Time Namespace Component RelatedObject Reason Message

openshift-ingress

e2e-derive-13c6a8db-openshift-default-797684c4c7-7b6lw

Scheduled

Successfully assigned openshift-ingress/e2e-derive-13c6a8db-openshift-default-797684c4c7-7b6lw to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-ipp-a-2b5a18-76496b8844-xjpx9

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-2b5a18-76496b8844-xjpx9 to ip-10-0-128-217.us-east-2.compute.internal

ai-tenants

e2e-autores-ced1b7

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-autores-ced1b7: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-explicit-9a69d1

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-explicit-9a69d1: MaasTenantConfig is not available to restore revocation operands

openshift-ingress

payload-processing-e2e-worker-w0-2e80fb-7f8cb49b89-d2pv9

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-2e80fb-7f8cb49b89-d2pv9 to ip-10-0-128-217.us-east-2.compute.internal

ai-tenants

e2e-extmod-b-204fba

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-204fba: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-inf-a-301be8

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-inf-a-301be8: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-inf-b-8743a5

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-inf-b-8743a5: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-ipp-a-2b5a18

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-ipp-a-2b5a18: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-ipp-b-836bba

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-ipp-b-836bba: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-mt-9cbd331c

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-mt-9cbd331c: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-mt-ce5e7da4

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-mt-ce5e7da4: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-praxis-a-000ffb

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-praxis-a-000ffb: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-praxis-b-d2e277

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-praxis-b-d2e277: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-shared-a-w4-bde944

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-shared-a-w4-bde944: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-shared-b-w4-9e7156

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-shared-b-w4-9e7156: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-single-main-dd40e2

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-single-main-dd40e2: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w0-2e80fb

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w0-2e80fb: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w3-a110ef

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w3-a110ef: MaasTenantConfig is not available to restore revocation operands

cert-manager

cert-manager-597b95cb84-ffv4m

Scheduled

Successfully assigned cert-manager/cert-manager-597b95cb84-ffv4m to ip-10-0-138-234.us-east-2.compute.internal

cert-manager

cert-manager-cainjector-5c57bbd4fc-457dp

Scheduled

Successfully assigned cert-manager/cert-manager-cainjector-5c57bbd4fc-457dp to ip-10-0-138-234.us-east-2.compute.internal

models-as-a-service

payload-processing-external-model-5ff7768868-lfrbv

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-5ff7768868-lfrbv to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-worker-single-main-dd40e2-d69d5d5d59rdvl

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-dd40e2-d69d5d5d59rdvl to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-worker-single-main-8d45bb-646b57d6bfgg2k

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-8d45bb-646b57d6bfgg2k to ip-10-0-138-234.us-east-2.compute.internal

models-as-a-service

payload-processing-external-model-5bf4f6c6d4-b44j6

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-5bf4f6c6d4-b44j6 to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-worker-single-main-8d45bb-646b57d6b2gvwf

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-8d45bb-646b57d6b2gvwf to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-worker-w3-a110ef-5766965cdf-gd9dj

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-a110ef-5766965cdf-gd9dj to ip-10-0-138-234.us-east-2.compute.internal

models-as-a-service

payload-processing-external-model-6cdf744db-49v9c

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-6cdf744db-49v9c to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-shared-b-w4-9e7156-f747b4965-v9sds

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-9e7156-f747b4965-v9sds to ip-10-0-128-217.us-east-2.compute.internal

models-as-a-service

payload-processing-external-model-6cdf744db-7tzrm

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-6cdf744db-7tzrm to ip-10-0-138-234.us-east-2.compute.internal

cert-manager

cert-manager-webhook-5dbbbf75cc-ztmqw

Scheduled

Successfully assigned cert-manager/cert-manager-webhook-5dbbbf75cc-ztmqw to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-inf-a-301be8-openshift-default-bf4f59db5-66ssd

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-301be8-openshift-default-bf4f59db5-66ssd to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-inf-a-301be8-openshift-default-bf4f59db5-cxz8s

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-301be8-openshift-default-bf4f59db5-cxz8s to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-inf-a-301be8-openshift-default-bf4f59db5-k5dvk

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-301be8-openshift-default-bf4f59db5-k5dvk to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-inf-a-301be8-openshift-default-bf4f59db5-rmcf7

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-301be8-openshift-default-bf4f59db5-rmcf7 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-shared-a-w4-bde944-c67f95877-bfj8t

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-bde944-c67f95877-bfj8t to ip-10-0-128-217.us-east-2.compute.internal

models-as-a-service

payload-processing-external-model-7b5bbbbbb8-27w26

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-7b5bbbbbb8-27w26 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-worker-w0-2e80fb-openshift-default-5585568575-rg5f7

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-2e80fb-openshift-default-5585568575-rg5f7 to ip-10-0-134-174.us-east-2.compute.internal

models-as-a-service

payload-processing-external-model-5bcd7997-kt9n9

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-5bcd7997-kt9n9 to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-praxis-b-d2e277-579fb69f6c-8plsv

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-praxis-b-d2e277-579fb69f6c-8plsv to ip-10-0-128-217.us-east-2.compute.internal

models-as-a-service

payload-processing-external-model-7b5bbbbbb8-ndszk

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-7b5bbbbbb8-ndszk to ip-10-0-138-234.us-east-2.compute.internal

kuadrant-system

authorino-56c6c9ddd5-dm2qp

Scheduled

Successfully assigned kuadrant-system/authorino-56c6c9ddd5-dm2qp to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-inf-b-8743a5-openshift-default-b59dcb9c-27nd6

Scheduled

Successfully assigned openshift-ingress/e2e-inf-b-8743a5-openshift-default-b59dcb9c-27nd6 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-inf-b-8743a5-openshift-default-b59dcb9c-mv6t8

Scheduled

Successfully assigned openshift-ingress/e2e-inf-b-8743a5-openshift-default-b59dcb9c-mv6t8 to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-inf-b-8743a5-openshift-default-b59dcb9c-sqnjm

Scheduled

Successfully assigned openshift-ingress/e2e-inf-b-8743a5-openshift-default-b59dcb9c-sqnjm to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-ipp-a-2b5a18-openshift-default-7d44b947f7-5klp9

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-a-2b5a18-openshift-default-7d44b947f7-5klp9 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-inf-a-301be8-9687dd9ff-n9tq4

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-301be8-9687dd9ff-n9tq4 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-ipp-b-836bba-openshift-default-887b755d9-l5rmq

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-b-836bba-openshift-default-887b755d9-l5rmq to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-204fba-6786ccc979-d7ct7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-204fba-6786ccc979-d7ct7 to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-629129-ddf565c67-76b79

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-629129-ddf565c67-76b79 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-explicit-9a69d1-7869845747-mnbt9

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-explicit-9a69d1-7869845747-mnbt9 to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-migrate-4c342351-gw-openshift-default-5bd6866449-t8rlj

Scheduled

Successfully assigned openshift-ingress/e2e-migrate-4c342351-gw-openshift-default-5bd6866449-t8rlj to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-mt-9cbd331c-openshift-default-574b7ff47c-6hx9j

Scheduled

Successfully assigned openshift-ingress/e2e-mt-9cbd331c-openshift-default-574b7ff47c-6hx9j to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-mt-9cbd331c-openshift-default-574b7ff47c-8pgt7

Scheduled

Successfully assigned openshift-ingress/e2e-mt-9cbd331c-openshift-default-574b7ff47c-8pgt7 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-mt-9cbd331c-openshift-default-574b7ff47c-g2dvk

Scheduled

Successfully assigned openshift-ingress/e2e-mt-9cbd331c-openshift-default-574b7ff47c-g2dvk to ip-10-0-138-234.us-east-2.compute.internal

kuadrant-system

authorino-57c4747b99-pjh9n

Scheduled

Successfully assigned kuadrant-system/authorino-57c4747b99-pjh9n to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-derive-13c6a8db-c677584cf-6h9fl

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-13c6a8db-c677584cf-6h9fl to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-autores-ced1b7-7557c8bd4c-pslr8

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-autores-ced1b7-7557c8bd4c-pslr8 to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-mt-ce5e7da4-openshift-default-5fb74bb54f-czsxf

Scheduled

Successfully assigned openshift-ingress/e2e-mt-ce5e7da4-openshift-default-5fb74bb54f-czsxf to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-api-b-ae13d4-8f94f446c-jcvtv

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-ae13d4-8f94f446c-jcvtv to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-api-a-a71884-94f55f9dc-pzflm

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-a71884-94f55f9dc-pzflm to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-ait-b9ae0feb-586d87c8c9-6hwhm

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-b9ae0feb-586d87c8c9-6hwhm to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-noten-0c5f4e-openshift-default-84f576f67-6k6gg

Scheduled

Successfully assigned openshift-ingress/e2e-noten-0c5f4e-openshift-default-84f576f67-6k6gg to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-ait-87e02313-5df95b66cf-lqlst

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-87e02313-5df95b66cf-lqlst to ip-10-0-128-217.us-east-2.compute.internal

kuadrant-system

authorino-57f7f8659f-m9jd9

Scheduled

Successfully assigned kuadrant-system/authorino-57f7f8659f-m9jd9 to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-praxis-a-000ffb-openshift-default-74665c677-9xtr6

Scheduled

Successfully assigned openshift-ingress/e2e-praxis-a-000ffb-openshift-default-74665c677-9xtr6 to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-praxis-a-000ffb-openshift-default-74665c677-lvtlw

Scheduled

Successfully assigned openshift-ingress/e2e-praxis-a-000ffb-openshift-default-74665c677-lvtlw to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-praxis-a-000ffb-openshift-default-74665c677-tx8bf

Scheduled

Successfully assigned openshift-ingress/e2e-praxis-a-000ffb-openshift-default-74665c677-tx8bf to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-extmod-b-204fba-openshift-default-8dd6555f7-xl5jm

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-204fba-openshift-default-8dd6555f7-xl5jm to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-ipp-a-2b5a18-7dbfd65bff-xxckt

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-2b5a18-7dbfd65bff-xxckt to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-ipp-a-2b5a18-85d6575455-fbmjk

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-2b5a18-85d6575455-fbmjk to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-ipp-b-836bba-6777bf66d7-vrfxq

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-836bba-6777bf66d7-vrfxq to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-ipp-b-836bba-6bb8db78bc-l4vc4

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-836bba-6bb8db78bc-l4vc4 to ip-10-0-138-234.us-east-2.compute.internal

kuadrant-system

authorino-6889cbd8c4-tckkg

Scheduled

Successfully assigned kuadrant-system/authorino-6889cbd8c4-tckkg to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-migrate-4c342351-7bf9d48b79-hx859

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-4c342351-7bf9d48b79-hx859 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-extmod-b-204fba-openshift-default-8dd6555f7-sn5sr

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-204fba-openshift-default-8dd6555f7-sn5sr to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-praxis-b-d2e277-openshift-default-94fd97fdf-t7vdm

Scheduled

Successfully assigned openshift-ingress/e2e-praxis-b-d2e277-openshift-default-94fd97fdf-t7vdm to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-extmod-b-204fba-openshift-default-8dd6555f7-r5zlx

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-204fba-openshift-default-8dd6555f7-r5zlx to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-mt-9cbd331c-b5c5f659-4jmx6

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-9cbd331c-b5c5f659-4jmx6 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-extmod-b-204fba-openshift-default-8dd6555f7-r2twl

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-204fba-openshift-default-8dd6555f7-r2twl to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-extmod-b-204fba-openshift-default-8dd6555f7-qvxr9

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-204fba-openshift-default-8dd6555f7-qvxr9 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-4cf62

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w4-bde944-openshift-default-75487f99f9-4cf62 to ip-10-0-134-174.us-east-2.compute.internal

kuadrant-system

authorino-6f6944bcdd-p8nf4

Scheduled

Successfully assigned kuadrant-system/authorino-6f6944bcdd-p8nf4 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-58jd2

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w4-bde944-openshift-default-75487f99f9-58jd2 to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-5lrv7

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w4-bde944-openshift-default-75487f99f9-5lrv7 to ip-10-0-128-217.us-east-2.compute.internal

kuadrant-system

authorino-795bcb67b4-xgtc6

Scheduled

Successfully assigned kuadrant-system/authorino-795bcb67b4-xgtc6 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-6f8mt

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w4-bde944-openshift-default-75487f99f9-6f8mt to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-lmdfc

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w4-bde944-openshift-default-75487f99f9-lmdfc to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-mt-9cbd331c-b5c5f659-tm6n7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-9cbd331c-b5c5f659-tm6n7 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-vq4ds

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w4-bde944-openshift-default-75487f99f9-vq4ds to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-extmod-b-204fba-openshift-default-8dd6555f7-mfjwz

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-204fba-openshift-default-8dd6555f7-mfjwz to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-ait-712810ea-77c968f8c9-4clwv

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-712810ea-77c968f8c9-4clwv to ip-10-0-128-217.us-east-2.compute.internal

kuadrant-system

authorino-operator-7b7bb4bc77-b66fk

Scheduled

Successfully assigned kuadrant-system/authorino-operator-7b7bb4bc77-b66fk to ip-10-0-128-217.us-east-2.compute.internal

models-as-a-service

payload-processing-external-model-87c8cf6-sftdh

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-87c8cf6-sftdh to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

router-default-68d88c448f-z5k5p

FailedScheduling

no nodes available to schedule pods

openshift-ingress

router-default-68d88c448f-z5k5p

FailedScheduling

no nodes available to schedule pods

openshift-ingress

router-default-68d88c448f-z5k5p

Scheduled

Successfully assigned openshift-ingress/router-default-68d88c448f-z5k5p to ip-10-0-134-174.us-east-2.compute.internal

models-as-a-service

payload-processing-external-model-55b55c785-vrkqt

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-55b55c785-vrkqt to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-praxis-a-000ffb-69745656c9-lmgkp

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-praxis-a-000ffb-69745656c9-lmgkp to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-mt-ce5e7da4-6687db7995-l2tvm

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-ce5e7da4-6687db7995-l2tvm to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-4rbgx

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-4rbgx to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-5msqr

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5msqr to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-5qw4z

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5qw4z to ip-10-0-138-234.us-east-2.compute.internal

models-as-a-service

payload-processing-external-model-f9c56bc9c-rph5n

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-f9c56bc9c-rph5n to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-2lqv8

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-2lqv8 to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-w22hv

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w4-bde944-openshift-default-75487f99f9-w22hv to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-wk5lx

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w4-bde944-openshift-default-75487f99f9-wk5lx to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-extmod-b-204fba-openshift-default-8dd6555f7-jqhv4

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-204fba-openshift-default-8dd6555f7-jqhv4 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-zt2tf

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w4-bde944-openshift-default-75487f99f9-zt2tf to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-zx7kj

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w4-bde944-openshift-default-75487f99f9-zx7kj to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-extmod-b-204fba-openshift-default-8dd6555f7-fvrvg

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-204fba-openshift-default-8dd6555f7-fvrvg to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-2cjnv

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-2cjnv to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-6mdlm

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-6mdlm to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-cwzxw

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-cwzxw to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-jsxcs

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-jsxcs to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-jzm5f

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-jzm5f to ip-10-0-128-217.us-east-2.compute.internal

kuadrant-system

dns-operator-controller-manager-8557cbdb84-tjlwm

Scheduled

Successfully assigned kuadrant-system/dns-operator-controller-manager-8557cbdb84-tjlwm to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-mt-9cbd331c-789b6c9bcf-v4jgn

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-9cbd331c-789b6c9bcf-v4jgn to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-6cftx

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-6cftx to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-6pk2b

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-6pk2b to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-2gmnx

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-2gmnx to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-mt-9cbd331c-789b6c9bcf-qscjb

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-9cbd331c-789b6c9bcf-qscjb to ip-10-0-128-217.us-east-2.compute.internal

opendatahub

ai-gateway-controller-6f755f6879-xh4h5

Scheduled

Successfully assigned opendatahub/ai-gateway-controller-6f755f6879-xh4h5 to ip-10-0-138-234.us-east-2.compute.internal

opendatahub

ai-gateway-controller-7d85675f98-pb8cr

Scheduled

Successfully assigned opendatahub/ai-gateway-controller-7d85675f98-pb8cr to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-4d57x

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-4d57x to ip-10-0-134-174.us-east-2.compute.internal

opendatahub

kserve-controller-manager-776cc66d47-cfpwn

Scheduled

Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-cfpwn to ip-10-0-134-174.us-east-2.compute.internal

opendatahub

kserve-module-controller-manager-7566cf5fc8-cwjgx

Scheduled

Successfully assigned opendatahub/kserve-module-controller-manager-7566cf5fc8-cwjgx to ip-10-0-128-217.us-east-2.compute.internal

opendatahub

llmisvc-controller-manager-6466c554d8-j8qxd

Scheduled

Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-j8qxd to ip-10-0-134-174.us-east-2.compute.internal

opendatahub

llmisvc-controller-manager-869d5dcc5d-lkqpk

Scheduled

Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-lkqpk to ip-10-0-134-174.us-east-2.compute.internal

opendatahub

maas-controller-5b555b5c85-jv2xg

Scheduled

Successfully assigned opendatahub/maas-controller-5b555b5c85-jv2xg to ip-10-0-134-174.us-east-2.compute.internal

opendatahub

maas-controller-5b555b5c85-sr2mh

Scheduled

Successfully assigned opendatahub/maas-controller-5b555b5c85-sr2mh to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-ait-62e67514-74478d9975-7sdxb

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-62e67514-74478d9975-7sdxb to ip-10-0-128-217.us-east-2.compute.internal

llm

premium-simulated-simulated-premium-kserve-5f5c4b459f-6hb6w

Scheduled

Successfully assigned llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-6hb6w to ip-10-0-128-217.us-east-2.compute.internal

kuadrant-system

kuadrant-console-plugin-66c4fb575d-52xrj

Scheduled

Successfully assigned kuadrant-system/kuadrant-console-plugin-66c4fb575d-52xrj to ip-10-0-128-217.us-east-2.compute.internal

ai-tenants

e2e-api-a-a71884

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-api-a-a71884: MaasTenantConfig is not available to restore revocation operands

openshift-ingress

e2e-extmod-b-204fba-openshift-default-8dd6555f7-9q2xr

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-204fba-openshift-default-8dd6555f7-9q2xr to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-pgrhg

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-pgrhg to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-mt-ce5e7da4-7749878675-xxb2t

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-ce5e7da4-7749878675-xxb2t to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-praxis-a-000ffb-76b97879b7-lbxd8

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-praxis-a-000ffb-76b97879b7-lbxd8 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-extmod-b-204fba-openshift-default-8dd6555f7-7glvz

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-204fba-openshift-default-8dd6555f7-7glvz to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-extmod-b-204fba-openshift-default-8dd6555f7-75t4b

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-204fba-openshift-default-8dd6555f7-75t4b to ip-10-0-138-234.us-east-2.compute.internal

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-5njvh

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-678d87766f-5njvh to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-rgcns

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-rgcns to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-s52zd

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-s52zd to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-x9j4q

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-x9j4q to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-zblc5

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-zblc5 to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-7b98bb678c-j6sq9

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-j6sq9 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-extmod-b-204fba-openshift-default-8dd6555f7-5msfb

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-204fba-openshift-default-8dd6555f7-5msfb to ip-10-0-128-217.us-east-2.compute.internal

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-mhvgh

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mhvgh to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-worker-single-main-8d45bb-openshift-default-5b7f975b9-mz4c6

Scheduled

Successfully assigned openshift-ingress/e2e-worker-single-main-8d45bb-openshift-default-5b7f975b9-mz4c6 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-6dbbcf55d8-n4d68

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-6dbbcf55d8-n4d68 to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-6c45b49d64-2dd4l

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-6c45b49d64-2dd4l to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-extmod-b-204fba-openshift-default-8dd6555f7-4kb6q

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-204fba-openshift-default-8dd6555f7-4kb6q to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-migrate-4c342351-7b78dfb54c-xjx6n

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-migrate-4c342351-7b78dfb54c-xjx6n to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-ipp-b-836bba-7b8f49cd68-2q26k

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-836bba-7b8f49cd68-2q26k to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-ipp-b-836bba-5577fcf647-9wmfx

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-836bba-5577fcf647-9wmfx to ip-10-0-138-234.us-east-2.compute.internal

kuadrant-system

kuadrant-operator-controller-manager-cf8469d7b-9vtdl

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9vtdl to ip-10-0-128-217.us-east-2.compute.internal

opendatahub

maas-controller-5b555b5c85-wbwdm

Scheduled

Successfully assigned opendatahub/maas-controller-5b555b5c85-wbwdm to ip-10-0-134-174.us-east-2.compute.internal

llm

e2e-distinct-simulated-kserve-7b998b79d9-vzmpw

Scheduled

Successfully assigned llm/e2e-distinct-simulated-kserve-7b998b79d9-vzmpw to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-ipp-a-2b5a18-69d476f7dc-8pfc8

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-2b5a18-69d476f7dc-8pfc8 to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-24rkw

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-24rkw to ip-10-0-128-217.us-east-2.compute.internal

opendatahub

maas-controller-796596f549-zjfq7

Scheduled

Successfully assigned opendatahub/maas-controller-796596f549-zjfq7 to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-inf-b-8743a5-6f4c48bc79-lb58d

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-8743a5-6f4c48bc79-lb58d to ip-10-0-128-217.us-east-2.compute.internal

opendatahub

model-serving-api-75bd89fdfc-2htfm

Scheduled

Successfully assigned opendatahub/model-serving-api-75bd89fdfc-2htfm to ip-10-0-138-234.us-east-2.compute.internal

opendatahub

odh-model-controller-769875db56-d2kkp

Scheduled

Successfully assigned opendatahub/odh-model-controller-769875db56-d2kkp to ip-10-0-138-234.us-east-2.compute.internal

llm

facebook-opt-125m-simulated-kserve-84b4ff656c-56c8w

Scheduled

Successfully assigned llm/facebook-opt-125m-simulated-kserve-84b4ff656c-56c8w to ip-10-0-128-217.us-east-2.compute.internal

opendatahub

test-curl-7360-0dc00d

Scheduled

Successfully assigned opendatahub/test-curl-7360-0dc00d to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-extmod-b-204fba-openshift-default-8dd6555f7-2q2rq

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-204fba-openshift-default-8dd6555f7-2q2rq to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

kube-auth-proxy-7c759d848c-x8mlt

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-7c759d848c-x8mlt to ip-10-0-138-234.us-east-2.compute.internal

kuadrant-system

limitador-limitador-74dfcfb894-svc5b

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-74dfcfb894-svc5b to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-praxis-b-d2e277-6bdc676c5f-g2r9n

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-praxis-b-d2e277-6bdc676c5f-g2r9n to ip-10-0-128-217.us-east-2.compute.internal

opendatahub

test-curl-7360-90a43e

Scheduled

Successfully assigned opendatahub/test-curl-7360-90a43e to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w4-bde944-99767bc7f-kbcnk

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-bde944-99767bc7f-kbcnk to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

kube-auth-proxy-7c759d848c-77mmj

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-7c759d848c-77mmj to ip-10-0-134-174.us-east-2.compute.internal

opendatahub

test-curl-7363-2b9420

Scheduled

Successfully assigned opendatahub/test-curl-7363-2b9420 to ip-10-0-134-174.us-east-2.compute.internal

opendatahub

test-curl-7363-a732ac

Scheduled

Successfully assigned opendatahub/test-curl-7363-a732ac to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-worker-single-main-dd40e2-openshift-default-5fd56bbf97qkv68

Scheduled

Successfully assigned openshift-ingress/e2e-worker-single-main-dd40e2-openshift-default-5fd56bbf97qkv68 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w4-9e7156-7957df6fc5-ctbhp

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-9e7156-7957df6fc5-ctbhp to ip-10-0-128-217.us-east-2.compute.internal

kuadrant-system

limitador-limitador-7c9b87d574-52pxc

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-7c9b87d574-52pxc to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-extmod-a-629129-openshift-default-dc77fd7d9-wzxjn

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-629129-openshift-default-dc77fd7d9-wzxjn to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-single-main-8d45bb-6dd776wgbw

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-8d45bb-6dd776wgbw to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-single-main-8d45bb-6dd778t8d6

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-8d45bb-6dd778t8d6 to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-single-main-dd40e2-6d5cdm2dt7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-dd40e2-6d5cdm2dt7 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w0-2e80fb-77fd4bc6bc-ncz2j

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-2e80fb-77fd4bc6bc-ncz2j to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-extmod-a-629129-openshift-default-dc77fd7d9-vrphk

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-629129-openshift-default-dc77fd7d9-vrphk to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w3-a110ef-c6fc947bc-hx6lh

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-a110ef-c6fc947bc-hx6lh to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-extmod-a-629129-openshift-default-dc77fd7d9-qbv6j

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-629129-openshift-default-dc77fd7d9-qbv6j to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-7cnfk

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7cnfk to ip-10-0-128-217.us-east-2.compute.internal

kuadrant-system

limitador-operator-controller-manager-5778bb4449-2l8n5

Scheduled

Successfully assigned kuadrant-system/limitador-operator-controller-manager-5778bb4449-2l8n5 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-7prvd

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7prvd to ip-10-0-138-234.us-east-2.compute.internal

opendatahub

test-curl-7366-0c0970

Scheduled

Successfully assigned opendatahub/test-curl-7366-0c0970 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-9nd6j

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-9nd6j to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

istiod-openshift-gateway-7ddd9cfdc8-scwch

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-scwch to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-cmgx5

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cmgx5 to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

istiod-openshift-gateway-7ddd9cfdc8-fmbxp

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-fmbxp to ip-10-0-138-234.us-east-2.compute.internal

opendatahub

test-curl-7366-5b297b

Scheduled

Successfully assigned opendatahub/test-curl-7366-5b297b to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

istiod-openshift-gateway-7ddd9cfdc8-6p6b2

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6p6b2 to ip-10-0-134-174.us-east-2.compute.internal

opendatahub

test-curl-7366-66627a

Scheduled

Successfully assigned opendatahub/test-curl-7366-66627a to ip-10-0-128-217.us-east-2.compute.internal

llm

echo-simulator-kserve-64b6f8f96c-npjcg

Scheduled

Successfully assigned llm/echo-simulator-kserve-64b6f8f96c-npjcg to ip-10-0-128-217.us-east-2.compute.internal

opendatahub

test-curl-7366-ae3ae1

Scheduled

Successfully assigned opendatahub/test-curl-7366-ae3ae1 to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-inf-a-301be8-74b7d9d5ff-5xjll

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-301be8-74b7d9d5ff-5xjll to ip-10-0-128-217.us-east-2.compute.internal

opendatahub

test-curl-7366-ae9450

Scheduled

Successfully assigned opendatahub/test-curl-7366-ae9450 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

istiod-openshift-gateway-7876bfcb74-4dlgl

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-4dlgl to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-d7szj

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-d7szj to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

openshift-ai-inference-openshift-ai-inference-6b78c66f8c-zwbkz

Scheduled

Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-zwbkz to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-f8wtz

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f8wtz to ip-10-0-128-217.us-east-2.compute.internal

opendatahub

test-curl-7366-e69be8

Scheduled

Successfully assigned opendatahub/test-curl-7366-e69be8 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-extmod-a-629129-openshift-default-dc77fd7d9-q7ntb

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-629129-openshift-default-dc77fd7d9-q7ntb to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

payload-processing-55fbbbf9dc-sbd6p

Scheduled

Successfully assigned openshift-ingress/payload-processing-55fbbbf9dc-sbd6p to ip-10-0-138-234.us-east-2.compute.internal

opendatahub

test-curl-7366-f11c97

Scheduled

Successfully assigned opendatahub/test-curl-7366-f11c97 to ip-10-0-128-217.us-east-2.compute.internal

llm

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8g4phf

Scheduled

Successfully assigned llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8g4phf to ip-10-0-138-234.us-east-2.compute.internal

opendatahub

usage-logs-tenancy-proxy-9c5d49bcd-7shdp

Scheduled

Successfully assigned opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-7shdp to ip-10-0-138-234.us-east-2.compute.internal

llm

e2e-distinct-2-simulated-kserve-cbd4f5586-h557j

Scheduled

Successfully assigned llm/e2e-distinct-2-simulated-kserve-cbd4f5586-h557j to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-extmod-a-629129-openshift-default-dc77fd7d9-fxn9s

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-629129-openshift-default-dc77fd7d9-fxn9s to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-processing-75fb58b5b7-xbsfb

Scheduled

Successfully assigned openshift-ingress/payload-processing-75fb58b5b7-xbsfb to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-processing-76459f68c5-48rr8

Scheduled

Successfully assigned openshift-ingress/payload-processing-76459f68c5-48rr8 to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-ait-62e67514-7b847995-hjrwz

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-62e67514-7b847995-hjrwz to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-extmod-a-629129-openshift-default-dc77fd7d9-cvdnx

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-629129-openshift-default-dc77fd7d9-cvdnx to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-worker-w0-2e80fb-openshift-default-5585568575-6krb6

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-2e80fb-openshift-default-5585568575-6krb6 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-worker-w0-2e80fb-openshift-default-5585568575-bcqxw

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-2e80fb-openshift-default-5585568575-bcqxw to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-ait-712810ea-8449cd864f-l84rs

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-712810ea-8449cd864f-l84rs to ip-10-0-128-217.us-east-2.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-f7698d755-48lmt

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-f7698d755-48lmt to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-worker-w0-2e80fb-openshift-default-5585568575-bssvc

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-2e80fb-openshift-default-5585568575-bssvc to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-worker-w0-2e80fb-openshift-default-5585568575-gkljq

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-2e80fb-openshift-default-5585568575-gkljq to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-explicit-9a69d1-openshift-default-5c97df6d45-844jv

Scheduled

Successfully assigned openshift-ingress/e2e-explicit-9a69d1-openshift-default-5c97df6d45-844jv to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-worker-w0-2e80fb-openshift-default-5585568575-k6wmf

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-2e80fb-openshift-default-5585568575-k6wmf to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-worker-w0-2e80fb-openshift-default-5585568575-l2jsl

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-2e80fb-openshift-default-5585568575-l2jsl to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-worker-w0-2e80fb-openshift-default-5585568575-ndhs8

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-2e80fb-openshift-default-5585568575-ndhs8 to ip-10-0-138-234.us-east-2.compute.internal

ai-tenants

e2e-extmod-a-629129

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-629129: MaasTenantConfig is not available to restore revocation operands

opendatahub-operator-system

opendatahub-operator-controller-manager-f7698d755-k98vj

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-f7698d755-k98vj to ip-10-0-134-174.us-east-2.compute.internal

ai-tenants

e2e-api-b-ae13d4

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-api-b-ae13d4: MaasTenantConfig is not available to restore revocation operands

llm

e2e-trlp-test-simulated-kserve-69b588897d-6wwp5

Scheduled

Successfully assigned llm/e2e-trlp-test-simulated-kserve-69b588897d-6wwp5 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-ait-87e02313-b88c5bc4f-4v9cc

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-87e02313-b88c5bc4f-4v9cc to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-worker-w0-2e80fb-openshift-default-5585568575-rmc6x

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-2e80fb-openshift-default-5585568575-rmc6x to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-zjvjd

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-zjvjd to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-worker-w0-2e80fb-openshift-default-5585568575-sk5vq

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-2e80fb-openshift-default-5585568575-sk5vq to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-wdc6m

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-wdc6m to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-autores-ced1b7-openshift-default-7d96c7f879-hgv8h

Scheduled

Successfully assigned openshift-ingress/e2e-autores-ced1b7-openshift-default-7d96c7f879-hgv8h to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-ait-b9ae0feb-fcd47598c-l25nf

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-b9ae0feb-fcd47598c-l25nf to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-api-a-a71884-7f8968dfdf-72wsf

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-a-a71884-7f8968dfdf-72wsf to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-worker-w3-a110ef-openshift-default-754f5584f9-2d492

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-a110ef-openshift-default-754f5584f9-2d492 to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-worker-w3-a110ef-openshift-default-754f5584f9-2rh7r

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-a110ef-openshift-default-754f5584f9-2rh7r to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-api-b-ae13d4-6d88f49fd5-bjfjb

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-b-ae13d4-6d88f49fd5-bjfjb to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-autores-ced1b7-865949c6d5-w8d7k

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-autores-ced1b7-865949c6d5-w8d7k to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-api-b-ae13d4-openshift-default-74cd94fc9f-mvzmt

Scheduled

Successfully assigned openshift-ingress/e2e-api-b-ae13d4-openshift-default-74cd94fc9f-mvzmt to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-worker-w3-a110ef-openshift-default-754f5584f9-42xxk

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-a110ef-openshift-default-754f5584f9-42xxk to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-vmkcq

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vmkcq to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-worker-w3-a110ef-openshift-default-754f5584f9-58hgh

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-a110ef-openshift-default-754f5584f9-58hgh to ip-10-0-134-174.us-east-2.compute.internal

llm

e2e-embedding-simulated-kserve-67bf594cd6-xg2gk

Scheduled

Successfully assigned llm/e2e-embedding-simulated-kserve-67bf594cd6-xg2gk to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-sqhmk

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-sqhmk to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-derive-13c6a8db-646b88c58c-69lf2

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-derive-13c6a8db-646b88c58c-69lf2 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-sn2pt

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-sn2pt to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-worker-w3-a110ef-openshift-default-754f5584f9-6fsp8

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-a110ef-openshift-default-754f5584f9-6fsp8 to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-api-a-a71884-openshift-default-6b7df8cf59-xxdrt

Scheduled

Successfully assigned openshift-ingress/e2e-api-a-a71884-openshift-default-6b7df8cf59-xxdrt to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-explicit-9a69d1-cbbb9c4d9-kp42c

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-explicit-9a69d1-cbbb9c4d9-kp42c to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-629129-9d8b9d665-qclfv

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-629129-9d8b9d665-qclfv to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-qwf92

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-qwf92 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-worker-w3-a110ef-openshift-default-754f5584f9-dg56j

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-a110ef-openshift-default-754f5584f9-dg56j to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-204fba-558699b45f-np9qr

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-204fba-558699b45f-np9qr to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

e2e-worker-w3-a110ef-openshift-default-754f5584f9-m8mhg

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-a110ef-openshift-default-754f5584f9-m8mhg to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-ait-b9ae0feb-openshift-default-558dd897b5-58kf5

Scheduled

Successfully assigned openshift-ingress/e2e-ait-b9ae0feb-openshift-default-558dd897b5-58kf5 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p2lgl

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-a110ef-openshift-default-754f5584f9-p2lgl to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p5zd5

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-a110ef-openshift-default-754f5584f9-p5zd5 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p8fzp

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-a110ef-openshift-default-754f5584f9-p8fzp to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-ait-87e02313-openshift-default-8749f6555-zhlms

Scheduled

Successfully assigned openshift-ingress/e2e-ait-87e02313-openshift-default-8749f6555-zhlms to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-qtkkt

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-qtkkt to ip-10-0-138-234.us-east-2.compute.internal

llm

e2e-prio-pool-4a8104f7-kserve-c99f876-5jksh

Scheduled

Successfully assigned llm/e2e-prio-pool-4a8104f7-kserve-c99f876-5jksh to ip-10-0-128-217.us-east-2.compute.internal

llm

e2e-same-id-b-bd1362d4

ModelNameConflictResolved

Model identity is no longer shared with any other MaaSModelRef in this namespace

llm

e2e-same-id-b-bd1362d4

ModelNameConflict

Model identity "publishers/llm/models/test/e2e-same-model-id" is shared with 1 other MaaSModelRef in this namespace: e2e-same-id-a-bd1362d4

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-mqzmk

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mqzmk to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-l4czw

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-l4czw to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-worker-w3-a110ef-openshift-default-754f5584f9-qkk8t

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-a110ef-openshift-default-754f5584f9-qkk8t to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-worker-w3-a110ef-openshift-default-754f5584f9-rgfsg

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-a110ef-openshift-default-754f5584f9-rgfsg to ip-10-0-138-234.us-east-2.compute.internal

llm

e2e-prio-pool-4a8104f7-kserve-router-scheduler-75f8877679-x2qwr

Scheduled

Successfully assigned llm/e2e-prio-pool-4a8104f7-kserve-router-scheduler-75f8877679-x2qwr to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-ait-712810ea-openshift-default-76c9fb4567-ncpx6

Scheduled

Successfully assigned openshift-ingress/e2e-ait-712810ea-openshift-default-76c9fb4567-ncpx6 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

e2e-worker-w3-a110ef-openshift-default-754f5584f9-s2h9r

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-a110ef-openshift-default-754f5584f9-s2h9r to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-worker-w3-a110ef-openshift-default-754f5584f9-wmdl4

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-a110ef-openshift-default-754f5584f9-wmdl4 to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-h4446

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-h4446 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-75fd9f5d47-fgc89

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fgc89 to ip-10-0-138-234.us-east-2.compute.internal

openshift-ingress

e2e-ait-62e67514-openshift-default-6cd4788d5-765tv

Scheduled

Successfully assigned openshift-ingress/e2e-ait-62e67514-openshift-default-6cd4788d5-765tv to ip-10-0-134-174.us-east-2.compute.internal

openshift-jobset-operator

jobset-controller-manager-7cf664dd55-w9468

Scheduled

Successfully assigned openshift-jobset-operator/jobset-controller-manager-7cf664dd55-w9468 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-6b748b6585knkfm

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b6585knkfm to ip-10-0-128-217.us-east-2.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-f7698d755-kkqg6

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-f7698d755-kkqg6 to ip-10-0-128-217.us-east-2.compute.internal

llm

e2e-same-id-b-bd1362d4-kserve-799bf7946f-7nrqs

Scheduled

Successfully assigned llm/e2e-same-id-b-bd1362d4-kserve-799bf7946f-7nrqs to ip-10-0-138-234.us-east-2.compute.internal

llm

e2e-same-id-a-bd1362d4

ModelNameConflict

Model identity "publishers/llm/models/test/e2e-same-model-id" is shared with 1 other MaaSModelRef in this namespace: e2e-same-id-b-bd1362d4

openshift-jobset-operator

jobset-operator-674cd59dc5-df694

Scheduled

Successfully assigned openshift-jobset-operator/jobset-operator-674cd59dc5-df694 to ip-10-0-138-234.us-east-2.compute.internal

llm

e2e-same-id-a-bd1362d4-kserve-5b987dd75c-5fgb8

Scheduled

Successfully assigned llm/e2e-same-id-a-bd1362d4-kserve-5b987dd75c-5fgb8 to ip-10-0-128-217.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-inf-b-8743a5-679877bf47-jj7dr

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-8743a5-679877bf47-jj7dr to ip-10-0-134-174.us-east-2.compute.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-68d88c448f from 0 to 1

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

replicaset-controller

router-default-68d88c448f

SuccessfulCreate

Created pod: router-default-68d88c448f-z5k5p
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts
(x5)

openshift-ingress

kubelet

router-default-68d88c448f-z5k5p

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
(x5)

openshift-ingress

kubelet

router-default-68d88c448f-z5k5p

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x5)

openshift-ingress

kubelet

router-default-68d88c448f-z5k5p

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x5)

openshift-ingress

kubelet

router-default-68d88c448f-z5k5p

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt

openshift-ingress

multus

router-default-68d88c448f-z5k5p

AddedInterface

Add eth0 [10.132.0.19/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-68d88c448f-z5k5p

Started

Container started

openshift-ingress

kubelet

router-default-68d88c448f-z5k5p

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

router-default-68d88c448f-z5k5p

Created

Container created

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

multus

istiod-openshift-gateway-7876bfcb74-4dlgl

AddedInterface

Add eth0 [10.132.0.27/23] from ovn-kubernetes

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7876bfcb74

SuccessfulCreate

Created pod: istiod-openshift-gateway-7876bfcb74-4dlgl

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-4dlgl

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8"

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-4dlgl

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-4dlgl

Started

Container started

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-4dlgl

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.654s (3.654s including waiting). Image size: 185476477 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-4dlgl

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-4dlgl

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

deployment-controller

openshift-ai-inference-openshift-ai-inference

ScalingReplicaSet

Scaled up replica set openshift-ai-inference-openshift-ai-inference-6b78c66f8c from 0 to 1

openshift-ingress

service-controller

openshift-ai-inference-openshift-ai-inference

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-6b78c66f8c-zwbkz

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-75fd9f5d47 from 0 to 1

openshift-ingress

multus

openshift-ai-inference-openshift-ai-inference-6b78c66f8c-zwbkz

AddedInterface

Add eth0 [10.134.0.27/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-75fd9f5d47-24rkw

openshift-ingress

replicaset-controller

openshift-ai-inference-openshift-ai-inference-6b78c66f8c

SuccessfulCreate

Created pod: openshift-ai-inference-openshift-ai-inference-6b78c66f8c-zwbkz

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-24rkw

AddedInterface

Add eth0 [10.134.0.26/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-24rkw

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

service-controller

openshift-ai-inference-openshift-ai-inference

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-6b78c66f8c-zwbkz

Started

Container started

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-6b78c66f8c-zwbkz

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 9.022s (9.022s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-6b78c66f8c-zwbkz

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-24rkw

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 10.562s (10.562s including waiting). Image size: 183555333 bytes.
(x2)

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

RequirementsUnknown

requirements not yet checked
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

RequirementsNotMet

one or more requirements couldn't be found

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-f7698d755

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-f7698d755-48lmt

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-f7698d755

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-f7698d755-k98vj

opendatahub-operator-system

deployment-controller

opendatahub-operator-controller-manager

ScalingReplicaSet

Scaled up replica set opendatahub-operator-controller-manager-f7698d755 from 0 to 3

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-f7698d755-kkqg6

AddedInterface

Add eth0 [10.134.0.29/23] from ovn-kubernetes

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-f7698d755

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-f7698d755-kkqg6

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

AllRequirementsMet

all requirements found, attempting install

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-f7698d755-kkqg6

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-f7698d755-k98vj

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallWaiting

Webhook install failed: conversionWebhook not ready

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-f7698d755-48lmt

AddedInterface

Add eth0 [10.133.0.19/23] from ovn-kubernetes

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-f7698d755-48lmt

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-f7698d755-k98vj

AddedInterface

Add eth0 [10.132.0.28/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

RequirementsNotMet

one or more requirements couldn't be found
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallSucceeded

waiting for install components to report healthy

opendatahub-operator-system

operator-lifecycle-manager

install-5x8pw

AppliedWithWarnings

1 warning(s) generated during installation of operator "opendatahub-operator.v3.6.0-ea.2" (CustomResourceDefinition "hardwareprofiles.infrastructure.opendatahub.io"): infrastructure.opendatahub.io/v1alpha1 is deprecated; please use infrastructure.opendatahub.io/v1
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallWaiting

installing: waiting for deployment opendatahub-operator-controller-manager to become ready: deployment "opendatahub-operator-controller-manager" not available: Deployment does not have minimum availability.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-f7698d755-kkqg6

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-f7698d755-kkqg6

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 3.29s (3.29s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-f7698d755-48lmt

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-f7698d755-48lmt

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-f7698d755-48lmt

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 2.938s (2.938s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-f7698d755-k98vj

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-f7698d755-k98vj

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-f7698d755-kkqg6

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-f7698d755-k98vj

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 3.114s (3.114s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-f7698d755-kkqg6

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 515ms (515ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-f7698d755-kkqg6

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-f7698d755-48lmt

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-f7698d755-48lmt

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 457ms (457ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-f7698d755-48lmt

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-f7698d755-k98vj

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-f7698d755-k98vj

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-f7698d755-48lmt

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-f7698d755-k98vj

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 379ms (379ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-f7698d755-k98vj

Created

Container created

opendatahub-operator-system

opendatahub-operator-controller-manager-f7698d755-k98vj_a72bcb9c-da92-4160-8d12-883d76e5d9fd

07ed84f7.opendatahub.io

LeaderElection

opendatahub-operator-controller-manager-f7698d755-k98vj_a72bcb9c-da92-4160-8d12-883d76e5d9fd became leader

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-f7698d755-kkqg6

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-f7698d755-kkqg6

Started

Container started
(x2)

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

RequirementsUnknown

requirements not yet checked
(x2)

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-tjlwm

Pulling

Pulling image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:d82688172d312a44d9978eacf7bd9e0402e9ba2967daf70b2ef30707f41faf4a"

kuadrant-system

replicaset-controller

dns-operator-controller-manager-8557cbdb84

SuccessfulCreate

Created pod: dns-operator-controller-manager-8557cbdb84-tjlwm

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

multus

dns-operator-controller-manager-8557cbdb84-tjlwm

AddedInterface

Add eth0 [10.134.0.30/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallWaiting

installing: waiting for deployment dns-operator-controller-manager to become ready: deployment "dns-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

deployment-controller

dns-operator-controller-manager

ScalingReplicaSet

Scaled up replica set dns-operator-controller-manager-8557cbdb84 from 0 to 1

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

RequirementsNotMet

one or more requirements couldn't be found

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

AllRequirementsMet

all requirements found, attempting install

openshift-jobset-operator

replicaset-controller

jobset-operator-674cd59dc5

SuccessfulCreate

Created pod: jobset-operator-674cd59dc5-df694

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallSucceeded

waiting for install components to report healthy

openshift-jobset-operator

deployment-controller

jobset-operator

ScalingReplicaSet

Scaled up replica set jobset-operator-674cd59dc5 from 0 to 1

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallWaiting

installing: waiting for deployment jobset-operator to become ready: deployment "jobset-operator" not available: Deployment does not have minimum availability.

openshift-jobset-operator

multus

jobset-operator-674cd59dc5-df694

AddedInterface

Add eth0 [10.133.0.20/23] from ovn-kubernetes

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-df694

Pulling

Pulling image "registry.redhat.io/job-set/jobset-rhel9-operator@sha256:a258511e57b511c5d2a54fae2b8165669d677fb3a69796aa1c78d224a499033e"

kuadrant-system

dns-operator-controller-manager-8557cbdb84-tjlwm_e291c709-c590-428b-ad9b-9e8c1e067e7c

a3f98d6c.kuadrant.io

LeaderElection

dns-operator-controller-manager-8557cbdb84-tjlwm_e291c709-c590-428b-ad9b-9e8c1e067e7c became leader

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-tjlwm

Created

Container created

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-tjlwm

Started

Container started

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-tjlwm

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:d82688172d312a44d9978eacf7bd9e0402e9ba2967daf70b2ef30707f41faf4a" in 3.266s (3.266s including waiting). Image size: 190804149 bytes.

kuadrant-system

multus

authorino-operator-7b7bb4bc77-b66fk

AddedInterface

Add eth0 [10.134.0.31/23] from ovn-kubernetes

kuadrant-system

deployment-controller

authorino-operator

ScalingReplicaSet

Scaled up replica set authorino-operator-7b7bb4bc77 from 0 to 1

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

replicaset-controller

authorino-operator-7b7bb4bc77

SuccessfulCreate

Created pod: authorino-operator-7b7bb4bc77-b66fk

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallWaiting

installing: waiting for deployment authorino-operator to become ready: deployment "authorino-operator" not available: Deployment does not have minimum availability.

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-b66fk

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:8a8cc5fb104032c23096f3c9cf2fbc3636d343ee7ee7974e51fe1f10d3a57589"

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

RequirementsUnknown

requirements not yet checked

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallSucceeded

install strategy completed with no errors

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-cf8469d7b

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-cf8469d7b-9vtdl

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-9vtdl

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e"

kuadrant-system

multus

kuadrant-operator-controller-manager-cf8469d7b-9vtdl

AddedInterface

Add eth0 [10.134.0.33/23] from ovn-kubernetes

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled up replica set kuadrant-operator-controller-manager-cf8469d7b from 0 to 1

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallSucceeded

install strategy completed with no errors

cert-manager

deployment-controller

cert-manager-cainjector

ScalingReplicaSet

Scaled up replica set cert-manager-cainjector-5c57bbd4fc from 0 to 1

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallWaiting

installing: waiting for deployment limitador-operator-controller-manager to become ready: deployment "limitador-operator-controller-manager" not available: Deployment does not have minimum availability.

cert-manager

deployment-controller

cert-manager

ScalingReplicaSet

Scaled up replica set cert-manager-597b95cb84 from 0 to 1

kuadrant-system

replicaset-controller

limitador-operator-controller-manager-5778bb4449

SuccessfulCreate

Created pod: limitador-operator-controller-manager-5778bb4449-2l8n5

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

deployment-controller

limitador-operator-controller-manager

ScalingReplicaSet

Scaled up replica set limitador-operator-controller-manager-5778bb4449 from 0 to 1
(x9)

cert-manager

replicaset-controller

cert-manager-cainjector-5c57bbd4fc

FailedCreate

Error creating: pods "cert-manager-cainjector-5c57bbd4fc-" is forbidden: error looking up service account cert-manager/cert-manager-cainjector: serviceaccount "cert-manager-cainjector" not found

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-b66fk

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:8a8cc5fb104032c23096f3c9cf2fbc3636d343ee7ee7974e51fe1f10d3a57589" in 12.974s (12.974s including waiting). Image size: 167738465 bytes.

cert-manager

replicaset-controller

cert-manager-cainjector-5c57bbd4fc

SuccessfulCreate

Created pod: cert-manager-cainjector-5c57bbd4fc-457dp

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-2l8n5

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:7fdc92b22c7d74e68e71ebf54d1c680fc8dbd47198286dc4f7f87b3d4b31472e"

kuadrant-system

multus

limitador-operator-controller-manager-5778bb4449-2l8n5

AddedInterface

Add eth0 [10.134.0.34/23] from ovn-kubernetes

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-df694

Pulled

Successfully pulled image "registry.redhat.io/job-set/jobset-rhel9-operator@sha256:a258511e57b511c5d2a54fae2b8165669d677fb3a69796aa1c78d224a499033e" in 15.702s (15.702s including waiting). Image size: 224874686 bytes.

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-df694

Started

Container started

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-df694

Created

Container created

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-457dp

Pulling

Pulling image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d"

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallSucceeded

install strategy completed with no errors

kuadrant-system

authorino-operator-7b7bb4bc77-b66fk_50843224-0c35-4f6b-b82e-3d81cc8568a1

aac3a15d.authorino.kuadrant.io

LeaderElection

authorino-operator-7b7bb4bc77-b66fk_50843224-0c35-4f6b-b82e-3d81cc8568a1 became leader

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-b66fk

Created

Container created

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-b66fk

Started

Container started

cert-manager

multus

cert-manager-cainjector-5c57bbd4fc-457dp

AddedInterface

Add eth0 [10.133.0.21/23] from ovn-kubernetes

openshift-jobset-operator

openshift-jobset-operator

openshift-jobset-operator-lock

LeaderElection

jobset-operator-674cd59dc5-df694_21eacc38-9709-4282-9ba9-94f35b6af34b became leader

kuadrant-system

limitador-operator-controller-manager-5778bb4449-2l8n5_fd798d22-abce-4a2a-82f0-8b0c514f25bc

3745a16e.kuadrant.io

LeaderElection

limitador-operator-controller-manager-5778bb4449-2l8n5_fd798d22-abce-4a2a-82f0-8b0c514f25bc became leader

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-2l8n5

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:7fdc92b22c7d74e68e71ebf54d1c680fc8dbd47198286dc4f7f87b3d4b31472e" in 2.404s (2.404s including waiting). Image size: 177967631 bytes.

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-2l8n5

Started

Container started

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-2l8n5

Created

Container created

kuadrant-system

kuadrant-operator-controller-manager-cf8469d7b-9vtdl_18d22d64-f4b9-4dac-8fb6-96d630cdfe9e

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-cf8469d7b-9vtdl_18d22d64-f4b9-4dac-8fb6-96d630cdfe9e became leader

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-9vtdl

Created

Container created

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-9vtdl

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" in 7.384s (7.384s including waiting). Image size: 424244516 bytes.
(x11)

cert-manager

replicaset-controller

cert-manager-597b95cb84

FailedCreate

Error creating: pods "cert-manager-597b95cb84-" is forbidden: error looking up service account cert-manager/cert-manager: serviceaccount "cert-manager" not found

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-9vtdl

Started

Container started

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-457dp

Pulled

Successfully pulled image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" in 3.659s (3.659s including waiting). Image size: 309650901 bytes.

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-457dp

Created

Container created

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-457dp

Started

Container started

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-6b748b6585 from 0 to 1

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-6b748b6585

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-6b748b6585knkfm

cert-manager

deployment-controller

cert-manager-webhook

ScalingReplicaSet

Scaled up replica set cert-manager-webhook-5dbbbf75cc from 0 to 1

cert-manager

replicaset-controller

cert-manager-webhook-5dbbbf75cc

SuccessfulCreate

Created pod: cert-manager-webhook-5dbbbf75cc-ztmqw

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-manager-role because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ServiceCreated

Created Service/jobset-webhook-service -n openshift-jobset-operator because it was missing

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-7c759d848c from 0 to 2

openshift-ingress

replicaset-controller

kube-auth-proxy-7c759d848c

SuccessfulCreate

Created pod: kube-auth-proxy-7c759d848c-77mmj

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-proxy-role because it was missing

openshift-ingress

replicaset-controller

kube-auth-proxy-7c759d848c

SuccessfulCreate

Created pod: kube-auth-proxy-7c759d848c-x8mlt

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-metrics-reader because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-manager-rolebinding because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ServiceAccountCreated

Created ServiceAccount/jobset-controller-manager -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-proxy-rolebinding because it was missing

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-ztmqw

Pulled

Container image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" already present on machine and can be accessed by the pod

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-52xrj

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:c8550de6790d58c5fa993bf83d41ef693d99a5739fcc0eed3fcaa8d5a7658044"

kuadrant-system

multus

kuadrant-console-plugin-66c4fb575d-52xrj

AddedInterface

Add eth0 [10.134.0.36/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-7c759d848c-x8mlt

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"

openshift-ingress

multus

kube-auth-proxy-7c759d848c-x8mlt

AddedInterface

Add eth0 [10.133.0.23/23] from ovn-kubernetes

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-6b748b6585knkfm

Started

Container started

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-6b748b6585knkfm

Created

Container created

cert-manager

multus

cert-manager-webhook-5dbbbf75cc-ztmqw

AddedInterface

Add eth0 [10.133.0.22/23] from ovn-kubernetes

kuadrant-system

deployment-controller

kuadrant-console-plugin

ScalingReplicaSet

Scaled up replica set kuadrant-console-plugin-66c4fb575d from 0 to 1

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-ztmqw

Created

Container created

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-6b748b6585knkfm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-ztmqw

Started

Container started

openshift-ingress

kubelet

kube-auth-proxy-7c759d848c-77mmj

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"

openshift-ingress

multus

kube-auth-proxy-7c759d848c-77mmj

AddedInterface

Add eth0 [10.132.0.29/23] from ovn-kubernetes

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-metrics-reader-rolebinding because it was missing

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-6b748b6585knkfm

AddedInterface

Add eth0 [10.134.0.35/23] from ovn-kubernetes

kuadrant-system

replicaset-controller

kuadrant-console-plugin-66c4fb575d

SuccessfulCreate

Created pod: kuadrant-console-plugin-66c4fb575d-52xrj

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-leader-election-role -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-manager-secrets-role -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ServiceCreated

Created Service/jobset-controller-manager-metrics-service -n openshift-jobset-operator because it was missing

kuadrant-system

replicaset-controller

authorino-56c6c9ddd5

SuccessfulCreate

Created pod: authorino-56c6c9ddd5-dm2qp

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-manager-secrets-rolebinding -n openshift-jobset-operator because it was missing

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-57f7f8659f from 0 to 1
(x5)

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

IssuerCreateFailed

Failed to create Issuer.cert-manager.io/jobset-selfsigned-issuer -n openshift-jobset-operator: Internal error occurred: failed calling webhook "webhook.cert-manager.io": failed to call webhook: Post "https://cert-manager-webhook.cert-manager.svc:443/validate?timeout=30s": no endpoints available for service "cert-manager-webhook"

kuadrant-system

multus

authorino-57f7f8659f-m9jd9

AddedInterface

Add eth0 [10.133.0.24/23] from ovn-kubernetes

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-56c6c9ddd5 from 0 to 1

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-leader-election-rolebinding -n openshift-jobset-operator because it was missing

kuadrant-system

replicaset-controller

authorino-57f7f8659f

SuccessfulCreate

Created pod: authorino-57f7f8659f-m9jd9

kuadrant-system

kubelet

authorino-57f7f8659f-m9jd9

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

cert-manager

replicaset-controller

cert-manager-597b95cb84

SuccessfulCreate

Created pod: cert-manager-597b95cb84-ffv4m

opendatahub

deployment-controller

kserve-module-controller-manager

ScalingReplicaSet

Scaled up replica set kserve-module-controller-manager-7566cf5fc8 from 0 to 1

opendatahub

replicaset-controller

kserve-module-controller-manager-7566cf5fc8

SuccessfulCreate

Created pod: kserve-module-controller-manager-7566cf5fc8-cwjgx

kuadrant-system

multus

authorino-56c6c9ddd5-dm2qp

AddedInterface

Add eth0 [10.134.0.37/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-56c6c9ddd5-dm2qp

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-cwjgx

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189"

openshift-ingress

kubelet

kube-auth-proxy-7c759d848c-77mmj

Started

Container started

openshift-ingress

kubelet

kube-auth-proxy-7c759d848c-x8mlt

Started

Container started

openshift-ingress

kubelet

kube-auth-proxy-7c759d848c-77mmj

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.934s (2.935s including waiting). Image size: 188904677 bytes.

openshift-ingress

kubelet

kube-auth-proxy-7c759d848c-77mmj

Created

Container created

openshift-ingress

kubelet

kube-auth-proxy-7c759d848c-x8mlt

Created

Container created

openshift-ingress

kubelet

kube-auth-proxy-7c759d848c-x8mlt

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.983s (2.983s including waiting). Image size: 188904677 bytes.

cert-manager

kubelet

cert-manager-597b95cb84-ffv4m

Started

Container started

opendatahub

multus

kserve-module-controller-manager-7566cf5fc8-cwjgx

AddedInterface

Add eth0 [10.134.0.38/23] from ovn-kubernetes

cert-manager

kubelet

cert-manager-597b95cb84-ffv4m

Created

Container created

cert-manager

multus

cert-manager-597b95cb84-ffv4m

AddedInterface

Add eth0 [10.133.0.25/23] from ovn-kubernetes

cert-manager

kubelet

cert-manager-597b95cb84-ffv4m

Pulled

Container image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" already present on machine and can be accessed by the pod

kuadrant-system

kubelet

authorino-56c6c9ddd5-dm2qp

Created

Container created

kuadrant-system

kubelet

authorino-56c6c9ddd5-dm2qp

Started

Container started

kuadrant-system

kubelet

authorino-56c6c9ddd5-dm2qp

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 2.613s (2.613s including waiting). Image size: 204096810 bytes.

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallSucceeded

install strategy completed with no errors

kuadrant-system

replicaset-controller

authorino-57f7f8659f

SuccessfulDelete

Deleted pod: authorino-57f7f8659f-m9jd9

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-57f7f8659f from 1 to 0

kuadrant-system

kubelet

authorino-57f7f8659f-m9jd9

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 3.291s (3.291s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-57f7f8659f-m9jd9

Created

Container created

kuadrant-system

kubelet

authorino-57f7f8659f-m9jd9

Started

Container started

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-cwjgx

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 2.979s (2.979s including waiting). Image size: 195729000 bytes.

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-cwjgx

Started

Container started

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-52xrj

Started

Container started

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-52xrj

Created

Container created

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-52xrj

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:c8550de6790d58c5fa993bf83d41ef693d99a5739fcc0eed3fcaa8d5a7658044" in 5.594s (5.594s including waiting). Image size: 342538866 bytes.

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-cwjgx

Created

Container created

kuadrant-system

kubelet

authorino-57f7f8659f-m9jd9

Killing

Stopping container authorino

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallSucceeded

install strategy completed with no errors

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" waiting for 1 outdated replica(s) to be terminated

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-678d87766f

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-678d87766f-5njvh

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: deployment changed old hash=6q1WY8Uae4JZmLF4M4Zj2lZF5tKn8wEFi4k5Ve, new hash=88W7Jv6nAXJm3xNFHpPpBPB6INrNa0R4IQ7iYv

kuadrant-system

multus

kuadrant-operator-controller-manager-678d87766f-5njvh

AddedInterface

Add eth0 [10.133.0.26/23] from ovn-kubernetes

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-5njvh

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e"

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

IssuerCreated

Created Issuer.cert-manager.io/jobset-selfsigned-issuer -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CertificateCreated

Created Certificate.cert-manager.io/jobset-metrics-cert -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CertificateCreated

Created Certificate.cert-manager.io/jobset-serving-cert -n openshift-jobset-operator because it was missing

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-5njvh

Created

Container created

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-5njvh

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" in 5.347s (5.347s including waiting). Image size: 424244516 bytes.

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-5njvh

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-74dfcfb894 from 0 to 1

kuadrant-system

replicaset-controller

limitador-limitador-74dfcfb894

SuccessfulCreate

Created pod: limitador-limitador-74dfcfb894-svc5b

kuadrant-system

replicaset-controller

limitador-limitador-7c9b87d574

SuccessfulCreate

Created pod: limitador-limitador-7c9b87d574-52pxc

kuadrant-system

multus

limitador-limitador-74dfcfb894-svc5b

AddedInterface

Add eth0 [10.132.0.30/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-svc5b

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751"

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-7c9b87d574 from 0 to 1

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-52pxc

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751"

kuadrant-system

multus

limitador-limitador-7c9b87d574-52pxc

AddedInterface

Add eth0 [10.133.0.27/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-52pxc

Created

Container created

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-52pxc

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751" in 1.119s (1.119s including waiting). Image size: 133379307 bytes.

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-52pxc

Started

Container started

opendatahub

kserve-module-controller-manager-7566cf5fc8-cwjgx_da59f62e-82a5-4e6d-90f6-25e3ce9e8ddf

kserve-module-controller-manager

LeaderElection

kserve-module-controller-manager-7566cf5fc8-cwjgx_da59f62e-82a5-4e6d-90f6-25e3ce9e8ddf became leader

openshift-jobset-operator

cert-manager-certificaterequests-issuer-ca

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-vault

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

MutatingWebhookConfigurationCreated

Created MutatingWebhookConfiguration.admissionregistration.k8s.io/jobset-mutating-webhook-configuration because it was missing

openshift-jobset-operator

cert-manager-certificates-request-manager

jobset-serving-cert

Requested

Created new CertificateRequest resource "jobset-serving-cert-1"

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-serving-cert-1

CertificateIssued

Certificate fetched from issuer successfully

openshift-jobset-operator

cert-manager-certificates-issuing

jobset-serving-cert

Issuing

The certificate has been successfully issued

openshift-jobset-operator

cert-manager-certificaterequests-issuer-venafi

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-venafi

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificates-trigger

jobset-serving-cert

Issuing

Issuing certificate as Secret does not exist

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-ca

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-metrics-cert-1

CertificateIssued

Certificate fetched from issuer successfully

openshift-jobset-operator

cert-manager-certificaterequests-issuer-acme

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-approver

jobset-metrics-cert-1

cert-manager.io

Certificate request has been approved by cert-manager.io

openshift-jobset-operator

cert-manager-certificaterequests-issuer-vault

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificates-key-manager

jobset-serving-cert

Generated

Stored new private key in temporary Secret resource "jobset-serving-cert-n77wl"

openshift-jobset-operator

cert-manager-certificates-issuing

jobset-metrics-cert

Issuing

The certificate has been successfully issued

openshift-jobset-operator

cert-manager-certificaterequests-issuer-acme

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-approver

jobset-serving-cert-1

cert-manager.io

Certificate request has been approved by cert-manager.io

openshift-jobset-operator

cert-manager-certificates-request-manager

jobset-metrics-cert

Requested

Created new CertificateRequest resource "jobset-metrics-cert-1"

openshift-jobset-operator

cert-manager-certificates-key-manager

jobset-metrics-cert

Generated

Stored new private key in temporary Secret resource "jobset-metrics-cert-7fpmf"

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ValidatingWebhookConfigurationCreated

Created ValidatingWebhookConfiguration.admissionregistration.k8s.io/jobset-validating-webhook-configuration because it was missing

openshift-jobset-operator

cert-manager-certificates-trigger

jobset-metrics-cert

Issuing

Issuing certificate as Secret does not exist

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ServiceMonitorCreated

Created ServiceMonitor.monitoring.coreos.com/jobset-controller-manager-metrics-monitor -n openshift-jobset-operator because it was missing

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-svc5b

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751" in 3.272s (3.272s including waiting). Image size: 133379307 bytes.

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-prometheus-k8s -n openshift-jobset-operator because it was missing

openshift-jobset-operator

multus

jobset-controller-manager-7cf664dd55-w9468

AddedInterface

Add eth0 [10.134.0.39/23] from ovn-kubernetes

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-prometheus-k8s -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

MutatingWebhookConfigurationUpdated

Updated MutatingWebhookConfiguration.admissionregistration.k8s.io/jobset-mutating-webhook-configuration because it changed

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ValidatingWebhookConfigurationUpdated

Updated ValidatingWebhookConfiguration.admissionregistration.k8s.io/jobset-validating-webhook-configuration because it changed

openshift-jobset-operator

deployment-controller

jobset-controller-manager

ScalingReplicaSet

Scaled up replica set jobset-controller-manager-7cf664dd55 from 0 to 1

openshift-jobset-operator

replicaset-controller

jobset-controller-manager-7cf664dd55

SuccessfulCreate

Created pod: jobset-controller-manager-7cf664dd55-w9468

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CustomResourceDefinitionCreated

Created CustomResourceDefinition.apiextensions.k8s.io/jobsets.jobset.x-k8s.io because it was missing

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-svc5b

Created

Container created

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-svc5b

Started

Container started

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

DeploymentCreated

Created Deployment.apps/jobset-controller-manager -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ConfigMapCreated

Created ConfigMap/jobset-manager-config -n openshift-jobset-operator because it was missing

openshift-jobset-operator

kubelet

jobset-controller-manager-7cf664dd55-w9468

Pulling

Pulling image "registry.redhat.io/job-set/jobset-rhel9@sha256:8d2a70fe90b2414cdb269d217703b7e5b3409f9276acf671721b1abc67a0b987"

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled down replica set kuadrant-operator-controller-manager-cf8469d7b from 1 to 0

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-cf8469d7b

SuccessfulDelete

Deleted pod: kuadrant-operator-controller-manager-cf8469d7b-9vtdl

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-9vtdl

Killing

Stopping container manager

openshift-jobset-operator

kubelet

jobset-controller-manager-7cf664dd55-w9468

Pulled

Successfully pulled image "registry.redhat.io/job-set/jobset-rhel9@sha256:8d2a70fe90b2414cdb269d217703b7e5b3409f9276acf671721b1abc67a0b987" in 3.68s (3.68s including waiting). Image size: 182184678 bytes.

openshift-jobset-operator

kubelet

jobset-controller-manager-7cf664dd55-w9468

Created

Container created

openshift-jobset-operator

jobset-controller-manager-7cf664dd55-w9468_bef3c1c2-7fdc-4bb6-8312-220dc174e1be

6d4f6a47.jobset.x-k8s.io

LeaderElection

jobset-controller-manager-7cf664dd55-w9468_bef3c1c2-7fdc-4bb6-8312-220dc174e1be became leader

openshift-jobset-operator

kubelet

jobset-controller-manager-7cf664dd55-w9468

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

kuadrant-system

replicaset-controller

limitador-limitador-74dfcfb894

SuccessfulDelete

Deleted pod: limitador-limitador-74dfcfb894-svc5b

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-svc5b

Killing

Stopping container limitador

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled down replica set limitador-limitador-74dfcfb894 from 1 to 0

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-5njvh_47e56764-c1a6-4af2-9632-990118c8762e

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-678d87766f-5njvh_47e56764-c1a6-4af2-9632-990118c8762e became leader

opendatahub

kubelet

odh-model-controller-769875db56-d2kkp

Pulling

Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775"

opendatahub

replicaset-controller

model-serving-api-75bd89fdfc

SuccessfulCreate

Created pod: model-serving-api-75bd89fdfc-2htfm

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulCreate

Created pod: istiod-openshift-gateway-7ddd9cfdc8-6p6b2

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1

opendatahub

replicaset-controller

kserve-controller-manager-776cc66d47

SuccessfulCreate

Created pod: kserve-controller-manager-776cc66d47-cfpwn

opendatahub

multus

model-serving-api-75bd89fdfc-2htfm

AddedInterface

Add eth0 [10.133.0.28/23] from ovn-kubernetes

opendatahub

kubelet

model-serving-api-75bd89fdfc-2htfm

Pulling

Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8"

opendatahub

replicaset-controller

llmisvc-controller-manager-869d5dcc5d

SuccessfulCreate

Created pod: llmisvc-controller-manager-869d5dcc5d-lkqpk

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1

opendatahub

kubelet

kserve-controller-manager-776cc66d47-cfpwn

Pulling

Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d"

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-lkqpk

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321"

opendatahub

multus

llmisvc-controller-manager-869d5dcc5d-lkqpk

AddedInterface

Add eth0 [10.132.0.33/23] from ovn-kubernetes

opendatahub

deployment-controller

kserve-controller-manager

ScalingReplicaSet

Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1

opendatahub

deployment-controller

model-serving-api

ScalingReplicaSet

Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1

opendatahub

multus

kserve-controller-manager-776cc66d47-cfpwn

AddedInterface

Add eth0 [10.132.0.32/23] from ovn-kubernetes

opendatahub

multus

odh-model-controller-769875db56-d2kkp

AddedInterface

Add eth0 [10.133.0.29/23] from ovn-kubernetes

opendatahub

deployment-controller

odh-model-controller

ScalingReplicaSet

Scaled up replica set odh-model-controller-769875db56 from 0 to 1

opendatahub

replicaset-controller

odh-model-controller-769875db56

SuccessfulCreate

Created pod: odh-model-controller-769875db56-d2kkp

openshift-ingress

multus

istiod-openshift-gateway-7ddd9cfdc8-6p6b2

AddedInterface

Add eth0 [10.132.0.34/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-6p6b2

Started

Container started

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-6p6b2

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-6p6b2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-4dlgl

Killing

Stopping container discovery

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7876bfcb74

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7876bfcb74-4dlgl

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0

opendatahub

kubelet

kserve-controller-manager-776cc66d47-cfpwn

Created

Container created

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-lkqpk

Created

Container created

opendatahub

kubelet

kserve-controller-manager-776cc66d47-cfpwn

Started

Container started

opendatahub

kubelet

kserve-controller-manager-776cc66d47-cfpwn

Pulled

Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 2.524s (2.524s including waiting). Image size: 247676581 bytes.

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-lkqpk

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 2.754s (2.754s including waiting). Image size: 235100981 bytes.

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-lkqpk

Started

Container started

opendatahub

kubelet

odh-model-controller-769875db56-d2kkp

Started

Container started

opendatahub

kubelet

odh-model-controller-769875db56-d2kkp

Created

Container created

opendatahub

kubelet

odh-model-controller-769875db56-d2kkp

Pulled

Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 2.991s (2.991s including waiting). Image size: 236755567 bytes.

opendatahub

kubelet

model-serving-api-75bd89fdfc-2htfm

Started

Container started

opendatahub

kubelet

model-serving-api-75bd89fdfc-2htfm

Pulled

Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 2.37s (2.37s including waiting). Image size: 184925761 bytes.

opendatahub

kubelet

model-serving-api-75bd89fdfc-2htfm

Created

Container created

opendatahub

kserve-controller-manager-776cc66d47-cfpwn_9e040974-be91-40ad-a4bc-8b7b224cd348

kserve-controller-manager-leader-lock

LeaderElection

kserve-controller-manager-776cc66d47-cfpwn_9e040974-be91-40ad-a4bc-8b7b224cd348 became leader

opendatahub

odh-model-controller-769875db56-d2kkp_3e273ccc-9b36-47bc-b43f-ef3fff2b7f04

odh-model-controller.opendatahub.io

LeaderElection

odh-model-controller-769875db56-d2kkp_3e273ccc-9b36-47bc-b43f-ef3fff2b7f04 became leader

opendatahub

llmisvc-controller-manager-869d5dcc5d-lkqpk_8507af78-5034-4c57-a618-6135eebf2ae8

llminferenceservice-kserve-controller-manager

LeaderElection

llmisvc-controller-manager-869d5dcc5d-lkqpk_8507af78-5034-4c57-a618-6135eebf2ae8 became leader
(x4)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub

multus

maas-controller-796596f549-zjfq7

AddedInterface

Add eth0 [10.132.0.35/23] from ovn-kubernetes

opendatahub

replicaset-controller

maas-controller-796596f549

SuccessfulCreate

Created pod: maas-controller-796596f549-zjfq7

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled up replica set maas-controller-796596f549 from 0 to 1

opendatahub

kubelet

maas-controller-796596f549-zjfq7

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:f6fe27d5bc148a871e93b1383cae04799a1b4bfc2d1f9858dad18ea42d6f2045"

opendatahub

replicaset-controller

maas-controller-5b555b5c85

SuccessfulCreate

Created pod: maas-controller-5b555b5c85-jv2xg

opendatahub

multus

maas-controller-5b555b5c85-jv2xg

AddedInterface

Add eth0 [10.132.0.36/23] from ovn-kubernetes

opendatahub

kubelet

maas-controller-5b555b5c85-jv2xg

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:f6fe27d5bc148a871e93b1383cae04799a1b4bfc2d1f9858dad18ea42d6f2045"

opendatahub

kubelet

maas-controller-796596f549-zjfq7

Started

Container started

opendatahub

kubelet

maas-controller-5b555b5c85-jv2xg

Started

Container started

opendatahub

kubelet

maas-controller-796596f549-zjfq7

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:f6fe27d5bc148a871e93b1383cae04799a1b4bfc2d1f9858dad18ea42d6f2045" in 3.955s (3.956s including waiting). Image size: 200094737 bytes.

opendatahub

kubelet

maas-controller-5b555b5c85-jv2xg

Created

Container created

opendatahub

kubelet

maas-controller-796596f549-zjfq7

Created

Container created

opendatahub

kubelet

maas-controller-5b555b5c85-jv2xg

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:f6fe27d5bc148a871e93b1383cae04799a1b4bfc2d1f9858dad18ea42d6f2045" in 3.429s (3.429s including waiting). Image size: 200094737 bytes.

opendatahub

maas-controller-796596f549-zjfq7_b34783c9-e895-4bb1-904f-6518bf2e5ea3

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-796596f549-zjfq7_b34783c9-e895-4bb1-904f-6518bf2e5ea3 became leader

opendatahub

deployment-controller

usage-logs-tenancy-proxy

ScalingReplicaSet

Scaled up replica set usage-logs-tenancy-proxy-9c5d49bcd from 0 to 1

opendatahub

replicaset-controller

usage-logs-tenancy-proxy-9c5d49bcd

SuccessfulCreate

Created pod: usage-logs-tenancy-proxy-9c5d49bcd-7shdp

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-7shdp

Pulling

Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172"

opendatahub

multus

usage-logs-tenancy-proxy-9c5d49bcd-7shdp

AddedInterface

Add eth0 [10.133.0.30/23] from ovn-kubernetes

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled down replica set maas-controller-796596f549 from 1 to 0

opendatahub

replicaset-controller

maas-controller-796596f549

SuccessfulDelete

Deleted pod: maas-controller-796596f549-zjfq7

opendatahub

kubelet

maas-controller-796596f549-zjfq7

Killing

Stopping container manager

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-7shdp

Pulled

Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 12.555s (12.555s including waiting). Image size: 1089502687 bytes.

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-7shdp

Started

Container started

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-7shdp

Created

Container created
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

opendatahub

maas-controller-5b555b5c85-jv2xg_03455c01-fd61-47eb-8e94-65f00a3a97e7

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-5b555b5c85-jv2xg_03455c01-fd61-47eb-8e94-65f00a3a97e7 became leader

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-6f6944bcdd from 0 to 1
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-24rkw

Started

Container started
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-24rkw

Created

Container created

kuadrant-system

replicaset-controller

authorino-6f6944bcdd

SuccessfulDelete

Deleted pod: authorino-6f6944bcdd-p8nf4

kuadrant-system

replicaset-controller

authorino-6f6944bcdd

SuccessfulCreate

Created pod: authorino-6f6944bcdd-p8nf4

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-6889cbd8c4 from 0 to 1

kuadrant-system

replicaset-controller

authorino-6889cbd8c4

SuccessfulCreate

Created pod: authorino-6889cbd8c4-tckkg

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-6f6944bcdd from 1 to 0

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-24rkw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-57c4747b99 from 0 to 1

kuadrant-system

kubelet

authorino-6889cbd8c4-tckkg

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

replicaset-controller

authorino-57c4747b99

SuccessfulCreate

Created pod: authorino-57c4747b99-pjh9n

kuadrant-system

kubelet

authorino-57c4747b99-pjh9n

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-6889cbd8c4 from 1 to 0

kuadrant-system

multus

authorino-6889cbd8c4-tckkg

AddedInterface

Add eth0 [10.134.0.41/23] from ovn-kubernetes

kuadrant-system

multus

authorino-57c4747b99-pjh9n

AddedInterface

Add eth0 [10.134.0.42/23] from ovn-kubernetes

kuadrant-system

replicaset-controller

authorino-6889cbd8c4

SuccessfulDelete

Deleted pod: authorino-6889cbd8c4-tckkg

kuadrant-system

kubelet

authorino-57c4747b99-pjh9n

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 711ms (711ms including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-57c4747b99-pjh9n

Started

Container started

kuadrant-system

kubelet

authorino-6889cbd8c4-tckkg

Started

Container started

kuadrant-system

kubelet

authorino-57c4747b99-pjh9n

Created

Container created

kuadrant-system

kubelet

authorino-6889cbd8c4-tckkg

Created

Container created

kuadrant-system

kubelet

authorino-6889cbd8c4-tckkg

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 797ms (797ms including waiting). Image size: 204096810 bytes.

kuadrant-system

replicaset-controller

authorino-56c6c9ddd5

SuccessfulDelete

Deleted pod: authorino-56c6c9ddd5-dm2qp

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-56c6c9ddd5 from 1 to 0

kuadrant-system

kubelet

authorino-6889cbd8c4-tckkg

Killing

Stopping container authorino

kuadrant-system

kubelet

authorino-56c6c9ddd5-dm2qp

Killing

Stopping container authorino

opendatahub

multus

llmisvc-controller-manager-6466c554d8-j8qxd

AddedInterface

Add eth0 [10.132.0.38/23] from ovn-kubernetes

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1

opendatahub

replicaset-controller

llmisvc-controller-manager-6466c554d8

SuccessfulCreate

Created pod: llmisvc-controller-manager-6466c554d8-j8qxd

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-j8qxd

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321"
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-24rkw

Unhealthy

Startup probe failed: Get "http://10.134.0.26:15021/healthz/ready": dial tcp 10.134.0.26:15021: connect: connection refused

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-j8qxd

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 383ms (383ms including waiting). Image size: 235100981 bytes.

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-j8qxd

Created

Container created

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-j8qxd

Started

Container started
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-24rkw

ProbeError

Startup probe error: Get "http://10.134.0.26:15021/healthz/ready": dial tcp 10.134.0.26:15021: connect: connection refused body:

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

opendatahub

replicaset-controller

llmisvc-controller-manager-869d5dcc5d

SuccessfulDelete

Deleted pod: llmisvc-controller-manager-869d5dcc5d-lkqpk

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-lkqpk

Killing

Stopping container manager

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0

opendatahub

kubelet

ai-gateway-controller-7d85675f98-pb8cr

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "ai-gateway-controller-metrics-tls" not found

opendatahub

replicaset-controller

ai-gateway-controller-7d85675f98

SuccessfulCreate

Created pod: ai-gateway-controller-7d85675f98-pb8cr

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled up replica set ai-gateway-controller-7d85675f98 from 0 to 1

opendatahub

multus

ai-gateway-controller-7d85675f98-pb8cr

AddedInterface

Add eth0 [10.133.0.32/23] from ovn-kubernetes

opendatahub

kubelet

ai-gateway-controller-7d85675f98-pb8cr

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:latest"
(x5)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub

kubelet

ai-gateway-controller-7d85675f98-pb8cr

Started

Container started

opendatahub

ai-gateway-controller-7d85675f98-pb8cr_a7d6e9f9-37e5-4df5-aa8f-0d6db682055c

ai-gateway-controller-leader.opendatahub.io

LeaderElection

ai-gateway-controller-7d85675f98-pb8cr_a7d6e9f9-37e5-4df5-aa8f-0d6db682055c became leader

opendatahub

kubelet

ai-gateway-controller-7d85675f98-pb8cr

Created

Container created

opendatahub

kubelet

ai-gateway-controller-7d85675f98-pb8cr

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:latest" in 2.861s (2.861s including waiting). Image size: 183979434 bytes.
(x4)

openshift-ingress

replicaset-controller

payload-processing-55fbbbf9dc

FailedCreate

Error creating: pods "payload-processing-55fbbbf9dc-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

multus

payload-pre-processing-7b98bb678c-j6sq9

AddedInterface

Add eth0 [10.134.0.43/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-55fbbbf9dc

SuccessfulCreate

Created pod: payload-processing-55fbbbf9dc-sbd6p

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-sbd6p

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

multus

payload-processing-55fbbbf9dc-sbd6p

AddedInterface

Add eth0 [10.133.0.33/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

FailedCreate

Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-j6sq9

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-55fbbbf9dc from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

SuccessfulCreate

Created pod: payload-pre-processing-7b98bb678c-j6sq9

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-j6sq9

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.974s (2.974s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-sbd6p

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-j6sq9

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-j6sq9

Created

Container created

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-sbd6p

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.519s (2.519s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-sbd6p

Started

Container started

opendatahub

llmisvc-controller-manager-6466c554d8-j8qxd_31abccc5-ff70-404b-89c1-3e6402839ccc

llminferenceservice-kserve-controller-manager

LeaderElection

llmisvc-controller-manager-6466c554d8-j8qxd_31abccc5-ff70-404b-89c1-3e6402839ccc became leader

opendatahub

replicaset-controller

ai-gateway-controller-6f755f6879

SuccessfulCreate

Created pod: ai-gateway-controller-6f755f6879-xh4h5

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled up replica set ai-gateway-controller-6f755f6879 from 0 to 1

opendatahub

kubelet

ai-gateway-controller-6f755f6879-xh4h5

Started

Container started

opendatahub

kubelet

ai-gateway-controller-6f755f6879-xh4h5

Created

Container created

opendatahub

kubelet

ai-gateway-controller-6f755f6879-xh4h5

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:latest" in 422ms (422ms including waiting). Image size: 183979434 bytes.

opendatahub

kubelet

ai-gateway-controller-6f755f6879-xh4h5

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:latest"

opendatahub

multus

ai-gateway-controller-6f755f6879-xh4h5

AddedInterface

Add eth0 [10.133.0.34/23] from ovn-kubernetes

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled down replica set ai-gateway-controller-7d85675f98 from 1 to 0

opendatahub

kubelet

ai-gateway-controller-7d85675f98-pb8cr

Killing

Stopping container manager

opendatahub

replicaset-controller

ai-gateway-controller-7d85675f98

SuccessfulDelete

Deleted pod: ai-gateway-controller-7d85675f98-pb8cr

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Deployment llm/e2e-distinct-2-simulated-kserve

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.DestinationRule llm/e2e-distinct-2-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.HTTPRoute llm/e2e-distinct-2-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Service llm/e2e-distinct-2-simulated-kserve-workload-svc

llm

replicaset-controller

e2e-distinct-2-simulated-kserve-cbd4f5586

SuccessfulCreate

Created pod: e2e-distinct-2-simulated-kserve-cbd4f5586-h557j

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine-default

llm

deployment-controller

e2e-distinct-2-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-distinct-2-simulated-kserve-cbd4f5586 from 0 to 1

llm

multus

e2e-distinct-2-simulated-kserve-cbd4f5586-h557j

AddedInterface

Add eth0 [10.133.0.35/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.NetworkPolicy llm/e2e-distinct-2-simulated-prometheus-scraping

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-h557j

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-h557j

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.466s (1.466s including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-h557j

Started

Container started

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-h557j

Created

Container created

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.HTTPRoute llm/e2e-distinct-simulated-kserve-route

llm

replicaset-controller

e2e-distinct-simulated-kserve-7b998b79d9

SuccessfulCreate

Created pod: e2e-distinct-simulated-kserve-7b998b79d9-vzmpw

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Deployment llm/e2e-distinct-simulated-kserve

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Service llm/e2e-distinct-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.DestinationRule llm/e2e-distinct-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine-default

llm

deployment-controller

e2e-distinct-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-distinct-simulated-kserve-7b998b79d9 from 0 to 1

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.NetworkPolicy llm/e2e-distinct-simulated-prometheus-scraping

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-vzmpw

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

multus

e2e-distinct-simulated-kserve-7b998b79d9-vzmpw

AddedInterface

Add eth0 [10.134.0.44/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.DestinationRule llm/e2e-embedding-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine-default

llm

deployment-controller

e2e-embedding-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-embedding-simulated-kserve-67bf594cd6 from 0 to 1

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Deployment llm/e2e-trlp-test-simulated-kserve

llm

multus

e2e-embedding-simulated-kserve-67bf594cd6-xg2gk

AddedInterface

Add eth0 [10.133.0.36/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.NetworkPolicy llm/e2e-embedding-simulated-prometheus-scraping

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-vzmpw

Started

Container started

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-vzmpw

Created

Container created

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Service llm/e2e-trlp-test-simulated-kserve-workload-svc

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-vzmpw

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.437s (1.437s including waiting). Image size: 98346788 bytes.

llm

deployment-controller

e2e-trlp-test-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-trlp-test-simulated-kserve-69b588897d from 0 to 1

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Deployment llm/e2e-embedding-simulated-kserve

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Service llm/e2e-embedding-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.HTTPRoute llm/e2e-embedding-simulated-kserve-route

llm

replicaset-controller

e2e-embedding-simulated-kserve-67bf594cd6

SuccessfulCreate

Created pod: e2e-embedding-simulated-kserve-67bf594cd6-xg2gk

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.DestinationRule llm/e2e-trlp-test-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.HTTPRoute llm/e2e-trlp-test-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine-default

llm

replicaset-controller

e2e-trlp-test-simulated-kserve-69b588897d

SuccessfulCreate

Created pod: e2e-trlp-test-simulated-kserve-69b588897d-6wwp5

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-6wwp5

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 284ms (284ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-6wwp5

Created

Container created

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-6wwp5

Started

Container started

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-xg2gk

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-6wwp5

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

multus

e2e-trlp-test-simulated-kserve-69b588897d-6wwp5

AddedInterface

Add eth0 [10.134.0.45/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.NetworkPolicy llm/e2e-trlp-test-simulated-prometheus-scraping

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-xg2gk

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 289ms (289ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-xg2gk

Created

Container created

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-xg2gk

Started

Container started

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.DestinationRule llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Deployment llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve

llm

replicaset-controller

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc

SuccessfulCreate

Created pod: e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8g4phf

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/e2e-unconab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.NetworkPolicy llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-prometheus-scraping

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/eab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.HTTPRoute llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Service llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc

llm

deployment-controller

e2e-unconfigured-facebook-opt-125m-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc from 0 to 1

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8g4phf

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8g4phf

Started

Container started

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8g4phf

Created

Container created

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8g4phf

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 276ms (276ms including waiting). Image size: 98346788 bytes.

llm

multus

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8g4phf

AddedInterface

Add eth0 [10.133.0.37/23] from ovn-kubernetes

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Deployment llm/echo-simulator-kserve

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.NetworkPolicy llm/echo-simulator-prometheus-scraping

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.DestinationRule llm/echo-simulator-kserve-workload-svc

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Service llm/echo-simulator-kserve-workload-svc

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Secret llm/echo-simulator-kserve-self-signed-certs

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.HTTPRoute llm/echo-simulator-kserve-route

llm

deployment-controller

echo-simulator-kserve

ScalingReplicaSet

Scaled up replica set echo-simulator-kserve-64b6f8f96c from 0 to 1

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine

llm

replicaset-controller

echo-simulator-kserve-64b6f8f96c

SuccessfulCreate

Created pod: echo-simulator-kserve-64b6f8f96c-npjcg

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine-default

llm

multus

echo-simulator-kserve-64b6f8f96c-npjcg

AddedInterface

Add eth0 [10.134.0.46/23] from ovn-kubernetes

llm

kubelet

echo-simulator-kserve-64b6f8f96c-npjcg

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0"

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs

llm

deployment-controller

facebook-opt-125m-simulated-kserve

ScalingReplicaSet

Scaled up replica set facebook-opt-125m-simulated-kserve-84b4ff656c from 0 to 1

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Deployment llm/facebook-opt-125m-simulated-kserve

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.NetworkPolicy llm/facebook-opt-125m-simulated-prometheus-scraping

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.HTTPRoute llm/facebook-opt-125m-simulated-kserve-route

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.DestinationRule llm/facebook-opt-125m-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Service llm/facebook-opt-125m-simulated-kserve-workload-svc

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-56c8w

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

multus

facebook-opt-125m-simulated-kserve-84b4ff656c-56c8w

AddedInterface

Add eth0 [10.134.0.47/23] from ovn-kubernetes

llm

replicaset-controller

facebook-opt-125m-simulated-kserve-84b4ff656c

SuccessfulCreate

Created pod: facebook-opt-125m-simulated-kserve-84b4ff656c-56c8w

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine-default

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-56c8w

Started

Container started

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-56c8w

Created

Container created

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-56c8w

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 321ms (321ms including waiting). Image size: 98346788 bytes.

llm

kubelet

echo-simulator-kserve-64b6f8f96c-npjcg

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0" in 2.175s (2.175s including waiting). Image size: 78208878 bytes.

llm

deployment-controller

premium-simulated-simulated-premium-kserve

ScalingReplicaSet

Scaled up replica set premium-simulated-simulated-premium-kserve-5f5c4b459f from 0 to 1

llm

kubelet

echo-simulator-kserve-64b6f8f96c-npjcg

Created

Container created

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.PodMonitor llm/p75d89895ad19b4ac62612d08bfe7f574-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs

llm

replicaset-controller

premium-simulated-simulated-premium-kserve-5f5c4b459f

SuccessfulCreate

Created pod: premium-simulated-simulated-premium-kserve-5f5c4b459f-6hb6w

opendatahub

ai-gateway-controller-6f755f6879-xh4h5_5fde2ac1-f391-44a6-b5d9-6f726f9aba66

ai-gateway-controller-leader.opendatahub.io

LeaderElection

ai-gateway-controller-6f755f6879-xh4h5_5fde2ac1-f391-44a6-b5d9-6f726f9aba66 became leader

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Deployment llm/premium-simulated-simulated-premium-kserve

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Service llm/premium-simulated-simulated-premium-kserve-workload-svc

llm

kubelet

echo-simulator-kserve-64b6f8f96c-npjcg

Started

Container started

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.HTTPRoute llm/premium-simulated-simulated-premium-kserve-route

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.DestinationRule llm/premium-simulated-simulated-premium-kserve-workload-svc

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.PodMonitor llm/premium-simulated-simulated-premium-kserve-llmisvc-engine

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.NetworkPolicy llm/premium-simulated-simulated-premium-prometheus-scraping

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-6hb6w

Created

Container created

llm

multus

premium-simulated-simulated-premium-kserve-5f5c4b459f-6hb6w

AddedInterface

Add eth0 [10.134.0.48/23] from ovn-kubernetes
(x2)

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Updated

Updated v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-6hb6w

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-6hb6w

Started

Container started

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-distinct-2-simulated] is Ready

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-6hb6w

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 316ms (316ms including waiting). Image size: 98346788 bytes.

llm

LLMInferenceServiceController

e2e-embedding-simulated

Updated

Updated v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-simulated

Updated

Updated v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Updated

Updated v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Updated

Updated v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs

llm

LLMInferenceServiceController

echo-simulator

Updated

Updated v1.Secret llm/echo-simulator-kserve-self-signed-certs

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Updated

Updated v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Updated

Updated v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-unconfigured-facebook-opt-125m-simulated] is Ready

llm

LLMInferenceServiceController

e2e-embedding-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-embedding-simulated] is Ready

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-trlp-test-simulated] is Ready

llm

LLMInferenceServiceController

e2e-distinct-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-distinct-simulated] is Ready

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

LLMInferenceServiceReady

LLMInferenceService [facebook-opt-125m-simulated] is Ready

llm

LLMInferenceServiceController

echo-simulator

LLMInferenceServiceReady

LLMInferenceService [echo-simulator] is Ready

kuadrant-system

replicaset-controller

authorino-795bcb67b4

SuccessfulCreate

Created pod: authorino-795bcb67b4-xgtc6

kuadrant-system

kubelet

authorino-795bcb67b4-xgtc6

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

multus

authorino-795bcb67b4-xgtc6

AddedInterface

Add eth0 [10.134.0.49/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-795bcb67b4-xgtc6

Started

Container started

kuadrant-system

kubelet

authorino-795bcb67b4-xgtc6

Created

Container created

kuadrant-system

kubelet

authorino-795bcb67b4-xgtc6

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 1.247s (1.247s including waiting). Image size: 204096810 bytes.

kuadrant-system

replicaset-controller

authorino-57c4747b99

SuccessfulDelete

Deleted pod: authorino-57c4747b99-pjh9n
(x2)

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

(combined from similar events): Scaled down replica set authorino-57c4747b99 from 1 to 0

kuadrant-system

kubelet

authorino-57c4747b99-pjh9n

Killing

Stopping container authorino

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-75fd9f5d47-sn2pt

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-75fd9f5d47-5qw4z

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-75fd9f5d47-f8wtz

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-sn2pt

AddedInterface

Add eth0 [10.132.0.39/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-5qw4z

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-f8wtz

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-5qw4z

AddedInterface

Add eth0 [10.133.0.38/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-f8wtz

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-f8wtz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-f8wtz

AddedInterface

Add eth0 [10.134.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-sn2pt

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-5qw4z

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.235s (4.235s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-5qw4z

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-5qw4z

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-sn2pt

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-sn2pt

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-sn2pt

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.767s (3.767s including waiting). Image size: 183555333 bytes.
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-f8wtz

Unhealthy

Startup probe failed: Get "http://10.134.0.50:15021/healthz/ready": dial tcp 10.134.0.50:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-f8wtz

ProbeError

Startup probe error: Get "http://10.134.0.50:15021/healthz/ready": dial tcp 10.134.0.50:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-sn2pt

Unhealthy

Startup probe failed: Get "http://10.132.0.39:15021/healthz/ready": dial tcp 10.132.0.39:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-5qw4z

Unhealthy

Startup probe failed: Get "http://10.133.0.38:15021/healthz/ready": dial tcp 10.133.0.38:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-5qw4z

ProbeError

Startup probe error: Get "http://10.133.0.38:15021/healthz/ready": dial tcp 10.133.0.38:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-sn2pt

ProbeError

Startup probe error: Get "http://10.132.0.39:15021/healthz/ready": dial tcp 10.132.0.39:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-75fd9f5d47-6pk2b

openshift-ingress

gateway_labeler_controller

e2e-worker-w3-a110ef

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-a110ef

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-75fd9f5d47-qtkkt

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-75fd9f5d47-h4446

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-75fd9f5d47-vmkcq

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 8; reason:

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-75fd9f5d47 from 4 to 8

openshift-ingress

deployment-controller

e2e-worker-w3-a110ef-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-a110ef-openshift-default-754f5584f9 from 0 to 1

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-6pk2b

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-6pk2b

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-6pk2b

AddedInterface

Add eth0 [10.133.0.39/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-vmkcq

AddedInterface

Add eth0 [10.132.0.40/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-qtkkt

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-qtkkt

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-qtkkt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-worker-w0-2e80fb-openshift-default-5585568575

SuccessfulCreate

Created pod: e2e-worker-w0-2e80fb-openshift-default-5585568575-gkljq

openshift-ingress

deployment-controller

e2e-praxis-a-000ffb-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-praxis-a-000ffb-openshift-default-74665c677 from 0 to 1

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-qtkkt

AddedInterface

Add eth0 [10.133.0.40/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w0-2e80fb-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-2e80fb-openshift-default-5585568575 from 0 to 1

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-6pk2b

Started

Container started

openshift-ingress

multus

e2e-worker-w0-2e80fb-openshift-default-5585568575-gkljq

AddedInterface

Add eth0 [10.134.0.53/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-vmkcq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-vmkcq

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-vmkcq

Started

Container started

openshift-ingress

gateway_labeler_controller

e2e-worker-w0-2e80fb

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-2e80fb

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-h4446

Started

Container started

openshift-ingress

replicaset-controller

e2e-worker-w3-a110ef-openshift-default-754f5584f9

SuccessfulCreate

Created pod: e2e-worker-w3-a110ef-openshift-default-754f5584f9-m8mhg

openshift-ingress

gateway_labeler_controller

e2e-praxis-a-000ffb

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-praxis-a-000ffb

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-h4446

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-h4446

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w3-a110ef-openshift-default-754f5584f9-m8mhg

AddedInterface

Add eth0 [10.134.0.52/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-h4446

AddedInterface

Add eth0 [10.134.0.51/23] from ovn-kubernetes

openshift-ingress

multus

e2e-praxis-a-000ffb-openshift-default-74665c677-9xtr6

AddedInterface

Add eth0 [10.133.0.41/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-praxis-a-000ffb-openshift-default-74665c677

SuccessfulCreate

Created pod: e2e-praxis-a-000ffb-openshift-default-74665c677-9xtr6

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w3-a110ef

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w3-a110ef-c6fc947bc from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w3-a110ef-5766965cdf

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w3-a110ef-5766965cdf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-a110ef: serviceaccount "payload-processing-e2e-worker-w3-a110ef" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w3-a110ef-5766965cdf

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w3-a110ef-5766965cdf-gd9dj

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w3-a110ef

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w3-a110ef-5766965cdf from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w3-a110ef-c6fc947bc

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w3-a110ef-c6fc947bc-hx6lh
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w3-a110ef-c6fc947bc

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w3-a110ef-c6fc947bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-a110ef: serviceaccount "payload-processing-e2e-worker-w3-a110ef" not found

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-a110ef-5766965cdf-gd9dj

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-a110ef-5766965cdf-gd9dj

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-a110ef-c6fc947bc-hx6lh

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-a110ef-c6fc947bc-hx6lh

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-worker-w3-a110ef-c6fc947bc-hx6lh

AddedInterface

Add eth0 [10.134.0.54/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-a110ef-5766965cdf-gd9dj

Started

Container started

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w0-2e80fb

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w0-2e80fb-77fd4bc6bc from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-2e80fb-77fd4bc6bc

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w0-2e80fb-77fd4bc6bc-ncz2j
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-2e80fb-77fd4bc6bc

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w0-2e80fb-77fd4bc6bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-2e80fb: serviceaccount "payload-processing-e2e-worker-w0-2e80fb" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-2e80fb-7f8cb49b89

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w0-2e80fb-7f8cb49b89-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-2e80fb: serviceaccount "payload-processing-e2e-worker-w0-2e80fb" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-2e80fb-7f8cb49b89

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w0-2e80fb-7f8cb49b89-d2pv9

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-a110ef-c6fc947bc-hx6lh

Started

Container started

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w0-2e80fb

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w0-2e80fb-7f8cb49b89 from 0 to 1

openshift-ingress

multus

payload-processing-e2e-worker-w3-a110ef-5766965cdf-gd9dj

AddedInterface

Add eth0 [10.133.0.42/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-2e80fb-7f8cb49b89-d2pv9

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-2e80fb-77fd4bc6bc-ncz2j

Started

Container started

openshift-ingress

multus

payload-processing-e2e-worker-w0-2e80fb-7f8cb49b89-d2pv9

AddedInterface

Add eth0 [10.134.0.56/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-2e80fb-7f8cb49b89-d2pv9

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-2e80fb-7f8cb49b89-d2pv9

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-worker-w0-2e80fb-77fd4bc6bc-ncz2j

AddedInterface

Add eth0 [10.134.0.55/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-2e80fb-77fd4bc6bc-ncz2j

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-2e80fb-77fd4bc6bc-ncz2j

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

payload-pre-processing-e2e-praxis-a-000ffb

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-praxis-a-000ffb-76b97879b7 from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-praxis-a-000ffb-76b97879b7-lbxd8

AddedInterface

Add eth0 [10.134.0.57/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-praxis-a-000ffb-69745656c9

SuccessfulCreate

Created pod: payload-processing-e2e-praxis-a-000ffb-69745656c9-lmgkp

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-praxis-a-000ffb-76b97879b7

SuccessfulCreate

Created pod: payload-pre-processing-e2e-praxis-a-000ffb-76b97879b7-lbxd8

openshift-ingress

deployment-controller

payload-processing-e2e-praxis-a-000ffb

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-praxis-a-000ffb-69745656c9 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-praxis-a-000ffb-69745656c9

FailedCreate

Error creating: pods "payload-processing-e2e-praxis-a-000ffb-69745656c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-praxis-a-000ffb: serviceaccount "payload-processing-e2e-praxis-a-000ffb" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-praxis-a-000ffb-76b97879b7

FailedCreate

Error creating: pods "payload-pre-processing-e2e-praxis-a-000ffb-76b97879b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-praxis-a-000ffb: serviceaccount "payload-processing-e2e-praxis-a-000ffb" not found

openshift-ingress

kubelet

payload-processing-e2e-praxis-a-000ffb-69745656c9-lmgkp

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-a-000ffb-76b97879b7-lbxd8

Started

Container started
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-qtkkt

Unhealthy

Startup probe failed: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-qtkkt

ProbeError

Startup probe error: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-6pk2b

ProbeError

Startup probe error: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-6pk2b

Unhealthy

Startup probe failed: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-h4446

Unhealthy

Startup probe failed: Get "http://10.134.0.51:15021/healthz/ready": dial tcp 10.134.0.51:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-h4446

ProbeError

Startup probe error: Get "http://10.134.0.51:15021/healthz/ready": dial tcp 10.134.0.51:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-vmkcq

ProbeError

Startup probe error: Get "http://10.132.0.40:15021/healthz/ready": dial tcp 10.132.0.40:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-vmkcq

Unhealthy

Startup probe failed: Get "http://10.132.0.40:15021/healthz/ready": dial tcp 10.132.0.40:15021: connect: connection refused

openshift-ingress

multus

payload-processing-e2e-praxis-a-000ffb-69745656c9-lmgkp

AddedInterface

Add eth0 [10.132.0.41/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-a-000ffb-76b97879b7-lbxd8

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-a-000ffb-76b97879b7-lbxd8

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-praxis-a-000ffb-69745656c9-lmgkp

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-qtkkt

ProbeError

Startup probe error: Get "http://10.133.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

payload-processing-e2e-praxis-a-000ffb-69745656c9-lmgkp

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.632s (2.632s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-e2e-praxis-a-000ffb-69745656c9-lmgkp

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-qtkkt

Unhealthy

Startup probe failed: Get "http://10.133.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 10; reason:

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-75fd9f5d47-4d57x

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-75fd9f5d47 from 8 to 10

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulCreate

(combined from similar events): Created pod: maas-default-gateway-openshift-default-75fd9f5d47-qwf92

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-qwf92

AddedInterface

Add eth0 [10.134.0.58/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-qwf92

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-qwf92

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-4d57x

AddedInterface

Add eth0 [10.132.0.42/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-4d57x

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-4d57x

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-qwf92

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-4d57x

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-m8mhg

ProbeError

Readiness probe error: Get "http://10.134.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-m8mhg

Unhealthy

Readiness probe failed: Get "http://10.134.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-m8mhg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-praxis-a-000ffb-openshift-default-74665c677-9xtr6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-m8mhg

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-m8mhg

Started

Container started
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-4d57x

Unhealthy

Startup probe failed: Get "http://10.132.0.42:15021/healthz/ready": dial tcp 10.132.0.42:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-praxis-a-000ffb-openshift-default-74665c677-9xtr6

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-praxis-a-000ffb-openshift-default-74665c677-9xtr6

Started

Container started
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-4d57x

ProbeError

Startup probe error: Get "http://10.132.0.42:15021/healthz/ready": dial tcp 10.132.0.42:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-qwf92

ProbeError

Startup probe error: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-qwf92

Unhealthy

Startup probe failed: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-4d57x

ProbeError

Startup probe error: Get "http://10.132.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-4d57x

Unhealthy

Startup probe failed: Get "http://10.132.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-h4446

ProbeError

Readiness probe error: Get "http://10.134.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

gateway_labeler_controller

e2e-praxis-b-d2e277

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-praxis-b-d2e277

openshift-ingress

replicaset-controller

e2e-praxis-b-d2e277-openshift-default-94fd97fdf

SuccessfulCreate

Created pod: e2e-praxis-b-d2e277-openshift-default-94fd97fdf-t7vdm

openshift-ingress

deployment-controller

e2e-praxis-b-d2e277-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-praxis-b-d2e277-openshift-default-94fd97fdf from 0 to 1

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-vmkcq

ProbeError

Readiness probe error: Get "http://10.132.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-vmkcq

Unhealthy

Readiness probe failed: Get "http://10.132.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-h4446

Unhealthy

Readiness probe failed: Get "http://10.134.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

multus

e2e-praxis-b-d2e277-openshift-default-94fd97fdf-t7vdm

AddedInterface

Add eth0 [10.133.0.46/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

e2e-praxis-a-000ffb-openshift-default-74665c677-9xtr6

Unhealthy

Startup probe failed: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-praxis-a-000ffb-openshift-default-74665c677-9xtr6

ProbeError

Startup probe error: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused body:
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-qwf92

Unhealthy

Startup probe failed: Get "http://10.134.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-qwf92

ProbeError

Startup probe error: Get "http://10.134.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-a110ef-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x8)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-m8mhg

Unhealthy

Startup probe failed: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-m8mhg

ProbeError

Startup probe error: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused body:

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-a110ef-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-praxis-b-d2e277-579fb69f6c

FailedCreate

Error creating: pods "payload-processing-e2e-praxis-b-d2e277-579fb69f6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-praxis-b-d2e277: serviceaccount "payload-processing-e2e-praxis-b-d2e277" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-praxis-b-d2e277

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-praxis-b-d2e277-6bdc676c5f from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-praxis-b-d2e277-6bdc676c5f

SuccessfulCreate

Created pod: payload-pre-processing-e2e-praxis-b-d2e277-6bdc676c5f-g2r9n
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-praxis-b-d2e277-6bdc676c5f

FailedCreate

Error creating: pods "payload-pre-processing-e2e-praxis-b-d2e277-6bdc676c5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-praxis-b-d2e277: serviceaccount "payload-processing-e2e-praxis-b-d2e277" not found

openshift-ingress

deployment-controller

payload-processing-e2e-praxis-b-d2e277

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-praxis-b-d2e277-579fb69f6c from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-praxis-b-d2e277-6bdc676c5f-g2r9n

AddedInterface

Add eth0 [10.134.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-b-d2e277-6bdc676c5f-g2r9n

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-praxis-b-d2e277-579fb69f6c-8plsv

AddedInterface

Add eth0 [10.134.0.61/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-praxis-b-d2e277-579fb69f6c

SuccessfulCreate

Created pod: payload-processing-e2e-praxis-b-d2e277-579fb69f6c-8plsv

openshift-ingress

kubelet

payload-processing-e2e-praxis-b-d2e277-579fb69f6c-8plsv

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-praxis-b-d2e277-579fb69f6c-8plsv

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-praxis-b-d2e277-579fb69f6c-8plsv

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-b-d2e277-6bdc676c5f-g2r9n

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-b-d2e277-6bdc676c5f-g2r9n

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-gkljq

ProbeError

Readiness probe error: Get "http://10.134.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-gkljq

Unhealthy

Readiness probe failed: Get "http://10.134.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-gkljq

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-gkljq

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-gkljq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-gkljq

ProbeError

Startup probe error: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-gkljq

Unhealthy

Startup probe failed: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused

openshift-ingress

deployment-controller

e2e-shared-a-w4-bde944-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w4-bde944-openshift-default-75487f99f9 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w4-bde944

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-bde944

openshift-ingress

replicaset-controller

e2e-shared-a-w4-bde944-openshift-default-75487f99f9

SuccessfulCreate

Created pod: e2e-shared-a-w4-bde944-openshift-default-75487f99f9-wk5lx
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-2e80fb-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-2e80fb-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

multus

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-wk5lx

AddedInterface

Add eth0 [10.134.0.62/23] from ovn-kubernetes
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-a-000ffb-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-a-000ffb-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w4-bde944

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w4-bde944-99767bc7f from 0 to 1

openshift-ingress

multus

payload-processing-e2e-shared-a-w4-bde944-c67f95877-bfj8t

AddedInterface

Add eth0 [10.134.0.63/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w4-bde944

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w4-bde944-c67f95877 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w4-bde944-c67f95877

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w4-bde944-c67f95877-bfj8t
(x4)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w4-bde944-c67f95877

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w4-bde944-c67f95877-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-bde944: serviceaccount "payload-processing-e2e-shared-a-w4-bde944" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w4-bde944-99767bc7f

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w4-bde944-99767bc7f-kbcnk
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w4-bde944-99767bc7f

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w4-bde944-99767bc7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-bde944: serviceaccount "payload-processing-e2e-shared-a-w4-bde944" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w4-bde944-99767bc7f-kbcnk

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w4-bde944-99767bc7f-kbcnk

AddedInterface

Add eth0 [10.134.0.64/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w4-bde944-99767bc7f-kbcnk

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w4-bde944-c67f95877-bfj8t

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w4-bde944-99767bc7f-kbcnk

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-praxis-b-d2e277-openshift-default-94fd97fdf-t7vdm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-praxis-b-d2e277-openshift-default-94fd97fdf-t7vdm

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-praxis-b-d2e277-openshift-default-94fd97fdf-t7vdm

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w4-bde944-c67f95877-bfj8t

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w4-bde944-c67f95877-bfj8t

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-praxis-b-d2e277-openshift-default-94fd97fdf-t7vdm

Unhealthy

Startup probe failed: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-praxis-b-d2e277-openshift-default-94fd97fdf-t7vdm

ProbeError

Startup probe error: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused body:

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-b-d2e277-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-b-d2e277-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-a110ef-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-a110ef-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

e2e-shared-b-w4-9e7156-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w4-9e7156-openshift-default-6bc7994879 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w4-9e7156

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-9e7156

openshift-ingress

replicaset-controller

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879

SuccessfulCreate

Created pod: e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-pgrhg

openshift-ingress

multus

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-pgrhg

AddedInterface

Add eth0 [10.134.0.68/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-wk5lx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-wk5lx

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-wk5lx

Created

Container created

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w4-9e7156

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w4-9e7156-7957df6fc5 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w4-9e7156-7957df6fc5

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w4-9e7156-7957df6fc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-9e7156: serviceaccount "payload-processing-e2e-shared-b-w4-9e7156" not found

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w4-9e7156

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w4-9e7156-f747b4965 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w4-9e7156-f747b4965

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w4-9e7156-f747b4965-v9sds
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w4-9e7156-f747b4965

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w4-9e7156-f747b4965-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-9e7156: serviceaccount "payload-processing-e2e-shared-b-w4-9e7156" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w4-9e7156-7957df6fc5

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w4-9e7156-7957df6fc5-ctbhp

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w4-9e7156-f747b4965-v9sds

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w4-9e7156-f747b4965-v9sds

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w4-9e7156-f747b4965-v9sds

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w4-9e7156-7957df6fc5-ctbhp

Started

Container started

openshift-ingress

multus

payload-processing-e2e-shared-b-w4-9e7156-f747b4965-v9sds

AddedInterface

Add eth0 [10.134.0.70/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w4-9e7156-7957df6fc5-ctbhp

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w4-9e7156-7957df6fc5-ctbhp

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w4-9e7156-7957df6fc5-ctbhp

AddedInterface

Add eth0 [10.134.0.69/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-wk5lx

ProbeError

Startup probe error: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-wk5lx

Unhealthy

Startup probe failed: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w4-bde944-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w4-bde944-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-rgfsg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-a110ef-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-rgfsg

Created

Container created

openshift-ingress

deployment-controller

e2e-worker-w3-a110ef-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-a110ef-openshift-default-754f5584f9 from 1 to 2
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-2e80fb-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-2e80fb-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

e2e-worker-w3-a110ef-openshift-default-754f5584f9

SuccessfulCreate

Created pod: e2e-worker-w3-a110ef-openshift-default-754f5584f9-rgfsg

openshift-ingress

multus

e2e-worker-w3-a110ef-openshift-default-754f5584f9-rgfsg

AddedInterface

Add eth0 [10.133.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-rgfsg

Started

Container started

openshift-ingress

deployment-controller

e2e-praxis-a-000ffb-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-praxis-a-000ffb-openshift-default-74665c677 from 1 to 2

openshift-ingress

replicaset-controller

e2e-praxis-a-000ffb-openshift-default-74665c677

SuccessfulCreate

Created pod: e2e-praxis-a-000ffb-openshift-default-74665c677-tx8bf

openshift-ingress

multus

e2e-praxis-a-000ffb-openshift-default-74665c677-tx8bf

AddedInterface

Add eth0 [10.132.0.51/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-a-000ffb-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-praxis-a-000ffb-openshift-default-74665c677-tx8bf

Started

Container started

openshift-ingress

kubelet

e2e-praxis-a-000ffb-openshift-default-74665c677-tx8bf

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-praxis-a-000ffb-openshift-default-74665c677-tx8bf

Created

Container created
(x6)

openshift-ingress

kubelet

e2e-praxis-a-000ffb-openshift-default-74665c677-tx8bf

Unhealthy

Startup probe failed: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-rgfsg

Unhealthy

Startup probe failed: Get "http://10.133.0.50:15021/healthz/ready": dial tcp 10.133.0.50:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-rgfsg

ProbeError

Startup probe error: Get "http://10.133.0.50:15021/healthz/ready": dial tcp 10.133.0.50:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-praxis-a-000ffb-openshift-default-74665c677-tx8bf

ProbeError

Startup probe error: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-rgfsg

Unhealthy

Startup probe failed: Get "http://10.133.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-rgfsg

ProbeError

Startup probe error: Get "http://10.133.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-b-d2e277-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-b-d2e277-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-pgrhg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-pgrhg

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-pgrhg

Started

Container started

openshift-ingress

multus

e2e-worker-w0-2e80fb-openshift-default-5585568575-ndhs8

AddedInterface

Add eth0 [10.133.0.51/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w0-2e80fb-openshift-default-5585568575

SuccessfulCreate

Created pod: e2e-worker-w0-2e80fb-openshift-default-5585568575-ndhs8

openshift-ingress

deployment-controller

e2e-worker-w0-2e80fb-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-2e80fb-openshift-default-5585568575 from 1 to 2

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-2e80fb-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-ndhs8

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-ndhs8

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-ndhs8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x6)

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-pgrhg

ProbeError

Startup probe error: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-pgrhg

Unhealthy

Startup probe failed: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-ndhs8

ProbeError

Startup probe error: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-ndhs8

Unhealthy

Startup probe failed: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-ndhs8

Unhealthy

Startup probe failed: Get "http://10.133.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-ndhs8

ProbeError

Startup probe error: Get "http://10.133.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-a-000ffb-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-praxis-a-000ffb-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-praxis-a-000ffb-openshift-default-74665c677 from 2 to 3

openshift-ingress

replicaset-controller

e2e-praxis-a-000ffb-openshift-default-74665c677

SuccessfulCreate

Created pod: e2e-praxis-a-000ffb-openshift-default-74665c677-lvtlw

openshift-ingress

kubelet

e2e-praxis-a-000ffb-openshift-default-74665c677-lvtlw

Started

Container started

openshift-ingress

kubelet

e2e-praxis-a-000ffb-openshift-default-74665c677-lvtlw

Created

Container created

openshift-ingress

kubelet

e2e-praxis-a-000ffb-openshift-default-74665c677-lvtlw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-praxis-a-000ffb-openshift-default-74665c677-lvtlw

AddedInterface

Add eth0 [10.134.0.73/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

e2e-praxis-a-000ffb-openshift-default-74665c677-lvtlw

Unhealthy

Startup probe failed: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-praxis-a-000ffb-openshift-default-74665c677-lvtlw

ProbeError

Startup probe error: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused body:

openshift-ingress

multus

e2e-worker-w3-a110ef-openshift-default-754f5584f9-s2h9r

AddedInterface

Add eth0 [10.133.0.52/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w3-a110ef-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-a110ef-openshift-default-754f5584f9 from 2 to 4
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w4-bde944-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w4-bde944-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

e2e-worker-w3-a110ef-openshift-default-754f5584f9

SuccessfulCreate

Created pod: e2e-worker-w3-a110ef-openshift-default-754f5584f9-s2h9r

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-a110ef-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

e2e-worker-w3-a110ef-openshift-default-754f5584f9-2d492

AddedInterface

Add eth0 [10.132.0.57/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w3-a110ef-openshift-default-754f5584f9

SuccessfulCreate

Created pod: e2e-worker-w3-a110ef-openshift-default-754f5584f9-2d492

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-2d492

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-s2h9r

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-s2h9r

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-s2h9r

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-2d492

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-2d492

Started

Container started
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w4-9e7156-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w4-9e7156-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x6)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-2d492

ProbeError

Startup probe error: Get "http://10.132.0.57:15021/healthz/ready": dial tcp 10.132.0.57:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-2d492

Unhealthy

Startup probe failed: Get "http://10.132.0.57:15021/healthz/ready": dial tcp 10.132.0.57:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-s2h9r

ProbeError

Startup probe error: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-s2h9r

Unhealthy

Startup probe failed: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused

opendatahub

kubelet

test-curl-7360-0dc00d

Pulling

Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5"

opendatahub

multus

test-curl-7360-0dc00d

AddedInterface

Add eth0 [10.134.0.74/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-a-000ffb-76b97879b7-lbxd8

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-praxis-a-000ffb-69745656c9-lmgkp

Killing

Stopping container payload-processing

opendatahub

kubelet

test-curl-7360-0dc00d

Pulled

Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 3.156s (3.156s including waiting). Image size: 106100986 bytes.

opendatahub

kubelet

test-curl-7360-0dc00d

Created

Container created

opendatahub

kubelet

test-curl-7360-0dc00d

Started

Container started

opendatahub

kubelet

test-curl-7360-90a43e

Created

Container created

opendatahub

kubelet

test-curl-7360-90a43e

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

multus

test-curl-7360-90a43e

AddedInterface

Add eth0 [10.134.0.75/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7360-90a43e

Started

Container started

opendatahub

kubelet

test-curl-7360-0dc00d

Killing

Stopping container test-curl-7360-0dc00d

openshift-ingress

gateway_labeler_controller

e2e-inf-a-301be8

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-301be8

openshift-ingress

deployment-controller

e2e-inf-a-301be8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-301be8-openshift-default-bf4f59db5 from 0 to 1

opendatahub

kubelet

test-curl-7360-90a43e

Killing

Stopping container test-curl-7360-90a43e

openshift-ingress

replicaset-controller

e2e-inf-a-301be8-openshift-default-bf4f59db5

SuccessfulCreate

Created pod: e2e-inf-a-301be8-openshift-default-bf4f59db5-k5dvk

openshift-ingress

multus

e2e-inf-a-301be8-openshift-default-bf4f59db5-k5dvk

AddedInterface

Add eth0 [10.134.0.77/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-301be8-74b7d9d5ff

FailedCreate

Error creating: pods "payload-processing-e2e-inf-a-301be8-74b7d9d5ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-301be8: serviceaccount "payload-processing-e2e-inf-a-301be8" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-301be8-74b7d9d5ff

SuccessfulCreate

Created pod: payload-processing-e2e-inf-a-301be8-74b7d9d5ff-5xjll

openshift-ingress

deployment-controller

payload-processing-e2e-inf-a-301be8

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-a-301be8-74b7d9d5ff from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-a-301be8

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-a-301be8-9687dd9ff from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-301be8-9687dd9ff

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-a-301be8-9687dd9ff-n9tq4
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-301be8-9687dd9ff

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-a-301be8-9687dd9ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-301be8: serviceaccount "payload-processing-e2e-inf-a-301be8" not found

openshift-ingress

kubelet

payload-processing-e2e-inf-a-301be8-74b7d9d5ff-5xjll

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-inf-a-301be8-74b7d9d5ff-5xjll

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-301be8-9687dd9ff-n9tq4

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-301be8-9687dd9ff-n9tq4

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-301be8-9687dd9ff-n9tq4

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-inf-a-301be8-9687dd9ff-n9tq4

AddedInterface

Add eth0 [10.134.0.80/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-inf-a-301be8-74b7d9d5ff-5xjll

AddedInterface

Add eth0 [10.134.0.79/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-inf-a-301be8-74b7d9d5ff-5xjll

Created

Container created

openshift-ingress

replicaset-controller

e2e-worker-w0-2e80fb-openshift-default-5585568575

SuccessfulCreate

Created pod: e2e-worker-w0-2e80fb-openshift-default-5585568575-bcqxw

openshift-ingress

deployment-controller

e2e-worker-w0-2e80fb-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-2e80fb-openshift-default-5585568575 from 2 to 4

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-2e80fb-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w0-2e80fb-openshift-default-5585568575

SuccessfulCreate

Created pod: e2e-worker-w0-2e80fb-openshift-default-5585568575-bssvc

openshift-ingress

multus

e2e-worker-w0-2e80fb-openshift-default-5585568575-bssvc

AddedInterface

Add eth0 [10.133.0.54/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w0-2e80fb-openshift-default-5585568575-bcqxw

AddedInterface

Add eth0 [10.132.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-bcqxw

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-bssvc

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-bssvc

Created

Container created

openshift-ingress

kubelet

e2e-praxis-a-000ffb-openshift-default-74665c677-9xtr6

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-bcqxw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

payload-processing-e2e-praxis-a-000ffb-69745656c9-lmgkp

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-bssvc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-bcqxw

Started

Container started

openshift-ingress

kubelet

e2e-praxis-a-000ffb-openshift-default-74665c677-lvtlw

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-praxis-a-000ffb-openshift-default-74665c677-tx8bf

Killing

Stopping container istio-proxy
(x6)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-bcqxw

Unhealthy

Startup probe failed: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-bcqxw

ProbeError

Startup probe error: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-bssvc

ProbeError

Startup probe error: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-bssvc

Unhealthy

Startup probe failed: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-a-000ffb-76b97879b7-lbxd8

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-b-d2e277-6bdc676c5f-g2r9n

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-praxis-b-d2e277-579fb69f6c-8plsv

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-58jd2

Created

Container created

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-58jd2

Started

Container started

openshift-ingress

deployment-controller

e2e-shared-a-w4-bde944-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w4-bde944-openshift-default-75487f99f9 from 1 to 2

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w4-bde944-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-shared-a-w4-bde944-openshift-default-75487f99f9

SuccessfulCreate

Created pod: e2e-shared-a-w4-bde944-openshift-default-75487f99f9-58jd2

openshift-ingress

multus

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-58jd2

AddedInterface

Add eth0 [10.133.0.55/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-a110ef-openshift-default

SuccessfulRescale

New size: 7; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w3-a110ef-openshift-default-754f5584f9

SuccessfulCreate

Created pod: e2e-worker-w3-a110ef-openshift-default-754f5584f9-wmdl4

openshift-ingress

replicaset-controller

e2e-worker-w3-a110ef-openshift-default-754f5584f9

SuccessfulCreate

Created pod: e2e-worker-w3-a110ef-openshift-default-754f5584f9-42xxk

openshift-ingress

replicaset-controller

e2e-worker-w3-a110ef-openshift-default-754f5584f9

SuccessfulCreate

Created pod: e2e-worker-w3-a110ef-openshift-default-754f5584f9-p5zd5

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-58jd2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

e2e-worker-w3-a110ef-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-a110ef-openshift-default-754f5584f9 from 4 to 7

openshift-ingress

multus

e2e-inf-b-8743a5-openshift-default-b59dcb9c-27nd6

AddedInterface

Add eth0 [10.134.0.83/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-42xxk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p5zd5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p5zd5

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p5zd5

Started

Container started

openshift-ingress

gateway_labeler_controller

e2e-inf-b-8743a5

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-8743a5

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-42xxk

Started

Container started

openshift-ingress

multus

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p5zd5

AddedInterface

Add eth0 [10.134.0.82/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-42xxk

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-wmdl4

Started

Container started

openshift-ingress

multus

e2e-worker-w3-a110ef-openshift-default-754f5584f9-42xxk

AddedInterface

Add eth0 [10.132.0.63/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w3-a110ef-openshift-default-754f5584f9-wmdl4

AddedInterface

Add eth0 [10.132.0.62/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-inf-b-8743a5-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-b-8743a5-openshift-default-b59dcb9c from 0 to 1

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-wmdl4

Created

Container created

openshift-ingress

replicaset-controller

e2e-inf-b-8743a5-openshift-default-b59dcb9c

SuccessfulCreate

Created pod: e2e-inf-b-8743a5-openshift-default-b59dcb9c-27nd6

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-wmdl4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-k5dvk

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-k5dvk

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-k5dvk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-8743a5-6f4c48bc79

SuccessfulCreate

Created pod: payload-processing-e2e-inf-b-8743a5-6f4c48bc79-lb58d

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-8743a5-679877bf47

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-b-8743a5-679877bf47-jj7dr
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-8743a5-6f4c48bc79

FailedCreate

Error creating: pods "payload-processing-e2e-inf-b-8743a5-6f4c48bc79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-8743a5: serviceaccount "payload-processing-e2e-inf-b-8743a5" not found

openshift-ingress

deployment-controller

payload-processing-e2e-inf-b-8743a5

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-b-8743a5-6f4c48bc79 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-b-8743a5

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-b-8743a5-679877bf47 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-8743a5-679877bf47

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-b-8743a5-679877bf47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-8743a5: serviceaccount "payload-processing-e2e-inf-b-8743a5" not found

openshift-ingress

multus

payload-processing-e2e-inf-b-8743a5-6f4c48bc79-lb58d

AddedInterface

Add eth0 [10.134.0.84/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-58jd2

Unhealthy

Startup probe failed: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-301be8-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-8743a5-679877bf47-jj7dr

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-8743a5-679877bf47-jj7dr

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-inf-b-8743a5-6f4c48bc79-lb58d

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-inf-b-8743a5-679877bf47-jj7dr

AddedInterface

Add eth0 [10.132.0.64/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-58jd2

ProbeError

Startup probe error: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused body:

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-301be8-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

payload-processing-e2e-inf-b-8743a5-6f4c48bc79-lb58d

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-8743a5-679877bf47-jj7dr

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-inf-b-8743a5-6f4c48bc79-lb58d

Created

Container created

openshift-ingress

kubelet

e2e-praxis-b-d2e277-openshift-default-94fd97fdf-t7vdm

Killing

Stopping container istio-proxy
(x7)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-42xxk

ProbeError

Startup probe error: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-wmdl4

Unhealthy

Startup probe failed: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-wmdl4

ProbeError

Startup probe error: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p5zd5

ProbeError

Startup probe error: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-42xxk

Unhealthy

Startup probe failed: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p5zd5

Unhealthy

Startup probe failed: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-extmod-a-629129-openshift-default-dc77fd7d9

SuccessfulCreate

Created pod: e2e-extmod-a-629129-openshift-default-dc77fd7d9-q7ntb

openshift-ingress

deployment-controller

e2e-extmod-a-629129-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-629129-openshift-default-dc77fd7d9 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-629129

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-629129
(x5)

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-k5dvk

ProbeError

Startup probe error: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-k5dvk

Unhealthy

Startup probe failed: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused

openshift-ingress

multus

e2e-extmod-a-629129-openshift-default-dc77fd7d9-q7ntb

AddedInterface

Add eth0 [10.133.0.56/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-praxis-b-d2e277-openshift-default-94fd97fdf-t7vdm

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-praxis-b-d2e277-openshift-default-94fd97fdf-t7vdm

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

opendatahub

kubelet

test-curl-7366-ae9450

Created

Container created

opendatahub

kubelet

test-curl-7366-ae9450

Started

Container started

opendatahub

multus

test-curl-7366-ae9450

AddedInterface

Add eth0 [10.134.0.86/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7366-ae9450

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-629129-9d8b9d665

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-629129-9d8b9d665-qclfv

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-629129

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-629129-ddf565c67 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-629129-9d8b9d665

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-629129-9d8b9d665-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-629129: serviceaccount "payload-processing-e2e-extmod-a-629129" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-629129-ddf565c67

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-629129-ddf565c67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-629129: serviceaccount "payload-processing-e2e-extmod-a-629129" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-629129-ddf565c67

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-629129-ddf565c67-76b79

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-629129

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-629129-9d8b9d665 from 0 to 1

opendatahub

kubelet

test-curl-7366-ae9450

Killing

Stopping container test-curl-7366-ae9450

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-629129-ddf565c67-76b79

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-629129-ddf565c67-76b79

AddedInterface

Add eth0 [10.134.0.87/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-629129-ddf565c67-76b79

Created

Container created

openshift-ingress

deployment-controller

e2e-worker-w3-a110ef-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-a110ef-openshift-default-754f5584f9 from 7 to 10

openshift-ingress

replicaset-controller

e2e-worker-w3-a110ef-openshift-default-754f5584f9

SuccessfulCreate

(combined from similar events): Created pod: e2e-worker-w3-a110ef-openshift-default-754f5584f9-p2lgl

openshift-ingress

replicaset-controller

e2e-worker-w3-a110ef-openshift-default-754f5584f9

SuccessfulCreate

Created pod: e2e-worker-w3-a110ef-openshift-default-754f5584f9-dg56j

openshift-ingress

replicaset-controller

e2e-worker-w3-a110ef-openshift-default-754f5584f9

SuccessfulCreate

Created pod: e2e-worker-w3-a110ef-openshift-default-754f5584f9-qkk8t

openshift-ingress

multus

payload-processing-e2e-extmod-a-629129-9d8b9d665-qclfv

AddedInterface

Add eth0 [10.134.0.88/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-629129-9d8b9d665-qclfv

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-629129-9d8b9d665-qclfv

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-629129-9d8b9d665-qclfv

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-629129-ddf565c67-76b79

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p2lgl

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-qkk8t

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-qkk8t

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-dg56j

Created

Container created

openshift-ingress

multus

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p2lgl

AddedInterface

Add eth0 [10.133.0.59/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p2lgl

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p2lgl

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w3-a110ef-openshift-default-754f5584f9-dg56j

AddedInterface

Add eth0 [10.134.0.89/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-qkk8t

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-dg56j

Started

Container started

openshift-ingress

multus

e2e-worker-w3-a110ef-openshift-default-754f5584f9-qkk8t

AddedInterface

Add eth0 [10.133.0.58/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-dg56j

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x7)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-dg56j

Unhealthy

Startup probe failed: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-dg56j

ProbeError

Startup probe error: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p2lgl

ProbeError

Startup probe error: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-qkk8t

Unhealthy

Startup probe failed: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused

openshift-ingress

multus

istiod-openshift-gateway-7ddd9cfdc8-fmbxp

AddedInterface

Add eth0 [10.133.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-fmbxp

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8"
(x8)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p2lgl

Unhealthy

Startup probe failed: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-qkk8t

ProbeError

Startup probe error: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulCreate

Created pod: istiod-openshift-gateway-7ddd9cfdc8-fmbxp

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-dg56j

Unhealthy

Startup probe failed: Get "http://10.134.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-dg56j

ProbeError

Startup probe error: Get "http://10.134.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-qkk8t

Unhealthy

Startup probe failed: Get "http://10.133.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-qkk8t

ProbeError

Startup probe error: Get "http://10.133.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p2lgl

ProbeError

Startup probe error: Get "http://10.133.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p2lgl

Unhealthy

Startup probe failed: Get "http://10.133.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-b-8743a5-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

e2e-inf-b-8743a5-openshift-default-b59dcb9c-27nd6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-inf-b-8743a5-openshift-default-b59dcb9c-27nd6

Started

Container started
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-b-8743a5-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

e2e-inf-b-8743a5-openshift-default-b59dcb9c-27nd6

Created

Container created

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w4-9e7156-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-fmbxp

Started

Container started

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-2cjnv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879

SuccessfulCreate

Created pod: e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-2cjnv

openshift-ingress

deployment-controller

e2e-shared-b-w4-9e7156-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w4-9e7156-openshift-default-6bc7994879 from 1 to 2

openshift-ingress

multus

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-2cjnv

AddedInterface

Add eth0 [10.132.0.69/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-2cjnv

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-fmbxp

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 9.39s (9.39s including waiting). Image size: 185476477 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-fmbxp

Created

Container created

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-2cjnv

Started

Container started

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-fmbxp

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-fmbxp

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-301be8-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-301be8-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x6)

openshift-ingress

kubelet

e2e-inf-b-8743a5-openshift-default-b59dcb9c-27nd6

ProbeError

Startup probe error: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-inf-b-8743a5-openshift-default-b59dcb9c-27nd6

Unhealthy

Startup probe failed: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-q7ntb

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-q7ntb

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-q7ntb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-2cjnv

Unhealthy

Startup probe failed: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-2cjnv

ProbeError

Startup probe error: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused body:

openshift-ingress

deployment-controller

e2e-extmod-b-204fba-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-204fba-openshift-default-8dd6555f7 from 0 to 1

openshift-ingress

replicaset-controller

e2e-extmod-b-204fba-openshift-default-8dd6555f7

SuccessfulCreate

Created pod: e2e-extmod-b-204fba-openshift-default-8dd6555f7-xl5jm

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-204fba

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-204fba

openshift-ingress

multus

e2e-extmod-b-204fba-openshift-default-8dd6555f7-xl5jm

AddedInterface

Add eth0 [10.132.0.70/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-2cjnv

Unhealthy

Startup probe failed: Get "http://10.132.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-2cjnv

ProbeError

Startup probe error: Get "http://10.132.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

opendatahub

kubelet

test-curl-7366-e69be8

Started

Container started

opendatahub

kubelet

test-curl-7366-e69be8

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p2lgl

Unhealthy

Readiness probe failed: Get "http://10.133.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p2lgl

ProbeError

Readiness probe error: Get "http://10.133.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-qkk8t

ProbeError

Readiness probe error: Get "http://10.133.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-qkk8t

Unhealthy

Readiness probe failed: Get "http://10.133.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

opendatahub

kubelet

test-curl-7366-e69be8

Created

Container created

opendatahub

multus

test-curl-7366-e69be8

AddedInterface

Add eth0 [10.134.0.95/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w0-2e80fb-openshift-default-5585568575

SuccessfulCreate

Created pod: e2e-worker-w0-2e80fb-openshift-default-5585568575-6krb6

openshift-ingress

replicaset-controller

e2e-worker-w0-2e80fb-openshift-default-5585568575

SuccessfulCreate

Created pod: e2e-worker-w0-2e80fb-openshift-default-5585568575-sk5vq

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-2e80fb-openshift-default

SuccessfulRescale

New size: 8; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w0-2e80fb-openshift-default-5585568575

SuccessfulCreate

Created pod: e2e-worker-w0-2e80fb-openshift-default-5585568575-l2jsl

openshift-ingress

replicaset-controller

e2e-worker-w0-2e80fb-openshift-default-5585568575

SuccessfulCreate

Created pod: e2e-worker-w0-2e80fb-openshift-default-5585568575-rmc6x

openshift-ingress

deployment-controller

e2e-worker-w0-2e80fb-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-2e80fb-openshift-default-5585568575 from 4 to 8

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-204fba

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-204fba-558699b45f from 0 to 1
(x6)

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-q7ntb

ProbeError

Startup probe error: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused body:

openshift-ingress

multus

e2e-worker-w0-2e80fb-openshift-default-5585568575-rmc6x

AddedInterface

Add eth0 [10.132.0.71/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-6krb6

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-6krb6

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-6krb6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w0-2e80fb-openshift-default-5585568575-6krb6

AddedInterface

Add eth0 [10.134.0.97/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-rmc6x

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7366-e69be8

Killing

Stopping container test-curl-7366-e69be8

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-l2jsl

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-l2jsl

Created

Container created

openshift-ingress

multus

e2e-worker-w0-2e80fb-openshift-default-5585568575-sk5vq

AddedInterface

Add eth0 [10.134.0.98/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-sk5vq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-sk5vq

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-l2jsl

Started

Container started
(x4)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-204fba-558699b45f

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-204fba-558699b45f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-204fba: serviceaccount "payload-processing-e2e-extmod-b-204fba" not found

opendatahub

kubelet

test-curl-7366-66627a

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-204fba-6786ccc979

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-204fba-6786ccc979-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-204fba: serviceaccount "payload-processing-e2e-extmod-b-204fba" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-204fba-6786ccc979

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-204fba-6786ccc979-d7ct7

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-204fba

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-204fba-6786ccc979 from 0 to 1

opendatahub

kubelet

test-curl-7366-66627a

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-rmc6x

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-rmc6x

Started

Container started

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-204fba-558699b45f

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-204fba-558699b45f-np9qr

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-b-8743a5-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-sk5vq

Started

Container started

opendatahub

kubelet

test-curl-7366-66627a

Created

Container created

opendatahub

multus

test-curl-7366-66627a

AddedInterface

Add eth0 [10.134.0.96/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w0-2e80fb-openshift-default-5585568575-l2jsl

AddedInterface

Add eth0 [10.132.0.72/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-q7ntb

Unhealthy

Startup probe failed: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-b-8743a5-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-204fba-6786ccc979-d7ct7

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-204fba-558699b45f-np9qr

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-204fba-6786ccc979-d7ct7

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-204fba-6786ccc979-d7ct7

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-204fba-558699b45f-np9qr

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-204fba-6786ccc979-d7ct7

AddedInterface

Add eth0 [10.132.0.74/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-204fba-558699b45f-np9qr

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-extmod-b-204fba-558699b45f-np9qr

AddedInterface

Add eth0 [10.132.0.73/23] from ovn-kubernetes

opendatahub

multus

test-curl-7366-f11c97

AddedInterface

Add eth0 [10.134.0.101/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7366-f11c97

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7366-f11c97

Created

Container created

opendatahub

kubelet

test-curl-7366-f11c97

Started

Container started

opendatahub

kubelet

test-curl-7366-66627a

Killing

Stopping container test-curl-7366-66627a

opendatahub

multus

test-curl-7366-0c0970

AddedInterface

Add eth0 [10.134.0.102/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7366-f11c97

Killing

Stopping container test-curl-7366-f11c97

opendatahub

kubelet

test-curl-7366-0c0970

Started

Container started

opendatahub

kubelet

test-curl-7366-0c0970

Created

Container created

opendatahub

kubelet

test-curl-7366-0c0970

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-inf-a-301be8-openshift-default-bf4f59db5

SuccessfulCreate

Created pod: e2e-inf-a-301be8-openshift-default-bf4f59db5-66ssd

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-301be8-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-inf-a-301be8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-301be8-openshift-default-bf4f59db5 from 1 to 2

openshift-ingress

multus

e2e-inf-a-301be8-openshift-default-bf4f59db5-66ssd

AddedInterface

Add eth0 [10.132.0.75/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7366-0c0970

Killing

Stopping container test-curl-7366-0c0970

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-66ssd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-66ssd

Created

Container created

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-66ssd

Started

Container started
(x7)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-rmc6x

ProbeError

Startup probe error: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-rmc6x

Unhealthy

Startup probe failed: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-sk5vq

ProbeError

Startup probe error: Get "http://10.134.0.98:15021/healthz/ready": dial tcp 10.134.0.98:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-6krb6

ProbeError

Startup probe error: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-l2jsl

Unhealthy

Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-6krb6

Unhealthy

Startup probe failed: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-sk5vq

Unhealthy

Startup probe failed: Get "http://10.134.0.98:15021/healthz/ready": dial tcp 10.134.0.98:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-l2jsl

ProbeError

Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-6krb6

Unhealthy

Startup probe failed: Get "http://10.134.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-6krb6

ProbeError

Startup probe error: Get "http://10.134.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-sk5vq

Unhealthy

Startup probe failed: Get "http://10.134.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-6hb6w

Unhealthy

Readiness probe failed: Get "https://10.134.0.48:8000/ready": net/http: request canceled (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-sk5vq

ProbeError

Startup probe error: Get "http://10.134.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-l2jsl

ProbeError

Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-l2jsl

Unhealthy

Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-rmc6x

Unhealthy

Startup probe failed: Get "http://10.132.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-rmc6x

ProbeError

Startup probe error: Get "http://10.132.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

deployment-controller

e2e-worker-w0-2e80fb-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-2e80fb-openshift-default-5585568575 from 8 to 10

openshift-ingress

replicaset-controller

e2e-worker-w0-2e80fb-openshift-default-5585568575

SuccessfulCreate

(combined from similar events): Created pod: e2e-worker-w0-2e80fb-openshift-default-5585568575-rg5f7

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-2e80fb-openshift-default

SuccessfulRescale

New size: 10; reason:

openshift-ingress

replicaset-controller

e2e-worker-w0-2e80fb-openshift-default-5585568575

SuccessfulCreate

Created pod: e2e-worker-w0-2e80fb-openshift-default-5585568575-k6wmf

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-rg5f7

Created

Container created

openshift-ingress

deployment-controller

e2e-inf-b-8743a5-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-b-8743a5-openshift-default-b59dcb9c from 1 to 2

openshift-ingress

kubelet

e2e-inf-b-8743a5-openshift-default-b59dcb9c-sqnjm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-k6wmf

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-k6wmf

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-rg5f7

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w0-2e80fb-openshift-default-5585568575-rg5f7

AddedInterface

Add eth0 [10.132.0.77/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-inf-b-8743a5-openshift-default-b59dcb9c

SuccessfulCreate

Created pod: e2e-inf-b-8743a5-openshift-default-b59dcb9c-sqnjm

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-b-8743a5-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-k6wmf

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-rg5f7

Started

Container started

openshift-ingress

multus

e2e-inf-b-8743a5-openshift-default-b59dcb9c-sqnjm

AddedInterface

Add eth0 [10.132.0.78/23] from ovn-kubernetes
(x7)

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-66ssd

Unhealthy

Startup probe failed: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-66ssd

ProbeError

Startup probe error: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused body:

openshift-ingress

multus

e2e-worker-w0-2e80fb-openshift-default-5585568575-k6wmf

AddedInterface

Add eth0 [10.133.0.62/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-inf-b-8743a5-openshift-default-b59dcb9c-sqnjm

Started

Container started

openshift-ingress

kubelet

e2e-inf-b-8743a5-openshift-default-b59dcb9c-sqnjm

Created

Container created
(x6)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-k6wmf

Unhealthy

Startup probe failed: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-k6wmf

ProbeError

Startup probe error: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-k6wmf

ProbeError

Startup probe error: Get "http://10.133.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-629129-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x8)

openshift-ingress

kubelet

e2e-inf-b-8743a5-openshift-default-b59dcb9c-sqnjm

ProbeError

Startup probe error: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

e2e-inf-b-8743a5-openshift-default-b59dcb9c-sqnjm

Unhealthy

Startup probe failed: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-629129-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-k6wmf

Unhealthy

Startup probe failed: Get "http://10.133.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x10)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-rg5f7

ProbeError

Startup probe error: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused body:
(x10)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-rg5f7

Unhealthy

Startup probe failed: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-rg5f7

ProbeError

Startup probe error: Get "http://10.132.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-rg5f7

Unhealthy

Startup probe failed: Get "http://10.132.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-xl5jm

Unhealthy

Readiness probe failed: Get "http://10.132.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-xl5jm

ProbeError

Readiness probe error: Get "http://10.132.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-xl5jm

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-xl5jm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-xl5jm

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-xl5jm

Unhealthy

Startup probe failed: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-xl5jm

ProbeError

Startup probe error: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused body:

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-204fba-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-204fba-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

multus

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-6mdlm

AddedInterface

Add eth0 [10.134.0.111/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-shared-b-w4-9e7156-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w4-9e7156-openshift-default-6bc7994879 from 2 to 4

openshift-ingress

replicaset-controller

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879

SuccessfulCreate

Created pod: e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-jsxcs

openshift-ingress

replicaset-controller

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879

SuccessfulCreate

Created pod: e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-6mdlm

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-jsxcs

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-6mdlm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w4-9e7156-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-jsxcs

AddedInterface

Add eth0 [10.133.0.63/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-6mdlm

Started

Container started

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-6mdlm

Created

Container created

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-jsxcs

Started

Container started

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-jsxcs

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-6mdlm

Unhealthy

Startup probe failed: Get "http://10.134.0.111:15021/healthz/ready": dial tcp 10.134.0.111:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-6mdlm

ProbeError

Startup probe error: Get "http://10.134.0.111:15021/healthz/ready": dial tcp 10.134.0.111:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-jsxcs

ProbeError

Startup probe error: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-jsxcs

Unhealthy

Startup probe failed: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-rg5f7

Unhealthy

Readiness probe failed: Get "http://10.132.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-rg5f7

ProbeError

Readiness probe error: Get "http://10.132.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

deployment-controller

e2e-inf-a-301be8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-301be8-openshift-default-bf4f59db5 from 2 to 3

openshift-ingress

replicaset-controller

e2e-inf-a-301be8-openshift-default-bf4f59db5

SuccessfulCreate

Created pod: e2e-inf-a-301be8-openshift-default-bf4f59db5-cxz8s

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-301be8-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-cxz8s

Created

Container created

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-cxz8s

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-cxz8s

Started

Container started

openshift-ingress

multus

e2e-inf-a-301be8-openshift-default-bf4f59db5-cxz8s

AddedInterface

Add eth0 [10.133.0.64/23] from ovn-kubernetes
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-204fba-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-204fba-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x6)

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-cxz8s

Unhealthy

Startup probe failed: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-cxz8s

ProbeError

Startup probe error: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

e2e-inf-b-8743a5-openshift-default-b59dcb9c

SuccessfulCreate

Created pod: e2e-inf-b-8743a5-openshift-default-b59dcb9c-mv6t8

openshift-ingress

deployment-controller

e2e-inf-b-8743a5-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-b-8743a5-openshift-default-b59dcb9c from 2 to 3

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-b-8743a5-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-inf-b-8743a5-openshift-default-b59dcb9c-mv6t8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-inf-b-8743a5-openshift-default-b59dcb9c-mv6t8

AddedInterface

Add eth0 [10.133.0.65/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-inf-b-8743a5-openshift-default-b59dcb9c-mv6t8

Created

Container created

openshift-ingress

kubelet

e2e-inf-b-8743a5-openshift-default-b59dcb9c-mv6t8

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-inf-b-8743a5-openshift-default-b59dcb9c-mv6t8

ProbeError

Startup probe error: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-inf-b-8743a5-openshift-default-b59dcb9c-mv6t8

Unhealthy

Startup probe failed: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-6hb6w

Unhealthy

Readiness probe failed: Get "https://10.134.0.48:8000/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-204fba-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-extmod-b-204fba-openshift-default-8dd6555f7

SuccessfulCreate

Created pod: e2e-extmod-b-204fba-openshift-default-8dd6555f7-qvxr9

openshift-ingress

deployment-controller

e2e-extmod-b-204fba-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-204fba-openshift-default-8dd6555f7 from 1 to 2

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-dg56j

Unhealthy

Readiness probe failed: Get "http://10.134.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-dg56j

ProbeError

Readiness probe error: Get "http://10.134.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-qvxr9

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-qvxr9

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-sk5vq

ProbeError

Readiness probe error: Get "http://10.134.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-sk5vq

Unhealthy

Readiness probe failed: Get "http://10.134.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

multus

e2e-extmod-b-204fba-openshift-default-8dd6555f7-qvxr9

AddedInterface

Add eth0 [10.134.0.121/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-qvxr9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w4-bde944-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-shared-a-w4-bde944-openshift-default-75487f99f9

SuccessfulCreate

Created pod: e2e-shared-a-w4-bde944-openshift-default-75487f99f9-4cf62

openshift-ingress

deployment-controller

e2e-shared-a-w4-bde944-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w4-bde944-openshift-default-75487f99f9 from 2 to 3

openshift-ingress

multus

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-4cf62

AddedInterface

Add eth0 [10.132.0.85/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-4cf62

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-4cf62

Started

Container started

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-4cf62

Created

Container created
(x7)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-qvxr9

Unhealthy

Startup probe failed: Get "http://10.134.0.121:15021/healthz/ready": dial tcp 10.134.0.121:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-qvxr9

ProbeError

Startup probe error: Get "http://10.134.0.121:15021/healthz/ready": dial tcp 10.134.0.121:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-4cf62

ProbeError

Startup probe error: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-4cf62

Unhealthy

Startup probe failed: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-301be8-9687dd9ff-n9tq4

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-301be8-74b7d9d5ff-5xjll

Killing

Stopping container payload-processing

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-301be8-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-inf-a-301be8-openshift-default-bf4f59db5

SuccessfulCreate

Created pod: e2e-inf-a-301be8-openshift-default-bf4f59db5-rmcf7

openshift-ingress

deployment-controller

e2e-inf-a-301be8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-301be8-openshift-default-bf4f59db5 from 3 to 4

openshift-ingress

multus

e2e-inf-a-301be8-openshift-default-bf4f59db5-rmcf7

AddedInterface

Add eth0 [10.134.0.127/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-rmcf7

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-rgfsg

Unhealthy

Readiness probe failed: Get "http://10.133.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-rmcf7

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-rgfsg

ProbeError

Readiness probe error: Get "http://10.133.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-66ssd

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-cxz8s

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p5zd5

Unhealthy

Readiness probe failed: Get "http://10.134.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-rmcf7

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p5zd5

ProbeError

Readiness probe error: Get "http://10.134.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-k6wmf

Unhealthy

Readiness probe failed: Get "http://10.133.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-k6wmf

ProbeError

Readiness probe error: Get "http://10.133.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-wmdl4

ProbeError

Readiness probe error: Get "http://10.132.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-k5dvk

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-rmcf7

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (e2e-inf-a-301be8-openshift-default-dockercfg-fxzjg); attempting to pull the image may not succeed.
(x2)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-wmdl4

Unhealthy

Readiness probe failed: Get "http://10.132.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-rmcf7

Unhealthy

Startup probe failed: Get "http://10.134.0.127:15021/healthz/ready": dial tcp 10.134.0.127:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-rmcf7

ProbeError

Startup probe error: Get "http://10.134.0.127:15021/healthz/ready": dial tcp 10.134.0.127:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-rmcf7

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-cxz8s

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-h4446

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-75fd9f5d47 from 10 to 8

openshift-ingress

kubelet

e2e-inf-a-301be8-openshift-default-bf4f59db5-cxz8s

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-f8wtz

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-f8wtz

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 8; reason: All metrics below target

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-h4446

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-8743a5-679877bf47-jj7dr

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-8743a5-6f4c48bc79-lb58d

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

payload-processing-e2e-inf-a-301be8-74b7d9d5ff-5xjll

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-301be8-9687dd9ff-n9tq4

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-2q2rq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-extmod-b-204fba-openshift-default-8dd6555f7-2q2rq

AddedInterface

Add eth0 [10.133.0.66/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-5msfb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

e2e-extmod-b-204fba-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-204fba-openshift-default-8dd6555f7 from 2 to 4

openshift-ingress

replicaset-controller

e2e-extmod-b-204fba-openshift-default-8dd6555f7

SuccessfulCreate

Created pod: e2e-extmod-b-204fba-openshift-default-8dd6555f7-5msfb

openshift-ingress

multus

e2e-extmod-b-204fba-openshift-default-8dd6555f7-5msfb

AddedInterface

Add eth0 [10.134.0.131/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-extmod-b-204fba-openshift-default-8dd6555f7

SuccessfulCreate

Created pod: e2e-extmod-b-204fba-openshift-default-8dd6555f7-2q2rq

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-204fba-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-5msfb

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-2q2rq

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-2q2rq

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-5msfb

Started

Container started

openshift-ingress

kubelet

e2e-inf-b-8743a5-openshift-default-b59dcb9c-mv6t8

Killing

Stopping container istio-proxy
(x6)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-5msfb

ProbeError

Startup probe error: Get "http://10.134.0.131:15021/healthz/ready": dial tcp 10.134.0.131:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-inf-b-8743a5-openshift-default-b59dcb9c-sqnjm

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-inf-b-8743a5-openshift-default-b59dcb9c-27nd6

Killing

Stopping container istio-proxy
(x6)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-5msfb

Unhealthy

Startup probe failed: Get "http://10.134.0.131:15021/healthz/ready": dial tcp 10.134.0.131:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-2q2rq

Unhealthy

Startup probe failed: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

payload-processing-e2e-inf-b-8743a5-6f4c48bc79-lb58d

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x7)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-2q2rq

ProbeError

Startup probe error: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-8743a5-679877bf47-jj7dr

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

gateway_labeler_controller

e2e-ait-712810ea

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-712810ea

openshift-ingress

deployment-controller

e2e-ait-712810ea-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-712810ea-openshift-default-76c9fb4567 from 0 to 1

openshift-ingress

replicaset-controller

e2e-ait-712810ea-openshift-default-76c9fb4567

SuccessfulCreate

Created pod: e2e-ait-712810ea-openshift-default-76c9fb4567-ncpx6

openshift-ingress

multus

e2e-ait-712810ea-openshift-default-76c9fb4567-ncpx6

AddedInterface

Add eth0 [10.134.0.135/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-8743a5-679877bf47-jj7dr

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.132.0.64:50052" within 1s: context deadline exceeded

openshift-ingress

deployment-controller

payload-processing-e2e-ait-712810ea

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-712810ea-8449cd864f from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-712810ea-8449cd864f

FailedCreate

Error creating: pods "payload-processing-e2e-ait-712810ea-8449cd864f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-712810ea: serviceaccount "payload-processing-e2e-ait-712810ea" not found
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-712810ea-77c968f8c9

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-712810ea-77c968f8c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-712810ea: serviceaccount "payload-processing-e2e-ait-712810ea" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-712810ea-77c968f8c9

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-712810ea-77c968f8c9-4clwv

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-712810ea

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-712810ea-77c968f8c9 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-712810ea-8449cd864f

SuccessfulCreate

Created pod: payload-processing-e2e-ait-712810ea-8449cd864f-l84rs

openshift-ingress

multus

payload-processing-e2e-ait-712810ea-8449cd864f-l84rs

AddedInterface

Add eth0 [10.134.0.136/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-712810ea-8449cd864f-l84rs

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-ait-712810ea-77c968f8c9-4clwv

AddedInterface

Add eth0 [10.134.0.137/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-712810ea-77c968f8c9-4clwv

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ait-712810ea-8449cd864f-l84rs

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ait-712810ea-8449cd864f-l84rs

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-712810ea-77c968f8c9-4clwv

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-712810ea-77c968f8c9-4clwv

Created

Container created

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-712810ea-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-712810ea-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-6hb6w

Unhealthy

Liveness probe failed: Get "https://10.134.0.48:8000/health": context deadline exceeded
(x3)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-s2h9r

ProbeError

Readiness probe error: Get "http://10.133.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x3)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-42xxk

Unhealthy

Readiness probe failed: Get "http://10.132.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x3)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-42xxk

ProbeError

Readiness probe error: Get "http://10.132.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x3)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-s2h9r

Unhealthy

Readiness probe failed: Get "http://10.133.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-r2twl

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-extmod-b-204fba-openshift-default-8dd6555f7-r2twl

AddedInterface

Add eth0 [10.132.0.91/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-extmod-b-204fba-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-204fba-openshift-default-8dd6555f7 from 4 to 6

openshift-ingress

multus

e2e-extmod-b-204fba-openshift-default-8dd6555f7-9q2xr

AddedInterface

Add eth0 [10.133.0.67/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-9q2xr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-extmod-b-204fba-openshift-default-8dd6555f7

SuccessfulCreate

Created pod: e2e-extmod-b-204fba-openshift-default-8dd6555f7-9q2xr

openshift-ingress

replicaset-controller

e2e-extmod-b-204fba-openshift-default-8dd6555f7

SuccessfulCreate

Created pod: e2e-extmod-b-204fba-openshift-default-8dd6555f7-r2twl

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-204fba-openshift-default

SuccessfulRescale

New size: 6; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-r2twl

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-9q2xr

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-r2twl

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-9q2xr

Created

Container created

openshift-ingress

deployment-controller

e2e-shared-a-w4-bde944-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w4-bde944-openshift-default-75487f99f9 from 3 to 5

openshift-ingress

replicaset-controller

e2e-shared-a-w4-bde944-openshift-default-75487f99f9

SuccessfulCreate

Created pod: e2e-shared-a-w4-bde944-openshift-default-75487f99f9-6f8mt

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w4-bde944-openshift-default

SuccessfulRescale

New size: 5; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-shared-a-w4-bde944-openshift-default-75487f99f9

SuccessfulCreate

Created pod: e2e-shared-a-w4-bde944-openshift-default-75487f99f9-zx7kj

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-6f8mt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-zx7kj

Started

Container started

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-6f8mt

Started

Container started

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-zx7kj

Created

Container created

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-zx7kj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-zx7kj

AddedInterface

Add eth0 [10.134.0.145/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-6f8mt

Created

Container created

openshift-ingress

multus

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-6f8mt

AddedInterface

Add eth0 [10.132.0.92/23] from ovn-kubernetes
(x5)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-9q2xr

Unhealthy

Startup probe failed: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-9q2xr

ProbeError

Startup probe error: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-ait-712810ea-openshift-default-76c9fb4567-ncpx6

Unhealthy

Readiness probe failed: Get "http://10.134.0.135:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-ait-712810ea-openshift-default-76c9fb4567-ncpx6

ProbeError

Readiness probe error: Get "http://10.134.0.135:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-r2twl

ProbeError

Startup probe error: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-r2twl

Unhealthy

Startup probe failed: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-ait-712810ea-openshift-default-76c9fb4567-ncpx6

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-ait-712810ea-openshift-default-76c9fb4567-ncpx6

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ait-712810ea-openshift-default-76c9fb4567-ncpx6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

LLMInferenceServiceNotReady

LLMInferenceService [premium-simulated-simulated-premium] is no longer Ready because of: MainWorkloadReady, WorkloadsReady
(x2)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-2d492

Unhealthy

Readiness probe failed: Get "http://10.132.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-712810ea-77c968f8c9-4clwv

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-712810ea-8449cd864f-l84rs

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-2d492

ProbeError

Readiness probe error: Get "http://10.132.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

deployment-controller

e2e-mt-9cbd331c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-9cbd331c-openshift-default-574b7ff47c from 0 to 1
(x9)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-zx7kj

ProbeError

Startup probe error: Get "http://10.134.0.145:15021/healthz/ready": dial tcp 10.134.0.145:15021: connect: connection refused body:
(x9)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-6f8mt

Unhealthy

Startup probe failed: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused
(x9)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-zx7kj

Unhealthy

Startup probe failed: Get "http://10.134.0.145:15021/healthz/ready": dial tcp 10.134.0.145:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-mt-9cbd331c-openshift-default-574b7ff47c

SuccessfulCreate

Created pod: e2e-mt-9cbd331c-openshift-default-574b7ff47c-8pgt7

openshift-ingress

gateway_labeler_controller

e2e-mt-9cbd331c

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-9cbd331c
(x9)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-6f8mt

ProbeError

Startup probe error: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused body:
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-712810ea-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-712810ea-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

multus

e2e-mt-9cbd331c-openshift-default-574b7ff47c-8pgt7

AddedInterface

Add eth0 [10.134.0.149/23] from ovn-kubernetes
(x9)

openshift-ingress

kubelet

e2e-ait-712810ea-openshift-default-76c9fb4567-ncpx6

Unhealthy

Startup probe failed: Get "http://10.134.0.135:15021/healthz/ready": dial tcp 10.134.0.135:15021: connect: connection refused
(x9)

openshift-ingress

kubelet

e2e-ait-712810ea-openshift-default-76c9fb4567-ncpx6

ProbeError

Startup probe error: Get "http://10.134.0.135:15021/healthz/ready": dial tcp 10.134.0.135:15021: connect: connection refused body:
(x3)

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

LLMInferenceServiceReady

LLMInferenceService [premium-simulated-simulated-premium] is Ready

openshift-ingress

gateway_labeler_controller

e2e-migrate-4c342351-gw

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-4c342351-gw

openshift-ingress

replicaset-controller

e2e-migrate-4c342351-gw-openshift-default-5bd6866449

SuccessfulCreate

Created pod: e2e-migrate-4c342351-gw-openshift-default-5bd6866449-t8rlj

openshift-ingress

deployment-controller

e2e-migrate-4c342351-gw-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-migrate-4c342351-gw-openshift-default-5bd6866449 from 0 to 1

openshift-ingress

kubelet

e2e-ait-712810ea-openshift-default-76c9fb4567-ncpx6

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (e2e-ait-712810ea-openshift-default-dockercfg-zbfsq); attempting to pull the image may not succeed.

openshift-ingress

multus

payload-pre-processing-e2e-mt-9cbd331c-b5c5f659-4jmx6

AddedInterface

Add eth0 [10.134.0.153/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-9cbd331c-789b6c9bcf-qscjb

Created

Container created
(x4)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-9cbd331c-789b6c9bcf

FailedCreate

Error creating: pods "payload-processing-e2e-mt-9cbd331c-789b6c9bcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-9cbd331c: serviceaccount "payload-processing-e2e-mt-9cbd331c" not found

openshift-ingress

kubelet

payload-processing-e2e-mt-9cbd331c-789b6c9bcf-qscjb

Started

Container started

openshift-ingress

kubelet

e2e-migrate-4c342351-gw-openshift-default-5bd6866449-t8rlj

Created

Container created

openshift-ingress

kubelet

e2e-migrate-4c342351-gw-openshift-default-5bd6866449-t8rlj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-migrate-4c342351-gw-openshift-default-5bd6866449-t8rlj

AddedInterface

Add eth0 [10.134.0.151/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-mt-9cbd331c

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-9cbd331c-789b6c9bcf from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-9cbd331c

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-9cbd331c-b5c5f659 from 0 to 1

openshift-ingress

kubelet

e2e-migrate-4c342351-gw-openshift-default-5bd6866449-t8rlj

Started

Container started

openshift-ingress

kubelet

e2e-ait-712810ea-openshift-default-76c9fb4567-ncpx6

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-9cbd331c-789b6c9bcf

SuccessfulCreate

Created pod: payload-processing-e2e-mt-9cbd331c-789b6c9bcf-qscjb

openshift-ingress

kubelet

payload-processing-e2e-mt-9cbd331c-789b6c9bcf-qscjb

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-mt-9cbd331c-789b6c9bcf-qscjb

AddedInterface

Add eth0 [10.134.0.152/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-9cbd331c-b5c5f659-4jmx6

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-9cbd331c-b5c5f659-4jmx6

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-9cbd331c-b5c5f659-4jmx6

Started

Container started

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-9cbd331c-b5c5f659

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-9cbd331c-b5c5f659-4jmx6
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-9cbd331c-b5c5f659

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-9cbd331c-b5c5f659-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-9cbd331c: serviceaccount "payload-processing-e2e-mt-9cbd331c" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-4c342351-7bf9d48b79

SuccessfulCreate

Created pod: payload-pre-processing-e2e-migrate-4c342351-7bf9d48b79-hx859

openshift-ingress

deployment-controller

payload-pre-processing-e2e-migrate-4c342351

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-migrate-4c342351-7bf9d48b79 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-4c342351-7bf9d48b79-hx859

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-migrate-4c342351-7bf9d48b79-hx859

AddedInterface

Add eth0 [10.134.0.154/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-migrate-4c342351-7b78dfb54c-xjx6n

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-4c342351-7b78dfb54c

FailedCreate

Error creating: pods "payload-processing-e2e-migrate-4c342351-7b78dfb54c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-4c342351: serviceaccount "payload-processing-e2e-migrate-4c342351" not found

openshift-ingress

deployment-controller

payload-processing-e2e-migrate-4c342351

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-migrate-4c342351-7b78dfb54c from 0 to 1

openshift-ingress

multus

payload-processing-e2e-migrate-4c342351-7b78dfb54c-xjx6n

AddedInterface

Add eth0 [10.132.0.93/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-4c342351-7bf9d48b79

FailedCreate

Error creating: pods "payload-pre-processing-e2e-migrate-4c342351-7bf9d48b79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-4c342351: serviceaccount "payload-processing-e2e-migrate-4c342351" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-4c342351-7b78dfb54c

SuccessfulCreate

Created pod: payload-processing-e2e-migrate-4c342351-7b78dfb54c-xjx6n

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-4c342351-7bf9d48b79-hx859

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-4c342351-7bf9d48b79-hx859

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-migrate-4c342351-7b78dfb54c-xjx6n

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-migrate-4c342351-7b78dfb54c-xjx6n

Created

Container created

openshift-ingress

replicaset-controller

e2e-extmod-b-204fba-openshift-default-8dd6555f7

SuccessfulCreate

Created pod: e2e-extmod-b-204fba-openshift-default-8dd6555f7-fvrvg

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-204fba-openshift-default

SuccessfulRescale

New size: 8; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-extmod-b-204fba-openshift-default-8dd6555f7

SuccessfulCreate

Created pod: e2e-extmod-b-204fba-openshift-default-8dd6555f7-jqhv4

openshift-ingress

deployment-controller

e2e-extmod-b-204fba-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-204fba-openshift-default-8dd6555f7 from 6 to 8

openshift-ingress

multus

e2e-extmod-b-204fba-openshift-default-8dd6555f7-fvrvg

AddedInterface

Add eth0 [10.132.0.95/23] from ovn-kubernetes

openshift-ingress

multus

e2e-extmod-b-204fba-openshift-default-8dd6555f7-jqhv4

AddedInterface

Add eth0 [10.134.0.160/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-jqhv4

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-jqhv4

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-fvrvg

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-fvrvg

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-jqhv4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-fvrvg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-shared-a-w4-bde944-openshift-default-75487f99f9

SuccessfulCreate

Created pod: e2e-shared-a-w4-bde944-openshift-default-75487f99f9-lmdfc
(x2)

openshift-ingress

kubelet

e2e-mt-9cbd331c-openshift-default-574b7ff47c-8pgt7

Created

Container created

openshift-ingress

deployment-controller

e2e-shared-a-w4-bde944-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w4-bde944-openshift-default-75487f99f9 from 5 to 7

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w4-bde944-openshift-default

SuccessfulRescale

New size: 7; reason: cpu resource utilization (percentage of request) above target
(x2)

openshift-ingress

kubelet

e2e-mt-9cbd331c-openshift-default-574b7ff47c-8pgt7

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-mt-9cbd331c-openshift-default-574b7ff47c-8pgt7

Started

Container started

openshift-ingress

replicaset-controller

e2e-shared-a-w4-bde944-openshift-default-75487f99f9

SuccessfulCreate

Created pod: e2e-shared-a-w4-bde944-openshift-default-75487f99f9-vq4ds

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-vq4ds

Created

Container created
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-migrate-4c342351-gw-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-vq4ds

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-lmdfc

Created

Container created

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-vq4ds

Started

Container started

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-lmdfc

Started

Container started

openshift-ingress

multus

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-vq4ds

AddedInterface

Add eth0 [10.133.0.69/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-lmdfc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-lmdfc

AddedInterface

Add eth0 [10.134.0.161/23] from ovn-kubernetes
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-migrate-4c342351-gw-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x6)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-jqhv4

Unhealthy

Startup probe failed: Get "http://10.134.0.160:15021/healthz/ready": dial tcp 10.134.0.160:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-fvrvg

ProbeError

Startup probe error: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-jqhv4

ProbeError

Startup probe error: Get "http://10.134.0.160:15021/healthz/ready": dial tcp 10.134.0.160:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-fvrvg

Unhealthy

Startup probe failed: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-4c342351-7bf9d48b79-hx859

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-migrate-4c342351-gw-openshift-default-5bd6866449-t8rlj

Unhealthy

Readiness probe failed: Get "http://10.134.0.151:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-migrate-4c342351-gw-openshift-default-5bd6866449-t8rlj

ProbeError

Readiness probe error: Get "http://10.134.0.151:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

payload-processing-e2e-migrate-4c342351-7b78dfb54c-xjx6n

Killing

Stopping container payload-processing
(x6)

openshift-ingress

kubelet

e2e-mt-9cbd331c-openshift-default-574b7ff47c-8pgt7

Unhealthy

Startup probe failed: Get "http://10.134.0.149:15021/healthz/ready": dial tcp 10.134.0.149:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-mt-9cbd331c-openshift-default-574b7ff47c-8pgt7

ProbeError

Startup probe error: Get "http://10.134.0.149:15021/healthz/ready": dial tcp 10.134.0.149:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-vq4ds

Unhealthy

Startup probe failed: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-vq4ds

ProbeError

Startup probe error: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-mt-9cbd331c-openshift-default-574b7ff47c-8pgt7

ProbeError

Startup probe error: Get "http://10.134.0.149:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-mt-9cbd331c-openshift-default-574b7ff47c-8pgt7

Unhealthy

Startup probe failed: Get "http://10.134.0.149:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x7)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-lmdfc

Unhealthy

Startup probe failed: Get "http://10.134.0.161:15021/healthz/ready": dial tcp 10.134.0.161:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-ait-62e67514-openshift-default-6cd4788d5

SuccessfulCreate

Created pod: e2e-ait-62e67514-openshift-default-6cd4788d5-765tv
(x7)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-lmdfc

ProbeError

Startup probe error: Get "http://10.134.0.161:15021/healthz/ready": dial tcp 10.134.0.161:15021: connect: connection refused body:

openshift-ingress

deployment-controller

e2e-ait-62e67514-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-62e67514-openshift-default-6cd4788d5 from 0 to 1

openshift-ingress

kubelet

e2e-migrate-4c342351-gw-openshift-default-5bd6866449-t8rlj

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-ait-62e67514

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-62e67514

openshift-ingress

multus

e2e-ait-62e67514-openshift-default-6cd4788d5-765tv

AddedInterface

Add eth0 [10.132.0.96/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-62e67514-74478d9975

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-62e67514-74478d9975-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-62e67514: serviceaccount "payload-processing-e2e-ait-62e67514" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-62e67514-7b847995

SuccessfulCreate

Created pod: payload-processing-e2e-ait-62e67514-7b847995-hjrwz

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-62e67514-74478d9975

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-62e67514-74478d9975-7sdxb
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-62e67514-7b847995

FailedCreate

Error creating: pods "payload-processing-e2e-ait-62e67514-7b847995-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-62e67514: serviceaccount "payload-processing-e2e-ait-62e67514" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-62e67514

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-62e67514-74478d9975 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-ait-62e67514

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-62e67514-7b847995 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-62e67514-74478d9975-7sdxb

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-62e67514-74478d9975-7sdxb

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-ait-62e67514-74478d9975-7sdxb

AddedInterface

Add eth0 [10.134.0.164/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-62e67514-74478d9975-7sdxb

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ait-62e67514-7b847995-hjrwz

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ait-62e67514-7b847995-hjrwz

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ait-62e67514-7b847995-hjrwz

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-ait-62e67514-7b847995-hjrwz

AddedInterface

Add eth0 [10.134.0.165/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 6; reason: All metrics below target

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-75fd9f5d47 from 8 to 6

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-6pk2b

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-4d57x

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-4d57x

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-6pk2b

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w4-9e7156-openshift-default

SuccessfulRescale

New size: 6; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-shared-b-w4-9e7156-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w4-9e7156-openshift-default-6bc7994879 from 4 to 6

openshift-ingress

replicaset-controller

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879

SuccessfulCreate

Created pod: e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-cwzxw

openshift-ingress

replicaset-controller

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879

SuccessfulCreate

Created pod: e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-s52zd

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-cwzxw

Created

Container created

openshift-ingress

multus

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-cwzxw

AddedInterface

Add eth0 [10.132.0.98/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-cwzxw

Started

Container started

openshift-ingress

multus

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-s52zd

AddedInterface

Add eth0 [10.133.0.70/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-s52zd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-s52zd

Created

Container created

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-cwzxw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-s52zd

Started

Container started
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-62e67514-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-62e67514-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-9cbd331c-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-9cbd331c-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x6)

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-s52zd

Unhealthy

Startup probe failed: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-s52zd

ProbeError

Startup probe error: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-ait-62e67514-openshift-default-6cd4788d5-765tv

ProbeError

Readiness probe error: Get "http://10.132.0.96:15021/healthz/ready": read tcp 10.132.0.2:48764->10.132.0.96:15021: read: connection reset by peer body:
(x6)

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-cwzxw

ProbeError

Startup probe error: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-cwzxw

Unhealthy

Startup probe failed: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused

openshift-ingress

kubelet

e2e-ait-62e67514-openshift-default-6cd4788d5-765tv

Unhealthy

Readiness probe failed: Get "http://10.132.0.96:15021/healthz/ready": read tcp 10.132.0.2:48764->10.132.0.96:15021: read: connection reset by peer
(x2)

openshift-ingress

kubelet

e2e-ait-62e67514-openshift-default-6cd4788d5-765tv

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ait-62e67514-openshift-default-6cd4788d5-765tv

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-ait-62e67514-openshift-default-6cd4788d5-765tv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x6)

openshift-ingress

kubelet

e2e-ait-62e67514-openshift-default-6cd4788d5-765tv

Unhealthy

Startup probe failed: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-ait-62e67514-openshift-default-6cd4788d5-765tv

ProbeError

Startup probe error: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-62e67514-74478d9975-7sdxb

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-62e67514-7b847995-hjrwz

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ait-62e67514-openshift-default-6cd4788d5-765tv

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-ait-87e02313-openshift-default-8749f6555

SuccessfulCreate

Created pod: e2e-ait-87e02313-openshift-default-8749f6555-zhlms
(x4)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-ndhs8

Unhealthy

Readiness probe failed: Get "http://10.133.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x3)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-bcqxw

ProbeError

Readiness probe error: Get "http://10.132.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

deployment-controller

e2e-ait-87e02313-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-87e02313-openshift-default-8749f6555 from 0 to 1
(x4)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-ndhs8

ProbeError

Readiness probe error: Get "http://10.133.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

gateway_labeler_controller

e2e-ait-87e02313

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-87e02313
(x3)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-bcqxw

Unhealthy

Readiness probe failed: Get "http://10.132.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

multus

e2e-ait-87e02313-openshift-default-8749f6555-zhlms

AddedInterface

Add eth0 [10.134.0.170/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-mt-9cbd331c-openshift-default-574b7ff47c-6hx9j

Created

Container created

openshift-ingress

kubelet

e2e-mt-9cbd331c-openshift-default-574b7ff47c-6hx9j

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-mt-9cbd331c-openshift-default-574b7ff47c

SuccessfulCreate

Created pod: e2e-mt-9cbd331c-openshift-default-574b7ff47c-6hx9j

openshift-ingress

deployment-controller

e2e-mt-9cbd331c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-9cbd331c-openshift-default-574b7ff47c from 1 to 2

openshift-ingress

kubelet

e2e-mt-9cbd331c-openshift-default-574b7ff47c-6hx9j

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-9cbd331c-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

e2e-mt-9cbd331c-openshift-default-574b7ff47c-6hx9j

AddedInterface

Add eth0 [10.132.0.99/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7363-2b9420

Pulling

Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5"

opendatahub

multus

test-curl-7363-2b9420

AddedInterface

Add eth0 [10.132.0.100/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-87e02313

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-87e02313-5df95b66cf from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-87e02313-5df95b66cf

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-87e02313-5df95b66cf-lqlst
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-87e02313-5df95b66cf

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-87e02313-5df95b66cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-87e02313: serviceaccount "payload-processing-e2e-ait-87e02313" not found

openshift-ingress

deployment-controller

payload-processing-e2e-ait-87e02313

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-87e02313-b88c5bc4f from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-87e02313-b88c5bc4f

FailedCreate

Error creating: pods "payload-processing-e2e-ait-87e02313-b88c5bc4f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-87e02313: serviceaccount "payload-processing-e2e-ait-87e02313" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-87e02313-b88c5bc4f

SuccessfulCreate

Created pod: payload-processing-e2e-ait-87e02313-b88c5bc4f-4v9cc

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-87e02313-5df95b66cf-lqlst

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-87e02313-5df95b66cf-lqlst

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ait-87e02313-b88c5bc4f-4v9cc

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-87e02313-5df95b66cf-lqlst

Started

Container started

openshift-ingress

multus

payload-processing-e2e-ait-87e02313-b88c5bc4f-4v9cc

AddedInterface

Add eth0 [10.134.0.172/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-87e02313-b88c5bc4f-4v9cc

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ait-87e02313-b88c5bc4f-4v9cc

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-ait-87e02313-5df95b66cf-lqlst

AddedInterface

Add eth0 [10.134.0.173/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

e2e-mt-9cbd331c-openshift-default-574b7ff47c-6hx9j

ProbeError

Startup probe error: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-mt-9cbd331c-openshift-default-574b7ff47c-6hx9j

Unhealthy

Startup probe failed: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused

opendatahub

kubelet

test-curl-7363-2b9420

Started

Container started

opendatahub

kubelet

test-curl-7363-2b9420

Created

Container created

opendatahub

kubelet

test-curl-7363-2b9420

Pulled

Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 11.74s (11.74s including waiting). Image size: 106100986 bytes.

opendatahub

kubelet

test-curl-7363-2b9420

Killing

Stopping container test-curl-7363-2b9420

openshift-ingress

kubelet

payload-processing-e2e-mt-9cbd331c-789b6c9bcf-qscjb

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-9cbd331c-b5c5f659-4jmx6

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ait-87e02313-openshift-default-8749f6555-zhlms

Unhealthy

Readiness probe failed: Get "http://10.134.0.170:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-ait-87e02313-openshift-default-8749f6555-zhlms

ProbeError

Readiness probe error: Get "http://10.134.0.170:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-ait-87e02313-openshift-default-8749f6555-zhlms

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-ait-87e02313-openshift-default-8749f6555-zhlms

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-ait-87e02313-openshift-default-8749f6555-zhlms

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-ait-87e02313-openshift-default-8749f6555-zhlms

Unhealthy

Startup probe failed: Get "http://10.134.0.170:15021/healthz/ready": dial tcp 10.134.0.170:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-ait-87e02313-openshift-default-8749f6555-zhlms

ProbeError

Startup probe error: Get "http://10.134.0.170:15021/healthz/ready": dial tcp 10.134.0.170:15021: connect: connection refused body:

openshift-ingress

deployment-controller

e2e-autores-ced1b7-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-autores-ced1b7-openshift-default-7d96c7f879 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-autores-ced1b7

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-autores-ced1b7

openshift-ingress

replicaset-controller

e2e-autores-ced1b7-openshift-default-7d96c7f879

SuccessfulCreate

Created pod: e2e-autores-ced1b7-openshift-default-7d96c7f879-hgv8h
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-87e02313-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-87e02313-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-processing-e2e-ait-87e02313-b88c5bc4f-4v9cc

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-87e02313-5df95b66cf-lqlst

Killing

Stopping container payload-pre-processing

openshift-ingress

multus

e2e-autores-ced1b7-openshift-default-7d96c7f879-hgv8h

AddedInterface

Add eth0 [10.132.0.103/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-l2jsl

Unhealthy

Readiness probe failed: Get "http://10.132.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-l2jsl

ProbeError

Readiness probe error: Get "http://10.132.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-9cbd331c-b5c5f659-4jmx6

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-mt-9cbd331c-789b6c9bcf-qscjb

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-ait-87e02313-openshift-default-8749f6555-zhlms

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-derive-13c6a8db

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-derive-13c6a8db

openshift-ingress

deployment-controller

e2e-derive-13c6a8db-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-derive-13c6a8db-openshift-default-797684c4c7 from 0 to 1

openshift-ingress

replicaset-controller

e2e-derive-13c6a8db-openshift-default-797684c4c7

SuccessfulCreate

Created pod: e2e-derive-13c6a8db-openshift-default-797684c4c7-7b6lw

openshift-ingress

multus

e2e-derive-13c6a8db-openshift-default-797684c4c7-7b6lw

AddedInterface

Add eth0 [10.134.0.182/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-ced1b7-7557c8bd4c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-autores-ced1b7-7557c8bd4c-pslr8

openshift-ingress

deployment-controller

payload-processing-e2e-autores-ced1b7

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-autores-ced1b7-865949c6d5 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-ced1b7-865949c6d5

FailedCreate

Error creating: pods "payload-processing-e2e-autores-ced1b7-865949c6d5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-ced1b7: serviceaccount "payload-processing-e2e-autores-ced1b7" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-autores-ced1b7

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-autores-ced1b7-7557c8bd4c from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-ced1b7-7557c8bd4c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-autores-ced1b7-7557c8bd4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-ced1b7: serviceaccount "payload-processing-e2e-autores-ced1b7" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-ced1b7-865949c6d5

SuccessfulCreate

Created pod: payload-processing-e2e-autores-ced1b7-865949c6d5-w8d7k

openshift-ingress

multus

payload-processing-e2e-autores-ced1b7-865949c6d5-w8d7k

AddedInterface

Add eth0 [10.134.0.183/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-ced1b7-7557c8bd4c-pslr8

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-ced1b7-7557c8bd4c-pslr8

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-ced1b7-7557c8bd4c-pslr8

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-autores-ced1b7-865949c6d5-w8d7k

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-autores-ced1b7-7557c8bd4c-pslr8

AddedInterface

Add eth0 [10.132.0.105/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-autores-ced1b7-865949c6d5-w8d7k

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-autores-ced1b7-865949c6d5-w8d7k

Created

Container created
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-13c6a8db-646b88c58c

FailedCreate

Error creating: pods "payload-processing-e2e-derive-13c6a8db-646b88c58c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-13c6a8db: serviceaccount "payload-processing-e2e-derive-13c6a8db" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-derive-13c6a8db

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-derive-13c6a8db-c677584cf from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-13c6a8db-c677584cf

SuccessfulCreate

Created pod: payload-pre-processing-e2e-derive-13c6a8db-c677584cf-6h9fl
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-13c6a8db-c677584cf

FailedCreate

Error creating: pods "payload-pre-processing-e2e-derive-13c6a8db-c677584cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-13c6a8db: serviceaccount "payload-processing-e2e-derive-13c6a8db" not found

openshift-ingress

deployment-controller

payload-processing-e2e-derive-13c6a8db

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-derive-13c6a8db-646b88c58c from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-13c6a8db-646b88c58c

SuccessfulCreate

Created pod: payload-processing-e2e-derive-13c6a8db-646b88c58c-69lf2

openshift-ingress

multus

payload-pre-processing-e2e-derive-13c6a8db-c677584cf-6h9fl

AddedInterface

Add eth0 [10.134.0.184/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-13c6a8db-c677584cf-6h9fl

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-derive-13c6a8db-646b88c58c-69lf2

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-derive-13c6a8db-646b88c58c-69lf2

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-derive-13c6a8db-646b88c58c-69lf2

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-13c6a8db-c677584cf-6h9fl

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-13c6a8db-c677584cf-6h9fl

Created

Container created

openshift-ingress

multus

payload-processing-e2e-derive-13c6a8db-646b88c58c-69lf2

AddedInterface

Add eth0 [10.134.0.185/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-mt-9cbd331c

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-9cbd331c-789b6c9bcf from 0 to 1
(x4)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-bssvc

Unhealthy

Readiness probe failed: Get "http://10.133.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x4)

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-bssvc

ProbeError

Readiness probe error: Get "http://10.133.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-9cbd331c-b5c5f659

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-9cbd331c-b5c5f659-tm6n7
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-9cbd331c-789b6c9bcf

FailedCreate

Error creating: pods "payload-processing-e2e-mt-9cbd331c-789b6c9bcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-9cbd331c: serviceaccount "payload-processing-e2e-mt-9cbd331c" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-9cbd331c-789b6c9bcf

SuccessfulCreate

Created pod: payload-processing-e2e-mt-9cbd331c-789b6c9bcf-v4jgn
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-9cbd331c-b5c5f659

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-9cbd331c-b5c5f659-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-9cbd331c: serviceaccount "payload-processing-e2e-mt-9cbd331c" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-9cbd331c

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-9cbd331c-b5c5f659 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-9cbd331c-b5c5f659-tm6n7

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-mt-9cbd331c-789b6c9bcf-v4jgn

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-mt-9cbd331c-789b6c9bcf-v4jgn

AddedInterface

Add eth0 [10.134.0.188/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-9cbd331c-789b6c9bcf-v4jgn

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-9cbd331c-b5c5f659-tm6n7

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-mt-9cbd331c-789b6c9bcf-v4jgn

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-mt-9cbd331c-b5c5f659-tm6n7

AddedInterface

Add eth0 [10.134.0.187/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-9cbd331c-b5c5f659-tm6n7

Started

Container started

openshift-ingress

deployment-controller

e2e-shared-a-w4-bde944-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w4-bde944-openshift-default-75487f99f9 from 7 to 10

openshift-ingress

replicaset-controller

e2e-shared-a-w4-bde944-openshift-default-75487f99f9

SuccessfulCreate

Created pod: e2e-shared-a-w4-bde944-openshift-default-75487f99f9-5lrv7

openshift-ingress

replicaset-controller

e2e-shared-a-w4-bde944-openshift-default-75487f99f9

SuccessfulCreate

Created pod: e2e-shared-a-w4-bde944-openshift-default-75487f99f9-w22hv

openshift-ingress

replicaset-controller

e2e-shared-a-w4-bde944-openshift-default-75487f99f9

SuccessfulCreate

(combined from similar events): Created pod: e2e-shared-a-w4-bde944-openshift-default-75487f99f9-zt2tf

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w4-bde944-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-w22hv

AddedInterface

Add eth0 [10.132.0.107/23] from ovn-kubernetes

openshift-ingress

multus

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-zt2tf

AddedInterface

Add eth0 [10.133.0.74/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-w22hv

Created

Container created

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-w22hv

Started

Container started

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-w22hv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-5lrv7

AddedInterface

Add eth0 [10.134.0.190/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-zt2tf

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-zt2tf

Started

Container started

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-5lrv7

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-zt2tf

Created

Container created

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-5lrv7

Started

Container started

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-5lrv7

Created

Container created

openshift-ingress

deployment-controller

e2e-shared-b-w4-9e7156-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w4-9e7156-openshift-default-6bc7994879 from 6 to 9

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w4-9e7156-openshift-default

SuccessfulRescale

New size: 9; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879

SuccessfulCreate

Created pod: e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-x9j4q

openshift-ingress

replicaset-controller

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879

SuccessfulCreate

Created pod: e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-rgcns

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-x9j4q

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-x9j4q

AddedInterface

Add eth0 [10.132.0.109/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879

SuccessfulCreate

Created pod: e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-zblc5

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-rgcns

Started

Container started

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-rgcns

Created

Container created

openshift-ingress

multus

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-rgcns

AddedInterface

Add eth0 [10.134.0.192/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-rgcns

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-zblc5

AddedInterface

Add eth0 [10.133.0.75/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-zblc5

Started

Container started

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-zblc5

Created

Container created

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-zblc5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-x9j4q

Started

Container started

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-x9j4q

Created

Container created
(x7)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-5lrv7

ProbeError

Startup probe error: Get "http://10.134.0.190:15021/healthz/ready": dial tcp 10.134.0.190:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-zt2tf

Unhealthy

Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-zt2tf

ProbeError

Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-5lrv7

Unhealthy

Startup probe failed: Get "http://10.134.0.190:15021/healthz/ready": dial tcp 10.134.0.190:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-w22hv

Unhealthy

Startup probe failed: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-w22hv

ProbeError

Startup probe error: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulCreate

Created pod: istiod-openshift-gateway-7ddd9cfdc8-scwch

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 3
(x2)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-5lrv7

Unhealthy

Startup probe failed: Get "http://10.134.0.190:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-5lrv7

ProbeError

Startup probe error: Get "http://10.134.0.190:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

multus

istiod-openshift-gateway-7ddd9cfdc8-scwch

AddedInterface

Add eth0 [10.134.0.196/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-w22hv

ProbeError

Startup probe error: Get "http://10.132.0.107:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-w22hv

Unhealthy

Startup probe failed: Get "http://10.132.0.107:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-scwch

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8"
(x8)

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-rgcns

ProbeError

Startup probe error: Get "http://10.134.0.192:15021/healthz/ready": dial tcp 10.134.0.192:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-zblc5

ProbeError

Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-zblc5

Unhealthy

Startup probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-rgcns

Unhealthy

Startup probe failed: Get "http://10.134.0.192:15021/healthz/ready": dial tcp 10.134.0.192:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-x9j4q

Unhealthy

Startup probe failed: Get "http://10.132.0.109:15021/healthz/ready": dial tcp 10.132.0.109:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-x9j4q

ProbeError

Startup probe error: Get "http://10.132.0.109:15021/healthz/ready": dial tcp 10.132.0.109:15021: connect: connection refused body:
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-derive-13c6a8db-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-derive-13c6a8db-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-processing-e2e-derive-13c6a8db-646b88c58c-69lf2

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-derive-13c6a8db-openshift-default-797684c4c7-7b6lw

Unhealthy

Readiness probe failed: Get "http://10.134.0.182:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-derive-13c6a8db-openshift-default-797684c4c7-7b6lw

ProbeError

Readiness probe error: Get "http://10.134.0.182:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-13c6a8db-c677584cf-6h9fl

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-autores-ced1b7-openshift-default-7d96c7f879-hgv8h

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-autores-ced1b7-openshift-default-7d96c7f879-hgv8h

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-autores-ced1b7-openshift-default-7d96c7f879-hgv8h

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7363-a732ac

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

multus

test-curl-7363-a732ac

AddedInterface

Add eth0 [10.132.0.112/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7363-a732ac

Created

Container created

opendatahub

kubelet

test-curl-7363-a732ac

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-derive-13c6a8db-openshift-default-797684c4c7-7b6lw

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-derive-13c6a8db-openshift-default-797684c4c7-7b6lw

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-derive-13c6a8db-openshift-default-797684c4c7-7b6lw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-autores-ced1b7-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-scwch

Started

Container started

opendatahub

kubelet

test-curl-7363-a732ac

Killing

Stopping container test-curl-7363-a732ac
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-autores-ced1b7-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-scwch

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 12.53s (12.53s including waiting). Image size: 185476477 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-scwch

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-scwch

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

deployment-controller

e2e-extmod-b-204fba-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-204fba-openshift-default-8dd6555f7 from 8 to 10

openshift-ingress

replicaset-controller

e2e-extmod-b-204fba-openshift-default-8dd6555f7

SuccessfulCreate

Created pod: e2e-extmod-b-204fba-openshift-default-8dd6555f7-75t4b

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-scwch

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-4kb6q

Started

Container started

openshift-ingress

multus

e2e-extmod-b-204fba-openshift-default-8dd6555f7-4kb6q

AddedInterface

Add eth0 [10.132.0.113/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-4kb6q

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-75t4b

Started

Container started

openshift-ingress

multus

e2e-extmod-b-204fba-openshift-default-8dd6555f7-75t4b

AddedInterface

Add eth0 [10.133.0.76/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-75t4b

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-75t4b

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-4kb6q

Created

Container created
(x6)

openshift-ingress

kubelet

e2e-autores-ced1b7-openshift-default-7d96c7f879-hgv8h

ProbeError

Startup probe error: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-autores-ced1b7-openshift-default-7d96c7f879-hgv8h

Unhealthy

Startup probe failed: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-derive-13c6a8db-openshift-default-797684c4c7-7b6lw

ProbeError

Startup probe error: Get "http://10.134.0.182:15021/healthz/ready": dial tcp 10.134.0.182:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-derive-13c6a8db-openshift-default-797684c4c7-7b6lw

Unhealthy

Startup probe failed: Get "http://10.134.0.182:15021/healthz/ready": dial tcp 10.134.0.182:15021: connect: connection refused

openshift-ingress

kubelet

e2e-derive-13c6a8db-openshift-default-797684c4c7-7b6lw

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-api-a-a71884

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-a71884

openshift-ingress

deployment-controller

e2e-api-a-a71884-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-a-a71884-openshift-default-6b7df8cf59 from 0 to 1
(x5)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-75t4b

Unhealthy

Startup probe failed: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-api-a-a71884-openshift-default-6b7df8cf59

SuccessfulCreate

Created pod: e2e-api-a-a71884-openshift-default-6b7df8cf59-xxdrt

openshift-ingress

multus

e2e-api-a-a71884-openshift-default-6b7df8cf59-xxdrt

AddedInterface

Add eth0 [10.132.0.114/23] from ovn-kubernetes
(x5)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-75t4b

ProbeError

Startup probe error: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-4kb6q

Unhealthy

Startup probe failed: Get "http://10.132.0.113:15021/healthz/ready": dial tcp 10.132.0.113:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-4kb6q

ProbeError

Startup probe error: Get "http://10.132.0.113:15021/healthz/ready": dial tcp 10.132.0.113:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-api-a-a71884-7f8968dfdf-72wsf

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-a-a71884

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-a-a71884-94f55f9dc from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-a71884-7f8968dfdf

FailedCreate

Error creating: pods "payload-processing-e2e-api-a-a71884-7f8968dfdf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-a71884: serviceaccount "payload-processing-e2e-api-a-a71884" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-a71884-7f8968dfdf

SuccessfulCreate

Created pod: payload-processing-e2e-api-a-a71884-7f8968dfdf-72wsf

openshift-ingress

deployment-controller

payload-processing-e2e-api-a-a71884

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-a-a71884-7f8968dfdf from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-api-a-a71884-94f55f9dc-pzflm

AddedInterface

Add eth0 [10.134.0.199/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-a71884-94f55f9dc-pzflm

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-api-a-a71884-7f8968dfdf-72wsf

AddedInterface

Add eth0 [10.134.0.198/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-a71884-94f55f9dc

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-a-a71884-94f55f9dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-a71884: serviceaccount "payload-processing-e2e-api-a-a71884" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-a71884-94f55f9dc

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-a-a71884-94f55f9dc-pzflm

openshift-ingress

kubelet

payload-processing-e2e-api-a-a71884-7f8968dfdf-72wsf

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-api-a-a71884-7f8968dfdf-72wsf

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-a71884-94f55f9dc-pzflm

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-a71884-94f55f9dc-pzflm

Started

Container started

openshift-ingress

replicaset-controller

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879

SuccessfulCreate

(combined from similar events): Created pod: e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-jzm5f

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-jzm5f

Started

Container started

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-jzm5f

Created

Container created

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-jzm5f

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-jzm5f

AddedInterface

Add eth0 [10.134.0.200/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w4-9e7156-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-shared-b-w4-9e7156-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w4-9e7156-openshift-default-6bc7994879 from 9 to 10
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-autores-ced1b7-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-autores-ced1b7-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x6)

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-jzm5f

Unhealthy

Startup probe failed: Get "http://10.134.0.200:15021/healthz/ready": dial tcp 10.134.0.200:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-jzm5f

ProbeError

Startup probe error: Get "http://10.134.0.200:15021/healthz/ready": dial tcp 10.134.0.200:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

e2e-mt-9cbd331c-openshift-default-574b7ff47c

SuccessfulCreate

Created pod: e2e-mt-9cbd331c-openshift-default-574b7ff47c-g2dvk

openshift-ingress

multus

e2e-mt-9cbd331c-openshift-default-574b7ff47c-g2dvk

AddedInterface

Add eth0 [10.133.0.78/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-mt-9cbd331c-openshift-default-574b7ff47c-g2dvk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

e2e-mt-9cbd331c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-9cbd331c-openshift-default-574b7ff47c from 2 to 3

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-9cbd331c-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-mt-9cbd331c-openshift-default-574b7ff47c-g2dvk

Started

Container started

openshift-ingress

kubelet

e2e-mt-9cbd331c-openshift-default-574b7ff47c-g2dvk

Created

Container created

openshift-ingress

replicaset-controller

e2e-api-b-ae13d4-openshift-default-74cd94fc9f

SuccessfulCreate

Created pod: e2e-api-b-ae13d4-openshift-default-74cd94fc9f-mvzmt

openshift-ingress

deployment-controller

e2e-api-b-ae13d4-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-b-ae13d4-openshift-default-74cd94fc9f from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-api-b-ae13d4

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-ae13d4
(x5)

openshift-ingress

kubelet

e2e-mt-9cbd331c-openshift-default-574b7ff47c-g2dvk

ProbeError

Startup probe error: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused body:

openshift-ingress

multus

e2e-api-b-ae13d4-openshift-default-74cd94fc9f-mvzmt

AddedInterface

Add eth0 [10.132.0.115/23] from ovn-kubernetes
(x5)

openshift-ingress

kubelet

e2e-mt-9cbd331c-openshift-default-574b7ff47c-g2dvk

Unhealthy

Startup probe failed: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-autores-ced1b7-865949c6d5-w8d7k

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-ced1b7-7557c8bd4c-pslr8

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-api-a-a71884-openshift-default-6b7df8cf59-xxdrt

ProbeError

Readiness probe error: Get "http://10.132.0.114:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-api-a-a71884-openshift-default-6b7df8cf59-xxdrt

Unhealthy

Readiness probe failed: Get "http://10.132.0.114:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-api-a-a71884-openshift-default-6b7df8cf59-xxdrt

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-api-a-a71884-openshift-default-6b7df8cf59-xxdrt

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-api-a-a71884-openshift-default-6b7df8cf59-xxdrt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-9cbd331c-b5c5f659-tm6n7

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-9cbd331c-789b6c9bcf-v4jgn

Killing

Stopping container payload-processing
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-ae13d4-6d88f49fd5

FailedCreate

Error creating: pods "payload-processing-e2e-api-b-ae13d4-6d88f49fd5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-ae13d4: serviceaccount "payload-processing-e2e-api-b-ae13d4" not found

openshift-ingress

deployment-controller

payload-processing-e2e-api-b-ae13d4

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-b-ae13d4-6d88f49fd5 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-b-ae13d4

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-b-ae13d4-8f94f446c from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-ae13d4-8f94f446c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-b-ae13d4-8f94f446c-jcvtv
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-ae13d4-8f94f446c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-b-ae13d4-8f94f446c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-ae13d4: serviceaccount "payload-processing-e2e-api-b-ae13d4" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-ae13d4-6d88f49fd5

SuccessfulCreate

Created pod: payload-processing-e2e-api-b-ae13d4-6d88f49fd5-bjfjb

openshift-ingress

multus

payload-pre-processing-e2e-api-b-ae13d4-8f94f446c-jcvtv

AddedInterface

Add eth0 [10.133.0.79/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-api-b-ae13d4-6d88f49fd5-bjfjb

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-ae13d4-8f94f446c-jcvtv

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-ae13d4-8f94f446c-jcvtv

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-ae13d4-8f94f446c-jcvtv

Started

Container started

openshift-ingress

multus

payload-processing-e2e-api-b-ae13d4-6d88f49fd5-bjfjb

AddedInterface

Add eth0 [10.133.0.80/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-api-b-ae13d4-6d88f49fd5-bjfjb

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-api-b-ae13d4-6d88f49fd5-bjfjb

Created

Container created
(x6)

openshift-ingress

kubelet

e2e-api-a-a71884-openshift-default-6b7df8cf59-xxdrt

Unhealthy

Startup probe failed: Get "http://10.132.0.114:15021/healthz/ready": dial tcp 10.132.0.114:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-api-a-a71884-openshift-default-6b7df8cf59-xxdrt

ProbeError

Startup probe error: Get "http://10.132.0.114:15021/healthz/ready": dial tcp 10.132.0.114:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-autores-ced1b7-openshift-default-7d96c7f879-hgv8h

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-autores-ced1b7-openshift-default-7d96c7f879-hgv8h

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-autores-ced1b7-openshift-default-7d96c7f879-hgv8h

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

gateway_labeler_controller

e2e-explicit-9a69d1

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-explicit-9a69d1

openshift-ingress

replicaset-controller

e2e-explicit-9a69d1-openshift-default-5c97df6d45

SuccessfulCreate

Created pod: e2e-explicit-9a69d1-openshift-default-5c97df6d45-844jv

openshift-ingress

deployment-controller

e2e-explicit-9a69d1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-explicit-9a69d1-openshift-default-5c97df6d45 from 0 to 1

openshift-ingress

kubelet

e2e-mt-9cbd331c-openshift-default-574b7ff47c-8pgt7

Killing

Stopping container istio-proxy

openshift-ingress

multus

e2e-explicit-9a69d1-openshift-default-5c97df6d45-844jv

AddedInterface

Add eth0 [10.134.0.201/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-mt-9cbd331c-openshift-default-574b7ff47c-g2dvk

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-9cbd331c-openshift-default-574b7ff47c-6hx9j

Killing

Stopping container istio-proxy
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-a-a71884-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-a-a71884-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-mt-9cbd331c-openshift-default-574b7ff47c-g2dvk

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-mt-9cbd331c-openshift-default-574b7ff47c-g2dvk

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-mt-9cbd331c-openshift-default-574b7ff47c-6hx9j

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-mt-9cbd331c-openshift-default-574b7ff47c-6hx9j

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-9a69d1-cbbb9c4d9

FailedCreate

Error creating: pods "payload-processing-e2e-explicit-9a69d1-cbbb9c4d9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-9a69d1: serviceaccount "payload-processing-e2e-explicit-9a69d1" not found

openshift-ingress

multus

payload-pre-processing-e2e-explicit-9a69d1-7869845747-mnbt9

AddedInterface

Add eth0 [10.132.0.116/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-explicit-9a69d1

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-explicit-9a69d1-7869845747 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-9a69d1-7869845747

SuccessfulCreate

Created pod: payload-pre-processing-e2e-explicit-9a69d1-7869845747-mnbt9
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-9a69d1-7869845747

FailedCreate

Error creating: pods "payload-pre-processing-e2e-explicit-9a69d1-7869845747-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-9a69d1: serviceaccount "payload-processing-e2e-explicit-9a69d1" not found

openshift-ingress

deployment-controller

payload-processing-e2e-explicit-9a69d1

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-explicit-9a69d1-cbbb9c4d9 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-9a69d1-cbbb9c4d9

SuccessfulCreate

Created pod: payload-processing-e2e-explicit-9a69d1-cbbb9c4d9-kp42c

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-9a69d1-7869845747-mnbt9

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-explicit-9a69d1-cbbb9c4d9-kp42c

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-explicit-9a69d1-cbbb9c4d9-kp42c

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-explicit-9a69d1-cbbb9c4d9-kp42c

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-explicit-9a69d1-cbbb9c4d9-kp42c

AddedInterface

Add eth0 [10.132.0.117/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-9a69d1-7869845747-mnbt9

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-9a69d1-7869845747-mnbt9

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-75fd9f5d47-7cnfk

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-75fd9f5d47-zjvjd

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-75fd9f5d47 from 6 to 10

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-75fd9f5d47-wdc6m

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-75fd9f5d47-7prvd

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-wdc6m

AddedInterface

Add eth0 [10.132.0.119/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-wdc6m

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

e2e-api-b-ae13d4-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-zjvjd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-7prvd

AddedInterface

Add eth0 [10.133.0.82/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-api-b-ae13d4-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-a-a71884-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-a-a71884-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-zjvjd

AddedInterface

Add eth0 [10.132.0.120/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-7prvd

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-7prvd

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-7prvd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-7cnfk

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-7cnfk

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-wdc6m

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-wdc6m

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-7cnfk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-zjvjd

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-zjvjd

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-7cnfk

AddedInterface

Add eth0 [10.134.0.203/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7366-5b297b

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

multus

test-curl-7366-5b297b

AddedInterface

Add eth0 [10.132.0.121/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7366-5b297b

Created

Container created

opendatahub

kubelet

test-curl-7366-5b297b

Started

Container started

openshift-ingress

kubelet

e2e-api-b-ae13d4-openshift-default-74cd94fc9f-mvzmt

Unhealthy

Readiness probe failed: Get "http://10.132.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-api-b-ae13d4-openshift-default-74cd94fc9f-mvzmt

ProbeError

Readiness probe error: Get "http://10.132.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

payload-processing-e2e-api-a-a71884-7f8968dfdf-72wsf

Killing

Stopping container payload-processing

opendatahub

kubelet

test-curl-7366-ae3ae1

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-a71884-94f55f9dc-pzflm

Killing

Stopping container payload-pre-processing

opendatahub

multus

test-curl-7366-ae3ae1

AddedInterface

Add eth0 [10.132.0.122/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7366-5b297b

Killing

Stopping container test-curl-7366-5b297b

opendatahub

kubelet

test-curl-7366-ae3ae1

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-api-b-ae13d4-openshift-default-74cd94fc9f-mvzmt

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-api-b-ae13d4-openshift-default-74cd94fc9f-mvzmt

Started

Container started

opendatahub

kubelet

test-curl-7366-ae3ae1

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-api-b-ae13d4-openshift-default-74cd94fc9f-mvzmt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-7prvd

Unhealthy

Startup probe failed: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-7prvd

ProbeError

Startup probe error: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-7cnfk

Unhealthy

Startup probe failed: Get "http://10.134.0.203:15021/healthz/ready": dial tcp 10.134.0.203:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-7cnfk

ProbeError

Startup probe error: Get "http://10.134.0.203:15021/healthz/ready": dial tcp 10.134.0.203:15021: connect: connection refused body:

opendatahub

kubelet

test-curl-7366-ae3ae1

Killing

Stopping container test-curl-7366-ae3ae1

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-7prvd

Unhealthy

Startup probe failed: Get "http://10.133.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x9)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-wdc6m

Unhealthy

Startup probe failed: Get "http://10.132.0.119:15021/healthz/ready": dial tcp 10.132.0.119:15021: connect: connection refused
(x9)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-wdc6m

ProbeError

Startup probe error: Get "http://10.132.0.119:15021/healthz/ready": dial tcp 10.132.0.119:15021: connect: connection refused body:
(x9)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-zjvjd

Unhealthy

Startup probe failed: Get "http://10.132.0.120:15021/healthz/ready": dial tcp 10.132.0.120:15021: connect: connection refused

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-7prvd

ProbeError

Startup probe error: Get "http://10.133.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x9)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-zjvjd

ProbeError

Startup probe error: Get "http://10.132.0.120:15021/healthz/ready": dial tcp 10.132.0.120:15021: connect: connection refused body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-7cnfk

Unhealthy

Startup probe failed: Get "http://10.134.0.203:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-7cnfk

ProbeError

Startup probe error: Get "http://10.134.0.203:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-zjvjd

Unhealthy

Startup probe failed: Get "http://10.132.0.120:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-zjvjd

ProbeError

Startup probe error: Get "http://10.132.0.120:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-wdc6m

ProbeError

Startup probe error: Get "http://10.132.0.119:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-wdc6m

Unhealthy

Startup probe failed: Get "http://10.132.0.119:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x6)

openshift-ingress

kubelet

e2e-api-b-ae13d4-openshift-default-74cd94fc9f-mvzmt

Unhealthy

Startup probe failed: Get "http://10.132.0.115:15021/healthz/ready": dial tcp 10.132.0.115:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-api-b-ae13d4-openshift-default-74cd94fc9f-mvzmt

ProbeError

Startup probe error: Get "http://10.132.0.115:15021/healthz/ready": dial tcp 10.132.0.115:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-api-a-a71884-openshift-default-6b7df8cf59-xxdrt

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-b-ae13d4-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-b-ae13d4-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-api-b-ae13d4-openshift-default-74cd94fc9f-mvzmt

Unhealthy

Startup probe failed: Get "http://10.132.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-api-b-ae13d4-openshift-default-74cd94fc9f-mvzmt

ProbeError

Startup probe error: Get "http://10.132.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-ae13d4-8f94f446c-jcvtv

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-ae13d4-6d88f49fd5-bjfjb

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

e2e-explicit-9a69d1-openshift-default-5c97df6d45-844jv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-explicit-9a69d1-openshift-default-5c97df6d45-844jv

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-explicit-9a69d1-openshift-default-5c97df6d45-844jv

Started

Container started
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-explicit-9a69d1-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-explicit-9a69d1-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-api-b-ae13d4-openshift-default-74cd94fc9f-mvzmt

Killing

Stopping container istio-proxy
(x7)

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-6hb6w

Unhealthy

Readiness probe failed: Get "https://10.134.0.48:8000/ready": context deadline exceeded

openshift-ingress

kubelet

e2e-api-b-ae13d4-openshift-default-74cd94fc9f-mvzmt

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-api-b-ae13d4-openshift-default-74cd94fc9f-mvzmt

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x10)

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-6hb6w

Unhealthy

Readiness probe failed: Get "https://10.134.0.48:8000/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
(x7)

openshift-ingress

kubelet

e2e-explicit-9a69d1-openshift-default-5c97df6d45-844jv

ProbeError

Startup probe error: Get "http://10.134.0.201:15021/healthz/ready": dial tcp 10.134.0.201:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-explicit-9a69d1-openshift-default-5c97df6d45-844jv

Unhealthy

Startup probe failed: Get "http://10.134.0.201:15021/healthz/ready": dial tcp 10.134.0.201:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

e2e-explicit-9a69d1-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-explicit-9a69d1-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x4)

models-as-a-service

replicaset-controller

payload-processing-external-model-f9c56bc9c

FailedCreate

Error creating: pods "payload-processing-external-model-f9c56bc9c-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-f9c56bc9c from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-f9c56bc9c

SuccessfulCreate

Created pod: payload-processing-external-model-f9c56bc9c-rph5n

models-as-a-service

multus

payload-processing-external-model-f9c56bc9c-rph5n

AddedInterface

Add eth0 [10.133.0.85/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-f9c56bc9c-rph5n

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-f9c56bc9c-rph5n

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-f9c56bc9c-rph5n

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-explicit-9a69d1-cbbb9c4d9-kp42c

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-9a69d1-7869845747-mnbt9

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-9a69d1-7869845747-mnbt9

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.132.0.116:50052" within 1s: context deadline exceeded

openshift-ingress

kubelet

payload-processing-e2e-explicit-9a69d1-cbbb9c4d9-kp42c

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.132.0.117:50052" within 1s: context deadline exceeded

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-87c8cf6 from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-87c8cf6

SuccessfulCreate

Created pod: payload-processing-external-model-87c8cf6-sftdh

models-as-a-service

multus

payload-processing-external-model-87c8cf6-sftdh

AddedInterface

Add eth0 [10.134.0.210/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-87c8cf6-sftdh

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-87c8cf6-sftdh

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-87c8cf6-sftdh

Started

Container started

openshift-ingress

kubelet

e2e-explicit-9a69d1-openshift-default-5c97df6d45-844jv

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-noten-0c5f4e-openshift-default-84f576f67

SuccessfulCreate

Created pod: e2e-noten-0c5f4e-openshift-default-84f576f67-6k6gg

openshift-ingress

deployment-controller

e2e-noten-0c5f4e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-noten-0c5f4e-openshift-default-84f576f67 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-noten-0c5f4e

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-noten-0c5f4e

openshift-ingress

multus

e2e-noten-0c5f4e-openshift-default-84f576f67-6k6gg

AddedInterface

Add eth0 [10.132.0.124/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-extmod-a-629129-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-629129-openshift-default-dc77fd7d9 from 1 to 2

openshift-ingress

replicaset-controller

e2e-extmod-a-629129-openshift-default-dc77fd7d9

SuccessfulCreate

Created pod: e2e-extmod-a-629129-openshift-default-dc77fd7d9-wzxjn

openshift-ingress

kubelet

e2e-noten-0c5f4e-openshift-default-84f576f67-6k6gg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-629129-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-wzxjn

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-wzxjn

Created

Container created

openshift-ingress

kubelet

e2e-noten-0c5f4e-openshift-default-84f576f67-6k6gg

Started

Container started

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-wzxjn

Started

Container started

openshift-ingress

kubelet

e2e-noten-0c5f4e-openshift-default-84f576f67-6k6gg

Created

Container created

openshift-ingress

multus

e2e-extmod-a-629129-openshift-default-dc77fd7d9-wzxjn

AddedInterface

Add eth0 [10.132.0.125/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-f9c56bc9c-rph5n

Killing

Stopping container payload-processing

models-as-a-service

replicaset-controller

payload-processing-external-model-f9c56bc9c

SuccessfulDelete

Deleted pod: payload-processing-external-model-f9c56bc9c-rph5n

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-f9c56bc9c from 1 to 0

openshift-ingress

replicaset-controller

e2e-worker-w3-a110ef-openshift-default-754f5584f9

SuccessfulDelete

Deleted pod: e2e-worker-w3-a110ef-openshift-default-754f5584f9-p2lgl

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-s2h9r

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-qkk8t

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p2lgl

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-a110ef-openshift-default

SuccessfulRescale

New size: 7; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-worker-w3-a110ef-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w3-a110ef-openshift-default-754f5584f9 from 10 to 7

openshift-ingress

replicaset-controller

e2e-worker-w3-a110ef-openshift-default-754f5584f9

SuccessfulDelete

Deleted pod: e2e-worker-w3-a110ef-openshift-default-754f5584f9-s2h9r

openshift-ingress

replicaset-controller

e2e-worker-w3-a110ef-openshift-default-754f5584f9

SuccessfulDelete

Deleted pod: e2e-worker-w3-a110ef-openshift-default-754f5584f9-qkk8t
(x5)

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-wzxjn

ProbeError

Startup probe error: Get "http://10.132.0.125:15021/healthz/ready": dial tcp 10.132.0.125:15021: connect: connection refused body:

models-as-a-service

replicaset-controller

payload-processing-external-model-7b5bbbbbb8

SuccessfulCreate

Created pod: payload-processing-external-model-7b5bbbbbb8-27w26
(x5)

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-wzxjn

Unhealthy

Startup probe failed: Get "http://10.132.0.125:15021/healthz/ready": dial tcp 10.132.0.125:15021: connect: connection refused

models-as-a-service

kubelet

payload-processing-external-model-7b5bbbbbb8-27w26

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

multus

payload-processing-external-model-7b5bbbbbb8-27w26

AddedInterface

Add eth0 [10.134.0.211/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-7b5bbbbbb8-27w26

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-7b5bbbbbb8-27w26

Started

Container started

openshift-ingress

kubelet

e2e-noten-0c5f4e-openshift-default-84f576f67-6k6gg

Killing

Stopping container istio-proxy

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-87c8cf6 from 1 to 0

models-as-a-service

replicaset-controller

payload-processing-external-model-87c8cf6

SuccessfulDelete

Deleted pod: payload-processing-external-model-87c8cf6-sftdh

models-as-a-service

kubelet

payload-processing-external-model-87c8cf6-sftdh

Killing

Stopping container payload-processing

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-5ff7768868 from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-5ff7768868

SuccessfulCreate

Created pod: payload-processing-external-model-5ff7768868-lfrbv

models-as-a-service

multus

payload-processing-external-model-5ff7768868-lfrbv

AddedInterface

Add eth0 [10.133.0.89/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-5ff7768868-lfrbv

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-5ff7768868-lfrbv

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-5ff7768868-lfrbv

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-7b5bbbbbb8-27w26

Killing

Stopping container payload-processing

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-7b5bbbbbb8 from 1 to 0

models-as-a-service

replicaset-controller

payload-processing-external-model-7b5bbbbbb8

SuccessfulDelete

Deleted pod: payload-processing-external-model-7b5bbbbbb8-27w26
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-zjvjd

ProbeError

Readiness probe error: Get "http://10.132.0.120:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-wdc6m

ProbeError

Readiness probe error: Get "http://10.132.0.119:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-wdc6m

Unhealthy

Readiness probe failed: Get "http://10.132.0.119:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-zjvjd

Unhealthy

Readiness probe failed: Get "http://10.132.0.120:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-2e80fb-77fd4bc6bc-ncz2j

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-2e80fb-7f8cb49b89-d2pv9

Killing

Stopping container payload-processing

models-as-a-service

kubelet

payload-processing-external-model-7b5bbbbbb8-ndszk

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-7b5bbbbbb8-ndszk

Created

Container created
(x2)

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-7b5bbbbbb8 from 0 to 1

models-as-a-service

kubelet

payload-processing-external-model-7b5bbbbbb8-ndszk

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

replicaset-controller

payload-processing-external-model-7b5bbbbbb8

SuccessfulCreate

Created pod: payload-processing-external-model-7b5bbbbbb8-ndszk

models-as-a-service

multus

payload-processing-external-model-7b5bbbbbb8-ndszk

AddedInterface

Add eth0 [10.133.0.91/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-7b5bbbbbb8-ndszk

Killing

Stopping container payload-processing

models-as-a-service

kubelet

payload-processing-external-model-5ff7768868-lfrbv

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-2e80fb-7f8cb49b89-d2pv9

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-2e80fb-77fd4bc6bc-ncz2j

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w4-bde944-c67f95877-bfj8t

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w4-bde944-99767bc7f-kbcnk

Killing

Stopping container payload-pre-processing

openshift-ingress

deployment-controller

e2e-extmod-a-629129-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-629129-openshift-default-dc77fd7d9 from 2 to 3

openshift-ingress

replicaset-controller

e2e-extmod-a-629129-openshift-default-dc77fd7d9

SuccessfulCreate

Created pod: e2e-extmod-a-629129-openshift-default-dc77fd7d9-fxn9s

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-629129-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

e2e-extmod-a-629129-openshift-default-dc77fd7d9-fxn9s

AddedInterface

Add eth0 [10.134.0.218/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-fxn9s

Started

Container started

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-fxn9s

Created

Container created

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-fxn9s

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-zx7kj

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-vq4ds

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-lmdfc

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-wk5lx

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-4cf62

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-58jd2

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-zt2tf

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-6f8mt

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-5lrv7

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w4-bde944-openshift-default-75487f99f9-w22hv

Killing

Stopping container istio-proxy
(x6)

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-fxn9s

ProbeError

Startup probe error: Get "http://10.134.0.218:15021/healthz/ready": dial tcp 10.134.0.218:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-fxn9s

Unhealthy

Startup probe failed: Get "http://10.134.0.218:15021/healthz/ready": dial tcp 10.134.0.218:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w4-9e7156-f747b4965-v9sds

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-rg5f7

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-rmc6x

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-bcqxw

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-gkljq

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w4-9e7156-7957df6fc5-ctbhp

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-k6wmf

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-bssvc

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-sk5vq

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-6krb6

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-l2jsl

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-ndhs8

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-ndhs8

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-ndhs8

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-bssvc

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-bssvc

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-bcqxw

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-bcqxw

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-k6wmf

Unhealthy

Readiness probe failed: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w0-2e80fb-openshift-default-5585568575-k6wmf

ProbeError

Readiness probe error: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w4-bde944-99767bc7f-kbcnk

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w4-bde944-c67f95877-bfj8t

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-jsxcs

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-s52zd

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-x9j4q

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-pgrhg

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-jzm5f

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-rgcns

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-zblc5

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-cwzxw

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-6mdlm

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-2cjnv

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-pgrhg

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-shared-b-w4-9e7156-openshift-default-6bc7994879-pgrhg

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w4-9e7156-7957df6fc5-ctbhp

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w4-9e7156-f747b4965-v9sds

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-629129-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-extmod-a-629129-openshift-default-dc77fd7d9

SuccessfulCreate

Created pod: e2e-extmod-a-629129-openshift-default-dc77fd7d9-qbv6j

openshift-ingress

multus

e2e-extmod-a-629129-openshift-default-dc77fd7d9-qbv6j

AddedInterface

Add eth0 [10.134.0.223/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-extmod-a-629129-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-629129-openshift-default-dc77fd7d9 from 3 to 4

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-qbv6j

Created

Container created

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-qbv6j

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-qbv6j

Started

Container started

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-5bf4f6c6d4 from 0 to 1
(x4)

models-as-a-service

replicaset-controller

payload-processing-external-model-5bf4f6c6d4

FailedCreate

Error creating: pods "payload-processing-external-model-5bf4f6c6d4-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

replicaset-controller

payload-processing-external-model-5bf4f6c6d4

SuccessfulCreate

Created pod: payload-processing-external-model-5bf4f6c6d4-b44j6

models-as-a-service

kubelet

payload-processing-external-model-5bf4f6c6d4-b44j6

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-5bf4f6c6d4-b44j6

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-qbv6j

Unhealthy

Startup probe failed: Get "http://10.134.0.223:15021/healthz/ready": dial tcp 10.134.0.223:15021: connect: connection refused

models-as-a-service

multus

payload-processing-external-model-5bf4f6c6d4-b44j6

AddedInterface

Add eth0 [10.133.0.97/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-5bf4f6c6d4-b44j6

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-qbv6j

ProbeError

Startup probe error: Get "http://10.134.0.223:15021/healthz/ready": dial tcp 10.134.0.223:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-r2twl

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-extmod-b-204fba-openshift-default-8dd6555f7

SuccessfulDelete

Deleted pod: e2e-extmod-b-204fba-openshift-default-8dd6555f7-r2twl

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-204fba-openshift-default

SuccessfulRescale

New size: 6; reason: All metrics below target

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-xl5jm

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-extmod-b-204fba-openshift-default-8dd6555f7

SuccessfulDelete

Deleted pod: e2e-extmod-b-204fba-openshift-default-8dd6555f7-xl5jm

openshift-ingress

deployment-controller

e2e-extmod-b-204fba-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-extmod-b-204fba-openshift-default-8dd6555f7 from 10 to 6

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-fvrvg

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-4kb6q

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-extmod-b-204fba-openshift-default-8dd6555f7

SuccessfulDelete

Deleted pod: e2e-extmod-b-204fba-openshift-default-8dd6555f7-4kb6q

openshift-ingress

replicaset-controller

e2e-extmod-b-204fba-openshift-default-8dd6555f7

SuccessfulDelete

Deleted pod: e2e-extmod-b-204fba-openshift-default-8dd6555f7-fvrvg

models-as-a-service

kubelet

payload-processing-external-model-5bf4f6c6d4-b44j6

Killing

Stopping container payload-processing
(x5)

models-as-a-service

replicaset-controller

payload-processing-external-model-6cdf744db

FailedCreate

Error creating: pods "payload-processing-external-model-6cdf744db-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-6cdf744db from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-6cdf744db

SuccessfulCreate

Created pod: payload-processing-external-model-6cdf744db-7tzrm

models-as-a-service

multus

payload-processing-external-model-6cdf744db-7tzrm

AddedInterface

Add eth0 [10.133.0.98/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-6cdf744db-7tzrm

Killing

Stopping container payload-processing

models-as-a-service

kubelet

payload-processing-external-model-6cdf744db-7tzrm

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-6cdf744db-7tzrm

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-6cdf744db-7tzrm

Started

Container started

models-as-a-service

replicaset-controller

payload-processing-external-model-6cdf744db

SuccessfulCreate

Created pod: payload-processing-external-model-6cdf744db-49v9c
(x5)

models-as-a-service

replicaset-controller

payload-processing-external-model-6cdf744db

FailedCreate

Error creating: pods "payload-processing-external-model-6cdf744db-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-6cdf744db from 0 to 1

models-as-a-service

kubelet

payload-processing-external-model-6cdf744db-49v9c

Started

Container started

models-as-a-service

multus

payload-processing-external-model-6cdf744db-49v9c

AddedInterface

Add eth0 [10.132.0.129/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-6cdf744db-49v9c

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (payload-processing-external-model-dockercfg-8xg25); attempting to pull the image may not succeed.

models-as-a-service

kubelet

payload-processing-external-model-6cdf744db-49v9c

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-6cdf744db-49v9c

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-6cdf744db-49v9c

Killing

Stopping container payload-processing

models-as-a-service

replicaset-controller

payload-processing-external-model-55b55c785

SuccessfulCreate

Created pod: payload-processing-external-model-55b55c785-vrkqt
(x4)

models-as-a-service

replicaset-controller

payload-processing-external-model-55b55c785

FailedCreate

Error creating: pods "payload-processing-external-model-55b55c785-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-55b55c785 from 0 to 1

models-as-a-service

kubelet

payload-processing-external-model-55b55c785-vrkqt

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-55b55c785-vrkqt

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

multus

payload-processing-external-model-55b55c785-vrkqt

AddedInterface

Add eth0 [10.133.0.99/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-55b55c785-vrkqt

Started

Container started
(x4)

models-as-a-service

kubelet

payload-processing-external-model-6cdf744db-49v9c

FailedMount

MountVolume.SetUp failed for volume "provider-credentials" : secret "agc-cred-bd224a8ca00c" not found
(x4)

models-as-a-service

kubelet

payload-processing-external-model-6cdf744db-49v9c

FailedMount

MountVolume.SetUp failed for volume "plugins-config-volume" : configmap "payload-processing-external-model-plugins" not found

models-as-a-service

kubelet

payload-processing-external-model-55b55c785-vrkqt

Killing

Stopping container payload-processing
(x4)

models-as-a-service

kubelet

payload-processing-external-model-55b55c785-vrkqt

FailedMount

MountVolume.SetUp failed for volume "provider-credentials" : secret "agc-cred-c3a9f12818cc" not found
(x4)

models-as-a-service

kubelet

payload-processing-external-model-55b55c785-vrkqt

FailedMount

MountVolume.SetUp failed for volume "plugins-config-volume" : configmap "payload-processing-external-model-plugins" not found

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-dg56j

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w3-a110ef-openshift-default-754f5584f9

SuccessfulDelete

Deleted pod: e2e-worker-w3-a110ef-openshift-default-754f5584f9-dg56j

openshift-ingress

deployment-controller

e2e-worker-w3-a110ef-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w3-a110ef-openshift-default-754f5584f9 from 7 to 6

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-a110ef-openshift-default

SuccessfulRescale

New size: 6; reason: All metrics below target

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-a110ef-c6fc947bc-hx6lh

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-a110ef-5766965cdf-gd9dj

Killing

Stopping container payload-processing

openshift-ingress

replicaset-controller

e2e-worker-w3-a110ef-openshift-default-754f5584f9

SuccessfulCreate

Created pod: e2e-worker-w3-a110ef-openshift-default-754f5584f9-2rh7r

openshift-ingress

replicaset-controller

e2e-worker-w3-a110ef-openshift-default-754f5584f9

SuccessfulCreate

Created pod: e2e-worker-w3-a110ef-openshift-default-754f5584f9-6fsp8

openshift-ingress

replicaset-controller

e2e-worker-w3-a110ef-openshift-default-754f5584f9

SuccessfulCreate

Created pod: e2e-worker-w3-a110ef-openshift-default-754f5584f9-58hgh

openshift-ingress

replicaset-controller

e2e-worker-w3-a110ef-openshift-default-754f5584f9

SuccessfulCreate

Created pod: e2e-worker-w3-a110ef-openshift-default-754f5584f9-p8fzp

openshift-ingress

deployment-controller

e2e-worker-w3-a110ef-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-a110ef-openshift-default-754f5584f9 from 6 to 10
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-a110ef-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-2rh7r

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-2rh7r

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w3-a110ef-openshift-default-754f5584f9-6fsp8

AddedInterface

Add eth0 [10.133.0.103/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-6fsp8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-58hgh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-2rh7r

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-6fsp8

Created

Container created

openshift-ingress

multus

e2e-worker-w3-a110ef-openshift-default-754f5584f9-58hgh

AddedInterface

Add eth0 [10.132.0.132/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p8fzp

Started

Container started

openshift-ingress

multus

e2e-worker-w3-a110ef-openshift-default-754f5584f9-2rh7r

AddedInterface

Add eth0 [10.134.0.229/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-6fsp8

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p8fzp

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p8fzp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p8fzp

AddedInterface

Add eth0 [10.133.0.102/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-58hgh

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-58hgh

Created

Container created
(x6)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-58hgh

ProbeError

Startup probe error: Get "http://10.132.0.132:15021/healthz/ready": dial tcp 10.132.0.132:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-58hgh

Unhealthy

Startup probe failed: Get "http://10.132.0.132:15021/healthz/ready": dial tcp 10.132.0.132:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-2rh7r

ProbeError

Startup probe error: Get "http://10.134.0.229:15021/healthz/ready": dial tcp 10.134.0.229:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-2rh7r

Unhealthy

Startup probe failed: Get "http://10.134.0.229:15021/healthz/ready": dial tcp 10.134.0.229:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-6fsp8

ProbeError

Startup probe error: Get "http://10.133.0.103:15021/healthz/ready": dial tcp 10.133.0.103:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p8fzp

Unhealthy

Startup probe failed: Get "http://10.133.0.102:15021/healthz/ready": dial tcp 10.133.0.102:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p8fzp

ProbeError

Startup probe error: Get "http://10.133.0.102:15021/healthz/ready": dial tcp 10.133.0.102:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-6fsp8

Unhealthy

Startup probe failed: Get "http://10.133.0.103:15021/healthz/ready": dial tcp 10.133.0.103:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-a110ef-5766965cdf-gd9dj

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-58hgh

ProbeError

Startup probe error: Get "http://10.132.0.132:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-58hgh

Unhealthy

Startup probe failed: Get "http://10.132.0.132:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-a110ef-c6fc947bc-hx6lh

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-6fsp8

ProbeError

Startup probe error: Get "http://10.133.0.103:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-6fsp8

Unhealthy

Startup probe failed: Get "http://10.133.0.103:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-6fsp8

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-42xxk

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p5zd5

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-p8fzp

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-rgfsg

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-58hgh

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-wmdl4

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-m8mhg

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-2rh7r

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-2d492

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-m8mhg

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w3-a110ef-openshift-default-754f5584f9-m8mhg

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

replicaset-controller

e2e-extmod-a-629129-openshift-default-dc77fd7d9

SuccessfulCreate

Created pod: e2e-extmod-a-629129-openshift-default-dc77fd7d9-cvdnx

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-629129-openshift-default

SuccessfulRescale

New size: 5; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-extmod-a-629129-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-629129-openshift-default-dc77fd7d9 from 4 to 5

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-cvdnx

Created

Container created

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-cvdnx

Started

Container started

openshift-ingress

multus

e2e-extmod-a-629129-openshift-default-dc77fd7d9-cvdnx

AddedInterface

Add eth0 [10.132.0.139/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-cvdnx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-cvdnx

Unhealthy

Startup probe failed: Get "http://10.132.0.139:15021/healthz/ready": dial tcp 10.132.0.139:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-cvdnx

ProbeError

Startup probe error: Get "http://10.132.0.139:15021/healthz/ready": dial tcp 10.132.0.139:15021: connect: connection refused body:

openshift-ingress

deployment-controller

e2e-extmod-a-629129-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-629129-openshift-default-dc77fd7d9 from 5 to 6

openshift-ingress

replicaset-controller

e2e-extmod-a-629129-openshift-default-dc77fd7d9

SuccessfulCreate

Created pod: e2e-extmod-a-629129-openshift-default-dc77fd7d9-vrphk

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-629129-openshift-default

SuccessfulRescale

New size: 6; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-sn5sr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-extmod-a-629129-openshift-default-dc77fd7d9-vrphk

AddedInterface

Add eth0 [10.133.0.106/23] from ovn-kubernetes

openshift-ingress

multus

e2e-extmod-b-204fba-openshift-default-8dd6555f7-sn5sr

AddedInterface

Add eth0 [10.132.0.141/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

e2e-extmod-b-204fba-openshift-default-8dd6555f7

SuccessfulCreate

(combined from similar events): Created pod: e2e-extmod-b-204fba-openshift-default-8dd6555f7-r5zlx

openshift-ingress

multus

e2e-extmod-b-204fba-openshift-default-8dd6555f7-r5zlx

AddedInterface

Add eth0 [10.134.0.231/23] from ovn-kubernetes
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-204fba-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

e2e-extmod-b-204fba-openshift-default-8dd6555f7-7glvz

AddedInterface

Add eth0 [10.132.0.143/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-extmod-b-204fba-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-204fba-openshift-default-8dd6555f7 from 6 to 10

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-vrphk

Started

Container started

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-vrphk

Created

Container created

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-vrphk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-mfjwz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-r5zlx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-mfjwz

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-7glvz

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-7glvz

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-r5zlx

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-7glvz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-extmod-b-204fba-openshift-default-8dd6555f7-mfjwz

AddedInterface

Add eth0 [10.132.0.142/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-sn5sr

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-sn5sr

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-mfjwz

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-r5zlx

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-7glvz

ProbeError

Startup probe error: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-sn5sr

ProbeError

Startup probe error: Get "http://10.132.0.141:15021/healthz/ready": dial tcp 10.132.0.141:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-mfjwz

Unhealthy

Startup probe failed: Get "http://10.132.0.142:15021/healthz/ready": dial tcp 10.132.0.142:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-sn5sr

Unhealthy

Startup probe failed: Get "http://10.132.0.141:15021/healthz/ready": dial tcp 10.132.0.141:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-7glvz

Unhealthy

Startup probe failed: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-r5zlx

ProbeError

Startup probe error: Get "http://10.134.0.231:15021/healthz/ready": dial tcp 10.134.0.231:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-r5zlx

Unhealthy

Startup probe failed: Get "http://10.134.0.231:15021/healthz/ready": dial tcp 10.134.0.231:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-mfjwz

ProbeError

Startup probe error: Get "http://10.132.0.142:15021/healthz/ready": dial tcp 10.132.0.142:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-vrphk

Unhealthy

Startup probe failed: Get "http://10.133.0.106:15021/healthz/ready": dial tcp 10.133.0.106:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-vrphk

ProbeError

Startup probe error: Get "http://10.133.0.106:15021/healthz/ready": dial tcp 10.133.0.106:15021: connect: connection refused body:

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-fmbxp

Killing

Stopping container discovery

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

SuccessfulRescale

New size: 2; reason: All metrics below target

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-fmbxp

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-629129-ddf565c67-76b79

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-629129-9d8b9d665-qclfv

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-qbv6j

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-cvdnx

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-fxn9s

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-wzxjn

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-q7ntb

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-vrphk

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-q7ntb

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-q7ntb

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-qbv6j

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-qbv6j

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-cvdnx

ProbeError

Readiness probe error: Get "http://10.132.0.139:15021/healthz/ready": dial tcp 10.132.0.139:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-cvdnx

Unhealthy

Readiness probe failed: Get "http://10.132.0.139:15021/healthz/ready": dial tcp 10.132.0.139:15021: connect: connection refused

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-vrphk

ProbeError

Readiness probe error: Get "http://10.133.0.106:15021/healthz/ready": dial tcp 10.133.0.106:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-extmod-a-629129-openshift-default-dc77fd7d9-vrphk

Unhealthy

Readiness probe failed: Get "http://10.133.0.106:15021/healthz/ready": dial tcp 10.133.0.106:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-204fba-6786ccc979-d7ct7

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-204fba-558699b45f-np9qr

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-629129-9d8b9d665-qclfv

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-629129-ddf565c67-76b79

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-sn5sr

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-9q2xr

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-2q2rq

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-7glvz

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-qvxr9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-mfjwz

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-r5zlx

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-5msfb

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-75t4b

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-jqhv4

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-75t4b

ProbeError

Readiness probe error: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-9q2xr

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-9q2xr

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-75t4b

Unhealthy

Readiness probe failed: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-5msfb

Unhealthy

Readiness probe failed: Get "http://10.134.0.131:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-extmod-b-204fba-openshift-default-8dd6555f7-5msfb

ProbeError

Readiness probe error: Get "http://10.134.0.131:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

replicaset-controller

e2e-ait-b9ae0feb-openshift-default-558dd897b5

SuccessfulCreate

Created pod: e2e-ait-b9ae0feb-openshift-default-558dd897b5-58kf5

openshift-ingress

gateway_labeler_controller

e2e-ait-b9ae0feb

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-b9ae0feb

openshift-ingress

deployment-controller

e2e-ait-b9ae0feb-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-b9ae0feb-openshift-default-558dd897b5 from 0 to 1

openshift-ingress

multus

e2e-ait-b9ae0feb-openshift-default-558dd897b5-58kf5

AddedInterface

Add eth0 [10.134.0.237/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-b9ae0feb-586d87c8c9

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-b9ae0feb-586d87c8c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-b9ae0feb: serviceaccount "payload-processing-e2e-ait-b9ae0feb" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-b9ae0feb-586d87c8c9

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-b9ae0feb-586d87c8c9-6hwhm

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-b9ae0feb

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-b9ae0feb-586d87c8c9 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-b9ae0feb-fcd47598c

SuccessfulCreate

Created pod: payload-processing-e2e-ait-b9ae0feb-fcd47598c-l25nf
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-b9ae0feb-fcd47598c

FailedCreate

Error creating: pods "payload-processing-e2e-ait-b9ae0feb-fcd47598c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-b9ae0feb: serviceaccount "payload-processing-e2e-ait-b9ae0feb" not found

openshift-ingress

deployment-controller

payload-processing-e2e-ait-b9ae0feb

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-b9ae0feb-fcd47598c from 0 to 1
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-204fba-6786ccc979-d7ct7

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

multus

payload-pre-processing-e2e-ait-b9ae0feb-586d87c8c9-6hwhm

AddedInterface

Add eth0 [10.133.0.112/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-ait-b9ae0feb-fcd47598c-l25nf

AddedInterface

Add eth0 [10.133.0.113/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-b9ae0feb-586d87c8c9-6hwhm

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ait-b9ae0feb-fcd47598c-l25nf

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ait-b9ae0feb-fcd47598c-l25nf

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ait-b9ae0feb-fcd47598c-l25nf

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-b9ae0feb-586d87c8c9-6hwhm

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-b9ae0feb-586d87c8c9-6hwhm

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-204fba-558699b45f-np9qr

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

e2e-ait-b9ae0feb-openshift-default-558dd897b5-58kf5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-ait-b9ae0feb-openshift-default-558dd897b5-58kf5

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ait-b9ae0feb-openshift-default-558dd897b5-58kf5

Created

Container created

openshift-ingress

kubelet

e2e-ait-b9ae0feb-openshift-default-558dd897b5-58kf5

ProbeError

Readiness probe error: Get "http://10.134.0.237:15021/healthz/ready": dial tcp 10.134.0.237:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-ait-b9ae0feb-openshift-default-558dd897b5-58kf5

Unhealthy

Readiness probe failed: Get "http://10.134.0.237:15021/healthz/ready": dial tcp 10.134.0.237:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 7; reason: All metrics below target

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-75fd9f5d47 from 10 to 7

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-vmkcq

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-wdc6m

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-zjvjd

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-wdc6m

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-zjvjd

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-vmkcq
(x5)

openshift-ingress

kubelet

e2e-ait-b9ae0feb-openshift-default-558dd897b5-58kf5

ProbeError

Startup probe error: Get "http://10.134.0.237:15021/healthz/ready": dial tcp 10.134.0.237:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ait-b9ae0feb-openshift-default-558dd897b5-58kf5

Unhealthy

Startup probe failed: Get "http://10.134.0.237:15021/healthz/ready": dial tcp 10.134.0.237:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-ait-b9ae0feb-fcd47598c-l25nf

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-b9ae0feb-586d87c8c9-6hwhm

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-b9ae0feb-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-b9ae0feb-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-ait-b9ae0feb-openshift-default-558dd897b5-58kf5

Killing

Stopping container istio-proxy

opendatahub

replicaset-controller

maas-controller-5b555b5c85

SuccessfulDelete

Deleted pod: maas-controller-5b555b5c85-jv2xg

opendatahub

kubelet

maas-controller-5b555b5c85-jv2xg

Killing

Stopping container manager

opendatahub

replicaset-controller

maas-controller-5b555b5c85

SuccessfulCreate

Created pod: maas-controller-5b555b5c85-wbwdm

opendatahub

kubelet

maas-controller-5b555b5c85-wbwdm

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:f6fe27d5bc148a871e93b1383cae04799a1b4bfc2d1f9858dad18ea42d6f2045"

opendatahub

multus

maas-controller-5b555b5c85-wbwdm

AddedInterface

Add eth0 [10.132.0.147/23] from ovn-kubernetes

opendatahub

kubelet

maas-controller-5b555b5c85-wbwdm

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:f6fe27d5bc148a871e93b1383cae04799a1b4bfc2d1f9858dad18ea42d6f2045" in 608ms (608ms including waiting). Image size: 200094737 bytes.

opendatahub

kubelet

maas-controller-5b555b5c85-wbwdm

Started

Container started

opendatahub

kubelet

maas-controller-5b555b5c85-wbwdm

Created

Container created

openshift-ingress

kubelet

e2e-ait-b9ae0feb-openshift-default-558dd897b5-58kf5

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-ait-b9ae0feb-openshift-default-558dd897b5-58kf5

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-2lqv8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-2lqv8

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-5msqr

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-5msqr

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-4rbgx

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-4rbgx

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-4rbgx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-2lqv8

AddedInterface

Add eth0 [10.132.0.150/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-5msqr

AddedInterface

Add eth0 [10.132.0.148/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-75fd9f5d47-4rbgx

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-75fd9f5d47-2lqv8

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-75fd9f5d47-5msqr

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-4rbgx

AddedInterface

Add eth0 [10.132.0.149/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-2lqv8

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-5msqr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

opendatahub

maas-controller-5b555b5c85-wbwdm_93756bac-b987-4c57-9a25-f91e47f228bd

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-5b555b5c85-wbwdm_93756bac-b987-4c57-9a25-f91e47f228bd became leader
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-4rbgx

ProbeError

Startup probe error: Get "http://10.132.0.149:15021/healthz/ready": dial tcp 10.132.0.149:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-5msqr

ProbeError

Startup probe error: Get "http://10.132.0.148:15021/healthz/ready": dial tcp 10.132.0.148:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-2lqv8

ProbeError

Startup probe error: Get "http://10.132.0.150:15021/healthz/ready": dial tcp 10.132.0.150:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-2lqv8

Unhealthy

Startup probe failed: Get "http://10.132.0.150:15021/healthz/ready": dial tcp 10.132.0.150:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-5msqr

Unhealthy

Startup probe failed: Get "http://10.132.0.148:15021/healthz/ready": dial tcp 10.132.0.148:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-4rbgx

Unhealthy

Startup probe failed: Get "http://10.132.0.149:15021/healthz/ready": dial tcp 10.132.0.149:15021: connect: connection refused
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-5msqr

Unhealthy

Startup probe failed: Get "http://10.132.0.148:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-2lqv8

Unhealthy

Startup probe failed: Get "http://10.132.0.150:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-5msqr

ProbeError

Startup probe error: Get "http://10.132.0.148:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-4rbgx

ProbeError

Startup probe error: Get "http://10.132.0.149:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-4rbgx

Unhealthy

Startup probe failed: Get "http://10.132.0.149:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-2lqv8

ProbeError

Startup probe error: Get "http://10.132.0.150:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled down replica set maas-controller-5b555b5c85 from 1 to 0

opendatahub

kubelet

maas-controller-5b555b5c85-wbwdm

Killing

Stopping container manager

opendatahub

replicaset-controller

maas-controller-5b555b5c85

SuccessfulDelete

Deleted pod: maas-controller-5b555b5c85-wbwdm
(x3)

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled up replica set maas-controller-5b555b5c85 from 0 to 1

opendatahub

replicaset-controller

maas-controller-5b555b5c85

SuccessfulCreate

Created pod: maas-controller-5b555b5c85-sr2mh

opendatahub

kubelet

maas-controller-5b555b5c85-sr2mh

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:f6fe27d5bc148a871e93b1383cae04799a1b4bfc2d1f9858dad18ea42d6f2045"

opendatahub

multus

maas-controller-5b555b5c85-sr2mh

AddedInterface

Add eth0 [10.133.0.114/23] from ovn-kubernetes

opendatahub

kubelet

maas-controller-5b555b5c85-sr2mh

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:f6fe27d5bc148a871e93b1383cae04799a1b4bfc2d1f9858dad18ea42d6f2045" in 1.732s (1.732s including waiting). Image size: 200094737 bytes.

opendatahub

kubelet

maas-controller-5b555b5c85-sr2mh

Created

Container created

opendatahub

kubelet

maas-controller-5b555b5c85-sr2mh

Started

Container started

opendatahub

maas-controller-5b555b5c85-sr2mh_8ba0bc3d-8571-4300-82ae-4345f46c28b9

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-5b555b5c85-sr2mh_8ba0bc3d-8571-4300-82ae-4345f46c28b9 became leader

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-scwch

Killing

Stopping container discovery

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-scwch

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-5msqr

Unhealthy

Readiness probe failed: Get "http://10.132.0.148:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

llm

LLMInferenceServiceController

e2e-prio-pool-4a8104f7

Created

Created v1.Deployment llm/e2e-prio-pool-4a8104f7-kserve-router-scheduler

llm

LLMInferenceServiceController

e2e-prio-pool-4a8104f7

Created

Created v1.Service llm/e2e-prio-pool-4a8104f7-epp-service

llm

LLMInferenceServiceController

e2e-prio-pool-4a8104f7

Created

Created v1.ServiceAccount llm/e2e-prio-pool-4a8104f7-epp-sa

llm

LLMInferenceServiceController

e2e-prio-pool-4a8104f7

Created

Created v1.ClusterRoleBinding /llm-e2e-prio-pool-4a8104f7-epp-auth-rb

llm

LLMInferenceServiceController

e2e-prio-pool-4a8104f7

Created

Created v1.Role llm/e2e-prio-pool-4a8104f7-epp-role

llm

LLMInferenceServiceController

e2e-prio-pool-4a8104f7

Created

Created v1.RoleBinding llm/e2e-prio-pool-4a8104f7-epp-rb

llm

LLMInferenceServiceController

e2e-prio-pool-4a8104f7

Created

Created v1.Service llm/e2e-prio-pool-4a8104f7-kserve-workload-svc

llm

deployment-controller

e2e-prio-pool-4a8104f7-kserve

ScalingReplicaSet

Scaled up replica set e2e-prio-pool-4a8104f7-kserve-c99f876 from 0 to 1

llm

LLMInferenceServiceController

e2e-prio-pool-4a8104f7

Created

Created v1.Secret llm/e2e-prio-pool-4a8104f7-kserve-self-signed-certs

llm

replicaset-controller

e2e-prio-pool-4a8104f7-kserve-router-scheduler-75f8877679

SuccessfulCreate

Created pod: e2e-prio-pool-4a8104f7-kserve-router-scheduler-75f8877679-x2qwr

llm

replicaset-controller

e2e-prio-pool-4a8104f7-kserve-c99f876

SuccessfulCreate

Created pod: e2e-prio-pool-4a8104f7-kserve-c99f876-5jksh

llm

deployment-controller

e2e-prio-pool-4a8104f7-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set e2e-prio-pool-4a8104f7-kserve-router-scheduler-75f8877679 from 0 to 1

llm

LLMInferenceServiceController

e2e-prio-pool-4a8104f7

Created

Created v1.Deployment llm/e2e-prio-pool-4a8104f7-kserve

llm

multus

e2e-prio-pool-4a8104f7-kserve-c99f876-5jksh

AddedInterface

Add eth0 [10.134.0.241/23] from ovn-kubernetes

llm

kubelet

e2e-prio-pool-4a8104f7-kserve-c99f876-5jksh

Created

Container created

llm

kubelet

e2e-prio-pool-4a8104f7-kserve-router-scheduler-75f8877679-x2qwr

Pulling

Pulling image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker@sha256:5fe24a45d9c8039b6db33e8be331dab766610e6eb60f9d2bec1ecc1153ca8827"

llm

kubelet

e2e-prio-pool-4a8104f7-kserve-c99f876-5jksh

Started

Container started

llm

kubelet

e2e-prio-pool-4a8104f7-kserve-c99f876-5jksh

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim@sha256:c3ba435081a4d032676b218ea34eb3a1c54507da0fade2f6297f9c37894fe0d1" already present on machine and can be accessed by the pod

llm

multus

e2e-prio-pool-4a8104f7-kserve-router-scheduler-75f8877679-x2qwr

AddedInterface

Add eth0 [10.133.0.115/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-prio-pool-4a8104f7

Updated

Updated v1.HTTPRoute llm/e2e-prio-pool-4a8104f7-kserve-route
(x9)

llm

LLMInferenceServiceController

e2e-prio-pool-4a8104f7

Created

(combined from similar events): Created v1.DestinationRule llm/e2e-prio-pool-4a8104f7-kserve-shadow-svc

llm

kubelet

e2e-prio-pool-4a8104f7-kserve-router-scheduler-75f8877679-x2qwr

Pulled

Successfully pulled image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker@sha256:5fe24a45d9c8039b6db33e8be331dab766610e6eb60f9d2bec1ecc1153ca8827" in 2.18s (2.18s including waiting). Image size: 184206420 bytes.

llm

kubelet

e2e-prio-pool-4a8104f7-kserve-router-scheduler-75f8877679-x2qwr

Started

Container started

llm

kubelet

e2e-prio-pool-4a8104f7-kserve-router-scheduler-75f8877679-x2qwr

Created

Container created
(x2)

llm

LLMInferenceServiceController

e2e-prio-pool-4a8104f7

Updated

Updated v1.Secret llm/e2e-prio-pool-4a8104f7-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-prio-pool-4a8104f7

LLMInferenceServiceReady

LLMInferenceService [e2e-prio-pool-4a8104f7] is Ready

llm

kubelet

e2e-prio-pool-4a8104f7-kserve-c99f876-5jksh

Killing

Stopping container main

llm

kubelet

e2e-prio-pool-4a8104f7-kserve-router-scheduler-75f8877679-x2qwr

Killing

Stopping container main

llm

LLMInferenceServiceController

e2e-prio-pool-4a8104f7

Deleted

Deleted v1.ClusterRoleBinding /llm-e2e-prio-pool-4a8104f7-epp-auth-rb

openshift-ingress

replicaset-controller

e2e-worker-single-main-dd40e2-openshift-default-5fd56bbf97

SuccessfulCreate

Created pod: e2e-worker-single-main-dd40e2-openshift-default-5fd56bbf97qkv68

openshift-ingress

gateway_labeler_controller

e2e-worker-single-main-dd40e2

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-dd40e2

openshift-ingress

deployment-controller

e2e-worker-single-main-dd40e2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-single-main-dd40e2-openshift-default-5fd56bbf97 from 0 to 1

openshift-ingress

multus

e2e-worker-single-main-dd40e2-openshift-default-5fd56bbf97qkv68

AddedInterface

Add eth0 [10.134.0.244/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-single-main-dd40e2-6d5cd64cd7

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-single-main-dd40e2-6d5cd64cd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-dd40e2: serviceaccount "payload-processing-e2e-worker-single-main-dd40e2" not found

openshift-ingress

deployment-controller

payload-processing-e2e-worker-single-main-dd40e2

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-single-main-dd40e2-d69d5d5d5 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-single-main-dd40e2-d69d5d5d5

FailedCreate

Error creating: pods "payload-processing-e2e-worker-single-main-dd40e2-d69d5d5d5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-dd40e2: serviceaccount "payload-processing-e2e-worker-single-main-dd40e2" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-single-main-dd40e2-d69d5d5d5

SuccessfulCreate

Created pod: payload-processing-e2e-worker-single-main-dd40e2-d69d5d5d59rdvl

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-single-main-dd40e2-6d5cd64cd7

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-single-main-dd40e2-6d5cdm2dt7

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-single-main-dd40e2

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-single-main-dd40e2-6d5cd64cd7 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-dd40e2-6d5cdm2dt7

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-dd40e2-d69d5d5d59rdvl

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-dd40e2-6d5cdm2dt7

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-dd40e2-6d5cdm2dt7

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-worker-single-main-dd40e2-6d5cdm2dt7

AddedInterface

Add eth0 [10.134.0.245/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-dd40e2-d69d5d5d59rdvl

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-dd40e2-d69d5d5d59rdvl

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-worker-single-main-dd40e2-d69d5d5d59rdvl

AddedInterface

Add eth0 [10.134.0.246/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-single-main-dd40e2-openshift-default-5fd56bbf97qkv68

ProbeError

Readiness probe error: Get "http://10.134.0.244:15021/healthz/ready": dial tcp 10.134.0.244:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-single-main-dd40e2-openshift-default-5fd56bbf97qkv68

Unhealthy

Readiness probe failed: Get "http://10.134.0.244:15021/healthz/ready": dial tcp 10.134.0.244:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-worker-single-main-dd40e2-openshift-default-5fd56bbf97qkv68

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-single-main-dd40e2-openshift-default-5fd56bbf97qkv68

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-single-main-dd40e2-openshift-default-5fd56bbf97qkv68

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x6)

openshift-ingress

kubelet

e2e-worker-single-main-dd40e2-openshift-default-5fd56bbf97qkv68

ProbeError

Startup probe error: Get "http://10.134.0.244:15021/healthz/ready": dial tcp 10.134.0.244:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-single-main-dd40e2-openshift-default-5fd56bbf97qkv68

Unhealthy

Startup probe failed: Get "http://10.134.0.244:15021/healthz/ready": dial tcp 10.134.0.244:15021: connect: connection refused
(x5)

llm

kubelet

e2e-prio-pool-4a8104f7-kserve-router-scheduler-75f8877679-x2qwr

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-single-main-dd40e2-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-single-main-dd40e2-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-single-main-dd40e2-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-single-main-dd40e2-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-dd40e2-6d5cdm2dt7

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-dd40e2-d69d5d5d59rdvl

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-worker-single-main-dd40e2-openshift-default-5fd56bbf97qkv68

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-dd40e2-6d5cdm2dt7

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-dd40e2-d69d5d5d59rdvl

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

llm

LLMInferenceServiceController

e2e-same-id-a-bd1362d4

Created

Created v1.Service llm/e2e-same-id-a-bd1362d4-kserve-workload-svc

llm

deployment-controller

e2e-same-id-a-bd1362d4-kserve

ScalingReplicaSet

Scaled up replica set e2e-same-id-a-bd1362d4-kserve-5b987dd75c from 0 to 1

llm

LLMInferenceServiceController

e2e-same-id-a-bd1362d4

Created

Created v1.PodMonitor llm/e2e-same-id-a-bd1362d4-kserve-llmisvc-engine

llm

replicaset-controller

e2e-same-id-a-bd1362d4-kserve-5b987dd75c

SuccessfulCreate

Created pod: e2e-same-id-a-bd1362d4-kserve-5b987dd75c-5fgb8

llm

LLMInferenceServiceController

e2e-same-id-a-bd1362d4

Created

Created v1.Deployment llm/e2e-same-id-a-bd1362d4-kserve

llm

LLMInferenceServiceController

e2e-same-id-a-bd1362d4

Created

Created v1.Secret llm/e2e-same-id-a-bd1362d4-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-same-id-a-bd1362d4

Created

Created v1.NetworkPolicy llm/e2e-same-id-a-bd1362d4-prometheus-scraping

llm

multus

e2e-same-id-a-bd1362d4-kserve-5b987dd75c-5fgb8

AddedInterface

Add eth0 [10.134.0.249/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-same-id-a-bd1362d4

Created

Created v1.HTTPRoute llm/e2e-same-id-a-bd1362d4-kserve-route

llm

LLMInferenceServiceController

e2e-same-id-a-bd1362d4

Created

Created v1.DestinationRule llm/e2e-same-id-a-bd1362d4-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-same-id-a-bd1362d4

Created

Created v1.PodMonitor llm/e2e-same-id-a-bd1362d4-kserve-llmisvc-engine-default

llm

kubelet

e2e-same-id-a-bd1362d4-kserve-5b987dd75c-5fgb8

Created

Container created

llm

kubelet

e2e-same-id-a-bd1362d4-kserve-5b987dd75c-5fgb8

Started

Container started

llm

kubelet

e2e-same-id-a-bd1362d4-kserve-5b987dd75c-5fgb8

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim@sha256:c3ba435081a4d032676b218ea34eb3a1c54507da0fade2f6297f9c37894fe0d1" already present on machine and can be accessed by the pod

llm

LLMInferenceServiceController

e2e-same-id-a-bd1362d4

Updated

Updated v1.Secret llm/e2e-same-id-a-bd1362d4-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-same-id-a-bd1362d4

LLMInferenceServiceReady

LLMInferenceService [e2e-same-id-a-bd1362d4] is Ready

llm

LLMInferenceServiceController

e2e-same-id-b-bd1362d4

Created

Created v1.Service llm/e2e-same-id-b-bd1362d4-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-same-id-b-bd1362d4

Created

Created v1.PodMonitor llm/e2e-same-id-b-bd1362d4-kserve-llmisvc-engine-default

llm

kubelet

e2e-same-id-b-bd1362d4-kserve-799bf7946f-7nrqs

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim@sha256:c3ba435081a4d032676b218ea34eb3a1c54507da0fade2f6297f9c37894fe0d1" already present on machine and can be accessed by the pod

llm

LLMInferenceServiceController

e2e-same-id-b-bd1362d4

Created

Created v1.NetworkPolicy llm/e2e-same-id-b-bd1362d4-prometheus-scraping

llm

kubelet

e2e-same-id-b-bd1362d4-kserve-799bf7946f-7nrqs

Started

Container started

llm

replicaset-controller

e2e-same-id-b-bd1362d4-kserve-799bf7946f

SuccessfulCreate

Created pod: e2e-same-id-b-bd1362d4-kserve-799bf7946f-7nrqs

llm

deployment-controller

e2e-same-id-b-bd1362d4-kserve

ScalingReplicaSet

Scaled up replica set e2e-same-id-b-bd1362d4-kserve-799bf7946f from 0 to 1

llm

LLMInferenceServiceController

e2e-same-id-b-bd1362d4

Created

Created v1.Secret llm/e2e-same-id-b-bd1362d4-kserve-self-signed-certs

llm

multus

e2e-same-id-b-bd1362d4-kserve-799bf7946f-7nrqs

AddedInterface

Add eth0 [10.133.0.116/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-same-id-b-bd1362d4

Created

Created v1.PodMonitor llm/e2e-same-id-b-bd1362d4-kserve-llmisvc-engine

llm

kubelet

e2e-same-id-b-bd1362d4-kserve-799bf7946f-7nrqs

Created

Container created

llm

LLMInferenceServiceController

e2e-same-id-b-bd1362d4

Created

Created v1.HTTPRoute llm/e2e-same-id-b-bd1362d4-kserve-route

llm

LLMInferenceServiceController

e2e-same-id-b-bd1362d4

Created

Created v1.DestinationRule llm/e2e-same-id-b-bd1362d4-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-same-id-b-bd1362d4

Created

Created v1.Deployment llm/e2e-same-id-b-bd1362d4-kserve
(x2)

llm

LLMInferenceServiceController

e2e-same-id-b-bd1362d4

Updated

Updated v1.Secret llm/e2e-same-id-b-bd1362d4-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-same-id-b-bd1362d4

LLMInferenceServiceReady

LLMInferenceService [e2e-same-id-b-bd1362d4] is Ready
(x8)

llm

LLMInferenceServiceController

e2e-same-id-b-bd1362d4

ModelNameCollision

model name "test/e2e-same-model-id" overlaps with e2e-same-id-a-bd1362d4 in namespace llm; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address llm/e2e-same-id-b-bd1362d4; safe to ignore if traffic splitting is planned
(x2)

llm

LLMInferenceServiceController

e2e-same-id-a-bd1362d4

ModelNameCollision

model name "test/e2e-same-model-id" overlaps with e2e-same-id-b-bd1362d4 in namespace llm; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address llm/e2e-same-id-a-bd1362d4; safe to ignore if traffic splitting is planned

llm

kubelet

e2e-same-id-a-bd1362d4-kserve-5b987dd75c-5fgb8

Killing

Stopping container main

llm

kubelet

e2e-same-id-b-bd1362d4-kserve-799bf7946f-7nrqs

Killing

Stopping container main

openshift-ingress

gateway_labeler_controller

e2e-ipp-a-2b5a18

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-2b5a18

openshift-ingress

replicaset-controller

e2e-ipp-a-2b5a18-openshift-default-7d44b947f7

SuccessfulCreate

Created pod: e2e-ipp-a-2b5a18-openshift-default-7d44b947f7-5klp9

openshift-ingress

deployment-controller

e2e-ipp-a-2b5a18-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-a-2b5a18-openshift-default-7d44b947f7 from 0 to 1

openshift-ingress

multus

e2e-ipp-a-2b5a18-openshift-default-7d44b947f7-5klp9

AddedInterface

Add eth0 [10.134.0.253/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-2b5a18-69d476f7dc

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-a-2b5a18-69d476f7dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-2b5a18: serviceaccount "payload-processing-e2e-ipp-a-2b5a18" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-2b5a18-7dbfd65bff

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-a-2b5a18-7dbfd65bff-xxckt
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-2b5a18-7dbfd65bff

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-a-2b5a18-7dbfd65bff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-2b5a18: serviceaccount "payload-processing-e2e-ipp-a-2b5a18" not found

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-a-2b5a18

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-a-2b5a18-69d476f7dc from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-a-2b5a18

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-a-2b5a18-7dbfd65bff from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-2b5a18-69d476f7dc

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-a-2b5a18-69d476f7dc-8pfc8

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-2b5a18-7dbfd65bff-xxckt

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-2b5a18-69d476f7dc-8pfc8

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-2b5a18-7dbfd65bff-xxckt

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-2b5a18-69d476f7dc-8pfc8

Created

Container created

openshift-ingress

multus

payload-processing-e2e-ipp-a-2b5a18-69d476f7dc-8pfc8

AddedInterface

Add eth0 [10.133.0.119/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-2b5a18-69d476f7dc-8pfc8

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-2b5a18-7dbfd65bff-xxckt

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-ipp-a-2b5a18-7dbfd65bff-xxckt

AddedInterface

Add eth0 [10.133.0.118/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-2b5a18-69d476f7dc-8pfc8

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-2b5a18-7dbfd65bff-xxckt

Killing

Stopping container payload-pre-processing
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-2b5a18-85d6575455

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-a-2b5a18-85d6575455-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-2b5a18: serviceaccount "payload-processing-e2e-ipp-a-2b5a18" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-2b5a18-76496b8844

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-a-2b5a18-76496b8844-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-2b5a18: serviceaccount "payload-processing-e2e-ipp-a-2b5a18" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-2b5a18-76496b8844

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-a-2b5a18-76496b8844-xjpx9

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-a-2b5a18

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-a-2b5a18-76496b8844 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-a-2b5a18

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-a-2b5a18-85d6575455 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-2b5a18-85d6575455

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-a-2b5a18-85d6575455-fbmjk

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-2b5a18-85d6575455-fbmjk

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-pre-processing-e2e-ipp-a-2b5a18-85d6575455-fbmjk

AddedInterface

Add eth0 [10.133.0.120/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-2b5a18-76496b8844-xjpx9

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-processing-e2e-ipp-a-2b5a18-76496b8844-xjpx9

AddedInterface

Add eth0 [10.134.0.254/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-2b5a18-85d6575455-fbmjk

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-2b5a18-85d6575455-fbmjk

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-2b5a18-85d6575455-fbmjk

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.49s (2.49s including waiting). Image size: 166620835 bytes.

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-2b5a18-76496b8844-xjpx9

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-2b5a18-76496b8844-xjpx9

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.594s (2.594s including waiting). Image size: 166620835 bytes.

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-2b5a18-76496b8844-xjpx9

Created

Container created

openshift-ingress

kubelet

e2e-ipp-a-2b5a18-openshift-default-7d44b947f7-5klp9

ProbeError

Readiness probe error: Get "http://10.134.0.253:15021/healthz/ready": read tcp 10.134.0.2:33492->10.134.0.253:15021: read: connection reset by peer body:

openshift-ingress

kubelet

e2e-ipp-a-2b5a18-openshift-default-7d44b947f7-5klp9

Unhealthy

Readiness probe failed: Get "http://10.134.0.253:15021/healthz/ready": read tcp 10.134.0.2:33492->10.134.0.253:15021: read: connection reset by peer
(x2)

openshift-ingress

kubelet

e2e-ipp-a-2b5a18-openshift-default-7d44b947f7-5klp9

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-ipp-a-2b5a18-openshift-default-7d44b947f7-5klp9

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ipp-a-2b5a18-openshift-default-7d44b947f7-5klp9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-ipp-a-2b5a18-openshift-default-7d44b947f7-5klp9

ProbeError

Startup probe error: Get "http://10.134.0.253:15021/healthz/ready": dial tcp 10.134.0.253:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ipp-a-2b5a18-openshift-default-7d44b947f7-5klp9

Unhealthy

Startup probe failed: Get "http://10.134.0.253:15021/healthz/ready": dial tcp 10.134.0.253:15021: connect: connection refused
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-a-2b5a18-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-a-2b5a18-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

e2e-ipp-b-836bba-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-b-836bba-openshift-default-887b755d9 from 0 to 1

openshift-ingress

replicaset-controller

e2e-ipp-b-836bba-openshift-default-887b755d9

SuccessfulCreate

Created pod: e2e-ipp-b-836bba-openshift-default-887b755d9-l5rmq

openshift-ingress

gateway_labeler_controller

e2e-ipp-b-836bba

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-836bba

openshift-ingress

multus

e2e-ipp-b-836bba-openshift-default-887b755d9-l5rmq

AddedInterface

Add eth0 [10.134.0.255/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-836bba-6777bf66d7

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-b-836bba-6777bf66d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-836bba: serviceaccount "payload-processing-e2e-ipp-b-836bba" not found

openshift-ingress

multus

payload-pre-processing-e2e-ipp-b-836bba-6777bf66d7-vrfxq

AddedInterface

Add eth0 [10.133.0.123/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-b-836bba

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-b-836bba-6777bf66d7 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-836bba-6777bf66d7

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-b-836bba-6777bf66d7-vrfxq

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-836bba-5577fcf647

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-b-836bba-5577fcf647-9wmfx

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-b-836bba

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-b-836bba-5577fcf647 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-836bba-6777bf66d7-vrfxq

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-836bba-5577fcf647-9wmfx

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-836bba-5577fcf647-9wmfx

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-836bba-6777bf66d7-vrfxq

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-836bba-6777bf66d7-vrfxq

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-ipp-b-836bba-5577fcf647-9wmfx

AddedInterface

Add eth0 [10.133.0.122/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-836bba-5577fcf647

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-b-836bba-5577fcf647-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-836bba: serviceaccount "payload-processing-e2e-ipp-b-836bba" not found

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-836bba-5577fcf647-9wmfx

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-836bba-6777bf66d7-vrfxq

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-836bba-5577fcf647-9wmfx

Killing

Stopping container payload-processing
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-836bba-6bb8db78bc

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-b-836bba-6bb8db78bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-836bba: serviceaccount "payload-processing-e2e-ipp-b-836bba" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-836bba-6bb8db78bc

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-b-836bba-6bb8db78bc-l4vc4

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-b-836bba

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-b-836bba-7b8f49cd68 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-b-836bba

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-b-836bba-6bb8db78bc from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-836bba-7b8f49cd68

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-b-836bba-7b8f49cd68-2q26k
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-836bba-7b8f49cd68

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-b-836bba-7b8f49cd68-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-836bba: serviceaccount "payload-processing-e2e-ipp-b-836bba" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-836bba-6bb8db78bc-l4vc4

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-836bba-7b8f49cd68-2q26k

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-836bba-6bb8db78bc-l4vc4

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-836bba-6bb8db78bc-l4vc4

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-836bba-7b8f49cd68-2q26k

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-ipp-b-836bba-6bb8db78bc-l4vc4

AddedInterface

Add eth0 [10.133.0.124/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-836bba-7b8f49cd68-2q26k

Created

Container created

openshift-ingress

multus

payload-processing-e2e-ipp-b-836bba-7b8f49cd68-2q26k

AddedInterface

Add eth0 [10.134.1.0/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-a-2b5a18-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-a-2b5a18-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

kubelet

e2e-ipp-b-836bba-openshift-default-887b755d9-l5rmq

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ipp-b-836bba-openshift-default-887b755d9-l5rmq

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-ipp-b-836bba-openshift-default-887b755d9-l5rmq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-ipp-b-836bba-openshift-default-887b755d9-l5rmq

ProbeError

Startup probe error: Get "http://10.134.0.255:15021/healthz/ready": dial tcp 10.134.0.255:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ipp-b-836bba-openshift-default-887b755d9-l5rmq

Unhealthy

Startup probe failed: Get "http://10.134.0.255:15021/healthz/ready": dial tcp 10.134.0.255:15021: connect: connection refused
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-b-836bba-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-b-836bba-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-2b5a18-85d6575455-fbmjk

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-2b5a18-76496b8844-xjpx9

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ipp-a-2b5a18-openshift-default-7d44b947f7-5klp9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-836bba-6bb8db78bc-l4vc4

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-836bba-7b8f49cd68-2q26k

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ipp-b-836bba-openshift-default-887b755d9-l5rmq

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-6c45b49d64 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-6c45b49d64

SuccessfulCreate

Created pod: payload-pre-processing-6c45b49d64-2dd4l

openshift-ingress

kubelet

payload-pre-processing-6c45b49d64-2dd4l

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-6c45b49d64-2dd4l

AddedInterface

Add eth0 [10.133.0.125/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-6c45b49d64-2dd4l

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-6c45b49d64-2dd4l

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-j6sq9

Killing

Stopping container payload-pre-processing

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

SuccessfulDelete

Deleted pod: payload-pre-processing-7b98bb678c-j6sq9

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled down replica set payload-pre-processing-7b98bb678c from 1 to 0

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-76459f68c5 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-76459f68c5

SuccessfulCreate

Created pod: payload-processing-76459f68c5-48rr8

openshift-ingress

kubelet

payload-processing-76459f68c5-48rr8

Created

Container created

openshift-ingress

kubelet

payload-processing-76459f68c5-48rr8

Started

Container started

openshift-ingress

multus

payload-processing-76459f68c5-48rr8

AddedInterface

Add eth0 [10.133.0.126/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-76459f68c5-48rr8

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-sbd6p

Killing

Stopping container payload-processing

openshift-ingress

replicaset-controller

payload-processing-55fbbbf9dc

SuccessfulDelete

Deleted pod: payload-processing-55fbbbf9dc-sbd6p

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled down replica set payload-processing-55fbbbf9dc from 1 to 0

openshift-ingress

replicaset-controller

payload-pre-processing-6dbbcf55d8

SuccessfulCreate

Created pod: payload-pre-processing-6dbbcf55d8-n4d68

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-6dbbcf55d8 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-75fb58b5b7

SuccessfulCreate

Created pod: payload-processing-75fb58b5b7-xbsfb

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-75fb58b5b7 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-6dbbcf55d8-n4d68

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-75fb58b5b7-xbsfb

AddedInterface

Add eth0 [10.134.1.2/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-6dbbcf55d8-n4d68

Created

Container created

openshift-ingress

kubelet

payload-processing-75fb58b5b7-xbsfb

Created

Container created

openshift-ingress

kubelet

payload-processing-75fb58b5b7-xbsfb

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-75fb58b5b7-xbsfb

Started

Container started

openshift-ingress

multus

payload-pre-processing-6dbbcf55d8-n4d68

AddedInterface

Add eth0 [10.133.0.127/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-6dbbcf55d8-n4d68

Started

Container started

openshift-ingress

replicaset-controller

payload-pre-processing-6c45b49d64

SuccessfulDelete

Deleted pod: payload-pre-processing-6c45b49d64-2dd4l

openshift-ingress

kubelet

payload-pre-processing-6c45b49d64-2dd4l

Killing

Stopping container payload-pre-processing

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled down replica set payload-pre-processing-6c45b49d64 from 1 to 0

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled down replica set payload-processing-76459f68c5 from 1 to 0

openshift-ingress

kubelet

payload-processing-76459f68c5-48rr8

Killing

Stopping container payload-processing

openshift-ingress

replicaset-controller

payload-processing-76459f68c5

SuccessfulDelete

Deleted pod: payload-processing-76459f68c5-48rr8

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-5bcd7997 from 0 to 1
(x5)

models-as-a-service

replicaset-controller

payload-processing-external-model-5bcd7997

FailedCreate

Error creating: pods "payload-processing-external-model-5bcd7997-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

replicaset-controller

payload-processing-external-model-5bcd7997

SuccessfulCreate

Created pod: payload-processing-external-model-5bcd7997-kt9n9

models-as-a-service

kubelet

payload-processing-external-model-5bcd7997-kt9n9

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-5bcd7997-kt9n9

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

multus

payload-processing-external-model-5bcd7997-kt9n9

AddedInterface

Add eth0 [10.133.0.128/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-5bcd7997-kt9n9

Created

Container created
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-qtkkt

ProbeError

Readiness probe error: Get "http://10.133.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-qtkkt

Unhealthy

Readiness probe failed: Get "http://10.133.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

models-as-a-service

kubelet

payload-processing-external-model-5bcd7997-kt9n9

Killing

Stopping container payload-processing
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-2lqv8

ProbeError

Readiness probe error: Get "http://10.132.0.150:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-2lqv8

Unhealthy

Readiness probe failed: Get "http://10.132.0.150:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-4rbgx

ProbeError

Readiness probe error: Get "http://10.132.0.149:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-5msqr

ProbeError

Readiness probe error: Get "http://10.132.0.148:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-7prvd

Unhealthy

Readiness probe failed: Get "http://10.133.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-7prvd

ProbeError

Readiness probe error: Get "http://10.133.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

gateway_labeler_controller

e2e-worker-single-main-8d45bb

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-8d45bb

openshift-ingress

replicaset-controller

e2e-worker-single-main-8d45bb-openshift-default-5b7f975b9

SuccessfulCreate

Created pod: e2e-worker-single-main-8d45bb-openshift-default-5b7f975b9-mz4c6

openshift-ingress

deployment-controller

e2e-worker-single-main-8d45bb-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-single-main-8d45bb-openshift-default-5b7f975b9 from 0 to 1

openshift-ingress

multus

e2e-worker-single-main-8d45bb-openshift-default-5b7f975b9-mz4c6

AddedInterface

Add eth0 [10.134.1.18/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-worker-single-main-8d45bb

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-single-main-8d45bb-646b57d6b5 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-single-main-8d45bb-6dd77b976c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-single-main-8d45bb-6dd77b976c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-8d45bb: serviceaccount "payload-processing-e2e-worker-single-main-8d45bb" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-single-main-8d45bb

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-single-main-8d45bb-6dd77b976c from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-single-main-8d45bb-6dd77b976c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-single-main-8d45bb-6dd776wgbw

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-8d45bb-6dd776wgbw

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-worker-single-main-8d45bb-6dd776wgbw

AddedInterface

Add eth0 [10.133.0.130/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-single-main-8d45bb-646b57d6b5

SuccessfulCreate

Created pod: payload-processing-e2e-worker-single-main-8d45bb-646b57d6b2gvwf
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-single-main-8d45bb-646b57d6b5

FailedCreate

Error creating: pods "payload-processing-e2e-worker-single-main-8d45bb-646b57d6b5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-8d45bb: serviceaccount "payload-processing-e2e-worker-single-main-8d45bb" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-8d45bb-6dd776wgbw

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-8d45bb-6dd776wgbw

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-8d45bb-646b57d6b2gvwf

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-8d45bb-646b57d6b2gvwf

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-8d45bb-646b57d6b2gvwf

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-worker-single-main-8d45bb-646b57d6b2gvwf

AddedInterface

Add eth0 [10.133.0.131/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-worker-single-main-8d45bb-openshift-default-5b7f975b9-mz4c6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-single-main-8d45bb-openshift-default-5b7f975b9-mz4c6

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-single-main-8d45bb-openshift-default-5b7f975b9-mz4c6

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-worker-single-main-8d45bb-openshift-default-5b7f975b9-mz4c6

Unhealthy

Startup probe failed: Get "http://10.134.1.18:15021/healthz/ready": dial tcp 10.134.1.18:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-single-main-8d45bb-openshift-default-5b7f975b9-mz4c6

ProbeError

Startup probe error: Get "http://10.134.1.18:15021/healthz/ready": dial tcp 10.134.1.18:15021: connect: connection refused body:
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-single-main-8d45bb-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-single-main-8d45bb-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-8d45bb-6dd776wgbw

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-8d45bb-646b57d6b2gvwf

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-8d45bb-6dd776wgbw

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-8d45bb-646b57d6b2gvwf

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

deployment-controller

payload-processing-e2e-worker-single-main-8d45bb

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-single-main-8d45bb-646b57d6b5 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-single-main-8d45bb-646b57d6b5

SuccessfulCreate

Created pod: payload-processing-e2e-worker-single-main-8d45bb-646b57d6bfgg2k

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-single-main-8d45bb-646b57d6b5

FailedCreate

Error creating: pods "payload-processing-e2e-worker-single-main-8d45bb-646b57d6b5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-8d45bb: serviceaccount "payload-processing-e2e-worker-single-main-8d45bb" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-single-main-8d45bb

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-single-main-8d45bb-6dd77b976c from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-single-main-8d45bb-6dd77b976c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-single-main-8d45bb-6dd778t8d6

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-single-main-8d45bb-6dd77b976c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-single-main-8d45bb-6dd77b976c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-8d45bb: serviceaccount "payload-processing-e2e-worker-single-main-8d45bb" not found

openshift-ingress

multus

payload-processing-e2e-worker-single-main-8d45bb-646b57d6bfgg2k

AddedInterface

Add eth0 [10.133.0.133/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-8d45bb-6dd778t8d6

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-8d45bb-6dd778t8d6

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-worker-single-main-8d45bb-6dd778t8d6

AddedInterface

Add eth0 [10.133.0.134/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-8d45bb-6dd778t8d6

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-8d45bb-646b57d6bfgg2k

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-8d45bb-646b57d6bfgg2k

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-8d45bb-646b57d6bfgg2k

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-8d45bb-6dd778t8d6

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-8d45bb-646b57d6bfgg2k

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-worker-single-main-8d45bb-openshift-default-5b7f975b9-mz4c6

Killing

Stopping container istio-proxy

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-678d87766f

SuccessfulDelete

Deleted pod: kuadrant-operator-controller-manager-678d87766f-5njvh

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-5njvh

Killing

Stopping container manager

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled down replica set kuadrant-operator-controller-manager-678d87766f from 1 to 0

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-678d87766f

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-678d87766f-mhvgh
(x2)

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled up replica set kuadrant-operator-controller-manager-678d87766f from 0 to 1

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

ComponentUnhealthy

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.
(x3)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-mhvgh

Pulled

Container image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" already present on machine and can be accessed by the pod

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-mhvgh

Started

Container started

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-mhvgh

Created

Container created

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

NeedsReinstall

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

multus

kuadrant-operator-controller-manager-678d87766f-mhvgh

AddedInterface

Add eth0 [10.133.0.135/23] from ovn-kubernetes
(x2)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.
(x2)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

AllRequirementsMet

all requirements found, attempting install
(x2)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallSucceeded

install strategy completed with no errors

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-mhvgh_5c53b1a7-d035-417b-aabb-760668147bcb

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-678d87766f-mhvgh_5c53b1a7-d035-417b-aabb-760668147bcb became leader
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-24rkw

Unhealthy

Readiness probe failed: Get "http://10.134.0.26:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-24rkw

ProbeError

Readiness probe error: Get "http://10.134.0.26:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-7cnfk

Unhealthy

Readiness probe failed: Get "http://10.134.0.203:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-sn2pt

ProbeError

Readiness probe error: Get "http://10.132.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-7cnfk

ProbeError

Readiness probe error: Get "http://10.134.0.203:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-5qw4z

Unhealthy

Readiness probe failed: Get "http://10.133.0.38:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-5qw4z

ProbeError

Readiness probe error: Get "http://10.133.0.38:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-sn2pt

Unhealthy

Readiness probe failed: Get "http://10.132.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-4rbgx

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-4rbgx

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-75fd9f5d47 from 10 to 4

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 4; reason: All metrics below target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-2lqv8

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-7prvd

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-sn2pt

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-7cnfk

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-5msqr

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-5msqr

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-2lqv8

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-7prvd

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-sn2pt

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-7cnfk

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-5qw4z

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-24rkw

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-75fd9f5d47 from 4 to 1

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-qtkkt

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-24rkw

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-qtkkt

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-5qw4z

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-75fd9f5d47-fgc89

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-75fd9f5d47-2gmnx
(x2)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target
(x2)

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-75fd9f5d47 from 1 to 4

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-2gmnx

AddedInterface

Add eth0 [10.132.0.159/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-2gmnx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-fgc89

AddedInterface

Add eth0 [10.133.0.137/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-9nd6j

AddedInterface

Add eth0 [10.134.1.28/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-9nd6j

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-2gmnx

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-2gmnx

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-fgc89

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-fgc89

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-9nd6j

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-9nd6j

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-fgc89

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-2gmnx

Unhealthy

Startup probe failed: Get "http://10.132.0.159:15021/healthz/ready": dial tcp 10.132.0.159:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-2gmnx

ProbeError

Startup probe error: Get "http://10.132.0.159:15021/healthz/ready": dial tcp 10.132.0.159:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-fgc89

Unhealthy

Startup probe failed: Get "http://10.133.0.137:15021/healthz/ready": dial tcp 10.133.0.137:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-fgc89

ProbeError

Startup probe error: Get "http://10.133.0.137:15021/healthz/ready": dial tcp 10.133.0.137:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-9nd6j

ProbeError

Startup probe error: Get "http://10.134.1.28:15021/healthz/ready": dial tcp 10.134.1.28:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-9nd6j

Unhealthy

Startup probe failed: Get "http://10.134.1.28:15021/healthz/ready": dial tcp 10.134.1.28:15021: connect: connection refused

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-75fd9f5d47 from 4 to 7

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-cmgx5

AddedInterface

Add eth0 [10.132.0.160/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-l4czw

AddedInterface

Add eth0 [10.133.0.138/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 7; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-6cftx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-6cftx

AddedInterface

Add eth0 [10.134.1.29/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-l4czw

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-cmgx5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-cmgx5

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-cmgx5

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-6cftx

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-6cftx

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-l4czw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-l4czw

Created

Container created
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-cmgx5

ProbeError

Startup probe error: Get "http://10.132.0.160:15021/healthz/ready": dial tcp 10.132.0.160:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-l4czw

ProbeError

Startup probe error: Get "http://10.133.0.138:15021/healthz/ready": dial tcp 10.133.0.138:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-cmgx5

Unhealthy

Startup probe failed: Get "http://10.132.0.160:15021/healthz/ready": dial tcp 10.132.0.160:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-l4czw

Unhealthy

Startup probe failed: Get "http://10.133.0.138:15021/healthz/ready": dial tcp 10.133.0.138:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-6cftx

Unhealthy

Startup probe failed: Get "http://10.134.1.29:15021/healthz/ready": dial tcp 10.134.1.29:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-6cftx

ProbeError

Startup probe error: Get "http://10.134.1.29:15021/healthz/ready": dial tcp 10.134.1.29:15021: connect: connection refused body:
(x2)

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-75fd9f5d47 from 7 to 10
(x3)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-75fd9f5d47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-75fd9f5d47-sqhmk

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-sqhmk

AddedInterface

Add eth0 [10.132.0.161/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-d7szj

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-mqzmk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-mqzmk

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-mqzmk

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-d7szj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-mqzmk

AddedInterface

Add eth0 [10.134.1.31/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-d7szj

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-sqhmk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-75fd9f5d47-d7szj

AddedInterface

Add eth0 [10.133.0.139/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-sqhmk

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-sqhmk

Created

Container created
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-mqzmk

Unhealthy

Startup probe failed: Get "http://10.134.1.31:15021/healthz/ready": dial tcp 10.134.1.31:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-mqzmk

ProbeError

Startup probe error: Get "http://10.134.1.31:15021/healthz/ready": dial tcp 10.134.1.31:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-sqhmk

ProbeError

Startup probe error: Get "http://10.132.0.161:15021/healthz/ready": dial tcp 10.132.0.161:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-sqhmk

Unhealthy

Startup probe failed: Get "http://10.132.0.161:15021/healthz/ready": dial tcp 10.132.0.161:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-d7szj

ProbeError

Startup probe error: Get "http://10.133.0.139:15021/healthz/ready": dial tcp 10.133.0.139:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-d7szj

Unhealthy

Startup probe failed: Get "http://10.133.0.139:15021/healthz/ready": dial tcp 10.133.0.139:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-mt-ce5e7da4-openshift-default-5fb74bb54f

SuccessfulCreate

Created pod: e2e-mt-ce5e7da4-openshift-default-5fb74bb54f-czsxf

openshift-ingress

deployment-controller

e2e-mt-ce5e7da4-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-ce5e7da4-openshift-default-5fb74bb54f from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-mt-ce5e7da4

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-ce5e7da4

openshift-ingress

multus

e2e-mt-ce5e7da4-openshift-default-5fb74bb54f-czsxf

AddedInterface

Add eth0 [10.134.1.32/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-ce5e7da4-7749878675

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-ce5e7da4-7749878675-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-ce5e7da4: serviceaccount "payload-processing-e2e-mt-ce5e7da4" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-ce5e7da4-6687db7995

SuccessfulCreate

Created pod: payload-processing-e2e-mt-ce5e7da4-6687db7995-l2tvm

openshift-ingress

deployment-controller

payload-processing-e2e-mt-ce5e7da4

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-ce5e7da4-6687db7995 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-ce5e7da4-6687db7995

FailedCreate

Error creating: pods "payload-processing-e2e-mt-ce5e7da4-6687db7995-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-ce5e7da4: serviceaccount "payload-processing-e2e-mt-ce5e7da4" not found

openshift-ingress

kubelet

payload-processing-e2e-mt-ce5e7da4-6687db7995-l2tvm

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-mt-ce5e7da4-6687db7995-l2tvm

Created

Container created

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-ce5e7da4

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-ce5e7da4-7749878675 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-ce5e7da4-7749878675-xxb2t

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-ce5e7da4-7749878675-xxb2t

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-ce5e7da4-7749878675-xxb2t

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-mt-ce5e7da4-7749878675-xxb2t

AddedInterface

Add eth0 [10.133.0.141/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-ce5e7da4-6687db7995-l2tvm

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-mt-ce5e7da4-6687db7995-l2tvm

AddedInterface

Add eth0 [10.133.0.142/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-ce5e7da4-7749878675

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-ce5e7da4-7749878675-xxb2t
(x2)

openshift-ingress

kubelet

e2e-mt-ce5e7da4-openshift-default-5fb74bb54f-czsxf

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-mt-ce5e7da4-openshift-default-5fb74bb54f-czsxf

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-mt-ce5e7da4-openshift-default-5fb74bb54f-czsxf

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-mt-ce5e7da4-openshift-default-5fb74bb54f-czsxf

Unhealthy

Startup probe failed: Get "http://10.134.1.32:15021/healthz/ready": dial tcp 10.134.1.32:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-ce5e7da4-openshift-default-5fb74bb54f-czsxf

ProbeError

Startup probe error: Get "http://10.134.1.32:15021/healthz/ready": dial tcp 10.134.1.32:15021: connect: connection refused body:
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-ce5e7da4-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-ce5e7da4-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-processing-e2e-mt-ce5e7da4-6687db7995-l2tvm

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-ce5e7da4-7749878675-xxb2t

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-mt-ce5e7da4-openshift-default-5fb74bb54f-czsxf

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-ce5e7da4-openshift-default-5fb74bb54f-czsxf

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-mt-ce5e7da4-openshift-default-5fb74bb54f-czsxf

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-mqzmk

ProbeError

Readiness probe error: Get "http://10.134.1.31:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-mqzmk

Unhealthy

Readiness probe failed: Get "http://10.134.1.31:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-cmgx5

ProbeError

Readiness probe error: Get "http://10.132.0.160:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-2gmnx

ProbeError

Readiness probe error: Get "http://10.132.0.159:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-cmgx5

Unhealthy

Readiness probe failed: Get "http://10.132.0.160:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-6cftx

Unhealthy

Readiness probe failed: Get "http://10.134.1.29:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-9nd6j

ProbeError

Readiness probe error: Get "http://10.134.1.28:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-9nd6j

Unhealthy

Readiness probe failed: Get "http://10.134.1.28:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-6cftx

ProbeError

Readiness probe error: Get "http://10.134.1.29:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-d7szj

ProbeError

Readiness probe error: Get "http://10.133.0.139:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-2gmnx

Unhealthy

Readiness probe failed: Get "http://10.132.0.159:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-d7szj

Unhealthy

Readiness probe failed: Get "http://10.133.0.139:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-sqhmk

ProbeError

Readiness probe error: Get "http://10.132.0.161:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-sqhmk

Unhealthy

Readiness probe failed: Get "http://10.132.0.161:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-l4czw

ProbeError

Readiness probe error: Get "http://10.133.0.138:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-fgc89

Unhealthy

Readiness probe failed: Get "http://10.133.0.137:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-fgc89

ProbeError

Readiness probe error: Get "http://10.133.0.137:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-l4czw

Unhealthy

Readiness probe failed: Get "http://10.133.0.138:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-qwf92

Unhealthy

Readiness probe failed: Get "http://10.134.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-75fd9f5d47-qwf92

ProbeError

Readiness probe error: Get "http://10.134.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body: