=== events in openshift-ingress (2026-11-03T23:25:56Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 50m Normal Scheduled pod/payload-processing-e2e-api-b-bbfc2e-6ddb5f6584-hc4kx Successfully assigned openshift-ingress/payload-processing-e2e-api-b-bbfc2e-6ddb5f6584-hc4kx to ip-10-0-139-73.ec2.internal 8m24s Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-22c413-748b6tg578 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-22c413-748b6tg578 to ip-10-0-128-81.ec2.internal 24m Normal Scheduled pod/e2e-ipp-b-162d7b-openshift-default-5d877b48fc-l8prt Successfully assigned openshift-ingress/e2e-ipp-b-162d7b-openshift-default-5d877b48fc-l8prt to ip-10-0-133-193.ec2.internal 63m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-50d738-5b4664c9c9-zldwf Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-50d738-5b4664c9c9-zldwf to ip-10-0-133-193.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-a-15e7a7-5f4f86db5d-mnf4m Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-15e7a7-5f4f86db5d-mnf4m to ip-10-0-133-193.ec2.internal 27m Normal Scheduled pod/payload-processing-e2e-ipp-a-15e7a7-79b4b8dbd4-pb92m Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-15e7a7-79b4b8dbd4-pb92m to ip-10-0-139-73.ec2.internal 28m Normal Scheduled pod/payload-processing-e2e-ipp-a-3920cb-57cdc6f756-nl8c2 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-3920cb-57cdc6f756-nl8c2 to ip-10-0-133-193.ec2.internal 29m Normal Scheduled pod/payload-processing-e2e-ipp-a-3920cb-675b9c8b6b-f2xtg Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-3920cb-675b9c8b6b-f2xtg to ip-10-0-139-73.ec2.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-a-4ad693-589cf5d8f6-lgfqh Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-4ad693-589cf5d8f6-lgfqh to ip-10-0-139-73.ec2.internal 57m Normal Scheduled pod/e2e-ait-1b279244-openshift-default-74dd88dd99-6px88 Successfully assigned openshift-ingress/e2e-ait-1b279244-openshift-default-74dd88dd99-6px88 to ip-10-0-128-81.ec2.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-9e540d-67db6b6b55-f687b Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-9e540d-67db6b6b55-f687b to ip-10-0-133-193.ec2.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-7940ea-6c48467979-8rjd5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-7940ea-6c48467979-8rjd5 to ip-10-0-133-193.ec2.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-b1ec3a-6fdb57959f-fxmdx Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-b1ec3a-6fdb57959f-fxmdx to ip-10-0-133-193.ec2.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-50b9b5-76b76f77c9-g6sth Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-50b9b5-76b76f77c9-g6sth to ip-10-0-133-193.ec2.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-4b5e2b-7dd898677f-6768h Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-4b5e2b-7dd898677f-6768h to ip-10-0-133-193.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-341c51-76d98b8f6c-cqm9p Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-341c51-76d98b8f6c-cqm9p to ip-10-0-139-73.ec2.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-derive-cbe6e297-848c8d947f-6rcr7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-cbe6e297-848c8d947f-6rcr7 to ip-10-0-139-73.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-bb1fa5-7948594b67-45z4r Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-bb1fa5-7948594b67-45z4r to ip-10-0-133-193.ec2.internal 4m45s Normal Scheduled pod/payload-pre-processing-e2e-authcfg-http-02e08d-6f4945bb79-sxdq6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-authcfg-http-02e08d-6f4945bb79-sxdq6 to ip-10-0-133-193.ec2.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-api-b-bbfc2e-78f95f4d55-5tprx Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-bbfc2e-78f95f4d55-5tprx to ip-10-0-133-193.ec2.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-api-b-b826b6-86b86bbbff-rkg2w Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-b826b6-86b86bbbff-rkg2w to ip-10-0-133-193.ec2.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-api-b-5dd3b1-7bdc56b4f5-cxmt6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-5dd3b1-7bdc56b4f5-cxmt6 to ip-10-0-133-193.ec2.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-api-b-22edc1-5bb4668bc-59h47 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-22edc1-5bb4668bc-59h47 to ip-10-0-133-193.ec2.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-api-a-65b0e2-5fc849b4cf-m2md4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-65b0e2-5fc849b4cf-m2md4 to ip-10-0-139-73.ec2.internal 56m Normal Scheduled pod/e2e-ait-51bfa2e1-openshift-default-6dcb64d7dc-7tnb9 Successfully assigned openshift-ingress/e2e-ait-51bfa2e1-openshift-default-6dcb64d7dc-7tnb9 to ip-10-0-128-81.ec2.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-api-a-4c5345-5548f954c-kk85c Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-4c5345-5548f954c-kk85c to ip-10-0-139-73.ec2.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-api-a-32fdf2-6767d498c5-np8rq Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-32fdf2-6767d498c5-np8rq to ip-10-0-133-193.ec2.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-api-a-2c0223-5954dfbb87-n5vnv Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-2c0223-5954dfbb87-n5vnv to ip-10-0-133-193.ec2.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-ait-d05919ef-5fbd9c9b67-rb26m Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-d05919ef-5fbd9c9b67-rb26m to ip-10-0-139-73.ec2.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-0a7065-7bc48cbbf9-psdjd Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-0a7065-7bc48cbbf9-psdjd to ip-10-0-128-81.ec2.internal 5m22s Normal Scheduled pod/payload-pre-processing-e2e-ait-a6ecd9bf-8bb8c667-vfwxn Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-a6ecd9bf-8bb8c667-vfwxn to ip-10-0-133-193.ec2.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-ait-51bfa2e1-559ff7c87f-ckt8j Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-51bfa2e1-559ff7c87f-ckt8j to ip-10-0-128-81.ec2.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-ait-1b279244-55d699df5f-8hq7t Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-1b279244-55d699df5f-8hq7t to ip-10-0-133-193.ec2.internal 24m Normal Scheduled pod/payload-processing-e2e-ipp-a-4ad693-5cb9c56db6-kvcc7 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-4ad693-5cb9c56db6-kvcc7 to ip-10-0-133-193.ec2.internal 28m Normal Scheduled pod/payload-processing-e2e-ipp-a-68410c-55c49569d6-rvsbr Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-68410c-55c49569d6-rvsbr to ip-10-0-133-193.ec2.internal 30m Normal Scheduled pod/payload-processing-e2e-ipp-a-68410c-7f86bfb976-d9z7w Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-68410c-7f86bfb976-d9z7w to ip-10-0-133-193.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-a-a4b994-65b8bd9f54-jszn5 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-a4b994-65b8bd9f54-jszn5 to ip-10-0-133-193.ec2.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-a-a4b994-696c454488-hc66w Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-a4b994-696c454488-hc66w to ip-10-0-128-81.ec2.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-b-162d7b-699f7476d4-tdvfk Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-162d7b-699f7476d4-tdvfk to ip-10-0-139-73.ec2.internal 5m27s Normal Scheduled pod/e2e-ait-a6ecd9bf-openshift-default-8f5fcfb57-8b7z2 Successfully assigned openshift-ingress/e2e-ait-a6ecd9bf-openshift-default-8f5fcfb57-8b7z2 to ip-10-0-133-193.ec2.internal 24m Normal Scheduled pod/payload-processing-e2e-ipp-b-162d7b-d6c56f5db-nggjh Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-162d7b-d6c56f5db-nggjh to ip-10-0-133-193.ec2.internal 25m Normal Scheduled pod/e2e-ipp-a-a4b994-openshift-default-556f895f89-lpg4l Successfully assigned openshift-ingress/e2e-ipp-a-a4b994-openshift-default-556f895f89-lpg4l to ip-10-0-128-81.ec2.internal 26m Normal Scheduled pod/payload-processing-e2e-ipp-b-4a01f9-57fb584d-fn5pv Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-4a01f9-57fb584d-fn5pv to ip-10-0-139-73.ec2.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-b-4a01f9-f7d5dd8f4-kzgmd Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-4a01f9-f7d5dd8f4-kzgmd to ip-10-0-139-73.ec2.internal 27m Normal Scheduled pod/payload-processing-e2e-ipp-b-f48f46-7c74fffd4b-b86lk Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-f48f46-7c74fffd4b-b86lk to ip-10-0-139-73.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-b-f48f46-7f5988976-2zk5f Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-f48f46-7f5988976-2zk5f to ip-10-0-133-193.ec2.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-b-f6dcc0-6b5944664-gzcth Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-f6dcc0-6b5944664-gzcth to ip-10-0-139-73.ec2.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-b-f6dcc0-85d8fff6db-tvftq Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-f6dcc0-85d8fff6db-tvftq to ip-10-0-139-73.ec2.internal 26m Normal Scheduled pod/payload-processing-e2e-ipp-b-ff80c0-55ddc664f8-xc54s Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-ff80c0-55ddc664f8-xc54s to ip-10-0-133-193.ec2.internal 28m Normal Scheduled pod/payload-processing-e2e-ipp-b-ff80c0-6794988c8d-w644b Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-ff80c0-6794988c8d-w644b to ip-10-0-139-73.ec2.internal 57m Normal Scheduled pod/payload-processing-e2e-migrate-b7b10505-5d8bb99fb-q2vm4 Successfully assigned openshift-ingress/payload-processing-e2e-migrate-b7b10505-5d8bb99fb-q2vm4 to ip-10-0-128-81.ec2.internal 12m Normal Scheduled pod/e2e-worker-single-main-d5590b-openshift-default-7bd898b6c7rlgk4 Successfully assigned openshift-ingress/e2e-worker-single-main-d5590b-openshift-default-7bd898b6c7rlgk4 to ip-10-0-133-193.ec2.internal 30m Normal Scheduled pod/e2e-ipp-a-68410c-openshift-default-7848d9f579-cqpd4 Successfully assigned openshift-ingress/e2e-ipp-a-68410c-openshift-default-7848d9f579-cqpd4 to ip-10-0-128-81.ec2.internal 63m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-5187e1-7968f4cdbc-klfgh Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-5187e1-7968f4cdbc-klfgh to ip-10-0-133-193.ec2.internal 56m Normal Scheduled pod/e2e-ait-d05919ef-openshift-default-7597d49499-9f7fq Successfully assigned openshift-ingress/e2e-ait-d05919ef-openshift-default-7597d49499-9f7fq to ip-10-0-139-73.ec2.internal 56m Normal Scheduled pod/payload-processing-e2e-mt-53879c92-6bf559b69b-plpwn Successfully assigned openshift-ingress/payload-processing-e2e-mt-53879c92-6bf559b69b-plpwn to ip-10-0-128-81.ec2.internal 64m Normal Scheduled pod/payload-pre-processing-7b98bb678c-wnlt4 Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-wnlt4 to ip-10-0-133-193.ec2.internal 63m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-553753-b48c58c59-ntcj2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-553753-b48c58c59-ntcj2 to ip-10-0-133-193.ec2.internal 49m Normal Scheduled pod/payload-processing-e2e-mt-8b1efeb6-5d7dd8ff94-5blct Successfully assigned openshift-ingress/payload-processing-e2e-mt-8b1efeb6-5d7dd8ff94-5blct to ip-10-0-133-193.ec2.internal 59m Normal Scheduled pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-8lzn6 Successfully assigned openshift-ingress/e2e-worker-w0-88a4d9-openshift-default-857757469c-8lzn6 to ip-10-0-133-193.ec2.internal 63m Normal Scheduled pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-dcb6t Successfully assigned openshift-ingress/e2e-worker-w0-88a4d9-openshift-default-857757469c-dcb6t to ip-10-0-128-81.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-mt-8b1efeb6-5d7dd8ff94-q446n Successfully assigned openshift-ingress/payload-processing-e2e-mt-8b1efeb6-5d7dd8ff94-q446n to ip-10-0-139-73.ec2.internal 22m Normal Scheduled pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-wkk84 Successfully assigned openshift-ingress/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-wkk84 to ip-10-0-133-193.ec2.internal 56m Normal Scheduled pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-g4fx2 Successfully assigned openshift-ingress/e2e-worker-w0-88a4d9-openshift-default-857757469c-g4fx2 to ip-10-0-128-81.ec2.internal 56m Normal Scheduled pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-hdk26 Successfully assigned openshift-ingress/e2e-worker-w0-88a4d9-openshift-default-857757469c-hdk26 to ip-10-0-133-193.ec2.internal 58m Normal Scheduled pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-nqgqh Successfully assigned openshift-ingress/e2e-worker-w0-88a4d9-openshift-default-857757469c-nqgqh to ip-10-0-133-193.ec2.internal 54m Normal Scheduled pod/payload-processing-e2e-shared-a-w0-0847aa-fc76d97d-m54ng Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-0847aa-fc76d97d-m54ng to ip-10-0-128-81.ec2.internal 50m Normal Scheduled pod/e2e-api-a-2c0223-openshift-default-6958d556b9-dwtg9 Successfully assigned openshift-ingress/e2e-api-a-2c0223-openshift-default-6958d556b9-dwtg9 to ip-10-0-128-81.ec2.internal 21m Normal Scheduled pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-q767t Successfully assigned openshift-ingress/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-q767t to ip-10-0-128-81.ec2.internal 69m Normal Scheduled pod/openshift-ai-inference-openshift-ai-inference-668d64f647-fg9gs Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-668d64f647-fg9gs to ip-10-0-128-81.ec2.internal 61m Normal Scheduled pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-shz2b Successfully assigned openshift-ingress/e2e-worker-w0-88a4d9-openshift-default-857757469c-shz2b to ip-10-0-139-73.ec2.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-124490-7cbff46899-7c5p6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-124490-7cbff46899-7c5p6 to ip-10-0-139-73.ec2.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-2d6076-dc5c6ff87-4d8d6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-2d6076-dc5c6ff87-4d8d6 to ip-10-0-133-193.ec2.internal 63m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-434ce3-7b5c6dc5b7-pgcvw Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-434ce3-7b5c6dc5b7-pgcvw to ip-10-0-133-193.ec2.internal 22m Normal Scheduled pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-m27fm Successfully assigned openshift-ingress/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-m27fm to ip-10-0-139-73.ec2.internal 38m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-d4b6bb-75d4df64b8-nvlff Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-d4b6bb-75d4df64b8-nvlff to ip-10-0-139-73.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-shared-a-w3-23d281-97b8bbd66-7dgnl Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-23d281-97b8bbd66-7dgnl to ip-10-0-139-73.ec2.internal 59m Normal Scheduled pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-sjh6w Successfully assigned openshift-ingress/e2e-worker-w0-88a4d9-openshift-default-857757469c-sjh6w to ip-10-0-128-81.ec2.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-71a7df-fdd88b4b7-sdbdj Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-71a7df-fdd88b4b7-sdbdj to ip-10-0-139-73.ec2.internal 47m Normal Scheduled pod/e2e-mt-8b1efeb6-openshift-default-5cf66d64d5-2gjrn Successfully assigned openshift-ingress/e2e-mt-8b1efeb6-openshift-default-5cf66d64d5-2gjrn to ip-10-0-128-81.ec2.internal 57m Normal Scheduled pod/e2e-migrate-b7b10505-gw-openshift-default-5778588977-67psd Successfully assigned openshift-ingress/e2e-migrate-b7b10505-gw-openshift-default-5778588977-67psd to ip-10-0-128-81.ec2.internal 56m Normal Scheduled pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-vlq4m Successfully assigned openshift-ingress/e2e-worker-w0-88a4d9-openshift-default-857757469c-vlq4m to ip-10-0-128-81.ec2.internal 51m Normal Scheduled pod/e2e-api-a-32fdf2-openshift-default-798db88bbc-52z75 Successfully assigned openshift-ingress/e2e-api-a-32fdf2-openshift-default-798db88bbc-52z75 to ip-10-0-128-81.ec2.internal 22m Normal Scheduled pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-l7fhj Successfully assigned openshift-ingress/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-l7fhj to ip-10-0-139-73.ec2.internal 58m Normal Scheduled pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-x96rf Successfully assigned openshift-ingress/e2e-worker-w0-88a4d9-openshift-default-857757469c-x96rf to ip-10-0-139-73.ec2.internal 56m Normal Scheduled pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-ztnxm Successfully assigned openshift-ingress/e2e-worker-w0-88a4d9-openshift-default-857757469c-ztnxm to ip-10-0-139-73.ec2.internal 54m Normal Scheduled pod/payload-processing-e2e-shared-a-w4-281732-645dfbdbf4-27m7g Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-281732-645dfbdbf4-27m7g to ip-10-0-133-193.ec2.internal 21m Normal Scheduled pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-j6vnf Successfully assigned openshift-ingress/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-j6vnf to ip-10-0-128-81.ec2.internal 35m Normal Scheduled pod/payload-processing-e2e-shared-a-w5-b023a2-f5bd44854-vjwb2 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w5-b023a2-f5bd44854-vjwb2 to ip-10-0-139-73.ec2.internal 52m Normal Scheduled pod/payload-processing-e2e-shared-a-w6-2a2eca-6cf9b6f4bd-9mxw7 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w6-2a2eca-6cf9b6f4bd-9mxw7 to ip-10-0-139-73.ec2.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-963936-6c6699ff5-mktm9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-963936-6c6699ff5-mktm9 to ip-10-0-133-193.ec2.internal 43m Normal Scheduled pod/payload-processing-e2e-inf-b-ccdb60-7fc6f896b-qrxhs Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-ccdb60-7fc6f896b-qrxhs to ip-10-0-133-193.ec2.internal 21m Normal Scheduled pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-h5g7f Successfully assigned openshift-ingress/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-h5g7f to ip-10-0-139-73.ec2.internal 56m Normal Scheduled pod/e2e-mt-53879c92-openshift-default-67dfffdb77-c47vg Successfully assigned openshift-ingress/e2e-mt-53879c92-openshift-default-67dfffdb77-c47vg to ip-10-0-139-73.ec2.internal 63m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-e19850-5898685ff-kd5l5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-e19850-5898685ff-kd5l5 to ip-10-0-128-81.ec2.internal 63m Normal Scheduled pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-4kq96 Successfully assigned openshift-ingress/e2e-worker-w1-e19850-openshift-default-9f855bd47-4kq96 to ip-10-0-139-73.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-shared-b-w0-a940d5-64d8f95f5b-l6fqz Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-a940d5-64d8f95f5b-l6fqz to ip-10-0-133-193.ec2.internal 64m Normal Scheduled pod/maas-default-gateway-openshift-default-7df854bf67-zqt58 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-zqt58 to ip-10-0-128-81.ec2.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-ccdb60-779c4b9f95-rrp8q Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-ccdb60-779c4b9f95-rrp8q to ip-10-0-139-73.ec2.internal 52m Normal Scheduled pod/e2e-api-a-4c5345-openshift-default-86899fff89-l8tzb Successfully assigned openshift-ingress/e2e-api-a-4c5345-openshift-default-86899fff89-l8tzb to ip-10-0-139-73.ec2.internal 16m Normal Scheduled pod/maas-default-gateway-openshift-default-7df854bf67-zfx74 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-zfx74 to ip-10-0-139-73.ec2.internal 53m Normal Scheduled pod/e2e-mt-8b1efeb6-openshift-default-5cf66d64d5-z8hk7 Successfully assigned openshift-ingress/e2e-mt-8b1efeb6-openshift-default-5cf66d64d5-z8hk7 to ip-10-0-133-193.ec2.internal 53m Normal Scheduled pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-2kdhp Successfully assigned openshift-ingress/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-2kdhp to ip-10-0-139-73.ec2.internal 21m Normal Scheduled pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-2bb6s Successfully assigned openshift-ingress/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-2bb6s to ip-10-0-133-193.ec2.internal 24m Normal Scheduled pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-224cq Successfully assigned openshift-ingress/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-224cq to ip-10-0-128-81.ec2.internal 36m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-4f9bb0-58c4b77cfb-cdgpk Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-4f9bb0-58c4b77cfb-cdgpk to ip-10-0-139-73.ec2.internal 52m Normal Scheduled pod/payload-processing-e2e-shared-b-w3-d8a3d1-85f887b8c4-gfs5q Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-d8a3d1-85f887b8c4-gfs5q to ip-10-0-139-73.ec2.internal 58m Normal Scheduled pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-7pkkv Successfully assigned openshift-ingress/e2e-worker-w1-e19850-openshift-default-9f855bd47-7pkkv to ip-10-0-139-73.ec2.internal 61m Normal Scheduled pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-84rpc Successfully assigned openshift-ingress/e2e-worker-w1-e19850-openshift-default-9f855bd47-84rpc to ip-10-0-128-81.ec2.internal 64m Normal Scheduled pod/maas-default-gateway-openshift-default-7df854bf67-xt8wx Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-xt8wx to ip-10-0-133-193.ec2.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-fe7ca0-54c86d5ff5-xqxp7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-fe7ca0-54c86d5ff5-xqxp7 to ip-10-0-139-73.ec2.internal 17m Normal Scheduled pod/maas-default-gateway-openshift-default-7df854bf67-w85lm Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-w85lm to ip-10-0-139-73.ec2.internal 54m Normal Scheduled pod/payload-processing-e2e-shared-b-w4-04988c-97c55b5f4-2jhm7 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-04988c-97c55b5f4-2jhm7 to ip-10-0-133-193.ec2.internal 58m Normal Scheduled pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-94n6d Successfully assigned openshift-ingress/e2e-worker-w1-e19850-openshift-default-9f855bd47-94n6d to ip-10-0-128-81.ec2.internal 50m Normal Scheduled pod/e2e-api-a-65b0e2-openshift-default-56d854d69c-tdh5n Successfully assigned openshift-ingress/e2e-api-a-65b0e2-openshift-default-56d854d69c-tdh5n to ip-10-0-128-81.ec2.internal 57m Normal Scheduled pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-f72v5 Successfully assigned openshift-ingress/e2e-worker-w1-e19850-openshift-default-9f855bd47-f72v5 to ip-10-0-128-81.ec2.internal 34m Normal Scheduled pod/payload-processing-e2e-shared-b-w5-9b841d-5c95644856-295pk Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w5-9b841d-5c95644856-295pk to ip-10-0-139-73.ec2.internal 51m Normal Scheduled pod/payload-processing-e2e-shared-b-w6-26f8ba-6f7bb8fcbd-mkqzk Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w6-26f8ba-6f7bb8fcbd-mkqzk to ip-10-0-133-193.ec2.internal 30m Normal Scheduled pod/e2e-ipp-a-3920cb-openshift-default-545b848d77-vkcv4 Successfully assigned openshift-ingress/e2e-ipp-a-3920cb-openshift-default-545b848d77-vkcv4 to ip-10-0-133-193.ec2.internal 58m Normal Scheduled pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-gwqld Successfully assigned openshift-ingress/e2e-worker-w1-e19850-openshift-default-9f855bd47-gwqld to ip-10-0-133-193.ec2.internal 57m Normal Scheduled pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-hz2jq Successfully assigned openshift-ingress/e2e-worker-w1-e19850-openshift-default-9f855bd47-hz2jq to ip-10-0-133-193.ec2.internal 69m Normal Scheduled pod/maas-default-gateway-openshift-default-7df854bf67-v828p Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-v828p to ip-10-0-128-81.ec2.internal 58m Normal Scheduled pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-q8mkq Successfully assigned openshift-ingress/e2e-worker-w1-e19850-openshift-default-9f855bd47-q8mkq to ip-10-0-133-193.ec2.internal 57m Normal Scheduled pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-thldf Successfully assigned openshift-ingress/e2e-worker-w1-e19850-openshift-default-9f855bd47-thldf to ip-10-0-139-73.ec2.internal 57m Normal Scheduled pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-vm8xs Successfully assigned openshift-ingress/e2e-worker-w1-e19850-openshift-default-9f855bd47-vm8xs to ip-10-0-128-81.ec2.internal 23m Normal Scheduled pod/e2e-ipp-a-3920cb-openshift-default-545b848d77-5x4gg Successfully assigned openshift-ingress/e2e-ipp-a-3920cb-openshift-default-545b848d77-5x4gg to ip-10-0-128-81.ec2.internal 8m24s Normal Scheduled pod/payload-processing-e2e-worker-single-main-22c413-578cb96cchrmtl Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-22c413-578cb96cchrmtl to ip-10-0-133-193.ec2.internal 12m Normal Scheduled pod/payload-processing-e2e-worker-single-main-d5590b-75b6cb5f9ll2ss Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-d5590b-75b6cb5f9ll2ss to ip-10-0-128-81.ec2.internal 64m Normal Scheduled pod/maas-default-gateway-openshift-default-7df854bf67-r4qjb Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-r4qjb to ip-10-0-139-73.ec2.internal 48m Normal Scheduled pod/e2e-api-b-22edc1-openshift-default-b9f7f8cb5-qvnsr Successfully assigned openshift-ingress/e2e-api-b-22edc1-openshift-default-b9f7f8cb5-qvnsr to ip-10-0-128-81.ec2.internal 63m Normal Scheduled pod/payload-processing-e2e-worker-w0-88a4d9-76c4574f8b-g8sjs Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-88a4d9-76c4574f8b-g8sjs to ip-10-0-128-81.ec2.internal 63m Normal Scheduled pod/payload-processing-e2e-worker-w1-e19850-565c99cd8d-qhx44 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-e19850-565c99cd8d-qhx44 to ip-10-0-133-193.ec2.internal 27m Normal Scheduled pod/e2e-ipp-a-15e7a7-openshift-default-7446d84559-dxsld Successfully assigned openshift-ingress/e2e-ipp-a-15e7a7-openshift-default-7446d84559-dxsld to ip-10-0-133-193.ec2.internal 63m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-88a4d9-797cb8d49f-mfqtn Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-88a4d9-797cb8d49f-mfqtn to ip-10-0-133-193.ec2.internal 12m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-d5590b-5575b4pmf8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-d5590b-5575b4pmf8 to ip-10-0-133-193.ec2.internal 68m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-575956cc9crbcbw Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-575956cc9crbcbw to ip-10-0-128-81.ec2.internal 57m Normal Scheduled pod/e2e-worker-w2-553753-openshift-default-558949869c-85x5q Successfully assigned openshift-ingress/e2e-worker-w2-553753-openshift-default-558949869c-85x5q to ip-10-0-139-73.ec2.internal 59m Normal Scheduled pod/e2e-worker-w2-553753-openshift-default-558949869c-8l4s6 Successfully assigned openshift-ingress/e2e-worker-w2-553753-openshift-default-558949869c-8l4s6 to ip-10-0-128-81.ec2.internal 8m30s Normal Scheduled pod/e2e-worker-single-main-22c413-openshift-default-685f6fd8d9qs4x5 Successfully assigned openshift-ingress/e2e-worker-single-main-22c413-openshift-default-685f6fd8d9qs4x5 to ip-10-0-133-193.ec2.internal 64m Normal Scheduled pod/maas-default-gateway-openshift-default-7df854bf67-qhqx9 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-qhqx9 to ip-10-0-133-193.ec2.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-15e7a7-5467566b7-npkkl Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-15e7a7-5467566b7-npkkl to ip-10-0-139-73.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-15e7a7-dd597bff5-j8vmx Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-15e7a7-dd597bff5-j8vmx to ip-10-0-133-193.ec2.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-3920cb-65745db9dc-czdrn Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-3920cb-65745db9dc-czdrn to ip-10-0-139-73.ec2.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-3920cb-754cdc574c-hk2m5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-3920cb-754cdc574c-hk2m5 to ip-10-0-133-193.ec2.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-4ad693-5685dfd7bc-vbh2d Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-4ad693-5685dfd7bc-vbh2d to ip-10-0-139-73.ec2.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-4ad693-5cfb5979-pq6rs Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-4ad693-5cfb5979-pq6rs to ip-10-0-133-193.ec2.internal 49m Normal Scheduled pod/e2e-api-b-5dd3b1-openshift-default-684dbd6c7c-k4v6l Successfully assigned openshift-ingress/e2e-api-b-5dd3b1-openshift-default-684dbd6c7c-k4v6l to ip-10-0-133-193.ec2.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-68410c-68bc774c4c-z576z Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-68410c-68bc774c4c-z576z to ip-10-0-133-193.ec2.internal 30m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-68410c-76b7f58695-w8gpk Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-68410c-76b7f58695-w8gpk to ip-10-0-133-193.ec2.internal 16m Normal Scheduled pod/maas-default-gateway-openshift-default-7df854bf67-mfkdg Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-mfkdg to ip-10-0-128-81.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-a4b994-7d779cd5bc-sw2ht Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-a4b994-7d779cd5bc-sw2ht to ip-10-0-133-193.ec2.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-a4b994-f6b9d5df9-54k5k Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-a4b994-f6b9d5df9-54k5k to ip-10-0-133-193.ec2.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-162d7b-74ddd95c5-rpng8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-162d7b-74ddd95c5-rpng8 to ip-10-0-139-73.ec2.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-162d7b-c8d776ff9-cswcp Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-162d7b-c8d776ff9-cswcp to ip-10-0-133-193.ec2.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-4a01f9-6df8b8cb59-l6zlr Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-4a01f9-6df8b8cb59-l6zlr to ip-10-0-139-73.ec2.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-4a01f9-cfdc97c95-pfc48 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-4a01f9-cfdc97c95-pfc48 to ip-10-0-128-81.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-f48f46-5bbdc5f47f-9x7w8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-f48f46-5bbdc5f47f-9x7w8 to ip-10-0-133-193.ec2.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-f48f46-9875d54dc-m8hbh Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-f48f46-9875d54dc-m8hbh to ip-10-0-139-73.ec2.internal 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-f6dcc0-7fc8d968b7-4fllg Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-f6dcc0-7fc8d968b7-4fllg to ip-10-0-139-73.ec2.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-f6dcc0-84b547ffbc-6v2rf Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-f6dcc0-84b547ffbc-6v2rf to ip-10-0-139-73.ec2.internal 16m Normal Scheduled pod/maas-default-gateway-openshift-default-7df854bf67-gvf4z Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-gvf4z to ip-10-0-139-73.ec2.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-ff80c0-5c7dbc8495-5pnw8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-ff80c0-5c7dbc8495-5pnw8 to ip-10-0-133-193.ec2.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-ff80c0-5d8df5b89f-cp5nc Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-ff80c0-5d8df5b89f-cp5nc to ip-10-0-133-193.ec2.internal 49m Normal Scheduled pod/e2e-api-b-b826b6-openshift-default-785fbbd7-rwr56 Successfully assigned openshift-ingress/e2e-api-b-b826b6-openshift-default-785fbbd7-rwr56 to ip-10-0-128-81.ec2.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-migrate-b7b10505-775cb59b75-96cjh Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-b7b10505-775cb59b75-96cjh to ip-10-0-128-81.ec2.internal 16m Normal Scheduled pod/maas-default-gateway-openshift-default-7df854bf67-gprr2 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-gprr2 to ip-10-0-128-81.ec2.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-mt-53879c92-668d9d8855-74s4z Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-53879c92-668d9d8855-74s4z to ip-10-0-128-81.ec2.internal 17m Normal Scheduled pod/maas-default-gateway-openshift-default-7df854bf67-fxflc Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-fxflc to ip-10-0-139-73.ec2.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-mt-8b1efeb6-68787c86c9-5wqpz Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-8b1efeb6-68787c86c9-5wqpz to ip-10-0-133-193.ec2.internal 16m Normal Scheduled pod/maas-default-gateway-openshift-default-7df854bf67-fkpql Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-fkpql to ip-10-0-133-193.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-mt-8b1efeb6-68787c86c9-wpjjk Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-8b1efeb6-68787c86c9-wpjjk to ip-10-0-139-73.ec2.internal 16m Normal Scheduled pod/maas-default-gateway-openshift-default-7df854bf67-f7pll Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-f7pll to ip-10-0-133-193.ec2.internal 53m Normal Scheduled pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-5jtnl Successfully assigned openshift-ingress/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-5jtnl to ip-10-0-133-193.ec2.internal 45m Normal Scheduled pod/payload-processing-e2e-inf-a-50b9b5-7f48f98986-ddh6w Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-50b9b5-7f48f98986-ddh6w to ip-10-0-139-73.ec2.internal 53m Normal Scheduled pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-7kgzf Successfully assigned openshift-ingress/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-7kgzf to ip-10-0-128-81.ec2.internal 53m Normal Scheduled pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-8xg84 Successfully assigned openshift-ingress/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-8xg84 to ip-10-0-139-73.ec2.internal 45m Normal Scheduled pod/payload-processing-e2e-inf-a-7940ea-56b89d4b64-pvwxl Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-7940ea-56b89d4b64-pvwxl to ip-10-0-128-81.ec2.internal 53m Normal Scheduled pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-db8c6 Successfully assigned openshift-ingress/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-db8c6 to ip-10-0-133-193.ec2.internal 60m Normal Scheduled pod/maas-default-gateway-openshift-default-7df854bf67-ddx5s Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-ddx5s to ip-10-0-128-81.ec2.internal 43m Normal Scheduled pod/e2e-inf-b-fe7ca0-openshift-default-7598bdb7d5-jqmrs Successfully assigned openshift-ingress/e2e-inf-b-fe7ca0-openshift-default-7598bdb7d5-jqmrs to ip-10-0-128-81.ec2.internal 51m Normal Scheduled pod/e2e-api-b-bbfc2e-openshift-default-6c585b5767-jlcxj Successfully assigned openshift-ingress/e2e-api-b-bbfc2e-openshift-default-6c585b5767-jlcxj to ip-10-0-128-81.ec2.internal 63m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-2d619f-5c9fd58999-69tjr Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-2d619f-5c9fd58999-69tjr to ip-10-0-133-193.ec2.internal 52m Normal Scheduled pod/e2e-shared-b-w6-26f8ba-openshift-default-569ccb4c87-pcd2l Successfully assigned openshift-ingress/e2e-shared-b-w6-26f8ba-openshift-default-569ccb4c87-pcd2l to ip-10-0-133-193.ec2.internal 58m Normal Scheduled pod/e2e-worker-w2-553753-openshift-default-558949869c-9fdlh Successfully assigned openshift-ingress/e2e-worker-w2-553753-openshift-default-558949869c-9fdlh to ip-10-0-139-73.ec2.internal 61m Normal Scheduled pod/e2e-worker-w2-553753-openshift-default-558949869c-9k6cb Successfully assigned openshift-ingress/e2e-worker-w2-553753-openshift-default-558949869c-9k6cb to ip-10-0-139-73.ec2.internal 26m Normal Scheduled pod/e2e-ipp-b-4a01f9-openshift-default-bb6d8b97f-wzwlk Successfully assigned openshift-ingress/e2e-ipp-b-4a01f9-openshift-default-bb6d8b97f-wzwlk to ip-10-0-128-81.ec2.internal 42m Normal Scheduled pod/payload-processing-e2e-inf-b-963936-589db57f54-5m29v Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-963936-589db57f54-5m29v to ip-10-0-133-193.ec2.internal 58m Normal Scheduled pod/e2e-worker-w2-553753-openshift-default-558949869c-fgvq5 Successfully assigned openshift-ingress/e2e-worker-w2-553753-openshift-default-558949869c-fgvq5 to ip-10-0-133-193.ec2.internal 63m Normal Scheduled pod/e2e-worker-w2-553753-openshift-default-558949869c-j5dc2 Successfully assigned openshift-ingress/e2e-worker-w2-553753-openshift-default-558949869c-j5dc2 to ip-10-0-128-81.ec2.internal 64m Normal Scheduled pod/maas-default-gateway-openshift-default-7df854bf67-d6hr4 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-d6hr4 to ip-10-0-139-73.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w0-0847aa-6d5b985b7-xt8mm Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0847aa-6d5b985b7-xt8mm to ip-10-0-128-81.ec2.internal 53m Normal Scheduled pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-mcjd8 Successfully assigned openshift-ingress/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-mcjd8 to ip-10-0-133-193.ec2.internal 25m Normal Scheduled pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-pt6sj Successfully assigned openshift-ingress/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-pt6sj to ip-10-0-128-81.ec2.internal 53m Normal Scheduled pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-rltc7 Successfully assigned openshift-ingress/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-rltc7 to ip-10-0-139-73.ec2.internal 54m Normal Scheduled pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-tdqsq Successfully assigned openshift-ingress/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-tdqsq to ip-10-0-128-81.ec2.internal 50m Normal Scheduled pod/payload-processing-e2e-inf-a-4b5e2b-56d495d764-wxzjw Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-4b5e2b-56d495d764-wxzjw to ip-10-0-133-193.ec2.internal 34m Normal Scheduled pod/e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff-8l4w8 Successfully assigned openshift-ingress/e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff-8l4w8 to ip-10-0-139-73.ec2.internal 60m Normal Scheduled pod/maas-default-gateway-openshift-default-7df854bf67-crcsv Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-crcsv to ip-10-0-139-73.ec2.internal 44m Normal Scheduled pod/e2e-inf-b-ccdb60-openshift-default-678d965f7c-cttbz Successfully assigned openshift-ingress/e2e-inf-b-ccdb60-openshift-default-678d965f7c-cttbz to ip-10-0-139-73.ec2.internal 4m50s Normal Scheduled pod/e2e-authcfg-http-02e08d-openshift-default-8d4777cb5-mdkrz Successfully assigned openshift-ingress/e2e-authcfg-http-02e08d-openshift-default-8d4777cb5-mdkrz to ip-10-0-133-193.ec2.internal 73m Normal Scheduled pod/router-default-68496b87c-6xdvq Successfully assigned openshift-ingress/router-default-68496b87c-6xdvq to ip-10-0-133-193.ec2.internal 73m Warning FailedScheduling pod/router-default-68496b87c-6xdvq no nodes available to schedule pods 79m Warning FailedScheduling pod/router-default-68496b87c-6xdvq no nodes available to schedule pods 58m Normal Scheduled pod/e2e-worker-w2-553753-openshift-default-558949869c-jzgb6 Successfully assigned openshift-ingress/e2e-worker-w2-553753-openshift-default-558949869c-jzgb6 to ip-10-0-128-81.ec2.internal 61m Normal Scheduled pod/e2e-worker-w2-553753-openshift-default-558949869c-ktl7d Successfully assigned openshift-ingress/e2e-worker-w2-553753-openshift-default-558949869c-ktl7d to ip-10-0-133-193.ec2.internal 63m Normal Scheduled pod/payload-processing-e2e-worker-w2-553753-cf69c5f4b-q2vgq Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-553753-cf69c5f4b-q2vgq to ip-10-0-128-81.ec2.internal 45m Normal Scheduled pod/payload-processing-e2e-inf-b-71a7df-5f4b644b-zwb9k Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-71a7df-5f4b644b-zwb9k to ip-10-0-139-73.ec2.internal 64m Normal Scheduled pod/payload-processing-7bb9489676-bnqlr Successfully assigned openshift-ingress/payload-processing-7bb9489676-bnqlr to ip-10-0-133-193.ec2.internal 42m Normal Scheduled pod/e2e-inf-b-963936-openshift-default-68b65f857c-phdvv Successfully assigned openshift-ingress/e2e-inf-b-963936-openshift-default-68b65f857c-phdvv to ip-10-0-139-73.ec2.internal 57m Normal Scheduled pod/e2e-worker-w2-553753-openshift-default-558949869c-mn9sk Successfully assigned openshift-ingress/e2e-worker-w2-553753-openshift-default-558949869c-mn9sk to ip-10-0-128-81.ec2.internal 59m Normal Scheduled pod/e2e-worker-w2-553753-openshift-default-558949869c-rdndg Successfully assigned openshift-ingress/e2e-worker-w2-553753-openshift-default-558949869c-rdndg to ip-10-0-133-193.ec2.internal 63m Normal Scheduled pod/payload-processing-e2e-worker-w6-2d619f-bdbc76d94-99hjl Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-2d619f-bdbc76d94-99hjl to ip-10-0-133-193.ec2.internal 27m Normal Scheduled pod/e2e-ipp-b-f48f46-openshift-default-5d6fb88f69-l2ttk Successfully assigned openshift-ingress/e2e-ipp-b-f48f46-openshift-default-5d6fb88f69-l2ttk to ip-10-0-128-81.ec2.internal 34m Normal Scheduled pod/e2e-shared-b-w5-9b841d-openshift-default-797cdf6799-fnr9n Successfully assigned openshift-ingress/e2e-shared-b-w5-9b841d-openshift-default-797cdf6799-fnr9n to ip-10-0-128-81.ec2.internal 55m Normal Scheduled pod/e2e-derive-cbe6e297-openshift-default-795c76bc85-vgf74 Successfully assigned openshift-ingress/e2e-derive-cbe6e297-openshift-default-795c76bc85-vgf74 to ip-10-0-128-81.ec2.internal 52m Normal Scheduled pod/e2e-shared-b-w4-04988c-openshift-default-d99f984b9-zmsb6 Successfully assigned openshift-ingress/e2e-shared-b-w4-04988c-openshift-default-d99f984b9-zmsb6 to ip-10-0-139-73.ec2.internal 48m Normal Scheduled pod/payload-processing-e2e-inf-b-2d6076-d576997dd-rh48p Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-2d6076-d576997dd-rh48p to ip-10-0-139-73.ec2.internal 56m Normal Scheduled pod/payload-processing-e2e-ait-1b279244-855ccd57cb-l9wh4 Successfully assigned openshift-ingress/payload-processing-e2e-ait-1b279244-855ccd57cb-l9wh4 to ip-10-0-128-81.ec2.internal 45m Normal Scheduled pod/e2e-inf-b-71a7df-openshift-default-74d8445657-zqkwp Successfully assigned openshift-ingress/e2e-inf-b-71a7df-openshift-default-74d8445657-zqkwp to ip-10-0-128-81.ec2.internal 54m Normal Scheduled pod/e2e-shared-b-w4-04988c-openshift-default-d99f984b9-trdjb Successfully assigned openshift-ingress/e2e-shared-b-w4-04988c-openshift-default-d99f984b9-trdjb to ip-10-0-128-81.ec2.internal 64m Normal Scheduled pod/maas-default-gateway-openshift-default-7df854bf67-5w59f Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-5w59f to ip-10-0-128-81.ec2.internal 46m Normal Scheduled pod/payload-processing-e2e-inf-a-9e540d-79df886dc8-q6kwt Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-9e540d-79df886dc8-q6kwt to ip-10-0-139-73.ec2.internal 44m Normal Scheduled pod/payload-processing-e2e-inf-a-341c51-857b566498-5d2tg Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-341c51-857b566498-5d2tg to ip-10-0-139-73.ec2.internal 56m Normal Scheduled pod/payload-processing-e2e-ait-51bfa2e1-6954676576-kr8sq Successfully assigned openshift-ingress/payload-processing-e2e-ait-51bfa2e1-6954676576-kr8sq to ip-10-0-139-73.ec2.internal 30m Normal Scheduled pod/e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7-xcxdb Successfully assigned openshift-ingress/e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7-xcxdb to ip-10-0-133-193.ec2.internal 52m Normal Scheduled pod/e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7-8q4n6 Successfully assigned openshift-ingress/e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7-8q4n6 to ip-10-0-128-81.ec2.internal 55m Normal Scheduled pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-5hj9v Successfully assigned openshift-ingress/e2e-worker-w3-5187e1-openshift-default-946849bdf-5hj9v to ip-10-0-128-81.ec2.internal 43m Normal Scheduled pod/e2e-inf-b-71a7df-openshift-default-74d8445657-j68rv Successfully assigned openshift-ingress/e2e-inf-b-71a7df-openshift-default-74d8445657-j68rv to ip-10-0-139-73.ec2.internal 55m Normal Scheduled pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-cflp5 Successfully assigned openshift-ingress/e2e-worker-w3-5187e1-openshift-default-946849bdf-cflp5 to ip-10-0-139-73.ec2.internal 44m Normal Scheduled pod/e2e-inf-a-341c51-openshift-default-56f6687b6f-dqgpg Successfully assigned openshift-ingress/e2e-inf-a-341c51-openshift-default-56f6687b6f-dqgpg to ip-10-0-128-81.ec2.internal 64m Normal Scheduled pod/maas-default-gateway-openshift-default-7df854bf67-2cwvd Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7df854bf67-2cwvd to ip-10-0-133-193.ec2.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w6-26f8ba-5b94f459cf-zrpcl Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w6-26f8ba-5b94f459cf-zrpcl to ip-10-0-133-193.ec2.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w5-9b841d-6447d9f595-qxv2f Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w5-9b841d-6447d9f595-qxv2f to ip-10-0-133-193.ec2.internal 56m Normal Scheduled pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-g45p4 Successfully assigned openshift-ingress/e2e-worker-w3-5187e1-openshift-default-946849bdf-g45p4 to ip-10-0-133-193.ec2.internal 55m Normal Scheduled pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-knhhd Successfully assigned openshift-ingress/e2e-worker-w3-5187e1-openshift-default-946849bdf-knhhd to ip-10-0-133-193.ec2.internal 56m Normal Scheduled pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-q88pc Successfully assigned openshift-ingress/e2e-worker-w3-5187e1-openshift-default-946849bdf-q88pc to ip-10-0-139-73.ec2.internal 43m Normal Scheduled pod/e2e-inf-b-71a7df-openshift-default-74d8445657-fv778 Successfully assigned openshift-ingress/e2e-inf-b-71a7df-openshift-default-74d8445657-fv778 to ip-10-0-133-193.ec2.internal 63m Normal Scheduled pod/payload-processing-e2e-worker-w3-5187e1-5d69cc969d-9qmml Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-5187e1-5d69cc969d-9qmml to ip-10-0-133-193.ec2.internal 59m Normal Scheduled pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-qslxh Successfully assigned openshift-ingress/e2e-worker-w3-5187e1-openshift-default-946849bdf-qslxh to ip-10-0-139-73.ec2.internal 5m21s Normal Scheduled pod/payload-processing-e2e-ait-a6ecd9bf-5ffcbfd4b-97clk Successfully assigned openshift-ingress/payload-processing-e2e-ait-a6ecd9bf-5ffcbfd4b-97clk to ip-10-0-133-193.ec2.internal 42m Normal Scheduled pod/payload-processing-e2e-inf-b-124490-77db798586-7ghqs Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-124490-77db798586-7ghqs to ip-10-0-133-193.ec2.internal 22m Normal Scheduled pod/e2e-ipp-b-f6dcc0-openshift-default-54f6df97-v8l4r Successfully assigned openshift-ingress/e2e-ipp-b-f6dcc0-openshift-default-54f6df97-v8l4r to ip-10-0-128-81.ec2.internal 68m Normal Scheduled pod/kube-auth-proxy-6b75ff7d64-dqsnv Successfully assigned openshift-ingress/kube-auth-proxy-6b75ff7d64-dqsnv to ip-10-0-133-193.ec2.internal 55m Normal Scheduled pod/payload-processing-e2e-ait-d05919ef-98cb9cd48-4lr79 Successfully assigned openshift-ingress/payload-processing-e2e-ait-d05919ef-98cb9cd48-4lr79 to ip-10-0-139-73.ec2.internal 50m Normal Scheduled pod/e2e-inf-a-4b5e2b-openshift-default-7496d87895-f5vss Successfully assigned openshift-ingress/e2e-inf-a-4b5e2b-openshift-default-7496d87895-f5vss to ip-10-0-139-73.ec2.internal 50m Normal Scheduled pod/payload-processing-e2e-api-a-2c0223-696c7b845d-txtnq Successfully assigned openshift-ingress/payload-processing-e2e-api-a-2c0223-696c7b845d-txtnq to ip-10-0-133-193.ec2.internal 43m Normal Scheduled pod/e2e-inf-b-71a7df-openshift-default-74d8445657-bgwbv Successfully assigned openshift-ingress/e2e-inf-b-71a7df-openshift-default-74d8445657-bgwbv to ip-10-0-128-81.ec2.internal 36m Normal Scheduled pod/e2e-shared-b-w2-4f9bb0-openshift-default-599dcdf4f9-pwn4t Successfully assigned openshift-ingress/e2e-shared-b-w2-4f9bb0-openshift-default-599dcdf4f9-pwn4t to ip-10-0-139-73.ec2.internal 63m Normal Scheduled pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-td5c9 Successfully assigned openshift-ingress/e2e-worker-w3-5187e1-openshift-default-946849bdf-td5c9 to ip-10-0-128-81.ec2.internal 68m Normal Scheduled pod/kube-auth-proxy-6b75ff7d64-48pqm Successfully assigned openshift-ingress/kube-auth-proxy-6b75ff7d64-48pqm to ip-10-0-139-73.ec2.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-d4b6bb-7678b5f4f9-d68cc Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-d4b6bb-7678b5f4f9-d68cc to ip-10-0-133-193.ec2.internal 24m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-mzm2m Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mzm2m to ip-10-0-133-193.ec2.internal 56m Normal Scheduled pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-wtzl9 Successfully assigned openshift-ingress/e2e-worker-w3-5187e1-openshift-default-946849bdf-wtzl9 to ip-10-0-128-81.ec2.internal 57m Normal Scheduled pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-z7tn8 Successfully assigned openshift-ingress/e2e-worker-w3-5187e1-openshift-default-946849bdf-z7tn8 to ip-10-0-133-193.ec2.internal 43m Normal Scheduled pod/payload-processing-e2e-inf-b-fe7ca0-55488976b6-5hnxs Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-fe7ca0-55488976b6-5hnxs to ip-10-0-133-193.ec2.internal 55m Normal Scheduled pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-zglz6 Successfully assigned openshift-ingress/e2e-worker-w3-5187e1-openshift-default-946849bdf-zglz6 to ip-10-0-128-81.ec2.internal 53m Normal Scheduled pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-xdg5b Successfully assigned openshift-ingress/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-xdg5b to ip-10-0-128-81.ec2.internal 45m Normal Scheduled pod/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-br8fv Successfully assigned openshift-ingress/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-br8fv to ip-10-0-128-81.ec2.internal 48m Normal Scheduled pod/e2e-inf-b-2d6076-openshift-default-5bdb59ccf9-48ljn Successfully assigned openshift-ingress/e2e-inf-b-2d6076-openshift-default-5bdb59ccf9-48ljn to ip-10-0-128-81.ec2.internal 53m Normal Scheduled pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-v6bzq Successfully assigned openshift-ingress/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-v6bzq to ip-10-0-139-73.ec2.internal 51m Normal Scheduled pod/payload-processing-e2e-api-a-32fdf2-7469cf674d-4c2l6 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-32fdf2-7469cf674d-4c2l6 to ip-10-0-133-193.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w4-04988c-9dcb589c9-xrtn5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-04988c-9dcb589c9-xrtn5 to ip-10-0-133-193.ec2.internal 58m Normal Scheduled pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-2jb6v Successfully assigned openshift-ingress/e2e-worker-w4-50d738-openshift-default-8d6476cbc-2jb6v to ip-10-0-139-73.ec2.internal 56m Normal Scheduled pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-6skbz Successfully assigned openshift-ingress/e2e-worker-w4-50d738-openshift-default-8d6476cbc-6skbz to ip-10-0-133-193.ec2.internal 56m Normal Scheduled pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-6wd75 Successfully assigned openshift-ingress/e2e-worker-w4-50d738-openshift-default-8d6476cbc-6wd75 to ip-10-0-128-81.ec2.internal 58m Normal Scheduled pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-cqz7h Successfully assigned openshift-ingress/e2e-worker-w4-50d738-openshift-default-8d6476cbc-cqz7h to ip-10-0-128-81.ec2.internal 56m Normal Scheduled pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-jrmj4 Successfully assigned openshift-ingress/e2e-worker-w4-50d738-openshift-default-8d6476cbc-jrmj4 to ip-10-0-133-193.ec2.internal 60m Normal Scheduled pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-nj6zd Successfully assigned openshift-ingress/e2e-worker-w4-50d738-openshift-default-8d6476cbc-nj6zd to ip-10-0-133-193.ec2.internal 52m Normal Scheduled pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-ksw7k Successfully assigned openshift-ingress/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-ksw7k to ip-10-0-133-193.ec2.internal 40m Normal Scheduled pod/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-jwjrc Successfully assigned openshift-ingress/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-jwjrc to ip-10-0-139-73.ec2.internal 67m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-vg6xv Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vg6xv to ip-10-0-128-81.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w4-281732-99d75d6c9-xrckt Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-281732-99d75d6c9-xrckt to ip-10-0-128-81.ec2.internal 63m Normal Scheduled pod/payload-processing-e2e-worker-w4-50d738-85555f75b4-p55sv Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-50d738-85555f75b4-p55sv to ip-10-0-133-193.ec2.internal 56m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-56cz2 Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56cz2 to ip-10-0-133-193.ec2.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w5-b023a2-678fcf5d79-x2g2r Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w5-b023a2-678fcf5d79-x2g2r to ip-10-0-139-73.ec2.internal 55m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-2xmvs Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2xmvs to ip-10-0-139-73.ec2.internal 42m Normal Scheduled pod/e2e-inf-b-124490-openshift-default-54b4ffbf45-88rs8 Successfully assigned openshift-ingress/e2e-inf-b-124490-openshift-default-54b4ffbf45-88rs8 to ip-10-0-133-193.ec2.internal 63m Normal Scheduled pod/payload-processing-e2e-worker-w5-434ce3-5797fc889b-dfrzp Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-434ce3-5797fc889b-dfrzp to ip-10-0-133-193.ec2.internal 46m Normal Scheduled pod/payload-processing-e2e-inf-b-0a7065-694768f4b4-5fq8k Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-0a7065-694768f4b4-5fq8k to ip-10-0-133-193.ec2.internal 53m Normal Scheduled pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-jpkff Successfully assigned openshift-ingress/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-jpkff to ip-10-0-133-193.ec2.internal 52m Normal Scheduled pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-fd2cz Successfully assigned openshift-ingress/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-fd2cz to ip-10-0-139-73.ec2.internal 54m Normal Scheduled pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-d7r4r Successfully assigned openshift-ingress/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-d7r4r to ip-10-0-128-81.ec2.internal 52m Normal Scheduled pod/payload-processing-e2e-api-a-4c5345-6df87f7798-fkts8 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-4c5345-6df87f7798-fkts8 to ip-10-0-133-193.ec2.internal 63m Normal Scheduled pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-npjdw Successfully assigned openshift-ingress/e2e-worker-w4-50d738-openshift-default-8d6476cbc-npjdw to ip-10-0-128-81.ec2.internal 61m Normal Scheduled pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-scbfp Successfully assigned openshift-ingress/e2e-worker-w4-50d738-openshift-default-8d6476cbc-scbfp to ip-10-0-139-73.ec2.internal 25m Normal Scheduled pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-4rhzm Successfully assigned openshift-ingress/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-4rhzm to ip-10-0-139-73.ec2.internal 46m Normal Scheduled pod/e2e-inf-b-0a7065-openshift-default-56696cf555-w2wcw Successfully assigned openshift-ingress/e2e-inf-b-0a7065-openshift-default-56696cf555-w2wcw to ip-10-0-128-81.ec2.internal 50m Normal Scheduled pod/payload-processing-e2e-api-a-65b0e2-765dd6d498-5n54k Successfully assigned openshift-ingress/payload-processing-e2e-api-a-65b0e2-765dd6d498-5n54k to ip-10-0-139-73.ec2.internal 28m Normal Scheduled pod/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-4rzs2 Successfully assigned openshift-ingress/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-4rzs2 to ip-10-0-139-73.ec2.internal 45m Normal Scheduled pod/e2e-inf-a-7940ea-openshift-default-75b99d856c-56bcx Successfully assigned openshift-ingress/e2e-inf-a-7940ea-openshift-default-75b99d856c-56bcx to ip-10-0-128-81.ec2.internal 69m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-zqttb Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-zqttb to ip-10-0-133-193.ec2.internal 30m Normal Scheduled pod/e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc-wbswg Successfully assigned openshift-ingress/e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc-wbswg to ip-10-0-133-193.ec2.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w6-2a2eca-66ff5f6f5f-8cw8r Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w6-2a2eca-66ff5f6f5f-8cw8r to ip-10-0-133-193.ec2.internal 38m Normal Scheduled pod/e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff-b4dkg Successfully assigned openshift-ingress/e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff-b4dkg to ip-10-0-128-81.ec2.internal 55m Normal Scheduled pod/payload-processing-e2e-derive-cbe6e297-545ffc4c86-qnkpg Successfully assigned openshift-ingress/payload-processing-e2e-derive-cbe6e297-545ffc4c86-qnkpg to ip-10-0-139-73.ec2.internal 44m Normal Scheduled pod/payload-processing-e2e-inf-a-bb1fa5-6694d587bb-tdvvr Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-bb1fa5-6694d587bb-tdvvr to ip-10-0-133-193.ec2.internal 53m Normal Scheduled pod/e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc-7d49j Successfully assigned openshift-ingress/e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc-7d49j to ip-10-0-128-81.ec2.internal 40m Normal Scheduled pod/e2e-inf-a-7940ea-openshift-default-75b99d856c-n9xtq Successfully assigned openshift-ingress/e2e-inf-a-7940ea-openshift-default-75b99d856c-n9xtq to ip-10-0-133-193.ec2.internal 42m Normal Scheduled pod/e2e-inf-b-0a7065-openshift-default-56696cf555-sf6vl Successfully assigned openshift-ingress/e2e-inf-b-0a7065-openshift-default-56696cf555-sf6vl to ip-10-0-139-73.ec2.internal 48m Normal Scheduled pod/payload-processing-e2e-api-b-22edc1-998976c8b-4whv2 Successfully assigned openshift-ingress/payload-processing-e2e-api-b-22edc1-998976c8b-4whv2 to ip-10-0-133-193.ec2.internal 27m Normal Scheduled pod/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-z2mwg Successfully assigned openshift-ingress/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-z2mwg to ip-10-0-128-81.ec2.internal 35m Normal Scheduled pod/e2e-shared-a-w5-b023a2-openshift-default-7bd456f44f-wshn9 Successfully assigned openshift-ingress/e2e-shared-a-w5-b023a2-openshift-default-7bd456f44f-wshn9 to ip-10-0-139-73.ec2.internal 57m Normal Scheduled pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-qzh84 Successfully assigned openshift-ingress/e2e-worker-w6-2d619f-openshift-default-85b84758df-qzh84 to ip-10-0-133-193.ec2.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w3-d8a3d1-b7986f8f9-s9n6t Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d8a3d1-b7986f8f9-s9n6t to ip-10-0-133-193.ec2.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-4f9bb0-6f8b69d999-g9rgk Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f9bb0-6f8b69d999-g9rgk to ip-10-0-139-73.ec2.internal 49m Normal Scheduled pod/payload-processing-e2e-api-b-5dd3b1-5b45c4ff8-rmshj Successfully assigned openshift-ingress/payload-processing-e2e-api-b-5dd3b1-5b45c4ff8-rmshj to ip-10-0-133-193.ec2.internal 54m Normal Scheduled pod/e2e-shared-a-w4-281732-openshift-default-868d787ccc-qgmvd Successfully assigned openshift-ingress/e2e-shared-a-w4-281732-openshift-default-868d787ccc-qgmvd to ip-10-0-128-81.ec2.internal 27m Normal Scheduled pod/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-nbfj6 Successfully assigned openshift-ingress/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-nbfj6 to ip-10-0-133-193.ec2.internal 49m Normal Scheduled pod/payload-processing-e2e-api-b-b826b6-65dbdbf768-v9wnw Successfully assigned openshift-ingress/payload-processing-e2e-api-b-b826b6-65dbdbf768-v9wnw to ip-10-0-133-193.ec2.internal 59m Normal Scheduled pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-2nggf Successfully assigned openshift-ingress/e2e-worker-w5-434ce3-openshift-default-58c7774759-2nggf to ip-10-0-128-81.ec2.internal 44m Normal Scheduled pod/e2e-inf-a-bb1fa5-openshift-default-95cc6c6f-8b6vq Successfully assigned openshift-ingress/e2e-inf-a-bb1fa5-openshift-default-95cc6c6f-8b6vq to ip-10-0-139-73.ec2.internal 57m Normal Scheduled pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-4wl8g Successfully assigned openshift-ingress/e2e-worker-w5-434ce3-openshift-default-58c7774759-4wl8g to ip-10-0-139-73.ec2.internal 63m Normal Scheduled pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-nnfpk Successfully assigned openshift-ingress/e2e-worker-w6-2d619f-openshift-default-85b84758df-nnfpk to ip-10-0-133-193.ec2.internal 4m45s Normal Scheduled pod/payload-processing-e2e-authcfg-http-02e08d-7d74877466-xrf9j Successfully assigned openshift-ingress/payload-processing-e2e-authcfg-http-02e08d-7d74877466-xrf9j to ip-10-0-133-193.ec2.internal 56m Normal Scheduled pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-6t8qk Successfully assigned openshift-ingress/e2e-worker-w5-434ce3-openshift-default-58c7774759-6t8qk to ip-10-0-139-73.ec2.internal 56m Normal Scheduled pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-bsswj Successfully assigned openshift-ingress/e2e-worker-w5-434ce3-openshift-default-58c7774759-bsswj to ip-10-0-133-193.ec2.internal 56m Normal Scheduled pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-m2b7t Successfully assigned openshift-ingress/e2e-worker-w5-434ce3-openshift-default-58c7774759-m2b7t to ip-10-0-139-73.ec2.internal 53m Normal Scheduled pod/e2e-shared-a-w3-23d281-openshift-default-5cb785989-8jstk Successfully assigned openshift-ingress/e2e-shared-a-w3-23d281-openshift-default-5cb785989-8jstk to ip-10-0-133-193.ec2.internal 47m Normal Scheduled pod/payload-processing-e2e-inf-a-b1ec3a-5c67ddd778-kzpcm Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-b1ec3a-5c67ddd778-kzpcm to ip-10-0-133-193.ec2.internal 20m Normal Scheduled pod/e2e-shared-a-w3-23d281-openshift-default-5cb785989-xdjl2 Successfully assigned openshift-ingress/e2e-shared-a-w3-23d281-openshift-default-5cb785989-xdjl2 to ip-10-0-139-73.ec2.internal 59m Normal Scheduled pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-nckfd Successfully assigned openshift-ingress/e2e-worker-w6-2d619f-openshift-default-85b84758df-nckfd to ip-10-0-128-81.ec2.internal 57m Normal Scheduled pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-gwb2l Successfully assigned openshift-ingress/e2e-worker-w6-2d619f-openshift-default-85b84758df-gwb2l to ip-10-0-128-81.ec2.internal 56m Normal Scheduled pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-mqzl2 Successfully assigned openshift-ingress/e2e-worker-w5-434ce3-openshift-default-58c7774759-mqzl2 to ip-10-0-139-73.ec2.internal 56m Normal Scheduled pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-fsnjn Successfully assigned openshift-ingress/e2e-worker-w6-2d619f-openshift-default-85b84758df-fsnjn to ip-10-0-128-81.ec2.internal 57m Normal Scheduled pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-cj9f5 Successfully assigned openshift-ingress/e2e-worker-w6-2d619f-openshift-default-85b84758df-cj9f5 to ip-10-0-139-73.ec2.internal 57m Normal Scheduled pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-cd47x Successfully assigned openshift-ingress/e2e-worker-w6-2d619f-openshift-default-85b84758df-cd47x to ip-10-0-139-73.ec2.internal 56m Normal Scheduled pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-btjw2 Successfully assigned openshift-ingress/e2e-worker-w6-2d619f-openshift-default-85b84758df-btjw2 to ip-10-0-133-193.ec2.internal 57m Normal Scheduled pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-9h2j6 Successfully assigned openshift-ingress/e2e-worker-w6-2d619f-openshift-default-85b84758df-9h2j6 to ip-10-0-128-81.ec2.internal 56m Normal Scheduled pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-2gnhr Successfully assigned openshift-ingress/e2e-worker-w6-2d619f-openshift-default-85b84758df-2gnhr to ip-10-0-139-73.ec2.internal 57m Normal Scheduled pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-r4zr7 Successfully assigned openshift-ingress/e2e-worker-w5-434ce3-openshift-default-58c7774759-r4zr7 to ip-10-0-128-81.ec2.internal 43m Normal Scheduled pod/e2e-inf-a-b1ec3a-openshift-default-7675569d9-kjjgn Successfully assigned openshift-ingress/e2e-inf-a-b1ec3a-openshift-default-7675569d9-kjjgn to ip-10-0-128-81.ec2.internal 63m Normal Scheduled pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-s5wxn Successfully assigned openshift-ingress/e2e-worker-w5-434ce3-openshift-default-58c7774759-s5wxn to ip-10-0-133-193.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w3-23d281-8599fb6555-qtrjc Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-23d281-8599fb6555-qtrjc to ip-10-0-139-73.ec2.internal 46m Normal Scheduled pod/e2e-inf-a-9e540d-openshift-default-74d865bc89-p5z9q Successfully assigned openshift-ingress/e2e-inf-a-9e540d-openshift-default-74d865bc89-p5z9q to ip-10-0-133-193.ec2.internal 56m Normal Scheduled pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-vgf7q Successfully assigned openshift-ingress/e2e-worker-w5-434ce3-openshift-default-58c7774759-vgf7q to ip-10-0-133-193.ec2.internal 56m Normal Scheduled pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-wmp98 Successfully assigned openshift-ingress/e2e-worker-w5-434ce3-openshift-default-58c7774759-wmp98 to ip-10-0-128-81.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w0-a940d5-5b5d7f6f59-nvgzv Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a940d5-5b5d7f6f59-nvgzv to ip-10-0-133-193.ec2.internal 47m Normal Scheduled pod/e2e-inf-a-b1ec3a-openshift-default-7675569d9-qjkx2 Successfully assigned openshift-ingress/e2e-inf-a-b1ec3a-openshift-default-7675569d9-qjkx2 to ip-10-0-133-193.ec2.internal 79m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-68496b87c from 0 to 1 79m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 79m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 79m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 79m Normal SuccessfulCreate replicaset/router-default-68496b87c Created pod: router-default-68496b87c-6xdvq 73m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 73m Warning FailedMount pod/router-default-68496b87c-6xdvq MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 73m Warning FailedMount pod/router-default-68496b87c-6xdvq MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 72m Warning FailedMount pod/router-default-68496b87c-6xdvq MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 72m Warning FailedMount pod/router-default-68496b87c-6xdvq MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 72m Normal Started pod/router-default-68496b87c-6xdvq Container started 72m Normal Created pod/router-default-68496b87c-6xdvq Container created 72m Normal Pulled pod/router-default-68496b87c-6xdvq Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 72m Normal AddedInterface pod/router-default-68496b87c-6xdvq Add eth0 [10.132.0.18/23] from ovn-kubernetes 69m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 69m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-zqttb Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 69m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-zqttb Add eth0 [10.132.0.24/23] from ovn-kubernetes 69m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-zqttb 69m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 69m Normal Started pod/istiod-openshift-gateway-7876bfcb74-zqttb Container started 69m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-zqttb Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.4s (4.4s including waiting). Image size: 185476477 bytes. 69m Normal Created pod/istiod-openshift-gateway-7876bfcb74-zqttb Container created 69m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-zqttb Readiness probe failed: HTTP probe failed with statuscode: 503 69m Normal Pulling pod/maas-default-gateway-openshift-default-7df854bf67-v828p Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 69m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-zqttb Readiness probe error: HTTP probe failed with statuscode: 503... 69m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7df854bf67 from 0 to 1 69m Normal SuccessfulCreate replicaset/openshift-ai-inference-openshift-ai-inference-668d64f647 Created pod: openshift-ai-inference-openshift-ai-inference-668d64f647-fg9gs 69m Normal ScalingReplicaSet deployment/openshift-ai-inference-openshift-ai-inference Scaled up replica set openshift-ai-inference-openshift-ai-inference-668d64f647 from 0 to 1 69m Normal EnsuringLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensuring load balancer 69m Normal AddedInterface pod/maas-default-gateway-openshift-default-7df854bf67-v828p Add eth0 [10.134.0.32/23] from ovn-kubernetes 69m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7df854bf67 Created pod: maas-default-gateway-openshift-default-7df854bf67-v828p 69m Normal Pulling pod/openshift-ai-inference-openshift-ai-inference-668d64f647-fg9gs Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 69m Normal AddedInterface pod/openshift-ai-inference-openshift-ai-inference-668d64f647-fg9gs Add eth0 [10.134.0.33/23] from ovn-kubernetes 69m Normal EnsuredLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensured load balancer 69m Normal Pulled pod/maas-default-gateway-openshift-default-7df854bf67-v828p Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 8.489s (8.489s including waiting). Image size: 183555333 bytes. 69m Normal Started pod/openshift-ai-inference-openshift-ai-inference-668d64f647-fg9gs Container started 69m Normal Pulled pod/openshift-ai-inference-openshift-ai-inference-668d64f647-fg9gs Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 9.764s (9.764s including waiting). Image size: 183555333 bytes. 69m Normal Created pod/openshift-ai-inference-openshift-ai-inference-668d64f647-fg9gs Container created 68m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 68m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 68m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-6b75ff7d64 from 0 to 2 68m Normal Pulling pod/kube-auth-proxy-6b75ff7d64-dqsnv Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 68m Normal AddedInterface pod/kube-auth-proxy-6b75ff7d64-48pqm Add eth0 [10.133.0.24/23] from ovn-kubernetes 68m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-575956cc9c Created pod: data-science-gateway-data-science-gateway-class-575956cc9crbcbw 68m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-575956cc9c from 0 to 1 68m Normal SuccessfulCreate replicaset/kube-auth-proxy-6b75ff7d64 Created pod: kube-auth-proxy-6b75ff7d64-48pqm 68m Normal Pulled pod/data-science-gateway-data-science-gateway-class-575956cc9crbcbw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 68m Normal SuccessfulCreate replicaset/kube-auth-proxy-6b75ff7d64 Created pod: kube-auth-proxy-6b75ff7d64-dqsnv 68m Normal AddedInterface pod/kube-auth-proxy-6b75ff7d64-dqsnv Add eth0 [10.132.0.26/23] from ovn-kubernetes 68m Normal Created pod/data-science-gateway-data-science-gateway-class-575956cc9crbcbw Container created 68m Normal Pulling pod/kube-auth-proxy-6b75ff7d64-48pqm Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 68m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-575956cc9crbcbw Add eth0 [10.134.0.42/23] from ovn-kubernetes 68m Normal Started pod/data-science-gateway-data-science-gateway-class-575956cc9crbcbw Container started 68m Normal Pulled pod/kube-auth-proxy-6b75ff7d64-dqsnv Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.334s (2.334s including waiting). Image size: 188904677 bytes. 68m Normal Created pod/kube-auth-proxy-6b75ff7d64-48pqm Container created 68m Normal Started pod/kube-auth-proxy-6b75ff7d64-dqsnv Container started 68m Normal Pulled pod/kube-auth-proxy-6b75ff7d64-48pqm Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.321s (2.321s including waiting). Image size: 188904677 bytes. 68m Normal Started pod/kube-auth-proxy-6b75ff7d64-48pqm Container started 68m Normal Created pod/kube-auth-proxy-6b75ff7d64-dqsnv Container created 67m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 67m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 67m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 67m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 67m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-vg6xv 67m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 67m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-vg6xv Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 67m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-vg6xv Add eth0 [10.134.0.46/23] from ovn-kubernetes 67m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-vg6xv Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.559s (2.559s including waiting). Image size: 185476477 bytes. 67m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-vg6xv Container started 67m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-vg6xv Container created 67m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-vg6xv Readiness probe failed: HTTP probe failed with statuscode: 503 67m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-vg6xv Readiness probe error: HTTP probe failed with statuscode: 503... 67m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 67m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-zqttb 67m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-zqttb Stopping container discovery 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Normal Started pod/maas-default-gateway-openshift-default-7df854bf67-v828p Container started 66m Normal Created pod/maas-default-gateway-openshift-default-7df854bf67-v828p Container created 66m Normal Pulled pod/maas-default-gateway-openshift-default-7df854bf67-v828p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-v828p Startup probe error: Get "http://10.134.0.32:15021/healthz/ready": dial tcp 10.134.0.32:15021: connect: connection refused... 66m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-v828p Startup probe failed: Get "http://10.134.0.32:15021/healthz/ready": dial tcp 10.134.0.32:15021: connect: connection refused 65m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Normal AddedInterface pod/maas-default-gateway-openshift-default-7df854bf67-5w59f Add eth0 [10.134.0.50/23] from ovn-kubernetes 64m Normal Pulling pod/maas-default-gateway-openshift-default-7df854bf67-d6hr4 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 64m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7df854bf67 Created pod: maas-default-gateway-openshift-default-7df854bf67-qhqx9 64m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7df854bf67 Created pod: maas-default-gateway-openshift-default-7df854bf67-d6hr4 64m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7df854bf67 Created pod: maas-default-gateway-openshift-default-7df854bf67-5w59f 64m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7df854bf67 from 1 to 4 64m Normal Pulling pod/maas-default-gateway-openshift-default-7df854bf67-qhqx9 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 64m Normal AddedInterface pod/maas-default-gateway-openshift-default-7df854bf67-d6hr4 Add eth0 [10.133.0.34/23] from ovn-kubernetes 64m Normal Started pod/maas-default-gateway-openshift-default-7df854bf67-5w59f Container started 64m Normal Created pod/maas-default-gateway-openshift-default-7df854bf67-5w59f Container created 64m Normal AddedInterface pod/maas-default-gateway-openshift-default-7df854bf67-qhqx9 Add eth0 [10.132.0.40/23] from ovn-kubernetes 64m Normal Pulled pod/maas-default-gateway-openshift-default-7df854bf67-5w59f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Pulled pod/maas-default-gateway-openshift-default-7df854bf67-qhqx9 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.664s (2.664s including waiting). Image size: 183555333 bytes. 64m Normal Created pod/maas-default-gateway-openshift-default-7df854bf67-d6hr4 Container created 64m Normal Pulled pod/maas-default-gateway-openshift-default-7df854bf67-d6hr4 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.792s (2.792s including waiting). Image size: 183555333 bytes. 64m Normal Created pod/maas-default-gateway-openshift-default-7df854bf67-qhqx9 Container created 64m Normal Started pod/maas-default-gateway-openshift-default-7df854bf67-qhqx9 Container started 64m Normal Started pod/maas-default-gateway-openshift-default-7df854bf67-d6hr4 Container started 64m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-5w59f Startup probe failed: Get "http://10.134.0.50:15021/healthz/ready": dial tcp 10.134.0.50:15021: connect: connection refused 64m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-5w59f Startup probe error: Get "http://10.134.0.50:15021/healthz/ready": dial tcp 10.134.0.50:15021: connect: connection refused... 64m Normal SuccessfulCreate replicaset/payload-processing-7bb9489676 Created pod: payload-processing-7bb9489676-bnqlr 64m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-wnlt4 64m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 64m Warning FailedCreate replicaset/payload-processing-7bb9489676 Error creating: pods "payload-processing-7bb9489676-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 64m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 64m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-7bb9489676 from 0 to 1 64m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-qhqx9 Startup probe failed: Get "http://10.132.0.40:15021/healthz/ready": dial tcp 10.132.0.40:15021: connect: connection refused 64m Normal Pulling pod/payload-processing-7bb9489676-bnqlr Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 64m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-wnlt4 Add eth0 [10.132.0.43/23] from ovn-kubernetes 64m Normal Pulling pod/payload-pre-processing-7b98bb678c-wnlt4 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 64m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-d6hr4 Startup probe error: Get "http://10.133.0.34:15021/healthz/ready": dial tcp 10.133.0.34:15021: connect: connection refused... 64m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-qhqx9 Startup probe error: Get "http://10.132.0.40:15021/healthz/ready": dial tcp 10.132.0.40:15021: connect: connection refused... 64m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-d6hr4 Startup probe failed: Get "http://10.133.0.34:15021/healthz/ready": dial tcp 10.133.0.34:15021: connect: connection refused 64m Normal AddedInterface pod/payload-processing-7bb9489676-bnqlr Add eth0 [10.132.0.42/23] from ovn-kubernetes 64m Normal Pulled pod/payload-processing-7bb9489676-bnqlr Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.115s (3.115s including waiting). Image size: 125353963 bytes. 64m Normal Started pod/payload-pre-processing-7b98bb678c-wnlt4 Container started 64m Normal Created pod/payload-processing-7bb9489676-bnqlr Container created 64m Normal Started pod/payload-processing-7bb9489676-bnqlr Container started 64m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-d6hr4 Startup probe error: Get "http://10.133.0.34:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 64m Normal Pulled pod/payload-pre-processing-7b98bb678c-wnlt4 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.003s (3.003s including waiting). Image size: 125353963 bytes. 64m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-d6hr4 Startup probe failed: Get "http://10.133.0.34:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 64m Normal Created pod/payload-pre-processing-7b98bb678c-wnlt4 Container created 64m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-qhqx9 Startup probe error: Get "http://10.132.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 64m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-qhqx9 Startup probe failed: Get "http://10.132.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 64m Normal Started pod/maas-default-gateway-openshift-default-7df854bf67-xt8wx Container started 64m Normal Started pod/maas-default-gateway-openshift-default-7df854bf67-2cwvd Container started 64m Normal AddedInterface pod/maas-default-gateway-openshift-default-7df854bf67-zqt58 Add eth0 [10.134.0.52/23] from ovn-kubernetes 64m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7df854bf67 Created pod: maas-default-gateway-openshift-default-7df854bf67-xt8wx 64m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7df854bf67 Created pod: maas-default-gateway-openshift-default-7df854bf67-zqt58 64m Normal AddedInterface pod/maas-default-gateway-openshift-default-7df854bf67-xt8wx Add eth0 [10.132.0.45/23] from ovn-kubernetes 64m Normal Pulled pod/maas-default-gateway-openshift-default-7df854bf67-zqt58 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Pulled pod/maas-default-gateway-openshift-default-7df854bf67-xt8wx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7df854bf67 Created pod: maas-default-gateway-openshift-default-7df854bf67-r4qjb 64m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: 64m Normal AddedInterface pod/maas-default-gateway-openshift-default-7df854bf67-r4qjb Add eth0 [10.133.0.35/23] from ovn-kubernetes 64m Normal Created pod/maas-default-gateway-openshift-default-7df854bf67-xt8wx Container created 64m Normal Pulled pod/maas-default-gateway-openshift-default-7df854bf67-r4qjb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Created pod/maas-default-gateway-openshift-default-7df854bf67-r4qjb Container created 64m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7df854bf67 Created pod: maas-default-gateway-openshift-default-7df854bf67-2cwvd 64m Normal Started pod/maas-default-gateway-openshift-default-7df854bf67-r4qjb Container started 64m Normal Created pod/maas-default-gateway-openshift-default-7df854bf67-2cwvd Container created 64m Normal Pulled pod/maas-default-gateway-openshift-default-7df854bf67-2cwvd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal AddedInterface pod/maas-default-gateway-openshift-default-7df854bf67-2cwvd Add eth0 [10.132.0.44/23] from ovn-kubernetes 64m Normal Started pod/maas-default-gateway-openshift-default-7df854bf67-zqt58 Container started 64m Normal Created pod/maas-default-gateway-openshift-default-7df854bf67-zqt58 Container created 64m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-xt8wx Startup probe error: Get "http://10.132.0.45:15021/healthz/ready": dial tcp 10.132.0.45:15021: connect: connection refused... 64m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-2cwvd Startup probe error: Get "http://10.132.0.44:15021/healthz/ready": dial tcp 10.132.0.44:15021: connect: connection refused... 64m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-2cwvd Startup probe failed: Get "http://10.132.0.44:15021/healthz/ready": dial tcp 10.132.0.44:15021: connect: connection refused 64m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-xt8wx Startup probe failed: Get "http://10.132.0.45:15021/healthz/ready": dial tcp 10.132.0.45:15021: connect: connection refused 64m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-r4qjb Startup probe failed: Get "http://10.133.0.35:15021/healthz/ready": dial tcp 10.133.0.35:15021: connect: connection refused 64m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-zqt58 Startup probe failed: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused 64m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-zqt58 Startup probe error: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused... 64m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-r4qjb Startup probe error: Get "http://10.133.0.35:15021/healthz/ready": dial tcp 10.133.0.35:15021: connect: connection refused... 63m Normal AddedLabel gateway/e2e-worker-w4-50d738 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-50d738 63m Normal AddedLabel gateway/e2e-worker-w6-2d619f Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-2d619f 63m Normal AddedLabel gateway/e2e-worker-w5-434ce3 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-434ce3 63m Normal AddedLabel gateway/e2e-worker-w3-5187e1 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-5187e1 63m Normal AddedLabel gateway/e2e-worker-w2-553753 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-553753 63m Normal AddedLabel gateway/e2e-worker-w1-e19850 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-e19850 63m Normal AddedLabel gateway/e2e-worker-w0-88a4d9 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-88a4d9 63m Normal SuccessfulCreate replicaset/e2e-worker-w6-2d619f-openshift-default-85b84758df Created pod: e2e-worker-w6-2d619f-openshift-default-85b84758df-nnfpk 63m Normal ScalingReplicaSet deployment/e2e-worker-w0-88a4d9-openshift-default Scaled up replica set e2e-worker-w0-88a4d9-openshift-default-857757469c from 0 to 1 63m Normal ScalingReplicaSet deployment/e2e-worker-w6-2d619f-openshift-default Scaled up replica set e2e-worker-w6-2d619f-openshift-default-85b84758df from 0 to 1 63m Normal SuccessfulCreate replicaset/e2e-worker-w1-e19850-openshift-default-9f855bd47 Created pod: e2e-worker-w1-e19850-openshift-default-9f855bd47-4kq96 63m Normal ScalingReplicaSet deployment/e2e-worker-w3-5187e1-openshift-default Scaled up replica set e2e-worker-w3-5187e1-openshift-default-946849bdf from 0 to 1 63m Normal AddedInterface pod/e2e-worker-w2-553753-openshift-default-558949869c-j5dc2 Add eth0 [10.134.0.55/23] from ovn-kubernetes 63m Normal ScalingReplicaSet deployment/e2e-worker-w1-e19850-openshift-default Scaled up replica set e2e-worker-w1-e19850-openshift-default-9f855bd47 from 0 to 1 63m Normal SuccessfulCreate replicaset/e2e-worker-w3-5187e1-openshift-default-946849bdf Created pod: e2e-worker-w3-5187e1-openshift-default-946849bdf-td5c9 63m Normal SuccessfulCreate replicaset/e2e-worker-w0-88a4d9-openshift-default-857757469c Created pod: e2e-worker-w0-88a4d9-openshift-default-857757469c-dcb6t 63m Normal SuccessfulCreate replicaset/e2e-worker-w4-50d738-openshift-default-8d6476cbc Created pod: e2e-worker-w4-50d738-openshift-default-8d6476cbc-npjdw 63m Normal ScalingReplicaSet deployment/e2e-worker-w5-434ce3-openshift-default Scaled up replica set e2e-worker-w5-434ce3-openshift-default-58c7774759 from 0 to 1 63m Normal AddedInterface pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-npjdw Add eth0 [10.134.0.54/23] from ovn-kubernetes 63m Normal SuccessfulCreate replicaset/e2e-worker-w2-553753-openshift-default-558949869c Created pod: e2e-worker-w2-553753-openshift-default-558949869c-j5dc2 63m Normal AddedInterface pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-nnfpk Add eth0 [10.132.0.46/23] from ovn-kubernetes 63m Normal SuccessfulCreate replicaset/e2e-worker-w5-434ce3-openshift-default-58c7774759 Created pod: e2e-worker-w5-434ce3-openshift-default-58c7774759-s5wxn 63m Normal ScalingReplicaSet deployment/e2e-worker-w2-553753-openshift-default Scaled up replica set e2e-worker-w2-553753-openshift-default-558949869c from 0 to 1 63m Normal ScalingReplicaSet deployment/e2e-worker-w4-50d738-openshift-default Scaled up replica set e2e-worker-w4-50d738-openshift-default-8d6476cbc from 0 to 1 63m Normal AddedInterface pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-s5wxn Add eth0 [10.132.0.47/23] from ovn-kubernetes 63m Normal AddedInterface pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-td5c9 Add eth0 [10.134.0.56/23] from ovn-kubernetes 63m Normal AddedInterface pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-4kq96 Add eth0 [10.133.0.36/23] from ovn-kubernetes 63m Normal AddedInterface pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-dcb6t Add eth0 [10.134.0.57/23] from ovn-kubernetes 63m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-50d738-5b4664c9c9 Error creating: pods "payload-pre-processing-e2e-worker-w4-50d738-5b4664c9c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-50d738: serviceaccount "payload-processing-e2e-worker-w4-50d738" not found 63m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-50d738-85555f75b4 Error creating: pods "payload-processing-e2e-worker-w4-50d738-85555f75b4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-50d738: serviceaccount "payload-processing-e2e-worker-w4-50d738" not found 63m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-50d738 Scaled up replica set payload-processing-e2e-worker-w4-50d738-85555f75b4 from 0 to 1 63m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-50d738 Scaled up replica set payload-pre-processing-e2e-worker-w4-50d738-5b4664c9c9 from 0 to 1 63m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-50d738-5b4664c9c9-zldwf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal Pulled pod/payload-processing-e2e-worker-w4-50d738-85555f75b4-p55sv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/payload-processing-e2e-worker-w4-50d738-85555f75b4-p55sv Add eth0 [10.132.0.48/23] from ovn-kubernetes 63m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-50d738-5b4664c9c9 Created pod: payload-pre-processing-e2e-worker-w4-50d738-5b4664c9c9-zldwf 63m Normal Started pod/payload-processing-e2e-worker-w4-50d738-85555f75b4-p55sv Container started 63m Normal Started pod/payload-pre-processing-e2e-worker-w4-50d738-5b4664c9c9-zldwf Container started 63m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-50d738-5b4664c9c9-zldwf Add eth0 [10.132.0.49/23] from ovn-kubernetes 63m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-50d738-85555f75b4 Created pod: payload-processing-e2e-worker-w4-50d738-85555f75b4-p55sv 63m Normal Created pod/payload-processing-e2e-worker-w4-50d738-85555f75b4-p55sv Container created 63m Normal Created pod/payload-pre-processing-e2e-worker-w4-50d738-5b4664c9c9-zldwf Container created 63m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-2d619f-5c9fd58999 Created pod: payload-pre-processing-e2e-worker-w6-2d619f-5c9fd58999-69tjr 63m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-2d619f Scaled up replica set payload-pre-processing-e2e-worker-w6-2d619f-5c9fd58999 from 0 to 1 63m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-2d619f-5c9fd58999 Error creating: pods "payload-pre-processing-e2e-worker-w6-2d619f-5c9fd58999-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-2d619f: serviceaccount "payload-processing-e2e-worker-w6-2d619f" not found 63m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-2d619f-bdbc76d94 Created pod: payload-processing-e2e-worker-w6-2d619f-bdbc76d94-99hjl 63m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-2d619f-bdbc76d94 Error creating: pods "payload-processing-e2e-worker-w6-2d619f-bdbc76d94-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-2d619f: serviceaccount "payload-processing-e2e-worker-w6-2d619f" not found 63m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-2d619f Scaled up replica set payload-processing-e2e-worker-w6-2d619f-bdbc76d94 from 0 to 1 63m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-e19850 Scaled up replica set payload-processing-e2e-worker-w1-e19850-565c99cd8d from 0 to 1 63m Normal Started pod/payload-processing-e2e-worker-w6-2d619f-bdbc76d94-99hjl Container started 63m Normal AddedInterface pod/payload-processing-e2e-worker-w1-e19850-565c99cd8d-qhx44 Add eth0 [10.132.0.52/23] from ovn-kubernetes 63m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-e19850 Scaled up replica set payload-pre-processing-e2e-worker-w1-e19850-5898685ff from 0 to 1 63m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-e19850-5898685ff Created pod: payload-pre-processing-e2e-worker-w1-e19850-5898685ff-kd5l5 63m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-e19850-5898685ff Error creating: pods "payload-pre-processing-e2e-worker-w1-e19850-5898685ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-e19850: serviceaccount "payload-processing-e2e-worker-w1-e19850" not found 63m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-e19850-565c99cd8d Error creating: pods "payload-processing-e2e-worker-w1-e19850-565c99cd8d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-e19850: serviceaccount "payload-processing-e2e-worker-w1-e19850" not found 63m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-e19850-565c99cd8d Created pod: payload-processing-e2e-worker-w1-e19850-565c99cd8d-qhx44 63m Normal AddedInterface pod/payload-processing-e2e-worker-w6-2d619f-bdbc76d94-99hjl Add eth0 [10.132.0.51/23] from ovn-kubernetes 63m Normal Pulled pod/payload-processing-e2e-worker-w6-2d619f-bdbc76d94-99hjl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-2d619f-5c9fd58999-69tjr Add eth0 [10.132.0.50/23] from ovn-kubernetes 63m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-2d619f-5c9fd58999-69tjr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal Created pod/payload-pre-processing-e2e-worker-w6-2d619f-5c9fd58999-69tjr Container created 63m Normal Started pod/payload-pre-processing-e2e-worker-w6-2d619f-5c9fd58999-69tjr Container started 63m Normal Created pod/payload-processing-e2e-worker-w6-2d619f-bdbc76d94-99hjl Container created 63m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-e19850-5898685ff-kd5l5 Add eth0 [10.134.0.58/23] from ovn-kubernetes 63m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-553753-b48c58c59 Error creating: pods "payload-pre-processing-e2e-worker-w2-553753-b48c58c59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-553753: serviceaccount "payload-processing-e2e-worker-w2-553753" not found 63m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-553753 Scaled up replica set payload-pre-processing-e2e-worker-w2-553753-b48c58c59 from 0 to 1 63m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-553753-b48c58c59-ntcj2 Add eth0 [10.132.0.53/23] from ovn-kubernetes 63m Normal Pulling pod/payload-pre-processing-e2e-worker-w1-e19850-5898685ff-kd5l5 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 63m Normal Pulled pod/payload-processing-e2e-worker-w1-e19850-565c99cd8d-qhx44 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-553753-b48c58c59-ntcj2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-553753 Scaled up replica set payload-processing-e2e-worker-w2-553753-cf69c5f4b from 0 to 1 63m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-553753-cf69c5f4b Created pod: payload-processing-e2e-worker-w2-553753-cf69c5f4b-q2vgq 63m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-553753-cf69c5f4b Error creating: pods "payload-processing-e2e-worker-w2-553753-cf69c5f4b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-553753: serviceaccount "payload-processing-e2e-worker-w2-553753" not found 63m Normal Created pod/payload-processing-e2e-worker-w1-e19850-565c99cd8d-qhx44 Container created 63m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-553753-b48c58c59 Created pod: payload-pre-processing-e2e-worker-w2-553753-b48c58c59-ntcj2 63m Normal Started pod/payload-processing-e2e-worker-w1-e19850-565c99cd8d-qhx44 Container started 63m Normal Started pod/payload-pre-processing-e2e-worker-w2-553753-b48c58c59-ntcj2 Container started 63m Normal Pulling pod/payload-processing-e2e-worker-w2-553753-cf69c5f4b-q2vgq Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 63m Normal AddedInterface pod/payload-processing-e2e-worker-w2-553753-cf69c5f4b-q2vgq Add eth0 [10.134.0.59/23] from ovn-kubernetes 63m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-434ce3-5797fc889b Created pod: payload-processing-e2e-worker-w5-434ce3-5797fc889b-dfrzp 63m Normal Created pod/payload-pre-processing-e2e-worker-w2-553753-b48c58c59-ntcj2 Container created 63m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-434ce3 Scaled up replica set payload-pre-processing-e2e-worker-w5-434ce3-7b5c6dc5b7 from 0 to 1 63m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-434ce3-5797fc889b Error creating: pods "payload-processing-e2e-worker-w5-434ce3-5797fc889b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-434ce3: serviceaccount "payload-processing-e2e-worker-w5-434ce3" not found 63m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-434ce3 Scaled up replica set payload-processing-e2e-worker-w5-434ce3-5797fc889b from 0 to 1 63m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-434ce3-7b5c6dc5b7 Error creating: pods "payload-pre-processing-e2e-worker-w5-434ce3-7b5c6dc5b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-434ce3: serviceaccount "payload-processing-e2e-worker-w5-434ce3" not found 63m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-434ce3-7b5c6dc5b7 Created pod: payload-pre-processing-e2e-worker-w5-434ce3-7b5c6dc5b7-pgcvw 63m Normal Pulled pod/payload-processing-e2e-worker-w5-434ce3-5797fc889b-dfrzp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal Started pod/payload-processing-e2e-worker-w5-434ce3-5797fc889b-dfrzp Container started 63m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-434ce3-7b5c6dc5b7-pgcvw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal Created pod/payload-pre-processing-e2e-worker-w1-e19850-5898685ff-kd5l5 Container created 63m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-e19850-5898685ff-kd5l5 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.726s (2.726s including waiting). Image size: 125353963 bytes. 63m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-434ce3-7b5c6dc5b7-pgcvw Add eth0 [10.132.0.55/23] from ovn-kubernetes 63m Normal Started pod/payload-pre-processing-e2e-worker-w1-e19850-5898685ff-kd5l5 Container started 63m Normal Created pod/payload-pre-processing-e2e-worker-w5-434ce3-7b5c6dc5b7-pgcvw Container created 63m Normal Started pod/payload-pre-processing-e2e-worker-w5-434ce3-7b5c6dc5b7-pgcvw Container started 63m Normal AddedInterface pod/payload-processing-e2e-worker-w5-434ce3-5797fc889b-dfrzp Add eth0 [10.132.0.54/23] from ovn-kubernetes 63m Normal Created pod/payload-processing-e2e-worker-w5-434ce3-5797fc889b-dfrzp Container created 63m Normal Created pod/payload-processing-e2e-worker-w2-553753-cf69c5f4b-q2vgq Container created 63m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-5187e1-7968f4cdbc Error creating: pods "payload-pre-processing-e2e-worker-w3-5187e1-7968f4cdbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-5187e1: serviceaccount "payload-processing-e2e-worker-w3-5187e1" not found 63m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-5187e1 Scaled up replica set payload-pre-processing-e2e-worker-w3-5187e1-7968f4cdbc from 0 to 1 63m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-5187e1 Scaled up replica set payload-processing-e2e-worker-w3-5187e1-5d69cc969d from 0 to 1 63m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-5187e1-7968f4cdbc Created pod: payload-pre-processing-e2e-worker-w3-5187e1-7968f4cdbc-klfgh 63m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-5187e1-5d69cc969d Created pod: payload-processing-e2e-worker-w3-5187e1-5d69cc969d-9qmml 63m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-5187e1-5d69cc969d Error creating: pods "payload-processing-e2e-worker-w3-5187e1-5d69cc969d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-5187e1: serviceaccount "payload-processing-e2e-worker-w3-5187e1" not found 63m Normal Pulled pod/payload-processing-e2e-worker-w2-553753-cf69c5f4b-q2vgq Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.563s (2.563s including waiting). Image size: 125353963 bytes. 63m Normal Started pod/payload-processing-e2e-worker-w2-553753-cf69c5f4b-q2vgq Container started 63m Normal AddedInterface pod/payload-processing-e2e-worker-w3-5187e1-5d69cc969d-9qmml Add eth0 [10.132.0.57/23] from ovn-kubernetes 63m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-88a4d9-797cb8d49f Created pod: payload-pre-processing-e2e-worker-w0-88a4d9-797cb8d49f-mfqtn 63m Normal Pulled pod/payload-processing-e2e-worker-w3-5187e1-5d69cc969d-9qmml Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-88a4d9-797cb8d49f-mfqtn Add eth0 [10.132.0.58/23] from ovn-kubernetes 63m Normal Created pod/payload-processing-e2e-worker-w3-5187e1-5d69cc969d-9qmml Container created 63m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-88a4d9 Scaled up replica set payload-pre-processing-e2e-worker-w0-88a4d9-797cb8d49f from 0 to 1 63m Normal Started pod/payload-processing-e2e-worker-w3-5187e1-5d69cc969d-9qmml Container started 63m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-88a4d9-797cb8d49f Error creating: pods "payload-pre-processing-e2e-worker-w0-88a4d9-797cb8d49f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-88a4d9: serviceaccount "payload-processing-e2e-worker-w0-88a4d9" not found 63m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-5187e1-7968f4cdbc-klfgh Add eth0 [10.132.0.56/23] from ovn-kubernetes 63m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-88a4d9-76c4574f8b Error creating: pods "payload-processing-e2e-worker-w0-88a4d9-76c4574f8b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-88a4d9: serviceaccount "payload-processing-e2e-worker-w0-88a4d9" not found 63m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-5187e1-7968f4cdbc-klfgh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-88a4d9-76c4574f8b Created pod: payload-processing-e2e-worker-w0-88a4d9-76c4574f8b-g8sjs 63m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-88a4d9 Scaled up replica set payload-processing-e2e-worker-w0-88a4d9-76c4574f8b from 0 to 1 63m Normal Created pod/payload-pre-processing-e2e-worker-w3-5187e1-7968f4cdbc-klfgh Container created 63m Normal Started pod/payload-pre-processing-e2e-worker-w3-5187e1-7968f4cdbc-klfgh Container started 63m Normal Created pod/payload-processing-e2e-worker-w0-88a4d9-76c4574f8b-g8sjs Container created 63m Normal Created pod/payload-pre-processing-e2e-worker-w0-88a4d9-797cb8d49f-mfqtn Container created 63m Normal Started pod/payload-processing-e2e-worker-w0-88a4d9-76c4574f8b-g8sjs Container started 63m Normal AddedInterface pod/payload-processing-e2e-worker-w0-88a4d9-76c4574f8b-g8sjs Add eth0 [10.134.0.61/23] from ovn-kubernetes 63m Normal Pulled pod/payload-processing-e2e-worker-w0-88a4d9-76c4574f8b-g8sjs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-88a4d9-797cb8d49f-mfqtn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal Started pod/payload-pre-processing-e2e-worker-w0-88a4d9-797cb8d49f-mfqtn Container started 63m Warning Unhealthy pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-nnfpk Readiness probe failed: Get "http://10.132.0.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 63m Warning ProbeError pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-nnfpk Readiness probe error: Get "http://10.132.0.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 63m Normal Created pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-nnfpk Container created 63m Normal Started pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-nnfpk Container started 63m Normal Pulled pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-nnfpk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Created pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-4kq96 Container created 63m Normal Pulled pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-4kq96 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Started pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-4kq96 Container started 63m Normal Pulled pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-s5wxn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Started pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-s5wxn Container started 63m Normal Created pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-s5wxn Container created 63m Warning ProbeError pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-nnfpk Startup probe error: Get "http://10.132.0.46:15021/healthz/ready": dial tcp 10.132.0.46:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-nnfpk Startup probe failed: Get "http://10.132.0.46:15021/healthz/ready": dial tcp 10.132.0.46:15021: connect: connection refused 63m Normal Pulled pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-td5c9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Created pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-td5c9 Container created 63m Normal Started pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-td5c9 Container started 63m Warning ProbeError pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-4kq96 Startup probe error: Get "http://10.133.0.36:15021/healthz/ready": dial tcp 10.133.0.36:15021: connect: connection refused... 63m Warning ProbeError pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-s5wxn Startup probe error: Get "http://10.132.0.47:15021/healthz/ready": dial tcp 10.132.0.47:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-s5wxn Startup probe failed: Get "http://10.132.0.47:15021/healthz/ready": dial tcp 10.132.0.47:15021: connect: connection refused 63m Warning Unhealthy pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-4kq96 Startup probe failed: Get "http://10.133.0.36:15021/healthz/ready": dial tcp 10.133.0.36:15021: connect: connection refused 63m Normal Pulled pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-dcb6t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Created pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-dcb6t Container created 63m Normal Started pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-dcb6t Container started 63m Warning ProbeError pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-dcb6t Readiness probe error: Get "http://10.134.0.57:15021/healthz/ready": read tcp 10.134.0.2:37752->10.134.0.57:15021: read: connection reset by peer... 63m Warning Unhealthy pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-dcb6t Readiness probe failed: Get "http://10.134.0.57:15021/healthz/ready": read tcp 10.134.0.2:37752->10.134.0.57:15021: read: connection reset by peer 63m Warning ProbeError pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-npjdw Readiness probe error: Get "http://10.134.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 63m Warning Unhealthy pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-npjdw Readiness probe failed: Get "http://10.134.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 63m Warning ProbeError pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-td5c9 Startup probe error: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-td5c9 Startup probe failed: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused 63m Normal Pulled pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-npjdw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Created pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-npjdw Container created 63m Normal Started pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-npjdw Container started 63m Warning ProbeError pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-dcb6t Startup probe error: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-dcb6t Startup probe failed: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused 63m Warning Unhealthy pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-npjdw Startup probe failed: Get "http://10.134.0.54:15021/healthz/ready": dial tcp 10.134.0.54:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-npjdw Startup probe error: Get "http://10.134.0.54:15021/healthz/ready": dial tcp 10.134.0.54:15021: connect: connection refused... 63m Normal Started pod/e2e-worker-w2-553753-openshift-default-558949869c-j5dc2 Container started 63m Normal Created pod/e2e-worker-w2-553753-openshift-default-558949869c-j5dc2 Container created 63m Normal Pulled pod/e2e-worker-w2-553753-openshift-default-558949869c-j5dc2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-434ce3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-553753-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-553753-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 62m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-434ce3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-88a4d9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-88a4d9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning Unhealthy pod/e2e-worker-w2-553753-openshift-default-558949869c-j5dc2 Startup probe failed: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w2-553753-openshift-default-558949869c-j5dc2 Startup probe error: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused... 62m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-2d619f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-434ce3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-2d619f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-e19850-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-e19850-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-50d738-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-50d738-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-434ce3-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 62m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-88a4d9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-88a4d9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-553753-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-553753-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-5187e1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-5187e1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Normal SuccessfulCreate replicaset/e2e-worker-w1-e19850-openshift-default-9f855bd47 Created pod: e2e-worker-w1-e19850-openshift-default-9f855bd47-84rpc 61m Normal AddedInterface pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-84rpc Add eth0 [10.134.0.63/23] from ovn-kubernetes 61m Normal Started pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-84rpc Container started 61m Normal Created pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-84rpc Container created 61m Normal ScalingReplicaSet deployment/e2e-worker-w1-e19850-openshift-default Scaled up replica set e2e-worker-w1-e19850-openshift-default-9f855bd47 from 1 to 2 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e19850-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 61m Normal Pulled pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-84rpc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Warning Unhealthy pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-84rpc Startup probe failed: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-84rpc Startup probe error: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused... 61m Normal ScalingReplicaSet deployment/e2e-worker-w0-88a4d9-openshift-default Scaled up replica set e2e-worker-w0-88a4d9-openshift-default-857757469c from 1 to 2 61m Normal SuccessfulCreate replicaset/e2e-worker-w0-88a4d9-openshift-default-857757469c Created pod: e2e-worker-w0-88a4d9-openshift-default-857757469c-shz2b 61m Normal Pulled pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-shz2b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Created pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-shz2b Container created 61m Normal Started pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-shz2b Container started 61m Normal AddedInterface pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-shz2b Add eth0 [10.133.0.47/23] from ovn-kubernetes 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-88a4d9-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 61m Warning Unhealthy pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-shz2b Startup probe failed: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-shz2b Startup probe error: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused... 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-50d738-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 61m Normal SuccessfulCreate replicaset/e2e-worker-w4-50d738-openshift-default-8d6476cbc Created pod: e2e-worker-w4-50d738-openshift-default-8d6476cbc-scbfp 61m Normal ScalingReplicaSet deployment/e2e-worker-w4-50d738-openshift-default Scaled up replica set e2e-worker-w4-50d738-openshift-default-8d6476cbc from 1 to 2 61m Normal ScalingReplicaSet deployment/e2e-worker-w2-553753-openshift-default Scaled up replica set e2e-worker-w2-553753-openshift-default-558949869c from 1 to 3 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-553753-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 61m Normal SuccessfulCreate replicaset/e2e-worker-w2-553753-openshift-default-558949869c Created pod: e2e-worker-w2-553753-openshift-default-558949869c-9k6cb 61m Normal SuccessfulCreate replicaset/e2e-worker-w2-553753-openshift-default-558949869c Created pod: e2e-worker-w2-553753-openshift-default-558949869c-ktl7d 61m Normal Created pod/e2e-worker-w2-553753-openshift-default-558949869c-9k6cb Container created 61m Normal Started pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-scbfp Container started 61m Normal Started pod/e2e-worker-w2-553753-openshift-default-558949869c-9k6cb Container started 61m Normal AddedInterface pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-scbfp Add eth0 [10.133.0.48/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-scbfp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w2-553753-openshift-default-558949869c-ktl7d Add eth0 [10.132.0.69/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w2-553753-openshift-default-558949869c-ktl7d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Created pod/e2e-worker-w2-553753-openshift-default-558949869c-ktl7d Container created 61m Normal Started pod/e2e-worker-w2-553753-openshift-default-558949869c-ktl7d Container started 61m Normal AddedInterface pod/e2e-worker-w2-553753-openshift-default-558949869c-9k6cb Add eth0 [10.133.0.49/23] from ovn-kubernetes 61m Normal Created pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-scbfp Container created 61m Normal Pulled pod/e2e-worker-w2-553753-openshift-default-558949869c-9k6cb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Warning Unhealthy pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-scbfp Startup probe failed: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-scbfp Startup probe error: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w2-553753-openshift-default-558949869c-ktl7d Startup probe failed: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w2-553753-openshift-default-558949869c-ktl7d Startup probe error: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w2-553753-openshift-default-558949869c-9k6cb Startup probe failed: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w2-553753-openshift-default-558949869c-9k6cb Startup probe error: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused... 60m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-2cwvd Readiness probe error: Get "http://10.132.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-xt8wx Readiness probe failed: Get "http://10.132.0.45:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-2cwvd Readiness probe failed: Get "http://10.132.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-xt8wx Readiness probe error: Get "http://10.132.0.45:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Normal SuccessfulCreate replicaset/e2e-worker-w4-50d738-openshift-default-8d6476cbc Created pod: e2e-worker-w4-50d738-openshift-default-8d6476cbc-nj6zd 60m Normal ScalingReplicaSet deployment/e2e-worker-w4-50d738-openshift-default Scaled up replica set e2e-worker-w4-50d738-openshift-default-8d6476cbc from 2 to 3 60m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-50d738-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 60m Normal Started pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-nj6zd Container started 60m Normal AddedInterface pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-nj6zd Add eth0 [10.132.0.83/23] from ovn-kubernetes 60m Normal Pulled pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-nj6zd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Created pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-nj6zd Container created 60m Normal Pulled pod/maas-default-gateway-openshift-default-7df854bf67-ddx5s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Started pod/maas-default-gateway-openshift-default-7df854bf67-crcsv Container started 60m Normal Started pod/maas-default-gateway-openshift-default-7df854bf67-ddx5s Container started 60m Normal AddedInterface pod/maas-default-gateway-openshift-default-7df854bf67-crcsv Add eth0 [10.133.0.50/23] from ovn-kubernetes 60m Normal Pulled pod/maas-default-gateway-openshift-default-7df854bf67-crcsv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Created pod/maas-default-gateway-openshift-default-7df854bf67-crcsv Container created 60m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7df854bf67 Created pod: maas-default-gateway-openshift-default-7df854bf67-crcsv 60m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7df854bf67 (combined from similar events): Created pod: maas-default-gateway-openshift-default-7df854bf67-ddx5s 60m Normal Created pod/maas-default-gateway-openshift-default-7df854bf67-ddx5s Container created 60m Normal AddedInterface pod/maas-default-gateway-openshift-default-7df854bf67-ddx5s Add eth0 [10.134.0.67/23] from ovn-kubernetes 60m Warning ProbeError pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-nj6zd Startup probe error: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-nj6zd Startup probe failed: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused 60m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-crcsv Startup probe error: Get "http://10.133.0.50:15021/healthz/ready": dial tcp 10.133.0.50:15021: connect: connection refused... 60m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-crcsv Startup probe failed: Get "http://10.133.0.50:15021/healthz/ready": dial tcp 10.133.0.50:15021: connect: connection refused 60m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-ddx5s Startup probe error: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused... 60m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-ddx5s Startup probe failed: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused 59m Normal ScalingReplicaSet deployment/e2e-worker-w6-2d619f-openshift-default Scaled up replica set e2e-worker-w6-2d619f-openshift-default-85b84758df from 1 to 2 59m Normal SuccessfulCreate replicaset/e2e-worker-w2-553753-openshift-default-558949869c Created pod: e2e-worker-w2-553753-openshift-default-558949869c-8l4s6 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-553753-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 59m Normal ScalingReplicaSet deployment/e2e-worker-w2-553753-openshift-default Scaled up replica set e2e-worker-w2-553753-openshift-default-558949869c from 3 to 4 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-2d619f-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-5187e1-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 59m Normal AddedInterface pod/e2e-worker-w2-553753-openshift-default-558949869c-8l4s6 Add eth0 [10.134.0.70/23] from ovn-kubernetes 59m Normal SuccessfulCreate replicaset/e2e-worker-w3-5187e1-openshift-default-946849bdf Created pod: e2e-worker-w3-5187e1-openshift-default-946849bdf-qslxh 59m Normal Created pod/e2e-worker-w2-553753-openshift-default-558949869c-8l4s6 Container created 59m Normal AddedInterface pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-nckfd Add eth0 [10.134.0.71/23] from ovn-kubernetes 59m Normal Pulled pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-nckfd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-nckfd Container created 59m Normal Started pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-nckfd Container started 59m Normal ScalingReplicaSet deployment/e2e-worker-w3-5187e1-openshift-default Scaled up replica set e2e-worker-w3-5187e1-openshift-default-946849bdf from 1 to 2 59m Normal Started pod/e2e-worker-w2-553753-openshift-default-558949869c-8l4s6 Container started 59m Normal SuccessfulCreate replicaset/e2e-worker-w6-2d619f-openshift-default-85b84758df Created pod: e2e-worker-w6-2d619f-openshift-default-85b84758df-nckfd 59m Normal Pulled pod/e2e-worker-w2-553753-openshift-default-558949869c-8l4s6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-qslxh Add eth0 [10.133.0.51/23] from ovn-kubernetes 59m Normal Pulled pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-qslxh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-qslxh Container created 59m Normal Started pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-qslxh Container started 59m Warning ProbeError pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-qslxh Startup probe error: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-qslxh Startup probe failed: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-nckfd Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-worker-w2-553753-openshift-default-558949869c-8l4s6 Startup probe failed: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w2-553753-openshift-default-558949869c-8l4s6 Startup probe error: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-nckfd Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused 59m Normal SuccessfulCreate replicaset/e2e-worker-w2-553753-openshift-default-558949869c Created pod: e2e-worker-w2-553753-openshift-default-558949869c-rdndg 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-553753-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 59m Normal ScalingReplicaSet deployment/e2e-worker-w2-553753-openshift-default Scaled up replica set e2e-worker-w2-553753-openshift-default-558949869c from 4 to 5 59m Normal Started pod/e2e-worker-w2-553753-openshift-default-558949869c-rdndg Container started 59m Normal Created pod/e2e-worker-w2-553753-openshift-default-558949869c-rdndg Container created 59m Normal Pulled pod/e2e-worker-w2-553753-openshift-default-558949869c-rdndg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/e2e-worker-w2-553753-openshift-default-558949869c-rdndg Add eth0 [10.132.0.87/23] from ovn-kubernetes 59m Warning Unhealthy pod/e2e-worker-w2-553753-openshift-default-558949869c-rdndg Startup probe failed: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w2-553753-openshift-default-558949869c-rdndg Startup probe error: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused... 59m Normal SuccessfulCreate replicaset/e2e-worker-w5-434ce3-openshift-default-58c7774759 Created pod: e2e-worker-w5-434ce3-openshift-default-58c7774759-2nggf 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-434ce3-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 59m Normal Pulled pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-2nggf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-2nggf Add eth0 [10.134.0.73/23] from ovn-kubernetes 59m Normal ScalingReplicaSet deployment/e2e-worker-w0-88a4d9-openshift-default Scaled up replica set e2e-worker-w0-88a4d9-openshift-default-857757469c from 2 to 4 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-88a4d9-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 59m Normal SuccessfulCreate replicaset/e2e-worker-w0-88a4d9-openshift-default-857757469c Created pod: e2e-worker-w0-88a4d9-openshift-default-857757469c-sjh6w 59m Normal SuccessfulCreate replicaset/e2e-worker-w0-88a4d9-openshift-default-857757469c Created pod: e2e-worker-w0-88a4d9-openshift-default-857757469c-8lzn6 59m Normal ScalingReplicaSet deployment/e2e-worker-w5-434ce3-openshift-default Scaled up replica set e2e-worker-w5-434ce3-openshift-default-58c7774759 from 1 to 2 59m Normal AddedInterface pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-8lzn6 Add eth0 [10.132.0.89/23] from ovn-kubernetes 59m Normal Started pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-sjh6w Container started 59m Normal Started pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-2nggf Container started 59m Normal Started pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-8lzn6 Container started 59m Normal Created pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-8lzn6 Container created 59m Normal Pulled pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-8lzn6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-2nggf Container created 59m Normal AddedInterface pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-sjh6w Add eth0 [10.134.0.74/23] from ovn-kubernetes 59m Normal Pulled pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-sjh6w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-sjh6w Container created 59m Warning Unhealthy pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-2nggf Startup probe failed: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused 59m Warning Unhealthy pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-8lzn6 Startup probe failed: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-8lzn6 Startup probe error: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused... 59m Warning ProbeError pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-2nggf Startup probe error: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused... 59m Warning ProbeError pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-sjh6w Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-sjh6w Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e19850-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 58m Normal ScalingReplicaSet deployment/e2e-worker-w1-e19850-openshift-default Scaled up replica set e2e-worker-w1-e19850-openshift-default-9f855bd47 from 2 to 3 58m Normal SuccessfulCreate replicaset/e2e-worker-w1-e19850-openshift-default-9f855bd47 Created pod: e2e-worker-w1-e19850-openshift-default-9f855bd47-gwqld 58m Normal Created pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-gwqld Container created 58m Normal Pulled pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-gwqld Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-gwqld Add eth0 [10.132.0.92/23] from ovn-kubernetes 58m Normal Started pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-gwqld Container started 58m Warning ProbeError pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-gwqld Startup probe error: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-gwqld Startup probe failed: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused 58m Normal SuccessfulCreate replicaset/e2e-worker-w2-553753-openshift-default-558949869c Created pod: e2e-worker-w2-553753-openshift-default-558949869c-jzgb6 58m Normal SuccessfulCreate replicaset/e2e-worker-w0-88a4d9-openshift-default-857757469c Created pod: e2e-worker-w0-88a4d9-openshift-default-857757469c-nqgqh 58m Normal SuccessfulCreate replicaset/e2e-worker-w0-88a4d9-openshift-default-857757469c Created pod: e2e-worker-w0-88a4d9-openshift-default-857757469c-x96rf 58m Normal AddedInterface pod/e2e-worker-w2-553753-openshift-default-558949869c-9fdlh Add eth0 [10.133.0.57/23] from ovn-kubernetes 58m Normal ScalingReplicaSet deployment/e2e-worker-w2-553753-openshift-default Scaled up replica set e2e-worker-w2-553753-openshift-default-558949869c from 5 to 8 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-553753-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 58m Normal Pulled pod/e2e-worker-w2-553753-openshift-default-558949869c-9fdlh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal SuccessfulCreate replicaset/e2e-worker-w2-553753-openshift-default-558949869c Created pod: e2e-worker-w2-553753-openshift-default-558949869c-9fdlh 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-88a4d9-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 58m Normal ScalingReplicaSet deployment/e2e-worker-w0-88a4d9-openshift-default Scaled up replica set e2e-worker-w0-88a4d9-openshift-default-857757469c from 4 to 6 58m Normal SuccessfulCreate replicaset/e2e-worker-w2-553753-openshift-default-558949869c Created pod: e2e-worker-w2-553753-openshift-default-558949869c-fgvq5 58m Normal AddedInterface pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-nqgqh Add eth0 [10.132.0.97/23] from ovn-kubernetes 58m Normal Pulled pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-nqgqh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/e2e-worker-w2-553753-openshift-default-558949869c-fgvq5 Container created 58m Normal Started pod/e2e-worker-w2-553753-openshift-default-558949869c-fgvq5 Container started 58m Normal AddedInterface pod/e2e-worker-w2-553753-openshift-default-558949869c-fgvq5 Add eth0 [10.132.0.96/23] from ovn-kubernetes 58m Normal Pulled pod/e2e-worker-w2-553753-openshift-default-558949869c-fgvq5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Pulled pod/e2e-worker-w2-553753-openshift-default-558949869c-jzgb6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/e2e-worker-w2-553753-openshift-default-558949869c-jzgb6 Add eth0 [10.134.0.76/23] from ovn-kubernetes 58m Normal Pulled pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-x96rf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Started pod/e2e-worker-w2-553753-openshift-default-558949869c-9fdlh Container started 58m Normal AddedInterface pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-x96rf Add eth0 [10.133.0.58/23] from ovn-kubernetes 58m Normal Created pod/e2e-worker-w2-553753-openshift-default-558949869c-9fdlh Container created 58m Normal Started pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-nqgqh Container started 58m Normal Created pod/e2e-worker-w2-553753-openshift-default-558949869c-jzgb6 Container created 58m Normal Created pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-x96rf Container created 58m Normal Started pod/e2e-worker-w2-553753-openshift-default-558949869c-jzgb6 Container started 58m Normal Started pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-x96rf Container started 58m Normal Created pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-nqgqh Container created 58m Warning Unhealthy pod/e2e-worker-w2-553753-openshift-default-558949869c-9fdlh Startup probe failed: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w2-553753-openshift-default-558949869c-9fdlh Startup probe error: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-x96rf Startup probe failed: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-nqgqh Startup probe error: Get "http://10.132.0.97:15021/healthz/ready": dial tcp 10.132.0.97:15021: connect: connection refused... 58m Warning ProbeError pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-x96rf Startup probe error: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w2-553753-openshift-default-558949869c-fgvq5 Startup probe failed: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused 58m Warning Unhealthy pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-nqgqh Startup probe failed: Get "http://10.132.0.97:15021/healthz/ready": dial tcp 10.132.0.97:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w2-553753-openshift-default-558949869c-fgvq5 Startup probe error: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w2-553753-openshift-default-558949869c-jzgb6 Startup probe failed: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w2-553753-openshift-default-558949869c-jzgb6 Startup probe error: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused... 58m Normal SuccessfulCreate replicaset/e2e-worker-w4-50d738-openshift-default-8d6476cbc Created pod: e2e-worker-w4-50d738-openshift-default-8d6476cbc-cqz7h 58m Normal SuccessfulCreate replicaset/e2e-worker-w1-e19850-openshift-default-9f855bd47 Created pod: e2e-worker-w1-e19850-openshift-default-9f855bd47-7pkkv 58m Normal Started pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-cqz7h Container started 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e19850-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 58m Normal ScalingReplicaSet deployment/e2e-worker-w1-e19850-openshift-default Scaled up replica set e2e-worker-w1-e19850-openshift-default-9f855bd47 from 3 to 6 58m Normal SuccessfulCreate replicaset/e2e-worker-w4-50d738-openshift-default-8d6476cbc Created pod: e2e-worker-w4-50d738-openshift-default-8d6476cbc-2jb6v 58m Normal Started pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-2jb6v Container started 58m Normal Created pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-2jb6v Container created 58m Normal Created pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-cqz7h Container created 58m Normal Pulled pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-2jb6v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Pulled pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-cqz7h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-cqz7h Add eth0 [10.134.0.78/23] from ovn-kubernetes 58m Normal AddedInterface pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-2jb6v Add eth0 [10.133.0.59/23] from ovn-kubernetes 58m Normal ScalingReplicaSet deployment/e2e-worker-w4-50d738-openshift-default Scaled up replica set e2e-worker-w4-50d738-openshift-default-8d6476cbc from 3 to 5 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-50d738-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 58m Normal SuccessfulCreate replicaset/e2e-worker-w1-e19850-openshift-default-9f855bd47 Created pod: e2e-worker-w1-e19850-openshift-default-9f855bd47-q8mkq 58m Normal SuccessfulCreate replicaset/e2e-worker-w1-e19850-openshift-default-9f855bd47 Created pod: e2e-worker-w1-e19850-openshift-default-9f855bd47-94n6d 58m Normal AddedInterface pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-94n6d Add eth0 [10.134.0.79/23] from ovn-kubernetes 58m Normal Started pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-q8mkq Container started 58m Normal AddedInterface pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-q8mkq Add eth0 [10.132.0.101/23] from ovn-kubernetes 58m Normal Started pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-7pkkv Container started 58m Normal Created pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-7pkkv Container created 58m Normal Pulled pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-q8mkq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Started pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-94n6d Container started 58m Normal Pulled pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-7pkkv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Pulled pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-94n6d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-7pkkv Add eth0 [10.133.0.60/23] from ovn-kubernetes 58m Normal Created pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-q8mkq Container created 58m Normal Created pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-94n6d Container created 58m Warning Unhealthy pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-2jb6v Startup probe failed: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-cqz7h Startup probe error: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-cqz7h Startup probe failed: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-2jb6v Startup probe error: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-q8mkq Startup probe failed: Get "http://10.132.0.101:15021/healthz/ready": dial tcp 10.132.0.101:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-7pkkv Startup probe error: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused... 58m Warning ProbeError pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-q8mkq Startup probe error: Get "http://10.132.0.101:15021/healthz/ready": dial tcp 10.132.0.101:15021: connect: connection refused... 58m Warning ProbeError pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-94n6d Startup probe error: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-94n6d Startup probe failed: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused 58m Warning Unhealthy pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-7pkkv Startup probe failed: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused 57m Normal ScalingReplicaSet deployment/e2e-worker-w6-2d619f-openshift-default Scaled up replica set e2e-worker-w6-2d619f-openshift-default-85b84758df from 2 to 4 57m Normal SuccessfulCreate replicaset/e2e-worker-w6-2d619f-openshift-default-85b84758df Created pod: e2e-worker-w6-2d619f-openshift-default-85b84758df-cd47x 57m Normal SuccessfulCreate replicaset/e2e-worker-w6-2d619f-openshift-default-85b84758df Created pod: e2e-worker-w6-2d619f-openshift-default-85b84758df-9h2j6 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-2d619f-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 57m Normal AddedInterface pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-cd47x Add eth0 [10.133.0.61/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-cd47x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-cd47x Container created 57m Normal Pulled pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-9h2j6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Started pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-cd47x Container started 57m Normal AddedInterface pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-9h2j6 Add eth0 [10.134.0.80/23] from ovn-kubernetes 57m Normal Created pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-9h2j6 Container created 57m Normal Started pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-9h2j6 Container started 57m Warning ProbeError pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-cd47x Startup probe error: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-cd47x Startup probe failed: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-9h2j6 Startup probe error: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-9h2j6 Startup probe failed: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused 57m Normal SuccessfulCreate replicaset/e2e-worker-w2-553753-openshift-default-558949869c (combined from similar events): Created pod: e2e-worker-w2-553753-openshift-default-558949869c-mn9sk 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e19850-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 57m Normal AddedInterface pod/e2e-worker-w2-553753-openshift-default-558949869c-mn9sk Add eth0 [10.134.0.81/23] from ovn-kubernetes 57m Normal ScalingReplicaSet deployment/e2e-worker-w3-5187e1-openshift-default Scaled up replica set e2e-worker-w3-5187e1-openshift-default-946849bdf from 2 to 3 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-5187e1-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 57m Normal SuccessfulCreate replicaset/e2e-worker-w3-5187e1-openshift-default-946849bdf Created pod: e2e-worker-w3-5187e1-openshift-default-946849bdf-z7tn8 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-553753-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 57m Normal ScalingReplicaSet deployment/e2e-worker-w2-553753-openshift-default Scaled up replica set e2e-worker-w2-553753-openshift-default-558949869c from 8 to 10 57m Normal SuccessfulCreate replicaset/e2e-worker-w1-e19850-openshift-default-9f855bd47 Created pod: e2e-worker-w1-e19850-openshift-default-9f855bd47-vm8xs 57m Normal ScalingReplicaSet deployment/e2e-worker-w1-e19850-openshift-default Scaled up replica set e2e-worker-w1-e19850-openshift-default-9f855bd47 from 6 to 7 57m Normal AddedInterface pod/e2e-worker-w2-553753-openshift-default-558949869c-85x5q Add eth0 [10.133.0.62/23] from ovn-kubernetes 57m Normal SuccessfulCreate replicaset/e2e-worker-w2-553753-openshift-default-558949869c Created pod: e2e-worker-w2-553753-openshift-default-558949869c-85x5q 57m Normal Pulled pod/e2e-worker-w2-553753-openshift-default-558949869c-85x5q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-z7tn8 Container created 57m Normal Started pod/e2e-worker-w2-553753-openshift-default-558949869c-85x5q Container started 57m Normal Created pod/e2e-worker-w2-553753-openshift-default-558949869c-mn9sk Container created 57m Normal Started pod/e2e-worker-w2-553753-openshift-default-558949869c-mn9sk Container started 57m Normal AddedInterface pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-z7tn8 Add eth0 [10.132.0.111/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-vm8xs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Pulled pod/e2e-worker-w2-553753-openshift-default-558949869c-mn9sk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Started pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-vm8xs Container started 57m Normal Created pod/e2e-worker-w2-553753-openshift-default-558949869c-85x5q Container created 57m Normal Created pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-vm8xs Container created 57m Normal Started pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-z7tn8 Container started 57m Normal AddedInterface pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-vm8xs Add eth0 [10.134.0.82/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-z7tn8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Warning ProbeError pod/e2e-worker-w2-553753-openshift-default-558949869c-85x5q Startup probe error: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w2-553753-openshift-default-558949869c-mn9sk Startup probe failed: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w2-553753-openshift-default-558949869c-mn9sk Startup probe error: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w2-553753-openshift-default-558949869c-85x5q Startup probe failed: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-vm8xs Startup probe error: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-vm8xs Startup probe failed: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-z7tn8 Startup probe error: Get "http://10.132.0.111:15021/healthz/ready": dial tcp 10.132.0.111:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-z7tn8 Startup probe failed: Get "http://10.132.0.111:15021/healthz/ready": dial tcp 10.132.0.111:15021: connect: connection refused 57m Normal ScalingReplicaSet deployment/e2e-worker-w5-434ce3-openshift-default Scaled up replica set e2e-worker-w5-434ce3-openshift-default-58c7774759 from 2 to 4 57m Normal SuccessfulCreate replicaset/e2e-worker-w5-434ce3-openshift-default-58c7774759 Created pod: e2e-worker-w5-434ce3-openshift-default-58c7774759-4wl8g 57m Normal SuccessfulCreate replicaset/e2e-worker-w5-434ce3-openshift-default-58c7774759 Created pod: e2e-worker-w5-434ce3-openshift-default-58c7774759-r4zr7 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-434ce3-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 57m Normal AddedInterface pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-4wl8g Add eth0 [10.133.0.63/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-4wl8g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-r4zr7 Add eth0 [10.134.0.84/23] from ovn-kubernetes 57m Normal Created pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-4wl8g Container created 57m Normal Started pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-4wl8g Container started 57m Warning Unhealthy pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-4kq96 Readiness probe failed: Get "http://10.133.0.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 57m Warning ProbeError pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-4kq96 Readiness probe error: Get "http://10.133.0.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 57m Normal Pulled pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-r4zr7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-r4zr7 Container created 57m Normal Started pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-r4zr7 Container started 57m Warning Unhealthy pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-4wl8g Startup probe failed: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-4wl8g Startup probe error: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-r4zr7 Startup probe error: Get "http://10.134.0.84:15021/healthz/ready": dial tcp 10.134.0.84:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-r4zr7 Startup probe failed: Get "http://10.134.0.84:15021/healthz/ready": dial tcp 10.134.0.84:15021: connect: connection refused 57m Normal SuccessfulCreate replicaset/e2e-worker-w6-2d619f-openshift-default-85b84758df Created pod: e2e-worker-w6-2d619f-openshift-default-85b84758df-qzh84 57m Normal SuccessfulCreate replicaset/e2e-worker-w6-2d619f-openshift-default-85b84758df Created pod: e2e-worker-w6-2d619f-openshift-default-85b84758df-cj9f5 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-2d619f-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 57m Normal SuccessfulCreate replicaset/e2e-worker-w6-2d619f-openshift-default-85b84758df Created pod: e2e-worker-w6-2d619f-openshift-default-85b84758df-gwb2l 57m Normal ScalingReplicaSet deployment/e2e-worker-w6-2d619f-openshift-default Scaled up replica set e2e-worker-w6-2d619f-openshift-default-85b84758df from 4 to 7 57m Normal Created pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-gwb2l Container created 57m Normal Pulled pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-cj9f5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-f72v5 Add eth0 [10.134.0.88/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-f72v5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Pulled pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-gwb2l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-gwb2l Add eth0 [10.134.0.87/23] from ovn-kubernetes 57m Normal SuccessfulCreate replicaset/e2e-worker-w1-e19850-openshift-default-9f855bd47 (combined from similar events): Created pod: e2e-worker-w1-e19850-openshift-default-9f855bd47-f72v5 57m Normal SuccessfulCreate replicaset/e2e-worker-w1-e19850-openshift-default-9f855bd47 Created pod: e2e-worker-w1-e19850-openshift-default-9f855bd47-hz2jq 57m Normal Started pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-gwb2l Container started 57m Normal AddedInterface pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-qzh84 Add eth0 [10.132.0.119/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-qzh84 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-qzh84 Container created 57m Normal Started pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-qzh84 Container started 57m Normal Started pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-cj9f5 Container started 57m Normal Created pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-f72v5 Container created 57m Normal Started pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-f72v5 Container started 57m Normal AddedInterface pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-hz2jq Add eth0 [10.132.0.120/23] from ovn-kubernetes 57m Normal ScalingReplicaSet deployment/e2e-worker-w1-e19850-openshift-default Scaled up replica set e2e-worker-w1-e19850-openshift-default-9f855bd47 from 7 to 10 57m Normal Created pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-cj9f5 Container created 57m Normal Started pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-hz2jq Container started 57m Normal Created pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-hz2jq Container created 57m Normal Pulled pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-hz2jq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal SuccessfulCreate replicaset/e2e-worker-w1-e19850-openshift-default-9f855bd47 Created pod: e2e-worker-w1-e19850-openshift-default-9f855bd47-thldf 57m Normal AddedInterface pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-cj9f5 Add eth0 [10.133.0.64/23] from ovn-kubernetes 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e19850-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 57m Normal Started pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-thldf Container started 57m Normal Created pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-thldf Container created 57m Normal Pulled pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-thldf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-thldf Add eth0 [10.133.0.65/23] from ovn-kubernetes 57m Normal AddedLabel gateway/e2e-migrate-b7b10505-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-b7b10505-gw 57m Normal SuccessfulCreate replicaset/e2e-migrate-b7b10505-gw-openshift-default-5778588977 Created pod: e2e-migrate-b7b10505-gw-openshift-default-5778588977-67psd 57m Normal ScalingReplicaSet deployment/e2e-migrate-b7b10505-gw-openshift-default Scaled up replica set e2e-migrate-b7b10505-gw-openshift-default-5778588977 from 0 to 1 57m Warning ProbeError pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-cj9f5 Startup probe error: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused... 57m Normal AddedInterface pod/e2e-migrate-b7b10505-gw-openshift-default-5778588977-67psd Add eth0 [10.134.0.89/23] from ovn-kubernetes 57m Warning Unhealthy pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-cj9f5 Startup probe failed: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused 57m Normal Pulled pod/e2e-migrate-b7b10505-gw-openshift-default-5778588977-67psd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Warning ProbeError pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-qzh84 Startup probe error: Get "http://10.132.0.119:15021/healthz/ready": dial tcp 10.132.0.119:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-qzh84 Startup probe failed: Get "http://10.132.0.119:15021/healthz/ready": dial tcp 10.132.0.119:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-thldf Startup probe failed: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused 57m Normal Created pod/e2e-migrate-b7b10505-gw-openshift-default-5778588977-67psd Container created 57m Normal Started pod/e2e-migrate-b7b10505-gw-openshift-default-5778588977-67psd Container started 57m Warning ProbeError pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-thldf Startup probe error: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-gwb2l Startup probe error: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-hz2jq Startup probe failed: Get "http://10.132.0.120:15021/healthz/ready": dial tcp 10.132.0.120:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-hz2jq Startup probe error: Get "http://10.132.0.120:15021/healthz/ready": dial tcp 10.132.0.120:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-gwb2l Startup probe failed: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-f72v5 Startup probe failed: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-f72v5 Startup probe error: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused... 57m Normal ScalingReplicaSet deployment/e2e-ait-1b279244-openshift-default Scaled up replica set e2e-ait-1b279244-openshift-default-74dd88dd99 from 0 to 1 57m Normal AddedLabel gateway/e2e-ait-1b279244 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-1b279244 57m Normal SuccessfulCreate replicaset/e2e-ait-1b279244-openshift-default-74dd88dd99 Created pod: e2e-ait-1b279244-openshift-default-74dd88dd99-6px88 57m Normal Started pod/e2e-ait-1b279244-openshift-default-74dd88dd99-6px88 Container started 57m Normal Created pod/e2e-ait-1b279244-openshift-default-74dd88dd99-6px88 Container created 57m Normal Pulled pod/e2e-ait-1b279244-openshift-default-74dd88dd99-6px88 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/e2e-ait-1b279244-openshift-default-74dd88dd99-6px88 Add eth0 [10.134.0.90/23] from ovn-kubernetes 57m Warning FailedCreate replicaset/payload-processing-e2e-migrate-b7b10505-5d8bb99fb Error creating: pods "payload-processing-e2e-migrate-b7b10505-5d8bb99fb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-b7b10505: serviceaccount "payload-processing-e2e-migrate-b7b10505" not found 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-b7b10505 Scaled up replica set payload-processing-e2e-migrate-b7b10505-5d8bb99fb from 0 to 1 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-b7b10505-775cb59b75 Error creating: pods "payload-pre-processing-e2e-migrate-b7b10505-775cb59b75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-b7b10505: serviceaccount "payload-processing-e2e-migrate-b7b10505" not found 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-b7b10505 Scaled up replica set payload-pre-processing-e2e-migrate-b7b10505-775cb59b75 from 0 to 1 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-b7b10505-775cb59b75 Created pod: payload-pre-processing-e2e-migrate-b7b10505-775cb59b75-96cjh 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-b7b10505-5d8bb99fb Created pod: payload-processing-e2e-migrate-b7b10505-5d8bb99fb-q2vm4 57m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-b7b10505-775cb59b75-96cjh Add eth0 [10.134.0.91/23] from ovn-kubernetes 57m Normal AddedInterface pod/payload-processing-e2e-migrate-b7b10505-5d8bb99fb-q2vm4 Add eth0 [10.134.0.92/23] from ovn-kubernetes 57m Normal Pulled pod/payload-processing-e2e-migrate-b7b10505-5d8bb99fb-q2vm4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Created pod/payload-processing-e2e-migrate-b7b10505-5d8bb99fb-q2vm4 Container created 57m Normal Started pod/payload-processing-e2e-migrate-b7b10505-5d8bb99fb-q2vm4 Container started 57m Normal Pulled pod/payload-pre-processing-e2e-migrate-b7b10505-775cb59b75-96cjh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Created pod/payload-pre-processing-e2e-migrate-b7b10505-775cb59b75-96cjh Container created 57m Normal Started pod/payload-pre-processing-e2e-migrate-b7b10505-775cb59b75-96cjh Container started 56m Normal SuccessfulCreate replicaset/e2e-worker-w4-50d738-openshift-default-8d6476cbc Created pod: e2e-worker-w4-50d738-openshift-default-8d6476cbc-jrmj4 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-50d738-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 56m Normal ScalingReplicaSet deployment/e2e-worker-w4-50d738-openshift-default Scaled up replica set e2e-worker-w4-50d738-openshift-default-8d6476cbc from 5 to 8 56m Normal SuccessfulCreate replicaset/e2e-worker-w4-50d738-openshift-default-8d6476cbc Created pod: e2e-worker-w4-50d738-openshift-default-8d6476cbc-6wd75 56m Normal SuccessfulCreate replicaset/e2e-worker-w4-50d738-openshift-default-8d6476cbc Created pod: e2e-worker-w4-50d738-openshift-default-8d6476cbc-6skbz 56m Normal AddedInterface pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-hdk26 Add eth0 [10.132.0.127/23] from ovn-kubernetes 56m Normal Started pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-6wd75 Container started 56m Normal AddedInterface pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-ztnxm Add eth0 [10.133.0.66/23] from ovn-kubernetes 56m Normal Pulled pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-ztnxm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal SuccessfulCreate replicaset/e2e-worker-w0-88a4d9-openshift-default-857757469c (combined from similar events): Created pod: e2e-worker-w0-88a4d9-openshift-default-857757469c-vlq4m 56m Normal ScalingReplicaSet deployment/e2e-worker-w0-88a4d9-openshift-default Scaled up replica set e2e-worker-w0-88a4d9-openshift-default-857757469c from 6 to 10 56m Normal SuccessfulCreate replicaset/e2e-worker-w0-88a4d9-openshift-default-857757469c Created pod: e2e-worker-w0-88a4d9-openshift-default-857757469c-ztnxm 56m Normal AddedInterface pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-6skbz Add eth0 [10.132.0.126/23] from ovn-kubernetes 56m Normal Pulled pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-6skbz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-6skbz Container created 56m Normal Started pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-6skbz Container started 56m Normal SuccessfulCreate replicaset/e2e-worker-w0-88a4d9-openshift-default-857757469c Created pod: e2e-worker-w0-88a4d9-openshift-default-857757469c-hdk26 56m Normal AddedInterface pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-6wd75 Add eth0 [10.134.0.93/23] from ovn-kubernetes 56m Normal SuccessfulCreate replicaset/e2e-worker-w0-88a4d9-openshift-default-857757469c Created pod: e2e-worker-w0-88a4d9-openshift-default-857757469c-g4fx2 56m Normal Started pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-jrmj4 Container started 56m Normal Pulled pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-6wd75 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-88a4d9-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 56m Normal Created pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-jrmj4 Container created 56m Normal Pulled pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-jrmj4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-jrmj4 Add eth0 [10.132.0.125/23] from ovn-kubernetes 56m Normal Created pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-6wd75 Container created 56m Normal AddedInterface pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-g4fx2 Add eth0 [10.134.0.94/23] from ovn-kubernetes 56m Normal Created pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-vlq4m Container created 56m Normal Created pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-hdk26 Container created 56m Normal Started pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-vlq4m Container started 56m Normal Pulled pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-vlq4m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-hdk26 Container started 56m Normal Pulled pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-g4fx2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Pulled pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-hdk26 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-ztnxm Container created 56m Normal Created pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-g4fx2 Container created 56m Normal Started pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-g4fx2 Container started 56m Normal Started pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-ztnxm Container started 56m Normal AddedInterface pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-vlq4m Add eth0 [10.134.0.95/23] from ovn-kubernetes 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-1b279244-855ccd57cb Created pod: payload-processing-e2e-ait-1b279244-855ccd57cb-l9wh4 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-1b279244-55d699df5f Created pod: payload-pre-processing-e2e-ait-1b279244-55d699df5f-8hq7t 56m Warning FailedCreate replicaset/payload-processing-e2e-ait-1b279244-855ccd57cb Error creating: pods "payload-processing-e2e-ait-1b279244-855ccd57cb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-1b279244: serviceaccount "payload-processing-e2e-ait-1b279244" not found 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-1b279244-55d699df5f Error creating: pods "payload-pre-processing-e2e-ait-1b279244-55d699df5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-1b279244: serviceaccount "payload-processing-e2e-ait-1b279244" not found 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-1b279244 Scaled up replica set payload-pre-processing-e2e-ait-1b279244-55d699df5f from 0 to 1 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-1b279244 Scaled up replica set payload-processing-e2e-ait-1b279244-855ccd57cb from 0 to 1 56m Normal Started pod/payload-pre-processing-e2e-ait-1b279244-55d699df5f-8hq7t Container started 56m Normal AddedInterface pod/payload-pre-processing-e2e-ait-1b279244-55d699df5f-8hq7t Add eth0 [10.132.0.128/23] from ovn-kubernetes 56m Normal Created pod/payload-pre-processing-e2e-ait-1b279244-55d699df5f-8hq7t Container created 56m Normal Pulled pod/payload-pre-processing-e2e-ait-1b279244-55d699df5f-8hq7t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Created pod/payload-processing-e2e-ait-1b279244-855ccd57cb-l9wh4 Container created 56m Normal Pulled pod/payload-processing-e2e-ait-1b279244-855ccd57cb-l9wh4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/payload-processing-e2e-ait-1b279244-855ccd57cb-l9wh4 Add eth0 [10.134.0.96/23] from ovn-kubernetes 56m Warning Unhealthy pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-6skbz Startup probe failed: Get "http://10.132.0.126:15021/healthz/ready": dial tcp 10.132.0.126:15021: connect: connection refused 56m Warning Unhealthy pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-jrmj4 Startup probe failed: Get "http://10.132.0.125:15021/healthz/ready": dial tcp 10.132.0.125:15021: connect: connection refused 56m Normal Started pod/payload-processing-e2e-ait-1b279244-855ccd57cb-l9wh4 Container started 56m Warning ProbeError pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-6skbz Startup probe error: Get "http://10.132.0.126:15021/healthz/ready": dial tcp 10.132.0.126:15021: connect: connection refused... 56m Warning ProbeError pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-jrmj4 Startup probe error: Get "http://10.132.0.125:15021/healthz/ready": dial tcp 10.132.0.125:15021: connect: connection refused... 56m Warning ProbeError pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-ztnxm Startup probe error: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-ztnxm Startup probe failed: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-hdk26 Startup probe error: Get "http://10.132.0.127:15021/healthz/ready": dial tcp 10.132.0.127:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-hdk26 Startup probe failed: Get "http://10.132.0.127:15021/healthz/ready": dial tcp 10.132.0.127:15021: connect: connection refused 56m Normal SuccessfulCreate replicaset/e2e-ait-51bfa2e1-openshift-default-6dcb64d7dc Created pod: e2e-ait-51bfa2e1-openshift-default-6dcb64d7dc-7tnb9 56m Normal ScalingReplicaSet deployment/e2e-ait-51bfa2e1-openshift-default Scaled up replica set e2e-ait-51bfa2e1-openshift-default-6dcb64d7dc from 0 to 1 56m Normal AddedLabel gateway/e2e-ait-51bfa2e1 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-51bfa2e1 56m Warning Unhealthy pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-6wd75 Startup probe failed: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-6wd75 Startup probe error: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-g4fx2 Startup probe failed: Get "http://10.134.0.94:15021/healthz/ready": dial tcp 10.134.0.94:15021: connect: connection refused 56m Warning Unhealthy pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-vlq4m Startup probe failed: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-vlq4m Startup probe error: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-6wd75 Startup probe failed: Get "http://10.134.0.93:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-g4fx2 Startup probe error: Get "http://10.134.0.94:15021/healthz/ready": dial tcp 10.134.0.94:15021: connect: connection refused... 56m Warning ProbeError pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-6wd75 Startup probe error: Get "http://10.134.0.93:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Normal AddedInterface pod/e2e-ait-51bfa2e1-openshift-default-6dcb64d7dc-7tnb9 Add eth0 [10.134.0.97/23] from ovn-kubernetes 56m Warning Unhealthy pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-r4zr7 Readiness probe failed: Get "http://10.134.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-r4zr7 Readiness probe error: Get "http://10.134.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Normal Pulled pod/e2e-ait-51bfa2e1-openshift-default-6dcb64d7dc-7tnb9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-ait-51bfa2e1-openshift-default-6dcb64d7dc-7tnb9 Container created 56m Normal Started pod/e2e-ait-51bfa2e1-openshift-default-6dcb64d7dc-7tnb9 Container started 56m Warning ProbeError pod/e2e-worker-w2-553753-openshift-default-558949869c-j5dc2 Readiness probe error: Get "http://10.134.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Warning Unhealthy pod/e2e-worker-w2-553753-openshift-default-558949869c-j5dc2 Readiness probe failed: Get "http://10.134.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-ait-51bfa2e1-openshift-default-6dcb64d7dc-7tnb9 Startup probe error: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-ait-51bfa2e1-openshift-default-6dcb64d7dc-7tnb9 Startup probe failed: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused 56m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-56cz2 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-1b279244-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-1b279244-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-56cz2 Add eth0 [10.132.0.133/23] from ovn-kubernetes 56m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-56cz2 Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 56m Warning FailedCreate replicaset/payload-processing-e2e-ait-51bfa2e1-6954676576 Error creating: pods "payload-processing-e2e-ait-51bfa2e1-6954676576-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-51bfa2e1: serviceaccount "payload-processing-e2e-ait-51bfa2e1" not found 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-51bfa2e1 Scaled up replica set payload-processing-e2e-ait-51bfa2e1-6954676576 from 0 to 1 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-51bfa2e1-6954676576 Created pod: payload-processing-e2e-ait-51bfa2e1-6954676576-kr8sq 56m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-56cz2 Container created 56m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-56cz2 Container started 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-51bfa2e1 Scaled up replica set payload-pre-processing-e2e-ait-51bfa2e1-559ff7c87f from 0 to 1 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-51bfa2e1-559ff7c87f Created pod: payload-pre-processing-e2e-ait-51bfa2e1-559ff7c87f-ckt8j 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-51bfa2e1-559ff7c87f Error creating: pods "payload-pre-processing-e2e-ait-51bfa2e1-559ff7c87f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-51bfa2e1: serviceaccount "payload-processing-e2e-ait-51bfa2e1" not found 56m Normal AddedInterface pod/payload-pre-processing-e2e-ait-51bfa2e1-559ff7c87f-ckt8j Add eth0 [10.134.0.99/23] from ovn-kubernetes 56m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-56cz2 Readiness probe error: HTTP probe failed with statuscode: 503... 56m Normal Pulling pod/payload-processing-e2e-ait-51bfa2e1-6954676576-kr8sq Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 56m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-56cz2 Readiness probe failed: HTTP probe failed with statuscode: 503 56m Normal Started pod/payload-pre-processing-e2e-ait-51bfa2e1-559ff7c87f-ckt8j Container started 56m Normal AddedInterface pod/payload-processing-e2e-ait-51bfa2e1-6954676576-kr8sq Add eth0 [10.133.0.68/23] from ovn-kubernetes 56m Normal Pulled pod/payload-pre-processing-e2e-ait-51bfa2e1-559ff7c87f-ckt8j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Created pod/payload-pre-processing-e2e-ait-51bfa2e1-559ff7c87f-ckt8j Container created 56m Normal Created pod/payload-processing-e2e-ait-51bfa2e1-6954676576-kr8sq Container created 56m Normal Started pod/payload-processing-e2e-ait-51bfa2e1-6954676576-kr8sq Container started 56m Normal Pulled pod/payload-processing-e2e-ait-51bfa2e1-6954676576-kr8sq Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.109s (2.109s including waiting). Image size: 125353963 bytes. 56m Normal SuccessfulCreate replicaset/e2e-worker-w6-2d619f-openshift-default-85b84758df Created pod: e2e-worker-w6-2d619f-openshift-default-85b84758df-2gnhr 56m Normal ScalingReplicaSet deployment/e2e-worker-w3-5187e1-openshift-default Scaled up replica set e2e-worker-w3-5187e1-openshift-default-946849bdf from 3 to 6 56m Normal ScalingReplicaSet deployment/e2e-worker-w5-434ce3-openshift-default Scaled up replica set e2e-worker-w5-434ce3-openshift-default-58c7774759 from 4 to 8 56m Normal SuccessfulCreate replicaset/e2e-worker-w5-434ce3-openshift-default-58c7774759 Created pod: e2e-worker-w5-434ce3-openshift-default-58c7774759-mqzl2 56m Normal SuccessfulCreate replicaset/e2e-worker-w6-2d619f-openshift-default-85b84758df Created pod: e2e-worker-w6-2d619f-openshift-default-85b84758df-fsnjn 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-5187e1-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 56m Normal SuccessfulCreate replicaset/e2e-worker-w5-434ce3-openshift-default-58c7774759 Created pod: e2e-worker-w5-434ce3-openshift-default-58c7774759-wmp98 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-2d619f-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 56m Normal SuccessfulCreate replicaset/e2e-worker-w6-2d619f-openshift-default-85b84758df (combined from similar events): Created pod: e2e-worker-w6-2d619f-openshift-default-85b84758df-btjw2 56m Normal ScalingReplicaSet deployment/e2e-worker-w6-2d619f-openshift-default Scaled up replica set e2e-worker-w6-2d619f-openshift-default-85b84758df from 7 to 10 56m Normal SuccessfulCreate replicaset/e2e-worker-w3-5187e1-openshift-default-946849bdf Created pod: e2e-worker-w3-5187e1-openshift-default-946849bdf-g45p4 56m Normal SuccessfulCreate replicaset/e2e-worker-w3-5187e1-openshift-default-946849bdf Created pod: e2e-worker-w3-5187e1-openshift-default-946849bdf-wtzl9 56m Normal SuccessfulCreate replicaset/e2e-worker-w3-5187e1-openshift-default-946849bdf Created pod: e2e-worker-w3-5187e1-openshift-default-946849bdf-q88pc 56m Normal SuccessfulCreate replicaset/e2e-worker-w5-434ce3-openshift-default-58c7774759 Created pod: e2e-worker-w5-434ce3-openshift-default-58c7774759-bsswj 56m Normal SuccessfulCreate replicaset/e2e-worker-w5-434ce3-openshift-default-58c7774759 Created pod: e2e-worker-w5-434ce3-openshift-default-58c7774759-m2b7t 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-434ce3-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 56m Normal Pulled pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-fsnjn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Pulled pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-g45p4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-mqzl2 Container started 56m Normal Pulled pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-mqzl2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-mqzl2 Add eth0 [10.133.0.72/23] from ovn-kubernetes 56m Normal Started pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-btjw2 Container started 56m Normal AddedInterface pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-g45p4 Add eth0 [10.132.0.135/23] from ovn-kubernetes 56m Normal Started pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-m2b7t Container started 56m Normal Created pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-m2b7t Container created 56m Normal Pulled pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-m2b7t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-wmp98 Container started 56m Normal AddedInterface pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-m2b7t Add eth0 [10.133.0.71/23] from ovn-kubernetes 56m Normal Created pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-wmp98 Container created 56m Normal Pulled pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-wmp98 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-2gnhr Container started 56m Normal Created pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-2gnhr Container created 56m Normal Pulled pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-2gnhr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-mqzl2 Container created 56m Normal Created pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-g45p4 Container created 56m Normal Started pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-g45p4 Container started 56m Normal AddedInterface pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-wmp98 Add eth0 [10.134.0.102/23] from ovn-kubernetes 56m Normal Started pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-wtzl9 Container started 56m Normal AddedInterface pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-2gnhr Add eth0 [10.133.0.69/23] from ovn-kubernetes 56m Normal Started pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-q88pc Container started 56m Normal Created pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-btjw2 Container created 56m Normal Created pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-q88pc Container created 56m Normal Pulled pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-q88pc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-q88pc Add eth0 [10.133.0.70/23] from ovn-kubernetes 56m Normal AddedInterface pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-fsnjn Add eth0 [10.134.0.100/23] from ovn-kubernetes 56m Normal Created pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-bsswj Container created 56m Normal Pulled pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-bsswj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-wtzl9 Container created 56m Normal AddedInterface pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-bsswj Add eth0 [10.132.0.136/23] from ovn-kubernetes 56m Normal AddedInterface pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-btjw2 Add eth0 [10.132.0.134/23] from ovn-kubernetes 56m Normal Pulled pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-btjw2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Pulled pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-wtzl9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-wtzl9 Add eth0 [10.134.0.101/23] from ovn-kubernetes 56m Normal Started pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-fsnjn Container started 56m Normal Created pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-fsnjn Container created 56m Normal Started pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-bsswj Container started 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-b7b10505-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-b7b10505-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning ProbeError pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-g45p4 Startup probe error: Get "http://10.132.0.135:15021/healthz/ready": dial tcp 10.132.0.135:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-btjw2 Startup probe failed: Get "http://10.132.0.134:15021/healthz/ready": dial tcp 10.132.0.134:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-btjw2 Startup probe error: Get "http://10.132.0.134:15021/healthz/ready": dial tcp 10.132.0.134:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-g45p4 Startup probe failed: Get "http://10.132.0.135:15021/healthz/ready": dial tcp 10.132.0.135:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-q88pc Startup probe error: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-bsswj Startup probe failed: Get "http://10.132.0.136:15021/healthz/ready": dial tcp 10.132.0.136:15021: connect: connection refused 56m Warning Unhealthy pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-q88pc Startup probe failed: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-bsswj Startup probe error: Get "http://10.132.0.136:15021/healthz/ready": dial tcp 10.132.0.136:15021: connect: connection refused... 56m Warning ProbeError pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-wtzl9 Startup probe error: Get "http://10.134.0.101:15021/healthz/ready": dial tcp 10.134.0.101:15021: connect: connection refused... 56m Warning ProbeError pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-wmp98 Startup probe error: Get "http://10.134.0.102:15021/healthz/ready": dial tcp 10.134.0.102:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-wtzl9 Startup probe failed: Get "http://10.134.0.101:15021/healthz/ready": dial tcp 10.134.0.101:15021: connect: connection refused 56m Warning Unhealthy pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-mqzl2 Startup probe failed: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-1b279244-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning ProbeError pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-fsnjn Startup probe error: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused... 56m Warning ProbeError pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-2gnhr Startup probe error: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused... 56m Warning ProbeError pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-mqzl2 Startup probe error: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-wmp98 Startup probe failed: Get "http://10.134.0.102:15021/healthz/ready": dial tcp 10.134.0.102:15021: connect: connection refused 56m Warning Unhealthy pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-fsnjn Startup probe failed: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused 56m Warning Unhealthy pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-2gnhr Startup probe failed: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused 56m Warning Unhealthy pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-m2b7t Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-m2b7t Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused... 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-1b279244-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-51bfa2e1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-51bfa2e1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Normal SuccessfulCreate replicaset/e2e-mt-53879c92-openshift-default-67dfffdb77 Created pod: e2e-mt-53879c92-openshift-default-67dfffdb77-c47vg 56m Normal ScalingReplicaSet deployment/e2e-mt-53879c92-openshift-default Scaled up replica set e2e-mt-53879c92-openshift-default-67dfffdb77 from 0 to 1 56m Normal AddedLabel gateway/e2e-mt-53879c92 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-53879c92 56m Normal AddedInterface pod/e2e-mt-53879c92-openshift-default-67dfffdb77-c47vg Add eth0 [10.133.0.73/23] from ovn-kubernetes 56m Normal SuccessfulCreate replicaset/e2e-worker-w5-434ce3-openshift-default-58c7774759 Created pod: e2e-worker-w5-434ce3-openshift-default-58c7774759-vgf7q 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-434ce3-openshift-default New size: 10; reason: 56m Normal ScalingReplicaSet deployment/e2e-worker-w5-434ce3-openshift-default Scaled up replica set e2e-worker-w5-434ce3-openshift-default-58c7774759 from 8 to 10 56m Normal SuccessfulCreate replicaset/e2e-worker-w5-434ce3-openshift-default-58c7774759 (combined from similar events): Created pod: e2e-worker-w5-434ce3-openshift-default-58c7774759-6t8qk 56m Normal Started pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-6t8qk Container started 56m Normal Started pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-vgf7q Container started 56m Normal Created pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-vgf7q Container created 56m Normal Killing pod/payload-pre-processing-e2e-migrate-b7b10505-775cb59b75-96cjh Stopping container payload-pre-processing 56m Normal Pulled pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-vgf7q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-vgf7q Add eth0 [10.132.0.140/23] from ovn-kubernetes 56m Normal AddedInterface pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-6t8qk Add eth0 [10.133.0.74/23] from ovn-kubernetes 56m Normal Killing pod/payload-processing-e2e-migrate-b7b10505-5d8bb99fb-q2vm4 Stopping container payload-processing 56m Normal Pulled pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-6t8qk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-6t8qk Container created 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-53879c92 Scaled up replica set payload-processing-e2e-mt-53879c92-6bf559b69b from 0 to 1 56m Warning FailedCreate replicaset/payload-processing-e2e-mt-53879c92-6bf559b69b Error creating: pods "payload-processing-e2e-mt-53879c92-6bf559b69b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-53879c92: serviceaccount "payload-processing-e2e-mt-53879c92" not found 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-53879c92-668d9d8855 Error creating: pods "payload-pre-processing-e2e-mt-53879c92-668d9d8855-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-53879c92: serviceaccount "payload-processing-e2e-mt-53879c92" not found 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-53879c92 Scaled up replica set payload-pre-processing-e2e-mt-53879c92-668d9d8855 from 0 to 1 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-53879c92-668d9d8855 Created pod: payload-pre-processing-e2e-mt-53879c92-668d9d8855-74s4z 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-53879c92-6bf559b69b Created pod: payload-processing-e2e-mt-53879c92-6bf559b69b-plpwn 56m Normal Killing pod/payload-pre-processing-e2e-ait-1b279244-55d699df5f-8hq7t Stopping container payload-pre-processing 56m Normal AddedInterface pod/payload-pre-processing-e2e-mt-53879c92-668d9d8855-74s4z Add eth0 [10.134.0.105/23] from ovn-kubernetes 56m Normal Started pod/payload-processing-e2e-mt-53879c92-6bf559b69b-plpwn Container started 56m Normal Created pod/payload-processing-e2e-mt-53879c92-6bf559b69b-plpwn Container created 56m Normal Killing pod/payload-processing-e2e-ait-1b279244-855ccd57cb-l9wh4 Stopping container payload-processing 56m Normal Pulled pod/payload-processing-e2e-mt-53879c92-6bf559b69b-plpwn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Pulled pod/payload-pre-processing-e2e-mt-53879c92-668d9d8855-74s4z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/payload-processing-e2e-mt-53879c92-6bf559b69b-plpwn Add eth0 [10.134.0.104/23] from ovn-kubernetes 56m Normal Created pod/payload-pre-processing-e2e-mt-53879c92-668d9d8855-74s4z Container created 56m Warning Unhealthy pod/payload-processing-e2e-migrate-b7b10505-5d8bb99fb-q2vm4 Readiness probe failed: timeout: failed to connect service "10.134.0.92:50052" within 1s: context deadline exceeded 56m Warning Unhealthy pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-vgf7q Startup probe failed: Get "http://10.132.0.140:15021/healthz/ready": dial tcp 10.132.0.140:15021: connect: connection refused 56m Warning Unhealthy pod/e2e-migrate-b7b10505-gw-openshift-default-5778588977-67psd Readiness probe failed: HTTP probe failed with statuscode: 503 56m Normal Killing pod/e2e-migrate-b7b10505-gw-openshift-default-5778588977-67psd Stopping container istio-proxy 56m Warning ProbeError pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-vgf7q Startup probe error: Get "http://10.132.0.140:15021/healthz/ready": dial tcp 10.132.0.140:15021: connect: connection refused... 56m Warning ProbeError pod/e2e-migrate-b7b10505-gw-openshift-default-5778588977-67psd Readiness probe error: HTTP probe failed with statuscode: 503... 56m Warning ProbeError pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-2nggf Readiness probe error: Get "http://10.134.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Normal AddedLabel gateway/e2e-ait-d05919ef Added label istio.io/rev=openshift-gateway to gateway e2e-ait-d05919ef 56m Warning Unhealthy pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-2nggf Readiness probe failed: Get "http://10.134.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Normal Started pod/payload-pre-processing-e2e-mt-53879c92-668d9d8855-74s4z Container started 56m Warning Unhealthy pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-6t8qk Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused 56m Normal ScalingReplicaSet deployment/e2e-ait-d05919ef-openshift-default Scaled up replica set e2e-ait-d05919ef-openshift-default-7597d49499 from 0 to 1 56m Warning ProbeError pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-6t8qk Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused... 56m Normal SuccessfulCreate replicaset/e2e-ait-d05919ef-openshift-default-7597d49499 Created pod: e2e-ait-d05919ef-openshift-default-7597d49499-9f7fq 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-51bfa2e1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-51bfa2e1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning ProbeError pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-wmp98 Readiness probe error: Get "http://10.134.0.102:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Warning Unhealthy pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-wtzl9 Readiness probe failed: Get "http://10.134.0.101:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-wtzl9 Readiness probe error: Get "http://10.134.0.101:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Normal AddedInterface pod/e2e-ait-d05919ef-openshift-default-7597d49499-9f7fq Add eth0 [10.133.0.75/23] from ovn-kubernetes 56m Warning Unhealthy pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-wmp98 Readiness probe failed: Get "http://10.134.0.102:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning Unhealthy pod/payload-pre-processing-e2e-ait-1b279244-55d699df5f-8hq7t Readiness probe failed: timeout: failed to connect service "10.132.0.128:50052" within 1s: context deadline exceeded 56m Warning Unhealthy pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-vgf7q Startup probe failed: Get "http://10.132.0.140:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-vgf7q Startup probe error: Get "http://10.132.0.140:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 55m Warning Unhealthy pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-6t8qk Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 55m Warning Unhealthy pod/payload-processing-7bb9489676-bnqlr Readiness probe failed: timeout: health rpc did not complete within 1s 55m Warning ProbeError pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-6t8qk Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 55m Warning Unhealthy pod/payload-processing-e2e-ait-1b279244-855ccd57cb-l9wh4 Readiness probe failed: timeout: failed to connect service "10.134.0.96:50052" within 1s: context deadline exceeded 55m Normal Killing pod/e2e-ait-1b279244-openshift-default-74dd88dd99-6px88 Stopping container istio-proxy 55m Normal AddedLabel gateway/e2e-derive-cbe6e297 Added label istio.io/rev=openshift-gateway to gateway e2e-derive-cbe6e297 55m Normal ScalingReplicaSet deployment/e2e-derive-cbe6e297-openshift-default Scaled up replica set e2e-derive-cbe6e297-openshift-default-795c76bc85 from 0 to 1 55m Normal SuccessfulCreate replicaset/e2e-derive-cbe6e297-openshift-default-795c76bc85 Created pod: e2e-derive-cbe6e297-openshift-default-795c76bc85-vgf74 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-d05919ef Scaled up replica set payload-processing-e2e-ait-d05919ef-98cb9cd48 from 0 to 1 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-d05919ef-98cb9cd48 Created pod: payload-processing-e2e-ait-d05919ef-98cb9cd48-4lr79 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-d05919ef-5fbd9c9b67 Error creating: pods "payload-pre-processing-e2e-ait-d05919ef-5fbd9c9b67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-d05919ef: serviceaccount "payload-processing-e2e-ait-d05919ef" not found 55m Warning FailedCreate replicaset/payload-processing-e2e-ait-d05919ef-98cb9cd48 Error creating: pods "payload-processing-e2e-ait-d05919ef-98cb9cd48-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-d05919ef: serviceaccount "payload-processing-e2e-ait-d05919ef" not found 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-d05919ef-5fbd9c9b67 Created pod: payload-pre-processing-e2e-ait-d05919ef-5fbd9c9b67-rb26m 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-d05919ef Scaled up replica set payload-pre-processing-e2e-ait-d05919ef-5fbd9c9b67 from 0 to 1 55m Normal Pulled pod/payload-pre-processing-e2e-ait-d05919ef-5fbd9c9b67-rb26m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/payload-pre-processing-e2e-ait-d05919ef-5fbd9c9b67-rb26m Add eth0 [10.133.0.76/23] from ovn-kubernetes 55m Normal AddedInterface pod/payload-processing-e2e-ait-d05919ef-98cb9cd48-4lr79 Add eth0 [10.133.0.77/23] from ovn-kubernetes 55m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w1-e19850-5898685ff-kd5l5 Readiness probe failed: timeout: health rpc did not complete within 1s 55m Normal Created pod/payload-processing-e2e-ait-d05919ef-98cb9cd48-4lr79 Container created 55m Normal Started pod/payload-processing-e2e-ait-d05919ef-98cb9cd48-4lr79 Container started 55m Normal Started pod/payload-pre-processing-e2e-ait-d05919ef-5fbd9c9b67-rb26m Container started 55m Normal Created pod/payload-pre-processing-e2e-ait-d05919ef-5fbd9c9b67-rb26m Container created 55m Warning Unhealthy pod/payload-pre-processing-e2e-mt-53879c92-668d9d8855-74s4z Readiness probe failed: timeout: health rpc did not complete within 1s 55m Normal Pulled pod/payload-processing-e2e-ait-d05919ef-98cb9cd48-4lr79 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Started pod/e2e-derive-cbe6e297-openshift-default-795c76bc85-vgf74 Container started 55m Normal Created pod/e2e-derive-cbe6e297-openshift-default-795c76bc85-vgf74 Container created 55m Normal Pulled pod/e2e-derive-cbe6e297-openshift-default-795c76bc85-vgf74 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/e2e-derive-cbe6e297-openshift-default-795c76bc85-vgf74 Add eth0 [10.134.0.109/23] from ovn-kubernetes 55m Warning Unhealthy pod/e2e-ait-1b279244-openshift-default-74dd88dd99-6px88 Readiness probe failed: Get "http://10.134.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 55m Warning ProbeError pod/e2e-ait-1b279244-openshift-default-74dd88dd99-6px88 Readiness probe error: Get "http://10.134.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-cbe6e297 Scaled up replica set payload-processing-e2e-derive-cbe6e297-545ffc4c86 from 0 to 1 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-cbe6e297-545ffc4c86 Created pod: payload-processing-e2e-derive-cbe6e297-545ffc4c86-qnkpg 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-5187e1-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 55m Normal Started pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-cflp5 Container started 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-cbe6e297-848c8d947f Created pod: payload-pre-processing-e2e-derive-cbe6e297-848c8d947f-6rcr7 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-cbe6e297-848c8d947f Error creating: pods "payload-pre-processing-e2e-derive-cbe6e297-848c8d947f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-cbe6e297: serviceaccount "payload-processing-e2e-derive-cbe6e297" not found 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-cbe6e297 Scaled up replica set payload-pre-processing-e2e-derive-cbe6e297-848c8d947f from 0 to 1 55m Normal AddedInterface pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-cflp5 Add eth0 [10.133.0.78/23] from ovn-kubernetes 55m Normal Created pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-cflp5 Container created 55m Normal ScalingReplicaSet deployment/e2e-worker-w3-5187e1-openshift-default Scaled up replica set e2e-worker-w3-5187e1-openshift-default-946849bdf from 6 to 10 55m Normal Pulled pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-cflp5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/payload-pre-processing-e2e-derive-cbe6e297-848c8d947f-6rcr7 Add eth0 [10.133.0.79/23] from ovn-kubernetes 55m Normal SuccessfulCreate replicaset/e2e-worker-w3-5187e1-openshift-default-946849bdf Created pod: e2e-worker-w3-5187e1-openshift-default-946849bdf-5hj9v 55m Normal SuccessfulCreate replicaset/e2e-worker-w3-5187e1-openshift-default-946849bdf Created pod: e2e-worker-w3-5187e1-openshift-default-946849bdf-cflp5 55m Normal AddedInterface pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-knhhd Add eth0 [10.132.0.147/23] from ovn-kubernetes 55m Normal Pulled pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-knhhd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-knhhd Container created 55m Normal Started pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-knhhd Container started 55m Normal SuccessfulCreate replicaset/e2e-worker-w3-5187e1-openshift-default-946849bdf (combined from similar events): Created pod: e2e-worker-w3-5187e1-openshift-default-946849bdf-zglz6 55m Normal SuccessfulCreate replicaset/e2e-worker-w3-5187e1-openshift-default-946849bdf Created pod: e2e-worker-w3-5187e1-openshift-default-946849bdf-knhhd 55m Warning FailedCreate replicaset/payload-processing-e2e-derive-cbe6e297-545ffc4c86 Error creating: pods "payload-processing-e2e-derive-cbe6e297-545ffc4c86-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-cbe6e297: serviceaccount "payload-processing-e2e-derive-cbe6e297" not found 55m Normal Created pod/payload-pre-processing-e2e-derive-cbe6e297-848c8d947f-6rcr7 Container created 55m Normal Started pod/payload-processing-e2e-derive-cbe6e297-545ffc4c86-qnkpg Container started 55m Normal Started pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-5hj9v Container started 55m Normal Pulled pod/payload-pre-processing-e2e-derive-cbe6e297-848c8d947f-6rcr7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Started pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-zglz6 Container started 55m Normal Created pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-zglz6 Container created 55m Normal Pulled pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-zglz6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Started pod/payload-pre-processing-e2e-derive-cbe6e297-848c8d947f-6rcr7 Container started 55m Normal AddedInterface pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-zglz6 Add eth0 [10.134.0.113/23] from ovn-kubernetes 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-cbe6e297-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Normal AddedInterface pod/payload-processing-e2e-derive-cbe6e297-545ffc4c86-qnkpg Add eth0 [10.133.0.80/23] from ovn-kubernetes 55m Normal Pulled pod/payload-processing-e2e-derive-cbe6e297-545ffc4c86-qnkpg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Created pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-5hj9v Container created 55m Normal Created pod/payload-processing-e2e-derive-cbe6e297-545ffc4c86-qnkpg Container created 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-cbe6e297-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Normal Pulled pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-5hj9v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-5hj9v Add eth0 [10.134.0.112/23] from ovn-kubernetes 55m Normal Killing pod/payload-pre-processing-e2e-ait-51bfa2e1-559ff7c87f-ckt8j Stopping container payload-pre-processing 55m Normal Killing pod/payload-processing-e2e-ait-51bfa2e1-6954676576-kr8sq Stopping container payload-processing 55m Warning ProbeError pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-knhhd Startup probe error: Get "http://10.132.0.147:15021/healthz/ready": dial tcp 10.132.0.147:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-cflp5 Startup probe failed: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-cflp5 Startup probe error: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-knhhd Startup probe failed: Get "http://10.132.0.147:15021/healthz/ready": dial tcp 10.132.0.147:15021: connect: connection refused 55m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 3 55m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: cpu resource utilization (percentage of request) above target 55m Normal Killing pod/e2e-ait-51bfa2e1-openshift-default-6dcb64d7dc-7tnb9 Stopping container istio-proxy 55m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-2xmvs Add eth0 [10.133.0.81/23] from ovn-kubernetes 55m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-2xmvs 55m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-2xmvs Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 55m Warning Unhealthy pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-zglz6 Startup probe failed: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-zglz6 Startup probe error: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-5hj9v Startup probe failed: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-5hj9v Startup probe error: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused... 55m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-2xmvs Container created 55m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-2xmvs Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.698s (2.698s including waiting). Image size: 185476477 bytes. 55m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-2xmvs Container started 55m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-2xmvs Readiness probe failed: HTTP probe failed with statuscode: 503 55m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-2xmvs Readiness probe error: HTTP probe failed with statuscode: 503... 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-cbe6e297-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-cbe6e297-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-d05919ef-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-d05919ef-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-qhqx9 Readiness probe failed: Get "http://10.132.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 55m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-d6hr4 Readiness probe failed: Get "http://10.133.0.34:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 55m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-d6hr4 Readiness probe error: Get "http://10.133.0.34:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 55m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-qhqx9 Readiness probe error: Get "http://10.132.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 55m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-v828p Readiness probe failed: Get "http://10.134.0.32:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 55m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-v828p Readiness probe error: Get "http://10.134.0.32:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 55m Normal Pulled pod/e2e-mt-53879c92-openshift-default-67dfffdb77-c47vg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Started pod/e2e-mt-53879c92-openshift-default-67dfffdb77-c47vg Container started 55m Normal Created pod/e2e-mt-53879c92-openshift-default-67dfffdb77-c47vg Container created 55m Normal Killing pod/payload-processing-e2e-ait-d05919ef-98cb9cd48-4lr79 Stopping container payload-processing 55m Normal Killing pod/payload-pre-processing-e2e-ait-d05919ef-5fbd9c9b67-rb26m Stopping container payload-pre-processing 55m Normal Pulled pod/e2e-ait-d05919ef-openshift-default-7597d49499-9f7fq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/e2e-ait-d05919ef-openshift-default-7597d49499-9f7fq Container created 55m Normal Started pod/e2e-ait-d05919ef-openshift-default-7597d49499-9f7fq Container started 54m Warning ProbeError pod/e2e-ait-d05919ef-openshift-default-7597d49499-9f7fq Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-ait-d05919ef-openshift-default-7597d49499-9f7fq Startup probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused 54m Warning Unhealthy pod/payload-processing-e2e-ait-d05919ef-98cb9cd48-4lr79 Readiness probe failed: timeout: failed to connect service "10.133.0.77:50052" within 1s: context deadline exceeded 54m Normal Killing pod/e2e-ait-d05919ef-openshift-default-7597d49499-9f7fq Stopping container istio-proxy 54m Warning ProbeError pod/e2e-mt-53879c92-openshift-default-67dfffdb77-c47vg Startup probe error: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-mt-53879c92-openshift-default-67dfffdb77-c47vg Startup probe failed: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused 54m Warning Unhealthy pod/payload-pre-processing-e2e-ait-d05919ef-5fbd9c9b67-rb26m Readiness probe failed: timeout: failed to connect service "10.133.0.76:50052" within 1s: context deadline exceeded 54m Normal AddedLabel gateway/e2e-shared-a-w4-281732 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-281732 54m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-281732-openshift-default Scaled up replica set e2e-shared-a-w4-281732-openshift-default-868d787ccc from 0 to 1 54m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-281732-openshift-default-868d787ccc Created pod: e2e-shared-a-w4-281732-openshift-default-868d787ccc-qgmvd 54m Normal AddedInterface pod/e2e-shared-a-w4-281732-openshift-default-868d787ccc-qgmvd Add eth0 [10.134.0.116/23] from ovn-kubernetes 54m Normal Killing pod/payload-processing-e2e-derive-cbe6e297-545ffc4c86-qnkpg Stopping container payload-processing 54m Normal Killing pod/payload-pre-processing-e2e-derive-cbe6e297-848c8d947f-6rcr7 Stopping container payload-pre-processing 54m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w4-281732-645dfbdbf4 Error creating: pods "payload-processing-e2e-shared-a-w4-281732-645dfbdbf4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-281732: serviceaccount "payload-processing-e2e-shared-a-w4-281732" not found 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w4-281732 Scaled up replica set payload-processing-e2e-shared-a-w4-281732-645dfbdbf4 from 0 to 1 54m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-0847aa-openshift-default-75d5774789 Created pod: e2e-shared-a-w0-0847aa-openshift-default-75d5774789-tdqsq 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w4-281732-99d75d6c9 Error creating: pods "payload-pre-processing-e2e-shared-a-w4-281732-99d75d6c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-281732: serviceaccount "payload-processing-e2e-shared-a-w4-281732" not found 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w4-281732 Scaled up replica set payload-pre-processing-e2e-shared-a-w4-281732-99d75d6c9 from 0 to 1 54m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-0847aa-openshift-default Scaled up replica set e2e-shared-a-w0-0847aa-openshift-default-75d5774789 from 0 to 1 54m Normal Killing pod/e2e-derive-cbe6e297-openshift-default-795c76bc85-vgf74 Stopping container istio-proxy 54m Normal AddedLabel gateway/e2e-shared-a-w0-0847aa Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-0847aa 54m Normal AddedInterface pod/payload-processing-e2e-shared-a-w4-281732-645dfbdbf4-27m7g Add eth0 [10.132.0.151/23] from ovn-kubernetes 54m Normal Created pod/payload-processing-e2e-shared-a-w4-281732-645dfbdbf4-27m7g Container created 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w4-281732-645dfbdbf4 Created pod: payload-processing-e2e-shared-a-w4-281732-645dfbdbf4-27m7g 54m Normal Started pod/payload-processing-e2e-shared-a-w4-281732-645dfbdbf4-27m7g Container started 54m Normal Pulled pod/payload-processing-e2e-shared-a-w4-281732-645dfbdbf4-27m7g Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w4-281732-99d75d6c9 Created pod: payload-pre-processing-e2e-shared-a-w4-281732-99d75d6c9-xrckt 54m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w4-281732-99d75d6c9-xrckt Add eth0 [10.134.0.118/23] from ovn-kubernetes 54m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w4-281732-99d75d6c9-xrckt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-tdqsq Add eth0 [10.134.0.117/23] from ovn-kubernetes 54m Normal Started pod/payload-pre-processing-e2e-shared-a-w4-281732-99d75d6c9-xrckt Container started 54m Normal Created pod/payload-pre-processing-e2e-shared-a-w4-281732-99d75d6c9-xrckt Container created 54m Warning Unhealthy pod/e2e-derive-cbe6e297-openshift-default-795c76bc85-vgf74 Readiness probe failed: HTTP probe failed with statuscode: 503 54m Warning ProbeError pod/e2e-derive-cbe6e297-openshift-default-795c76bc85-vgf74 Readiness probe error: HTTP probe failed with statuscode: 503... 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w0-0847aa-6d5b985b7 Error creating: pods "payload-pre-processing-e2e-shared-a-w0-0847aa-6d5b985b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-0847aa: serviceaccount "payload-processing-e2e-shared-a-w0-0847aa" not found 54m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w0-0847aa-fc76d97d Error creating: pods "payload-processing-e2e-shared-a-w0-0847aa-fc76d97d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-0847aa: serviceaccount "payload-processing-e2e-shared-a-w0-0847aa" not found 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w0-0847aa-6d5b985b7 Created pod: payload-pre-processing-e2e-shared-a-w0-0847aa-6d5b985b7-xt8mm 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w0-0847aa-fc76d97d Created pod: payload-processing-e2e-shared-a-w0-0847aa-fc76d97d-m54ng 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w0-0847aa Scaled up replica set payload-processing-e2e-shared-a-w0-0847aa-fc76d97d from 0 to 1 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w0-0847aa Scaled up replica set payload-pre-processing-e2e-shared-a-w0-0847aa-6d5b985b7 from 0 to 1 54m Normal Created pod/payload-processing-e2e-shared-a-w0-0847aa-fc76d97d-m54ng Container created 54m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w0-0847aa-6d5b985b7-xt8mm Add eth0 [10.134.0.120/23] from ovn-kubernetes 54m Normal Pulled pod/payload-processing-e2e-shared-a-w0-0847aa-fc76d97d-m54ng Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Started pod/payload-processing-e2e-shared-a-w0-0847aa-fc76d97d-m54ng Container started 54m Normal Started pod/payload-pre-processing-e2e-shared-a-w0-0847aa-6d5b985b7-xt8mm Container started 54m Normal Created pod/payload-pre-processing-e2e-shared-a-w0-0847aa-6d5b985b7-xt8mm Container created 54m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w0-0847aa-6d5b985b7-xt8mm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/payload-processing-e2e-shared-a-w0-0847aa-fc76d97d-m54ng Add eth0 [10.134.0.119/23] from ovn-kubernetes 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-r4qjb Readiness probe failed: Get "http://10.133.0.35:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-r4qjb Readiness probe error: Get "http://10.133.0.35:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-5w59f Readiness probe failed: Get "http://10.134.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-5w59f Readiness probe error: Get "http://10.134.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-53879c92-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-53879c92-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-0847aa-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-0847aa-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Normal AddedLabel gateway/e2e-shared-b-w4-04988c Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-04988c 54m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-04988c-openshift-default-d99f984b9 Created pod: e2e-shared-b-w4-04988c-openshift-default-d99f984b9-trdjb 54m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-04988c-openshift-default Scaled up replica set e2e-shared-b-w4-04988c-openshift-default-d99f984b9 from 0 to 1 54m Normal AddedInterface pod/e2e-shared-b-w4-04988c-openshift-default-d99f984b9-trdjb Add eth0 [10.134.0.123/23] from ovn-kubernetes 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w4-04988c Scaled up replica set payload-processing-e2e-shared-b-w4-04988c-97c55b5f4 from 0 to 1 54m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w4-04988c-97c55b5f4 Error creating: pods "payload-processing-e2e-shared-b-w4-04988c-97c55b5f4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-04988c: serviceaccount "payload-processing-e2e-shared-b-w4-04988c" not found 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w4-04988c Scaled up replica set payload-pre-processing-e2e-shared-b-w4-04988c-9dcb589c9 from 0 to 1 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w4-04988c-9dcb589c9 Error creating: pods "payload-pre-processing-e2e-shared-b-w4-04988c-9dcb589c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-04988c: serviceaccount "payload-processing-e2e-shared-b-w4-04988c" not found 54m Normal AddedInterface pod/payload-processing-e2e-shared-b-w4-04988c-97c55b5f4-2jhm7 Add eth0 [10.132.0.153/23] from ovn-kubernetes 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w4-04988c-97c55b5f4 Created pod: payload-processing-e2e-shared-b-w4-04988c-97c55b5f4-2jhm7 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w4-04988c-9dcb589c9 Created pod: payload-pre-processing-e2e-shared-b-w4-04988c-9dcb589c9-xrtn5 54m Normal Created pod/payload-processing-e2e-shared-b-w4-04988c-97c55b5f4-2jhm7 Container created 54m Normal Pulled pod/payload-processing-e2e-shared-b-w4-04988c-97c55b5f4-2jhm7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w4-04988c-9dcb589c9-xrtn5 Add eth0 [10.132.0.152/23] from ovn-kubernetes 54m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w4-04988c-9dcb589c9-xrtn5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Created pod/payload-pre-processing-e2e-shared-b-w4-04988c-9dcb589c9-xrtn5 Container created 54m Normal Started pod/payload-processing-e2e-shared-b-w4-04988c-97c55b5f4-2jhm7 Container started 54m Normal Started pod/payload-pre-processing-e2e-shared-b-w4-04988c-9dcb589c9-xrtn5 Container started 54m Normal AddedLabel gateway/e2e-shared-b-w0-a940d5 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-a940d5 54m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c Created pod: e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-d7r4r 54m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-a940d5-openshift-default Scaled up replica set e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c from 0 to 1 54m Normal AddedInterface pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-d7r4r Add eth0 [10.134.0.124/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w0-a940d5-5b5d7f6f59 Created pod: payload-pre-processing-e2e-shared-b-w0-a940d5-5b5d7f6f59-nvgzv 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w0-a940d5-64d8f95f5b Created pod: payload-processing-e2e-shared-b-w0-a940d5-64d8f95f5b-l6fqz 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w0-a940d5-5b5d7f6f59 Error creating: pods "payload-pre-processing-e2e-shared-b-w0-a940d5-5b5d7f6f59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-a940d5: serviceaccount "payload-processing-e2e-shared-b-w0-a940d5" not found 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w0-a940d5 Scaled up replica set payload-processing-e2e-shared-b-w0-a940d5-64d8f95f5b from 0 to 1 53m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w0-a940d5-64d8f95f5b Error creating: pods "payload-processing-e2e-shared-b-w0-a940d5-64d8f95f5b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-a940d5: serviceaccount "payload-processing-e2e-shared-b-w0-a940d5" not found 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w0-a940d5 Scaled up replica set payload-pre-processing-e2e-shared-b-w0-a940d5-5b5d7f6f59 from 0 to 1 53m Normal Created pod/payload-pre-processing-e2e-shared-b-w0-a940d5-5b5d7f6f59-nvgzv Container created 53m Normal Created pod/payload-processing-e2e-shared-b-w0-a940d5-64d8f95f5b-l6fqz Container created 53m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w0-a940d5-5b5d7f6f59-nvgzv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Started pod/payload-pre-processing-e2e-shared-b-w0-a940d5-5b5d7f6f59-nvgzv Container started 53m Normal AddedInterface pod/payload-processing-e2e-shared-b-w0-a940d5-64d8f95f5b-l6fqz Add eth0 [10.132.0.155/23] from ovn-kubernetes 53m Normal Started pod/payload-processing-e2e-shared-b-w0-a940d5-64d8f95f5b-l6fqz Container started 53m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w0-a940d5-5b5d7f6f59-nvgzv Add eth0 [10.132.0.154/23] from ovn-kubernetes 53m Normal Pulled pod/payload-processing-e2e-shared-b-w0-a940d5-64d8f95f5b-l6fqz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Warning ProbeError pod/e2e-shared-a-w4-281732-openshift-default-868d787ccc-qgmvd Readiness probe error: Get "http://10.134.0.116:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 53m Warning Unhealthy pod/e2e-shared-a-w4-281732-openshift-default-868d787ccc-qgmvd Readiness probe failed: Get "http://10.134.0.116:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 53m Normal AddedInterface pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-2kdhp Add eth0 [10.133.0.83/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-0847aa-openshift-default-75d5774789 Created pod: e2e-shared-a-w0-0847aa-openshift-default-75d5774789-8xg84 53m Normal AddedInterface pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-db8c6 Add eth0 [10.132.0.159/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-0847aa-openshift-default-75d5774789 Created pod: e2e-shared-a-w0-0847aa-openshift-default-75d5774789-db8c6 53m Normal Pulled pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-8xg84 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-8xg84 Add eth0 [10.133.0.82/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-0847aa-openshift-default-75d5774789 Created pod: e2e-shared-a-w0-0847aa-openshift-default-75d5774789-2kdhp 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-0847aa-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 53m Normal Created pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-8xg84 Container created 53m Normal Pulled pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-2kdhp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-0847aa-openshift-default Scaled up replica set e2e-shared-a-w0-0847aa-openshift-default-75d5774789 from 1 to 4 53m Normal Pulled pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-db8c6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-2kdhp Container created 53m Normal Started pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-2kdhp Container started 53m Normal Started pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-db8c6 Container started 53m Normal Pulled pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-tdqsq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-8xg84 Container started 53m Normal Created pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-db8c6 Container created 53m Normal Pulled pod/e2e-shared-a-w4-281732-openshift-default-868d787ccc-qgmvd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-shared-a-w4-281732-openshift-default-868d787ccc-qgmvd Container started 53m Normal Created pod/e2e-shared-a-w4-281732-openshift-default-868d787ccc-qgmvd Container created 53m Normal Started pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-tdqsq Container started 53m Normal Created pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-tdqsq Container created 53m Normal Killing pod/payload-pre-processing-e2e-mt-53879c92-668d9d8855-74s4z Stopping container payload-pre-processing 53m Normal Killing pod/payload-processing-e2e-mt-53879c92-6bf559b69b-plpwn Stopping container payload-processing 53m Warning ProbeError pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-8xg84 Startup probe error: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-db8c6 Startup probe error: Get "http://10.132.0.159:15021/healthz/ready": dial tcp 10.132.0.159:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-db8c6 Startup probe failed: Get "http://10.132.0.159:15021/healthz/ready": dial tcp 10.132.0.159:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-8xg84 Startup probe failed: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-2kdhp Startup probe failed: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused 53m Warning ProbeError pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-2kdhp Startup probe error: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused... 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-281732-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-281732-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Warning ProbeError pod/e2e-shared-a-w4-281732-openshift-default-868d787ccc-qgmvd Startup probe error: Get "http://10.134.0.116:15021/healthz/ready": dial tcp 10.134.0.116:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-tdqsq Startup probe error: Get "http://10.134.0.117:15021/healthz/ready": dial tcp 10.134.0.117:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-tdqsq Startup probe failed: Get "http://10.134.0.117:15021/healthz/ready": dial tcp 10.134.0.117:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-shared-a-w4-281732-openshift-default-868d787ccc-qgmvd Startup probe failed: Get "http://10.134.0.116:15021/healthz/ready": dial tcp 10.134.0.116:15021: connect: connection refused 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-0847aa-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-0847aa-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Normal Killing pod/e2e-mt-53879c92-openshift-default-67dfffdb77-c47vg Stopping container istio-proxy 53m Normal AddedLabel gateway/e2e-shared-a-w3-23d281 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-23d281 53m Warning FailedUpdateStatus horizontalpodautoscaler/e2e-mt-53879c92-openshift-default horizontalpodautoscalers.autoscaling "e2e-mt-53879c92-openshift-default" not found 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-53879c92-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-53879c92-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Warning Unhealthy pod/e2e-mt-53879c92-openshift-default-67dfffdb77-c47vg Readiness probe failed: HTTP probe failed with statuscode: 503 53m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-23d281-openshift-default Scaled up replica set e2e-shared-a-w3-23d281-openshift-default-5cb785989 from 0 to 1 53m Warning ProbeError pod/e2e-mt-53879c92-openshift-default-67dfffdb77-c47vg Readiness probe error: HTTP probe failed with statuscode: 503... 53m Normal AddedInterface pod/e2e-shared-a-w3-23d281-openshift-default-5cb785989-8jstk Add eth0 [10.132.0.160/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-23d281-openshift-default-5cb785989 Created pod: e2e-shared-a-w3-23d281-openshift-default-5cb785989-8jstk 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-281732-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Normal SuccessfulCreate replicaset/e2e-mt-8b1efeb6-openshift-default-5cf66d64d5 Created pod: e2e-mt-8b1efeb6-openshift-default-5cf66d64d5-z8hk7 53m Normal ScalingReplicaSet deployment/e2e-mt-8b1efeb6-openshift-default Scaled up replica set e2e-mt-8b1efeb6-openshift-default-5cf66d64d5 from 0 to 1 53m Normal AddedLabel gateway/e2e-mt-8b1efeb6 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-8b1efeb6 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-281732-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Normal AddedInterface pod/e2e-mt-8b1efeb6-openshift-default-5cf66d64d5-z8hk7 Add eth0 [10.132.0.161/23] from ovn-kubernetes 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w3-23d281 Scaled up replica set payload-processing-e2e-shared-a-w3-23d281-97b8bbd66 from 0 to 1 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w3-23d281-8599fb6555 Error creating: pods "payload-pre-processing-e2e-shared-a-w3-23d281-8599fb6555-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-23d281: serviceaccount "payload-processing-e2e-shared-a-w3-23d281" not found 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w3-23d281-97b8bbd66 Created pod: payload-processing-e2e-shared-a-w3-23d281-97b8bbd66-7dgnl 53m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w3-23d281-97b8bbd66 Error creating: pods "payload-processing-e2e-shared-a-w3-23d281-97b8bbd66-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-23d281: serviceaccount "payload-processing-e2e-shared-a-w3-23d281" not found 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w3-23d281 Scaled up replica set payload-pre-processing-e2e-shared-a-w3-23d281-8599fb6555 from 0 to 1 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w3-23d281-8599fb6555 Created pod: payload-pre-processing-e2e-shared-a-w3-23d281-8599fb6555-qtrjc 53m Normal Created pod/payload-processing-e2e-shared-a-w3-23d281-97b8bbd66-7dgnl Container created 53m Normal Created pod/payload-pre-processing-e2e-shared-a-w3-23d281-8599fb6555-qtrjc Container created 53m Normal Started pod/payload-processing-e2e-shared-a-w3-23d281-97b8bbd66-7dgnl Container started 53m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w3-23d281-8599fb6555-qtrjc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Started pod/payload-pre-processing-e2e-shared-a-w3-23d281-8599fb6555-qtrjc Container started 53m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w3-23d281-8599fb6555-qtrjc Add eth0 [10.133.0.85/23] from ovn-kubernetes 53m Normal Pulled pod/payload-processing-e2e-shared-a-w3-23d281-97b8bbd66-7dgnl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/payload-processing-e2e-shared-a-w3-23d281-97b8bbd66-7dgnl Add eth0 [10.133.0.84/23] from ovn-kubernetes 53m Normal AddedLabel gateway/e2e-shared-a-w6-2a2eca Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w6-2a2eca 53m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-2a2eca-openshift-default Scaled up replica set e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc from 0 to 1 53m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc Created pod: e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc-7d49j 53m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-0847aa-openshift-default Scaled up replica set e2e-shared-a-w0-0847aa-openshift-default-75d5774789 from 4 to 8 53m Normal AddedInterface pod/e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc-7d49j Add eth0 [10.134.0.131/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-8b1efeb6-68787c86c9 Created pod: payload-pre-processing-e2e-mt-8b1efeb6-68787c86c9-wpjjk 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-0847aa-openshift-default New size: 8; reason: All metrics below target 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-8b1efeb6 Scaled up replica set payload-processing-e2e-mt-8b1efeb6-5d7dd8ff94 from 0 to 1 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-8b1efeb6-68787c86c9 Error creating: pods "payload-pre-processing-e2e-mt-8b1efeb6-68787c86c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-8b1efeb6: serviceaccount "payload-processing-e2e-mt-8b1efeb6" not found 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-8b1efeb6 Scaled up replica set payload-pre-processing-e2e-mt-8b1efeb6-68787c86c9 from 0 to 1 53m Warning FailedCreate replicaset/payload-processing-e2e-mt-8b1efeb6-5d7dd8ff94 Error creating: pods "payload-processing-e2e-mt-8b1efeb6-5d7dd8ff94-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-8b1efeb6: serviceaccount "payload-processing-e2e-mt-8b1efeb6" not found 53m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-0847aa-openshift-default-75d5774789 Created pod: e2e-shared-a-w0-0847aa-openshift-default-75d5774789-7kgzf 53m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-0847aa-openshift-default-75d5774789 Created pod: e2e-shared-a-w0-0847aa-openshift-default-75d5774789-5jtnl 53m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-0847aa-openshift-default-75d5774789 Created pod: e2e-shared-a-w0-0847aa-openshift-default-75d5774789-rltc7 53m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-0847aa-openshift-default-75d5774789 Created pod: e2e-shared-a-w0-0847aa-openshift-default-75d5774789-mcjd8 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-8b1efeb6-5d7dd8ff94 Created pod: payload-processing-e2e-mt-8b1efeb6-5d7dd8ff94-q446n 53m Normal Pulled pod/payload-pre-processing-e2e-mt-8b1efeb6-68787c86c9-wpjjk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-7kgzf Add eth0 [10.134.0.132/23] from ovn-kubernetes 53m Normal Created pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-rltc7 Container created 53m Normal Started pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-rltc7 Container started 53m Normal Started pod/payload-processing-e2e-mt-8b1efeb6-5d7dd8ff94-q446n Container started 53m Normal Created pod/payload-processing-e2e-mt-8b1efeb6-5d7dd8ff94-q446n Container created 53m Normal Pulled pod/payload-processing-e2e-mt-8b1efeb6-5d7dd8ff94-q446n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-mcjd8 Container started 53m Normal Created pod/payload-pre-processing-e2e-mt-8b1efeb6-68787c86c9-wpjjk Container created 53m Normal Created pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-7kgzf Container created 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-a940d5-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 53m Normal Pulled pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-rltc7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-rltc7 Add eth0 [10.133.0.88/23] from ovn-kubernetes 53m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-a940d5-openshift-default Scaled up replica set e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c from 1 to 4 53m Normal Pulled pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-7kgzf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/payload-pre-processing-e2e-mt-8b1efeb6-68787c86c9-wpjjk Add eth0 [10.133.0.87/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c Created pod: e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-xdg5b 53m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c Created pod: e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-jpkff 53m Normal Started pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-5jtnl Container started 53m Normal Created pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-5jtnl Container created 53m Normal Pulled pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-5jtnl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c Created pod: e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-v6bzq 53m Normal AddedInterface pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-5jtnl Add eth0 [10.132.0.162/23] from ovn-kubernetes 53m Normal AddedInterface pod/payload-processing-e2e-mt-8b1efeb6-5d7dd8ff94-q446n Add eth0 [10.133.0.86/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-mcjd8 Add eth0 [10.132.0.163/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-mcjd8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/payload-pre-processing-e2e-mt-8b1efeb6-68787c86c9-wpjjk Container started 53m Normal Created pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-mcjd8 Container created 53m Normal Started pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-7kgzf Container started 53m Normal Started pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-jpkff Container started 53m Normal AddedInterface pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-xdg5b Add eth0 [10.134.0.133/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-jpkff Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-v6bzq Add eth0 [10.133.0.89/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-jpkff Add eth0 [10.132.0.164/23] from ovn-kubernetes 53m Normal Created pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-v6bzq Container created 53m Normal Started pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-v6bzq Container started 53m Normal Created pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-jpkff Container created 53m Normal Pulled pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-xdg5b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-xdg5b Container created 53m Normal Started pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-xdg5b Container started 53m Normal Pulled pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-v6bzq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-04988c-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 52m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-04988c-openshift-default Scaled up replica set e2e-shared-b-w4-04988c-openshift-default-d99f984b9 from 1 to 2 52m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-04988c-openshift-default-d99f984b9 Created pod: e2e-shared-b-w4-04988c-openshift-default-d99f984b9-zmsb6 52m Normal Created pod/e2e-shared-b-w4-04988c-openshift-default-d99f984b9-zmsb6 Container created 52m Normal Pulled pod/e2e-shared-b-w4-04988c-openshift-default-d99f984b9-zmsb6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Started pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-d7r4r Container started 52m Normal Started pod/e2e-shared-b-w4-04988c-openshift-default-d99f984b9-trdjb Container started 52m Warning ProbeError pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-d7r4r Readiness probe error: Get "http://10.134.0.124:15021/healthz/ready": dial tcp 10.134.0.124:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-d7r4r Readiness probe failed: Get "http://10.134.0.124:15021/healthz/ready": dial tcp 10.134.0.124:15021: connect: connection refused 52m Normal Created pod/e2e-shared-b-w4-04988c-openshift-default-d99f984b9-trdjb Container created 52m Normal Pulled pod/e2e-shared-b-w4-04988c-openshift-default-d99f984b9-trdjb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Pulled pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-d7r4r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-shared-b-w4-04988c-openshift-default-d99f984b9-zmsb6 Add eth0 [10.133.0.90/23] from ovn-kubernetes 52m Normal Created pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-d7r4r Container created 52m Normal Started pod/e2e-shared-b-w4-04988c-openshift-default-d99f984b9-zmsb6 Container started 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w6-2a2eca Scaled up replica set payload-processing-e2e-shared-a-w6-2a2eca-6cf9b6f4bd from 0 to 1 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w6-2a2eca Scaled up replica set payload-pre-processing-e2e-shared-a-w6-2a2eca-66ff5f6f5f from 0 to 1 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w6-2a2eca-66ff5f6f5f Error creating: pods "payload-pre-processing-e2e-shared-a-w6-2a2eca-66ff5f6f5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-2a2eca: serviceaccount "payload-processing-e2e-shared-a-w6-2a2eca" not found 52m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w6-2a2eca-6cf9b6f4bd Error creating: pods "payload-processing-e2e-shared-a-w6-2a2eca-6cf9b6f4bd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-2a2eca: serviceaccount "payload-processing-e2e-shared-a-w6-2a2eca" not found 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w6-2a2eca-6cf9b6f4bd Created pod: payload-processing-e2e-shared-a-w6-2a2eca-6cf9b6f4bd-9mxw7 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w6-2a2eca-66ff5f6f5f Created pod: payload-pre-processing-e2e-shared-a-w6-2a2eca-66ff5f6f5f-8cw8r 52m Normal Pulled pod/payload-processing-e2e-shared-a-w6-2a2eca-6cf9b6f4bd-9mxw7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w6-2a2eca-66ff5f6f5f-8cw8r Add eth0 [10.132.0.166/23] from ovn-kubernetes 52m Warning Unhealthy pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-jpkff Startup probe failed: Get "http://10.132.0.164:15021/healthz/ready": dial tcp 10.132.0.164:15021: connect: connection refused 52m Normal Created pod/payload-processing-e2e-shared-a-w6-2a2eca-6cf9b6f4bd-9mxw7 Container created 52m Normal Started pod/payload-processing-e2e-shared-a-w6-2a2eca-6cf9b6f4bd-9mxw7 Container started 52m Warning ProbeError pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-jpkff Startup probe error: Get "http://10.132.0.164:15021/healthz/ready": dial tcp 10.132.0.164:15021: connect: connection refused... 52m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w6-2a2eca-66ff5f6f5f-8cw8r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Warning ProbeError pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-rltc7 Startup probe error: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-v6bzq Startup probe error: Get "http://10.133.0.89:15021/healthz/ready": dial tcp 10.133.0.89:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-5jtnl Startup probe error: Get "http://10.132.0.162:15021/healthz/ready": dial tcp 10.132.0.162:15021: connect: connection refused... 52m Normal Started pod/payload-pre-processing-e2e-shared-a-w6-2a2eca-66ff5f6f5f-8cw8r Container started 52m Warning Unhealthy pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-v6bzq Startup probe failed: Get "http://10.133.0.89:15021/healthz/ready": dial tcp 10.133.0.89:15021: connect: connection refused 52m Normal Created pod/payload-pre-processing-e2e-shared-a-w6-2a2eca-66ff5f6f5f-8cw8r Container created 52m Warning Unhealthy pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-5jtnl Startup probe failed: Get "http://10.132.0.162:15021/healthz/ready": dial tcp 10.132.0.162:15021: connect: connection refused 52m Normal AddedInterface pod/payload-processing-e2e-shared-a-w6-2a2eca-6cf9b6f4bd-9mxw7 Add eth0 [10.133.0.91/23] from ovn-kubernetes 52m Warning Unhealthy pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-rltc7 Startup probe failed: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused 52m Warning ProbeError pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-mcjd8 Startup probe error: Get "http://10.132.0.163:15021/healthz/ready": dial tcp 10.132.0.163:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-mcjd8 Startup probe failed: Get "http://10.132.0.163:15021/healthz/ready": dial tcp 10.132.0.163:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-7kgzf Startup probe failed: Get "http://10.134.0.132:15021/healthz/ready": dial tcp 10.134.0.132:15021: connect: connection refused 52m Warning ProbeError pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-7kgzf Startup probe error: Get "http://10.134.0.132:15021/healthz/ready": dial tcp 10.134.0.132:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-xdg5b Startup probe error: Get "http://10.134.0.133:15021/healthz/ready": dial tcp 10.134.0.133:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-xdg5b Startup probe failed: Get "http://10.134.0.133:15021/healthz/ready": dial tcp 10.134.0.133:15021: connect: connection refused 52m Warning ProbeError pod/e2e-shared-b-w4-04988c-openshift-default-d99f984b9-zmsb6 Startup probe error: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-shared-b-w4-04988c-openshift-default-d99f984b9-zmsb6 Startup probe failed: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-d7r4r Startup probe failed: Get "http://10.134.0.124:15021/healthz/ready": dial tcp 10.134.0.124:15021: connect: connection refused 52m Warning ProbeError pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-d7r4r Startup probe error: Get "http://10.134.0.124:15021/healthz/ready": dial tcp 10.134.0.124:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-shared-b-w4-04988c-openshift-default-d99f984b9-trdjb Startup probe error: Get "http://10.134.0.123:15021/healthz/ready": dial tcp 10.134.0.123:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-shared-b-w4-04988c-openshift-default-d99f984b9-trdjb Startup probe failed: Get "http://10.134.0.123:15021/healthz/ready": dial tcp 10.134.0.123:15021: connect: connection refused 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-a940d5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-a940d5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-04988c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-04988c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-ddx5s Readiness probe error: Get "http://10.134.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-ddx5s Readiness probe failed: Get "http://10.134.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-a940d5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-a940d5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-04988c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-04988c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c Created pod: e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-ksw7k 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-a940d5-openshift-default New size: 6; reason: All metrics below target 52m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-a940d5-openshift-default Scaled up replica set e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c from 4 to 6 52m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c Created pod: e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-fd2cz 52m Normal Pulled pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-fd2cz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-ksw7k Add eth0 [10.132.0.167/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-ksw7k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-fd2cz Container created 52m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-d8a3d1-openshift-default Scaled up replica set e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7 from 0 to 1 52m Normal Created pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-ksw7k Container created 52m Normal AddedInterface pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-fd2cz Add eth0 [10.133.0.92/23] from ovn-kubernetes 52m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7 Created pod: e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7-8q4n6 52m Normal AddedLabel gateway/e2e-shared-b-w3-d8a3d1 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-d8a3d1 52m Normal Started pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-fd2cz Container started 52m Normal Started pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-ksw7k Container started 52m Normal AddedInterface pod/e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7-8q4n6 Add eth0 [10.134.0.137/23] from ovn-kubernetes 52m Warning Unhealthy pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-ksw7k Startup probe failed: Get "http://10.132.0.167:15021/healthz/ready": dial tcp 10.132.0.167:15021: connect: connection refused 52m Warning ProbeError pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-ksw7k Startup probe error: Get "http://10.132.0.167:15021/healthz/ready": dial tcp 10.132.0.167:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-fd2cz Startup probe failed: Get "http://10.133.0.92:15021/healthz/ready": dial tcp 10.133.0.92:15021: connect: connection refused 52m Warning ProbeError pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-fd2cz Startup probe error: Get "http://10.133.0.92:15021/healthz/ready": dial tcp 10.133.0.92:15021: connect: connection refused... 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w3-d8a3d1 Scaled up replica set payload-processing-e2e-shared-b-w3-d8a3d1-85f887b8c4 from 0 to 1 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w3-d8a3d1 Scaled up replica set payload-pre-processing-e2e-shared-b-w3-d8a3d1-b7986f8f9 from 0 to 1 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w3-d8a3d1-b7986f8f9 Error creating: pods "payload-pre-processing-e2e-shared-b-w3-d8a3d1-b7986f8f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-d8a3d1: serviceaccount "payload-processing-e2e-shared-b-w3-d8a3d1" not found 52m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w3-d8a3d1-b7986f8f9-s9n6t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w3-d8a3d1-b7986f8f9-s9n6t Add eth0 [10.132.0.168/23] from ovn-kubernetes 52m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w3-d8a3d1-85f887b8c4 Error creating: pods "payload-processing-e2e-shared-b-w3-d8a3d1-85f887b8c4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-d8a3d1: serviceaccount "payload-processing-e2e-shared-b-w3-d8a3d1" not found 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w3-d8a3d1-85f887b8c4 Created pod: payload-processing-e2e-shared-b-w3-d8a3d1-85f887b8c4-gfs5q 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w3-d8a3d1-b7986f8f9 Created pod: payload-pre-processing-e2e-shared-b-w3-d8a3d1-b7986f8f9-s9n6t 52m Normal Created pod/payload-processing-e2e-shared-b-w3-d8a3d1-85f887b8c4-gfs5q Container created 52m Normal Started pod/payload-processing-e2e-shared-b-w3-d8a3d1-85f887b8c4-gfs5q Container started 52m Normal Created pod/payload-pre-processing-e2e-shared-b-w3-d8a3d1-b7986f8f9-s9n6t Container created 52m Normal Pulled pod/payload-processing-e2e-shared-b-w3-d8a3d1-85f887b8c4-gfs5q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal AddedLabel gateway/e2e-api-a-4c5345 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-4c5345 52m Normal AddedInterface pod/payload-processing-e2e-shared-b-w3-d8a3d1-85f887b8c4-gfs5q Add eth0 [10.133.0.93/23] from ovn-kubernetes 52m Normal Started pod/payload-pre-processing-e2e-shared-b-w3-d8a3d1-b7986f8f9-s9n6t Container started 52m Normal SuccessfulCreate replicaset/e2e-api-a-4c5345-openshift-default-86899fff89 Created pod: e2e-api-a-4c5345-openshift-default-86899fff89-l8tzb 52m Normal ScalingReplicaSet deployment/e2e-api-a-4c5345-openshift-default Scaled up replica set e2e-api-a-4c5345-openshift-default-86899fff89 from 0 to 1 52m Normal AddedInterface pod/e2e-api-a-4c5345-openshift-default-86899fff89-l8tzb Add eth0 [10.133.0.94/23] from ovn-kubernetes 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-4c5345 Scaled up replica set payload-pre-processing-e2e-api-a-4c5345-5548f954c from 0 to 1 52m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-26f8ba-openshift-default Scaled up replica set e2e-shared-b-w6-26f8ba-openshift-default-569ccb4c87 from 0 to 1 52m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-26f8ba-openshift-default-569ccb4c87 Created pod: e2e-shared-b-w6-26f8ba-openshift-default-569ccb4c87-pcd2l 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-4c5345 Scaled up replica set payload-processing-e2e-api-a-4c5345-6df87f7798 from 0 to 1 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-4c5345-6df87f7798 Created pod: payload-processing-e2e-api-a-4c5345-6df87f7798-fkts8 52m Warning FailedCreate replicaset/payload-processing-e2e-api-a-4c5345-6df87f7798 Error creating: pods "payload-processing-e2e-api-a-4c5345-6df87f7798-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-4c5345: serviceaccount "payload-processing-e2e-api-a-4c5345" not found 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-4c5345-5548f954c Created pod: payload-pre-processing-e2e-api-a-4c5345-5548f954c-kk85c 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-4c5345-5548f954c Error creating: pods "payload-pre-processing-e2e-api-a-4c5345-5548f954c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-4c5345: serviceaccount "payload-processing-e2e-api-a-4c5345" not found 52m Normal AddedLabel gateway/e2e-shared-b-w6-26f8ba Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w6-26f8ba 52m Normal Pulled pod/payload-pre-processing-e2e-api-a-4c5345-5548f954c-kk85c Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Created pod/payload-pre-processing-e2e-api-a-4c5345-5548f954c-kk85c Container created 52m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-4c5345-5548f954c-kk85c Add eth0 [10.133.0.95/23] from ovn-kubernetes 52m Normal Pulled pod/payload-processing-e2e-api-a-4c5345-6df87f7798-fkts8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/payload-processing-e2e-api-a-4c5345-6df87f7798-fkts8 Add eth0 [10.132.0.171/23] from ovn-kubernetes 52m Normal Started pod/payload-pre-processing-e2e-api-a-4c5345-5548f954c-kk85c Container started 52m Normal Created pod/payload-processing-e2e-api-a-4c5345-6df87f7798-fkts8 Container created 52m Normal Started pod/payload-processing-e2e-api-a-4c5345-6df87f7798-fkts8 Container started 52m Normal AddedInterface pod/e2e-shared-b-w6-26f8ba-openshift-default-569ccb4c87-pcd2l Add eth0 [10.132.0.170/23] from ovn-kubernetes 51m Normal Killing pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-vlq4m Stopping container istio-proxy 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-88a4d9-openshift-default New size: 8; reason: All metrics below target 51m Normal Killing pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-g4fx2 Stopping container istio-proxy 51m Normal SuccessfulDelete replicaset/e2e-worker-w0-88a4d9-openshift-default-857757469c Deleted pod: e2e-worker-w0-88a4d9-openshift-default-857757469c-g4fx2 51m Normal SuccessfulDelete replicaset/e2e-worker-w0-88a4d9-openshift-default-857757469c Deleted pod: e2e-worker-w0-88a4d9-openshift-default-857757469c-vlq4m 51m Normal ScalingReplicaSet deployment/e2e-worker-w0-88a4d9-openshift-default Scaled down replica set e2e-worker-w0-88a4d9-openshift-default-857757469c from 10 to 8 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w6-26f8ba Scaled up replica set payload-pre-processing-e2e-shared-b-w6-26f8ba-5b94f459cf from 0 to 1 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w6-26f8ba-6f7bb8fcbd Created pod: payload-processing-e2e-shared-b-w6-26f8ba-6f7bb8fcbd-mkqzk 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w6-26f8ba-5b94f459cf Error creating: pods "payload-pre-processing-e2e-shared-b-w6-26f8ba-5b94f459cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-26f8ba: serviceaccount "payload-processing-e2e-shared-b-w6-26f8ba" not found 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w6-26f8ba-5b94f459cf Created pod: payload-pre-processing-e2e-shared-b-w6-26f8ba-5b94f459cf-zrpcl 51m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w6-26f8ba-6f7bb8fcbd Error creating: pods "payload-processing-e2e-shared-b-w6-26f8ba-6f7bb8fcbd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-26f8ba: serviceaccount "payload-processing-e2e-shared-b-w6-26f8ba" not found 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w6-26f8ba Scaled up replica set payload-processing-e2e-shared-b-w6-26f8ba-6f7bb8fcbd from 0 to 1 51m Normal Pulled pod/payload-processing-e2e-shared-b-w6-26f8ba-6f7bb8fcbd-mkqzk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/payload-processing-e2e-shared-b-w6-26f8ba-6f7bb8fcbd-mkqzk Add eth0 [10.132.0.173/23] from ovn-kubernetes 51m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w6-26f8ba-5b94f459cf-zrpcl Add eth0 [10.132.0.172/23] from ovn-kubernetes 51m Normal Started pod/payload-processing-e2e-shared-b-w6-26f8ba-6f7bb8fcbd-mkqzk Container started 51m Normal Created pod/payload-processing-e2e-shared-b-w6-26f8ba-6f7bb8fcbd-mkqzk Container created 51m Warning ProbeError pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-g4fx2 Readiness probe error: HTTP probe failed with statuscode: 503... 51m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w6-26f8ba-5b94f459cf-zrpcl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Created pod/payload-pre-processing-e2e-shared-b-w6-26f8ba-5b94f459cf-zrpcl Container created 51m Normal Started pod/payload-pre-processing-e2e-shared-b-w6-26f8ba-5b94f459cf-zrpcl Container started 51m Warning ProbeError pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-vlq4m Readiness probe error: HTTP probe failed with statuscode: 503... 51m Warning ProbeError pod/e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc-7d49j Readiness probe error: Get "http://10.134.0.131:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc-7d49j Readiness probe failed: Get "http://10.134.0.131:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Normal Pulled pod/e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc-7d49j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc-7d49j Container created 51m Normal Started pod/e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc-7d49j Container started 51m Normal Created pod/e2e-mt-8b1efeb6-openshift-default-5cf66d64d5-z8hk7 Container created 51m Normal Pulled pod/e2e-mt-8b1efeb6-openshift-default-5cf66d64d5-z8hk7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Pulled pod/e2e-shared-a-w3-23d281-openshift-default-5cb785989-8jstk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-shared-a-w3-23d281-openshift-default-5cb785989-8jstk Container created 51m Normal Started pod/e2e-shared-a-w3-23d281-openshift-default-5cb785989-8jstk Container started 51m Normal Started pod/e2e-mt-8b1efeb6-openshift-default-5cf66d64d5-z8hk7 Container started 51m Warning ProbeError pod/e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc-7d49j Startup probe error: Get "http://10.134.0.131:15021/healthz/ready": dial tcp 10.134.0.131:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc-7d49j Startup probe failed: Get "http://10.134.0.131:15021/healthz/ready": dial tcp 10.134.0.131:15021: connect: connection refused 51m Warning ProbeError pod/e2e-mt-8b1efeb6-openshift-default-5cf66d64d5-z8hk7 Startup probe error: Get "http://10.132.0.161:15021/healthz/ready": dial tcp 10.132.0.161:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-shared-a-w3-23d281-openshift-default-5cb785989-8jstk Startup probe error: Get "http://10.132.0.160:15021/healthz/ready": dial tcp 10.132.0.160:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-mt-8b1efeb6-openshift-default-5cf66d64d5-z8hk7 Startup probe failed: Get "http://10.132.0.161:15021/healthz/ready": dial tcp 10.132.0.161:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-shared-a-w3-23d281-openshift-default-5cb785989-8jstk Startup probe failed: Get "http://10.132.0.160:15021/healthz/ready": dial tcp 10.132.0.160:15021: connect: connection refused 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-26f8ba-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-26f8ba-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Normal Killing pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-jrmj4 Stopping container istio-proxy 51m Normal ScalingReplicaSet deployment/e2e-worker-w4-50d738-openshift-default Scaled down replica set e2e-worker-w4-50d738-openshift-default-8d6476cbc from 8 to 6 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-50d738-openshift-default New size: 6; reason: All metrics below target 51m Normal SuccessfulDelete replicaset/e2e-worker-w4-50d738-openshift-default-8d6476cbc Deleted pod: e2e-worker-w4-50d738-openshift-default-8d6476cbc-jrmj4 51m Normal SuccessfulDelete replicaset/e2e-worker-w4-50d738-openshift-default-8d6476cbc Deleted pod: e2e-worker-w4-50d738-openshift-default-8d6476cbc-6wd75 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-2d619f-openshift-default New size: 8; reason: All metrics below target 51m Normal Killing pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-6wd75 Stopping container istio-proxy 51m Normal Killing pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-fsnjn Stopping container istio-proxy 51m Normal SuccessfulDelete replicaset/e2e-worker-w6-2d619f-openshift-default-85b84758df Deleted pod: e2e-worker-w6-2d619f-openshift-default-85b84758df-nckfd 51m Normal SuccessfulDelete replicaset/e2e-worker-w6-2d619f-openshift-default-85b84758df Deleted pod: e2e-worker-w6-2d619f-openshift-default-85b84758df-fsnjn 51m Normal ScalingReplicaSet deployment/e2e-worker-w6-2d619f-openshift-default Scaled down replica set e2e-worker-w6-2d619f-openshift-default-85b84758df from 10 to 8 51m Normal Killing pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-nckfd Stopping container istio-proxy 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-23d281-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-23d281-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-8b1efeb6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-8b1efeb6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-2a2eca-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-2a2eca-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning ProbeError pod/e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7-8q4n6 Readiness probe error: Get "http://10.134.0.137:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7-8q4n6 Readiness probe failed: Get "http://10.134.0.137:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-23d281-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-23d281-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-8b1efeb6-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-8b1efeb6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Normal Pulled pod/e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7-8q4n6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedLabel gateway/e2e-api-a-32fdf2 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-32fdf2 51m Normal ScalingReplicaSet deployment/e2e-api-a-32fdf2-openshift-default Scaled up replica set e2e-api-a-32fdf2-openshift-default-798db88bbc from 0 to 1 51m Normal SuccessfulCreate replicaset/e2e-api-a-32fdf2-openshift-default-798db88bbc Created pod: e2e-api-a-32fdf2-openshift-default-798db88bbc-52z75 51m Normal Created pod/e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7-8q4n6 Container created 51m Normal Started pod/e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7-8q4n6 Container started 51m Normal AddedInterface pod/e2e-api-a-32fdf2-openshift-default-798db88bbc-52z75 Add eth0 [10.134.0.141/23] from ovn-kubernetes 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-2a2eca-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-2a2eca-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning FailedCreate replicaset/payload-processing-e2e-api-a-32fdf2-7469cf674d Error creating: pods "payload-processing-e2e-api-a-32fdf2-7469cf674d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-32fdf2: serviceaccount "payload-processing-e2e-api-a-32fdf2" not found 51m Normal AddedLabel gateway/e2e-api-b-bbfc2e Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-bbfc2e 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-32fdf2-6767d498c5 Error creating: pods "payload-pre-processing-e2e-api-a-32fdf2-6767d498c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-32fdf2: serviceaccount "payload-processing-e2e-api-a-32fdf2" not found 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-32fdf2 Scaled up replica set payload-pre-processing-e2e-api-a-32fdf2-6767d498c5 from 0 to 1 51m Normal ScalingReplicaSet deployment/e2e-api-b-bbfc2e-openshift-default Scaled up replica set e2e-api-b-bbfc2e-openshift-default-6c585b5767 from 0 to 1 51m Normal SuccessfulCreate replicaset/e2e-api-b-bbfc2e-openshift-default-6c585b5767 Created pod: e2e-api-b-bbfc2e-openshift-default-6c585b5767-jlcxj 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-32fdf2 Scaled up replica set payload-processing-e2e-api-a-32fdf2-7469cf674d from 0 to 1 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-32fdf2-6767d498c5 Created pod: payload-pre-processing-e2e-api-a-32fdf2-6767d498c5-np8rq 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-d8a3d1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Normal AddedInterface pod/payload-processing-e2e-api-a-32fdf2-7469cf674d-4c2l6 Add eth0 [10.132.0.176/23] from ovn-kubernetes 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-d8a3d1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-32fdf2-7469cf674d Created pod: payload-processing-e2e-api-a-32fdf2-7469cf674d-4c2l6 50m Warning ProbeError pod/e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7-8q4n6 Startup probe error: Get "http://10.134.0.137:15021/healthz/ready": dial tcp 10.134.0.137:15021: connect: connection refused... 50m Normal Pulled pod/payload-pre-processing-e2e-api-a-32fdf2-6767d498c5-np8rq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Created pod/payload-processing-e2e-api-a-32fdf2-7469cf674d-4c2l6 Container created 50m Normal Started pod/payload-processing-e2e-api-a-32fdf2-7469cf674d-4c2l6 Container started 50m Normal Started pod/payload-pre-processing-e2e-api-a-32fdf2-6767d498c5-np8rq Container started 50m Normal Pulled pod/payload-processing-e2e-api-a-32fdf2-7469cf674d-4c2l6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Created pod/payload-pre-processing-e2e-api-a-32fdf2-6767d498c5-np8rq Container created 50m Warning Unhealthy pod/e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7-8q4n6 Startup probe failed: Get "http://10.134.0.137:15021/healthz/ready": dial tcp 10.134.0.137:15021: connect: connection refused 50m Normal AddedInterface pod/e2e-api-b-bbfc2e-openshift-default-6c585b5767-jlcxj Add eth0 [10.134.0.142/23] from ovn-kubernetes 50m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-32fdf2-6767d498c5-np8rq Add eth0 [10.132.0.175/23] from ovn-kubernetes 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-bbfc2e Scaled up replica set payload-processing-e2e-api-b-bbfc2e-6ddb5f6584 from 0 to 1 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-bbfc2e-6ddb5f6584 Created pod: payload-processing-e2e-api-b-bbfc2e-6ddb5f6584-hc4kx 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-bbfc2e-78f95f4d55 Error creating: pods "payload-pre-processing-e2e-api-b-bbfc2e-78f95f4d55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-bbfc2e: serviceaccount "payload-processing-e2e-api-b-bbfc2e" not found 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-bbfc2e Scaled up replica set payload-pre-processing-e2e-api-b-bbfc2e-78f95f4d55 from 0 to 1 50m Warning FailedCreate replicaset/payload-processing-e2e-api-b-bbfc2e-6ddb5f6584 Error creating: pods "payload-processing-e2e-api-b-bbfc2e-6ddb5f6584-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-bbfc2e: serviceaccount "payload-processing-e2e-api-b-bbfc2e" not found 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-bbfc2e-78f95f4d55 Created pod: payload-pre-processing-e2e-api-b-bbfc2e-78f95f4d55-5tprx 50m Normal Pulled pod/payload-pre-processing-e2e-api-b-bbfc2e-78f95f4d55-5tprx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/payload-processing-e2e-api-b-bbfc2e-6ddb5f6584-hc4kx Add eth0 [10.133.0.96/23] from ovn-kubernetes 50m Normal Started pod/payload-processing-e2e-api-b-bbfc2e-6ddb5f6584-hc4kx Container started 50m Normal Created pod/payload-processing-e2e-api-b-bbfc2e-6ddb5f6584-hc4kx Container created 50m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-bbfc2e-78f95f4d55-5tprx Add eth0 [10.132.0.177/23] from ovn-kubernetes 50m Normal Created pod/payload-pre-processing-e2e-api-b-bbfc2e-78f95f4d55-5tprx Container created 50m Normal Started pod/payload-pre-processing-e2e-api-b-bbfc2e-78f95f4d55-5tprx Container started 50m Normal Pulled pod/payload-processing-e2e-api-b-bbfc2e-6ddb5f6584-hc4kx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal SuccessfulCreate replicaset/e2e-api-a-65b0e2-openshift-default-56d854d69c Created pod: e2e-api-a-65b0e2-openshift-default-56d854d69c-tdh5n 50m Normal ScalingReplicaSet deployment/e2e-api-a-65b0e2-openshift-default Scaled up replica set e2e-api-a-65b0e2-openshift-default-56d854d69c from 0 to 1 50m Normal AddedLabel gateway/e2e-api-a-65b0e2 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-65b0e2 50m Normal AddedInterface pod/e2e-api-a-65b0e2-openshift-default-56d854d69c-tdh5n Add eth0 [10.134.0.143/23] from ovn-kubernetes 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-d8a3d1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-d8a3d1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning FailedCreate replicaset/payload-processing-e2e-api-a-65b0e2-765dd6d498 Error creating: pods "payload-processing-e2e-api-a-65b0e2-765dd6d498-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-65b0e2: serviceaccount "payload-processing-e2e-api-a-65b0e2" not found 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-65b0e2-765dd6d498 Created pod: payload-processing-e2e-api-a-65b0e2-765dd6d498-5n54k 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-65b0e2-5fc849b4cf Error creating: pods "payload-pre-processing-e2e-api-a-65b0e2-5fc849b4cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-65b0e2: serviceaccount "payload-processing-e2e-api-a-65b0e2" not found 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-65b0e2 Scaled up replica set payload-processing-e2e-api-a-65b0e2-765dd6d498 from 0 to 1 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-65b0e2 Scaled up replica set payload-pre-processing-e2e-api-a-65b0e2-5fc849b4cf from 0 to 1 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-65b0e2-5fc849b4cf Created pod: payload-pre-processing-e2e-api-a-65b0e2-5fc849b4cf-m2md4 50m Normal Killing pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-6t8qk Stopping container istio-proxy 50m Normal Killing pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-94n6d Stopping container istio-proxy 50m Normal Created pod/payload-pre-processing-e2e-api-a-65b0e2-5fc849b4cf-m2md4 Container created 50m Normal Killing pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-btjw2 Stopping container istio-proxy 50m Normal ScalingReplicaSet deployment/e2e-worker-w5-434ce3-openshift-default Scaled down replica set e2e-worker-w5-434ce3-openshift-default-58c7774759 from 10 to 7 50m Normal Killing pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-2jb6v Stopping container istio-proxy 50m Normal SuccessfulDelete replicaset/e2e-worker-w5-434ce3-openshift-default-58c7774759 Deleted pod: e2e-worker-w5-434ce3-openshift-default-58c7774759-mqzl2 50m Normal SuccessfulDelete replicaset/e2e-worker-w5-434ce3-openshift-default-58c7774759 Deleted pod: e2e-worker-w5-434ce3-openshift-default-58c7774759-6t8qk 50m Normal Killing pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-4wl8g Stopping container istio-proxy 50m Normal SuccessfulDelete replicaset/e2e-worker-w6-2d619f-openshift-default-85b84758df Deleted pod: e2e-worker-w6-2d619f-openshift-default-85b84758df-cd47x 50m Normal SuccessfulDelete replicaset/e2e-worker-w6-2d619f-openshift-default-85b84758df Deleted pod: e2e-worker-w6-2d619f-openshift-default-85b84758df-btjw2 50m Normal SuccessfulDelete replicaset/e2e-worker-w6-2d619f-openshift-default-85b84758df Deleted pod: e2e-worker-w6-2d619f-openshift-default-85b84758df-qzh84 50m Normal SuccessfulDelete replicaset/e2e-worker-w1-e19850-openshift-default-9f855bd47 Deleted pod: e2e-worker-w1-e19850-openshift-default-9f855bd47-94n6d 50m Normal SuccessfulDelete replicaset/e2e-worker-w1-e19850-openshift-default-9f855bd47 Deleted pod: e2e-worker-w1-e19850-openshift-default-9f855bd47-vm8xs 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-434ce3-openshift-default New size: 7; reason: All metrics below target 50m Normal SuccessfulDelete replicaset/e2e-worker-w1-e19850-openshift-default-9f855bd47 Deleted pod: e2e-worker-w1-e19850-openshift-default-9f855bd47-f72v5 50m Normal ScalingReplicaSet deployment/e2e-worker-w4-50d738-openshift-default Scaled down replica set e2e-worker-w4-50d738-openshift-default-8d6476cbc from 6 to 5 50m Normal Killing pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-cd47x Stopping container istio-proxy 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-50d738-openshift-default New size: 5; reason: All metrics below target 50m Normal Started pod/payload-pre-processing-e2e-api-a-65b0e2-5fc849b4cf-m2md4 Container started 50m Normal ScalingReplicaSet deployment/e2e-worker-w1-e19850-openshift-default Scaled down replica set e2e-worker-w1-e19850-openshift-default-9f855bd47 from 10 to 7 50m Normal Pulled pod/payload-pre-processing-e2e-api-a-65b0e2-5fc849b4cf-m2md4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-65b0e2-5fc849b4cf-m2md4 Add eth0 [10.133.0.97/23] from ovn-kubernetes 50m Normal SuccessfulDelete replicaset/e2e-worker-w5-434ce3-openshift-default-58c7774759 Deleted pod: e2e-worker-w5-434ce3-openshift-default-58c7774759-4wl8g 50m Normal ScalingReplicaSet deployment/e2e-worker-w0-88a4d9-openshift-default Scaled down replica set e2e-worker-w0-88a4d9-openshift-default-857757469c from 8 to 5 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-88a4d9-openshift-default New size: 5; reason: All metrics below target 50m Normal SuccessfulDelete replicaset/e2e-worker-w4-50d738-openshift-default-8d6476cbc Deleted pod: e2e-worker-w4-50d738-openshift-default-8d6476cbc-2jb6v 50m Normal SuccessfulDelete replicaset/e2e-worker-w0-88a4d9-openshift-default-857757469c Deleted pod: e2e-worker-w0-88a4d9-openshift-default-857757469c-x96rf 50m Normal Killing pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-f72v5 Stopping container istio-proxy 50m Normal SuccessfulDelete replicaset/e2e-worker-w0-88a4d9-openshift-default-857757469c Deleted pod: e2e-worker-w0-88a4d9-openshift-default-857757469c-nqgqh 50m Normal SuccessfulDelete replicaset/e2e-worker-w0-88a4d9-openshift-default-857757469c Deleted pod: e2e-worker-w0-88a4d9-openshift-default-857757469c-hdk26 50m Normal Started pod/payload-processing-e2e-api-a-65b0e2-765dd6d498-5n54k Container started 50m Normal Killing pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-qzh84 Stopping container istio-proxy 50m Normal Created pod/payload-processing-e2e-api-a-65b0e2-765dd6d498-5n54k Container created 50m Normal Pulled pod/payload-processing-e2e-api-a-65b0e2-765dd6d498-5n54k Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Killing pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-x96rf Stopping container istio-proxy 50m Normal Killing pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-vm8xs Stopping container istio-proxy 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-2d619f-openshift-default New size: 5; reason: All metrics below target 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e19850-openshift-default New size: 7; reason: All metrics below target 50m Normal Killing pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-mqzl2 Stopping container istio-proxy 50m Normal Killing pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-nqgqh Stopping container istio-proxy 50m Normal Killing pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-hdk26 Stopping container istio-proxy 50m Normal AddedInterface pod/payload-processing-e2e-api-a-65b0e2-765dd6d498-5n54k Add eth0 [10.133.0.98/23] from ovn-kubernetes 50m Normal ScalingReplicaSet deployment/e2e-worker-w6-2d619f-openshift-default Scaled down replica set e2e-worker-w6-2d619f-openshift-default-85b84758df from 8 to 5 50m Warning ProbeError pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-6t8qk Readiness probe error: HTTP probe failed with statuscode: 503... 50m Normal Started pod/e2e-shared-b-w6-26f8ba-openshift-default-569ccb4c87-pcd2l Container started 50m Normal Pulled pod/e2e-shared-b-w6-26f8ba-openshift-default-569ccb4c87-pcd2l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-api-a-4c5345-openshift-default-86899fff89-l8tzb Container started 50m Normal Created pod/e2e-api-a-4c5345-openshift-default-86899fff89-l8tzb Container created 50m Normal Pulled pod/e2e-api-a-4c5345-openshift-default-86899fff89-l8tzb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-shared-b-w6-26f8ba-openshift-default-569ccb4c87-pcd2l Container created 50m Warning Unhealthy pod/e2e-shared-b-w6-26f8ba-openshift-default-569ccb4c87-pcd2l Startup probe failed: Get "http://10.132.0.170:15021/healthz/ready": dial tcp 10.132.0.170:15021: connect: connection refused 50m Warning ProbeError pod/e2e-shared-b-w6-26f8ba-openshift-default-569ccb4c87-pcd2l Startup probe error: Get "http://10.132.0.170:15021/healthz/ready": dial tcp 10.132.0.170:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-api-a-4c5345-openshift-default-86899fff89-l8tzb Startup probe failed: Get "http://10.133.0.94:15021/healthz/ready": dial tcp 10.133.0.94:15021: connect: connection refused 50m Warning ProbeError pod/e2e-api-a-4c5345-openshift-default-86899fff89-l8tzb Startup probe error: Get "http://10.133.0.94:15021/healthz/ready": dial tcp 10.133.0.94:15021: connect: connection refused... 50m Normal Killing pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-gwqld Stopping container istio-proxy 50m Normal SuccessfulDelete replicaset/e2e-worker-w2-553753-openshift-default-558949869c Deleted pod: e2e-worker-w2-553753-openshift-default-558949869c-mn9sk 50m Normal Killing pod/e2e-worker-w2-553753-openshift-default-558949869c-8l4s6 Stopping container istio-proxy 50m Normal ScalingReplicaSet deployment/e2e-worker-w1-e19850-openshift-default Scaled down replica set e2e-worker-w1-e19850-openshift-default-9f855bd47 from 7 to 5 50m Normal Killing pod/e2e-worker-w2-553753-openshift-default-558949869c-rdndg Stopping container istio-proxy 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-434ce3-openshift-default New size: 5; reason: All metrics below target 50m Normal Killing pod/e2e-worker-w2-553753-openshift-default-558949869c-mn9sk Stopping container istio-proxy 50m Normal ScalingReplicaSet deployment/e2e-worker-w5-434ce3-openshift-default Scaled down replica set e2e-worker-w5-434ce3-openshift-default-58c7774759 from 7 to 5 50m Normal SuccessfulDelete replicaset/e2e-worker-w2-553753-openshift-default-558949869c Deleted pod: e2e-worker-w2-553753-openshift-default-558949869c-8l4s6 50m Normal SuccessfulDelete replicaset/e2e-worker-w5-434ce3-openshift-default-58c7774759 Deleted pod: e2e-worker-w5-434ce3-openshift-default-58c7774759-wmp98 50m Normal SuccessfulDelete replicaset/e2e-worker-w2-553753-openshift-default-558949869c Deleted pod: e2e-worker-w2-553753-openshift-default-558949869c-j5dc2 50m Normal SuccessfulDelete replicaset/e2e-worker-w2-553753-openshift-default-558949869c Deleted pod: e2e-worker-w2-553753-openshift-default-558949869c-jzgb6 50m Normal Killing pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-wmp98 Stopping container istio-proxy 50m Normal SuccessfulDelete replicaset/e2e-worker-w5-434ce3-openshift-default-58c7774759 Deleted pod: e2e-worker-w5-434ce3-openshift-default-58c7774759-vgf7q 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-553753-openshift-default New size: 5; reason: All metrics below target 50m Normal Killing pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-hz2jq Stopping container istio-proxy 50m Normal SuccessfulDelete replicaset/e2e-worker-w2-553753-openshift-default-558949869c Deleted pod: e2e-worker-w2-553753-openshift-default-558949869c-rdndg 50m Normal SuccessfulDelete replicaset/e2e-worker-w1-e19850-openshift-default-9f855bd47 Deleted pod: e2e-worker-w1-e19850-openshift-default-9f855bd47-hz2jq 50m Normal SuccessfulDelete replicaset/e2e-worker-w1-e19850-openshift-default-9f855bd47 Deleted pod: e2e-worker-w1-e19850-openshift-default-9f855bd47-gwqld 50m Normal Killing pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-vgf7q Stopping container istio-proxy 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e19850-openshift-default New size: 5; reason: All metrics below target 50m Normal Killing pod/e2e-worker-w2-553753-openshift-default-558949869c-j5dc2 Stopping container istio-proxy 50m Normal Killing pod/e2e-worker-w2-553753-openshift-default-558949869c-jzgb6 Stopping container istio-proxy 50m Normal ScalingReplicaSet deployment/e2e-worker-w2-553753-openshift-default Scaled down replica set e2e-worker-w2-553753-openshift-default-558949869c from 10 to 5 50m Warning Unhealthy pod/e2e-worker-w2-553753-openshift-default-558949869c-j5dc2 Readiness probe failed: HTTP probe failed with statuscode: 503 50m Warning ProbeError pod/e2e-worker-w2-553753-openshift-default-558949869c-j5dc2 Readiness probe error: HTTP probe failed with statuscode: 503... 50m Normal AddedInterface pod/e2e-inf-a-4b5e2b-openshift-default-7496d87895-f5vss Add eth0 [10.133.0.100/23] from ovn-kubernetes 50m Normal ScalingReplicaSet deployment/e2e-inf-a-4b5e2b-openshift-default Scaled up replica set e2e-inf-a-4b5e2b-openshift-default-7496d87895 from 0 to 1 50m Normal AddedLabel gateway/e2e-inf-a-4b5e2b Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-4b5e2b 50m Normal SuccessfulCreate replicaset/e2e-inf-a-4b5e2b-openshift-default-7496d87895 Created pod: e2e-inf-a-4b5e2b-openshift-default-7496d87895-f5vss 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-4c5345-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-4c5345-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning Unhealthy pod/e2e-worker-w2-553753-openshift-default-558949869c-8l4s6 Readiness probe failed: Get "http://10.134.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w2-553753-openshift-default-558949869c-8l4s6 Readiness probe error: Get "http://10.134.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-4b5e2b Scaled up replica set payload-processing-e2e-inf-a-4b5e2b-56d495d764 from 0 to 1 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-4b5e2b Scaled up replica set payload-pre-processing-e2e-inf-a-4b5e2b-7dd898677f from 0 to 1 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-4b5e2b-7dd898677f Error creating: pods "payload-pre-processing-e2e-inf-a-4b5e2b-7dd898677f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-4b5e2b: serviceaccount "payload-processing-e2e-inf-a-4b5e2b" not found 50m Normal Started pod/payload-pre-processing-e2e-inf-a-4b5e2b-7dd898677f-6768h Container started 50m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-4b5e2b-56d495d764 Error creating: pods "payload-processing-e2e-inf-a-4b5e2b-56d495d764-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-4b5e2b: serviceaccount "payload-processing-e2e-inf-a-4b5e2b" not found 50m Normal Pulled pod/payload-processing-e2e-inf-a-4b5e2b-56d495d764-wxzjw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Pulled pod/payload-pre-processing-e2e-inf-a-4b5e2b-7dd898677f-6768h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-4b5e2b-7dd898677f Created pod: payload-pre-processing-e2e-inf-a-4b5e2b-7dd898677f-6768h 50m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-4b5e2b-7dd898677f-6768h Add eth0 [10.132.0.180/23] from ovn-kubernetes 50m Normal AddedInterface pod/payload-processing-e2e-inf-a-4b5e2b-56d495d764-wxzjw Add eth0 [10.132.0.181/23] from ovn-kubernetes 50m Normal Created pod/payload-pre-processing-e2e-inf-a-4b5e2b-7dd898677f-6768h Container created 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-4b5e2b-56d495d764 Created pod: payload-processing-e2e-inf-a-4b5e2b-56d495d764-wxzjw 50m Normal Killing pod/payload-pre-processing-e2e-mt-8b1efeb6-68787c86c9-wpjjk Stopping container payload-pre-processing 50m Normal Started pod/payload-processing-e2e-inf-a-4b5e2b-56d495d764-wxzjw Container started 50m Normal Created pod/payload-processing-e2e-inf-a-4b5e2b-56d495d764-wxzjw Container created 50m Normal ScalingReplicaSet deployment/e2e-api-a-2c0223-openshift-default Scaled up replica set e2e-api-a-2c0223-openshift-default-6958d556b9 from 0 to 1 50m Normal AddedLabel gateway/e2e-api-a-2c0223 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-2c0223 50m Normal SuccessfulCreate replicaset/e2e-api-a-2c0223-openshift-default-6958d556b9 Created pod: e2e-api-a-2c0223-openshift-default-6958d556b9-dwtg9 50m Normal Killing pod/payload-processing-e2e-mt-8b1efeb6-5d7dd8ff94-q446n Stopping container payload-processing 50m Normal AddedInterface pod/e2e-api-a-2c0223-openshift-default-6958d556b9-dwtg9 Add eth0 [10.134.0.145/23] from ovn-kubernetes 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-2c0223-5954dfbb87 Error creating: pods "payload-pre-processing-e2e-api-a-2c0223-5954dfbb87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-2c0223: serviceaccount "payload-processing-e2e-api-a-2c0223" not found 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-2c0223-5954dfbb87 Created pod: payload-pre-processing-e2e-api-a-2c0223-5954dfbb87-n5vnv 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-2c0223 Scaled up replica set payload-pre-processing-e2e-api-a-2c0223-5954dfbb87 from 0 to 1 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-2c0223-696c7b845d Created pod: payload-processing-e2e-api-a-2c0223-696c7b845d-txtnq 50m Warning FailedCreate replicaset/payload-processing-e2e-api-a-2c0223-696c7b845d Error creating: pods "payload-processing-e2e-api-a-2c0223-696c7b845d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-2c0223: serviceaccount "payload-processing-e2e-api-a-2c0223" not found 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-2c0223 Scaled up replica set payload-processing-e2e-api-a-2c0223-696c7b845d from 0 to 1 50m Normal Started pod/payload-pre-processing-e2e-api-a-2c0223-5954dfbb87-n5vnv Container started 50m Normal Pulled pod/payload-processing-e2e-api-a-2c0223-696c7b845d-txtnq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-2c0223-5954dfbb87-n5vnv Add eth0 [10.132.0.183/23] from ovn-kubernetes 50m Normal AddedInterface pod/payload-processing-e2e-api-a-2c0223-696c7b845d-txtnq Add eth0 [10.132.0.184/23] from ovn-kubernetes 50m Normal Pulled pod/payload-pre-processing-e2e-api-a-2c0223-5954dfbb87-n5vnv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Created pod/payload-pre-processing-e2e-api-a-2c0223-5954dfbb87-n5vnv Container created 50m Normal Created pod/payload-processing-e2e-api-a-2c0223-696c7b845d-txtnq Container created 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-4c5345-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Normal Started pod/payload-processing-e2e-api-a-2c0223-696c7b845d-txtnq Container started 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-4c5345-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-26f8ba-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-26f8ba-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Normal Killing pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-zglz6 Stopping container istio-proxy 49m Normal Killing pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-5hj9v Stopping container istio-proxy 49m Normal ScalingReplicaSet deployment/e2e-worker-w3-5187e1-openshift-default Scaled down replica set e2e-worker-w3-5187e1-openshift-default-946849bdf from 10 to 7 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-5187e1-openshift-default New size: 7; reason: All metrics below target 49m Normal SuccessfulDelete replicaset/e2e-worker-w3-5187e1-openshift-default-946849bdf Deleted pod: e2e-worker-w3-5187e1-openshift-default-946849bdf-zglz6 49m Normal SuccessfulDelete replicaset/e2e-worker-w3-5187e1-openshift-default-946849bdf Deleted pod: e2e-worker-w3-5187e1-openshift-default-946849bdf-5hj9v 49m Normal SuccessfulDelete replicaset/e2e-worker-w3-5187e1-openshift-default-946849bdf Deleted pod: e2e-worker-w3-5187e1-openshift-default-946849bdf-wtzl9 49m Normal Killing pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-wtzl9 Stopping container istio-proxy 49m Normal SuccessfulCreate replicaset/e2e-api-b-b826b6-openshift-default-785fbbd7 Created pod: e2e-api-b-b826b6-openshift-default-785fbbd7-rwr56 49m Normal AddedLabel gateway/e2e-api-b-b826b6 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-b826b6 49m Normal ScalingReplicaSet deployment/e2e-api-b-b826b6-openshift-default Scaled up replica set e2e-api-b-b826b6-openshift-default-785fbbd7 from 0 to 1 49m Normal AddedInterface pod/e2e-api-b-b826b6-openshift-default-785fbbd7-rwr56 Add eth0 [10.134.0.147/23] from ovn-kubernetes 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-5187e1-openshift-default New size: 6; reason: All metrics below target 49m Normal Killing pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-z7tn8 Stopping container istio-proxy 49m Normal SuccessfulDelete replicaset/e2e-worker-w3-5187e1-openshift-default-946849bdf Deleted pod: e2e-worker-w3-5187e1-openshift-default-946849bdf-z7tn8 49m Normal ScalingReplicaSet deployment/e2e-worker-w3-5187e1-openshift-default Scaled down replica set e2e-worker-w3-5187e1-openshift-default-946849bdf from 7 to 6 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-b826b6-86b86bbbff Error creating: pods "payload-pre-processing-e2e-api-b-b826b6-86b86bbbff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-b826b6: serviceaccount "payload-processing-e2e-api-b-b826b6" not found 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-b826b6 Scaled up replica set payload-pre-processing-e2e-api-b-b826b6-86b86bbbff from 0 to 1 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-b826b6-86b86bbbff Created pod: payload-pre-processing-e2e-api-b-b826b6-86b86bbbff-rkg2w 49m Normal Started pod/e2e-api-a-32fdf2-openshift-default-798db88bbc-52z75 Container started 49m Normal Created pod/e2e-api-a-32fdf2-openshift-default-798db88bbc-52z75 Container created 49m Normal Pulled pod/e2e-api-a-32fdf2-openshift-default-798db88bbc-52z75 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-b826b6 Scaled up replica set payload-processing-e2e-api-b-b826b6-65dbdbf768 from 0 to 1 49m Warning FailedCreate replicaset/payload-processing-e2e-api-b-b826b6-65dbdbf768 Error creating: pods "payload-processing-e2e-api-b-b826b6-65dbdbf768-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-b826b6: serviceaccount "payload-processing-e2e-api-b-b826b6" not found 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-b826b6-65dbdbf768 Created pod: payload-processing-e2e-api-b-b826b6-65dbdbf768-v9wnw 49m Normal Pulled pod/payload-processing-e2e-api-b-b826b6-65dbdbf768-v9wnw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-b826b6-86b86bbbff-rkg2w Add eth0 [10.132.0.186/23] from ovn-kubernetes 49m Normal Pulled pod/payload-pre-processing-e2e-api-b-b826b6-86b86bbbff-rkg2w Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Created pod/payload-pre-processing-e2e-api-b-b826b6-86b86bbbff-rkg2w Container created 49m Normal Started pod/payload-pre-processing-e2e-api-b-b826b6-86b86bbbff-rkg2w Container started 49m Normal AddedInterface pod/payload-processing-e2e-api-b-b826b6-65dbdbf768-v9wnw Add eth0 [10.132.0.187/23] from ovn-kubernetes 49m Normal Started pod/payload-processing-e2e-api-b-b826b6-65dbdbf768-v9wnw Container started 49m Normal Created pod/payload-processing-e2e-api-b-b826b6-65dbdbf768-v9wnw Container created 49m Warning ProbeError pod/e2e-api-a-32fdf2-openshift-default-798db88bbc-52z75 Startup probe error: Get "http://10.134.0.141:15021/healthz/ready": dial tcp 10.134.0.141:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-api-a-32fdf2-openshift-default-798db88bbc-52z75 Startup probe failed: Get "http://10.134.0.141:15021/healthz/ready": dial tcp 10.134.0.141:15021: connect: connection refused 49m Normal Killing pod/payload-pre-processing-e2e-api-a-4c5345-5548f954c-kk85c Stopping container payload-pre-processing 49m Normal Killing pod/payload-processing-e2e-api-a-4c5345-6df87f7798-fkts8 Stopping container payload-processing 49m Normal ScalingReplicaSet deployment/e2e-api-b-5dd3b1-openshift-default Scaled up replica set e2e-api-b-5dd3b1-openshift-default-684dbd6c7c from 0 to 1 49m Normal AddedInterface pod/e2e-api-b-5dd3b1-openshift-default-684dbd6c7c-k4v6l Add eth0 [10.132.0.188/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/e2e-api-b-5dd3b1-openshift-default-684dbd6c7c Created pod: e2e-api-b-5dd3b1-openshift-default-684dbd6c7c-k4v6l 49m Normal AddedLabel gateway/e2e-api-b-5dd3b1 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-5dd3b1 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-32fdf2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-32fdf2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-5dd3b1 Scaled up replica set payload-processing-e2e-api-b-5dd3b1-5b45c4ff8 from 0 to 1 49m Warning FailedCreate replicaset/payload-processing-e2e-api-b-5dd3b1-5b45c4ff8 Error creating: pods "payload-processing-e2e-api-b-5dd3b1-5b45c4ff8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-5dd3b1: serviceaccount "payload-processing-e2e-api-b-5dd3b1" not found 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-5dd3b1 Scaled up replica set payload-pre-processing-e2e-api-b-5dd3b1-7bdc56b4f5 from 0 to 1 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-5dd3b1-7bdc56b4f5 Created pod: payload-pre-processing-e2e-api-b-5dd3b1-7bdc56b4f5-cxmt6 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-5dd3b1-5b45c4ff8 Created pod: payload-processing-e2e-api-b-5dd3b1-5b45c4ff8-rmshj 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-5dd3b1-7bdc56b4f5 Error creating: pods "payload-pre-processing-e2e-api-b-5dd3b1-7bdc56b4f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-5dd3b1: serviceaccount "payload-processing-e2e-api-b-5dd3b1" not found 49m Normal AddedInterface pod/payload-processing-e2e-api-b-5dd3b1-5b45c4ff8-rmshj Add eth0 [10.132.0.190/23] from ovn-kubernetes 49m Normal Created pod/payload-pre-processing-e2e-api-b-5dd3b1-7bdc56b4f5-cxmt6 Container created 49m Normal Started pod/payload-processing-e2e-api-b-5dd3b1-5b45c4ff8-rmshj Container started 49m Normal Pulled pod/payload-pre-processing-e2e-api-b-5dd3b1-7bdc56b4f5-cxmt6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Created pod/payload-processing-e2e-api-b-5dd3b1-5b45c4ff8-rmshj Container created 49m Normal Pulled pod/payload-processing-e2e-api-b-5dd3b1-5b45c4ff8-rmshj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-5dd3b1-7bdc56b4f5-cxmt6 Add eth0 [10.132.0.189/23] from ovn-kubernetes 49m Normal Started pod/payload-pre-processing-e2e-api-b-5dd3b1-7bdc56b4f5-cxmt6 Container started 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-8b1efeb6 Scaled up replica set payload-pre-processing-e2e-mt-8b1efeb6-68787c86c9 from 0 to 1 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-8b1efeb6 Scaled up replica set payload-processing-e2e-mt-8b1efeb6-5d7dd8ff94 from 0 to 1 49m Warning FailedCreate replicaset/payload-processing-e2e-mt-8b1efeb6-5d7dd8ff94 Error creating: pods "payload-processing-e2e-mt-8b1efeb6-5d7dd8ff94-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-8b1efeb6: serviceaccount "payload-processing-e2e-mt-8b1efeb6" not found 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-8b1efeb6-68787c86c9 Created pod: payload-pre-processing-e2e-mt-8b1efeb6-68787c86c9-5wqpz 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-8b1efeb6-5d7dd8ff94 Created pod: payload-processing-e2e-mt-8b1efeb6-5d7dd8ff94-5blct 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-8b1efeb6-68787c86c9 Error creating: pods "payload-pre-processing-e2e-mt-8b1efeb6-68787c86c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-8b1efeb6: serviceaccount "payload-processing-e2e-mt-8b1efeb6" not found 49m Normal AddedInterface pod/payload-pre-processing-e2e-mt-8b1efeb6-68787c86c9-5wqpz Add eth0 [10.132.0.192/23] from ovn-kubernetes 49m Normal Pulled pod/payload-pre-processing-e2e-mt-8b1efeb6-68787c86c9-5wqpz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/payload-processing-e2e-mt-8b1efeb6-5d7dd8ff94-5blct Add eth0 [10.132.0.191/23] from ovn-kubernetes 49m Normal Created pod/payload-processing-e2e-mt-8b1efeb6-5d7dd8ff94-5blct Container created 49m Normal Started pod/payload-processing-e2e-mt-8b1efeb6-5d7dd8ff94-5blct Container started 49m Normal Created pod/payload-pre-processing-e2e-mt-8b1efeb6-68787c86c9-5wqpz Container created 49m Normal Started pod/payload-pre-processing-e2e-mt-8b1efeb6-68787c86c9-5wqpz Container started 49m Normal Pulled pod/payload-processing-e2e-mt-8b1efeb6-5d7dd8ff94-5blct Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-32fdf2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Normal Started pod/e2e-inf-a-4b5e2b-openshift-default-7496d87895-f5vss Container started 49m Normal Created pod/e2e-inf-a-4b5e2b-openshift-default-7496d87895-f5vss Container created 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-32fdf2-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Normal Pulled pod/e2e-inf-a-4b5e2b-openshift-default-7496d87895-f5vss Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Pulled pod/e2e-api-a-65b0e2-openshift-default-56d854d69c-tdh5n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-api-a-65b0e2-openshift-default-56d854d69c-tdh5n Container started 49m Normal Created pod/e2e-api-a-65b0e2-openshift-default-56d854d69c-tdh5n Container created 49m Warning ProbeError pod/e2e-inf-a-4b5e2b-openshift-default-7496d87895-f5vss Startup probe error: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-inf-a-4b5e2b-openshift-default-7496d87895-f5vss Startup probe failed: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused 48m Warning ProbeError pod/e2e-api-a-65b0e2-openshift-default-56d854d69c-tdh5n Startup probe error: Get "http://10.134.0.143:15021/healthz/ready": dial tcp 10.134.0.143:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-api-a-65b0e2-openshift-default-56d854d69c-tdh5n Startup probe failed: Get "http://10.134.0.143:15021/healthz/ready": dial tcp 10.134.0.143:15021: connect: connection refused 48m Normal ScalingReplicaSet deployment/e2e-inf-b-2d6076-openshift-default Scaled up replica set e2e-inf-b-2d6076-openshift-default-5bdb59ccf9 from 0 to 1 48m Normal SuccessfulCreate replicaset/e2e-inf-b-2d6076-openshift-default-5bdb59ccf9 Created pod: e2e-inf-b-2d6076-openshift-default-5bdb59ccf9-48ljn 48m Normal AddedLabel gateway/e2e-inf-b-2d6076 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-2d6076 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-5dd3b1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-5dd3b1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Normal AddedInterface pod/e2e-inf-b-2d6076-openshift-default-5bdb59ccf9-48ljn Add eth0 [10.134.0.151/23] from ovn-kubernetes 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-2d6076 Scaled up replica set payload-processing-e2e-inf-b-2d6076-d576997dd from 0 to 1 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-2d6076-dc5c6ff87 Error creating: pods "payload-pre-processing-e2e-inf-b-2d6076-dc5c6ff87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-2d6076: serviceaccount "payload-processing-e2e-inf-b-2d6076" not found 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-2d6076 Scaled up replica set payload-pre-processing-e2e-inf-b-2d6076-dc5c6ff87 from 0 to 1 48m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-2d6076-d576997dd Error creating: pods "payload-processing-e2e-inf-b-2d6076-d576997dd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-2d6076: serviceaccount "payload-processing-e2e-inf-b-2d6076" not found 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-2d6076-dc5c6ff87 Created pod: payload-pre-processing-e2e-inf-b-2d6076-dc5c6ff87-4d8d6 48m Normal Started pod/payload-pre-processing-e2e-inf-b-2d6076-dc5c6ff87-4d8d6 Container started 48m Normal Created pod/payload-pre-processing-e2e-inf-b-2d6076-dc5c6ff87-4d8d6 Container created 48m Normal Pulled pod/payload-pre-processing-e2e-inf-b-2d6076-dc5c6ff87-4d8d6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-2d6076-dc5c6ff87-4d8d6 Add eth0 [10.132.0.193/23] from ovn-kubernetes 48m Normal Started pod/payload-processing-e2e-inf-b-2d6076-d576997dd-rh48p Container started 48m Normal Created pod/payload-processing-e2e-inf-b-2d6076-d576997dd-rh48p Container created 48m Normal Pulled pod/payload-processing-e2e-inf-b-2d6076-d576997dd-rh48p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-2d6076-d576997dd Created pod: payload-processing-e2e-inf-b-2d6076-d576997dd-rh48p 48m Normal AddedInterface pod/payload-processing-e2e-inf-b-2d6076-d576997dd-rh48p Add eth0 [10.133.0.101/23] from ovn-kubernetes 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-65b0e2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-65b0e2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Normal AddedLabel gateway/e2e-api-b-22edc1 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-22edc1 48m Normal ScalingReplicaSet deployment/e2e-api-b-22edc1-openshift-default Scaled up replica set e2e-api-b-22edc1-openshift-default-b9f7f8cb5 from 0 to 1 48m Normal SuccessfulCreate replicaset/e2e-api-b-22edc1-openshift-default-b9f7f8cb5 Created pod: e2e-api-b-22edc1-openshift-default-b9f7f8cb5-qvnsr 48m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-0847aa-openshift-default Scaled down replica set e2e-shared-a-w0-0847aa-openshift-default-75d5774789 from 8 to 6 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-0847aa-openshift-default New size: 6; reason: All metrics below target 48m Normal SuccessfulDelete replicaset/e2e-shared-a-w0-0847aa-openshift-default-75d5774789 Deleted pod: e2e-shared-a-w0-0847aa-openshift-default-75d5774789-rltc7 48m Normal SuccessfulDelete replicaset/e2e-shared-a-w0-0847aa-openshift-default-75d5774789 Deleted pod: e2e-shared-a-w0-0847aa-openshift-default-75d5774789-mcjd8 48m Normal AddedInterface pod/e2e-api-b-22edc1-openshift-default-b9f7f8cb5-qvnsr Add eth0 [10.134.0.152/23] from ovn-kubernetes 48m Normal Killing pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-rltc7 Stopping container istio-proxy 48m Normal Killing pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-mcjd8 Stopping container istio-proxy 48m Warning FailedCreate replicaset/payload-processing-e2e-api-b-22edc1-998976c8b Error creating: pods "payload-processing-e2e-api-b-22edc1-998976c8b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-22edc1: serviceaccount "payload-processing-e2e-api-b-22edc1" not found 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-22edc1 Scaled up replica set payload-pre-processing-e2e-api-b-22edc1-5bb4668bc from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-22edc1-5bb4668bc Created pod: payload-pre-processing-e2e-api-b-22edc1-5bb4668bc-59h47 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-22edc1-5bb4668bc Error creating: pods "payload-pre-processing-e2e-api-b-22edc1-5bb4668bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-22edc1: serviceaccount "payload-processing-e2e-api-b-22edc1" not found 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-22edc1 Scaled up replica set payload-processing-e2e-api-b-22edc1-998976c8b from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-22edc1-998976c8b Created pod: payload-processing-e2e-api-b-22edc1-998976c8b-4whv2 48m Normal Created pod/payload-processing-e2e-api-b-22edc1-998976c8b-4whv2 Container created 48m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-22edc1-5bb4668bc-59h47 Add eth0 [10.132.0.195/23] from ovn-kubernetes 48m Normal Pulled pod/payload-pre-processing-e2e-api-b-22edc1-5bb4668bc-59h47 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-processing-e2e-api-b-22edc1-998976c8b-4whv2 Add eth0 [10.132.0.194/23] from ovn-kubernetes 48m Normal Pulled pod/payload-processing-e2e-api-b-22edc1-998976c8b-4whv2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Started pod/payload-pre-processing-e2e-api-b-22edc1-5bb4668bc-59h47 Container started 48m Normal Started pod/payload-processing-e2e-api-b-22edc1-998976c8b-4whv2 Container started 48m Normal Created pod/payload-pre-processing-e2e-api-b-22edc1-5bb4668bc-59h47 Container created 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-4b5e2b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-4b5e2b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-65b0e2-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-65b0e2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Normal Killing pod/e2e-api-a-4c5345-openshift-default-86899fff89-l8tzb Stopping container istio-proxy 48m Normal Started pod/e2e-api-b-b826b6-openshift-default-785fbbd7-rwr56 Container started 48m Normal Created pod/e2e-api-b-b826b6-openshift-default-785fbbd7-rwr56 Container created 48m Normal Pulled pod/e2e-api-b-b826b6-openshift-default-785fbbd7-rwr56 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Pulled pod/e2e-api-a-2c0223-openshift-default-6958d556b9-dwtg9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-api-a-2c0223-openshift-default-6958d556b9-dwtg9 Container created 48m Normal Started pod/e2e-api-a-2c0223-openshift-default-6958d556b9-dwtg9 Container started 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-b826b6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-b826b6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning Unhealthy pod/e2e-api-b-b826b6-openshift-default-785fbbd7-rwr56 Startup probe failed: Get "http://10.134.0.147:15021/healthz/ready": dial tcp 10.134.0.147:15021: connect: connection refused 48m Warning ProbeError pod/e2e-api-b-b826b6-openshift-default-785fbbd7-rwr56 Startup probe error: Get "http://10.134.0.147:15021/healthz/ready": dial tcp 10.134.0.147:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-api-a-2c0223-openshift-default-6958d556b9-dwtg9 Startup probe failed: Get "http://10.134.0.145:15021/healthz/ready": dial tcp 10.134.0.145:15021: connect: connection refused 48m Warning ProbeError pod/e2e-api-a-2c0223-openshift-default-6958d556b9-dwtg9 Startup probe error: Get "http://10.134.0.145:15021/healthz/ready": dial tcp 10.134.0.145:15021: connect: connection refused... 48m Normal Killing pod/payload-processing-e2e-api-b-bbfc2e-6ddb5f6584-hc4kx Stopping container payload-processing 48m Normal Killing pod/payload-pre-processing-e2e-api-b-bbfc2e-78f95f4d55-5tprx Stopping container payload-pre-processing 48m Normal Killing pod/payload-pre-processing-e2e-api-a-32fdf2-6767d498c5-np8rq Stopping container payload-pre-processing 48m Normal Killing pod/payload-processing-e2e-api-a-32fdf2-7469cf674d-4c2l6 Stopping container payload-processing 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-b826b6-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-b826b6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning ProbeError pod/e2e-api-b-bbfc2e-openshift-default-6c585b5767-jlcxj Readiness probe error: Get "http://10.134.0.142:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-api-b-bbfc2e-openshift-default-6c585b5767-jlcxj Readiness probe failed: Get "http://10.134.0.142:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Normal Pulled pod/e2e-api-b-bbfc2e-openshift-default-6c585b5767-jlcxj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Started pod/e2e-api-b-bbfc2e-openshift-default-6c585b5767-jlcxj Container started 47m Normal Created pod/e2e-api-b-bbfc2e-openshift-default-6c585b5767-jlcxj Container created 47m Warning Unhealthy pod/e2e-api-b-bbfc2e-openshift-default-6c585b5767-jlcxj Startup probe failed: Get "http://10.134.0.142:15021/healthz/ready": dial tcp 10.134.0.142:15021: connect: connection refused 47m Warning ProbeError pod/e2e-api-b-bbfc2e-openshift-default-6c585b5767-jlcxj Startup probe error: Get "http://10.134.0.142:15021/healthz/ready": dial tcp 10.134.0.142:15021: connect: connection refused... 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-2c0223-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-2c0223-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-bbfc2e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-bbfc2e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Normal Killing pod/payload-pre-processing-e2e-api-a-65b0e2-5fc849b4cf-m2md4 Stopping container payload-pre-processing 47m Normal Killing pod/payload-processing-e2e-api-a-65b0e2-765dd6d498-5n54k Stopping container payload-processing 47m Normal Killing pod/payload-pre-processing-e2e-mt-8b1efeb6-68787c86c9-5wqpz Stopping container payload-pre-processing 47m Normal Killing pod/payload-processing-e2e-mt-8b1efeb6-5d7dd8ff94-5blct Stopping container payload-processing 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-8b1efeb6-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 47m Normal SuccessfulCreate replicaset/e2e-mt-8b1efeb6-openshift-default-5cf66d64d5 Created pod: e2e-mt-8b1efeb6-openshift-default-5cf66d64d5-2gjrn 47m Normal ScalingReplicaSet deployment/e2e-mt-8b1efeb6-openshift-default Scaled up replica set e2e-mt-8b1efeb6-openshift-default-5cf66d64d5 from 1 to 2 47m Normal Created pod/e2e-mt-8b1efeb6-openshift-default-5cf66d64d5-2gjrn Container created 47m Normal Pulled pod/e2e-mt-8b1efeb6-openshift-default-5cf66d64d5-2gjrn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Started pod/e2e-mt-8b1efeb6-openshift-default-5cf66d64d5-2gjrn Container started 47m Normal AddedInterface pod/e2e-mt-8b1efeb6-openshift-default-5cf66d64d5-2gjrn Add eth0 [10.134.0.154/23] from ovn-kubernetes 47m Normal SuccessfulCreate replicaset/e2e-inf-a-b1ec3a-openshift-default-7675569d9 Created pod: e2e-inf-a-b1ec3a-openshift-default-7675569d9-qjkx2 47m Normal ScalingReplicaSet deployment/e2e-inf-a-b1ec3a-openshift-default Scaled up replica set e2e-inf-a-b1ec3a-openshift-default-7675569d9 from 0 to 1 47m Normal AddedLabel gateway/e2e-inf-a-b1ec3a Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-b1ec3a 47m Normal AddedInterface pod/e2e-inf-a-b1ec3a-openshift-default-7675569d9-qjkx2 Add eth0 [10.132.0.201/23] from ovn-kubernetes 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-0847aa-openshift-default New size: 5; reason: All metrics below target 47m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-0847aa-openshift-default Scaled down replica set e2e-shared-a-w0-0847aa-openshift-default-75d5774789 from 6 to 5 47m Normal SuccessfulDelete replicaset/e2e-shared-a-w0-0847aa-openshift-default-75d5774789 Deleted pod: e2e-shared-a-w0-0847aa-openshift-default-75d5774789-db8c6 47m Normal Killing pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-db8c6 Stopping container istio-proxy 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-b1ec3a Scaled up replica set payload-processing-e2e-inf-a-b1ec3a-5c67ddd778 from 0 to 1 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-bbfc2e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-b1ec3a-6fdb57959f Created pod: payload-pre-processing-e2e-inf-a-b1ec3a-6fdb57959f-fxmdx 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-b1ec3a-6fdb57959f Error creating: pods "payload-pre-processing-e2e-inf-a-b1ec3a-6fdb57959f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-b1ec3a: serviceaccount "payload-processing-e2e-inf-a-b1ec3a" not found 47m Warning ProbeError pod/e2e-mt-8b1efeb6-openshift-default-5cf66d64d5-2gjrn Startup probe error: Get "http://10.134.0.154:15021/healthz/ready": dial tcp 10.134.0.154:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-mt-8b1efeb6-openshift-default-5cf66d64d5-2gjrn Startup probe failed: Get "http://10.134.0.154:15021/healthz/ready": dial tcp 10.134.0.154:15021: connect: connection refused 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-b1ec3a Scaled up replica set payload-pre-processing-e2e-inf-a-b1ec3a-6fdb57959f from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-b1ec3a-5c67ddd778 Created pod: payload-processing-e2e-inf-a-b1ec3a-5c67ddd778-kzpcm 47m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-b1ec3a-5c67ddd778 Error creating: pods "payload-processing-e2e-inf-a-b1ec3a-5c67ddd778-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-b1ec3a: serviceaccount "payload-processing-e2e-inf-a-b1ec3a" not found 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-bbfc2e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Normal Pulled pod/payload-pre-processing-e2e-inf-a-b1ec3a-6fdb57959f-fxmdx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Started pod/payload-processing-e2e-inf-a-b1ec3a-5c67ddd778-kzpcm Container started 47m Normal Pulled pod/payload-processing-e2e-inf-a-b1ec3a-5c67ddd778-kzpcm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-processing-e2e-inf-a-b1ec3a-5c67ddd778-kzpcm Container created 47m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-b1ec3a-6fdb57959f-fxmdx Add eth0 [10.132.0.203/23] from ovn-kubernetes 47m Normal Started pod/payload-pre-processing-e2e-inf-a-b1ec3a-6fdb57959f-fxmdx Container started 47m Normal AddedInterface pod/payload-processing-e2e-inf-a-b1ec3a-5c67ddd778-kzpcm Add eth0 [10.132.0.202/23] from ovn-kubernetes 47m Normal Created pod/payload-pre-processing-e2e-inf-a-b1ec3a-6fdb57959f-fxmdx Container created 47m Normal Pulled pod/e2e-inf-b-2d6076-openshift-default-5bdb59ccf9-48ljn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-api-b-22edc1-openshift-default-b9f7f8cb5-qvnsr Container created 47m Normal Pulled pod/e2e-api-b-22edc1-openshift-default-b9f7f8cb5-qvnsr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-inf-b-2d6076-openshift-default-5bdb59ccf9-48ljn Container created 47m Normal Started pod/e2e-api-b-22edc1-openshift-default-b9f7f8cb5-qvnsr Container started 47m Normal Started pod/e2e-inf-b-2d6076-openshift-default-5bdb59ccf9-48ljn Container started 47m Normal Killing pod/payload-processing-e2e-api-a-2c0223-696c7b845d-txtnq Stopping container payload-processing 47m Normal Killing pod/payload-pre-processing-e2e-api-a-2c0223-5954dfbb87-n5vnv Stopping container payload-pre-processing 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-22edc1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning ProbeError pod/e2e-inf-b-2d6076-openshift-default-5bdb59ccf9-48ljn Startup probe error: Get "http://10.134.0.151:15021/healthz/ready": dial tcp 10.134.0.151:15021: connect: connection refused... 47m Warning ProbeError pod/e2e-api-b-22edc1-openshift-default-b9f7f8cb5-qvnsr Startup probe error: Get "http://10.134.0.152:15021/healthz/ready": dial tcp 10.134.0.152:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-inf-b-2d6076-openshift-default-5bdb59ccf9-48ljn Startup probe failed: Get "http://10.134.0.151:15021/healthz/ready": dial tcp 10.134.0.151:15021: connect: connection refused 47m Warning Unhealthy pod/e2e-api-b-22edc1-openshift-default-b9f7f8cb5-qvnsr Startup probe failed: Get "http://10.134.0.152:15021/healthz/ready": dial tcp 10.134.0.152:15021: connect: connection refused 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-22edc1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Normal Killing pod/e2e-api-a-32fdf2-openshift-default-798db88bbc-52z75 Stopping container istio-proxy 47m Warning ProbeError pod/e2e-api-a-32fdf2-openshift-default-798db88bbc-52z75 Readiness probe error: HTTP probe failed with statuscode: 503... 47m Warning Unhealthy pod/e2e-api-a-32fdf2-openshift-default-798db88bbc-52z75 Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-2d6076-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-2d6076-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal Killing pod/e2e-api-b-bbfc2e-openshift-default-6c585b5767-jlcxj Stopping container istio-proxy 46m Normal Killing pod/payload-pre-processing-e2e-api-b-b826b6-86b86bbbff-rkg2w Stopping container payload-pre-processing 46m Normal Killing pod/payload-processing-e2e-api-b-b826b6-65dbdbf768-v9wnw Stopping container payload-processing 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-22edc1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-22edc1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning Unhealthy pod/e2e-api-b-bbfc2e-openshift-default-6c585b5767-jlcxj Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning ProbeError pod/e2e-api-b-bbfc2e-openshift-default-6c585b5767-jlcxj Readiness probe error: HTTP probe failed with statuscode: 503... 46m Normal SuccessfulDelete replicaset/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c Deleted pod: e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-v6bzq 46m Normal SuccessfulDelete replicaset/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c Deleted pod: e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-d7r4r 46m Normal Killing pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-v6bzq Stopping container istio-proxy 46m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-a940d5-openshift-default Scaled down replica set e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c from 6 to 4 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-a940d5-openshift-default New size: 4; reason: All metrics below target 46m Normal Killing pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-d7r4r Stopping container istio-proxy 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-2d6076-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-2d6076-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal SuccessfulDelete replicaset/e2e-worker-w6-2d619f-openshift-default-85b84758df Deleted pod: e2e-worker-w6-2d619f-openshift-default-85b84758df-9h2j6 46m Normal Killing pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-ztnxm Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/e2e-worker-w4-50d738-openshift-default-8d6476cbc Deleted pod: e2e-worker-w4-50d738-openshift-default-8d6476cbc-6skbz 46m Normal Killing pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-6skbz Stopping container istio-proxy 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-2d619f-openshift-default New size: 4; reason: All metrics below target 46m Normal ScalingReplicaSet deployment/e2e-worker-w6-2d619f-openshift-default Scaled down replica set e2e-worker-w6-2d619f-openshift-default-85b84758df from 5 to 4 46m Normal ScalingReplicaSet deployment/e2e-worker-w4-50d738-openshift-default Scaled down replica set e2e-worker-w4-50d738-openshift-default-8d6476cbc from 5 to 4 46m Normal SuccessfulDelete replicaset/e2e-worker-w0-88a4d9-openshift-default-857757469c Deleted pod: e2e-worker-w0-88a4d9-openshift-default-857757469c-ztnxm 46m Normal SuccessfulCreate replicaset/e2e-inf-a-9e540d-openshift-default-74d865bc89 Created pod: e2e-inf-a-9e540d-openshift-default-74d865bc89-p5z9q 46m Normal Killing pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-9h2j6 Stopping container istio-proxy 46m Normal ScalingReplicaSet deployment/e2e-inf-a-9e540d-openshift-default Scaled up replica set e2e-inf-a-9e540d-openshift-default-74d865bc89 from 0 to 1 46m Normal ScalingReplicaSet deployment/e2e-worker-w0-88a4d9-openshift-default Scaled down replica set e2e-worker-w0-88a4d9-openshift-default-857757469c from 5 to 4 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-88a4d9-openshift-default New size: 4; reason: All metrics below target 46m Normal AddedLabel gateway/e2e-inf-a-9e540d Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-9e540d 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-50d738-openshift-default New size: 4; reason: All metrics below target 46m Normal AddedInterface pod/e2e-inf-a-9e540d-openshift-default-74d865bc89-p5z9q Add eth0 [10.132.0.208/23] from ovn-kubernetes 46m Normal Killing pod/e2e-api-a-65b0e2-openshift-default-56d854d69c-tdh5n Stopping container istio-proxy 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-9e540d Scaled up replica set payload-processing-e2e-inf-a-9e540d-79df886dc8 from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-9e540d-79df886dc8 Created pod: payload-processing-e2e-inf-a-9e540d-79df886dc8-q6kwt 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-9e540d-67db6b6b55 Created pod: payload-pre-processing-e2e-inf-a-9e540d-67db6b6b55-f687b 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-9e540d Scaled up replica set payload-pre-processing-e2e-inf-a-9e540d-67db6b6b55 from 0 to 1 46m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-9e540d-79df886dc8 Error creating: pods "payload-processing-e2e-inf-a-9e540d-79df886dc8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-9e540d: serviceaccount "payload-processing-e2e-inf-a-9e540d" not found 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-9e540d-67db6b6b55 Error creating: pods "payload-pre-processing-e2e-inf-a-9e540d-67db6b6b55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-9e540d: serviceaccount "payload-processing-e2e-inf-a-9e540d" not found 46m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-9e540d-67db6b6b55-f687b Add eth0 [10.132.0.210/23] from ovn-kubernetes 46m Normal Started pod/payload-processing-e2e-inf-a-9e540d-79df886dc8-q6kwt Container started 46m Normal Started pod/payload-pre-processing-e2e-inf-a-9e540d-67db6b6b55-f687b Container started 46m Normal Created pod/payload-pre-processing-e2e-inf-a-9e540d-67db6b6b55-f687b Container created 46m Normal Pulled pod/payload-pre-processing-e2e-inf-a-9e540d-67db6b6b55-f687b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Pulled pod/payload-processing-e2e-inf-a-9e540d-79df886dc8-q6kwt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/payload-processing-e2e-inf-a-9e540d-79df886dc8-q6kwt Container created 46m Normal AddedInterface pod/payload-processing-e2e-inf-a-9e540d-79df886dc8-q6kwt Add eth0 [10.133.0.106/23] from ovn-kubernetes 46m Normal Killing pod/e2e-mt-8b1efeb6-openshift-default-5cf66d64d5-z8hk7 Stopping container istio-proxy 46m Normal Killing pod/e2e-mt-8b1efeb6-openshift-default-5cf66d64d5-2gjrn Stopping container istio-proxy 46m Warning Unhealthy pod/e2e-mt-8b1efeb6-openshift-default-5cf66d64d5-2gjrn Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning ProbeError pod/e2e-mt-8b1efeb6-openshift-default-5cf66d64d5-2gjrn Readiness probe error: HTTP probe failed with statuscode: 503... 46m Normal Killing pod/payload-processing-e2e-api-b-5dd3b1-5b45c4ff8-rmshj Stopping container payload-processing 46m Normal Killing pod/payload-pre-processing-e2e-api-b-5dd3b1-7bdc56b4f5-cxmt6 Stopping container payload-pre-processing 46m Normal Pulled pod/e2e-api-b-5dd3b1-openshift-default-684dbd6c7c-k4v6l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-api-b-5dd3b1-openshift-default-684dbd6c7c-k4v6l Container created 46m Normal Started pod/e2e-api-b-5dd3b1-openshift-default-684dbd6c7c-k4v6l Container started 46m Normal SuccessfulCreate replicaset/e2e-inf-b-0a7065-openshift-default-56696cf555 Created pod: e2e-inf-b-0a7065-openshift-default-56696cf555-w2wcw 46m Warning Unhealthy pod/e2e-api-b-5dd3b1-openshift-default-684dbd6c7c-k4v6l Startup probe failed: Get "http://10.132.0.188:15021/healthz/ready": dial tcp 10.132.0.188:15021: connect: connection refused 46m Warning ProbeError pod/e2e-api-b-5dd3b1-openshift-default-684dbd6c7c-k4v6l Startup probe error: Get "http://10.132.0.188:15021/healthz/ready": dial tcp 10.132.0.188:15021: connect: connection refused... 46m Normal ScalingReplicaSet deployment/e2e-inf-b-0a7065-openshift-default Scaled up replica set e2e-inf-b-0a7065-openshift-default-56696cf555 from 0 to 1 46m Normal AddedLabel gateway/e2e-inf-b-0a7065 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-0a7065 46m Normal AddedInterface pod/e2e-inf-b-0a7065-openshift-default-56696cf555-w2wcw Add eth0 [10.134.0.155/23] from ovn-kubernetes 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-0a7065 Scaled up replica set payload-processing-e2e-inf-b-0a7065-694768f4b4 from 0 to 1 46m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-0a7065-694768f4b4 Error creating: pods "payload-processing-e2e-inf-b-0a7065-694768f4b4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-0a7065: serviceaccount "payload-processing-e2e-inf-b-0a7065" not found 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-0a7065-7bc48cbbf9 Error creating: pods "payload-pre-processing-e2e-inf-b-0a7065-7bc48cbbf9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-0a7065: serviceaccount "payload-processing-e2e-inf-b-0a7065" not found 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-0a7065 Scaled up replica set payload-pre-processing-e2e-inf-b-0a7065-7bc48cbbf9 from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-0a7065-694768f4b4 Created pod: payload-processing-e2e-inf-b-0a7065-694768f4b4-5fq8k 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-0a7065-7bc48cbbf9 Created pod: payload-pre-processing-e2e-inf-b-0a7065-7bc48cbbf9-psdjd 46m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-0a7065-7bc48cbbf9-psdjd Add eth0 [10.134.0.156/23] from ovn-kubernetes 46m Normal AddedInterface pod/payload-processing-e2e-inf-b-0a7065-694768f4b4-5fq8k Add eth0 [10.132.0.211/23] from ovn-kubernetes 46m Normal Created pod/payload-processing-e2e-inf-b-0a7065-694768f4b4-5fq8k Container created 46m Normal Started pod/payload-processing-e2e-inf-b-0a7065-694768f4b4-5fq8k Container started 46m Normal Pulled pod/payload-processing-e2e-inf-b-0a7065-694768f4b4-5fq8k Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Started pod/payload-pre-processing-e2e-inf-b-0a7065-7bc48cbbf9-psdjd Container started 46m Normal Pulled pod/payload-pre-processing-e2e-inf-b-0a7065-7bc48cbbf9-psdjd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/payload-pre-processing-e2e-inf-b-0a7065-7bc48cbbf9-psdjd Container created 45m Normal Killing pod/e2e-api-a-2c0223-openshift-default-6958d556b9-dwtg9 Stopping container istio-proxy 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-5dd3b1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-5dd3b1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning Unhealthy pod/e2e-api-a-2c0223-openshift-default-6958d556b9-dwtg9 Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/e2e-api-a-2c0223-openshift-default-6958d556b9-dwtg9 Readiness probe error: HTTP probe failed with statuscode: 503... 45m Normal Killing pod/payload-pre-processing-e2e-api-b-22edc1-5bb4668bc-59h47 Stopping container payload-pre-processing 45m Normal Killing pod/payload-processing-e2e-api-b-22edc1-998976c8b-4whv2 Stopping container payload-processing 45m Normal SuccessfulDelete replicaset/e2e-worker-w1-e19850-openshift-default-9f855bd47 Deleted pod: e2e-worker-w1-e19850-openshift-default-9f855bd47-7pkkv 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-434ce3-openshift-default New size: 4; reason: All metrics below target 45m Normal ScalingReplicaSet deployment/e2e-worker-w5-434ce3-openshift-default Scaled down replica set e2e-worker-w5-434ce3-openshift-default-58c7774759 from 5 to 4 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e19850-openshift-default New size: 4; reason: All metrics below target 45m Normal ScalingReplicaSet deployment/e2e-worker-w1-e19850-openshift-default Scaled down replica set e2e-worker-w1-e19850-openshift-default-9f855bd47 from 5 to 4 45m Normal Killing pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-r4zr7 Stopping container istio-proxy 45m Normal Killing pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-7pkkv Stopping container istio-proxy 45m Normal SuccessfulDelete replicaset/e2e-worker-w5-434ce3-openshift-default-58c7774759 Deleted pod: e2e-worker-w5-434ce3-openshift-default-58c7774759-r4zr7 45m Normal Started pod/e2e-inf-a-b1ec3a-openshift-default-7675569d9-qjkx2 Container started 45m Normal Created pod/e2e-inf-a-b1ec3a-openshift-default-7675569d9-qjkx2 Container created 45m Normal Pulled pod/e2e-inf-a-b1ec3a-openshift-default-7675569d9-qjkx2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Warning ProbeError pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-7pkkv Readiness probe error: Get "http://10.133.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning Unhealthy pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-7pkkv Readiness probe failed: Get "http://10.133.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning ProbeError pod/e2e-inf-a-b1ec3a-openshift-default-7675569d9-qjkx2 Startup probe error: Get "http://10.132.0.201:15021/healthz/ready": dial tcp 10.132.0.201:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-inf-a-b1ec3a-openshift-default-7675569d9-qjkx2 Startup probe failed: Get "http://10.132.0.201:15021/healthz/ready": dial tcp 10.132.0.201:15021: connect: connection refused 45m Normal Killing pod/e2e-api-b-b826b6-openshift-default-785fbbd7-rwr56 Stopping container istio-proxy 45m Normal SuccessfulCreate replicaset/e2e-inf-a-7940ea-openshift-default-75b99d856c Created pod: e2e-inf-a-7940ea-openshift-default-75b99d856c-56bcx 45m Normal AddedLabel gateway/e2e-inf-a-7940ea Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-7940ea 45m Normal ScalingReplicaSet deployment/e2e-inf-a-7940ea-openshift-default Scaled up replica set e2e-inf-a-7940ea-openshift-default-75b99d856c from 0 to 1 45m Normal AddedInterface pod/e2e-inf-a-7940ea-openshift-default-75b99d856c-56bcx Add eth0 [10.134.0.158/23] from ovn-kubernetes 45m Warning Unhealthy pod/e2e-api-b-b826b6-openshift-default-785fbbd7-rwr56 Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/e2e-api-b-b826b6-openshift-default-785fbbd7-rwr56 Readiness probe error: HTTP probe failed with statuscode: 503... 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e19850-openshift-default New size: 3; reason: All metrics below target 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-553753-openshift-default New size: 3; reason: All metrics below target 45m Normal ScalingReplicaSet deployment/e2e-worker-w2-553753-openshift-default Scaled down replica set e2e-worker-w2-553753-openshift-default-558949869c from 5 to 3 45m Normal Killing pod/e2e-worker-w2-553753-openshift-default-558949869c-85x5q Stopping container istio-proxy 45m Normal Killing pod/e2e-worker-w2-553753-openshift-default-558949869c-9fdlh Stopping container istio-proxy 45m Normal SuccessfulDelete replicaset/e2e-worker-w2-553753-openshift-default-558949869c Deleted pod: e2e-worker-w2-553753-openshift-default-558949869c-85x5q 45m Normal SuccessfulDelete replicaset/e2e-worker-w2-553753-openshift-default-558949869c Deleted pod: e2e-worker-w2-553753-openshift-default-558949869c-9fdlh 45m Normal SuccessfulDelete replicaset/e2e-worker-w1-e19850-openshift-default-9f855bd47 Deleted pod: e2e-worker-w1-e19850-openshift-default-9f855bd47-thldf 45m Normal Killing pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-thldf Stopping container istio-proxy 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-7940ea-56b89d4b64 Created pod: payload-processing-e2e-inf-a-7940ea-56b89d4b64-pvwxl 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-7940ea-6c48467979 Error creating: pods "payload-pre-processing-e2e-inf-a-7940ea-6c48467979-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-7940ea: serviceaccount "payload-processing-e2e-inf-a-7940ea" not found 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-7940ea-6c48467979 Created pod: payload-pre-processing-e2e-inf-a-7940ea-6c48467979-8rjd5 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-7940ea Scaled up replica set payload-pre-processing-e2e-inf-a-7940ea-6c48467979 from 0 to 1 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-7940ea Scaled up replica set payload-processing-e2e-inf-a-7940ea-56b89d4b64 from 0 to 1 45m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-7940ea-56b89d4b64 Error creating: pods "payload-processing-e2e-inf-a-7940ea-56b89d4b64-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-7940ea: serviceaccount "payload-processing-e2e-inf-a-7940ea" not found 45m Normal Started pod/payload-processing-e2e-inf-a-7940ea-56b89d4b64-pvwxl Container started 45m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-7940ea-6c48467979-8rjd5 Add eth0 [10.132.0.213/23] from ovn-kubernetes 45m Normal Created pod/payload-processing-e2e-inf-a-7940ea-56b89d4b64-pvwxl Container created 45m Normal Pulled pod/payload-processing-e2e-inf-a-7940ea-56b89d4b64-pvwxl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/payload-processing-e2e-inf-a-7940ea-56b89d4b64-pvwxl Add eth0 [10.134.0.160/23] from ovn-kubernetes 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-b1ec3a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-b1ec3a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Normal Started pod/payload-pre-processing-e2e-inf-a-7940ea-6c48467979-8rjd5 Container started 45m Normal Created pod/payload-pre-processing-e2e-inf-a-7940ea-6c48467979-8rjd5 Container created 45m Normal Pulled pod/payload-pre-processing-e2e-inf-a-7940ea-6c48467979-8rjd5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Warning Unhealthy pod/e2e-worker-w2-553753-openshift-default-558949869c-85x5q Readiness probe failed: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused 45m Warning ProbeError pod/e2e-worker-w2-553753-openshift-default-558949869c-85x5q Readiness probe error: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused... 45m Normal AddedLabel gateway/e2e-inf-b-71a7df Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-71a7df 45m Normal ScalingReplicaSet deployment/e2e-inf-b-71a7df-openshift-default Scaled up replica set e2e-inf-b-71a7df-openshift-default-74d8445657 from 0 to 1 45m Normal SuccessfulCreate replicaset/e2e-inf-b-71a7df-openshift-default-74d8445657 Created pod: e2e-inf-b-71a7df-openshift-default-74d8445657-zqkwp 45m Normal AddedInterface pod/e2e-inf-b-71a7df-openshift-default-74d8445657-zqkwp Add eth0 [10.134.0.161/23] from ovn-kubernetes 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-71a7df Scaled up replica set payload-processing-e2e-inf-b-71a7df-5f4b644b from 0 to 1 45m Normal Created pod/payload-pre-processing-e2e-inf-b-71a7df-fdd88b4b7-sdbdj Container created 45m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-71a7df-5f4b644b Error creating: pods "payload-processing-e2e-inf-b-71a7df-5f4b644b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-71a7df: serviceaccount "payload-processing-e2e-inf-b-71a7df" not found 45m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-71a7df-fdd88b4b7-sdbdj Add eth0 [10.133.0.108/23] from ovn-kubernetes 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-71a7df Scaled up replica set payload-pre-processing-e2e-inf-b-71a7df-fdd88b4b7 from 0 to 1 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-71a7df-fdd88b4b7 Error creating: pods "payload-pre-processing-e2e-inf-b-71a7df-fdd88b4b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-71a7df: serviceaccount "payload-processing-e2e-inf-b-71a7df" not found 45m Normal Created pod/payload-processing-e2e-inf-b-71a7df-5f4b644b-zwb9k Container created 45m Normal Started pod/payload-pre-processing-e2e-inf-b-71a7df-fdd88b4b7-sdbdj Container started 45m Normal Pulled pod/payload-processing-e2e-inf-b-71a7df-5f4b644b-zwb9k Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-71a7df-fdd88b4b7 Created pod: payload-pre-processing-e2e-inf-b-71a7df-fdd88b4b7-sdbdj 45m Normal Pulled pod/payload-pre-processing-e2e-inf-b-71a7df-fdd88b4b7-sdbdj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Started pod/payload-processing-e2e-inf-b-71a7df-5f4b644b-zwb9k Container started 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-71a7df-5f4b644b Created pod: payload-processing-e2e-inf-b-71a7df-5f4b644b-zwb9k 45m Normal AddedInterface pod/payload-processing-e2e-inf-b-71a7df-5f4b644b-zwb9k Add eth0 [10.133.0.107/23] from ovn-kubernetes 45m Normal Killing pod/e2e-api-b-5dd3b1-openshift-default-684dbd6c7c-k4v6l Stopping container istio-proxy 45m Normal SuccessfulCreate replicaset/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f Created pod: e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-br8fv 45m Normal ScalingReplicaSet deployment/e2e-inf-a-50b9b5-openshift-default Scaled up replica set e2e-inf-a-50b9b5-openshift-default-6f7ffb6f from 0 to 1 45m Normal AddedLabel gateway/e2e-inf-a-50b9b5 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-50b9b5 45m Normal AddedInterface pod/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-br8fv Add eth0 [10.134.0.162/23] from ovn-kubernetes 45m Warning Unhealthy pod/e2e-api-b-5dd3b1-openshift-default-684dbd6c7c-k4v6l Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/e2e-api-b-5dd3b1-openshift-default-684dbd6c7c-k4v6l Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-b1ec3a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-b1ec3a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-50b9b5-76b76f77c9-g6sth Add eth0 [10.132.0.214/23] from ovn-kubernetes 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-50b9b5 Scaled up replica set payload-pre-processing-e2e-inf-a-50b9b5-76b76f77c9 from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-50b9b5-76b76f77c9 Created pod: payload-pre-processing-e2e-inf-a-50b9b5-76b76f77c9-g6sth 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-50b9b5 Scaled up replica set payload-processing-e2e-inf-a-50b9b5-7f48f98986 from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-50b9b5-7f48f98986 Created pod: payload-processing-e2e-inf-a-50b9b5-7f48f98986-ddh6w 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-50b9b5-76b76f77c9 Error creating: pods "payload-pre-processing-e2e-inf-a-50b9b5-76b76f77c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-50b9b5: serviceaccount "payload-processing-e2e-inf-a-50b9b5" not found 45m Normal AddedInterface pod/payload-processing-e2e-inf-a-50b9b5-7f48f98986-ddh6w Add eth0 [10.133.0.109/23] from ovn-kubernetes 45m Normal Created pod/payload-processing-e2e-inf-a-50b9b5-7f48f98986-ddh6w Container created 45m Normal Created pod/payload-pre-processing-e2e-inf-a-50b9b5-76b76f77c9-g6sth Container created 45m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-50b9b5-7f48f98986 Error creating: pods "payload-processing-e2e-inf-a-50b9b5-7f48f98986-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-50b9b5: serviceaccount "payload-processing-e2e-inf-a-50b9b5" not found 45m Normal Pulled pod/payload-pre-processing-e2e-inf-a-50b9b5-76b76f77c9-g6sth Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Started pod/payload-processing-e2e-inf-a-50b9b5-7f48f98986-ddh6w Container started 45m Normal Pulled pod/payload-processing-e2e-inf-a-50b9b5-7f48f98986-ddh6w Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Started pod/payload-pre-processing-e2e-inf-a-50b9b5-76b76f77c9-g6sth Container started 44m Normal ScalingReplicaSet deployment/e2e-worker-w3-5187e1-openshift-default Scaled down replica set e2e-worker-w3-5187e1-openshift-default-946849bdf from 6 to 4 44m Normal SuccessfulDelete replicaset/e2e-worker-w3-5187e1-openshift-default-946849bdf Deleted pod: e2e-worker-w3-5187e1-openshift-default-946849bdf-q88pc 44m Normal SuccessfulDelete replicaset/e2e-worker-w3-5187e1-openshift-default-946849bdf Deleted pod: e2e-worker-w3-5187e1-openshift-default-946849bdf-qslxh 44m Normal Killing pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-qslxh Stopping container istio-proxy 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-5187e1-openshift-default New size: 4; reason: All metrics below target 44m Normal Killing pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-q88pc Stopping container istio-proxy 44m Warning ProbeError pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-qslxh Readiness probe error: HTTP probe failed with statuscode: 503... 44m Warning Unhealthy pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-qslxh Readiness probe failed: HTTP probe failed with statuscode: 503 44m Normal Created pod/e2e-inf-a-9e540d-openshift-default-74d865bc89-p5z9q Container created 44m Normal Started pod/e2e-inf-b-0a7065-openshift-default-56696cf555-w2wcw Container started 44m Normal Created pod/e2e-inf-b-0a7065-openshift-default-56696cf555-w2wcw Container created 44m Normal Started pod/e2e-inf-a-9e540d-openshift-default-74d865bc89-p5z9q Container started 44m Normal Pulled pod/e2e-inf-a-9e540d-openshift-default-74d865bc89-p5z9q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Pulled pod/e2e-inf-b-0a7065-openshift-default-56696cf555-w2wcw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-9e540d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-9e540d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal Killing pod/e2e-api-b-22edc1-openshift-default-b9f7f8cb5-qvnsr Stopping container istio-proxy 44m Normal AddedLabel gateway/e2e-inf-a-bb1fa5 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-bb1fa5 44m Normal ScalingReplicaSet deployment/e2e-inf-a-bb1fa5-openshift-default Scaled up replica set e2e-inf-a-bb1fa5-openshift-default-95cc6c6f from 0 to 1 44m Normal SuccessfulCreate replicaset/e2e-inf-a-bb1fa5-openshift-default-95cc6c6f Created pod: e2e-inf-a-bb1fa5-openshift-default-95cc6c6f-8b6vq 44m Normal AddedInterface pod/e2e-inf-a-bb1fa5-openshift-default-95cc6c6f-8b6vq Add eth0 [10.133.0.111/23] from ovn-kubernetes 44m Normal ScalingReplicaSet deployment/e2e-inf-a-341c51-openshift-default Scaled up replica set e2e-inf-a-341c51-openshift-default-56f6687b6f from 0 to 1 44m Normal SuccessfulCreate replicaset/e2e-inf-a-341c51-openshift-default-56f6687b6f Created pod: e2e-inf-a-341c51-openshift-default-56f6687b6f-dqgpg 44m Normal AddedLabel gateway/e2e-inf-a-341c51 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-341c51 44m Warning ProbeError pod/e2e-inf-a-9e540d-openshift-default-74d865bc89-p5z9q Startup probe error: Get "http://10.132.0.208:15021/healthz/ready": dial tcp 10.132.0.208:15021: connect: connection refused... 44m Normal AddedInterface pod/e2e-inf-a-341c51-openshift-default-56f6687b6f-dqgpg Add eth0 [10.134.0.164/23] from ovn-kubernetes 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-71a7df-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-71a7df-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning Unhealthy pod/e2e-inf-a-9e540d-openshift-default-74d865bc89-p5z9q Startup probe failed: Get "http://10.132.0.208:15021/healthz/ready": dial tcp 10.132.0.208:15021: connect: connection refused 44m Warning ProbeError pod/e2e-inf-b-0a7065-openshift-default-56696cf555-w2wcw Startup probe error: Get "http://10.134.0.155:15021/healthz/ready": dial tcp 10.134.0.155:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-inf-b-0a7065-openshift-default-56696cf555-w2wcw Startup probe failed: Get "http://10.134.0.155:15021/healthz/ready": dial tcp 10.134.0.155:15021: connect: connection refused 44m Warning Unhealthy pod/e2e-api-b-22edc1-openshift-default-b9f7f8cb5-qvnsr Readiness probe failed: Get "http://10.134.0.152:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-bb1fa5 Scaled up replica set payload-pre-processing-e2e-inf-a-bb1fa5-7948594b67 from 0 to 1 44m Warning ProbeError pod/e2e-api-b-22edc1-openshift-default-b9f7f8cb5-qvnsr Readiness probe error: Get "http://10.134.0.152:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-bb1fa5-7948594b67 Created pod: payload-pre-processing-e2e-inf-a-bb1fa5-7948594b67-45z4r 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-bb1fa5-7948594b67 Error creating: pods "payload-pre-processing-e2e-inf-a-bb1fa5-7948594b67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-bb1fa5: serviceaccount "payload-processing-e2e-inf-a-bb1fa5" not found 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-bb1fa5-6694d587bb Created pod: payload-processing-e2e-inf-a-bb1fa5-6694d587bb-tdvvr 44m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-bb1fa5-6694d587bb Error creating: pods "payload-processing-e2e-inf-a-bb1fa5-6694d587bb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-bb1fa5: serviceaccount "payload-processing-e2e-inf-a-bb1fa5" not found 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-bb1fa5 Scaled up replica set payload-processing-e2e-inf-a-bb1fa5-6694d587bb from 0 to 1 44m Normal Created pod/payload-processing-e2e-inf-a-bb1fa5-6694d587bb-tdvvr Container created 44m Normal AddedInterface pod/payload-processing-e2e-inf-a-bb1fa5-6694d587bb-tdvvr Add eth0 [10.132.0.217/23] from ovn-kubernetes 44m Normal Started pod/payload-processing-e2e-inf-a-bb1fa5-6694d587bb-tdvvr Container started 44m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-bb1fa5-7948594b67-45z4r Add eth0 [10.132.0.216/23] from ovn-kubernetes 44m Normal Started pod/payload-pre-processing-e2e-inf-a-bb1fa5-7948594b67-45z4r Container started 44m Normal Pulled pod/payload-pre-processing-e2e-inf-a-bb1fa5-7948594b67-45z4r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Pulled pod/payload-processing-e2e-inf-a-bb1fa5-6694d587bb-tdvvr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-pre-processing-e2e-inf-a-bb1fa5-7948594b67-45z4r Container created 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-341c51 Scaled up replica set payload-pre-processing-e2e-inf-a-341c51-76d98b8f6c from 0 to 1 44m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-341c51-857b566498 Error creating: pods "payload-processing-e2e-inf-a-341c51-857b566498-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-341c51: serviceaccount "payload-processing-e2e-inf-a-341c51" not found 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-341c51 Scaled up replica set payload-processing-e2e-inf-a-341c51-857b566498 from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-341c51-857b566498 Created pod: payload-processing-e2e-inf-a-341c51-857b566498-5d2tg 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-341c51-76d98b8f6c Created pod: payload-pre-processing-e2e-inf-a-341c51-76d98b8f6c-cqm9p 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-341c51-76d98b8f6c Error creating: pods "payload-pre-processing-e2e-inf-a-341c51-76d98b8f6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-341c51: serviceaccount "payload-processing-e2e-inf-a-341c51" not found 44m Normal Created pod/payload-processing-e2e-inf-a-341c51-857b566498-5d2tg Container created 44m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-341c51-76d98b8f6c-cqm9p Add eth0 [10.133.0.113/23] from ovn-kubernetes 44m Normal AddedInterface pod/payload-processing-e2e-inf-a-341c51-857b566498-5d2tg Add eth0 [10.133.0.114/23] from ovn-kubernetes 44m Normal Pulled pod/payload-pre-processing-e2e-inf-a-341c51-76d98b8f6c-cqm9p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Pulled pod/payload-processing-e2e-inf-a-341c51-857b566498-5d2tg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Started pod/payload-processing-e2e-inf-a-341c51-857b566498-5d2tg Container started 44m Normal Created pod/payload-pre-processing-e2e-inf-a-341c51-76d98b8f6c-cqm9p Container created 44m Normal Started pod/payload-pre-processing-e2e-inf-a-341c51-76d98b8f6c-cqm9p Container started 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-0a7065-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-0a7065-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-0a7065-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-0a7065-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-9e540d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-9e540d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal SuccessfulCreate replicaset/e2e-inf-b-ccdb60-openshift-default-678d965f7c Created pod: e2e-inf-b-ccdb60-openshift-default-678d965f7c-cttbz 44m Normal ScalingReplicaSet deployment/e2e-inf-b-ccdb60-openshift-default Scaled up replica set e2e-inf-b-ccdb60-openshift-default-678d965f7c from 0 to 1 44m Normal AddedLabel gateway/e2e-inf-b-ccdb60 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-ccdb60 44m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-0847aa-openshift-default Scaled down replica set e2e-shared-a-w0-0847aa-openshift-default-75d5774789 from 5 to 3 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-0847aa-openshift-default New size: 3; reason: All metrics below target 44m Normal SuccessfulDelete replicaset/e2e-shared-a-w0-0847aa-openshift-default-75d5774789 Deleted pod: e2e-shared-a-w0-0847aa-openshift-default-75d5774789-7kgzf 44m Normal Killing pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-tdqsq Stopping container istio-proxy 44m Normal SuccessfulDelete replicaset/e2e-shared-a-w0-0847aa-openshift-default-75d5774789 Deleted pod: e2e-shared-a-w0-0847aa-openshift-default-75d5774789-tdqsq 44m Normal Killing pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-7kgzf Stopping container istio-proxy 44m Normal AddedInterface pod/e2e-inf-b-ccdb60-openshift-default-678d965f7c-cttbz Add eth0 [10.133.0.115/23] from ovn-kubernetes 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-88a4d9-openshift-default New size: 3; reason: All metrics below target 43m Normal ScalingReplicaSet deployment/e2e-worker-w0-88a4d9-openshift-default Scaled down replica set e2e-worker-w0-88a4d9-openshift-default-857757469c from 4 to 3 43m Normal SuccessfulDelete replicaset/e2e-worker-w0-88a4d9-openshift-default-857757469c Deleted pod: e2e-worker-w0-88a4d9-openshift-default-857757469c-sjh6w 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-ccdb60-779c4b9f95 Error creating: pods "payload-pre-processing-e2e-inf-b-ccdb60-779c4b9f95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-ccdb60: serviceaccount "payload-processing-e2e-inf-b-ccdb60" not found 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-ccdb60-779c4b9f95 Created pod: payload-pre-processing-e2e-inf-b-ccdb60-779c4b9f95-rrp8q 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-ccdb60 Scaled up replica set payload-pre-processing-e2e-inf-b-ccdb60-779c4b9f95 from 0 to 1 43m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-ccdb60-7fc6f896b Error creating: pods "payload-processing-e2e-inf-b-ccdb60-7fc6f896b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-ccdb60: serviceaccount "payload-processing-e2e-inf-b-ccdb60" not found 43m Normal Killing pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-sjh6w Stopping container istio-proxy 43m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-ccdb60-779c4b9f95-rrp8q Add eth0 [10.133.0.116/23] from ovn-kubernetes 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-ccdb60-7fc6f896b Created pod: payload-processing-e2e-inf-b-ccdb60-7fc6f896b-qrxhs 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-ccdb60 Scaled up replica set payload-processing-e2e-inf-b-ccdb60-7fc6f896b from 0 to 1 43m Normal Created pod/payload-processing-e2e-inf-b-ccdb60-7fc6f896b-qrxhs Container created 43m Normal Started pod/payload-pre-processing-e2e-inf-b-ccdb60-779c4b9f95-rrp8q Container started 43m Warning Unhealthy pod/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-br8fv Readiness probe failed: Get "http://10.134.0.162:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Normal Pulled pod/payload-pre-processing-e2e-inf-b-ccdb60-779c4b9f95-rrp8q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Warning ProbeError pod/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-br8fv Readiness probe error: Get "http://10.134.0.162:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Normal Created pod/payload-pre-processing-e2e-inf-b-ccdb60-779c4b9f95-rrp8q Container created 43m Normal Started pod/payload-processing-e2e-inf-b-ccdb60-7fc6f896b-qrxhs Container started 43m Normal Pulled pod/payload-processing-e2e-inf-b-ccdb60-7fc6f896b-qrxhs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-processing-e2e-inf-b-ccdb60-7fc6f896b-qrxhs Add eth0 [10.132.0.219/23] from ovn-kubernetes 43m Warning ProbeError pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-sjh6w Readiness probe error: HTTP probe failed with statuscode: 503... 43m Warning Unhealthy pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-sjh6w Readiness probe failed: HTTP probe failed with statuscode: 503 43m Normal ScalingReplicaSet deployment/e2e-inf-b-71a7df-openshift-default Scaled up replica set e2e-inf-b-71a7df-openshift-default-74d8445657 from 1 to 4 43m Normal SuccessfulCreate replicaset/e2e-inf-b-71a7df-openshift-default-74d8445657 Created pod: e2e-inf-b-71a7df-openshift-default-74d8445657-j68rv 43m Normal SuccessfulCreate replicaset/e2e-inf-b-71a7df-openshift-default-74d8445657 Created pod: e2e-inf-b-71a7df-openshift-default-74d8445657-fv778 43m Normal SuccessfulCreate replicaset/e2e-inf-b-71a7df-openshift-default-74d8445657 Created pod: e2e-inf-b-71a7df-openshift-default-74d8445657-bgwbv 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-71a7df-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 43m Normal Created pod/e2e-inf-b-71a7df-openshift-default-74d8445657-j68rv Container created 43m Normal Created pod/e2e-inf-b-71a7df-openshift-default-74d8445657-fv778 Container created 43m Normal AddedInterface pod/e2e-inf-b-71a7df-openshift-default-74d8445657-j68rv Add eth0 [10.133.0.117/23] from ovn-kubernetes 43m Normal AddedInterface pod/e2e-inf-b-71a7df-openshift-default-74d8445657-fv778 Add eth0 [10.132.0.220/23] from ovn-kubernetes 43m Normal Pulled pod/e2e-inf-b-71a7df-openshift-default-74d8445657-fv778 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Pulled pod/e2e-inf-b-71a7df-openshift-default-74d8445657-j68rv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-inf-b-71a7df-openshift-default-74d8445657-j68rv Container started 43m Normal Pulled pod/e2e-inf-b-71a7df-openshift-default-74d8445657-bgwbv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-inf-b-71a7df-openshift-default-74d8445657-fv778 Container started 43m Normal AddedInterface pod/e2e-inf-b-71a7df-openshift-default-74d8445657-bgwbv Add eth0 [10.134.0.167/23] from ovn-kubernetes 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-ccdb60-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal Started pod/e2e-inf-b-71a7df-openshift-default-74d8445657-bgwbv Container started 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-ccdb60-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal Created pod/e2e-inf-b-71a7df-openshift-default-74d8445657-bgwbv Container created 43m Warning ProbeError pod/e2e-inf-b-71a7df-openshift-default-74d8445657-zqkwp Readiness probe error: Get "http://10.134.0.161:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Warning Unhealthy pod/e2e-inf-b-71a7df-openshift-default-74d8445657-zqkwp Readiness probe failed: Get "http://10.134.0.161:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Normal Created pod/e2e-inf-a-7940ea-openshift-default-75b99d856c-56bcx Container created 43m Normal Started pod/e2e-inf-b-71a7df-openshift-default-74d8445657-zqkwp Container started 43m Normal Created pod/e2e-inf-b-71a7df-openshift-default-74d8445657-zqkwp Container created 43m Normal Pulled pod/e2e-inf-b-71a7df-openshift-default-74d8445657-zqkwp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Created pod/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-br8fv Container created 43m Normal Pulled pod/e2e-inf-a-7940ea-openshift-default-75b99d856c-56bcx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-inf-a-7940ea-openshift-default-75b99d856c-56bcx Container started 43m Normal Pulled pod/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-br8fv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-br8fv Container started 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-7940ea-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-7940ea-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning ProbeError pod/e2e-inf-b-71a7df-openshift-default-74d8445657-j68rv Startup probe error: Get "http://10.133.0.117:15021/healthz/ready": dial tcp 10.133.0.117:15021: connect: connection refused... 43m Warning ProbeError pod/e2e-inf-b-71a7df-openshift-default-74d8445657-fv778 Startup probe error: Get "http://10.132.0.220:15021/healthz/ready": dial tcp 10.132.0.220:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-inf-b-71a7df-openshift-default-74d8445657-fv778 Startup probe failed: Get "http://10.132.0.220:15021/healthz/ready": dial tcp 10.132.0.220:15021: connect: connection refused 43m Warning Unhealthy pod/e2e-inf-b-71a7df-openshift-default-74d8445657-j68rv Startup probe failed: Get "http://10.133.0.117:15021/healthz/ready": dial tcp 10.133.0.117:15021: connect: connection refused 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-50b9b5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-50b9b5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning Unhealthy pod/e2e-inf-b-71a7df-openshift-default-74d8445657-bgwbv Startup probe failed: Get "http://10.134.0.167:15021/healthz/ready": dial tcp 10.134.0.167:15021: connect: connection refused 43m Warning ProbeError pod/e2e-inf-b-71a7df-openshift-default-74d8445657-bgwbv Startup probe error: Get "http://10.134.0.167:15021/healthz/ready": dial tcp 10.134.0.167:15021: connect: connection refused... 43m Warning Unhealthy pod/payload-processing-e2e-shared-a-w0-0847aa-fc76d97d-m54ng Readiness probe failed: timeout: failed to connect service "10.134.0.119:50052" within 1s: context deadline exceeded 43m Normal SuccessfulCreate replicaset/e2e-inf-a-b1ec3a-openshift-default-7675569d9 Created pod: e2e-inf-a-b1ec3a-openshift-default-7675569d9-kjjgn 43m Normal ScalingReplicaSet deployment/e2e-inf-a-b1ec3a-openshift-default Scaled up replica set e2e-inf-a-b1ec3a-openshift-default-7675569d9 from 1 to 2 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-b1ec3a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 43m Normal AddedLabel gateway/e2e-inf-b-fe7ca0 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-fe7ca0 43m Normal ScalingReplicaSet deployment/e2e-inf-b-fe7ca0-openshift-default Scaled up replica set e2e-inf-b-fe7ca0-openshift-default-7598bdb7d5 from 0 to 1 43m Warning Unhealthy pod/e2e-inf-b-71a7df-openshift-default-74d8445657-zqkwp Startup probe failed: Get "http://10.134.0.161:15021/healthz/ready": dial tcp 10.134.0.161:15021: connect: connection refused 43m Warning ProbeError pod/e2e-inf-b-71a7df-openshift-default-74d8445657-zqkwp Startup probe error: Get "http://10.134.0.161:15021/healthz/ready": dial tcp 10.134.0.161:15021: connect: connection refused... 43m Normal SuccessfulCreate replicaset/e2e-inf-b-fe7ca0-openshift-default-7598bdb7d5 Created pod: e2e-inf-b-fe7ca0-openshift-default-7598bdb7d5-jqmrs 43m Warning ProbeError pod/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-br8fv Startup probe error: Get "http://10.134.0.162:15021/healthz/ready": dial tcp 10.134.0.162:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-br8fv Startup probe failed: Get "http://10.134.0.162:15021/healthz/ready": dial tcp 10.134.0.162:15021: connect: connection refused 43m Warning ProbeError pod/e2e-inf-a-7940ea-openshift-default-75b99d856c-56bcx Startup probe error: Get "http://10.134.0.158:15021/healthz/ready": dial tcp 10.134.0.158:15021: connect: connection refused... 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-71a7df-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-71a7df-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning Unhealthy pod/e2e-inf-a-7940ea-openshift-default-75b99d856c-56bcx Startup probe failed: Get "http://10.134.0.158:15021/healthz/ready": dial tcp 10.134.0.158:15021: connect: connection refused 43m Normal AddedInterface pod/e2e-inf-a-b1ec3a-openshift-default-7675569d9-kjjgn Add eth0 [10.134.0.170/23] from ovn-kubernetes 43m Normal Pulled pod/e2e-inf-a-b1ec3a-openshift-default-7675569d9-kjjgn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/e2e-inf-b-fe7ca0-openshift-default-7598bdb7d5-jqmrs Add eth0 [10.134.0.171/23] from ovn-kubernetes 43m Normal Created pod/e2e-inf-a-b1ec3a-openshift-default-7675569d9-kjjgn Container created 43m Normal Started pod/e2e-inf-a-b1ec3a-openshift-default-7675569d9-kjjgn Container started 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-fe7ca0-54c86d5ff5 Created pod: payload-pre-processing-e2e-inf-b-fe7ca0-54c86d5ff5-xqxp7 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-fe7ca0 Scaled up replica set payload-processing-e2e-inf-b-fe7ca0-55488976b6 from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-fe7ca0-55488976b6 Created pod: payload-processing-e2e-inf-b-fe7ca0-55488976b6-5hnxs 43m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-fe7ca0-55488976b6 Error creating: pods "payload-processing-e2e-inf-b-fe7ca0-55488976b6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-fe7ca0: serviceaccount "payload-processing-e2e-inf-b-fe7ca0" not found 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-fe7ca0-54c86d5ff5 Error creating: pods "payload-pre-processing-e2e-inf-b-fe7ca0-54c86d5ff5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-fe7ca0: serviceaccount "payload-processing-e2e-inf-b-fe7ca0" not found 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-fe7ca0 Scaled up replica set payload-pre-processing-e2e-inf-b-fe7ca0-54c86d5ff5 from 0 to 1 43m Normal Created pod/payload-pre-processing-e2e-inf-b-fe7ca0-54c86d5ff5-xqxp7 Container created 43m Normal AddedInterface pod/payload-processing-e2e-inf-b-fe7ca0-55488976b6-5hnxs Add eth0 [10.132.0.221/23] from ovn-kubernetes 43m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-fe7ca0-54c86d5ff5-xqxp7 Add eth0 [10.133.0.119/23] from ovn-kubernetes 43m Normal Pulled pod/payload-pre-processing-e2e-inf-b-fe7ca0-54c86d5ff5-xqxp7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-pre-processing-e2e-inf-b-fe7ca0-54c86d5ff5-xqxp7 Container started 43m Normal Pulled pod/payload-processing-e2e-inf-b-fe7ca0-55488976b6-5hnxs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Warning Unhealthy pod/e2e-inf-a-b1ec3a-openshift-default-7675569d9-kjjgn Startup probe failed: Get "http://10.134.0.170:15021/healthz/ready": dial tcp 10.134.0.170:15021: connect: connection refused 43m Normal Created pod/payload-processing-e2e-inf-b-fe7ca0-55488976b6-5hnxs Container created 43m Warning ProbeError pod/e2e-inf-a-b1ec3a-openshift-default-7675569d9-kjjgn Startup probe error: Get "http://10.134.0.170:15021/healthz/ready": dial tcp 10.134.0.170:15021: connect: connection refused... 43m Normal Started pod/payload-processing-e2e-inf-b-fe7ca0-55488976b6-5hnxs Container started 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-7940ea-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-7940ea-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-50b9b5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-50b9b5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal SuccessfulCreate replicaset/e2e-inf-b-963936-openshift-default-68b65f857c Created pod: e2e-inf-b-963936-openshift-default-68b65f857c-phdvv 43m Normal ScalingReplicaSet deployment/e2e-inf-b-963936-openshift-default Scaled up replica set e2e-inf-b-963936-openshift-default-68b65f857c from 0 to 1 43m Normal AddedLabel gateway/e2e-inf-b-963936 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-963936 42m Normal AddedInterface pod/e2e-inf-b-963936-openshift-default-68b65f857c-phdvv Add eth0 [10.133.0.120/23] from ovn-kubernetes 42m Normal Pulled pod/e2e-inf-a-bb1fa5-openshift-default-95cc6c6f-8b6vq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Started pod/e2e-inf-a-bb1fa5-openshift-default-95cc6c6f-8b6vq Container started 42m Normal Created pod/e2e-inf-a-bb1fa5-openshift-default-95cc6c6f-8b6vq Container created 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-963936-6c6699ff5 Error creating: pods "payload-pre-processing-e2e-inf-b-963936-6c6699ff5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-963936: serviceaccount "payload-processing-e2e-inf-b-963936" not found 42m Normal Pulled pod/e2e-inf-a-341c51-openshift-default-56f6687b6f-dqgpg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Started pod/e2e-inf-a-341c51-openshift-default-56f6687b6f-dqgpg Container started 42m Normal Created pod/e2e-inf-a-341c51-openshift-default-56f6687b6f-dqgpg Container created 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-963936 Scaled up replica set payload-processing-e2e-inf-b-963936-589db57f54 from 0 to 1 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-963936 Scaled up replica set payload-pre-processing-e2e-inf-b-963936-6c6699ff5 from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-963936-589db57f54 Created pod: payload-processing-e2e-inf-b-963936-589db57f54-5m29v 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-963936-6c6699ff5 Created pod: payload-pre-processing-e2e-inf-b-963936-6c6699ff5-mktm9 42m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-963936-589db57f54 Error creating: pods "payload-processing-e2e-inf-b-963936-589db57f54-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-963936: serviceaccount "payload-processing-e2e-inf-b-963936" not found 42m Normal Pulled pod/payload-processing-e2e-inf-b-963936-589db57f54-5m29v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Created pod/payload-pre-processing-e2e-inf-b-963936-6c6699ff5-mktm9 Container created 42m Normal Created pod/payload-processing-e2e-inf-b-963936-589db57f54-5m29v Container created 42m Normal Started pod/payload-processing-e2e-inf-b-963936-589db57f54-5m29v Container started 42m Normal AddedInterface pod/payload-processing-e2e-inf-b-963936-589db57f54-5m29v Add eth0 [10.132.0.223/23] from ovn-kubernetes 42m Normal Started pod/payload-pre-processing-e2e-inf-b-963936-6c6699ff5-mktm9 Container started 42m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-963936-6c6699ff5-mktm9 Add eth0 [10.132.0.222/23] from ovn-kubernetes 42m Normal Pulled pod/payload-pre-processing-e2e-inf-b-963936-6c6699ff5-mktm9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal SuccessfulCreate replicaset/e2e-inf-b-124490-openshift-default-54b4ffbf45 Created pod: e2e-inf-b-124490-openshift-default-54b4ffbf45-88rs8 42m Normal ScalingReplicaSet deployment/e2e-inf-b-124490-openshift-default Scaled up replica set e2e-inf-b-124490-openshift-default-54b4ffbf45 from 0 to 1 42m Normal AddedLabel gateway/e2e-inf-b-124490 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-124490 42m Normal AddedInterface pod/e2e-inf-b-124490-openshift-default-54b4ffbf45-88rs8 Add eth0 [10.132.0.224/23] from ovn-kubernetes 42m Warning Unhealthy pod/e2e-inf-a-bb1fa5-openshift-default-95cc6c6f-8b6vq Startup probe failed: Get "http://10.133.0.111:15021/healthz/ready": dial tcp 10.133.0.111:15021: connect: connection refused 42m Warning ProbeError pod/e2e-inf-a-bb1fa5-openshift-default-95cc6c6f-8b6vq Startup probe error: Get "http://10.133.0.111:15021/healthz/ready": dial tcp 10.133.0.111:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-inf-a-341c51-openshift-default-56f6687b6f-dqgpg Startup probe failed: Get "http://10.134.0.164:15021/healthz/ready": dial tcp 10.134.0.164:15021: connect: connection refused 42m Warning ProbeError pod/e2e-inf-a-341c51-openshift-default-56f6687b6f-dqgpg Startup probe error: Get "http://10.134.0.164:15021/healthz/ready": dial tcp 10.134.0.164:15021: connect: connection refused... 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-124490-7cbff46899 Error creating: pods "payload-pre-processing-e2e-inf-b-124490-7cbff46899-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-124490: serviceaccount "payload-processing-e2e-inf-b-124490" not found 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-124490-7cbff46899 Created pod: payload-pre-processing-e2e-inf-b-124490-7cbff46899-7c5p6 42m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-124490-7cbff46899-7c5p6 Add eth0 [10.133.0.121/23] from ovn-kubernetes 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-124490-77db798586 Created pod: payload-processing-e2e-inf-b-124490-77db798586-7ghqs 42m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-124490-77db798586 Error creating: pods "payload-processing-e2e-inf-b-124490-77db798586-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-124490: serviceaccount "payload-processing-e2e-inf-b-124490" not found 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-124490 Scaled up replica set payload-pre-processing-e2e-inf-b-124490-7cbff46899 from 0 to 1 42m Normal Pulled pod/payload-pre-processing-e2e-inf-b-124490-7cbff46899-7c5p6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-124490 Scaled up replica set payload-processing-e2e-inf-b-124490-77db798586 from 0 to 1 42m Normal SuccessfulDelete replicaset/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c Deleted pod: e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-jpkff 42m Normal AddedInterface pod/payload-processing-e2e-inf-b-124490-77db798586-7ghqs Add eth0 [10.132.0.225/23] from ovn-kubernetes 42m Normal Created pod/payload-pre-processing-e2e-inf-b-124490-7cbff46899-7c5p6 Container created 42m Normal Created pod/payload-processing-e2e-inf-b-124490-77db798586-7ghqs Container created 42m Normal Pulled pod/payload-processing-e2e-inf-b-124490-77db798586-7ghqs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Started pod/payload-pre-processing-e2e-inf-b-124490-7cbff46899-7c5p6 Container started 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-a940d5-openshift-default New size: 3; reason: All metrics below target 42m Normal Started pod/payload-processing-e2e-inf-b-124490-77db798586-7ghqs Container started 42m Normal Killing pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-jpkff Stopping container istio-proxy 42m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-a940d5-openshift-default Scaled down replica set e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c from 4 to 3 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-50d738-openshift-default New size: 3; reason: All metrics below target 42m Normal SuccessfulDelete replicaset/e2e-worker-w4-50d738-openshift-default-8d6476cbc Deleted pod: e2e-worker-w4-50d738-openshift-default-8d6476cbc-cqz7h 42m Normal Killing pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-cqz7h Stopping container istio-proxy 42m Normal ScalingReplicaSet deployment/e2e-worker-w4-50d738-openshift-default Scaled down replica set e2e-worker-w4-50d738-openshift-default-8d6476cbc from 4 to 3 42m Normal ScalingReplicaSet deployment/e2e-worker-w6-2d619f-openshift-default Scaled down replica set e2e-worker-w6-2d619f-openshift-default-85b84758df from 4 to 3 42m Normal SuccessfulDelete replicaset/e2e-worker-w5-434ce3-openshift-default-58c7774759 Deleted pod: e2e-worker-w5-434ce3-openshift-default-58c7774759-bsswj 42m Normal SuccessfulDelete replicaset/e2e-worker-w6-2d619f-openshift-default-85b84758df Deleted pod: e2e-worker-w6-2d619f-openshift-default-85b84758df-cj9f5 42m Normal ScalingReplicaSet deployment/e2e-worker-w5-434ce3-openshift-default Scaled down replica set e2e-worker-w5-434ce3-openshift-default-58c7774759 from 4 to 3 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-434ce3-openshift-default New size: 3; reason: All metrics below target 42m Normal Killing pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-bsswj Stopping container istio-proxy 42m Normal Killing pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-cj9f5 Stopping container istio-proxy 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-2d619f-openshift-default New size: 3; reason: All metrics below target 42m Normal SuccessfulCreate replicaset/e2e-inf-b-0a7065-openshift-default-56696cf555 Created pod: e2e-inf-b-0a7065-openshift-default-56696cf555-sf6vl 42m Normal ScalingReplicaSet deployment/e2e-inf-b-0a7065-openshift-default Scaled up replica set e2e-inf-b-0a7065-openshift-default-56696cf555 from 1 to 2 42m Normal Created pod/e2e-inf-b-0a7065-openshift-default-56696cf555-sf6vl Container created 42m Normal Pulled pod/e2e-inf-b-0a7065-openshift-default-56696cf555-sf6vl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/e2e-inf-b-0a7065-openshift-default-56696cf555-sf6vl Add eth0 [10.133.0.122/23] from ovn-kubernetes 42m Normal Started pod/e2e-inf-b-0a7065-openshift-default-56696cf555-sf6vl Container started 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-0a7065-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-bb1fa5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-bb1fa5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning Unhealthy pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-cqz7h Readiness probe failed: HTTP probe failed with statuscode: 503 42m Warning ProbeError pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-cqz7h Readiness probe error: HTTP probe failed with statuscode: 503... 42m Warning Unhealthy pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-cj9f5 Readiness probe failed: HTTP probe failed with statuscode: 503 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-341c51-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-341c51-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning ProbeError pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-cj9f5 Readiness probe error: HTTP probe failed with statuscode: 503... 42m Warning ProbeError pod/e2e-inf-b-0a7065-openshift-default-56696cf555-sf6vl Startup probe error: Get "http://10.133.0.122:15021/healthz/ready": dial tcp 10.133.0.122:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-inf-b-0a7065-openshift-default-56696cf555-sf6vl Startup probe failed: Get "http://10.133.0.122:15021/healthz/ready": dial tcp 10.133.0.122:15021: connect: connection refused 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-bb1fa5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-bb1fa5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal Created pod/e2e-inf-b-ccdb60-openshift-default-678d965f7c-cttbz Container created 42m Normal Pulled pod/e2e-inf-b-ccdb60-openshift-default-678d965f7c-cttbz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Started pod/e2e-inf-b-ccdb60-openshift-default-678d965f7c-cttbz Container started 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-341c51-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-341c51-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning Unhealthy pod/e2e-inf-b-ccdb60-openshift-default-678d965f7c-cttbz Startup probe failed: Get "http://10.133.0.115:15021/healthz/ready": dial tcp 10.133.0.115:15021: connect: connection refused 42m Warning ProbeError pod/e2e-inf-b-ccdb60-openshift-default-678d965f7c-cttbz Startup probe error: Get "http://10.133.0.115:15021/healthz/ready": dial tcp 10.133.0.115:15021: connect: connection refused... 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-ccdb60-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-ccdb60-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning Unhealthy pod/e2e-inf-b-fe7ca0-openshift-default-7598bdb7d5-jqmrs Readiness probe failed: Get "http://10.134.0.171:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Warning ProbeError pod/e2e-inf-b-fe7ca0-openshift-default-7598bdb7d5-jqmrs Readiness probe error: Get "http://10.134.0.171:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Normal Started pod/e2e-inf-b-fe7ca0-openshift-default-7598bdb7d5-jqmrs Container started 41m Normal Pulled pod/e2e-inf-b-fe7ca0-openshift-default-7598bdb7d5-jqmrs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-inf-b-fe7ca0-openshift-default-7598bdb7d5-jqmrs Container created 41m Warning ProbeError pod/e2e-inf-b-fe7ca0-openshift-default-7598bdb7d5-jqmrs Startup probe error: Get "http://10.134.0.171:15021/healthz/ready": dial tcp 10.134.0.171:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-inf-b-fe7ca0-openshift-default-7598bdb7d5-jqmrs Startup probe failed: Get "http://10.134.0.171:15021/healthz/ready": dial tcp 10.134.0.171:15021: connect: connection refused 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-fe7ca0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-fe7ca0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal Killing pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-q8mkq Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w1-e19850-openshift-default-9f855bd47 Deleted pod: e2e-worker-w1-e19850-openshift-default-9f855bd47-q8mkq 41m Warning ProbeError pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-q8mkq Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-q8mkq Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-fe7ca0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-fe7ca0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal Pulled pod/e2e-inf-b-963936-openshift-default-68b65f857c-phdvv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-inf-b-124490-openshift-default-54b4ffbf45-88rs8 Container created 40m Normal Pulled pod/e2e-inf-b-124490-openshift-default-54b4ffbf45-88rs8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-inf-b-963936-openshift-default-68b65f857c-phdvv Container created 40m Normal Started pod/e2e-inf-b-963936-openshift-default-68b65f857c-phdvv Container started 40m Normal Started pod/e2e-inf-b-124490-openshift-default-54b4ffbf45-88rs8 Container started 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-963936-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-963936-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning Unhealthy pod/e2e-inf-b-124490-openshift-default-54b4ffbf45-88rs8 Startup probe failed: Get "http://10.132.0.224:15021/healthz/ready": dial tcp 10.132.0.224:15021: connect: connection refused 40m Warning ProbeError pod/e2e-inf-b-124490-openshift-default-54b4ffbf45-88rs8 Startup probe error: Get "http://10.132.0.224:15021/healthz/ready": dial tcp 10.132.0.224:15021: connect: connection refused... 40m Normal SuccessfulCreate replicaset/e2e-inf-a-7940ea-openshift-default-75b99d856c Created pod: e2e-inf-a-7940ea-openshift-default-75b99d856c-n9xtq 40m Normal ScalingReplicaSet deployment/e2e-inf-a-7940ea-openshift-default Scaled up replica set e2e-inf-a-7940ea-openshift-default-75b99d856c from 1 to 2 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-7940ea-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 40m Warning Unhealthy pod/e2e-inf-b-963936-openshift-default-68b65f857c-phdvv Startup probe failed: Get "http://10.133.0.120:15021/healthz/ready": dial tcp 10.133.0.120:15021: connect: connection refused 40m Warning ProbeError pod/e2e-inf-b-963936-openshift-default-68b65f857c-phdvv Startup probe error: Get "http://10.133.0.120:15021/healthz/ready": dial tcp 10.133.0.120:15021: connect: connection refused... 40m Normal Started pod/e2e-inf-a-7940ea-openshift-default-75b99d856c-n9xtq Container started 40m Normal Pulled pod/e2e-inf-a-7940ea-openshift-default-75b99d856c-n9xtq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/e2e-inf-a-7940ea-openshift-default-75b99d856c-n9xtq Add eth0 [10.132.0.251/23] from ovn-kubernetes 40m Normal Created pod/e2e-inf-a-7940ea-openshift-default-75b99d856c-n9xtq Container created 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-50b9b5-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 40m Normal ScalingReplicaSet deployment/e2e-inf-a-50b9b5-openshift-default Scaled up replica set e2e-inf-a-50b9b5-openshift-default-6f7ffb6f from 1 to 2 40m Normal SuccessfulCreate replicaset/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f Created pod: e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-jwjrc 40m Normal Created pod/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-jwjrc Container created 40m Normal Started pod/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-jwjrc Container started 40m Normal Pulled pod/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-jwjrc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-jwjrc Add eth0 [10.133.0.135/23] from ovn-kubernetes 40m Warning Unhealthy pod/e2e-inf-a-7940ea-openshift-default-75b99d856c-n9xtq Startup probe failed: Get "http://10.132.0.251:15021/healthz/ready": dial tcp 10.132.0.251:15021: connect: connection refused 40m Warning ProbeError pod/e2e-inf-a-7940ea-openshift-default-75b99d856c-n9xtq Startup probe error: Get "http://10.132.0.251:15021/healthz/ready": dial tcp 10.132.0.251:15021: connect: connection refused... 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-124490-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-124490-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning Unhealthy pod/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-jwjrc Startup probe failed: Get "http://10.133.0.135:15021/healthz/ready": dial tcp 10.133.0.135:15021: connect: connection refused 40m Warning ProbeError pod/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-jwjrc Startup probe error: Get "http://10.133.0.135:15021/healthz/ready": dial tcp 10.133.0.135:15021: connect: connection refused... 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-553753-openshift-default New size: 2; reason: All metrics below target 40m Normal Killing pod/e2e-worker-w2-553753-openshift-default-558949869c-fgvq5 Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w2-553753-openshift-default-558949869c Deleted pod: e2e-worker-w2-553753-openshift-default-558949869c-fgvq5 40m Normal ScalingReplicaSet deployment/e2e-worker-w2-553753-openshift-default Scaled down replica set e2e-worker-w2-553753-openshift-default-558949869c from 3 to 2 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-124490-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-124490-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning Unhealthy pod/e2e-worker-w2-553753-openshift-default-558949869c-fgvq5 Readiness probe failed: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w2-553753-openshift-default-558949869c-fgvq5 Readiness probe error: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused... 40m Normal Killing pod/payload-pre-processing-e2e-inf-a-4b5e2b-7dd898677f-6768h Stopping container payload-pre-processing 40m Normal Killing pod/payload-processing-e2e-inf-a-4b5e2b-56d495d764-wxzjw Stopping container payload-processing 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-963936-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-963936-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-4b5e2b-7dd898677f-6768h Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 39m Warning Unhealthy pod/payload-processing-e2e-inf-a-4b5e2b-56d495d764-wxzjw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 39m Normal ScalingReplicaSet deployment/e2e-worker-w3-5187e1-openshift-default Scaled down replica set e2e-worker-w3-5187e1-openshift-default-946849bdf from 4 to 3 39m Normal Killing pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-g45p4 Stopping container istio-proxy 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-5187e1-openshift-default New size: 3; reason: All metrics below target 39m Normal SuccessfulDelete replicaset/e2e-worker-w3-5187e1-openshift-default-946849bdf Deleted pod: e2e-worker-w3-5187e1-openshift-default-946849bdf-g45p4 39m Normal Killing pod/e2e-inf-a-4b5e2b-openshift-default-7496d87895-f5vss Stopping container istio-proxy 39m Normal Killing pod/payload-processing-e2e-inf-b-2d6076-d576997dd-rh48p Stopping container payload-processing 39m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-0847aa-openshift-default Scaled down replica set e2e-shared-a-w0-0847aa-openshift-default-75d5774789 from 3 to 2 39m Normal Killing pod/payload-pre-processing-e2e-inf-b-2d6076-dc5c6ff87-4d8d6 Stopping container payload-pre-processing 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-0847aa-openshift-default New size: 2; reason: All metrics below target 39m Normal SuccessfulDelete replicaset/e2e-shared-a-w0-0847aa-openshift-default-75d5774789 Deleted pod: e2e-shared-a-w0-0847aa-openshift-default-75d5774789-8xg84 39m Normal Killing pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-8xg84 Stopping container istio-proxy 39m Normal Killing pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-fd2cz Stopping container istio-proxy 39m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-a940d5-openshift-default Scaled down replica set e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c from 3 to 2 39m Normal SuccessfulDelete replicaset/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c Deleted pod: e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-fd2cz 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-a940d5-openshift-default New size: 2; reason: All metrics below target 39m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-04988c-openshift-default Scaled down replica set e2e-shared-b-w4-04988c-openshift-default-d99f984b9 from 2 to 1 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-04988c-openshift-default New size: 1; reason: All metrics below target 39m Normal Killing pod/e2e-shared-b-w4-04988c-openshift-default-d99f984b9-zmsb6 Stopping container istio-proxy 39m Normal SuccessfulDelete replicaset/e2e-shared-b-w4-04988c-openshift-default-d99f984b9 Deleted pod: e2e-shared-b-w4-04988c-openshift-default-d99f984b9-zmsb6 39m Warning Unhealthy pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-8xg84 Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning ProbeError pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-8xg84 Readiness probe error: HTTP probe failed with statuscode: 503... 39m Normal Killing pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-gwb2l Stopping container istio-proxy 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-2d619f-openshift-default New size: 2; reason: All metrics below target 39m Normal SuccessfulDelete replicaset/e2e-worker-w6-2d619f-openshift-default-85b84758df Deleted pod: e2e-worker-w6-2d619f-openshift-default-85b84758df-gwb2l 39m Normal Killing pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-dcb6t Stopping container istio-proxy 39m Normal ScalingReplicaSet deployment/e2e-worker-w0-88a4d9-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w0-88a4d9-openshift-default-857757469c from 3 to 2 39m Normal SuccessfulDelete replicaset/e2e-worker-w0-88a4d9-openshift-default-857757469c Deleted pod: e2e-worker-w0-88a4d9-openshift-default-857757469c-dcb6t 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-88a4d9-openshift-default New size: 2; reason: All metrics below target 39m Warning ProbeError pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-dcb6t Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-dcb6t Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning ProbeError pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-gwb2l Readiness probe error: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-gwb2l Readiness probe failed: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused 38m Warning Unhealthy pod/payload-processing-e2e-inf-b-2d6076-d576997dd-rh48p Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 38m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-2d6076-dc5c6ff87-4d8d6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 38m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7df854bf67 Deleted pod: maas-default-gateway-openshift-default-7df854bf67-ddx5s 38m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7df854bf67 from 10 to 9 38m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 9; reason: All metrics below target 38m Normal Killing pod/maas-default-gateway-openshift-default-7df854bf67-ddx5s Stopping container istio-proxy 38m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-ddx5s Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-ddx5s Readiness probe error: HTTP probe failed with statuscode: 503... 38m Normal Killing pod/e2e-inf-b-2d6076-openshift-default-5bdb59ccf9-48ljn Stopping container istio-proxy 38m Normal AddedLabel gateway/e2e-shared-a-w2-d4b6bb Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-d4b6bb 38m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-d4b6bb-openshift-default Scaled up replica set e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff from 0 to 1 38m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff Created pod: e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff-b4dkg 38m Normal AddedInterface pod/e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff-b4dkg Add eth0 [10.134.0.176/23] from ovn-kubernetes 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-50d738-openshift-default New size: 2; reason: All metrics below target 38m Normal SuccessfulDelete replicaset/e2e-worker-w4-50d738-openshift-default-8d6476cbc Deleted pod: e2e-worker-w4-50d738-openshift-default-8d6476cbc-scbfp 38m Normal Killing pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-scbfp Stopping container istio-proxy 38m Normal SuccessfulDelete replicaset/e2e-worker-w5-434ce3-openshift-default-58c7774759 Deleted pod: e2e-worker-w5-434ce3-openshift-default-58c7774759-m2b7t 38m Normal Killing pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-m2b7t Stopping container istio-proxy 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-434ce3-openshift-default New size: 2; reason: All metrics below target 38m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7df854bf67 Deleted pod: maas-default-gateway-openshift-default-7df854bf67-crcsv 38m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7df854bf67 Deleted pod: maas-default-gateway-openshift-default-7df854bf67-qhqx9 38m Warning ProbeError pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-scbfp Readiness probe error: HTTP probe failed with statuscode: 503... 38m Warning Unhealthy pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-scbfp Readiness probe failed: HTTP probe failed with statuscode: 503 38m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: All metrics below target 38m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7df854bf67 from 9 to 6 38m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7df854bf67 Deleted pod: maas-default-gateway-openshift-default-7df854bf67-d6hr4 38m Normal Killing pod/maas-default-gateway-openshift-default-7df854bf67-crcsv Stopping container istio-proxy 38m Normal Killing pod/maas-default-gateway-openshift-default-7df854bf67-qhqx9 Stopping container istio-proxy 38m Normal Killing pod/maas-default-gateway-openshift-default-7df854bf67-d6hr4 Stopping container istio-proxy 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-d4b6bb Scaled up replica set payload-processing-e2e-shared-a-w2-d4b6bb-75d4df64b8 from 0 to 1 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-d4b6bb-75d4df64b8 Created pod: payload-processing-e2e-shared-a-w2-d4b6bb-75d4df64b8-nvlff 38m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-d4b6bb-75d4df64b8 Error creating: pods "payload-processing-e2e-shared-a-w2-d4b6bb-75d4df64b8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-d4b6bb: serviceaccount "payload-processing-e2e-shared-a-w2-d4b6bb" not found 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-d4b6bb-7678b5f4f9 Created pod: payload-pre-processing-e2e-shared-a-w2-d4b6bb-7678b5f4f9-d68cc 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-d4b6bb-7678b5f4f9 Error creating: pods "payload-pre-processing-e2e-shared-a-w2-d4b6bb-7678b5f4f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-d4b6bb: serviceaccount "payload-processing-e2e-shared-a-w2-d4b6bb" not found 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-d4b6bb Scaled up replica set payload-pre-processing-e2e-shared-a-w2-d4b6bb-7678b5f4f9 from 0 to 1 38m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-d4b6bb-7678b5f4f9-d68cc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-d4b6bb-7678b5f4f9-d68cc Container created 38m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-d4b6bb-7678b5f4f9-d68cc Container started 38m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-d4b6bb-7678b5f4f9-d68cc Add eth0 [10.132.1.3/23] from ovn-kubernetes 38m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-d4b6bb-75d4df64b8-nvlff Add eth0 [10.133.0.136/23] from ovn-kubernetes 38m Normal Pulled pod/payload-processing-e2e-shared-a-w2-d4b6bb-75d4df64b8-nvlff Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Created pod/payload-processing-e2e-shared-a-w2-d4b6bb-75d4df64b8-nvlff Container created 38m Normal Started pod/payload-processing-e2e-shared-a-w2-d4b6bb-75d4df64b8-nvlff Container started 38m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-crcsv Readiness probe error: HTTP probe failed with statuscode: 503... 38m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-crcsv Readiness probe failed: HTTP probe failed with statuscode: 503 38m Normal Killing pod/payload-processing-e2e-inf-a-b1ec3a-5c67ddd778-kzpcm Stopping container payload-processing 38m Normal Killing pod/payload-pre-processing-e2e-inf-a-b1ec3a-6fdb57959f-fxmdx Stopping container payload-pre-processing 37m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-b1ec3a-6fdb57959f-fxmdx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Warning Unhealthy pod/e2e-inf-a-b1ec3a-openshift-default-7675569d9-qjkx2 Readiness probe failed: HTTP probe failed with statuscode: 503 37m Normal Killing pod/e2e-inf-a-b1ec3a-openshift-default-7675569d9-kjjgn Stopping container istio-proxy 37m Normal Killing pod/e2e-inf-a-b1ec3a-openshift-default-7675569d9-qjkx2 Stopping container istio-proxy 37m Warning ProbeError pod/e2e-inf-a-b1ec3a-openshift-default-7675569d9-qjkx2 Readiness probe error: HTTP probe failed with statuscode: 503... 37m Normal Killing pod/payload-processing-e2e-inf-b-0a7065-694768f4b4-5fq8k Stopping container payload-processing 37m Normal Killing pod/payload-pre-processing-e2e-inf-b-0a7065-7bc48cbbf9-psdjd Stopping container payload-pre-processing 37m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-2xmvs Stopping container discovery 37m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-2xmvs 37m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 37m Normal Killing pod/payload-processing-e2e-inf-a-9e540d-79df886dc8-q6kwt Stopping container payload-processing 37m Normal Killing pod/payload-pre-processing-e2e-inf-a-9e540d-67db6b6b55-f687b Stopping container payload-pre-processing 37m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-0a7065-7bc48cbbf9-psdjd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-9e540d-67db6b6b55-f687b Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Normal Killing pod/e2e-inf-b-0a7065-openshift-default-56696cf555-w2wcw Stopping container istio-proxy 36m Normal Killing pod/e2e-inf-b-0a7065-openshift-default-56696cf555-sf6vl Stopping container istio-proxy 36m Normal AddedLabel gateway/e2e-shared-b-w2-4f9bb0 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-4f9bb0 36m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-4f9bb0-openshift-default Scaled up replica set e2e-shared-b-w2-4f9bb0-openshift-default-599dcdf4f9 from 0 to 1 36m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-4f9bb0-openshift-default-599dcdf4f9 Created pod: e2e-shared-b-w2-4f9bb0-openshift-default-599dcdf4f9-pwn4t 36m Normal AddedInterface pod/e2e-shared-b-w2-4f9bb0-openshift-default-599dcdf4f9-pwn4t Add eth0 [10.133.0.139/23] from ovn-kubernetes 36m Normal Killing pod/e2e-inf-a-9e540d-openshift-default-74d865bc89-p5z9q Stopping container istio-proxy 36m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-4f9bb0-58c4b77cfb Error creating: pods "payload-processing-e2e-shared-b-w2-4f9bb0-58c4b77cfb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-4f9bb0: serviceaccount "payload-processing-e2e-shared-b-w2-4f9bb0" not found 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-4f9bb0 Scaled up replica set payload-pre-processing-e2e-shared-b-w2-4f9bb0-6f8b69d999 from 0 to 1 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-4f9bb0 Scaled up replica set payload-processing-e2e-shared-b-w2-4f9bb0-58c4b77cfb from 0 to 1 36m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-4f9bb0-6f8b69d999-g9rgk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Created pod/payload-processing-e2e-shared-b-w2-4f9bb0-58c4b77cfb-cdgpk Container created 36m Normal Started pod/payload-processing-e2e-shared-b-w2-4f9bb0-58c4b77cfb-cdgpk Container started 36m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-4f9bb0-58c4b77cfb-cdgpk Add eth0 [10.133.0.140/23] from ovn-kubernetes 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-4f9bb0-58c4b77cfb Created pod: payload-processing-e2e-shared-b-w2-4f9bb0-58c4b77cfb-cdgpk 36m Normal Pulled pod/payload-processing-e2e-shared-b-w2-4f9bb0-58c4b77cfb-cdgpk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-4f9bb0-6f8b69d999 Error creating: pods "payload-pre-processing-e2e-shared-b-w2-4f9bb0-6f8b69d999-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-4f9bb0: serviceaccount "payload-processing-e2e-shared-b-w2-4f9bb0" not found 36m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-4f9bb0-6f8b69d999-g9rgk Container started 36m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-4f9bb0-6f8b69d999-g9rgk Add eth0 [10.133.0.141/23] from ovn-kubernetes 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-4f9bb0-6f8b69d999 Created pod: payload-pre-processing-e2e-shared-b-w2-4f9bb0-6f8b69d999-g9rgk 36m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-4f9bb0-6f8b69d999-g9rgk Container created 36m Normal Killing pod/payload-pre-processing-e2e-inf-b-71a7df-fdd88b4b7-sdbdj Stopping container payload-pre-processing 36m Normal Killing pod/payload-processing-e2e-inf-b-71a7df-5f4b644b-zwb9k Stopping container payload-processing 36m Warning Unhealthy pod/payload-processing-e2e-inf-b-71a7df-5f4b644b-zwb9k Readiness probe failed: timeout: failed to connect service "10.133.0.107:50052" within 1s: context deadline exceeded 36m Normal Pulled pod/e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff-b4dkg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff-b4dkg Container started 36m Normal Created pod/e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff-b4dkg Container created 36m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-71a7df-fdd88b4b7-sdbdj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Warning Unhealthy pod/e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff-b4dkg Startup probe failed: Get "http://10.134.0.176:15021/healthz/ready": dial tcp 10.134.0.176:15021: connect: connection refused 36m Warning ProbeError pod/e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff-b4dkg Startup probe error: Get "http://10.134.0.176:15021/healthz/ready": dial tcp 10.134.0.176:15021: connect: connection refused... 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-4f9bb0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-4f9bb0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-d4b6bb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-d4b6bb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Normal Killing pod/e2e-inf-b-71a7df-openshift-default-74d8445657-fv778 Stopping container istio-proxy 35m Normal Killing pod/e2e-inf-b-71a7df-openshift-default-74d8445657-j68rv Stopping container istio-proxy 35m Normal Killing pod/e2e-inf-b-71a7df-openshift-default-74d8445657-bgwbv Stopping container istio-proxy 35m Normal Killing pod/e2e-inf-b-71a7df-openshift-default-74d8445657-zqkwp Stopping container istio-proxy 35m Normal AddedLabel gateway/e2e-shared-a-w5-b023a2 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w5-b023a2 35m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-b023a2-openshift-default Scaled up replica set e2e-shared-a-w5-b023a2-openshift-default-7bd456f44f from 0 to 1 35m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-b023a2-openshift-default-7bd456f44f Created pod: e2e-shared-a-w5-b023a2-openshift-default-7bd456f44f-wshn9 35m Warning Unhealthy pod/e2e-inf-b-71a7df-openshift-default-74d8445657-j68rv Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-inf-b-71a7df-openshift-default-74d8445657-j68rv Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-inf-b-71a7df-openshift-default-74d8445657-fv778 Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-inf-b-71a7df-openshift-default-74d8445657-fv778 Readiness probe error: HTTP probe failed with statuscode: 503... 35m Normal AddedInterface pod/e2e-shared-a-w5-b023a2-openshift-default-7bd456f44f-wshn9 Add eth0 [10.133.0.144/23] from ovn-kubernetes 35m Warning Unhealthy pod/e2e-inf-b-71a7df-openshift-default-74d8445657-bgwbv Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-inf-b-71a7df-openshift-default-74d8445657-bgwbv Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w5-b023a2-f5bd44854 Error creating: pods "payload-processing-e2e-shared-a-w5-b023a2-f5bd44854-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-b023a2: serviceaccount "payload-processing-e2e-shared-a-w5-b023a2" not found 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w5-b023a2-f5bd44854 Created pod: payload-processing-e2e-shared-a-w5-b023a2-f5bd44854-vjwb2 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w5-b023a2-678fcf5d79 Error creating: pods "payload-pre-processing-e2e-shared-a-w5-b023a2-678fcf5d79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-b023a2: serviceaccount "payload-processing-e2e-shared-a-w5-b023a2" not found 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w5-b023a2-678fcf5d79 Created pod: payload-pre-processing-e2e-shared-a-w5-b023a2-678fcf5d79-x2g2r 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w5-b023a2 Scaled up replica set payload-pre-processing-e2e-shared-a-w5-b023a2-678fcf5d79 from 0 to 1 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w5-b023a2 Scaled up replica set payload-processing-e2e-shared-a-w5-b023a2-f5bd44854 from 0 to 1 35m Normal Started pod/payload-pre-processing-e2e-shared-a-w5-b023a2-678fcf5d79-x2g2r Container started 35m Normal Created pod/payload-pre-processing-e2e-shared-a-w5-b023a2-678fcf5d79-x2g2r Container created 35m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w5-b023a2-678fcf5d79-x2g2r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w5-b023a2-678fcf5d79-x2g2r Add eth0 [10.133.0.146/23] from ovn-kubernetes 35m Normal AddedInterface pod/payload-processing-e2e-shared-a-w5-b023a2-f5bd44854-vjwb2 Add eth0 [10.133.0.145/23] from ovn-kubernetes 35m Normal Started pod/payload-processing-e2e-shared-a-w5-b023a2-f5bd44854-vjwb2 Container started 35m Normal Created pod/payload-processing-e2e-shared-a-w5-b023a2-f5bd44854-vjwb2 Container created 35m Normal Pulled pod/payload-processing-e2e-shared-a-w5-b023a2-f5bd44854-vjwb2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-d4b6bb-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-d4b6bb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-5187e1-openshift-default New size: 2; reason: All metrics below target 35m Normal ScalingReplicaSet deployment/e2e-worker-w2-553753-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w2-553753-openshift-default-558949869c from 2 to 1 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-553753-openshift-default New size: 1; reason: All metrics below target 35m Normal Killing pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-td5c9 Stopping container istio-proxy 35m Normal SuccessfulDelete replicaset/e2e-worker-w2-553753-openshift-default-558949869c Deleted pod: e2e-worker-w2-553753-openshift-default-558949869c-ktl7d 35m Normal SuccessfulDelete replicaset/e2e-worker-w3-5187e1-openshift-default-946849bdf Deleted pod: e2e-worker-w3-5187e1-openshift-default-946849bdf-td5c9 35m Normal Killing pod/e2e-worker-w2-553753-openshift-default-558949869c-ktl7d Stopping container istio-proxy 35m Normal Killing pod/maas-default-gateway-openshift-default-7df854bf67-5w59f Stopping container istio-proxy 35m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: All metrics below target 35m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7df854bf67 from 6 to 4 35m Normal Killing pod/maas-default-gateway-openshift-default-7df854bf67-v828p Stopping container istio-proxy 35m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7df854bf67 Deleted pod: maas-default-gateway-openshift-default-7df854bf67-v828p 35m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7df854bf67 Deleted pod: maas-default-gateway-openshift-default-7df854bf67-5w59f 35m Warning ProbeError pod/e2e-worker-w2-553753-openshift-default-558949869c-ktl7d Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-worker-w2-553753-openshift-default-558949869c-ktl7d Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-5w59f Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-5w59f Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-shared-b-w2-4f9bb0-openshift-default-599dcdf4f9-pwn4t Readiness probe error: Get "http://10.133.0.139:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 35m Warning Unhealthy pod/e2e-shared-b-w2-4f9bb0-openshift-default-599dcdf4f9-pwn4t Readiness probe failed: Get "http://10.133.0.139:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 35m Normal Started pod/e2e-shared-b-w2-4f9bb0-openshift-default-599dcdf4f9-pwn4t Container started 35m Normal Created pod/e2e-shared-b-w2-4f9bb0-openshift-default-599dcdf4f9-pwn4t Container created 35m Normal Pulled pod/e2e-shared-b-w2-4f9bb0-openshift-default-599dcdf4f9-pwn4t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Warning ProbeError pod/e2e-shared-b-w2-4f9bb0-openshift-default-599dcdf4f9-pwn4t Startup probe error: Get "http://10.133.0.139:15021/healthz/ready": dial tcp 10.133.0.139:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-shared-b-w2-4f9bb0-openshift-default-599dcdf4f9-pwn4t Startup probe failed: Get "http://10.133.0.139:15021/healthz/ready": dial tcp 10.133.0.139:15021: connect: connection refused 34m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-0847aa-openshift-default Scaled down replica set e2e-shared-a-w0-0847aa-openshift-default-75d5774789 from 2 to 1 34m Normal Killing pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-5jtnl Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/e2e-shared-a-w0-0847aa-openshift-default-75d5774789 Deleted pod: e2e-shared-a-w0-0847aa-openshift-default-75d5774789-5jtnl 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-0847aa-openshift-default New size: 1; reason: All metrics below target 34m Warning ProbeError pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-5jtnl Readiness probe error: Get "http://10.132.0.162:15021/healthz/ready": dial tcp 10.132.0.162:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-5jtnl Readiness probe failed: Get "http://10.132.0.162:15021/healthz/ready": dial tcp 10.132.0.162:15021: connect: connection refused 34m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-a940d5-openshift-default Scaled down replica set e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c from 2 to 1 34m Normal Killing pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-ksw7k Stopping container istio-proxy 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-a940d5-openshift-default New size: 1; reason: All metrics below target 34m Normal SuccessfulDelete replicaset/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c Deleted pod: e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-ksw7k 34m Warning Unhealthy pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-ksw7k Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning ProbeError pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-ksw7k Readiness probe error: HTTP probe failed with statuscode: 503... 34m Normal Killing pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-nj6zd Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/e2e-worker-w4-50d738-openshift-default-8d6476cbc Deleted pod: e2e-worker-w4-50d738-openshift-default-8d6476cbc-nj6zd 34m Normal ScalingReplicaSet deployment/e2e-worker-w4-50d738-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w4-50d738-openshift-default-8d6476cbc from 2 to 1 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-50d738-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 34m Warning ProbeError pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-nj6zd Readiness probe error: HTTP probe failed with statuscode: 503... 34m Warning Unhealthy pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-nj6zd Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-4f9bb0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-4f9bb0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Normal Pulled pod/e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff-8l4w8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff Created pod: e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff-8l4w8 34m Normal AddedInterface pod/e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff-8l4w8 Add eth0 [10.133.0.147/23] from ovn-kubernetes 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-d4b6bb-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 34m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-d4b6bb-openshift-default Scaled up replica set e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff from 1 to 2 34m Normal Started pod/e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff-8l4w8 Container started 34m Normal Created pod/e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff-8l4w8 Container created 34m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-9b841d-openshift-default Scaled up replica set e2e-shared-b-w5-9b841d-openshift-default-797cdf6799 from 0 to 1 34m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-9b841d-openshift-default-797cdf6799 Created pod: e2e-shared-b-w5-9b841d-openshift-default-797cdf6799-fnr9n 34m Normal AddedLabel gateway/e2e-shared-b-w5-9b841d Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w5-9b841d 34m Normal AddedInterface pod/e2e-shared-b-w5-9b841d-openshift-default-797cdf6799-fnr9n Add eth0 [10.134.0.179/23] from ovn-kubernetes 34m Warning ProbeError pod/e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff-8l4w8 Startup probe error: Get "http://10.133.0.147:15021/healthz/ready": dial tcp 10.133.0.147:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff-8l4w8 Startup probe failed: Get "http://10.133.0.147:15021/healthz/ready": dial tcp 10.133.0.147:15021: connect: connection refused 34m Normal Killing pod/payload-pre-processing-e2e-inf-a-7940ea-6c48467979-8rjd5 Stopping container payload-pre-processing 34m Normal Killing pod/payload-processing-e2e-inf-a-7940ea-56b89d4b64-pvwxl Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-inf-a-50b9b5-76b76f77c9-g6sth Stopping container payload-pre-processing 34m Normal Killing pod/payload-processing-e2e-inf-a-50b9b5-7f48f98986-ddh6w Stopping container payload-processing 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w5-9b841d Scaled up replica set payload-pre-processing-e2e-shared-b-w5-9b841d-6447d9f595 from 0 to 1 34m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w5-9b841d-6447d9f595-qxv2f Add eth0 [10.132.1.16/23] from ovn-kubernetes 34m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w5-9b841d-6447d9f595-qxv2f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Created pod/payload-pre-processing-e2e-shared-b-w5-9b841d-6447d9f595-qxv2f Container created 34m Normal Pulled pod/payload-processing-e2e-shared-b-w5-9b841d-5c95644856-295pk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/payload-processing-e2e-shared-b-w5-9b841d-5c95644856-295pk Add eth0 [10.133.0.148/23] from ovn-kubernetes 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w5-9b841d-6447d9f595 Created pod: payload-pre-processing-e2e-shared-b-w5-9b841d-6447d9f595-qxv2f 34m Normal Created pod/payload-processing-e2e-shared-b-w5-9b841d-5c95644856-295pk Container created 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w5-9b841d-5c95644856 Created pod: payload-processing-e2e-shared-b-w5-9b841d-5c95644856-295pk 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w5-9b841d-6447d9f595 Error creating: pods "payload-pre-processing-e2e-shared-b-w5-9b841d-6447d9f595-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-9b841d: serviceaccount "payload-processing-e2e-shared-b-w5-9b841d" not found 34m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w5-9b841d-5c95644856 Error creating: pods "payload-processing-e2e-shared-b-w5-9b841d-5c95644856-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-9b841d: serviceaccount "payload-processing-e2e-shared-b-w5-9b841d" not found 34m Normal Started pod/payload-processing-e2e-shared-b-w5-9b841d-5c95644856-295pk Container started 34m Normal Started pod/payload-pre-processing-e2e-shared-b-w5-9b841d-6447d9f595-qxv2f Container started 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w5-9b841d Scaled up replica set payload-processing-e2e-shared-b-w5-9b841d-5c95644856 from 0 to 1 34m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-50b9b5-76b76f77c9-g6sth Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning Unhealthy pod/payload-processing-e2e-inf-a-50b9b5-7f48f98986-ddh6w Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-7940ea-6c48467979-8rjd5 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Normal Killing pod/e2e-inf-a-7940ea-openshift-default-75b99d856c-56bcx Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-a-7940ea-openshift-default-75b99d856c-n9xtq Stopping container istio-proxy 33m Normal Killing pod/payload-pre-processing-e2e-inf-b-ccdb60-779c4b9f95-rrp8q Stopping container payload-pre-processing 33m Normal Killing pod/payload-processing-e2e-inf-b-ccdb60-7fc6f896b-qrxhs Stopping container payload-processing 33m Normal Started pod/e2e-shared-a-w5-b023a2-openshift-default-7bd456f44f-wshn9 Container started 33m Normal Pulled pod/e2e-shared-a-w5-b023a2-openshift-default-7bd456f44f-wshn9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-shared-a-w5-b023a2-openshift-default-7bd456f44f-wshn9 Container created 33m Warning ProbeError pod/e2e-shared-a-w5-b023a2-openshift-default-7bd456f44f-wshn9 Startup probe error: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-shared-a-w5-b023a2-openshift-default-7bd456f44f-wshn9 Startup probe failed: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused 33m Normal Killing pod/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-jwjrc Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-br8fv Stopping container istio-proxy 33m Warning ProbeError pod/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-br8fv Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning ProbeError pod/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-jwjrc Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-br8fv Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning Unhealthy pod/e2e-inf-a-50b9b5-openshift-default-6f7ffb6f-jwjrc Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-ccdb60-779c4b9f95-rrp8q Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Warning Unhealthy pod/payload-processing-e2e-inf-b-ccdb60-7fc6f896b-qrxhs Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-b023a2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-b023a2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Normal Killing pod/payload-processing-e2e-inf-b-fe7ca0-55488976b6-5hnxs Stopping container payload-processing 33m Normal Killing pod/payload-pre-processing-e2e-inf-b-fe7ca0-54c86d5ff5-xqxp7 Stopping container payload-pre-processing 33m Normal Killing pod/e2e-inf-b-ccdb60-openshift-default-678d965f7c-cttbz Stopping container istio-proxy 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-b023a2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-b023a2-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-fe7ca0-54c86d5ff5-xqxp7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Normal Started pod/e2e-shared-b-w5-9b841d-openshift-default-797cdf6799-fnr9n Container started 32m Normal Created pod/e2e-shared-b-w5-9b841d-openshift-default-797cdf6799-fnr9n Container created 32m Normal Pulled pod/e2e-shared-b-w5-9b841d-openshift-default-797cdf6799-fnr9n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-9b841d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-9b841d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning Unhealthy pod/e2e-shared-b-w5-9b841d-openshift-default-797cdf6799-fnr9n Startup probe failed: Get "http://10.134.0.179:15021/healthz/ready": dial tcp 10.134.0.179:15021: connect: connection refused 32m Warning ProbeError pod/e2e-shared-b-w5-9b841d-openshift-default-797cdf6799-fnr9n Startup probe error: Get "http://10.134.0.179:15021/healthz/ready": dial tcp 10.134.0.179:15021: connect: connection refused... 32m Normal Killing pod/e2e-inf-b-fe7ca0-openshift-default-7598bdb7d5-jqmrs Stopping container istio-proxy 32m Normal Killing pod/payload-processing-e2e-inf-a-341c51-857b566498-5d2tg Stopping container payload-processing 32m Normal Killing pod/payload-pre-processing-e2e-inf-a-341c51-76d98b8f6c-cqm9p Stopping container payload-pre-processing 32m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-341c51-76d98b8f6c-cqm9p Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Warning Unhealthy pod/payload-processing-e2e-inf-a-341c51-857b566498-5d2tg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-9b841d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-9b841d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Normal Killing pod/e2e-inf-a-341c51-openshift-default-56f6687b6f-dqgpg Stopping container istio-proxy 31m Normal Killing pod/payload-processing-e2e-inf-b-124490-77db798586-7ghqs Stopping container payload-processing 31m Normal Killing pod/payload-pre-processing-e2e-inf-b-124490-7cbff46899-7c5p6 Stopping container payload-pre-processing 31m Normal Killing pod/payload-pre-processing-e2e-inf-a-bb1fa5-7948594b67-45z4r Stopping container payload-pre-processing 31m Normal Killing pod/payload-processing-e2e-inf-a-bb1fa5-6694d587bb-tdvvr Stopping container payload-processing 31m Normal SuccessfulDelete replicaset/e2e-worker-w5-434ce3-openshift-default-58c7774759 Deleted pod: e2e-worker-w5-434ce3-openshift-default-58c7774759-s5wxn 31m Normal ScalingReplicaSet deployment/e2e-worker-w1-e19850-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w1-e19850-openshift-default-9f855bd47 from 2 to 1 31m Normal Killing pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-84rpc Stopping container istio-proxy 31m Normal SuccessfulDelete replicaset/e2e-worker-w6-2d619f-openshift-default-85b84758df Deleted pod: e2e-worker-w6-2d619f-openshift-default-85b84758df-nnfpk 31m Normal ScalingReplicaSet deployment/e2e-worker-w5-434ce3-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w5-434ce3-openshift-default-58c7774759 from 2 to 1 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-2d619f-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 31m Normal ScalingReplicaSet deployment/e2e-worker-w6-2d619f-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w6-2d619f-openshift-default-85b84758df from 2 to 1 31m Normal Killing pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-nnfpk Stopping container istio-proxy 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-434ce3-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 31m Normal SuccessfulDelete replicaset/e2e-worker-w1-e19850-openshift-default-9f855bd47 Deleted pod: e2e-worker-w1-e19850-openshift-default-9f855bd47-84rpc 31m Normal Killing pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-s5wxn Stopping container istio-proxy 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e19850-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 31m Warning ProbeError pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-84rpc Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning Unhealthy pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-84rpc Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-124490-7cbff46899-7c5p6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Normal Killing pod/e2e-inf-a-bb1fa5-openshift-default-95cc6c6f-8b6vq Stopping container istio-proxy 31m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-bb1fa5-7948594b67-45z4r Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Warning Unhealthy pod/payload-processing-e2e-inf-a-bb1fa5-6694d587bb-tdvvr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc Created pod: e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc-wbswg 30m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-2a2eca-openshift-default Scaled up replica set e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc from 1 to 2 30m Normal Killing pod/e2e-inf-b-124490-openshift-default-54b4ffbf45-88rs8 Stopping container istio-proxy 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-2a2eca-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 30m Normal Pulled pod/e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc-wbswg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Started pod/e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc-wbswg Container started 30m Normal Created pod/e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc-wbswg Container created 30m Normal Killing pod/payload-pre-processing-e2e-inf-b-963936-6c6699ff5-mktm9 Stopping container payload-pre-processing 30m Normal Killing pod/payload-processing-e2e-inf-b-963936-589db57f54-5m29v Stopping container payload-processing 30m Normal AddedInterface pod/e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc-wbswg Add eth0 [10.132.1.30/23] from ovn-kubernetes 30m Warning Unhealthy pod/e2e-inf-b-124490-openshift-default-54b4ffbf45-88rs8 Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning ProbeError pod/e2e-inf-b-124490-openshift-default-54b4ffbf45-88rs8 Readiness probe error: HTTP probe failed with statuscode: 503... 30m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7 Created pod: e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7-xcxdb 30m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-d8a3d1-openshift-default Scaled up replica set e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7 from 1 to 2 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-d8a3d1-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 30m Normal AddedInterface pod/e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7-xcxdb Add eth0 [10.132.1.31/23] from ovn-kubernetes 30m Normal Pulled pod/e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7-xcxdb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Started pod/e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7-xcxdb Container started 30m Normal Created pod/e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7-xcxdb Container created 30m Warning Unhealthy pod/e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc-wbswg Startup probe failed: Get "http://10.132.1.30:15021/healthz/ready": dial tcp 10.132.1.30:15021: connect: connection refused 30m Warning ProbeError pod/e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc-wbswg Startup probe error: Get "http://10.132.1.30:15021/healthz/ready": dial tcp 10.132.1.30:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7-xcxdb Startup probe failed: Get "http://10.132.1.31:15021/healthz/ready": dial tcp 10.132.1.31:15021: connect: connection refused 30m Warning ProbeError pod/e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7-xcxdb Startup probe error: Get "http://10.132.1.31:15021/healthz/ready": dial tcp 10.132.1.31:15021: connect: connection refused... 30m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-963936-6c6699ff5-mktm9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Warning Unhealthy pod/payload-processing-e2e-inf-b-963936-589db57f54-5m29v Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Normal SuccessfulDelete replicaset/e2e-worker-w3-5187e1-openshift-default-946849bdf Deleted pod: e2e-worker-w3-5187e1-openshift-default-946849bdf-cflp5 30m Normal Killing pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-cflp5 Stopping container istio-proxy 30m Normal ScalingReplicaSet deployment/e2e-worker-w3-5187e1-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w3-5187e1-openshift-default-946849bdf from 2 to 1 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-5187e1-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 30m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7df854bf67 from 4 to 3 30m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7df854bf67 Deleted pod: maas-default-gateway-openshift-default-7df854bf67-2cwvd 30m Normal Killing pod/maas-default-gateway-openshift-default-7df854bf67-2cwvd Stopping container istio-proxy 30m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: All metrics below target 30m Warning Unhealthy pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-cflp5 Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-2cwvd Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning ProbeError pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-cflp5 Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-2cwvd Readiness probe error: HTTP probe failed with statuscode: 503... 30m Normal Killing pod/e2e-inf-b-963936-openshift-default-68b65f857c-phdvv Stopping container istio-proxy 30m Normal AddedLabel gateway/e2e-ipp-a-68410c Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-68410c 30m Normal ScalingReplicaSet deployment/e2e-ipp-a-68410c-openshift-default Scaled up replica set e2e-ipp-a-68410c-openshift-default-7848d9f579 from 0 to 1 30m Normal SuccessfulCreate replicaset/e2e-ipp-a-68410c-openshift-default-7848d9f579 Created pod: e2e-ipp-a-68410c-openshift-default-7848d9f579-cqpd4 30m Normal AddedInterface pod/e2e-ipp-a-68410c-openshift-default-7848d9f579-cqpd4 Add eth0 [10.134.0.186/23] from ovn-kubernetes 30m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-68410c Scaled up replica set payload-pre-processing-e2e-ipp-a-68410c-76b7f58695 from 0 to 1 30m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-68410c-76b7f58695 Created pod: payload-pre-processing-e2e-ipp-a-68410c-76b7f58695-w8gpk 30m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-68410c Scaled up replica set payload-processing-e2e-ipp-a-68410c-7f86bfb976 from 0 to 1 30m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-68410c-76b7f58695 Error creating: pods "payload-pre-processing-e2e-ipp-a-68410c-76b7f58695-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-68410c: serviceaccount "payload-processing-e2e-ipp-a-68410c" not found 30m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-68410c-7f86bfb976 Error creating: pods "payload-processing-e2e-ipp-a-68410c-7f86bfb976-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-68410c: serviceaccount "payload-processing-e2e-ipp-a-68410c" not found 30m Normal AddedInterface pod/payload-processing-e2e-ipp-a-68410c-7f86bfb976-d9z7w Add eth0 [10.132.1.33/23] from ovn-kubernetes 30m Normal Started pod/payload-processing-e2e-ipp-a-68410c-7f86bfb976-d9z7w Container started 30m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-68410c-76b7f58695-w8gpk Add eth0 [10.132.1.32/23] from ovn-kubernetes 30m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-68410c-7f86bfb976 Created pod: payload-processing-e2e-ipp-a-68410c-7f86bfb976-d9z7w 30m Normal Pulled pod/payload-processing-e2e-ipp-a-68410c-7f86bfb976-d9z7w Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal Started pod/payload-pre-processing-e2e-ipp-a-68410c-76b7f58695-w8gpk Container started 30m Normal Created pod/payload-processing-e2e-ipp-a-68410c-7f86bfb976-d9z7w Container created 30m Normal Created pod/payload-pre-processing-e2e-ipp-a-68410c-76b7f58695-w8gpk Container created 30m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-68410c-76b7f58695-w8gpk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal ScalingReplicaSet deployment/e2e-ipp-a-3920cb-openshift-default Scaled up replica set e2e-ipp-a-3920cb-openshift-default-545b848d77 from 0 to 1 30m Normal SuccessfulCreate replicaset/e2e-ipp-a-3920cb-openshift-default-545b848d77 Created pod: e2e-ipp-a-3920cb-openshift-default-545b848d77-vkcv4 30m Normal AddedLabel gateway/e2e-ipp-a-3920cb Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-3920cb 30m Normal AddedInterface pod/e2e-ipp-a-3920cb-openshift-default-545b848d77-vkcv4 Add eth0 [10.132.1.34/23] from ovn-kubernetes 29m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-3920cb-65745db9dc-czdrn Add eth0 [10.133.0.152/23] from ovn-kubernetes 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-3920cb Scaled up replica set payload-processing-e2e-ipp-a-3920cb-675b9c8b6b from 0 to 1 29m Normal AddedInterface pod/payload-processing-e2e-ipp-a-3920cb-675b9c8b6b-f2xtg Add eth0 [10.133.0.151/23] from ovn-kubernetes 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-3920cb Scaled up replica set payload-pre-processing-e2e-ipp-a-3920cb-65745db9dc from 0 to 1 29m Normal Started pod/payload-processing-e2e-ipp-a-3920cb-675b9c8b6b-f2xtg Container started 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-3920cb-65745db9dc Error creating: pods "payload-pre-processing-e2e-ipp-a-3920cb-65745db9dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-3920cb: serviceaccount "payload-processing-e2e-ipp-a-3920cb" not found 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-3920cb-675b9c8b6b Created pod: payload-processing-e2e-ipp-a-3920cb-675b9c8b6b-f2xtg 29m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-3920cb-675b9c8b6b Error creating: pods "payload-processing-e2e-ipp-a-3920cb-675b9c8b6b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-3920cb: serviceaccount "payload-processing-e2e-ipp-a-3920cb" not found 29m Normal Pulled pod/payload-processing-e2e-ipp-a-3920cb-675b9c8b6b-f2xtg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-3920cb-65745db9dc-czdrn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-3920cb-65745db9dc Created pod: payload-pre-processing-e2e-ipp-a-3920cb-65745db9dc-czdrn 29m Normal Created pod/payload-processing-e2e-ipp-a-3920cb-675b9c8b6b-f2xtg Container created 29m Normal Started pod/payload-pre-processing-e2e-ipp-a-3920cb-65745db9dc-czdrn Container started 29m Normal Created pod/payload-pre-processing-e2e-ipp-a-3920cb-65745db9dc-czdrn Container created 29m Normal Pulled pod/e2e-ipp-a-68410c-openshift-default-7848d9f579-cqpd4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Created pod/e2e-ipp-a-68410c-openshift-default-7848d9f579-cqpd4 Container created 29m Normal Started pod/e2e-ipp-a-68410c-openshift-default-7848d9f579-cqpd4 Container started 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-68410c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-68410c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Normal Killing pod/payload-processing-e2e-ipp-a-68410c-7f86bfb976-d9z7w Stopping container payload-processing 29m Normal Killing pod/payload-pre-processing-e2e-ipp-a-68410c-76b7f58695-w8gpk Stopping container payload-pre-processing 29m Warning ProbeError pod/e2e-ipp-a-68410c-openshift-default-7848d9f579-cqpd4 Startup probe error: Get "http://10.134.0.186:15021/healthz/ready": dial tcp 10.134.0.186:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-ipp-a-68410c-openshift-default-7848d9f579-cqpd4 Startup probe failed: Get "http://10.134.0.186:15021/healthz/ready": dial tcp 10.134.0.186:15021: connect: connection refused 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-68410c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-68410c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Normal Killing pod/payload-pre-processing-e2e-ipp-a-3920cb-65745db9dc-czdrn Stopping container payload-pre-processing 28m Normal Killing pod/payload-processing-e2e-ipp-a-3920cb-675b9c8b6b-f2xtg Stopping container payload-processing 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-68410c-68bc774c4c Error creating: pods "payload-pre-processing-e2e-ipp-a-68410c-68bc774c4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-68410c: serviceaccount "payload-processing-e2e-ipp-a-68410c" not found 28m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-68410c-55c49569d6 Error creating: pods "payload-processing-e2e-ipp-a-68410c-55c49569d6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-68410c: serviceaccount "payload-processing-e2e-ipp-a-68410c" not found 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-68410c Scaled up replica set payload-pre-processing-e2e-ipp-a-68410c-68bc774c4c from 0 to 1 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-68410c Scaled up replica set payload-processing-e2e-ipp-a-68410c-55c49569d6 from 0 to 1 28m Normal Pulling pod/payload-pre-processing-e2e-ipp-a-68410c-68bc774c4c-z576z Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 28m Normal Pulling pod/payload-processing-e2e-ipp-a-68410c-55c49569d6-rvsbr Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 28m Normal AddedInterface pod/payload-processing-e2e-ipp-a-68410c-55c49569d6-rvsbr Add eth0 [10.132.1.36/23] from ovn-kubernetes 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-68410c-68bc774c4c Created pod: payload-pre-processing-e2e-ipp-a-68410c-68bc774c4c-z576z 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-68410c-68bc774c4c-z576z Add eth0 [10.132.1.35/23] from ovn-kubernetes 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-68410c-55c49569d6 Created pod: payload-processing-e2e-ipp-a-68410c-55c49569d6-rvsbr 28m Normal Pulled pod/payload-processing-e2e-ipp-a-68410c-55c49569d6-rvsbr Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.719s (1.719s including waiting). Image size: 166620835 bytes. 28m Normal Created pod/payload-pre-processing-e2e-ipp-a-68410c-68bc774c4c-z576z Container created 28m Normal Started pod/payload-pre-processing-e2e-ipp-a-68410c-68bc774c4c-z576z Container started 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-68410c-68bc774c4c-z576z Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.834s (1.834s including waiting). Image size: 166620835 bytes. 28m Normal Started pod/payload-processing-e2e-ipp-a-68410c-55c49569d6-rvsbr Container started 28m Normal Created pod/payload-processing-e2e-ipp-a-68410c-55c49569d6-rvsbr Container created 28m Warning Unhealthy pod/e2e-ipp-a-3920cb-openshift-default-545b848d77-vkcv4 Readiness probe failed: Get "http://10.132.1.34:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 28m Warning ProbeError pod/e2e-ipp-a-3920cb-openshift-default-545b848d77-vkcv4 Readiness probe error: Get "http://10.132.1.34:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 28m Normal ScalingReplicaSet deployment/e2e-ipp-b-ff80c0-openshift-default Scaled up replica set e2e-ipp-b-ff80c0-openshift-default-6587d8df9f from 0 to 1 28m Normal AddedLabel gateway/e2e-ipp-b-ff80c0 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-ff80c0 28m Normal SuccessfulCreate replicaset/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f Created pod: e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-4rzs2 28m Normal AddedInterface pod/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-4rzs2 Add eth0 [10.133.0.154/23] from ovn-kubernetes 28m Normal Created pod/e2e-ipp-a-3920cb-openshift-default-545b848d77-vkcv4 Container created 28m Normal Pulled pod/e2e-ipp-a-3920cb-openshift-default-545b848d77-vkcv4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Started pod/e2e-ipp-a-3920cb-openshift-default-545b848d77-vkcv4 Container started 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-ff80c0-5d8df5b89f Error creating: pods "payload-pre-processing-e2e-ipp-b-ff80c0-5d8df5b89f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-ff80c0: serviceaccount "payload-processing-e2e-ipp-b-ff80c0" not found 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-ff80c0 Scaled up replica set payload-pre-processing-e2e-ipp-b-ff80c0-5d8df5b89f from 0 to 1 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-ff80c0-6794988c8d Created pod: payload-processing-e2e-ipp-b-ff80c0-6794988c8d-w644b 28m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-ff80c0-6794988c8d Error creating: pods "payload-processing-e2e-ipp-b-ff80c0-6794988c8d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-ff80c0: serviceaccount "payload-processing-e2e-ipp-b-ff80c0" not found 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-ff80c0 Scaled up replica set payload-processing-e2e-ipp-b-ff80c0-6794988c8d from 0 to 1 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-ff80c0-5d8df5b89f Created pod: payload-pre-processing-e2e-ipp-b-ff80c0-5d8df5b89f-cp5nc 28m Normal Started pod/payload-processing-e2e-ipp-b-ff80c0-6794988c8d-w644b Container started 28m Warning Unhealthy pod/e2e-ipp-a-3920cb-openshift-default-545b848d77-vkcv4 Startup probe failed: Get "http://10.132.1.34:15021/healthz/ready": dial tcp 10.132.1.34:15021: connect: connection refused 28m Warning ProbeError pod/e2e-ipp-a-3920cb-openshift-default-545b848d77-vkcv4 Startup probe error: Get "http://10.132.1.34:15021/healthz/ready": dial tcp 10.132.1.34:15021: connect: connection refused... 28m Normal AddedInterface pod/payload-processing-e2e-ipp-b-ff80c0-6794988c8d-w644b Add eth0 [10.133.0.155/23] from ovn-kubernetes 28m Normal Pulled pod/payload-processing-e2e-ipp-b-ff80c0-6794988c8d-w644b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Created pod/payload-processing-e2e-ipp-b-ff80c0-6794988c8d-w644b Container created 28m Normal Started pod/payload-pre-processing-e2e-ipp-b-ff80c0-5d8df5b89f-cp5nc Container started 28m Normal Created pod/payload-pre-processing-e2e-ipp-b-ff80c0-5d8df5b89f-cp5nc Container created 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-ff80c0-5d8df5b89f-cp5nc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-ff80c0-5d8df5b89f-cp5nc Add eth0 [10.132.1.39/23] from ovn-kubernetes 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-3920cb Scaled up replica set payload-pre-processing-e2e-ipp-a-3920cb-754cdc574c from 0 to 1 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-3920cb Scaled up replica set payload-processing-e2e-ipp-a-3920cb-57cdc6f756 from 0 to 1 28m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-3920cb-57cdc6f756 Error creating: pods "payload-processing-e2e-ipp-a-3920cb-57cdc6f756-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-3920cb: serviceaccount "payload-processing-e2e-ipp-a-3920cb" not found 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-3920cb-57cdc6f756 Created pod: payload-processing-e2e-ipp-a-3920cb-57cdc6f756-nl8c2 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-3920cb-754cdc574c Error creating: pods "payload-pre-processing-e2e-ipp-a-3920cb-754cdc574c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-3920cb: serviceaccount "payload-processing-e2e-ipp-a-3920cb" not found 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-3920cb-754cdc574c Created pod: payload-pre-processing-e2e-ipp-a-3920cb-754cdc574c-hk2m5 28m Normal Created pod/payload-pre-processing-e2e-ipp-a-3920cb-754cdc574c-hk2m5 Container created 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-3920cb-754cdc574c-hk2m5 Add eth0 [10.132.1.40/23] from ovn-kubernetes 28m Normal Pulled pod/payload-processing-e2e-ipp-a-3920cb-57cdc6f756-nl8c2 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 28m Normal Created pod/payload-processing-e2e-ipp-a-3920cb-57cdc6f756-nl8c2 Container created 28m Normal Started pod/payload-pre-processing-e2e-ipp-a-3920cb-754cdc574c-hk2m5 Container started 28m Normal Started pod/payload-processing-e2e-ipp-a-3920cb-57cdc6f756-nl8c2 Container started 28m Normal AddedInterface pod/payload-processing-e2e-ipp-a-3920cb-57cdc6f756-nl8c2 Add eth0 [10.132.1.41/23] from ovn-kubernetes 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-3920cb-754cdc574c-hk2m5 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-3920cb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-3920cb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Normal ScalingReplicaSet deployment/e2e-ipp-b-f48f46-openshift-default Scaled up replica set e2e-ipp-b-f48f46-openshift-default-5d6fb88f69 from 0 to 1 28m Normal AddedLabel gateway/e2e-ipp-b-f48f46 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-f48f46 28m Normal SuccessfulCreate replicaset/e2e-ipp-b-f48f46-openshift-default-5d6fb88f69 Created pod: e2e-ipp-b-f48f46-openshift-default-5d6fb88f69-l2ttk 27m Normal AddedInterface pod/e2e-ipp-b-f48f46-openshift-default-5d6fb88f69-l2ttk Add eth0 [10.134.0.189/23] from ovn-kubernetes 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-f48f46 Scaled up replica set payload-processing-e2e-ipp-b-f48f46-7c74fffd4b from 0 to 1 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-f48f46-7c74fffd4b Error creating: pods "payload-processing-e2e-ipp-b-f48f46-7c74fffd4b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-f48f46: serviceaccount "payload-processing-e2e-ipp-b-f48f46" not found 27m Normal AddedInterface pod/payload-processing-e2e-ipp-b-f48f46-7c74fffd4b-b86lk Add eth0 [10.133.0.156/23] from ovn-kubernetes 27m Normal Pulled pod/payload-processing-e2e-ipp-b-f48f46-7c74fffd4b-b86lk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal Started pod/payload-processing-e2e-ipp-b-f48f46-7c74fffd4b-b86lk Container started 27m Normal Created pod/payload-pre-processing-e2e-ipp-b-f48f46-9875d54dc-m8hbh Container created 27m Normal Started pod/payload-pre-processing-e2e-ipp-b-f48f46-9875d54dc-m8hbh Container started 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-f48f46 Scaled up replica set payload-pre-processing-e2e-ipp-b-f48f46-9875d54dc from 0 to 1 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-f48f46-9875d54dc Created pod: payload-pre-processing-e2e-ipp-b-f48f46-9875d54dc-m8hbh 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-f48f46-9875d54dc-m8hbh Add eth0 [10.133.0.157/23] from ovn-kubernetes 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-f48f46-9875d54dc Error creating: pods "payload-pre-processing-e2e-ipp-b-f48f46-9875d54dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-f48f46: serviceaccount "payload-processing-e2e-ipp-b-f48f46" not found 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-f48f46-9875d54dc-m8hbh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal Created pod/payload-processing-e2e-ipp-b-f48f46-7c74fffd4b-b86lk Container created 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-f48f46-7c74fffd4b Created pod: payload-processing-e2e-ipp-b-f48f46-7c74fffd4b-b86lk 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-3920cb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-3920cb-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 27m Normal SuccessfulCreate replicaset/e2e-ipp-a-15e7a7-openshift-default-7446d84559 Created pod: e2e-ipp-a-15e7a7-openshift-default-7446d84559-dxsld 27m Normal ScalingReplicaSet deployment/e2e-ipp-a-15e7a7-openshift-default Scaled up replica set e2e-ipp-a-15e7a7-openshift-default-7446d84559 from 0 to 1 27m Normal AddedLabel gateway/e2e-ipp-a-15e7a7 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-15e7a7 27m Normal AddedInterface pod/e2e-ipp-a-15e7a7-openshift-default-7446d84559-dxsld Add eth0 [10.132.1.42/23] from ovn-kubernetes 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-15e7a7-79b4b8dbd4 Error creating: pods "payload-processing-e2e-ipp-a-15e7a7-79b4b8dbd4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-15e7a7: serviceaccount "payload-processing-e2e-ipp-a-15e7a7" not found 27m Normal Created pod/payload-processing-e2e-ipp-a-15e7a7-79b4b8dbd4-pb92m Container created 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-15e7a7 Scaled up replica set payload-processing-e2e-ipp-a-15e7a7-79b4b8dbd4 from 0 to 1 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-15e7a7-5467566b7 Created pod: payload-pre-processing-e2e-ipp-a-15e7a7-5467566b7-npkkl 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-15e7a7-5467566b7 Error creating: pods "payload-pre-processing-e2e-ipp-a-15e7a7-5467566b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-15e7a7: serviceaccount "payload-processing-e2e-ipp-a-15e7a7" not found 27m Normal Pulled pod/payload-processing-e2e-ipp-a-15e7a7-79b4b8dbd4-pb92m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-15e7a7-5467566b7-npkkl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-15e7a7-5467566b7-npkkl Add eth0 [10.133.0.159/23] from ovn-kubernetes 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-15e7a7-79b4b8dbd4 Created pod: payload-processing-e2e-ipp-a-15e7a7-79b4b8dbd4-pb92m 27m Normal Started pod/payload-processing-e2e-ipp-a-15e7a7-79b4b8dbd4-pb92m Container started 27m Normal AddedInterface pod/payload-processing-e2e-ipp-a-15e7a7-79b4b8dbd4-pb92m Add eth0 [10.133.0.158/23] from ovn-kubernetes 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-15e7a7 Scaled up replica set payload-pre-processing-e2e-ipp-a-15e7a7-5467566b7 from 0 to 1 27m Normal Created pod/payload-pre-processing-e2e-ipp-a-15e7a7-5467566b7-npkkl Container created 27m Normal Started pod/payload-pre-processing-e2e-ipp-a-15e7a7-5467566b7-npkkl Container started 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-15e7a7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-15e7a7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Normal SuccessfulCreate replicaset/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f Created pod: e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-z2mwg 27m Normal SuccessfulCreate replicaset/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f Created pod: e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-nbfj6 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-b-ff80c0-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 27m Normal ScalingReplicaSet deployment/e2e-ipp-b-ff80c0-openshift-default Scaled up replica set e2e-ipp-b-ff80c0-openshift-default-6587d8df9f from 1 to 3 27m Normal Pulled pod/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-nbfj6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-nbfj6 Add eth0 [10.132.1.44/23] from ovn-kubernetes 27m Normal Started pod/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-z2mwg Container started 27m Normal Created pod/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-z2mwg Container created 27m Normal Pulled pod/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-z2mwg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-z2mwg Add eth0 [10.134.0.192/23] from ovn-kubernetes 27m Normal Started pod/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-nbfj6 Container started 27m Normal Created pod/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-nbfj6 Container created 27m Normal Started pod/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-4rzs2 Container started 27m Normal Created pod/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-4rzs2 Container created 27m Normal Pulled pod/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-4rzs2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Warning ProbeError pod/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-nbfj6 Startup probe error: Get "http://10.132.1.44:15021/healthz/ready": dial tcp 10.132.1.44:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-nbfj6 Startup probe failed: Get "http://10.132.1.44:15021/healthz/ready": dial tcp 10.132.1.44:15021: connect: connection refused 27m Warning Unhealthy pod/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-z2mwg Startup probe failed: Get "http://10.134.0.192:15021/healthz/ready": dial tcp 10.134.0.192:15021: connect: connection refused 27m Warning ProbeError pod/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-z2mwg Startup probe error: Get "http://10.134.0.192:15021/healthz/ready": dial tcp 10.134.0.192:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-4rzs2 Startup probe failed: Get "http://10.133.0.154:15021/healthz/ready": dial tcp 10.133.0.154:15021: connect: connection refused 27m Warning ProbeError pod/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-4rzs2 Startup probe error: Get "http://10.133.0.154:15021/healthz/ready": dial tcp 10.133.0.154:15021: connect: connection refused... 27m Normal Killing pod/payload-processing-e2e-ipp-b-ff80c0-6794988c8d-w644b Stopping container payload-processing 27m Normal Killing pod/payload-pre-processing-e2e-ipp-b-ff80c0-5d8df5b89f-cp5nc Stopping container payload-pre-processing 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-ff80c0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-ff80c0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-ff80c0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-ff80c0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Normal Started pod/e2e-ipp-b-f48f46-openshift-default-5d6fb88f69-l2ttk Container started 26m Normal Created pod/e2e-ipp-b-f48f46-openshift-default-5d6fb88f69-l2ttk Container created 26m Normal Pulled pod/e2e-ipp-b-f48f46-openshift-default-5d6fb88f69-l2ttk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Killing pod/payload-processing-e2e-ipp-b-f48f46-7c74fffd4b-b86lk Stopping container payload-processing 26m Warning ProbeError pod/e2e-ipp-b-f48f46-openshift-default-5d6fb88f69-l2ttk Startup probe error: Get "http://10.134.0.189:15021/healthz/ready": dial tcp 10.134.0.189:15021: connect: connection refused... 26m Normal Killing pod/payload-pre-processing-e2e-ipp-b-f48f46-9875d54dc-m8hbh Stopping container payload-pre-processing 26m Warning Unhealthy pod/e2e-ipp-b-f48f46-openshift-default-5d6fb88f69-l2ttk Startup probe failed: Get "http://10.134.0.189:15021/healthz/ready": dial tcp 10.134.0.189:15021: connect: connection refused 26m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-ff80c0-55ddc664f8 Error creating: pods "payload-processing-e2e-ipp-b-ff80c0-55ddc664f8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-ff80c0: serviceaccount "payload-processing-e2e-ipp-b-ff80c0" not found 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-ff80c0-55ddc664f8 Created pod: payload-processing-e2e-ipp-b-ff80c0-55ddc664f8-xc54s 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-ff80c0 Scaled up replica set payload-processing-e2e-ipp-b-ff80c0-55ddc664f8 from 0 to 1 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-ff80c0-5c7dbc8495 Created pod: payload-pre-processing-e2e-ipp-b-ff80c0-5c7dbc8495-5pnw8 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-ff80c0-5c7dbc8495 Error creating: pods "payload-pre-processing-e2e-ipp-b-ff80c0-5c7dbc8495-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-ff80c0: serviceaccount "payload-processing-e2e-ipp-b-ff80c0" not found 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-ff80c0 Scaled up replica set payload-pre-processing-e2e-ipp-b-ff80c0-5c7dbc8495 from 0 to 1 26m Normal Pulled pod/payload-processing-e2e-ipp-b-ff80c0-55ddc664f8-xc54s Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 26m Normal Created pod/payload-processing-e2e-ipp-b-ff80c0-55ddc664f8-xc54s Container created 26m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-ff80c0-5c7dbc8495-5pnw8 Add eth0 [10.132.1.52/23] from ovn-kubernetes 26m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-ff80c0-5c7dbc8495-5pnw8 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 26m Normal Started pod/payload-processing-e2e-ipp-b-ff80c0-55ddc664f8-xc54s Container started 26m Normal Started pod/payload-pre-processing-e2e-ipp-b-ff80c0-5c7dbc8495-5pnw8 Container started 26m Normal Created pod/payload-pre-processing-e2e-ipp-b-ff80c0-5c7dbc8495-5pnw8 Container created 26m Normal AddedInterface pod/payload-processing-e2e-ipp-b-ff80c0-55ddc664f8-xc54s Add eth0 [10.132.1.53/23] from ovn-kubernetes 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-f48f46-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-f48f46-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Normal Killing pod/payload-processing-e2e-ipp-a-15e7a7-79b4b8dbd4-pb92m Stopping container payload-processing 26m Normal AddedInterface pod/e2e-ipp-b-4a01f9-openshift-default-bb6d8b97f-wzwlk Add eth0 [10.134.0.193/23] from ovn-kubernetes 26m Normal Killing pod/payload-pre-processing-e2e-ipp-a-15e7a7-5467566b7-npkkl Stopping container payload-pre-processing 26m Normal SuccessfulCreate replicaset/e2e-ipp-b-4a01f9-openshift-default-bb6d8b97f Created pod: e2e-ipp-b-4a01f9-openshift-default-bb6d8b97f-wzwlk 26m Normal ScalingReplicaSet deployment/e2e-ipp-b-4a01f9-openshift-default Scaled up replica set e2e-ipp-b-4a01f9-openshift-default-bb6d8b97f from 0 to 1 26m Normal AddedLabel gateway/e2e-ipp-b-4a01f9 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-4a01f9 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-4a01f9 Scaled up replica set payload-pre-processing-e2e-ipp-b-4a01f9-6df8b8cb59 from 0 to 1 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-4a01f9-6df8b8cb59 Created pod: payload-pre-processing-e2e-ipp-b-4a01f9-6df8b8cb59-l6zlr 26m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-4a01f9-57fb584d Error creating: pods "payload-processing-e2e-ipp-b-4a01f9-57fb584d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-4a01f9: serviceaccount "payload-processing-e2e-ipp-b-4a01f9" not found 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-4a01f9-57fb584d Created pod: payload-processing-e2e-ipp-b-4a01f9-57fb584d-fn5pv 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-4a01f9 Scaled up replica set payload-processing-e2e-ipp-b-4a01f9-57fb584d from 0 to 1 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-4a01f9-6df8b8cb59 Error creating: pods "payload-pre-processing-e2e-ipp-b-4a01f9-6df8b8cb59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-4a01f9: serviceaccount "payload-processing-e2e-ipp-b-4a01f9" not found 26m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-4a01f9-6df8b8cb59-l6zlr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal Started pod/payload-pre-processing-e2e-ipp-b-4a01f9-6df8b8cb59-l6zlr Container started 26m Normal Pulled pod/payload-processing-e2e-ipp-b-4a01f9-57fb584d-fn5pv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/payload-processing-e2e-ipp-b-4a01f9-57fb584d-fn5pv Add eth0 [10.133.0.162/23] from ovn-kubernetes 26m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-4a01f9-6df8b8cb59-l6zlr Add eth0 [10.133.0.163/23] from ovn-kubernetes 26m Normal Started pod/payload-processing-e2e-ipp-b-4a01f9-57fb584d-fn5pv Container started 26m Normal Created pod/payload-pre-processing-e2e-ipp-b-4a01f9-6df8b8cb59-l6zlr Container created 26m Normal Created pod/payload-processing-e2e-ipp-b-4a01f9-57fb584d-fn5pv Container created 25m Normal AddedLabel gateway/e2e-ipp-a-a4b994 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-a4b994 25m Normal SuccessfulCreate replicaset/e2e-ipp-a-a4b994-openshift-default-556f895f89 Created pod: e2e-ipp-a-a4b994-openshift-default-556f895f89-lpg4l 25m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-56cz2 Stopping container discovery 25m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-56cz2 25m Normal ScalingReplicaSet deployment/e2e-ipp-a-a4b994-openshift-default Scaled up replica set e2e-ipp-a-a4b994-openshift-default-556f895f89 from 0 to 1 25m Normal AddedInterface pod/e2e-ipp-a-a4b994-openshift-default-556f895f89-lpg4l Add eth0 [10.134.0.194/23] from ovn-kubernetes 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-a4b994 Scaled up replica set payload-pre-processing-e2e-ipp-a-a4b994-7d779cd5bc from 0 to 1 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-a4b994 Scaled up replica set payload-processing-e2e-ipp-a-a4b994-65b8bd9f54 from 0 to 1 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-a4b994-7d779cd5bc Created pod: payload-pre-processing-e2e-ipp-a-a4b994-7d779cd5bc-sw2ht 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-88a4d9-openshift-default New size: 1; reason: All metrics below target 25m Normal ScalingReplicaSet deployment/e2e-worker-w0-88a4d9-openshift-default Scaled down replica set e2e-worker-w0-88a4d9-openshift-default-857757469c from 2 to 1 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-a4b994-65b8bd9f54 Error creating: pods "payload-processing-e2e-ipp-a-a4b994-65b8bd9f54-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-a4b994: serviceaccount "payload-processing-e2e-ipp-a-a4b994" not found 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-a4b994-65b8bd9f54 Created pod: payload-processing-e2e-ipp-a-a4b994-65b8bd9f54-jszn5 25m Normal SuccessfulDelete replicaset/e2e-worker-w0-88a4d9-openshift-default-857757469c Deleted pod: e2e-worker-w0-88a4d9-openshift-default-857757469c-shz2b 25m Normal Killing pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-shz2b Stopping container istio-proxy 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-f48f46 Scaled up replica set payload-processing-e2e-ipp-b-f48f46-7f5988976 from 0 to 1 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-a4b994-7d779cd5bc Error creating: pods "payload-pre-processing-e2e-ipp-a-a4b994-7d779cd5bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-a4b994: serviceaccount "payload-processing-e2e-ipp-a-a4b994" not found 25m Normal Pulled pod/payload-processing-e2e-ipp-a-a4b994-65b8bd9f54-jszn5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-f48f46 Scaled up replica set payload-pre-processing-e2e-ipp-b-f48f46-5bbdc5f47f from 0 to 1 25m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7df854bf67 from 3 to 2 25m Normal AddedInterface pod/payload-processing-e2e-ipp-a-a4b994-65b8bd9f54-jszn5 Add eth0 [10.132.1.73/23] from ovn-kubernetes 25m Normal Started pod/payload-pre-processing-e2e-ipp-a-a4b994-7d779cd5bc-sw2ht Container started 25m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 2; reason: All metrics below target 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-f48f46-7f5988976 Created pod: payload-processing-e2e-ipp-b-f48f46-7f5988976-2zk5f 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-f48f46-7f5988976 Error creating: pods "payload-processing-e2e-ipp-b-f48f46-7f5988976-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-f48f46: serviceaccount "payload-processing-e2e-ipp-b-f48f46" not found 25m Normal Created pod/payload-pre-processing-e2e-ipp-a-a4b994-7d779cd5bc-sw2ht Container created 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-a4b994-7d779cd5bc-sw2ht Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-a4b994-7d779cd5bc-sw2ht Add eth0 [10.132.1.72/23] from ovn-kubernetes 25m Warning Unhealthy pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-shz2b Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning ProbeError pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-shz2b Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-f48f46-5bbdc5f47f Error creating: pods "payload-pre-processing-e2e-ipp-b-f48f46-5bbdc5f47f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-f48f46: serviceaccount "payload-processing-e2e-ipp-b-f48f46" not found 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-f48f46-5bbdc5f47f Created pod: payload-pre-processing-e2e-ipp-b-f48f46-5bbdc5f47f-9x7w8 25m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7df854bf67 Deleted pod: maas-default-gateway-openshift-default-7df854bf67-r4qjb 25m Normal Killing pod/maas-default-gateway-openshift-default-7df854bf67-r4qjb Stopping container istio-proxy 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-f48f46-5bbdc5f47f-9x7w8 Add eth0 [10.132.1.74/23] from ovn-kubernetes 25m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-r4qjb Readiness probe failed: HTTP probe failed with statuscode: 503 25m Normal Started pod/payload-pre-processing-e2e-ipp-b-f48f46-5bbdc5f47f-9x7w8 Container started 25m Normal Created pod/payload-pre-processing-e2e-ipp-b-f48f46-5bbdc5f47f-9x7w8 Container created 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-f48f46-5bbdc5f47f-9x7w8 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal Started pod/payload-processing-e2e-ipp-a-a4b994-65b8bd9f54-jszn5 Container started 25m Normal AddedInterface pod/payload-processing-e2e-ipp-b-f48f46-7f5988976-2zk5f Add eth0 [10.132.1.75/23] from ovn-kubernetes 25m Normal Created pod/payload-processing-e2e-ipp-b-f48f46-7f5988976-2zk5f Container created 25m Normal Pulled pod/payload-processing-e2e-ipp-b-f48f46-7f5988976-2zk5f Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal Started pod/payload-processing-e2e-ipp-b-f48f46-7f5988976-2zk5f Container started 25m Normal Created pod/payload-processing-e2e-ipp-a-a4b994-65b8bd9f54-jszn5 Container created 25m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-r4qjb Readiness probe error: HTTP probe failed with statuscode: 503... 25m Normal Created pod/e2e-ipp-a-15e7a7-openshift-default-7446d84559-dxsld Container created 25m Normal Started pod/e2e-ipp-a-15e7a7-openshift-default-7446d84559-dxsld Container started 25m Normal Pulled pod/e2e-ipp-a-15e7a7-openshift-default-7446d84559-dxsld Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c Created pod: e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-4rhzm 25m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-a940d5-openshift-default Scaled up replica set e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c from 1 to 2 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-a940d5-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 25m Warning ProbeError pod/e2e-ipp-a-15e7a7-openshift-default-7446d84559-dxsld Startup probe error: Get "http://10.132.1.42:15021/healthz/ready": dial tcp 10.132.1.42:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-ipp-a-15e7a7-openshift-default-7446d84559-dxsld Startup probe failed: Get "http://10.132.1.42:15021/healthz/ready": dial tcp 10.132.1.42:15021: connect: connection refused 25m Normal AddedInterface pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-4rhzm Add eth0 [10.133.0.174/23] from ovn-kubernetes 25m Normal Pulled pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-4rhzm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Created pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-4rhzm Container created 25m Normal Started pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-4rhzm Container started 25m Warning Unhealthy pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-4rhzm Startup probe failed: Get "http://10.133.0.174:15021/healthz/ready": dial tcp 10.133.0.174:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-4rhzm Startup probe error: Get "http://10.133.0.174:15021/healthz/ready": dial tcp 10.133.0.174:15021: connect: connection refused... 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-15e7a7 Scaled up replica set payload-processing-e2e-ipp-a-15e7a7-5f4f86db5d from 0 to 1 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-15e7a7 Scaled up replica set payload-pre-processing-e2e-ipp-a-15e7a7-dd597bff5 from 0 to 1 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-15e7a7-dd597bff5 Error creating: pods "payload-pre-processing-e2e-ipp-a-15e7a7-dd597bff5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-15e7a7: serviceaccount "payload-processing-e2e-ipp-a-15e7a7" not found 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-15e7a7-dd597bff5-j8vmx Add eth0 [10.132.1.78/23] from ovn-kubernetes 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-15e7a7-5f4f86db5d Error creating: pods "payload-processing-e2e-ipp-a-15e7a7-5f4f86db5d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-15e7a7: serviceaccount "payload-processing-e2e-ipp-a-15e7a7" not found 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-15e7a7-5f4f86db5d Created pod: payload-processing-e2e-ipp-a-15e7a7-5f4f86db5d-mnf4m 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-15e7a7-dd597bff5 Created pod: payload-pre-processing-e2e-ipp-a-15e7a7-dd597bff5-j8vmx 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-15e7a7-dd597bff5-j8vmx Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal Pulled pod/payload-processing-e2e-ipp-a-15e7a7-5f4f86db5d-mnf4m Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal Created pod/payload-processing-e2e-ipp-a-15e7a7-5f4f86db5d-mnf4m Container created 25m Normal AddedInterface pod/payload-processing-e2e-ipp-a-15e7a7-5f4f86db5d-mnf4m Add eth0 [10.132.1.79/23] from ovn-kubernetes 25m Normal Started pod/payload-processing-e2e-ipp-a-15e7a7-5f4f86db5d-mnf4m Container started 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-0847aa-openshift-default Scaled up replica set e2e-shared-a-w0-0847aa-openshift-default-75d5774789 from 1 to 2 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-0847aa-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 25m Normal Created pod/payload-pre-processing-e2e-ipp-a-15e7a7-dd597bff5-j8vmx Container created 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-0847aa-openshift-default-75d5774789 Created pod: e2e-shared-a-w0-0847aa-openshift-default-75d5774789-pt6sj 25m Normal Started pod/payload-pre-processing-e2e-ipp-a-15e7a7-dd597bff5-j8vmx Container started 25m Normal AddedInterface pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-pt6sj Add eth0 [10.134.0.196/23] from ovn-kubernetes 25m Normal Started pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-pt6sj Container started 25m Normal Created pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-pt6sj Container created 25m Normal Pulled pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-pt6sj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Warning Unhealthy pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-pt6sj Startup probe failed: Get "http://10.134.0.196:15021/healthz/ready": dial tcp 10.134.0.196:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-pt6sj Startup probe error: Get "http://10.134.0.196:15021/healthz/ready": dial tcp 10.134.0.196:15021: connect: connection refused... 25m Normal Killing pod/payload-processing-e2e-ipp-a-68410c-55c49569d6-rvsbr Stopping container payload-processing 25m Normal Killing pod/payload-pre-processing-e2e-ipp-a-68410c-68bc774c4c-z576z Stopping container payload-pre-processing 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-15e7a7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-15e7a7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Normal Killing pod/e2e-ipp-a-68410c-openshift-default-7848d9f579-cqpd4 Stopping container istio-proxy 24m Normal Killing pod/payload-pre-processing-e2e-ipp-b-ff80c0-5c7dbc8495-5pnw8 Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-ipp-b-ff80c0-55ddc664f8-xc54s Stopping container payload-processing 24m Normal Killing pod/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-z2mwg Stopping container istio-proxy 24m Normal Killing pod/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-4rzs2 Stopping container istio-proxy 24m Normal Killing pod/e2e-ipp-b-ff80c0-openshift-default-6587d8df9f-nbfj6 Stopping container istio-proxy 24m Normal Killing pod/payload-pre-processing-e2e-worker-w1-e19850-5898685ff-kd5l5 Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-worker-w1-e19850-565c99cd8d-qhx44 Stopping container payload-processing 24m Normal Killing pod/payload-processing-e2e-ipp-b-4a01f9-57fb584d-fn5pv Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-ipp-b-4a01f9-6df8b8cb59-l6zlr Stopping container payload-pre-processing 24m Normal Pulled pod/e2e-ipp-b-4a01f9-openshift-default-bb6d8b97f-wzwlk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal AddedLabel gateway/e2e-ipp-a-4ad693 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-4ad693 24m Normal Started pod/e2e-ipp-a-a4b994-openshift-default-556f895f89-lpg4l Container started 24m Normal Created pod/e2e-ipp-a-a4b994-openshift-default-556f895f89-lpg4l Container created 24m Normal ScalingReplicaSet deployment/e2e-ipp-a-4ad693-openshift-default Scaled up replica set e2e-ipp-a-4ad693-openshift-default-c44fc7bbf from 0 to 1 24m Normal SuccessfulCreate replicaset/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf Created pod: e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-224cq 24m Normal Created pod/e2e-ipp-b-4a01f9-openshift-default-bb6d8b97f-wzwlk Container created 24m Normal Pulled pod/e2e-ipp-a-a4b994-openshift-default-556f895f89-lpg4l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Started pod/e2e-ipp-b-4a01f9-openshift-default-bb6d8b97f-wzwlk Container started 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-a4b994-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-a4b994-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Normal AddedInterface pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-224cq Add eth0 [10.134.0.197/23] from ovn-kubernetes 24m Warning Unhealthy pod/e2e-ipp-a-a4b994-openshift-default-556f895f89-lpg4l Startup probe failed: Get "http://10.134.0.194:15021/healthz/ready": dial tcp 10.134.0.194:15021: connect: connection refused 24m Warning ProbeError pod/e2e-ipp-a-a4b994-openshift-default-556f895f89-lpg4l Startup probe error: Get "http://10.134.0.194:15021/healthz/ready": dial tcp 10.134.0.194:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-ipp-b-4a01f9-openshift-default-bb6d8b97f-wzwlk Startup probe failed: Get "http://10.134.0.193:15021/healthz/ready": dial tcp 10.134.0.193:15021: connect: connection refused 24m Warning ProbeError pod/e2e-ipp-b-4a01f9-openshift-default-bb6d8b97f-wzwlk Startup probe error: Get "http://10.134.0.193:15021/healthz/ready": dial tcp 10.134.0.193:15021: connect: connection refused... 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-4ad693-5cfb5979 Error creating: pods "payload-pre-processing-e2e-ipp-a-4ad693-5cfb5979-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-4ad693: serviceaccount "payload-processing-e2e-ipp-a-4ad693" not found 24m Normal Killing pod/payload-processing-e2e-ipp-a-a4b994-65b8bd9f54-jszn5 Stopping container payload-processing 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-4ad693 Scaled up replica set payload-processing-e2e-ipp-a-4ad693-5cb9c56db6 from 0 to 1 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-4ad693-5cb9c56db6 Created pod: payload-processing-e2e-ipp-a-4ad693-5cb9c56db6-kvcc7 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-4ad693-5cb9c56db6 Error creating: pods "payload-processing-e2e-ipp-a-4ad693-5cb9c56db6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-4ad693: serviceaccount "payload-processing-e2e-ipp-a-4ad693" not found 24m Normal Killing pod/payload-pre-processing-e2e-ipp-a-a4b994-7d779cd5bc-sw2ht Stopping container payload-pre-processing 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-4ad693 Scaled up replica set payload-pre-processing-e2e-ipp-a-4ad693-5cfb5979 from 0 to 1 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-4ad693-5cfb5979 Created pod: payload-pre-processing-e2e-ipp-a-4ad693-5cfb5979-pq6rs 24m Normal Created pod/payload-processing-e2e-ipp-a-4ad693-5cb9c56db6-kvcc7 Container created 24m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-4ad693-5cfb5979-pq6rs Add eth0 [10.132.1.87/23] from ovn-kubernetes 24m Normal Pulled pod/payload-processing-e2e-ipp-a-4ad693-5cb9c56db6-kvcc7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal Started pod/payload-pre-processing-e2e-ipp-a-4ad693-5cfb5979-pq6rs Container started 24m Normal AddedInterface pod/payload-processing-e2e-ipp-a-4ad693-5cb9c56db6-kvcc7 Add eth0 [10.132.1.88/23] from ovn-kubernetes 24m Normal Started pod/payload-processing-e2e-ipp-a-4ad693-5cb9c56db6-kvcc7 Container started 24m Normal AddedLabel gateway/e2e-ipp-b-162d7b Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-162d7b 24m Normal Created pod/payload-pre-processing-e2e-ipp-a-4ad693-5cfb5979-pq6rs Container created 24m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-4ad693-5cfb5979-pq6rs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal SuccessfulCreate replicaset/e2e-ipp-b-162d7b-openshift-default-5d877b48fc Created pod: e2e-ipp-b-162d7b-openshift-default-5d877b48fc-l8prt 24m Normal ScalingReplicaSet deployment/e2e-ipp-b-162d7b-openshift-default Scaled up replica set e2e-ipp-b-162d7b-openshift-default-5d877b48fc from 0 to 1 24m Normal AddedInterface pod/e2e-ipp-b-162d7b-openshift-default-5d877b48fc-l8prt Add eth0 [10.132.1.90/23] from ovn-kubernetes 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-162d7b-c8d776ff9 Error creating: pods "payload-pre-processing-e2e-ipp-b-162d7b-c8d776ff9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-162d7b: serviceaccount "payload-processing-e2e-ipp-b-162d7b" not found 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-162d7b-c8d776ff9 Created pod: payload-pre-processing-e2e-ipp-b-162d7b-c8d776ff9-cswcp 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-162d7b-d6c56f5db Error creating: pods "payload-processing-e2e-ipp-b-162d7b-d6c56f5db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-162d7b: serviceaccount "payload-processing-e2e-ipp-b-162d7b" not found 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-162d7b Scaled up replica set payload-pre-processing-e2e-ipp-b-162d7b-c8d776ff9 from 0 to 1 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-162d7b-d6c56f5db Created pod: payload-processing-e2e-ipp-b-162d7b-d6c56f5db-nggjh 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-162d7b Scaled up replica set payload-processing-e2e-ipp-b-162d7b-d6c56f5db from 0 to 1 24m Normal AddedInterface pod/payload-processing-e2e-ipp-b-162d7b-d6c56f5db-nggjh Add eth0 [10.132.1.92/23] from ovn-kubernetes 24m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-162d7b-c8d776ff9-cswcp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal Started pod/payload-pre-processing-e2e-ipp-b-162d7b-c8d776ff9-cswcp Container started 24m Normal Pulled pod/payload-processing-e2e-ipp-b-162d7b-d6c56f5db-nggjh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal Created pod/payload-processing-e2e-ipp-b-162d7b-d6c56f5db-nggjh Container created 24m Normal Started pod/payload-processing-e2e-ipp-b-162d7b-d6c56f5db-nggjh Container started 24m Normal Created pod/payload-pre-processing-e2e-ipp-b-162d7b-c8d776ff9-cswcp Container created 24m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-162d7b-c8d776ff9-cswcp Add eth0 [10.132.1.91/23] from ovn-kubernetes 24m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-mzm2m 24m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-mzm2m Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 24m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 24m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-mzm2m Container created 24m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-mzm2m Add eth0 [10.132.1.93/23] from ovn-kubernetes 24m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-mzm2m Container started 24m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-4a01f9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-4a01f9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-mzm2m Readiness probe failed: HTTP probe failed with statuscode: 503 24m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-mzm2m Readiness probe error: HTTP probe failed with statuscode: 503... 24m Normal Killing pod/payload-processing-e2e-ipp-a-15e7a7-5f4f86db5d-mnf4m Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-ipp-a-15e7a7-dd597bff5-j8vmx Stopping container payload-pre-processing 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-a4b994-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-a4b994-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-4a01f9 Scaled up replica set payload-processing-e2e-ipp-b-4a01f9-f7d5dd8f4 from 0 to 1 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-4a01f9 Scaled up replica set payload-pre-processing-e2e-ipp-b-4a01f9-cfdc97c95 from 0 to 1 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-4a01f9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-4a01f9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Normal Killing pod/e2e-ipp-a-15e7a7-openshift-default-7446d84559-dxsld Stopping container istio-proxy 23m Warning Unhealthy pod/e2e-ipp-a-15e7a7-openshift-default-7446d84559-dxsld Readiness probe failed: HTTP probe failed with statuscode: 503 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-4a01f9-cfdc97c95 Error creating: pods "payload-pre-processing-e2e-ipp-b-4a01f9-cfdc97c95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-4a01f9: serviceaccount "payload-processing-e2e-ipp-b-4a01f9" not found 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-4a01f9-cfdc97c95 Created pod: payload-pre-processing-e2e-ipp-b-4a01f9-cfdc97c95-pfc48 23m Normal AddedInterface pod/payload-processing-e2e-ipp-b-4a01f9-f7d5dd8f4-kzgmd Add eth0 [10.133.0.178/23] from ovn-kubernetes 23m Warning ProbeError pod/e2e-ipp-a-15e7a7-openshift-default-7446d84559-dxsld Readiness probe error: HTTP probe failed with statuscode: 503... 23m Normal Pulling pod/payload-processing-e2e-ipp-b-4a01f9-f7d5dd8f4-kzgmd Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-4a01f9-f7d5dd8f4 Created pod: payload-processing-e2e-ipp-b-4a01f9-f7d5dd8f4-kzgmd 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-4a01f9-f7d5dd8f4 Error creating: pods "payload-processing-e2e-ipp-b-4a01f9-f7d5dd8f4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-4a01f9: serviceaccount "payload-processing-e2e-ipp-b-4a01f9" not found 23m Normal Pulling pod/payload-pre-processing-e2e-ipp-b-4a01f9-cfdc97c95-pfc48 Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-4a01f9-cfdc97c95-pfc48 Add eth0 [10.134.0.198/23] from ovn-kubernetes 23m Normal Pulled pod/payload-processing-e2e-ipp-b-4a01f9-f7d5dd8f4-kzgmd Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.117s (1.117s including waiting). Image size: 166620835 bytes. 23m Normal Created pod/payload-processing-e2e-ipp-b-4a01f9-f7d5dd8f4-kzgmd Container created 23m Normal Started pod/payload-processing-e2e-ipp-b-4a01f9-f7d5dd8f4-kzgmd Container started 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-4a01f9-cfdc97c95-pfc48 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.943s (1.943s including waiting). Image size: 166620835 bytes. 23m Normal Created pod/payload-pre-processing-e2e-ipp-b-4a01f9-cfdc97c95-pfc48 Container created 23m Normal Started pod/payload-pre-processing-e2e-ipp-b-4a01f9-cfdc97c95-pfc48 Container started 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-a4b994-696c454488 Error creating: pods "payload-processing-e2e-ipp-a-a4b994-696c454488-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-a4b994: serviceaccount "payload-processing-e2e-ipp-a-a4b994" not found 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-a4b994-f6b9d5df9 Error creating: pods "payload-pre-processing-e2e-ipp-a-a4b994-f6b9d5df9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-a4b994: serviceaccount "payload-processing-e2e-ipp-a-a4b994" not found 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-a4b994 Scaled up replica set payload-pre-processing-e2e-ipp-a-a4b994-f6b9d5df9 from 0 to 1 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-a4b994 Scaled up replica set payload-processing-e2e-ipp-a-a4b994-696c454488 from 0 to 1 23m Normal Created pod/payload-pre-processing-e2e-ipp-a-a4b994-f6b9d5df9-54k5k Container created 23m Normal Started pod/payload-pre-processing-e2e-ipp-a-a4b994-f6b9d5df9-54k5k Container started 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-a4b994-696c454488 Created pod: payload-processing-e2e-ipp-a-a4b994-696c454488-hc66w 23m Normal Pulled pod/payload-processing-e2e-ipp-a-a4b994-696c454488-hc66w Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-a4b994-f6b9d5df9 Created pod: payload-pre-processing-e2e-ipp-a-a4b994-f6b9d5df9-54k5k 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-a4b994-f6b9d5df9-54k5k Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-a4b994-f6b9d5df9-54k5k Add eth0 [10.132.1.97/23] from ovn-kubernetes 23m Normal AddedInterface pod/payload-processing-e2e-ipp-a-a4b994-696c454488-hc66w Add eth0 [10.134.0.200/23] from ovn-kubernetes 23m Normal Started pod/payload-processing-e2e-ipp-a-a4b994-696c454488-hc66w Container started 23m Normal Created pod/payload-processing-e2e-ipp-a-a4b994-696c454488-hc66w Container created 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-162d7b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-162d7b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Normal SuccessfulCreate replicaset/e2e-ipp-a-3920cb-openshift-default-545b848d77 Created pod: e2e-ipp-a-3920cb-openshift-default-545b848d77-5x4gg 23m Normal ScalingReplicaSet deployment/e2e-ipp-a-3920cb-openshift-default Scaled up replica set e2e-ipp-a-3920cb-openshift-default-545b848d77 from 1 to 2 23m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-a-3920cb-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 23m Normal AddedInterface pod/e2e-ipp-a-3920cb-openshift-default-545b848d77-5x4gg Add eth0 [10.134.0.201/23] from ovn-kubernetes 23m Normal Started pod/e2e-ipp-a-3920cb-openshift-default-545b848d77-5x4gg Container started 23m Normal Created pod/e2e-ipp-a-3920cb-openshift-default-545b848d77-5x4gg Container created 23m Normal Pulled pod/e2e-ipp-a-3920cb-openshift-default-545b848d77-5x4gg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Warning ProbeError pod/e2e-ipp-a-3920cb-openshift-default-545b848d77-5x4gg Startup probe error: Get "http://10.134.0.201:15021/healthz/ready": dial tcp 10.134.0.201:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-ipp-a-3920cb-openshift-default-545b848d77-5x4gg Startup probe failed: Get "http://10.134.0.201:15021/healthz/ready": dial tcp 10.134.0.201:15021: connect: connection refused 23m Normal Killing pod/payload-processing-e2e-ipp-b-4a01f9-f7d5dd8f4-kzgmd Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-ipp-b-4a01f9-cfdc97c95-pfc48 Stopping container payload-pre-processing 23m Normal Killing pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-4kq96 Stopping container istio-proxy 23m Warning ProbeError pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-4kq96 Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning Unhealthy pod/e2e-worker-w1-e19850-openshift-default-9f855bd47-4kq96 Readiness probe failed: HTTP probe failed with statuscode: 503 23m Normal Killing pod/e2e-ipp-b-4a01f9-openshift-default-bb6d8b97f-wzwlk Stopping container istio-proxy 23m Warning ProbeError pod/e2e-ipp-b-4a01f9-openshift-default-bb6d8b97f-wzwlk Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning Unhealthy pod/e2e-ipp-b-4a01f9-openshift-default-bb6d8b97f-wzwlk Readiness probe failed: HTTP probe failed with statuscode: 503 22m Normal Killing pod/payload-processing-e2e-ipp-a-4ad693-5cb9c56db6-kvcc7 Stopping container payload-processing 22m Normal Killing pod/payload-pre-processing-e2e-ipp-a-4ad693-5cfb5979-pq6rs Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-shared-a-w2-d4b6bb-75d4df64b8-nvlff Stopping container payload-processing 22m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-d4b6bb-7678b5f4f9-d68cc Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-ipp-b-162d7b-d6c56f5db-nggjh Stopping container payload-processing 22m Normal Killing pod/payload-pre-processing-e2e-ipp-b-162d7b-c8d776ff9-cswcp Stopping container payload-pre-processing 22m Warning Unhealthy pod/e2e-ipp-b-162d7b-openshift-default-5d877b48fc-l8prt Readiness probe failed: Get "http://10.132.1.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 22m Warning ProbeError pod/e2e-ipp-b-162d7b-openshift-default-5d877b48fc-l8prt Readiness probe error: Get "http://10.132.1.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 22m Normal Pulled pod/e2e-ipp-b-162d7b-openshift-default-5d877b48fc-l8prt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Created pod/e2e-ipp-b-162d7b-openshift-default-5d877b48fc-l8prt Container created 22m Normal Started pod/e2e-ipp-b-162d7b-openshift-default-5d877b48fc-l8prt Container started 22m Warning ProbeError pod/e2e-ipp-b-162d7b-openshift-default-5d877b48fc-l8prt Startup probe error: Get "http://10.132.1.90:15021/healthz/ready": dial tcp 10.132.1.90:15021: connect: connection refused... 22m Normal Killing pod/payload-processing-e2e-ipp-a-3920cb-57cdc6f756-nl8c2 Stopping container payload-processing 22m Warning Unhealthy pod/e2e-ipp-b-162d7b-openshift-default-5d877b48fc-l8prt Startup probe failed: Get "http://10.132.1.90:15021/healthz/ready": dial tcp 10.132.1.90:15021: connect: connection refused 22m Normal Killing pod/payload-pre-processing-e2e-ipp-a-3920cb-754cdc574c-hk2m5 Stopping container payload-pre-processing 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-4ad693-589cf5d8f6 Created pod: payload-processing-e2e-ipp-a-4ad693-589cf5d8f6-lgfqh 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-4ad693-589cf5d8f6 Error creating: pods "payload-processing-e2e-ipp-a-4ad693-589cf5d8f6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-4ad693: serviceaccount "payload-processing-e2e-ipp-a-4ad693" not found 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-4ad693 Scaled up replica set payload-processing-e2e-ipp-a-4ad693-589cf5d8f6 from 0 to 1 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-4ad693-5685dfd7bc Created pod: payload-pre-processing-e2e-ipp-a-4ad693-5685dfd7bc-vbh2d 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-4ad693-5685dfd7bc Error creating: pods "payload-pre-processing-e2e-ipp-a-4ad693-5685dfd7bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-4ad693: serviceaccount "payload-processing-e2e-ipp-a-4ad693" not found 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-4ad693 Scaled up replica set payload-pre-processing-e2e-ipp-a-4ad693-5685dfd7bc from 0 to 1 22m Normal AddedInterface pod/payload-processing-e2e-ipp-a-4ad693-589cf5d8f6-lgfqh Add eth0 [10.133.0.181/23] from ovn-kubernetes 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-4ad693-5685dfd7bc-vbh2d Add eth0 [10.133.0.180/23] from ovn-kubernetes 22m Normal Pulled pod/payload-processing-e2e-ipp-a-4ad693-589cf5d8f6-lgfqh Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal Created pod/payload-processing-e2e-ipp-a-4ad693-589cf5d8f6-lgfqh Container created 22m Normal Started pod/payload-pre-processing-e2e-ipp-a-4ad693-5685dfd7bc-vbh2d Container started 22m Normal Created pod/payload-pre-processing-e2e-ipp-a-4ad693-5685dfd7bc-vbh2d Container created 22m Normal Started pod/payload-processing-e2e-ipp-a-4ad693-589cf5d8f6-lgfqh Container started 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-4ad693-5685dfd7bc-vbh2d Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-162d7b-74ddd95c5 Error creating: pods "payload-pre-processing-e2e-ipp-b-162d7b-74ddd95c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-162d7b: serviceaccount "payload-processing-e2e-ipp-b-162d7b" not found 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-162d7b-699f7476d4 Created pod: payload-processing-e2e-ipp-b-162d7b-699f7476d4-tdvfk 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-162d7b Scaled up replica set payload-pre-processing-e2e-ipp-b-162d7b-74ddd95c5 from 0 to 1 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-162d7b-699f7476d4 Error creating: pods "payload-processing-e2e-ipp-b-162d7b-699f7476d4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-162d7b: serviceaccount "payload-processing-e2e-ipp-b-162d7b" not found 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-162d7b-74ddd95c5 Created pod: payload-pre-processing-e2e-ipp-b-162d7b-74ddd95c5-rpng8 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-162d7b Scaled up replica set payload-processing-e2e-ipp-b-162d7b-699f7476d4 from 0 to 1 22m Normal ScalingReplicaSet deployment/e2e-ipp-a-4ad693-openshift-default Scaled up replica set e2e-ipp-a-4ad693-openshift-default-c44fc7bbf from 1 to 4 22m Normal Created pod/payload-pre-processing-e2e-ipp-b-162d7b-74ddd95c5-rpng8 Container created 22m Normal Started pod/payload-pre-processing-e2e-ipp-b-162d7b-74ddd95c5-rpng8 Container started 22m Normal AddedInterface pod/payload-processing-e2e-ipp-b-162d7b-699f7476d4-tdvfk Add eth0 [10.133.0.183/23] from ovn-kubernetes 22m Normal SuccessfulCreate replicaset/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf Created pod: e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-m27fm 22m Normal Pulled pod/payload-processing-e2e-ipp-b-162d7b-699f7476d4-tdvfk Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-162d7b-74ddd95c5-rpng8 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-162d7b-74ddd95c5-rpng8 Add eth0 [10.133.0.182/23] from ovn-kubernetes 22m Normal SuccessfulCreate replicaset/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf Created pod: e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-wkk84 22m Normal Pulled pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-224cq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Created pod/payload-processing-e2e-ipp-b-162d7b-699f7476d4-tdvfk Container created 22m Normal Created pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-224cq Container created 22m Normal SuccessfulCreate replicaset/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf Created pod: e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-l7fhj 22m Normal Started pod/payload-processing-e2e-ipp-b-162d7b-699f7476d4-tdvfk Container started 22m Normal Started pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-224cq Container started 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-a-4ad693-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 22m Normal AddedInterface pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-wkk84 Add eth0 [10.132.1.101/23] from ovn-kubernetes 22m Normal Created pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-m27fm Container created 22m Normal Created pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-wkk84 Container created 22m Normal Started pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-l7fhj Container started 22m Normal Created pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-l7fhj Container created 22m Normal Pulled pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-l7fhj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Pulled pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-wkk84 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-l7fhj Add eth0 [10.133.0.185/23] from ovn-kubernetes 22m Normal AddedInterface pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-m27fm Add eth0 [10.133.0.184/23] from ovn-kubernetes 22m Normal Pulled pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-m27fm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Started pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-m27fm Container started 22m Normal Started pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-wkk84 Container started 22m Normal SuccessfulCreate replicaset/e2e-ipp-b-f6dcc0-openshift-default-54f6df97 Created pod: e2e-ipp-b-f6dcc0-openshift-default-54f6df97-v8l4r 22m Normal ScalingReplicaSet deployment/e2e-ipp-b-f6dcc0-openshift-default Scaled up replica set e2e-ipp-b-f6dcc0-openshift-default-54f6df97 from 0 to 1 22m Normal AddedLabel gateway/e2e-ipp-b-f6dcc0 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-f6dcc0 22m Normal Killing pod/e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff-8l4w8 Stopping container istio-proxy 22m Normal Killing pod/e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff-b4dkg Stopping container istio-proxy 22m Normal AddedInterface pod/e2e-ipp-b-f6dcc0-openshift-default-54f6df97-v8l4r Add eth0 [10.134.0.202/23] from ovn-kubernetes 22m Warning Unhealthy pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-224cq Startup probe failed: Get "http://10.134.0.197:15021/healthz/ready": dial tcp 10.134.0.197:15021: connect: connection refused 22m Warning ProbeError pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-224cq Startup probe error: Get "http://10.134.0.197:15021/healthz/ready": dial tcp 10.134.0.197:15021: connect: connection refused... 22m Warning Unhealthy pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-l7fhj Startup probe failed: Get "http://10.133.0.185:15021/healthz/ready": dial tcp 10.133.0.185:15021: connect: connection refused 22m Warning ProbeError pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-l7fhj Startup probe error: Get "http://10.133.0.185:15021/healthz/ready": dial tcp 10.133.0.185:15021: connect: connection refused... 22m Warning Unhealthy pod/e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff-b4dkg Readiness probe failed: HTTP probe failed with statuscode: 503 22m Warning Unhealthy pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-m27fm Startup probe failed: Get "http://10.133.0.184:15021/healthz/ready": dial tcp 10.133.0.184:15021: connect: connection refused 22m Warning ProbeError pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-wkk84 Startup probe error: Get "http://10.132.1.101:15021/healthz/ready": dial tcp 10.132.1.101:15021: connect: connection refused... 22m Warning ProbeError pod/e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff-b4dkg Readiness probe error: HTTP probe failed with statuscode: 503... 22m Warning ProbeError pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-m27fm Startup probe error: Get "http://10.133.0.184:15021/healthz/ready": dial tcp 10.133.0.184:15021: connect: connection refused... 22m Warning Unhealthy pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-wkk84 Startup probe failed: Get "http://10.132.1.101:15021/healthz/ready": dial tcp 10.132.1.101:15021: connect: connection refused 22m Warning ProbeError pod/e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff-8l4w8 Readiness probe error: HTTP probe failed with statuscode: 503... 22m Warning Unhealthy pod/e2e-shared-a-w2-d4b6bb-openshift-default-788fd77bff-8l4w8 Readiness probe failed: HTTP probe failed with statuscode: 503 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-f6dcc0 Scaled up replica set payload-processing-e2e-ipp-b-f6dcc0-85d8fff6db from 0 to 1 22m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-162d7b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-f6dcc0-85d8fff6db Error creating: pods "payload-processing-e2e-ipp-b-f6dcc0-85d8fff6db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-f6dcc0: serviceaccount "payload-processing-e2e-ipp-b-f6dcc0" not found 22m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-162d7b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-f6dcc0-84b547ffbc Error creating: pods "payload-pre-processing-e2e-ipp-b-f6dcc0-84b547ffbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-f6dcc0: serviceaccount "payload-processing-e2e-ipp-b-f6dcc0" not found 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-f6dcc0 Scaled up replica set payload-pre-processing-e2e-ipp-b-f6dcc0-84b547ffbc from 0 to 1 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-f6dcc0-84b547ffbc-6v2rf Add eth0 [10.133.0.187/23] from ovn-kubernetes 22m Normal Created pod/payload-pre-processing-e2e-ipp-b-f6dcc0-84b547ffbc-6v2rf Container created 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-f6dcc0-84b547ffbc-6v2rf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-f6dcc0-85d8fff6db Created pod: payload-processing-e2e-ipp-b-f6dcc0-85d8fff6db-tvftq 22m Normal Created pod/payload-processing-e2e-ipp-b-f6dcc0-85d8fff6db-tvftq Container created 22m Normal Pulled pod/payload-processing-e2e-ipp-b-f6dcc0-85d8fff6db-tvftq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/payload-processing-e2e-ipp-b-f6dcc0-85d8fff6db-tvftq Add eth0 [10.133.0.186/23] from ovn-kubernetes 22m Normal Started pod/payload-processing-e2e-ipp-b-f6dcc0-85d8fff6db-tvftq Container started 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-f6dcc0-84b547ffbc Created pod: payload-pre-processing-e2e-ipp-b-f6dcc0-84b547ffbc-6v2rf 22m Normal Started pod/payload-pre-processing-e2e-ipp-b-f6dcc0-84b547ffbc-6v2rf Container started 21m Normal Killing pod/payload-processing-e2e-shared-b-w2-4f9bb0-58c4b77cfb-cdgpk Stopping container payload-processing 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-4ad693-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-4ad693-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-4f9bb0-6f8b69d999-g9rgk Stopping container payload-pre-processing 21m Normal Killing pod/e2e-ipp-a-3920cb-openshift-default-545b848d77-5x4gg Stopping container istio-proxy 21m Normal Killing pod/e2e-ipp-a-3920cb-openshift-default-545b848d77-vkcv4 Stopping container istio-proxy 21m Normal Killing pod/payload-processing-e2e-ipp-a-a4b994-696c454488-hc66w Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-ipp-a-a4b994-f6b9d5df9-54k5k Stopping container payload-pre-processing 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-4ad693-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-4ad693-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 21m Normal Killing pod/e2e-ipp-a-a4b994-openshift-default-556f895f89-lpg4l Stopping container istio-proxy 21m Warning ProbeError pod/e2e-ipp-a-a4b994-openshift-default-556f895f89-lpg4l Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning Unhealthy pod/e2e-ipp-a-a4b994-openshift-default-556f895f89-lpg4l Readiness probe failed: HTTP probe failed with statuscode: 503 21m Normal SuccessfulCreate replicaset/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf Created pod: e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-q767t 21m Normal SuccessfulCreate replicaset/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf Created pod: e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-2bb6s 21m Normal ScalingReplicaSet deployment/e2e-ipp-a-4ad693-openshift-default Scaled up replica set e2e-ipp-a-4ad693-openshift-default-c44fc7bbf from 4 to 8 21m Normal SuccessfulCreate replicaset/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf Created pod: e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-h5g7f 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-a-4ad693-openshift-default New size: 8; reason: All metrics below target 21m Normal SuccessfulCreate replicaset/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf Created pod: e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-j6vnf 21m Normal AddedInterface pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-j6vnf Add eth0 [10.134.0.205/23] from ovn-kubernetes 21m Normal AddedInterface pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-q767t Add eth0 [10.134.0.204/23] from ovn-kubernetes 21m Normal Created pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-j6vnf Container created 21m Normal AddedInterface pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-2bb6s Add eth0 [10.132.1.106/23] from ovn-kubernetes 21m Normal Started pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-j6vnf Container started 21m Normal Pulled pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-2bb6s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-h5g7f Add eth0 [10.133.0.188/23] from ovn-kubernetes 21m Normal Pulled pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-h5g7f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Created pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-2bb6s Container created 21m Normal Pulled pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-j6vnf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Created pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-h5g7f Container created 21m Normal Pulled pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-q767t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Created pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-q767t Container created 21m Normal Started pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-q767t Container started 21m Normal Started pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-h5g7f Container started 21m Normal Started pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-2bb6s Container started 21m Normal Killing pod/e2e-shared-b-w2-4f9bb0-openshift-default-599dcdf4f9-pwn4t Stopping container istio-proxy 21m Warning Unhealthy pod/e2e-shared-b-w2-4f9bb0-openshift-default-599dcdf4f9-pwn4t Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning ProbeError pod/e2e-shared-b-w2-4f9bb0-openshift-default-599dcdf4f9-pwn4t Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning Unhealthy pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-2bb6s Startup probe failed: Get "http://10.132.1.106:15021/healthz/ready": dial tcp 10.132.1.106:15021: connect: connection refused 21m Warning ProbeError pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-2bb6s Startup probe error: Get "http://10.132.1.106:15021/healthz/ready": dial tcp 10.132.1.106:15021: connect: connection refused... 21m Warning Unhealthy pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-h5g7f Startup probe failed: Get "http://10.133.0.188:15021/healthz/ready": dial tcp 10.133.0.188:15021: connect: connection refused 21m Warning Unhealthy pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-q767t Startup probe failed: Get "http://10.134.0.204:15021/healthz/ready": dial tcp 10.134.0.204:15021: connect: connection refused 21m Warning ProbeError pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-h5g7f Startup probe error: Get "http://10.133.0.188:15021/healthz/ready": dial tcp 10.133.0.188:15021: connect: connection refused... 21m Warning Unhealthy pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-j6vnf Startup probe failed: Get "http://10.134.0.205:15021/healthz/ready": dial tcp 10.134.0.205:15021: connect: connection refused 21m Warning ProbeError pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-q767t Startup probe error: Get "http://10.134.0.204:15021/healthz/ready": dial tcp 10.134.0.204:15021: connect: connection refused... 21m Warning ProbeError pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-j6vnf Startup probe error: Get "http://10.134.0.205:15021/healthz/ready": dial tcp 10.134.0.205:15021: connect: connection refused... 21m Normal Killing pod/payload-processing-e2e-worker-w2-553753-cf69c5f4b-q2vgq Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-worker-w2-553753-b48c58c59-ntcj2 Stopping container payload-pre-processing 21m Normal Killing pod/payload-pre-processing-e2e-ipp-b-f48f46-5bbdc5f47f-9x7w8 Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-ipp-b-162d7b-699f7476d4-tdvfk Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-ipp-b-162d7b-74ddd95c5-rpng8 Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-ipp-b-f48f46-7f5988976-2zk5f Stopping container payload-processing 20m Normal Killing pod/e2e-ipp-b-f48f46-openshift-default-5d6fb88f69-l2ttk Stopping container istio-proxy 20m Normal Killing pod/e2e-ipp-b-162d7b-openshift-default-5d877b48fc-l8prt Stopping container istio-proxy 20m Normal Created pod/e2e-ipp-b-f6dcc0-openshift-default-54f6df97-v8l4r Container created 20m Normal Pulled pod/e2e-ipp-b-f6dcc0-openshift-default-54f6df97-v8l4r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Killing pod/payload-pre-processing-e2e-ipp-b-f6dcc0-84b547ffbc-6v2rf Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-ipp-b-f6dcc0-85d8fff6db-tvftq Stopping container payload-processing 20m Normal Started pod/e2e-ipp-b-f6dcc0-openshift-default-54f6df97-v8l4r Container started 20m Warning ProbeError pod/e2e-ipp-b-162d7b-openshift-default-5d877b48fc-l8prt Readiness probe error: HTTP probe failed with statuscode: 503... 20m Warning ProbeError pod/e2e-ipp-b-f6dcc0-openshift-default-54f6df97-v8l4r Readiness probe error: Get "http://10.134.0.202:15021/healthz/ready": dial tcp 10.134.0.202:15021: connect: connection refused... 20m Warning Unhealthy pod/e2e-ipp-b-f6dcc0-openshift-default-54f6df97-v8l4r Readiness probe failed: Get "http://10.134.0.202:15021/healthz/ready": dial tcp 10.134.0.202:15021: connect: connection refused 20m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-553753-b48c58c59-ntcj2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/e2e-ipp-b-162d7b-openshift-default-5d877b48fc-l8prt Readiness probe failed: HTTP probe failed with statuscode: 503 20m Warning ProbeError pod/e2e-ipp-b-f6dcc0-openshift-default-54f6df97-v8l4r Startup probe error: Get "http://10.134.0.202:15021/healthz/ready": dial tcp 10.134.0.202:15021: connect: connection refused... 20m Warning Unhealthy pod/e2e-ipp-b-f6dcc0-openshift-default-54f6df97-v8l4r Startup probe failed: Get "http://10.134.0.202:15021/healthz/ready": dial tcp 10.134.0.202:15021: connect: connection refused 20m Normal Killing pod/payload-pre-processing-e2e-shared-a-w4-281732-99d75d6c9-xrckt Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-shared-a-w4-281732-645dfbdbf4-27m7g Stopping container payload-processing 20m Normal Killing pod/payload-processing-e2e-shared-a-w5-b023a2-f5bd44854-vjwb2 Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-shared-a-w5-b023a2-678fcf5d79-x2g2r Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-ipp-a-4ad693-589cf5d8f6-lgfqh Stopping container payload-processing 20m Warning FailedToUpdateEndpoint endpoints/payload-processing-e2e-ipp-a-4ad693 Failed to update endpoint openshift-ingress/payload-processing-e2e-ipp-a-4ad693: Operation cannot be fulfilled on endpoints "payload-processing-e2e-ipp-a-4ad693": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-processing-e2e-ipp-a-4ad693, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 7a1288d7-fefc-4e40-a19a-d1be065400e0, UID in object meta: 20m Normal Killing pod/payload-pre-processing-e2e-ipp-a-4ad693-5685dfd7bc-vbh2d Stopping container payload-pre-processing 20m Normal Killing pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-l7fhj Stopping container istio-proxy 20m Normal Killing pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-2bb6s Stopping container istio-proxy 20m Normal Killing pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-224cq Stopping container istio-proxy 20m Normal Killing pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-wkk84 Stopping container istio-proxy 20m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-f6dcc0 Scaled up replica set payload-processing-e2e-ipp-b-f6dcc0-6b5944664 from 0 to 1 20m Normal Killing pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-j6vnf Stopping container istio-proxy 20m Normal Killing pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-q767t Stopping container istio-proxy 20m Normal Killing pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-h5g7f Stopping container istio-proxy 20m Normal Killing pod/e2e-ipp-a-4ad693-openshift-default-c44fc7bbf-m27fm Stopping container istio-proxy 20m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-f6dcc0-7fc8d968b7 Created pod: payload-pre-processing-e2e-ipp-b-f6dcc0-7fc8d968b7-4fllg 20m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-f6dcc0-6b5944664 Created pod: payload-processing-e2e-ipp-b-f6dcc0-6b5944664-gzcth 20m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-f6dcc0-7fc8d968b7 Error creating: pods "payload-pre-processing-e2e-ipp-b-f6dcc0-7fc8d968b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-f6dcc0: serviceaccount "payload-processing-e2e-ipp-b-f6dcc0" not found 20m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-f6dcc0-6b5944664 Error creating: pods "payload-processing-e2e-ipp-b-f6dcc0-6b5944664-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-f6dcc0: serviceaccount "payload-processing-e2e-ipp-b-f6dcc0" not found 20m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-f6dcc0 Scaled up replica set payload-pre-processing-e2e-ipp-b-f6dcc0-7fc8d968b7 from 0 to 1 20m Normal Pulled pod/payload-processing-e2e-ipp-b-f6dcc0-6b5944664-gzcth Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-f6dcc0-7fc8d968b7-4fllg Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal Created pod/payload-processing-e2e-ipp-b-f6dcc0-6b5944664-gzcth Container created 20m Normal Started pod/payload-pre-processing-e2e-ipp-b-f6dcc0-7fc8d968b7-4fllg Container started 20m Normal Started pod/payload-processing-e2e-ipp-b-f6dcc0-6b5944664-gzcth Container started 20m Normal AddedInterface pod/payload-processing-e2e-ipp-b-f6dcc0-6b5944664-gzcth Add eth0 [10.133.0.190/23] from ovn-kubernetes 20m Normal Created pod/payload-pre-processing-e2e-ipp-b-f6dcc0-7fc8d968b7-4fllg Container created 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-f6dcc0-7fc8d968b7-4fllg Add eth0 [10.133.0.189/23] from ovn-kubernetes 20m Normal Killing pod/e2e-worker-w2-553753-openshift-default-558949869c-9k6cb Stopping container istio-proxy 20m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-f6dcc0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Normal Killing pod/e2e-shared-a-w4-281732-openshift-default-868d787ccc-qgmvd Stopping container istio-proxy 20m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-f6dcc0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Warning ProbeError pod/e2e-shared-a-w4-281732-openshift-default-868d787ccc-qgmvd Readiness probe error: HTTP probe failed with statuscode: 503... 20m Warning Unhealthy pod/e2e-shared-a-w4-281732-openshift-default-868d787ccc-qgmvd Readiness probe failed: HTTP probe failed with statuscode: 503 20m Normal Killing pod/e2e-shared-a-w5-b023a2-openshift-default-7bd456f44f-wshn9 Stopping container istio-proxy 20m Normal Killing pod/payload-processing-e2e-shared-a-w6-2a2eca-6cf9b6f4bd-9mxw7 Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-shared-a-w6-2a2eca-66ff5f6f5f-8cw8r Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-shared-b-w4-04988c-97c55b5f4-2jhm7 Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-shared-b-w4-04988c-9dcb589c9-xrtn5 Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-shared-b-w5-9b841d-5c95644856-295pk Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-ipp-b-f6dcc0-7fc8d968b7-4fllg Stopping container payload-pre-processing 20m Normal Killing pod/payload-pre-processing-e2e-shared-b-w5-9b841d-6447d9f595-qxv2f Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-ipp-b-f6dcc0-6b5944664-gzcth Stopping container payload-processing 20m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-23d281-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 20m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-23d281-openshift-default-5cb785989 Created pod: e2e-shared-a-w3-23d281-openshift-default-5cb785989-xdjl2 20m Normal AddedInterface pod/e2e-shared-a-w3-23d281-openshift-default-5cb785989-xdjl2 Add eth0 [10.133.0.191/23] from ovn-kubernetes 20m Normal Pulled pod/e2e-shared-a-w3-23d281-openshift-default-5cb785989-xdjl2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-23d281-openshift-default Scaled up replica set e2e-shared-a-w3-23d281-openshift-default-5cb785989 from 1 to 2 20m Normal Started pod/e2e-shared-a-w3-23d281-openshift-default-5cb785989-xdjl2 Container started 20m Normal Created pod/e2e-shared-a-w3-23d281-openshift-default-5cb785989-xdjl2 Container created 20m Normal Killing pod/e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc-7d49j Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-a-w6-2a2eca-openshift-default-5dd76969dc-wbswg Stopping container istio-proxy 19m Warning Unhealthy pod/e2e-shared-a-w3-23d281-openshift-default-5cb785989-xdjl2 Startup probe failed: Get "http://10.133.0.191:15021/healthz/ready": dial tcp 10.133.0.191:15021: connect: connection refused 19m Warning ProbeError pod/e2e-shared-a-w3-23d281-openshift-default-5cb785989-xdjl2 Startup probe error: Get "http://10.133.0.191:15021/healthz/ready": dial tcp 10.133.0.191:15021: connect: connection refused... 19m Normal Killing pod/e2e-ipp-b-f6dcc0-openshift-default-54f6df97-v8l4r Stopping container istio-proxy 19m Warning Unhealthy pod/payload-processing-e2e-shared-a-w6-2a2eca-6cf9b6f4bd-9mxw7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w6-2a2eca-66ff5f6f5f-8cw8r Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/payload-pre-processing-e2e-shared-b-w6-26f8ba-5b94f459cf-zrpcl Stopping container payload-pre-processing 19m Normal Killing pod/e2e-shared-b-w5-9b841d-openshift-default-797cdf6799-fnr9n Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-b-w4-04988c-openshift-default-d99f984b9-trdjb Stopping container istio-proxy 19m Normal Killing pod/payload-processing-e2e-shared-b-w6-26f8ba-6f7bb8fcbd-mkqzk Stopping container payload-processing 19m Warning ProbeError pod/e2e-shared-b-w4-04988c-openshift-default-d99f984b9-trdjb Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning Unhealthy pod/e2e-shared-b-w4-04988c-openshift-default-d99f984b9-trdjb Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning ProbeError pod/e2e-shared-b-w5-9b841d-openshift-default-797cdf6799-fnr9n Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning Unhealthy pod/e2e-shared-b-w5-9b841d-openshift-default-797cdf6799-fnr9n Readiness probe failed: HTTP probe failed with statuscode: 503 19m Normal Killing pod/payload-pre-processing-e2e-shared-a-w0-0847aa-6d5b985b7-xt8mm Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-shared-a-w0-0847aa-fc76d97d-m54ng Stopping container payload-processing 19m Normal Killing pod/payload-pre-processing-e2e-worker-w5-434ce3-7b5c6dc5b7-pgcvw Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-worker-w5-434ce3-5797fc889b-dfrzp Stopping container payload-processing 19m Normal Killing pod/payload-processing-e2e-worker-w4-50d738-85555f75b4-p55sv Stopping container payload-processing 19m Normal Killing pod/payload-pre-processing-e2e-worker-w4-50d738-5b4664c9c9-zldwf Stopping container payload-pre-processing 19m Normal Killing pod/e2e-shared-b-w6-26f8ba-openshift-default-569ccb4c87-pcd2l Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-pt6sj Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w0-0847aa-openshift-default-75d5774789-2kdhp Stopping container istio-proxy 19m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w6-26f8ba-5b94f459cf-zrpcl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-processing-e2e-shared-b-w6-26f8ba-6f7bb8fcbd-mkqzk Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/payload-pre-processing-e2e-worker-w6-2d619f-5c9fd58999-69tjr Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-worker-w6-2d619f-bdbc76d94-99hjl Stopping container payload-processing 19m Normal Killing pod/payload-processing-e2e-shared-b-w0-a940d5-64d8f95f5b-l6fqz Stopping container payload-processing 19m Normal Killing pod/payload-pre-processing-e2e-shared-b-w0-a940d5-5b5d7f6f59-nvgzv Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-shared-a-w3-23d281-97b8bbd66-7dgnl Stopping container payload-processing 19m Normal Killing pod/payload-pre-processing-e2e-shared-a-w3-23d281-8599fb6555-qtrjc Stopping container payload-pre-processing 19m Normal Killing pod/e2e-worker-w4-50d738-openshift-default-8d6476cbc-npjdw Stopping container istio-proxy 19m Normal Killing pod/e2e-worker-w5-434ce3-openshift-default-58c7774759-2nggf Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w3-23d281-openshift-default-5cb785989-xdjl2 Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w3-23d281-openshift-default-5cb785989-8jstk Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-xdg5b Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-b-w0-a940d5-openshift-default-767f4ff56c-4rhzm Stopping container istio-proxy 19m Warning Unhealthy pod/payload-processing-e2e-shared-a-w3-23d281-97b8bbd66-7dgnl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w3-23d281-8599fb6555-qtrjc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Normal Killing pod/payload-processing-e2e-shared-b-w3-d8a3d1-85f887b8c4-gfs5q Stopping container payload-processing 18m Normal Killing pod/e2e-worker-w6-2d619f-openshift-default-85b84758df-2gnhr Stopping container istio-proxy 18m Normal Killing pod/payload-pre-processing-e2e-shared-b-w3-d8a3d1-b7986f8f9-s9n6t Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-worker-w0-88a4d9-76c4574f8b-g8sjs Stopping container payload-processing 18m Normal Killing pod/payload-pre-processing-e2e-worker-w0-88a4d9-797cb8d49f-mfqtn Stopping container payload-pre-processing 18m Normal Killing pod/e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7-8q4n6 Stopping container istio-proxy 18m Normal Killing pod/e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7-xcxdb Stopping container istio-proxy 18m Warning ProbeError pod/e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7-8q4n6 Readiness probe error: HTTP probe failed with statuscode: 503... 18m Warning Unhealthy pod/e2e-shared-b-w3-d8a3d1-openshift-default-657d6f85c7-8q4n6 Readiness probe failed: HTTP probe failed with statuscode: 503 18m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w3-d8a3d1-b7986f8f9-s9n6t Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-processing-e2e-shared-b-w3-d8a3d1-85f887b8c4-gfs5q Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Normal Killing pod/payload-processing-e2e-worker-w3-5187e1-5d69cc969d-9qmml Stopping container payload-processing 18m Normal Killing pod/payload-pre-processing-e2e-worker-w3-5187e1-7968f4cdbc-klfgh Stopping container payload-pre-processing 18m Normal Killing pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-8lzn6 Stopping container istio-proxy 18m Warning Unhealthy pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-8lzn6 Readiness probe failed: HTTP probe failed with statuscode: 503 18m Warning ProbeError pod/e2e-worker-w0-88a4d9-openshift-default-857757469c-8lzn6 Readiness probe error: HTTP probe failed with statuscode: 503... 18m Normal Killing pod/e2e-worker-w3-5187e1-openshift-default-946849bdf-knhhd Stopping container istio-proxy 17m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7df854bf67 Created pod: maas-default-gateway-openshift-default-7df854bf67-w85lm 17m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 17m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7df854bf67 from 2 to 4 17m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7df854bf67 Created pod: maas-default-gateway-openshift-default-7df854bf67-fxflc 17m Normal Started pod/maas-default-gateway-openshift-default-7df854bf67-fxflc Container started 17m Normal Created pod/maas-default-gateway-openshift-default-7df854bf67-fxflc Container created 17m Normal Started pod/maas-default-gateway-openshift-default-7df854bf67-w85lm Container started 17m Normal Created pod/maas-default-gateway-openshift-default-7df854bf67-w85lm Container created 17m Normal Pulled pod/maas-default-gateway-openshift-default-7df854bf67-w85lm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal AddedInterface pod/maas-default-gateway-openshift-default-7df854bf67-w85lm Add eth0 [10.133.0.192/23] from ovn-kubernetes 17m Normal Pulled pod/maas-default-gateway-openshift-default-7df854bf67-fxflc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal AddedInterface pod/maas-default-gateway-openshift-default-7df854bf67-fxflc Add eth0 [10.133.0.193/23] from ovn-kubernetes 17m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-fxflc Startup probe failed: Get "http://10.133.0.193:15021/healthz/ready": dial tcp 10.133.0.193:15021: connect: connection refused 17m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-fxflc Startup probe error: Get "http://10.133.0.193:15021/healthz/ready": dial tcp 10.133.0.193:15021: connect: connection refused... 17m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-w85lm Startup probe failed: Get "http://10.133.0.192:15021/healthz/ready": dial tcp 10.133.0.192:15021: connect: connection refused 17m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-w85lm Startup probe error: Get "http://10.133.0.192:15021/healthz/ready": dial tcp 10.133.0.192:15021: connect: connection refused... 16m Normal Pulled pod/maas-default-gateway-openshift-default-7df854bf67-fkpql Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal AddedInterface pod/maas-default-gateway-openshift-default-7df854bf67-gvf4z Add eth0 [10.133.0.194/23] from ovn-kubernetes 16m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 16m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7df854bf67 from 4 to 8 16m Normal AddedInterface pod/maas-default-gateway-openshift-default-7df854bf67-mfkdg Add eth0 [10.134.0.213/23] from ovn-kubernetes 16m Normal Pulled pod/maas-default-gateway-openshift-default-7df854bf67-mfkdg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal Created pod/maas-default-gateway-openshift-default-7df854bf67-mfkdg Container created 16m Normal Started pod/maas-default-gateway-openshift-default-7df854bf67-mfkdg Container started 16m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7df854bf67 Created pod: maas-default-gateway-openshift-default-7df854bf67-f7pll 16m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7df854bf67 Created pod: maas-default-gateway-openshift-default-7df854bf67-mfkdg 16m Normal Started pod/maas-default-gateway-openshift-default-7df854bf67-fkpql Container started 16m Normal Created pod/maas-default-gateway-openshift-default-7df854bf67-fkpql Container created 16m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7df854bf67 Created pod: maas-default-gateway-openshift-default-7df854bf67-gvf4z 16m Normal AddedInterface pod/maas-default-gateway-openshift-default-7df854bf67-fkpql Add eth0 [10.132.1.139/23] from ovn-kubernetes 16m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7df854bf67 Created pod: maas-default-gateway-openshift-default-7df854bf67-fkpql 16m Normal Started pod/maas-default-gateway-openshift-default-7df854bf67-gvf4z Container started 16m Normal Created pod/maas-default-gateway-openshift-default-7df854bf67-gvf4z Container created 16m Normal Pulled pod/maas-default-gateway-openshift-default-7df854bf67-gvf4z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal AddedInterface pod/maas-default-gateway-openshift-default-7df854bf67-f7pll Add eth0 [10.132.1.140/23] from ovn-kubernetes 16m Normal Pulled pod/maas-default-gateway-openshift-default-7df854bf67-f7pll Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal Created pod/maas-default-gateway-openshift-default-7df854bf67-f7pll Container created 16m Normal Started pod/maas-default-gateway-openshift-default-7df854bf67-f7pll Container started 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-mfkdg Startup probe failed: Get "http://10.134.0.213:15021/healthz/ready": dial tcp 10.134.0.213:15021: connect: connection refused 16m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-mfkdg Startup probe error: Get "http://10.134.0.213:15021/healthz/ready": dial tcp 10.134.0.213:15021: connect: connection refused... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-fkpql Startup probe failed: Get "http://10.132.1.139:15021/healthz/ready": dial tcp 10.132.1.139:15021: connect: connection refused 16m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-fkpql Startup probe error: Get "http://10.132.1.139:15021/healthz/ready": dial tcp 10.132.1.139:15021: connect: connection refused... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-gvf4z Startup probe failed: Get "http://10.133.0.194:15021/healthz/ready": dial tcp 10.133.0.194:15021: connect: connection refused 16m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-gvf4z Startup probe error: Get "http://10.133.0.194:15021/healthz/ready": dial tcp 10.133.0.194:15021: connect: connection refused... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-f7pll Startup probe failed: Get "http://10.132.1.140:15021/healthz/ready": dial tcp 10.132.1.140:15021: connect: connection refused 16m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-f7pll Startup probe error: Get "http://10.132.1.140:15021/healthz/ready": dial tcp 10.132.1.140:15021: connect: connection refused... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-gvf4z Startup probe failed: Get "http://10.133.0.194:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 16m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-mfkdg Startup probe error: Get "http://10.134.0.213:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 16m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-gvf4z Startup probe error: Get "http://10.133.0.194:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-mfkdg Startup probe failed: Get "http://10.134.0.213:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 16m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-fkpql Startup probe error: Get "http://10.132.1.139:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-fkpql Startup probe failed: Get "http://10.132.1.139:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-f7pll Startup probe failed: Get "http://10.132.1.140:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 16m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-f7pll Startup probe error: Get "http://10.132.1.140:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 16m Normal Started pod/maas-default-gateway-openshift-default-7df854bf67-zfx74 Container started 16m Normal Created pod/maas-default-gateway-openshift-default-7df854bf67-zfx74 Container created 16m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7df854bf67 from 8 to 10 16m Normal AddedInterface pod/maas-default-gateway-openshift-default-7df854bf67-gprr2 Add eth0 [10.134.0.215/23] from ovn-kubernetes 16m Normal Pulled pod/maas-default-gateway-openshift-default-7df854bf67-gprr2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal Created pod/maas-default-gateway-openshift-default-7df854bf67-gprr2 Container created 16m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 16m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7df854bf67 Created pod: maas-default-gateway-openshift-default-7df854bf67-zfx74 16m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7df854bf67 Created pod: maas-default-gateway-openshift-default-7df854bf67-gprr2 16m Normal Started pod/maas-default-gateway-openshift-default-7df854bf67-gprr2 Container started 16m Normal AddedInterface pod/maas-default-gateway-openshift-default-7df854bf67-zfx74 Add eth0 [10.133.0.195/23] from ovn-kubernetes 16m Normal Pulled pod/maas-default-gateway-openshift-default-7df854bf67-zfx74 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-gprr2 Startup probe failed: Get "http://10.134.0.215:15021/healthz/ready": dial tcp 10.134.0.215:15021: connect: connection refused 15m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-zfx74 Startup probe error: Get "http://10.133.0.195:15021/healthz/ready": dial tcp 10.133.0.195:15021: connect: connection refused... 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-zfx74 Startup probe failed: Get "http://10.133.0.195:15021/healthz/ready": dial tcp 10.133.0.195:15021: connect: connection refused 15m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-gprr2 Startup probe error: Get "http://10.134.0.215:15021/healthz/ready": dial tcp 10.134.0.215:15021: connect: connection refused... 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-gprr2 Startup probe failed: Get "http://10.134.0.215:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 15m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-zfx74 Startup probe error: Get "http://10.133.0.195:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 15m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-gprr2 Startup probe error: Get "http://10.134.0.215:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-zfx74 Startup probe failed: Get "http://10.133.0.195:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 15m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 15m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-mzm2m 15m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-mzm2m Stopping container discovery 15m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 13m Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-zqt58 Readiness probe error: Get "http://10.134.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-zqt58 Readiness probe failed: Get "http://10.134.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 12m Normal AddedLabel gateway/e2e-worker-single-main-d5590b Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-d5590b 12m Normal ScalingReplicaSet deployment/e2e-worker-single-main-d5590b-openshift-default Scaled up replica set e2e-worker-single-main-d5590b-openshift-default-7bd898b6c7 from 0 to 1 12m Normal AddedInterface pod/e2e-worker-single-main-d5590b-openshift-default-7bd898b6c7rlgk4 Add eth0 [10.132.1.141/23] from ovn-kubernetes 12m Normal SuccessfulCreate replicaset/e2e-worker-single-main-d5590b-openshift-default-7bd898b6c7 Created pod: e2e-worker-single-main-d5590b-openshift-default-7bd898b6c7rlgk4 12m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-d5590b-5575b4pmf8 Add eth0 [10.132.1.142/23] from ovn-kubernetes 12m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-d5590b Scaled up replica set payload-pre-processing-e2e-worker-single-main-d5590b-5575b9459 from 0 to 1 12m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-d5590b-75b6cb5f98 Error creating: pods "payload-processing-e2e-worker-single-main-d5590b-75b6cb5f98-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-d5590b: serviceaccount "payload-processing-e2e-worker-single-main-d5590b" not found 12m Normal Started pod/payload-pre-processing-e2e-worker-single-main-d5590b-5575b4pmf8 Container started 12m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-d5590b-5575b9459 Error creating: pods "payload-pre-processing-e2e-worker-single-main-d5590b-5575b9459-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-d5590b: serviceaccount "payload-processing-e2e-worker-single-main-d5590b" not found 12m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-d5590b-5575b9459 Created pod: payload-pre-processing-e2e-worker-single-main-d5590b-5575b4pmf8 12m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-d5590b-5575b4pmf8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 12m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-d5590b-75b6cb5f98 Created pod: payload-processing-e2e-worker-single-main-d5590b-75b6cb5f9ll2ss 12m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-d5590b Scaled up replica set payload-processing-e2e-worker-single-main-d5590b-75b6cb5f98 from 0 to 1 12m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-d5590b-75b6cb5f9ll2ss Add eth0 [10.134.0.228/23] from ovn-kubernetes 12m Normal Created pod/payload-pre-processing-e2e-worker-single-main-d5590b-5575b4pmf8 Container created 12m Normal Pulled pod/payload-processing-e2e-worker-single-main-d5590b-75b6cb5f9ll2ss Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 12m Normal Created pod/payload-processing-e2e-worker-single-main-d5590b-75b6cb5f9ll2ss Container created 12m Normal Started pod/payload-processing-e2e-worker-single-main-d5590b-75b6cb5f9ll2ss Container started 11m Warning ProbeError pod/e2e-worker-single-main-d5590b-openshift-default-7bd898b6c7rlgk4 Readiness probe error: Get "http://10.132.1.141:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 11m Warning Unhealthy pod/e2e-worker-single-main-d5590b-openshift-default-7bd898b6c7rlgk4 Readiness probe failed: Get "http://10.132.1.141:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 11m Normal Started pod/e2e-worker-single-main-d5590b-openshift-default-7bd898b6c7rlgk4 Container started 11m Normal Created pod/e2e-worker-single-main-d5590b-openshift-default-7bd898b6c7rlgk4 Container created 11m Normal Pulled pod/e2e-worker-single-main-d5590b-openshift-default-7bd898b6c7rlgk4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 11m Warning ProbeError pod/e2e-worker-single-main-d5590b-openshift-default-7bd898b6c7rlgk4 Startup probe error: Get "http://10.132.1.141:15021/healthz/ready": dial tcp 10.132.1.141:15021: connect: connection refused... 11m Warning Unhealthy pod/e2e-worker-single-main-d5590b-openshift-default-7bd898b6c7rlgk4 Startup probe failed: Get "http://10.132.1.141:15021/healthz/ready": dial tcp 10.132.1.141:15021: connect: connection refused 11m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-d5590b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 11m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-d5590b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 11m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-d5590b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 11m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-d5590b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 11m Normal Killing pod/payload-processing-e2e-worker-single-main-d5590b-75b6cb5f9ll2ss Stopping container payload-processing 11m Normal Killing pod/payload-pre-processing-e2e-worker-single-main-d5590b-5575b4pmf8 Stopping container payload-pre-processing 10m Normal Killing pod/e2e-worker-single-main-d5590b-openshift-default-7bd898b6c7rlgk4 Stopping container istio-proxy 10m Warning Unhealthy pod/e2e-worker-single-main-d5590b-openshift-default-7bd898b6c7rlgk4 Readiness probe failed: HTTP probe failed with statuscode: 503 10m Warning ProbeError pod/e2e-worker-single-main-d5590b-openshift-default-7bd898b6c7rlgk4 Readiness probe error: HTTP probe failed with statuscode: 503... 10m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-d5590b-5575b4pmf8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 10m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-d5590b-75b6cb5f9ll2ss Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 8m30s Normal AddedLabel gateway/e2e-worker-single-main-22c413 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-22c413 8m30s Normal SuccessfulCreate replicaset/e2e-worker-single-main-22c413-openshift-default-685f6fd8d9 Created pod: e2e-worker-single-main-22c413-openshift-default-685f6fd8d9qs4x5 8m30s Normal ScalingReplicaSet deployment/e2e-worker-single-main-22c413-openshift-default Scaled up replica set e2e-worker-single-main-22c413-openshift-default-685f6fd8d9 from 0 to 1 8m29s Normal AddedInterface pod/e2e-worker-single-main-22c413-openshift-default-685f6fd8d9qs4x5 Add eth0 [10.132.1.147/23] from ovn-kubernetes 8m24s Normal Pulled pod/payload-processing-e2e-worker-single-main-22c413-578cb96cchrmtl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m24s Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-22c413 Scaled up replica set payload-processing-e2e-worker-single-main-22c413-578cb96cc8 from 0 to 1 8m24s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-22c413 Scaled up replica set payload-pre-processing-e2e-worker-single-main-22c413-748b657747 from 0 to 1 8m24s Normal AddedInterface pod/payload-processing-e2e-worker-single-main-22c413-578cb96cchrmtl Add eth0 [10.132.1.148/23] from ovn-kubernetes 8m24s Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-22c413-578cb96cc8 Created pod: payload-processing-e2e-worker-single-main-22c413-578cb96cchrmtl 8m24s Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-22c413-578cb96cc8 Error creating: pods "payload-processing-e2e-worker-single-main-22c413-578cb96cc8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-22c413: serviceaccount "payload-processing-e2e-worker-single-main-22c413" not found 8m24s Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-22c413-748b657747 Error creating: pods "payload-pre-processing-e2e-worker-single-main-22c413-748b657747-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-22c413: serviceaccount "payload-processing-e2e-worker-single-main-22c413" not found 8m24s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-22c413-748b657747 Created pod: payload-pre-processing-e2e-worker-single-main-22c413-748b6tg578 8m23s Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-22c413-748b6tg578 Add eth0 [10.134.0.234/23] from ovn-kubernetes 8m23s Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-22c413-748b6tg578 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m23s Normal Started pod/payload-processing-e2e-worker-single-main-22c413-578cb96cchrmtl Container started 8m23s Normal Created pod/payload-pre-processing-e2e-worker-single-main-22c413-748b6tg578 Container created 8m23s Normal Started pod/payload-pre-processing-e2e-worker-single-main-22c413-748b6tg578 Container started 8m23s Normal Created pod/payload-processing-e2e-worker-single-main-22c413-578cb96cchrmtl Container created 8m15s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-22c413-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 8m15s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-22c413-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 8m11s Warning Unhealthy pod/e2e-worker-single-main-22c413-openshift-default-685f6fd8d9qs4x5 Readiness probe failed: Get "http://10.132.1.147:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 8m11s Warning ProbeError pod/e2e-worker-single-main-22c413-openshift-default-685f6fd8d9qs4x5 Readiness probe error: Get "http://10.132.1.147:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 8m10s Normal Started pod/e2e-worker-single-main-22c413-openshift-default-685f6fd8d9qs4x5 Container started 8m10s Normal Created pod/e2e-worker-single-main-22c413-openshift-default-685f6fd8d9qs4x5 Container created 8m10s Normal Pulled pod/e2e-worker-single-main-22c413-openshift-default-685f6fd8d9qs4x5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 8m5s Warning Unhealthy pod/e2e-worker-single-main-22c413-openshift-default-685f6fd8d9qs4x5 Startup probe failed: Get "http://10.132.1.147:15021/healthz/ready": dial tcp 10.132.1.147:15021: connect: connection refused 8m5s Warning ProbeError pod/e2e-worker-single-main-22c413-openshift-default-685f6fd8d9qs4x5 Startup probe error: Get "http://10.132.1.147:15021/healthz/ready": dial tcp 10.132.1.147:15021: connect: connection refused... 7m45s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-22c413-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 7m45s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-22c413-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 7m17s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-22c413-748b6tg578 Stopping container payload-pre-processing 7m17s Normal Killing pod/payload-processing-e2e-worker-single-main-22c413-578cb96cchrmtl Stopping container payload-processing 7m11s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-22c413-748b6tg578 Readiness probe failed: timeout: failed to connect service "10.134.0.234:50052" within 1s: context deadline exceeded 7m9s Normal Killing pod/e2e-worker-single-main-22c413-openshift-default-685f6fd8d9qs4x5 Stopping container istio-proxy 6m6s Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-zfx74 Readiness probe error: Get "http://10.133.0.195:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 6m6s Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-gprr2 Readiness probe failed: Get "http://10.134.0.215:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 6m6s Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-f7pll Readiness probe failed: Get "http://10.132.1.140:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 6m6s Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-gprr2 Readiness probe error: Get "http://10.134.0.215:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 6m6s Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-f7pll Readiness probe error: Get "http://10.132.1.140:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 6m6s Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-zfx74 Readiness probe failed: Get "http://10.133.0.195:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 5m51s Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-mfkdg Readiness probe error: Get "http://10.134.0.213:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 5m51s Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-fkpql Readiness probe failed: Get "http://10.132.1.139:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 5m51s Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-fkpql Readiness probe error: Get "http://10.132.1.139:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 5m51s Warning ProbeError pod/maas-default-gateway-openshift-default-7df854bf67-gvf4z Readiness probe error: Get "http://10.133.0.194:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 5m51s Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-gvf4z Readiness probe failed: Get "http://10.133.0.194:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 5m51s Warning Unhealthy pod/maas-default-gateway-openshift-default-7df854bf67-mfkdg Readiness probe failed: Get "http://10.134.0.213:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 5m28s Normal SuccessfulCreate replicaset/e2e-ait-a6ecd9bf-openshift-default-8f5fcfb57 Created pod: e2e-ait-a6ecd9bf-openshift-default-8f5fcfb57-8b7z2 5m28s Normal ScalingReplicaSet deployment/e2e-ait-a6ecd9bf-openshift-default Scaled up replica set e2e-ait-a6ecd9bf-openshift-default-8f5fcfb57 from 0 to 1 5m28s Normal AddedLabel gateway/e2e-ait-a6ecd9bf Added label istio.io/rev=openshift-gateway to gateway e2e-ait-a6ecd9bf 5m27s Normal AddedInterface pod/e2e-ait-a6ecd9bf-openshift-default-8f5fcfb57-8b7z2 Add eth0 [10.132.1.154/23] from ovn-kubernetes 5m22s Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-a6ecd9bf-5ffcbfd4b Created pod: payload-processing-e2e-ait-a6ecd9bf-5ffcbfd4b-97clk 5m22s Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-a6ecd9bf-8bb8c667 Error creating: pods "payload-pre-processing-e2e-ait-a6ecd9bf-8bb8c667-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-a6ecd9bf: serviceaccount "payload-processing-e2e-ait-a6ecd9bf" not found 5m22s Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-a6ecd9bf Scaled up replica set payload-processing-e2e-ait-a6ecd9bf-5ffcbfd4b from 0 to 1 5m22s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-a6ecd9bf Scaled up replica set payload-pre-processing-e2e-ait-a6ecd9bf-8bb8c667 from 0 to 1 5m22s Warning FailedCreate replicaset/payload-processing-e2e-ait-a6ecd9bf-5ffcbfd4b Error creating: pods "payload-processing-e2e-ait-a6ecd9bf-5ffcbfd4b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-a6ecd9bf: serviceaccount "payload-processing-e2e-ait-a6ecd9bf" not found 5m22s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-a6ecd9bf-8bb8c667 Created pod: payload-pre-processing-e2e-ait-a6ecd9bf-8bb8c667-vfwxn 5m21s Normal Created pod/payload-processing-e2e-ait-a6ecd9bf-5ffcbfd4b-97clk Container created 5m21s Normal AddedInterface pod/payload-pre-processing-e2e-ait-a6ecd9bf-8bb8c667-vfwxn Add eth0 [10.132.1.155/23] from ovn-kubernetes 5m21s Normal Pulled pod/payload-pre-processing-e2e-ait-a6ecd9bf-8bb8c667-vfwxn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m21s Normal Created pod/payload-pre-processing-e2e-ait-a6ecd9bf-8bb8c667-vfwxn Container created 5m21s Normal Started pod/payload-pre-processing-e2e-ait-a6ecd9bf-8bb8c667-vfwxn Container started 5m21s Normal AddedInterface pod/payload-processing-e2e-ait-a6ecd9bf-5ffcbfd4b-97clk Add eth0 [10.132.1.156/23] from ovn-kubernetes 5m21s Normal Started pod/payload-processing-e2e-ait-a6ecd9bf-5ffcbfd4b-97clk Container started 5m21s Normal Pulled pod/payload-processing-e2e-ait-a6ecd9bf-5ffcbfd4b-97clk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m9s Warning Unhealthy pod/e2e-ait-a6ecd9bf-openshift-default-8f5fcfb57-8b7z2 Readiness probe failed: Get "http://10.132.1.154:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 5m9s Warning ProbeError pod/e2e-ait-a6ecd9bf-openshift-default-8f5fcfb57-8b7z2 Readiness probe error: Get "http://10.132.1.154:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 5m8s Normal Pulled pod/e2e-ait-a6ecd9bf-openshift-default-8f5fcfb57-8b7z2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 5m8s Normal Started pod/e2e-ait-a6ecd9bf-openshift-default-8f5fcfb57-8b7z2 Container started 5m8s Normal Created pod/e2e-ait-a6ecd9bf-openshift-default-8f5fcfb57-8b7z2 Container created 5m3s Warning ProbeError pod/e2e-ait-a6ecd9bf-openshift-default-8f5fcfb57-8b7z2 Startup probe error: Get "http://10.132.1.154:15021/healthz/ready": dial tcp 10.132.1.154:15021: connect: connection refused... 5m3s Warning Unhealthy pod/e2e-ait-a6ecd9bf-openshift-default-8f5fcfb57-8b7z2 Startup probe failed: Get "http://10.132.1.154:15021/healthz/ready": dial tcp 10.132.1.154:15021: connect: connection refused 4m58s Normal Killing pod/payload-pre-processing-e2e-ait-a6ecd9bf-8bb8c667-vfwxn Stopping container payload-pre-processing 4m58s Normal Killing pod/payload-processing-e2e-ait-a6ecd9bf-5ffcbfd4b-97clk Stopping container payload-processing 4m57s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-a6ecd9bf-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m57s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-a6ecd9bf-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m52s Normal Killing pod/e2e-ait-a6ecd9bf-openshift-default-8f5fcfb57-8b7z2 Stopping container istio-proxy 4m51s Normal ScalingReplicaSet deployment/e2e-authcfg-http-02e08d-openshift-default Scaled up replica set e2e-authcfg-http-02e08d-openshift-default-8d4777cb5 from 0 to 1 4m51s Normal AddedLabel gateway/e2e-authcfg-http-02e08d Added label istio.io/rev=openshift-gateway to gateway e2e-authcfg-http-02e08d 4m51s Normal SuccessfulCreate replicaset/e2e-authcfg-http-02e08d-openshift-default-8d4777cb5 Created pod: e2e-authcfg-http-02e08d-openshift-default-8d4777cb5-mdkrz 4m50s Normal AddedInterface pod/e2e-authcfg-http-02e08d-openshift-default-8d4777cb5-mdkrz Add eth0 [10.132.1.158/23] from ovn-kubernetes 4m46s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-authcfg-http-02e08d Scaled up replica set payload-pre-processing-e2e-authcfg-http-02e08d-6f4945bb79 from 0 to 1 4m46s Normal SuccessfulCreate replicaset/payload-processing-e2e-authcfg-http-02e08d-7d74877466 Created pod: payload-processing-e2e-authcfg-http-02e08d-7d74877466-xrf9j 4m46s Warning FailedCreate replicaset/payload-processing-e2e-authcfg-http-02e08d-7d74877466 Error creating: pods "payload-processing-e2e-authcfg-http-02e08d-7d74877466-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-02e08d: serviceaccount "payload-processing-e2e-authcfg-http-02e08d" not found 4m46s Warning FailedCreate replicaset/payload-pre-processing-e2e-authcfg-http-02e08d-6f4945bb79 Error creating: pods "payload-pre-processing-e2e-authcfg-http-02e08d-6f4945bb79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-02e08d: serviceaccount "payload-processing-e2e-authcfg-http-02e08d" not found 4m46s Normal ScalingReplicaSet deployment/payload-processing-e2e-authcfg-http-02e08d Scaled up replica set payload-processing-e2e-authcfg-http-02e08d-7d74877466 from 0 to 1 4m46s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-authcfg-http-02e08d-6f4945bb79 Created pod: payload-pre-processing-e2e-authcfg-http-02e08d-6f4945bb79-sxdq6 4m45s Normal AddedInterface pod/payload-pre-processing-e2e-authcfg-http-02e08d-6f4945bb79-sxdq6 Add eth0 [10.132.1.160/23] from ovn-kubernetes 4m45s Normal Started pod/payload-pre-processing-e2e-authcfg-http-02e08d-6f4945bb79-sxdq6 Container started 4m45s Normal Created pod/payload-processing-e2e-authcfg-http-02e08d-7d74877466-xrf9j Container created 4m45s Normal Started pod/payload-processing-e2e-authcfg-http-02e08d-7d74877466-xrf9j Container started 4m45s Normal AddedInterface pod/payload-processing-e2e-authcfg-http-02e08d-7d74877466-xrf9j Add eth0 [10.132.1.159/23] from ovn-kubernetes 4m45s Normal Pulled pod/payload-pre-processing-e2e-authcfg-http-02e08d-6f4945bb79-sxdq6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 4m45s Normal Created pod/payload-pre-processing-e2e-authcfg-http-02e08d-6f4945bb79-sxdq6 Container created 4m45s Normal Pulled pod/payload-processing-e2e-authcfg-http-02e08d-7d74877466-xrf9j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 4m33s Warning ProbeError pod/e2e-authcfg-http-02e08d-openshift-default-8d4777cb5-mdkrz Readiness probe error: Get "http://10.132.1.158:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 4m33s Warning Unhealthy pod/e2e-authcfg-http-02e08d-openshift-default-8d4777cb5-mdkrz Readiness probe failed: Get "http://10.132.1.158:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 4m32s Normal Pulled pod/e2e-authcfg-http-02e08d-openshift-default-8d4777cb5-mdkrz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 4m32s Normal Created pod/e2e-authcfg-http-02e08d-openshift-default-8d4777cb5-mdkrz Container created 4m32s Normal Started pod/e2e-authcfg-http-02e08d-openshift-default-8d4777cb5-mdkrz Container started 4m26s Warning Unhealthy pod/e2e-authcfg-http-02e08d-openshift-default-8d4777cb5-mdkrz Startup probe failed: Get "http://10.132.1.158:15021/healthz/ready": dial tcp 10.132.1.158:15021: connect: connection refused 4m26s Warning ProbeError pod/e2e-authcfg-http-02e08d-openshift-default-8d4777cb5-mdkrz Startup probe error: Get "http://10.132.1.158:15021/healthz/ready": dial tcp 10.132.1.158:15021: connect: connection refused... 3m51s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-02e08d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 3m51s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-02e08d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 3m2s Normal Killing pod/payload-processing-e2e-authcfg-http-02e08d-7d74877466-xrf9j Stopping container payload-processing 3m2s Normal Killing pod/payload-pre-processing-e2e-authcfg-http-02e08d-6f4945bb79-sxdq6 Stopping container payload-pre-processing 2m47s Normal Killing pod/e2e-authcfg-http-02e08d-openshift-default-8d4777cb5-mdkrz Stopping container istio-proxy