=== events in openshift-ingress (2026-10-30T20:08:45Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 37m Normal Scheduled pod/payload-pre-processing-e2e-derive-5cb1c5cd-8565fccdcf-s4zvb Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-5cb1c5cd-8565fccdcf-s4zvb to ip-10-0-138-233.us-east-2.compute.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w3-fdb355-699984f479-clsd4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-fdb355-699984f479-clsd4 to ip-10-0-138-233.us-east-2.compute.internal 37m Normal Scheduled pod/e2e-migrate-5e333f16-gw-openshift-default-7c796c78f-kpx57 Successfully assigned openshift-ingress/e2e-migrate-5e333f16-gw-openshift-default-7c796c78f-kpx57 to ip-10-0-137-84.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-wkhn2 Successfully assigned openshift-ingress/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-wkhn2 to ip-10-0-140-127.us-east-2.compute.internal 37m Normal Scheduled pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-tclm9 Successfully assigned openshift-ingress/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-tclm9 to ip-10-0-140-127.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-2dmhf Successfully assigned openshift-ingress/e2e-worker-w5-60577a-openshift-default-7c8cb87747-2dmhf to ip-10-0-137-84.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-6lg8j Successfully assigned openshift-ingress/e2e-worker-w5-60577a-openshift-default-7c8cb87747-6lg8j to ip-10-0-138-233.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-fz2s8 Successfully assigned openshift-ingress/e2e-worker-w5-60577a-openshift-default-7c8cb87747-fz2s8 to ip-10-0-138-233.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-jggxr Successfully assigned openshift-ingress/e2e-worker-w5-60577a-openshift-default-7c8cb87747-jggxr to ip-10-0-137-84.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-inf-b-d651c4-openshift-default-775869444c-9wh89 Successfully assigned openshift-ingress/e2e-inf-b-d651c4-openshift-default-775869444c-9wh89 to ip-10-0-138-233.us-east-2.compute.internal 37m Normal Scheduled pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-p58s7 Successfully assigned openshift-ingress/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-p58s7 to ip-10-0-137-84.us-east-2.compute.internal 37m Normal Scheduled pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-jrr8j Successfully assigned openshift-ingress/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-jrr8j to ip-10-0-138-233.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-7f784 Successfully assigned openshift-ingress/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-7f784 to ip-10-0-138-233.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-gqmnf Successfully assigned openshift-ingress/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-gqmnf to ip-10-0-137-84.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-fmf6b Successfully assigned openshift-ingress/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-fmf6b to ip-10-0-138-233.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-jrrrp Successfully assigned openshift-ingress/e2e-worker-w5-60577a-openshift-default-7c8cb87747-jrrrp to ip-10-0-138-233.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-m9f79 Successfully assigned openshift-ingress/e2e-worker-w5-60577a-openshift-default-7c8cb87747-m9f79 to ip-10-0-140-127.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-ncnjw Successfully assigned openshift-ingress/e2e-worker-w5-60577a-openshift-default-7c8cb87747-ncnjw to ip-10-0-138-233.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-ddzt2 Successfully assigned openshift-ingress/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-ddzt2 to ip-10-0-138-233.us-east-2.compute.internal 36m Normal Scheduled pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-bqssp Successfully assigned openshift-ingress/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-bqssp to ip-10-0-137-84.us-east-2.compute.internal 47m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-6455b4886fw65nx Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fw65nx to ip-10-0-137-84.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-npzrt Successfully assigned openshift-ingress/e2e-worker-w5-60577a-openshift-default-7c8cb87747-npzrt to ip-10-0-137-84.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-ait-2ae6628f-openshift-default-856f6c7c6f-q5n4b Successfully assigned openshift-ingress/e2e-ait-2ae6628f-openshift-default-856f6c7c6f-q5n4b to ip-10-0-138-233.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-sbl2p Successfully assigned openshift-ingress/e2e-worker-w5-60577a-openshift-default-7c8cb87747-sbl2p to ip-10-0-140-127.us-east-2.compute.internal 52m Normal Scheduled pod/router-default-75866dc98c-4mnhf Successfully assigned openshift-ingress/router-default-75866dc98c-4mnhf to ip-10-0-137-84.us-east-2.compute.internal 52m Warning FailedScheduling pod/router-default-75866dc98c-4mnhf no nodes available to schedule pods 59m Warning FailedScheduling pod/router-default-75866dc98c-4mnhf no nodes available to schedule pods 37m Normal Scheduled pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-8b2zb Successfully assigned openshift-ingress/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-8b2zb to ip-10-0-138-233.us-east-2.compute.internal 27m Normal Scheduled pod/e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c-xmzml Successfully assigned openshift-ingress/e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c-xmzml to ip-10-0-140-127.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-75rps Successfully assigned openshift-ingress/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-75rps to ip-10-0-140-127.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-c6gjl Successfully assigned openshift-ingress/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-c6gjl to ip-10-0-140-127.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-4g5wx Successfully assigned openshift-ingress/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-4g5wx to ip-10-0-137-84.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-w5hjp Successfully assigned openshift-ingress/e2e-worker-w5-60577a-openshift-default-7c8cb87747-w5hjp to ip-10-0-140-127.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-jx2g9 Successfully assigned openshift-ingress/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-jx2g9 to ip-10-0-137-84.us-east-2.compute.internal 29m Normal Scheduled pod/e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c-qj56r Successfully assigned openshift-ingress/e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c-qj56r to ip-10-0-137-84.us-east-2.compute.internal 5m23s Normal Scheduled pod/e2e-ait-58847295-openshift-default-8f844cdfc-m4s26 Successfully assigned openshift-ingress/e2e-ait-58847295-openshift-default-8f844cdfc-m4s26 to ip-10-0-138-233.us-east-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-worker-w6-01b24e-7b4dc88878-4fbph Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-01b24e-7b4dc88878-4fbph to ip-10-0-138-233.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-whfvm Successfully assigned openshift-ingress/e2e-worker-w3-ce69f3-openshift-default-6f98547647-whfvm to ip-10-0-138-233.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-qxlwv Successfully assigned openshift-ingress/e2e-worker-w3-ce69f3-openshift-default-6f98547647-qxlwv to ip-10-0-137-84.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-mn4mp Successfully assigned openshift-ingress/e2e-worker-w3-ce69f3-openshift-default-6f98547647-mn4mp to ip-10-0-138-233.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-lkn8z Successfully assigned openshift-ingress/e2e-worker-w3-ce69f3-openshift-default-6f98547647-lkn8z to ip-10-0-140-127.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-kzwqv Successfully assigned openshift-ingress/e2e-worker-w3-ce69f3-openshift-default-6f98547647-kzwqv to ip-10-0-137-84.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-jgbnq Successfully assigned openshift-ingress/e2e-worker-w3-ce69f3-openshift-default-6f98547647-jgbnq to ip-10-0-137-84.us-east-2.compute.internal 34m Normal Scheduled pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-pgfhg Successfully assigned openshift-ingress/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-pgfhg to ip-10-0-138-233.us-east-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-worker-w5-60577a-646d79c448-2w78j Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-60577a-646d79c448-2w78j to ip-10-0-138-233.us-east-2.compute.internal 35m Normal Scheduled pod/e2e-shared-b-w3-976b80-openshift-default-69d4fb4fc9-6sdwr Successfully assigned openshift-ingress/e2e-shared-b-w3-976b80-openshift-default-69d4fb4fc9-6sdwr to ip-10-0-137-84.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-57r8q Successfully assigned openshift-ingress/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-57r8q to ip-10-0-138-233.us-east-2.compute.internal 8m16s Normal Scheduled pod/e2e-worker-single-main-cb7041-openshift-default-776964d669mzkbp Successfully assigned openshift-ingress/e2e-worker-single-main-cb7041-openshift-default-776964d669mzkbp to ip-10-0-138-233.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-28g9t Successfully assigned openshift-ingress/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-28g9t to ip-10-0-137-84.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-6smj6 Successfully assigned openshift-ingress/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-6smj6 to ip-10-0-140-127.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-8pztb Successfully assigned openshift-ingress/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-8pztb to ip-10-0-140-127.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-9z5vs Successfully assigned openshift-ingress/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-9z5vs to ip-10-0-138-233.us-east-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-worker-w4-ad96a4-7988fc87b-bpvc9 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-ad96a4-7988fc87b-bpvc9 to ip-10-0-138-233.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-bdpxx Successfully assigned openshift-ingress/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-bdpxx to ip-10-0-140-127.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-fpzgc Successfully assigned openshift-ingress/e2e-worker-w3-ce69f3-openshift-default-6f98547647-fpzgc to ip-10-0-137-84.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-dxnzj Successfully assigned openshift-ingress/e2e-worker-w3-ce69f3-openshift-default-6f98547647-dxnzj to ip-10-0-140-127.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-ait-77930c58-openshift-default-5d78dcfbdf-hcqvq Successfully assigned openshift-ingress/e2e-ait-77930c58-openshift-default-5d78dcfbdf-hcqvq to ip-10-0-140-127.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-dwtpl Successfully assigned openshift-ingress/e2e-worker-w3-ce69f3-openshift-default-6f98547647-dwtpl to ip-10-0-138-233.us-east-2.compute.internal 4m42s Normal Scheduled pod/payload-pre-processing-e2e-authcfg-http-64f2bb-7dbf5f4555-gns4h Successfully assigned openshift-ingress/payload-pre-processing-e2e-authcfg-http-64f2bb-7dbf5f4555-gns4h to ip-10-0-138-233.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-vm6lq Successfully assigned openshift-ingress/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-vm6lq to ip-10-0-140-127.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-5bwkh Successfully assigned openshift-ingress/e2e-worker-w3-ce69f3-openshift-default-6f98547647-5bwkh to ip-10-0-140-127.us-east-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-worker-w3-ce69f3-598446f65b-z7qlw Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-ce69f3-598446f65b-z7qlw to ip-10-0-138-233.us-east-2.compute.internal 27m Normal Scheduled pod/e2e-inf-b-47110f-openshift-default-6dc75dc96c-jtp46 Successfully assigned openshift-ingress/e2e-inf-b-47110f-openshift-default-6dc75dc96c-jtp46 to ip-10-0-137-84.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-d7zl9 Successfully assigned openshift-ingress/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-d7zl9 to ip-10-0-138-233.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-xbjth Successfully assigned openshift-ingress/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-xbjth to ip-10-0-138-233.us-east-2.compute.internal 36m Normal Scheduled pod/e2e-shared-b-w0-ae425f-openshift-default-76874f9f6c-vszwf Successfully assigned openshift-ingress/e2e-shared-b-w0-ae425f-openshift-default-76874f9f6c-vszwf to ip-10-0-137-84.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-d6kpm Successfully assigned openshift-ingress/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-d6kpm to ip-10-0-138-233.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-gspwm Successfully assigned openshift-ingress/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-gspwm to ip-10-0-138-233.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-npw9t Successfully assigned openshift-ingress/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-npw9t to ip-10-0-137-84.us-east-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-worker-w2-b623ca-867779ffc6-6dg44 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-b623ca-867779ffc6-6dg44 to ip-10-0-138-233.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-gsk6q Successfully assigned openshift-ingress/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-gsk6q to ip-10-0-137-84.us-east-2.compute.internal 33m Normal Scheduled pod/e2e-api-a-73f96d-openshift-default-66d877c755-9nvtp Successfully assigned openshift-ingress/e2e-api-a-73f96d-openshift-default-66d877c755-9nvtp to ip-10-0-140-127.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-lf4jg Successfully assigned openshift-ingress/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-lf4jg to ip-10-0-137-84.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-pjznf Successfully assigned openshift-ingress/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-pjznf to ip-10-0-137-84.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-q8kz9 Successfully assigned openshift-ingress/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-q8kz9 to ip-10-0-137-84.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-inf-b-47110f-openshift-default-6dc75dc96c-dcf9r Successfully assigned openshift-ingress/e2e-inf-b-47110f-openshift-default-6dc75dc96c-dcf9r to ip-10-0-140-127.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-zcdmk Successfully assigned openshift-ingress/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-zcdmk to ip-10-0-137-84.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-smclj Successfully assigned openshift-ingress/e2e-worker-w2-b623ca-openshift-default-68847758ff-smclj to ip-10-0-138-233.us-east-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-worker-w1-419d0a-688c6dfd6-8qmrl Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-419d0a-688c6dfd6-8qmrl to ip-10-0-138-233.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-qj2fm Successfully assigned openshift-ingress/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-qj2fm to ip-10-0-137-84.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-t2llg Successfully assigned openshift-ingress/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-t2llg to ip-10-0-140-127.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-t8psh Successfully assigned openshift-ingress/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-t8psh to ip-10-0-137-84.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-vwbfz Successfully assigned openshift-ingress/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-vwbfz to ip-10-0-138-233.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-rbvpb Successfully assigned openshift-ingress/e2e-worker-w2-b623ca-openshift-default-68847758ff-rbvpb to ip-10-0-137-84.us-east-2.compute.internal 36m Normal Scheduled pod/e2e-shared-a-w3-fdb355-openshift-default-d67d8d799-b6tcd Successfully assigned openshift-ingress/e2e-shared-a-w3-fdb355-openshift-default-d67d8d799-b6tcd to ip-10-0-138-233.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-t8wnk Successfully assigned openshift-ingress/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-t8wnk to ip-10-0-140-127.us-east-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-worker-w0-ae63bf-56c67d7d7b-mgtt5 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-ae63bf-56c67d7d7b-mgtt5 to ip-10-0-137-84.us-east-2.compute.internal 8m12s Normal Scheduled pod/payload-processing-e2e-worker-single-main-cb7041-9c74d8c9dzmn2j Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-cb7041-9c74d8c9dzmn2j to ip-10-0-138-233.us-east-2.compute.internal 11m Normal Scheduled pod/payload-processing-e2e-worker-single-main-98b568-75c89f78cgkxbl Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-98b568-75c89f78cgkxbl to ip-10-0-138-233.us-east-2.compute.internal 35m Normal Scheduled pod/payload-processing-e2e-shared-b-w3-976b80-646cd76644-rlkvm Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-976b80-646cd76644-rlkvm to ip-10-0-137-84.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-api-a-82012b-openshift-default-67457b4d8c-fx2xx Successfully assigned openshift-ingress/e2e-api-a-82012b-openshift-default-67457b4d8c-fx2xx to ip-10-0-138-233.us-east-2.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-ca9d14-746f5b5598-6jt2z Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-ca9d14-746f5b5598-6jt2z to ip-10-0-137-84.us-east-2.compute.internal 36m Normal Scheduled pod/payload-processing-e2e-shared-b-w0-ae425f-5885c86dd4-4wvh9 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-ae425f-5885c86dd4-4wvh9 to ip-10-0-137-84.us-east-2.compute.internal 36m Normal Scheduled pod/payload-processing-e2e-shared-a-w3-fdb355-75d6685876-vw9vc Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-fdb355-75d6685876-vw9vc to ip-10-0-138-233.us-east-2.compute.internal 35m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-59f017-694c87888d-lw9vk Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-59f017-694c87888d-lw9vk to ip-10-0-137-84.us-east-2.compute.internal 37m Normal Scheduled pod/payload-processing-e2e-shared-a-w0-395ccc-656669bc5b-sg8dz Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-395ccc-656669bc5b-sg8dz to ip-10-0-138-233.us-east-2.compute.internal 36m Normal Scheduled pod/payload-processing-e2e-mt-2fd9a937-7b6c4cdc98-p4kpl Successfully assigned openshift-ingress/payload-processing-e2e-mt-2fd9a937-7b6c4cdc98-p4kpl to ip-10-0-138-233.us-east-2.compute.internal 35m Normal Scheduled pod/payload-processing-e2e-mt-20e66fa4-54557c4fd4-j8gh8 Successfully assigned openshift-ingress/payload-processing-e2e-mt-20e66fa4-54557c4fd4-j8gh8 to ip-10-0-137-84.us-east-2.compute.internal 32m Normal Scheduled pod/payload-processing-e2e-mt-20e66fa4-54557c4fd4-8ffbj Successfully assigned openshift-ingress/payload-processing-e2e-mt-20e66fa4-54557c4fd4-8ffbj to ip-10-0-137-84.us-east-2.compute.internal 37m Normal Scheduled pod/payload-processing-e2e-migrate-5e333f16-68c977c4bd-rhgmr Successfully assigned openshift-ingress/payload-processing-e2e-migrate-5e333f16-68c977c4bd-rhgmr to ip-10-0-137-84.us-east-2.compute.internal 30m Normal Scheduled pod/payload-processing-e2e-inf-b-d651c4-7f667dbdfb-48tkv Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-d651c4-7f667dbdfb-48tkv to ip-10-0-138-233.us-east-2.compute.internal 28m Normal Scheduled pod/payload-processing-e2e-inf-b-ac37c8-69dbd9fb6d-xjzdf Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-ac37c8-69dbd9fb6d-xjzdf to ip-10-0-137-84.us-east-2.compute.internal 27m Normal Scheduled pod/payload-processing-e2e-inf-b-47110f-65f4f5b6b4-5528c Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-47110f-65f4f5b6b4-5528c to ip-10-0-137-84.us-east-2.compute.internal 30m Normal Scheduled pod/payload-processing-e2e-inf-b-1e13df-68d544b494-c828d Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-1e13df-68d544b494-c828d to ip-10-0-138-233.us-east-2.compute.internal 31m Normal Scheduled pod/payload-processing-e2e-inf-a-89a3f6-7b8cb866f4-5zqx4 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-89a3f6-7b8cb866f4-5zqx4 to ip-10-0-138-233.us-east-2.compute.internal 29m Normal Scheduled pod/payload-processing-e2e-inf-a-744b57-597d6c8986-4v2zh Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-744b57-597d6c8986-4v2zh to ip-10-0-137-84.us-east-2.compute.internal 31m Normal Scheduled pod/payload-processing-e2e-inf-a-579ddb-6756775486-ghzlj Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-579ddb-6756775486-ghzlj to ip-10-0-137-84.us-east-2.compute.internal 34m Normal Scheduled pod/e2e-api-a-c7880b-openshift-default-56db9c9557-9vwgs Successfully assigned openshift-ingress/e2e-api-a-c7880b-openshift-default-56db9c9557-9vwgs to ip-10-0-140-127.us-east-2.compute.internal 28m Normal Scheduled pod/payload-processing-e2e-inf-a-3e35df-7f747b7fb-gdjfs Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-3e35df-7f747b7fb-gdjfs to ip-10-0-137-84.us-east-2.compute.internal 37m Normal Scheduled pod/payload-processing-e2e-derive-5cb1c5cd-9cc6f45d4-98mg5 Successfully assigned openshift-ingress/payload-processing-e2e-derive-5cb1c5cd-9cc6f45d4-98mg5 to ip-10-0-138-233.us-east-2.compute.internal 4m42s Normal Scheduled pod/payload-processing-e2e-authcfg-http-64f2bb-5cbb5f6456-x96sh Successfully assigned openshift-ingress/payload-processing-e2e-authcfg-http-64f2bb-5cbb5f6456-x96sh to ip-10-0-138-233.us-east-2.compute.internal 33m Normal Scheduled pod/payload-processing-e2e-api-b-95d2ac-75f686bc4d-v7vw5 Successfully assigned openshift-ingress/payload-processing-e2e-api-b-95d2ac-75f686bc4d-v7vw5 to ip-10-0-137-84.us-east-2.compute.internal 32m Normal Scheduled pod/payload-processing-e2e-api-b-6a6348-677c897664-5q8vf Successfully assigned openshift-ingress/payload-processing-e2e-api-b-6a6348-677c897664-5q8vf to ip-10-0-137-84.us-east-2.compute.internal 29m Normal Scheduled pod/payload-processing-e2e-api-b-22af66-7b5b988746-mm9zl Successfully assigned openshift-ingress/payload-processing-e2e-api-b-22af66-7b5b988746-mm9zl to ip-10-0-137-84.us-east-2.compute.internal 30m Normal Scheduled pod/payload-processing-e2e-api-b-156838-79f475dcd-x2qg9 Successfully assigned openshift-ingress/payload-processing-e2e-api-b-156838-79f475dcd-x2qg9 to ip-10-0-137-84.us-east-2.compute.internal 30m Normal Scheduled pod/payload-processing-e2e-api-a-d1d256-5b65b66d9d-2djrn Successfully assigned openshift-ingress/payload-processing-e2e-api-a-d1d256-5b65b66d9d-2djrn to ip-10-0-137-84.us-east-2.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-api-a-c7880b-d5b8b5f94-gwr52 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-c7880b-d5b8b5f94-gwr52 to ip-10-0-137-84.us-east-2.compute.internal 31m Normal Scheduled pod/payload-processing-e2e-api-a-82012b-64d6c8f676-wv8kj Successfully assigned openshift-ingress/payload-processing-e2e-api-a-82012b-64d6c8f676-wv8kj to ip-10-0-138-233.us-east-2.compute.internal 32m Normal Scheduled pod/payload-processing-e2e-api-a-73f96d-767bd9568-4pjpt Successfully assigned openshift-ingress/payload-processing-e2e-api-a-73f96d-767bd9568-4pjpt to ip-10-0-138-233.us-east-2.compute.internal 38m Normal Scheduled pod/payload-processing-e2e-ait-77930c58-f86d9d8fd-gg9bz Successfully assigned openshift-ingress/payload-processing-e2e-ait-77930c58-f86d9d8fd-gg9bz to ip-10-0-138-233.us-east-2.compute.internal 5m19s Normal Scheduled pod/payload-processing-e2e-ait-58847295-865b6f598b-q29b7 Successfully assigned openshift-ingress/payload-processing-e2e-ait-58847295-865b6f598b-q29b7 to ip-10-0-138-233.us-east-2.compute.internal 37m Normal Scheduled pod/payload-processing-e2e-ait-2ae6628f-54ff895c9d-4d9pk Successfully assigned openshift-ingress/payload-processing-e2e-ait-2ae6628f-54ff895c9d-4d9pk to ip-10-0-137-84.us-east-2.compute.internal 37m Normal Scheduled pod/payload-processing-e2e-ait-229db90d-6f45bf4ddb-q875f Successfully assigned openshift-ingress/payload-processing-e2e-ait-229db90d-6f45bf4ddb-q875f to ip-10-0-137-84.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-vz4wm Successfully assigned openshift-ingress/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-vz4wm to ip-10-0-140-127.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-api-a-d1d256-openshift-default-979b4ccbf-lm6hp Successfully assigned openshift-ingress/e2e-api-a-d1d256-openshift-default-979b4ccbf-lm6hp to ip-10-0-137-84.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-kqc8x Successfully assigned openshift-ingress/e2e-worker-w2-b623ca-openshift-default-68847758ff-kqc8x to ip-10-0-138-233.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-g8pb5 Successfully assigned openshift-ingress/e2e-worker-w2-b623ca-openshift-default-68847758ff-g8pb5 to ip-10-0-138-233.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-d2xbp Successfully assigned openshift-ingress/e2e-worker-w2-b623ca-openshift-default-68847758ff-d2xbp to ip-10-0-137-84.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-bsbrr Successfully assigned openshift-ingress/e2e-worker-w2-b623ca-openshift-default-68847758ff-bsbrr to ip-10-0-140-127.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-6z6zf Successfully assigned openshift-ingress/e2e-worker-w2-b623ca-openshift-default-68847758ff-6z6zf to ip-10-0-138-233.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-5frq8 Successfully assigned openshift-ingress/e2e-worker-w2-b623ca-openshift-default-68847758ff-5frq8 to ip-10-0-140-127.us-east-2.compute.internal 44m Normal Scheduled pod/payload-processing-5f78687d5b-pqrwr Successfully assigned openshift-ingress/payload-processing-5f78687d5b-pqrwr to ip-10-0-137-84.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-5dpgx Successfully assigned openshift-ingress/e2e-worker-w2-b623ca-openshift-default-68847758ff-5dpgx to ip-10-0-140-127.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-2d82j Successfully assigned openshift-ingress/e2e-worker-w2-b623ca-openshift-default-68847758ff-2d82j to ip-10-0-137-84.us-east-2.compute.internal 28m Normal Scheduled pod/e2e-inf-b-1e13df-openshift-default-58bb9b8669-cf9fq Successfully assigned openshift-ingress/e2e-inf-b-1e13df-openshift-default-58bb9b8669-cf9fq to ip-10-0-140-127.us-east-2.compute.internal 48m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-wdm9c Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-wdm9c to ip-10-0-138-233.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-inf-b-1e13df-openshift-default-58bb9b8669-c6zvx Successfully assigned openshift-ingress/e2e-inf-b-1e13df-openshift-default-58bb9b8669-c6zvx to ip-10-0-137-84.us-east-2.compute.internal 37m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-2zxhf Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2zxhf to ip-10-0-140-127.us-east-2.compute.internal 46m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-mfsgz Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mfsgz to ip-10-0-138-233.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-api-b-156838-openshift-default-6cc4b565b5-ljxwd Successfully assigned openshift-ingress/e2e-api-b-156838-openshift-default-6cc4b565b5-ljxwd to ip-10-0-137-84.us-east-2.compute.internal 36m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-p5c6s Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p5c6s to ip-10-0-140-127.us-east-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-01b24e-f7447dc8c-z5lr6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-01b24e-f7447dc8c-z5lr6 to ip-10-0-138-233.us-east-2.compute.internal 39m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-zt46p Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zt46p to ip-10-0-137-84.us-east-2.compute.internal 47m Normal Scheduled pod/kube-auth-proxy-7c5f4bd96d-fsk2s Successfully assigned openshift-ingress/kube-auth-proxy-7c5f4bd96d-fsk2s to ip-10-0-140-127.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-inf-a-89a3f6-openshift-default-547f79fc89-njwhz Successfully assigned openshift-ingress/e2e-inf-a-89a3f6-openshift-default-547f79fc89-njwhz to ip-10-0-138-233.us-east-2.compute.internal 47m Normal Scheduled pod/kube-auth-proxy-7c5f4bd96d-rhxkl Successfully assigned openshift-ingress/kube-auth-proxy-7c5f4bd96d-rhxkl to ip-10-0-138-233.us-east-2.compute.internal 28m Normal Scheduled pod/e2e-inf-a-89a3f6-openshift-default-547f79fc89-d75jj Successfully assigned openshift-ingress/e2e-inf-a-89a3f6-openshift-default-547f79fc89-d75jj to ip-10-0-137-84.us-east-2.compute.internal 36m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-5cd48 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-5cd48 to ip-10-0-138-233.us-east-2.compute.internal 36m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-5jm6z Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-5jm6z to ip-10-0-138-233.us-east-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-60577a-785b689d99-wh2hs Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-60577a-785b689d99-wh2hs to ip-10-0-138-233.us-east-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-742kv Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-742kv to ip-10-0-138-233.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-2ch45 Successfully assigned openshift-ingress/e2e-worker-w1-419d0a-openshift-default-55766cf857-2ch45 to ip-10-0-138-233.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-6snr9 Successfully assigned openshift-ingress/e2e-worker-w1-419d0a-openshift-default-55766cf857-6snr9 to ip-10-0-138-233.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-6x6nn Successfully assigned openshift-ingress/e2e-worker-w1-419d0a-openshift-default-55766cf857-6x6nn to ip-10-0-138-233.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-api-b-22af66-openshift-default-5f7c494769-gqftx Successfully assigned openshift-ingress/e2e-api-b-22af66-openshift-default-5f7c494769-gqftx to ip-10-0-138-233.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-fqcxb Successfully assigned openshift-ingress/e2e-worker-w1-419d0a-openshift-default-55766cf857-fqcxb to ip-10-0-140-127.us-east-2.compute.internal 34m Normal Scheduled pod/e2e-shared-b-w2-ca9d14-openshift-default-65f96b67bf-klhmt Successfully assigned openshift-ingress/e2e-shared-b-w2-ca9d14-openshift-default-65f96b67bf-klhmt to ip-10-0-137-84.us-east-2.compute.internal 35m Normal Scheduled pod/e2e-shared-a-w2-59f017-openshift-default-ddc89b68c-lshmc Successfully assigned openshift-ingress/e2e-shared-a-w2-59f017-openshift-default-ddc89b68c-lshmc to ip-10-0-137-84.us-east-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-ad96a4-746867c7d7-lnlgf Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-ad96a4-746867c7d7-lnlgf to ip-10-0-137-84.us-east-2.compute.internal 9m58s Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-82vrf Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-82vrf to ip-10-0-138-233.us-east-2.compute.internal 9m43s Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-bjr2f Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-bjr2f to ip-10-0-138-233.us-east-2.compute.internal 9m43s Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-ctqxf Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ctqxf to ip-10-0-140-127.us-east-2.compute.internal 36m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-f5qx7 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-f5qx7 to ip-10-0-137-84.us-east-2.compute.internal 40m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-frx85 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-frx85 to ip-10-0-140-127.us-east-2.compute.internal 9m43s Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-ftjzp Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftjzp to ip-10-0-137-84.us-east-2.compute.internal 9m58s Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-jrm57 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-jrm57 to ip-10-0-137-84.us-east-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-ce69f3-f897b54f5-hv6zc Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-ce69f3-f897b54f5-hv6zc to ip-10-0-138-233.us-east-2.compute.internal 29m Normal Scheduled pod/e2e-inf-a-744b57-openshift-default-6c5cbf6bb9-zctmd Successfully assigned openshift-ingress/e2e-inf-a-744b57-openshift-default-6c5cbf6bb9-zctmd to ip-10-0-137-84.us-east-2.compute.internal 36m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-lnkcf Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lnkcf to ip-10-0-138-233.us-east-2.compute.internal 9m43s Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-ndnrw Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ndnrw to ip-10-0-138-233.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-jssx7 Successfully assigned openshift-ingress/e2e-worker-w1-419d0a-openshift-default-55766cf857-jssx7 to ip-10-0-140-127.us-east-2.compute.internal 32m Normal Scheduled pod/e2e-api-b-6a6348-openshift-default-7bd9c5bc47-cmtk5 Successfully assigned openshift-ingress/e2e-api-b-6a6348-openshift-default-7bd9c5bc47-cmtk5 to ip-10-0-137-84.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-lr27f Successfully assigned openshift-ingress/e2e-worker-w1-419d0a-openshift-default-55766cf857-lr27f to ip-10-0-137-84.us-east-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-ndqct Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ndqct to ip-10-0-140-127.us-east-2.compute.internal 37m Normal Scheduled pod/e2e-shared-a-w0-395ccc-openshift-default-5dcf9cb54c-fnlf6 Successfully assigned openshift-ingress/e2e-shared-a-w0-395ccc-openshift-default-5dcf9cb54c-fnlf6 to ip-10-0-137-84.us-east-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-b623ca-5c58d97f4c-p6qfh Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-b623ca-5c58d97f4c-p6qfh to ip-10-0-138-233.us-east-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-q4492 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-q4492 to ip-10-0-137-84.us-east-2.compute.internal 40m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-q4g9n Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-q4g9n to ip-10-0-138-233.us-east-2.compute.internal 11m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-qbqj6 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-qbqj6 to ip-10-0-138-233.us-east-2.compute.internal 40m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-qdh96 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-qdh96 to ip-10-0-137-84.us-east-2.compute.internal 27m Normal Scheduled pod/e2e-inf-a-744b57-openshift-default-6c5cbf6bb9-b74k6 Successfully assigned openshift-ingress/e2e-inf-a-744b57-openshift-default-6c5cbf6bb9-b74k6 to ip-10-0-138-233.us-east-2.compute.internal 36m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-qj2g6 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-qj2g6 to ip-10-0-137-84.us-east-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-s6ldn Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s6ldn to ip-10-0-138-233.us-east-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-419d0a-57688785ff-fjn4j Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-419d0a-57688785ff-fjn4j to ip-10-0-138-233.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-mkgcz Successfully assigned openshift-ingress/e2e-worker-w1-419d0a-openshift-default-55766cf857-mkgcz to ip-10-0-138-233.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-nkc9c Successfully assigned openshift-ingress/e2e-worker-w1-419d0a-openshift-default-55766cf857-nkc9c to ip-10-0-137-84.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-rkbbc Successfully assigned openshift-ingress/e2e-worker-w1-419d0a-openshift-default-55766cf857-rkbbc to ip-10-0-137-84.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-tmlkg Successfully assigned openshift-ingress/e2e-worker-w1-419d0a-openshift-default-55766cf857-tmlkg to ip-10-0-140-127.us-east-2.compute.internal 9m58s Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-s7gc9 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-s7gc9 to ip-10-0-140-127.us-east-2.compute.internal 33m Normal Scheduled pod/e2e-api-b-95d2ac-openshift-default-7f6cb95d55-t6xhk Successfully assigned openshift-ingress/e2e-api-b-95d2ac-openshift-default-7f6cb95d55-t6xhk to ip-10-0-137-84.us-east-2.compute.internal 48m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-x2wnb Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-x2wnb to ip-10-0-140-127.us-east-2.compute.internal 36m Normal Scheduled pod/e2e-mt-2fd9a937-openshift-default-76594d45b5-55g7m Successfully assigned openshift-ingress/e2e-mt-2fd9a937-openshift-default-76594d45b5-55g7m to ip-10-0-137-84.us-east-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-ae63bf-76d99cc4d7-fdtz4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-ae63bf-76d99cc4d7-fdtz4 to ip-10-0-137-84.us-east-2.compute.internal 8m12s Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-cb7041-68c6d7lt67 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-cb7041-68c6d7lt67 to ip-10-0-138-233.us-east-2.compute.internal 11m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-98b568-58446m98kd Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-98b568-58446m98kd to ip-10-0-138-233.us-east-2.compute.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w3-976b80-6869f686f9-9cs2n Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-976b80-6869f686f9-9cs2n to ip-10-0-138-233.us-east-2.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-ca9d14-5c478d7cc5-wtmp4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ca9d14-5c478d7cc5-wtmp4 to ip-10-0-137-84.us-east-2.compute.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w0-ae425f-79c9ff7db7-z7nf7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-ae425f-79c9ff7db7-z7nf7 to ip-10-0-137-84.us-east-2.compute.internal 33m Normal Scheduled pod/payload-pre-processing-e2e-api-b-95d2ac-74dbfdbb99-fjbtf Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-95d2ac-74dbfdbb99-fjbtf to ip-10-0-137-84.us-east-2.compute.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-59f017-5fb9d9fcf5-5xlfc Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-59f017-5fb9d9fcf5-5xlfc to ip-10-0-138-233.us-east-2.compute.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w0-395ccc-7d4b79d8f9-gn47s Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-395ccc-7d4b79d8f9-gn47s to ip-10-0-137-84.us-east-2.compute.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-mt-2fd9a937-7c795b4979-8kvvc Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-2fd9a937-7c795b4979-8kvvc to ip-10-0-137-84.us-east-2.compute.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-mt-20e66fa4-997965c87-hsg4m Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-20e66fa4-997965c87-hsg4m to ip-10-0-137-84.us-east-2.compute.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-mt-20e66fa4-997965c87-7v4rf Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-20e66fa4-997965c87-7v4rf to ip-10-0-137-84.us-east-2.compute.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-migrate-5e333f16-f5f6bd6dc-jfs9n Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-5e333f16-f5f6bd6dc-jfs9n to ip-10-0-137-84.us-east-2.compute.internal 30m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-d651c4-75cbc576bc-tp8q9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-d651c4-75cbc576bc-tp8q9 to ip-10-0-138-233.us-east-2.compute.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-ac37c8-57cb47d8c-4cprd Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-ac37c8-57cb47d8c-4cprd to ip-10-0-137-84.us-east-2.compute.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-47110f-7cdc4b47f5-2brw4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-47110f-7cdc4b47f5-2brw4 to ip-10-0-137-84.us-east-2.compute.internal 4m46s Normal Scheduled pod/e2e-authcfg-http-64f2bb-openshift-default-6c68c554d9-dwc8m Successfully assigned openshift-ingress/e2e-authcfg-http-64f2bb-openshift-default-6c68c554d9-dwc8m to ip-10-0-138-233.us-east-2.compute.internal 30m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-1e13df-78b7ccf4bc-8ldn2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-1e13df-78b7ccf4bc-8ldn2 to ip-10-0-137-84.us-east-2.compute.internal 31m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-89a3f6-db46ffd87-8l4s5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-89a3f6-db46ffd87-8l4s5 to ip-10-0-137-84.us-east-2.compute.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-744b57-77cc66bfc9-2fcz9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-744b57-77cc66bfc9-2fcz9 to ip-10-0-137-84.us-east-2.compute.internal 31m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-579ddb-9b6756cf9-gfdfg Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-579ddb-9b6756cf9-gfdfg to ip-10-0-137-84.us-east-2.compute.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-3e35df-7874fb7595-2v6xk Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-3e35df-7874fb7595-2v6xk to ip-10-0-137-84.us-east-2.compute.internal 11m Normal Scheduled pod/e2e-worker-single-main-98b568-openshift-default-96865bb7c-bxslr Successfully assigned openshift-ingress/e2e-worker-single-main-98b568-openshift-default-96865bb7c-bxslr to ip-10-0-138-233.us-east-2.compute.internal 11m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-zgd28 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgd28 to ip-10-0-137-84.us-east-2.compute.internal 37m Normal Scheduled pod/e2e-ait-229db90d-openshift-default-8696475765-tkr28 Successfully assigned openshift-ingress/e2e-ait-229db90d-openshift-default-8696475765-tkr28 to ip-10-0-138-233.us-east-2.compute.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-api-b-6a6348-79bcfb7655-cpxz9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-6a6348-79bcfb7655-cpxz9 to ip-10-0-137-84.us-east-2.compute.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-api-b-22af66-5bf7f46cd7-ffx8p Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-22af66-5bf7f46cd7-ffx8p to ip-10-0-137-84.us-east-2.compute.internal 30m Normal Scheduled pod/payload-pre-processing-e2e-api-b-156838-6749758ddc-tnv4d Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-156838-6749758ddc-tnv4d to ip-10-0-137-84.us-east-2.compute.internal 30m Normal Scheduled pod/payload-pre-processing-e2e-api-a-d1d256-5b889bbcc5-qb7k8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-d1d256-5b889bbcc5-qb7k8 to ip-10-0-137-84.us-east-2.compute.internal 37m Normal Scheduled pod/e2e-derive-5cb1c5cd-openshift-default-67897f4645-w4hvx Successfully assigned openshift-ingress/e2e-derive-5cb1c5cd-openshift-default-67897f4645-w4hvx to ip-10-0-138-233.us-east-2.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-api-a-c7880b-c46dcb8b7-dkz2k Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-c7880b-c46dcb8b7-dkz2k to ip-10-0-137-84.us-east-2.compute.internal 31m Normal Scheduled pod/payload-pre-processing-e2e-api-a-82012b-5987f89c59-xqtgs Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-82012b-5987f89c59-xqtgs to ip-10-0-137-84.us-east-2.compute.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-api-a-73f96d-65c48fbd9f-98p49 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-73f96d-65c48fbd9f-98p49 to ip-10-0-137-84.us-east-2.compute.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-ait-77930c58-75d6fb56d7-s8tgk Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-77930c58-75d6fb56d7-s8tgk to ip-10-0-138-233.us-east-2.compute.internal 5m19s Normal Scheduled pod/payload-pre-processing-e2e-ait-58847295-7cbc7fcfc5-7srtc Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-58847295-7cbc7fcfc5-7srtc to ip-10-0-138-233.us-east-2.compute.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-ait-2ae6628f-6fbdd54f79-qh5cw Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-2ae6628f-6fbdd54f79-qh5cw to ip-10-0-138-233.us-east-2.compute.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-ait-229db90d-7b97c78b67-wn2bb Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-229db90d-7b97c78b67-wn2bb to ip-10-0-138-233.us-east-2.compute.internal 36m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-zl27m Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zl27m to ip-10-0-140-127.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-inf-a-579ddb-openshift-default-677f4b6845-mwmnb Successfully assigned openshift-ingress/e2e-inf-a-579ddb-openshift-default-677f4b6845-mwmnb to ip-10-0-137-84.us-east-2.compute.internal 48m Normal Scheduled pod/openshift-ai-inference-openshift-ai-inference-64695b4bc9-qmb5h Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-64695b4bc9-qmb5h to ip-10-0-137-84.us-east-2.compute.internal 28m Normal Scheduled pod/e2e-inf-a-3e35df-openshift-default-67646bf575-7psjn Successfully assigned openshift-ingress/e2e-inf-a-3e35df-openshift-default-67646bf575-7psjn to ip-10-0-140-127.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-inf-a-3e35df-openshift-default-67646bf575-jmxjq Successfully assigned openshift-ingress/e2e-inf-a-3e35df-openshift-default-67646bf575-jmxjq to ip-10-0-137-84.us-east-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-7b98bb678c-v5vhh Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-v5vhh to ip-10-0-138-233.us-east-2.compute.internal 59m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-75866dc98c from 0 to 1 59m Normal SuccessfulCreate replicaset/router-default-75866dc98c Created pod: router-default-75866dc98c-4mnhf 58m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 58m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 58m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 52m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 52m Warning FailedMount pod/router-default-75866dc98c-4mnhf MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 52m Warning FailedMount pod/router-default-75866dc98c-4mnhf MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 52m Warning FailedMount pod/router-default-75866dc98c-4mnhf MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 52m Normal Pulled pod/router-default-75866dc98c-4mnhf Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 52m Normal Started pod/router-default-75866dc98c-4mnhf Container started 52m Normal Created pod/router-default-75866dc98c-4mnhf Container created 52m Normal AddedInterface pod/router-default-75866dc98c-4mnhf Add eth0 [10.132.0.12/23] from ovn-kubernetes 48m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 48m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-wdm9c 48m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 48m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-wdm9c Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 48m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-wdm9c Add eth0 [10.134.0.21/23] from ovn-kubernetes 48m Normal Created pod/istiod-openshift-gateway-7876bfcb74-wdm9c Container created 48m Normal Started pod/istiod-openshift-gateway-7876bfcb74-wdm9c Container started 48m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-wdm9c Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.564s (3.564s including waiting). Image size: 185476477 bytes. 48m Normal EnsuringLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensuring load balancer 48m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-wdm9c Readiness probe error: HTTP probe failed with statuscode: 503... 48m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-wdm9c Readiness probe failed: HTTP probe failed with statuscode: 503 48m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-5bc884dd87 from 0 to 1 48m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-x2wnb 48m Normal SuccessfulCreate replicaset/openshift-ai-inference-openshift-ai-inference-64695b4bc9 Created pod: openshift-ai-inference-openshift-ai-inference-64695b4bc9-qmb5h 48m Normal ScalingReplicaSet deployment/openshift-ai-inference-openshift-ai-inference Scaled up replica set openshift-ai-inference-openshift-ai-inference-64695b4bc9 from 0 to 1 48m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-x2wnb Add eth0 [10.133.0.18/23] from ovn-kubernetes 48m Normal Pulling pod/openshift-ai-inference-openshift-ai-inference-64695b4bc9-qmb5h Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 48m Normal AddedInterface pod/openshift-ai-inference-openshift-ai-inference-64695b4bc9-qmb5h Add eth0 [10.132.0.32/23] from ovn-kubernetes 48m Normal Pulling pod/maas-default-gateway-openshift-default-5bc884dd87-x2wnb Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 48m Normal EnsuredLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensured load balancer 48m Normal Created pod/openshift-ai-inference-openshift-ai-inference-64695b4bc9-qmb5h Container created 48m Normal Started pod/openshift-ai-inference-openshift-ai-inference-64695b4bc9-qmb5h Container started 48m Normal Pulled pod/openshift-ai-inference-openshift-ai-inference-64695b4bc9-qmb5h Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.268s (3.268s including waiting). Image size: 183555333 bytes. 48m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-x2wnb Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 6.53s (6.53s including waiting). Image size: 183555333 bytes. 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Normal SuccessfulCreate replicaset/kube-auth-proxy-7c5f4bd96d Created pod: kube-auth-proxy-7c5f4bd96d-fsk2s 47m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-6455b4886f from 0 to 1 47m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-6455b4886f Created pod: data-science-gateway-data-science-gateway-class-6455b4886fw65nx 47m Normal SuccessfulCreate replicaset/kube-auth-proxy-7c5f4bd96d Created pod: kube-auth-proxy-7c5f4bd96d-rhxkl 47m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-7c5f4bd96d from 0 to 2 47m Normal AddedInterface pod/kube-auth-proxy-7c5f4bd96d-fsk2s Add eth0 [10.133.0.26/23] from ovn-kubernetes 47m Normal Pulling pod/kube-auth-proxy-7c5f4bd96d-rhxkl Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 47m Normal Created pod/data-science-gateway-data-science-gateway-class-6455b4886fw65nx Container created 47m Normal Pulled pod/data-science-gateway-data-science-gateway-class-6455b4886fw65nx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-6455b4886fw65nx Add eth0 [10.132.0.39/23] from ovn-kubernetes 47m Normal Pulling pod/kube-auth-proxy-7c5f4bd96d-fsk2s Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 47m Normal Started pod/data-science-gateway-data-science-gateway-class-6455b4886fw65nx Container started 47m Normal AddedInterface pod/kube-auth-proxy-7c5f4bd96d-rhxkl Add eth0 [10.134.0.24/23] from ovn-kubernetes 47m Normal Started pod/kube-auth-proxy-7c5f4bd96d-fsk2s Container started 47m Normal Created pod/kube-auth-proxy-7c5f4bd96d-fsk2s Container created 47m Normal Pulled pod/kube-auth-proxy-7c5f4bd96d-fsk2s Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.522s (2.522s including waiting). Image size: 188904677 bytes. 47m Normal Pulled pod/kube-auth-proxy-7c5f4bd96d-rhxkl Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 3.032s (3.032s including waiting). Image size: 188904677 bytes. 47m Normal Created pod/kube-auth-proxy-7c5f4bd96d-rhxkl Container created 47m Normal Started pod/kube-auth-proxy-7c5f4bd96d-rhxkl Container started 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-mfsgz Add eth0 [10.134.0.29/23] from ovn-kubernetes 46m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 46m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-mfsgz 46m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-mfsgz Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 46m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-mfsgz Container created 46m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-mfsgz Container started 46m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-mfsgz Readiness probe error: HTTP probe failed with statuscode: 503... 46m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-mfsgz Readiness probe failed: HTTP probe failed with statuscode: 503 46m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-wdm9c 46m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 46m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-wdm9c Stopping container discovery 46m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-x2wnb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-x2wnb Container created 44m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-x2wnb Container started 44m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-x2wnb Startup probe failed: Get "http://10.133.0.18:15021/healthz/ready": dial tcp 10.133.0.18:15021: connect: connection refused 44m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-x2wnb Startup probe error: Get "http://10.133.0.18:15021/healthz/ready": dial tcp 10.133.0.18:15021: connect: connection refused... 44m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-x2wnb Startup probe failed: Get "http://10.133.0.18:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-x2wnb Startup probe error: Get "http://10.133.0.18:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-5f78687d5b from 0 to 1 44m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 44m Normal AddedInterface pod/payload-processing-5f78687d5b-pqrwr Add eth0 [10.132.0.50/23] from ovn-kubernetes 44m Normal Pulling pod/payload-processing-5f78687d5b-pqrwr Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 44m Warning FailedCreate replicaset/payload-processing-5f78687d5b Error creating: pods "payload-processing-5f78687d5b-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 44m Normal SuccessfulCreate replicaset/payload-processing-5f78687d5b Created pod: payload-processing-5f78687d5b-pqrwr 44m Normal Pulling pod/payload-pre-processing-7b98bb678c-v5vhh Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 44m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-v5vhh 44m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 44m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-v5vhh Add eth0 [10.134.0.37/23] from ovn-kubernetes 44m Normal Created pod/payload-pre-processing-7b98bb678c-v5vhh Container created 44m Normal Started pod/payload-pre-processing-7b98bb678c-v5vhh Container started 44m Normal Started pod/payload-processing-5f78687d5b-pqrwr Container started 44m Normal Created pod/payload-processing-5f78687d5b-pqrwr Container created 44m Normal Pulled pod/payload-processing-5f78687d5b-pqrwr Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.036s (2.036s including waiting). Image size: 125353965 bytes. 44m Normal Pulled pod/payload-pre-processing-7b98bb678c-v5vhh Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.047s (2.047s including waiting). Image size: 125353965 bytes. 44m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal SuccessfulCreate replicaset/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77 Created pod: e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-q8kz9 43m Normal AddedLabel gateway/e2e-worker-w3-ce69f3 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-ce69f3 43m Normal AddedLabel gateway/e2e-worker-w4-ad96a4 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-ad96a4 43m Normal ScalingReplicaSet deployment/e2e-worker-w6-01b24e-openshift-default Scaled up replica set e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77 from 0 to 1 43m Normal AddedLabel gateway/e2e-worker-w6-01b24e Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-01b24e 43m Normal AddedLabel gateway/e2e-worker-w1-419d0a Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-419d0a 43m Normal AddedLabel gateway/e2e-worker-w0-ae63bf Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-ae63bf 43m Normal AddedLabel gateway/e2e-worker-w5-60577a Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-60577a 43m Normal AddedLabel gateway/e2e-worker-w2-b623ca Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-b623ca 43m Normal AddedInterface pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-d2xbp Add eth0 [10.132.0.52/23] from ovn-kubernetes 43m Normal SuccessfulCreate replicaset/e2e-worker-w2-b623ca-openshift-default-68847758ff Created pod: e2e-worker-w2-b623ca-openshift-default-68847758ff-d2xbp 43m Normal SuccessfulCreate replicaset/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 Created pod: e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-4g5wx 43m Normal ScalingReplicaSet deployment/e2e-worker-w1-419d0a-openshift-default Scaled up replica set e2e-worker-w1-419d0a-openshift-default-55766cf857 from 0 to 1 43m Normal SuccessfulCreate replicaset/e2e-worker-w3-ce69f3-openshift-default-6f98547647 Created pod: e2e-worker-w3-ce69f3-openshift-default-6f98547647-5bwkh 43m Normal ScalingReplicaSet deployment/e2e-worker-w5-60577a-openshift-default Scaled up replica set e2e-worker-w5-60577a-openshift-default-7c8cb87747 from 0 to 1 43m Normal ScalingReplicaSet deployment/e2e-worker-w3-ce69f3-openshift-default Scaled up replica set e2e-worker-w3-ce69f3-openshift-default-6f98547647 from 0 to 1 43m Normal AddedInterface pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-q8kz9 Add eth0 [10.132.0.51/23] from ovn-kubernetes 43m Normal SuccessfulCreate replicaset/e2e-worker-w5-60577a-openshift-default-7c8cb87747 Created pod: e2e-worker-w5-60577a-openshift-default-7c8cb87747-jggxr 43m Normal ScalingReplicaSet deployment/e2e-worker-w2-b623ca-openshift-default Scaled up replica set e2e-worker-w2-b623ca-openshift-default-68847758ff from 0 to 1 43m Normal ScalingReplicaSet deployment/e2e-worker-w0-ae63bf-openshift-default Scaled up replica set e2e-worker-w0-ae63bf-openshift-default-556d46cdff from 0 to 1 43m Normal SuccessfulCreate replicaset/e2e-worker-w0-ae63bf-openshift-default-556d46cdff Created pod: e2e-worker-w0-ae63bf-openshift-default-556d46cdff-8pztb 43m Normal ScalingReplicaSet deployment/e2e-worker-w4-ad96a4-openshift-default Scaled up replica set e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 from 0 to 1 43m Normal SuccessfulCreate replicaset/e2e-worker-w1-419d0a-openshift-default-55766cf857 Created pod: e2e-worker-w1-419d0a-openshift-default-55766cf857-lr27f 43m Normal AddedInterface pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-jggxr Add eth0 [10.132.0.53/23] from ovn-kubernetes 43m Normal AddedInterface pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-8pztb Add eth0 [10.133.0.37/23] from ovn-kubernetes 43m Normal AddedInterface pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-lr27f Add eth0 [10.132.0.54/23] from ovn-kubernetes 43m Normal AddedInterface pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-4g5wx Add eth0 [10.132.0.55/23] from ovn-kubernetes 43m Normal AddedInterface pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-5bwkh Add eth0 [10.133.0.36/23] from ovn-kubernetes 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-ce69f3-598446f65b Created pod: payload-processing-e2e-worker-w3-ce69f3-598446f65b-z7qlw 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-ce69f3 Scaled up replica set payload-processing-e2e-worker-w3-ce69f3-598446f65b from 0 to 1 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-ce69f3 Scaled up replica set payload-pre-processing-e2e-worker-w3-ce69f3-f897b54f5 from 0 to 1 43m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-ce69f3-f897b54f5-hv6zc Add eth0 [10.134.0.39/23] from ovn-kubernetes 43m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-ce69f3-f897b54f5-hv6zc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-ce69f3-f897b54f5 Created pod: payload-pre-processing-e2e-worker-w3-ce69f3-f897b54f5-hv6zc 43m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-ce69f3-598446f65b Error creating: pods "payload-processing-e2e-worker-w3-ce69f3-598446f65b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-ce69f3: serviceaccount "payload-processing-e2e-worker-w3-ce69f3" not found 43m Normal AddedInterface pod/payload-processing-e2e-worker-w3-ce69f3-598446f65b-z7qlw Add eth0 [10.134.0.40/23] from ovn-kubernetes 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-ce69f3-f897b54f5 Error creating: pods "payload-pre-processing-e2e-worker-w3-ce69f3-f897b54f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-ce69f3: serviceaccount "payload-processing-e2e-worker-w3-ce69f3" not found 43m Normal Pulled pod/payload-processing-e2e-worker-w3-ce69f3-598446f65b-z7qlw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-processing-e2e-worker-w3-ce69f3-598446f65b-z7qlw Container started 43m Normal Started pod/payload-pre-processing-e2e-worker-w3-ce69f3-f897b54f5-hv6zc Container started 43m Normal Created pod/payload-pre-processing-e2e-worker-w3-ce69f3-f897b54f5-hv6zc Container created 43m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-01b24e-7b4dc88878 Error creating: pods "payload-processing-e2e-worker-w6-01b24e-7b4dc88878-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-01b24e: serviceaccount "payload-processing-e2e-worker-w6-01b24e" not found 43m Normal Created pod/payload-processing-e2e-worker-w3-ce69f3-598446f65b-z7qlw Container created 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-60577a-785b689d99 Error creating: pods "payload-pre-processing-e2e-worker-w5-60577a-785b689d99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-60577a: serviceaccount "payload-processing-e2e-worker-w5-60577a" not found 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-01b24e Scaled up replica set payload-pre-processing-e2e-worker-w6-01b24e-f7447dc8c from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-60577a-785b689d99 Created pod: payload-pre-processing-e2e-worker-w5-60577a-785b689d99-wh2hs 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-60577a Scaled up replica set payload-pre-processing-e2e-worker-w5-60577a-785b689d99 from 0 to 1 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-01b24e Scaled up replica set payload-processing-e2e-worker-w6-01b24e-7b4dc88878 from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-01b24e-7b4dc88878 Created pod: payload-processing-e2e-worker-w6-01b24e-7b4dc88878-4fbph 43m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-60577a-646d79c448 Error creating: pods "payload-processing-e2e-worker-w5-60577a-646d79c448-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-60577a: serviceaccount "payload-processing-e2e-worker-w5-60577a" not found 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-60577a-646d79c448 Created pod: payload-processing-e2e-worker-w5-60577a-646d79c448-2w78j 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-60577a Scaled up replica set payload-processing-e2e-worker-w5-60577a-646d79c448 from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-01b24e-f7447dc8c Created pod: payload-pre-processing-e2e-worker-w6-01b24e-f7447dc8c-z5lr6 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-01b24e-f7447dc8c Error creating: pods "payload-pre-processing-e2e-worker-w6-01b24e-f7447dc8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-01b24e: serviceaccount "payload-processing-e2e-worker-w6-01b24e" not found 43m Normal AddedInterface pod/payload-processing-e2e-worker-w5-60577a-646d79c448-2w78j Add eth0 [10.134.0.45/23] from ovn-kubernetes 43m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-60577a-785b689d99-wh2hs Add eth0 [10.134.0.44/23] from ovn-kubernetes 43m Normal Started pod/payload-pre-processing-e2e-worker-w6-01b24e-f7447dc8c-z5lr6 Container started 43m Normal Created pod/payload-pre-processing-e2e-worker-w6-01b24e-f7447dc8c-z5lr6 Container created 43m Normal Created pod/payload-processing-e2e-worker-w6-01b24e-7b4dc88878-4fbph Container created 43m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-01b24e-f7447dc8c-z5lr6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-processing-e2e-worker-w6-01b24e-7b4dc88878-4fbph Container started 43m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-01b24e-f7447dc8c-z5lr6 Add eth0 [10.134.0.42/23] from ovn-kubernetes 43m Normal AddedInterface pod/payload-processing-e2e-worker-w6-01b24e-7b4dc88878-4fbph Add eth0 [10.134.0.43/23] from ovn-kubernetes 43m Normal Started pod/payload-processing-e2e-worker-w5-60577a-646d79c448-2w78j Container started 43m Normal Created pod/payload-processing-e2e-worker-w5-60577a-646d79c448-2w78j Container created 43m Normal Pulled pod/payload-processing-e2e-worker-w5-60577a-646d79c448-2w78j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-419d0a Scaled up replica set payload-processing-e2e-worker-w1-419d0a-688c6dfd6 from 0 to 1 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-419d0a-57688785ff Error creating: pods "payload-pre-processing-e2e-worker-w1-419d0a-57688785ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-419d0a: serviceaccount "payload-processing-e2e-worker-w1-419d0a" not found 43m Normal Started pod/payload-pre-processing-e2e-worker-w5-60577a-785b689d99-wh2hs Container started 43m Normal Created pod/payload-pre-processing-e2e-worker-w5-60577a-785b689d99-wh2hs Container created 43m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-60577a-785b689d99-wh2hs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Pulled pod/payload-processing-e2e-worker-w6-01b24e-7b4dc88878-4fbph Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-419d0a Scaled up replica set payload-pre-processing-e2e-worker-w1-419d0a-57688785ff from 0 to 1 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-b623ca-5c58d97f4c Error creating: pods "payload-pre-processing-e2e-worker-w2-b623ca-5c58d97f4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-b623ca: serviceaccount "payload-processing-e2e-worker-w2-b623ca" not found 43m Normal Pulled pod/payload-processing-e2e-worker-w1-419d0a-688c6dfd6-8qmrl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-b623ca Scaled up replica set payload-processing-e2e-worker-w2-b623ca-867779ffc6 from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-b623ca-867779ffc6 Created pod: payload-processing-e2e-worker-w2-b623ca-867779ffc6-6dg44 43m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-b623ca-867779ffc6 Error creating: pods "payload-processing-e2e-worker-w2-b623ca-867779ffc6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-b623ca: serviceaccount "payload-processing-e2e-worker-w2-b623ca" not found 43m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-419d0a-688c6dfd6 Error creating: pods "payload-processing-e2e-worker-w1-419d0a-688c6dfd6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-419d0a: serviceaccount "payload-processing-e2e-worker-w1-419d0a" not found 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-b623ca Scaled up replica set payload-pre-processing-e2e-worker-w2-b623ca-5c58d97f4c from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-b623ca-5c58d97f4c Created pod: payload-pre-processing-e2e-worker-w2-b623ca-5c58d97f4c-p6qfh 43m Normal Started pod/payload-processing-e2e-worker-w1-419d0a-688c6dfd6-8qmrl Container started 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-419d0a-57688785ff Created pod: payload-pre-processing-e2e-worker-w1-419d0a-57688785ff-fjn4j 43m Normal Created pod/payload-processing-e2e-worker-w1-419d0a-688c6dfd6-8qmrl Container created 43m Normal AddedInterface pod/payload-processing-e2e-worker-w1-419d0a-688c6dfd6-8qmrl Add eth0 [10.134.0.47/23] from ovn-kubernetes 43m Normal Started pod/payload-pre-processing-e2e-worker-w1-419d0a-57688785ff-fjn4j Container started 43m Normal Created pod/payload-pre-processing-e2e-worker-w1-419d0a-57688785ff-fjn4j Container created 43m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-419d0a-57688785ff-fjn4j Add eth0 [10.134.0.46/23] from ovn-kubernetes 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-419d0a-688c6dfd6 Created pod: payload-processing-e2e-worker-w1-419d0a-688c6dfd6-8qmrl 43m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-419d0a-57688785ff-fjn4j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-ad96a4-746867c7d7-lnlgf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Pulled pod/payload-processing-e2e-worker-w4-ad96a4-7988fc87b-bpvc9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-processing-e2e-worker-w2-b623ca-867779ffc6-6dg44 Container created 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-ad96a4 Scaled up replica set payload-pre-processing-e2e-worker-w4-ad96a4-746867c7d7 from 0 to 1 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-ad96a4-746867c7d7 Error creating: pods "payload-pre-processing-e2e-worker-w4-ad96a4-746867c7d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-ad96a4: serviceaccount "payload-processing-e2e-worker-w4-ad96a4" not found 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-ad96a4 Scaled up replica set payload-processing-e2e-worker-w4-ad96a4-7988fc87b from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-ad96a4-7988fc87b Created pod: payload-processing-e2e-worker-w4-ad96a4-7988fc87b-bpvc9 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-ae63bf Scaled up replica set payload-pre-processing-e2e-worker-w0-ae63bf-76d99cc4d7 from 0 to 1 43m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-ad96a4-746867c7d7-lnlgf Add eth0 [10.132.0.56/23] from ovn-kubernetes 43m Normal Started pod/payload-pre-processing-e2e-worker-w2-b623ca-5c58d97f4c-p6qfh Container started 43m Normal Created pod/payload-pre-processing-e2e-worker-w2-b623ca-5c58d97f4c-p6qfh Container created 43m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-b623ca-5c58d97f4c-p6qfh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Pulled pod/payload-processing-e2e-worker-w2-b623ca-867779ffc6-6dg44 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-processing-e2e-worker-w2-b623ca-867779ffc6-6dg44 Container started 43m Normal AddedInterface pod/payload-processing-e2e-worker-w4-ad96a4-7988fc87b-bpvc9 Add eth0 [10.134.0.51/23] from ovn-kubernetes 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-ad96a4-746867c7d7 Created pod: payload-pre-processing-e2e-worker-w4-ad96a4-746867c7d7-lnlgf 43m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-ad96a4-7988fc87b Error creating: pods "payload-processing-e2e-worker-w4-ad96a4-7988fc87b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-ad96a4: serviceaccount "payload-processing-e2e-worker-w4-ad96a4" not found 43m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-b623ca-5c58d97f4c-p6qfh Add eth0 [10.134.0.49/23] from ovn-kubernetes 43m Normal AddedInterface pod/payload-processing-e2e-worker-w2-b623ca-867779ffc6-6dg44 Add eth0 [10.134.0.50/23] from ovn-kubernetes 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-ae63bf Scaled up replica set payload-processing-e2e-worker-w0-ae63bf-56c67d7d7b from 0 to 1 43m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-ae63bf-56c67d7d7b Error creating: pods "payload-processing-e2e-worker-w0-ae63bf-56c67d7d7b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-ae63bf: serviceaccount "payload-processing-e2e-worker-w0-ae63bf" not found 43m Normal Started pod/payload-pre-processing-e2e-worker-w4-ad96a4-746867c7d7-lnlgf Container started 43m Normal Created pod/payload-pre-processing-e2e-worker-w0-ae63bf-76d99cc4d7-fdtz4 Container created 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-ae63bf-56c67d7d7b Created pod: payload-processing-e2e-worker-w0-ae63bf-56c67d7d7b-mgtt5 43m Normal Started pod/payload-processing-e2e-worker-w4-ad96a4-7988fc87b-bpvc9 Container started 43m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-ae63bf-76d99cc4d7-fdtz4 Add eth0 [10.132.0.58/23] from ovn-kubernetes 43m Normal Created pod/payload-processing-e2e-worker-w0-ae63bf-56c67d7d7b-mgtt5 Container created 43m Normal Created pod/payload-processing-e2e-worker-w4-ad96a4-7988fc87b-bpvc9 Container created 43m Normal Pulled pod/payload-processing-e2e-worker-w0-ae63bf-56c67d7d7b-mgtt5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-ae63bf-76d99cc4d7-fdtz4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-processing-e2e-worker-w0-ae63bf-56c67d7d7b-mgtt5 Container started 43m Normal Created pod/payload-pre-processing-e2e-worker-w4-ad96a4-746867c7d7-lnlgf Container created 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-ae63bf-76d99cc4d7 Created pod: payload-pre-processing-e2e-worker-w0-ae63bf-76d99cc4d7-fdtz4 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-ae63bf-76d99cc4d7 Error creating: pods "payload-pre-processing-e2e-worker-w0-ae63bf-76d99cc4d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-ae63bf: serviceaccount "payload-processing-e2e-worker-w0-ae63bf" not found 43m Normal Started pod/payload-pre-processing-e2e-worker-w0-ae63bf-76d99cc4d7-fdtz4 Container started 43m Normal AddedInterface pod/payload-processing-e2e-worker-w0-ae63bf-56c67d7d7b-mgtt5 Add eth0 [10.132.0.57/23] from ovn-kubernetes 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-01b24e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-01b24e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal Created pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-5bwkh Container created 43m Warning Unhealthy pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-5bwkh Readiness probe failed: Get "http://10.133.0.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Warning ProbeError pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-5bwkh Readiness probe error: Get "http://10.133.0.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Normal Started pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-5bwkh Container started 43m Normal Pulled pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-5bwkh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Warning Unhealthy pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-5bwkh Startup probe failed: Get "http://10.133.0.36:15021/healthz/ready": dial tcp 10.133.0.36:15021: connect: connection refused 42m Warning ProbeError pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-5bwkh Startup probe error: Get "http://10.133.0.36:15021/healthz/ready": dial tcp 10.133.0.36:15021: connect: connection refused... 42m Normal Started pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-jggxr Container started 42m Normal Created pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-jggxr Container created 42m Normal Pulled pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-jggxr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Pulled pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-q8kz9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-q8kz9 Container created 42m Normal Started pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-q8kz9 Container started 42m Normal Created pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-8pztb Container created 42m Normal Started pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-8pztb Container started 42m Normal Pulled pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-8pztb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-60577a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-60577a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning ProbeError pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-jggxr Startup probe error: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-jggxr Startup probe failed: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused 42m Warning Unhealthy pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-q8kz9 Startup probe failed: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused 42m Warning ProbeError pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-q8kz9 Startup probe error: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused... 42m Warning ProbeError pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-lr27f Readiness probe error: Get "http://10.132.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Warning Unhealthy pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-lr27f Readiness probe failed: Get "http://10.132.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Warning Unhealthy pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-8pztb Startup probe failed: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused 42m Warning ProbeError pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-8pztb Startup probe error: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused... 42m Normal Created pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-d2xbp Container created 42m Normal Created pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-4g5wx Container created 42m Normal Started pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-4g5wx Container started 42m Normal Pulled pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-4g5wx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Pulled pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-d2xbp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Started pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-d2xbp Container started 42m Normal Pulled pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-lr27f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-lr27f Container created 42m Normal Started pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-lr27f Container started 42m Warning Unhealthy pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-4g5wx Startup probe failed: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused 42m Warning ProbeError pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-d2xbp Startup probe error: Get "http://10.132.0.52:15021/healthz/ready": dial tcp 10.132.0.52:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-d2xbp Startup probe failed: Get "http://10.132.0.52:15021/healthz/ready": dial tcp 10.132.0.52:15021: connect: connection refused 42m Warning ProbeError pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-4g5wx Startup probe error: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused... 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-ce69f3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-ae63bf-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-ce69f3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning ProbeError pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-lr27f Startup probe error: Get "http://10.132.0.54:15021/healthz/ready": dial tcp 10.132.0.54:15021: connect: connection refused... 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-ae63bf-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-ad96a4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-ad96a4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-419d0a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-419d0a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning Unhealthy pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-lr27f Startup probe failed: Get "http://10.132.0.54:15021/healthz/ready": dial tcp 10.132.0.54:15021: connect: connection refused 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-419d0a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-60577a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-ad96a4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-419d0a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-01b24e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-01b24e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-ad96a4-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-60577a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-ae63bf-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-ae63bf-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-b623ca-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-b623ca-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-ad96a4-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 41m Normal SuccessfulCreate replicaset/e2e-worker-w5-60577a-openshift-default-7c8cb87747 Created pod: e2e-worker-w5-60577a-openshift-default-7c8cb87747-sbl2p 41m Normal ScalingReplicaSet deployment/e2e-worker-w4-ad96a4-openshift-default Scaled up replica set e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 from 1 to 2 41m Normal SuccessfulCreate replicaset/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 Created pod: e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-wkhn2 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-60577a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 41m Normal ScalingReplicaSet deployment/e2e-worker-w5-60577a-openshift-default Scaled up replica set e2e-worker-w5-60577a-openshift-default-7c8cb87747 from 1 to 2 41m Normal Pulled pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-wkhn2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Started pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-wkhn2 Container started 41m Normal Created pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-wkhn2 Container created 41m Normal AddedInterface pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-wkhn2 Add eth0 [10.133.0.43/23] from ovn-kubernetes 41m Normal AddedInterface pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-sbl2p Add eth0 [10.133.0.42/23] from ovn-kubernetes 41m Normal Pulled pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-sbl2p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-sbl2p Container created 41m Normal Started pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-sbl2p Container started 41m Warning Unhealthy pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-sbl2p Startup probe failed: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused 41m Warning ProbeError pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-sbl2p Startup probe error: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused... 41m Warning ProbeError pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-wkhn2 Startup probe error: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-wkhn2 Startup probe failed: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused 41m Normal Started pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-bdpxx Container started 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-419d0a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 41m Normal SuccessfulCreate replicaset/e2e-worker-w1-419d0a-openshift-default-55766cf857 Created pod: e2e-worker-w1-419d0a-openshift-default-55766cf857-fqcxb 41m Normal ScalingReplicaSet deployment/e2e-worker-w6-01b24e-openshift-default Scaled up replica set e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77 from 1 to 2 41m Normal SuccessfulCreate replicaset/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77 Created pod: e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-bdpxx 41m Normal Created pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-bdpxx Container created 41m Normal ScalingReplicaSet deployment/e2e-worker-w1-419d0a-openshift-default Scaled up replica set e2e-worker-w1-419d0a-openshift-default-55766cf857 from 1 to 2 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-01b24e-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 41m Normal Pulled pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-bdpxx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-bdpxx Add eth0 [10.133.0.44/23] from ovn-kubernetes 41m Normal Started pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-fqcxb Container started 41m Normal AddedInterface pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-fqcxb Add eth0 [10.133.0.45/23] from ovn-kubernetes 41m Normal Pulled pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-fqcxb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-fqcxb Container created 41m Warning Unhealthy pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-fqcxb Startup probe failed: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused 41m Warning ProbeError pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-bdpxx Startup probe error: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused... 41m Warning ProbeError pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-fqcxb Startup probe error: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-bdpxx Startup probe failed: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused 41m Normal ScalingReplicaSet deployment/e2e-worker-w2-b623ca-openshift-default Scaled up replica set e2e-worker-w2-b623ca-openshift-default-68847758ff from 1 to 2 41m Normal Pulled pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-bsbrr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal SuccessfulCreate replicaset/e2e-worker-w2-b623ca-openshift-default-68847758ff Created pod: e2e-worker-w2-b623ca-openshift-default-68847758ff-bsbrr 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-b623ca-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 41m Normal AddedInterface pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-bsbrr Add eth0 [10.133.0.46/23] from ovn-kubernetes 41m Normal Created pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-bsbrr Container created 41m Normal Started pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-bsbrr Container started 41m Normal Pulled pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-28g9t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal SuccessfulCreate replicaset/e2e-worker-w0-ae63bf-openshift-default-556d46cdff Created pod: e2e-worker-w0-ae63bf-openshift-default-556d46cdff-28g9t 41m Normal AddedInterface pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-28g9t Add eth0 [10.132.0.71/23] from ovn-kubernetes 41m Normal Started pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-28g9t Container started 41m Normal Created pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-28g9t Container created 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ae63bf-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 41m Normal ScalingReplicaSet deployment/e2e-worker-w0-ae63bf-openshift-default Scaled up replica set e2e-worker-w0-ae63bf-openshift-default-556d46cdff from 1 to 2 41m Warning Unhealthy pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-bsbrr Startup probe failed: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused 41m Warning ProbeError pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-bsbrr Startup probe error: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused... 41m Warning ProbeError pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-28g9t Startup probe error: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-28g9t Startup probe failed: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused 40m Normal ScalingReplicaSet deployment/e2e-worker-w6-01b24e-openshift-default Scaled up replica set e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77 from 2 to 3 40m Normal SuccessfulCreate replicaset/e2e-worker-w5-60577a-openshift-default-7c8cb87747 Created pod: e2e-worker-w5-60577a-openshift-default-7c8cb87747-ncnjw 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-ad96a4-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 40m Normal SuccessfulCreate replicaset/e2e-worker-w1-419d0a-openshift-default-55766cf857 Created pod: e2e-worker-w1-419d0a-openshift-default-55766cf857-6snr9 40m Normal SuccessfulCreate replicaset/e2e-worker-w1-419d0a-openshift-default-55766cf857 Created pod: e2e-worker-w1-419d0a-openshift-default-55766cf857-nkc9c 40m Normal SuccessfulCreate replicaset/e2e-worker-w3-ce69f3-openshift-default-6f98547647 Created pod: e2e-worker-w3-ce69f3-openshift-default-6f98547647-fpzgc 40m Normal ScalingReplicaSet deployment/e2e-worker-w1-419d0a-openshift-default Scaled up replica set e2e-worker-w1-419d0a-openshift-default-55766cf857 from 2 to 4 40m Normal ScalingReplicaSet deployment/e2e-worker-w3-ce69f3-openshift-default Scaled up replica set e2e-worker-w3-ce69f3-openshift-default-6f98547647 from 1 to 2 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-01b24e-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-ce69f3-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 40m Normal SuccessfulCreate replicaset/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77 Created pod: e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-d7zl9 40m Normal Pulling pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-d7zl9 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 40m Normal AddedInterface pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-d7zl9 Add eth0 [10.134.0.61/23] from ovn-kubernetes 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-419d0a-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-60577a-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 40m Normal ScalingReplicaSet deployment/e2e-worker-w5-60577a-openshift-default Scaled up replica set e2e-worker-w5-60577a-openshift-default-7c8cb87747 from 2 to 3 40m Normal Pulling pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-ddzt2 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 40m Normal AddedInterface pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-ncnjw Add eth0 [10.134.0.62/23] from ovn-kubernetes 40m Normal Created pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-nkc9c Container created 40m Normal Pulled pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-fpzgc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-fpzgc Add eth0 [10.132.0.85/23] from ovn-kubernetes 40m Normal Pulled pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-nkc9c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-6snr9 Add eth0 [10.134.0.63/23] from ovn-kubernetes 40m Normal Pulling pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-ncnjw Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 40m Normal AddedInterface pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-nkc9c Add eth0 [10.132.0.84/23] from ovn-kubernetes 40m Normal Started pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-fpzgc Container started 40m Normal Created pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-fpzgc Container created 40m Normal Pulling pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-6snr9 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 40m Normal AddedInterface pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-ddzt2 Add eth0 [10.134.0.64/23] from ovn-kubernetes 40m Normal AddedInterface pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-gqmnf Add eth0 [10.132.0.86/23] from ovn-kubernetes 40m Normal Pulled pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-gqmnf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-gqmnf Container created 40m Normal Started pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-gqmnf Container started 40m Normal ScalingReplicaSet deployment/e2e-worker-w4-ad96a4-openshift-default Scaled up replica set e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 from 2 to 4 40m Normal Started pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-nkc9c Container started 40m Normal SuccessfulCreate replicaset/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 Created pod: e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-ddzt2 40m Normal SuccessfulCreate replicaset/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 Created pod: e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-gqmnf 40m Normal Created pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-ncnjw Container created 40m Normal Pulled pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-ddzt2 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.727s (4.727s including waiting). Image size: 183555333 bytes. 40m Normal Created pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-6snr9 Container created 40m Normal Created pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-d7zl9 Container created 40m Normal Started pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-d7zl9 Container started 40m Normal Started pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-6snr9 Container started 40m Normal Pulled pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-d7zl9 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 5.281s (5.281s including waiting). Image size: 183555333 bytes. 40m Normal Started pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-ncnjw Container started 40m Normal Pulled pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-ncnjw Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.821s (4.821s including waiting). Image size: 183555333 bytes. 40m Normal Pulled pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-6snr9 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.999s (4.999s including waiting). Image size: 183555333 bytes. 40m Normal Started pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-ddzt2 Container started 40m Normal Created pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-ddzt2 Container created 40m Warning ProbeError pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-fpzgc Startup probe error: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused... 40m Warning ProbeError pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-nkc9c Startup probe error: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-fpzgc Startup probe failed: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused 40m Warning Unhealthy pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-nkc9c Startup probe failed: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-gqmnf Startup probe error: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-gqmnf Startup probe failed: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-fpzgc Startup probe error: Get "http://10.132.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Warning Unhealthy pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-nkc9c Startup probe failed: Get "http://10.132.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Warning ProbeError pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-nkc9c Startup probe error: Get "http://10.132.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Warning Unhealthy pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-fpzgc Startup probe failed: Get "http://10.132.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Warning Unhealthy pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-ncnjw Startup probe failed: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-ncnjw Startup probe error: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused... 40m Warning ProbeError pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-d7zl9 Startup probe error: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-d7zl9 Startup probe failed: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused 40m Warning Unhealthy pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-ddzt2 Startup probe failed: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused 40m Warning Unhealthy pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-6snr9 Startup probe failed: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-6snr9 Startup probe error: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused... 40m Warning ProbeError pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-ddzt2 Startup probe error: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused... 40m Normal ScalingReplicaSet deployment/e2e-worker-w6-01b24e-openshift-default Scaled up replica set e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77 from 3 to 4 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-01b24e-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 40m Normal Pulled pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-pjznf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-pjznf Add eth0 [10.132.0.89/23] from ovn-kubernetes 40m Normal SuccessfulCreate replicaset/e2e-worker-w2-b623ca-openshift-default-68847758ff Created pod: e2e-worker-w2-b623ca-openshift-default-68847758ff-kqc8x 40m Normal SuccessfulCreate replicaset/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77 Created pod: e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-pjznf 40m Normal ScalingReplicaSet deployment/e2e-worker-w2-b623ca-openshift-default Scaled up replica set e2e-worker-w2-b623ca-openshift-default-68847758ff from 2 to 4 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-b623ca-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 40m Normal SuccessfulCreate replicaset/e2e-worker-w2-b623ca-openshift-default-68847758ff Created pod: e2e-worker-w2-b623ca-openshift-default-68847758ff-5dpgx 40m Normal Started pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-kqc8x Container started 40m Normal Started pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-pjznf Container started 40m Normal Created pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-pjznf Container created 40m Normal Created pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-kqc8x Container created 40m Normal Pulled pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-kqc8x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-kqc8x Add eth0 [10.134.0.65/23] from ovn-kubernetes 40m Normal Started pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-5dpgx Container started 40m Normal Created pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-5dpgx Container created 40m Normal Pulled pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-5dpgx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-5dpgx Add eth0 [10.133.0.49/23] from ovn-kubernetes 40m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-5bc884dd87 from 1 to 4 40m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-frx85 40m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-qdh96 40m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-q4g9n 40m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 40m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-q4g9n Add eth0 [10.134.0.66/23] from ovn-kubernetes 40m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-q4g9n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-frx85 Add eth0 [10.133.0.51/23] from ovn-kubernetes 40m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-qdh96 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-qdh96 Add eth0 [10.132.0.90/23] from ovn-kubernetes 40m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-frx85 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-frx85 Container created 40m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-frx85 Container started 40m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-q4g9n Container started 40m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-qdh96 Container created 40m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-q4g9n Container created 40m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-qdh96 Container started 40m Warning Unhealthy pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-kqc8x Startup probe failed: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-kqc8x Startup probe error: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-pjznf Startup probe failed: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-5dpgx Startup probe error: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-5dpgx Startup probe failed: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-pjznf Startup probe error: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused... 40m Warning ProbeError pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-pjznf Startup probe error: Get "http://10.132.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Warning Unhealthy pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-pjznf Startup probe failed: Get "http://10.132.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-q4g9n Startup probe failed: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused 40m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-q4g9n Startup probe error: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused... 40m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-frx85 Startup probe error: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused... 40m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-frx85 Startup probe failed: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused 40m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-qdh96 Startup probe error: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused... 40m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-qdh96 Startup probe failed: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused 40m Normal ScalingReplicaSet deployment/e2e-worker-w5-60577a-openshift-default Scaled up replica set e2e-worker-w5-60577a-openshift-default-7c8cb87747 from 3 to 5 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-60577a-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 40m Normal SuccessfulCreate replicaset/e2e-worker-w5-60577a-openshift-default-7c8cb87747 Created pod: e2e-worker-w5-60577a-openshift-default-7c8cb87747-npzrt 40m Normal SuccessfulCreate replicaset/e2e-worker-w5-60577a-openshift-default-7c8cb87747 Created pod: e2e-worker-w5-60577a-openshift-default-7c8cb87747-fz2s8 40m Normal Pulled pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-fz2s8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ae63bf-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 40m Normal Created pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-fz2s8 Container created 40m Normal AddedInterface pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-vwbfz Add eth0 [10.134.0.68/23] from ovn-kubernetes 40m Normal AddedInterface pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-fz2s8 Add eth0 [10.134.0.67/23] from ovn-kubernetes 40m Normal Created pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-vwbfz Container created 40m Normal Started pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-vwbfz Container started 40m Normal Started pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-npzrt Container started 40m Normal AddedInterface pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-npw9t Add eth0 [10.132.0.94/23] from ovn-kubernetes 40m Normal Pulled pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-npw9t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal SuccessfulCreate replicaset/e2e-worker-w0-ae63bf-openshift-default-556d46cdff Created pod: e2e-worker-w0-ae63bf-openshift-default-556d46cdff-vwbfz 40m Normal Started pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-fz2s8 Container started 40m Normal ScalingReplicaSet deployment/e2e-worker-w0-ae63bf-openshift-default Scaled up replica set e2e-worker-w0-ae63bf-openshift-default-556d46cdff from 2 to 4 40m Normal Created pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-npw9t Container created 40m Normal Pulled pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-vwbfz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal SuccessfulCreate replicaset/e2e-worker-w0-ae63bf-openshift-default-556d46cdff Created pod: e2e-worker-w0-ae63bf-openshift-default-556d46cdff-npw9t 40m Normal AddedInterface pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-npzrt Add eth0 [10.132.0.92/23] from ovn-kubernetes 40m Normal Pulled pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-npzrt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-npzrt Container created 40m Normal Started pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-npw9t Container started 40m Warning Unhealthy pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-fz2s8 Startup probe failed: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-fz2s8 Startup probe error: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-vwbfz Startup probe failed: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-vwbfz Startup probe error: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-fz2s8 Startup probe failed: Get "http://10.134.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Warning Unhealthy pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-npzrt Startup probe failed: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-npzrt Startup probe error: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused... 40m Warning ProbeError pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-fz2s8 Startup probe error: Get "http://10.134.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Warning Unhealthy pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-npw9t Startup probe failed: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-npw9t Startup probe error: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused... 40m Warning ProbeError pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-npzrt Startup probe error: Get "http://10.132.0.92:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Warning Unhealthy pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-npzrt Startup probe failed: Get "http://10.132.0.92:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-ce69f3-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 40m Normal Pulled pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-t8wnk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal SuccessfulCreate replicaset/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77 Created pod: e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-lf4jg 40m Normal SuccessfulCreate replicaset/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77 Created pod: e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-t8wnk 40m Normal SuccessfulCreate replicaset/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77 Created pod: e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-57r8q 40m Normal AddedInterface pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-t8wnk Add eth0 [10.133.0.56/23] from ovn-kubernetes 40m Normal Pulled pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-57r8q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-01b24e-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 40m Normal ScalingReplicaSet deployment/e2e-worker-w3-ce69f3-openshift-default Scaled up replica set e2e-worker-w3-ce69f3-openshift-default-6f98547647 from 2 to 4 40m Normal AddedInterface pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-lf4jg Add eth0 [10.132.0.96/23] from ovn-kubernetes 40m Normal ScalingReplicaSet deployment/e2e-worker-w6-01b24e-openshift-default Scaled up replica set e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77 from 4 to 7 40m Normal SuccessfulCreate replicaset/e2e-worker-w3-ce69f3-openshift-default-6f98547647 Created pod: e2e-worker-w3-ce69f3-openshift-default-6f98547647-whfvm 40m Normal AddedInterface pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-57r8q Add eth0 [10.134.0.69/23] from ovn-kubernetes 40m Normal Created pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-t8wnk Container created 40m Normal SuccessfulCreate replicaset/e2e-worker-w3-ce69f3-openshift-default-6f98547647 Created pod: e2e-worker-w3-ce69f3-openshift-default-6f98547647-mn4mp 40m Normal Created pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-lf4jg Container created 40m Normal Pulled pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-lf4jg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-whfvm Add eth0 [10.134.0.70/23] from ovn-kubernetes 40m Normal Pulled pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-whfvm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-whfvm Container created 40m Normal Started pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-whfvm Container started 40m Normal Started pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-lf4jg Container started 40m Normal Started pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-t8wnk Container started 40m Normal Started pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-57r8q Container started 40m Normal Created pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-57r8q Container created 40m Normal AddedInterface pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-mn4mp Add eth0 [10.134.0.71/23] from ovn-kubernetes 40m Normal Pulled pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-mn4mp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-mn4mp Container created 40m Normal Started pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-mn4mp Container started 39m Warning ProbeError pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-t8wnk Startup probe error: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-t8wnk Startup probe failed: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused 39m Warning Unhealthy pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-lf4jg Startup probe failed: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused 39m Warning Unhealthy pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-57r8q Startup probe failed: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused 39m Warning ProbeError pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-lf4jg Startup probe error: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused... 39m Warning ProbeError pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-57r8q Startup probe error: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-mn4mp Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused 39m Warning ProbeError pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-whfvm Startup probe error: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-whfvm Startup probe failed: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused 39m Warning ProbeError pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-mn4mp Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused... 39m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 39m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 39m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-zt46p 39m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-zt46p Add eth0 [10.132.0.99/23] from ovn-kubernetes 39m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-zt46p Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 39m Normal SuccessfulCreate replicaset/e2e-worker-w1-419d0a-openshift-default-55766cf857 Created pod: e2e-worker-w1-419d0a-openshift-default-55766cf857-jssx7 39m Normal SuccessfulCreate replicaset/e2e-worker-w2-b623ca-openshift-default-68847758ff Created pod: e2e-worker-w2-b623ca-openshift-default-68847758ff-g8pb5 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-419d0a-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 39m Normal SuccessfulCreate replicaset/e2e-worker-w1-419d0a-openshift-default-55766cf857 Created pod: e2e-worker-w1-419d0a-openshift-default-55766cf857-2ch45 39m Normal SuccessfulCreate replicaset/e2e-worker-w2-b623ca-openshift-default-68847758ff Created pod: e2e-worker-w2-b623ca-openshift-default-68847758ff-6z6zf 39m Normal ScalingReplicaSet deployment/e2e-worker-w2-b623ca-openshift-default Scaled up replica set e2e-worker-w2-b623ca-openshift-default-68847758ff from 4 to 7 39m Normal SuccessfulCreate replicaset/e2e-worker-w2-b623ca-openshift-default-68847758ff Created pod: e2e-worker-w2-b623ca-openshift-default-68847758ff-2d82j 39m Normal SuccessfulCreate replicaset/e2e-worker-w1-419d0a-openshift-default-55766cf857 Created pod: e2e-worker-w1-419d0a-openshift-default-55766cf857-tmlkg 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-b623ca-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 39m Normal ScalingReplicaSet deployment/e2e-worker-w1-419d0a-openshift-default Scaled up replica set e2e-worker-w1-419d0a-openshift-default-55766cf857 from 4 to 8 39m Normal SuccessfulCreate replicaset/e2e-worker-w1-419d0a-openshift-default-55766cf857 Created pod: e2e-worker-w1-419d0a-openshift-default-55766cf857-6x6nn 39m Normal AddedInterface pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-tmlkg Add eth0 [10.133.0.59/23] from ovn-kubernetes 39m Normal Started pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-tmlkg Container started 39m Normal Created pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-jssx7 Container created 39m Normal Pulled pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-jssx7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-jssx7 Add eth0 [10.133.0.60/23] from ovn-kubernetes 39m Normal Started pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-jssx7 Container started 39m Normal Created pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-tmlkg Container created 39m Normal Pulled pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-tmlkg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-2ch45 Add eth0 [10.134.0.75/23] from ovn-kubernetes 39m Normal AddedInterface pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-g8pb5 Add eth0 [10.134.0.72/23] from ovn-kubernetes 39m Normal AddedInterface pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-6x6nn Add eth0 [10.134.0.74/23] from ovn-kubernetes 39m Normal Started pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-g8pb5 Container started 39m Normal Created pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-g8pb5 Container created 39m Normal Pulled pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-g8pb5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-6z6zf Add eth0 [10.134.0.73/23] from ovn-kubernetes 39m Normal Pulled pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-6z6zf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-6z6zf Container created 39m Normal AddedInterface pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-2d82j Add eth0 [10.132.0.100/23] from ovn-kubernetes 39m Normal Started pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-6z6zf Container started 39m Normal Pulled pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-2ch45 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Pulled pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-2d82j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-2d82j Container created 39m Normal Started pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-6x6nn Container started 39m Normal Created pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-6x6nn Container created 39m Normal Pulled pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-6x6nn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Started pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-2d82j Container started 39m Normal Started pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-2ch45 Container started 39m Normal Created pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-2ch45 Container created 39m Warning ProbeError pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-tmlkg Startup probe error: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused... 39m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-zt46p Container started 39m Warning Unhealthy pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-jssx7 Startup probe failed: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused 39m Warning ProbeError pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-jssx7 Startup probe error: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-tmlkg Startup probe failed: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused 39m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-zt46p Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 9.412s (9.412s including waiting). Image size: 185476477 bytes. 39m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-zt46p Container created 39m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-zt46p Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-zt46p Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning ProbeError pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-2d82j Startup probe error: Get "http://10.132.0.100:15021/healthz/ready": dial tcp 10.132.0.100:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-2d82j Startup probe failed: Get "http://10.132.0.100:15021/healthz/ready": dial tcp 10.132.0.100:15021: connect: connection refused 39m Warning Unhealthy pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-6z6zf Startup probe failed: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused 39m Warning ProbeError pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-6z6zf Startup probe error: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-2ch45 Startup probe failed: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused 39m Warning ProbeError pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-2ch45 Startup probe error: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused... 39m Warning ProbeError pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-g8pb5 Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-g8pb5 Startup probe failed: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused 39m Warning Unhealthy pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-6x6nn Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused 39m Warning ProbeError pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-6x6nn Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused... 39m Normal SuccessfulCreate replicaset/e2e-worker-w5-60577a-openshift-default-7c8cb87747 Created pod: e2e-worker-w5-60577a-openshift-default-7c8cb87747-m9f79 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-60577a-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 39m Normal ScalingReplicaSet deployment/e2e-worker-w1-419d0a-openshift-default Scaled up replica set e2e-worker-w1-419d0a-openshift-default-55766cf857 from 8 to 9 39m Normal SuccessfulCreate replicaset/e2e-worker-w5-60577a-openshift-default-7c8cb87747 Created pod: e2e-worker-w5-60577a-openshift-default-7c8cb87747-w5hjp 39m Normal SuccessfulCreate replicaset/e2e-worker-w1-419d0a-openshift-default-55766cf857 Created pod: e2e-worker-w1-419d0a-openshift-default-55766cf857-rkbbc 39m Normal ScalingReplicaSet deployment/e2e-worker-w5-60577a-openshift-default Scaled up replica set e2e-worker-w5-60577a-openshift-default-7c8cb87747 from 5 to 9 39m Normal SuccessfulCreate replicaset/e2e-worker-w5-60577a-openshift-default-7c8cb87747 Created pod: e2e-worker-w5-60577a-openshift-default-7c8cb87747-2dmhf 39m Normal SuccessfulCreate replicaset/e2e-worker-w5-60577a-openshift-default-7c8cb87747 Created pod: e2e-worker-w5-60577a-openshift-default-7c8cb87747-jrrrp 39m Normal AddedInterface pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-d6kpm Add eth0 [10.134.0.77/23] from ovn-kubernetes 39m Normal SuccessfulCreate replicaset/e2e-worker-w0-ae63bf-openshift-default-556d46cdff Created pod: e2e-worker-w0-ae63bf-openshift-default-556d46cdff-d6kpm 39m Normal AddedInterface pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-w5hjp Add eth0 [10.133.0.62/23] from ovn-kubernetes 39m Normal Created pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-2dmhf Container created 39m Normal Pulled pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-2dmhf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Started pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-2dmhf Container started 39m Normal AddedInterface pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-2dmhf Add eth0 [10.132.0.105/23] from ovn-kubernetes 39m Normal AddedInterface pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-rkbbc Add eth0 [10.132.0.106/23] from ovn-kubernetes 39m Normal ScalingReplicaSet deployment/e2e-worker-w0-ae63bf-openshift-default Scaled up replica set e2e-worker-w0-ae63bf-openshift-default-556d46cdff from 4 to 5 39m Normal Created pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-rkbbc Container created 39m Normal Pulled pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-d6kpm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ae63bf-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 39m Normal Started pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-w5hjp Container started 39m Normal Started pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-rkbbc Container started 39m Normal Created pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-w5hjp Container created 39m Normal AddedInterface pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-jrrrp Add eth0 [10.134.0.76/23] from ovn-kubernetes 39m Normal Pulled pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-jrrrp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-jrrrp Container created 39m Normal Started pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-jrrrp Container started 39m Normal Pulled pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-w5hjp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Started pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-m9f79 Container started 39m Normal Pulled pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-rkbbc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-m9f79 Add eth0 [10.133.0.63/23] from ovn-kubernetes 39m Normal Pulled pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-m9f79 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-m9f79 Container created 39m Normal Started pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-d6kpm Container started 39m Normal Created pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-d6kpm Container created 39m Warning Unhealthy pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-jrrrp Startup probe failed: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused 39m Warning Unhealthy pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-d6kpm Startup probe failed: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused 39m Warning Unhealthy pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-w5hjp Startup probe failed: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused 39m Warning ProbeError pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-w5hjp Startup probe error: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-m9f79 Startup probe failed: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused 39m Warning ProbeError pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-rkbbc Startup probe error: Get "http://10.132.0.106:15021/healthz/ready": dial tcp 10.132.0.106:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-rkbbc Startup probe failed: Get "http://10.132.0.106:15021/healthz/ready": dial tcp 10.132.0.106:15021: connect: connection refused 39m Warning ProbeError pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-m9f79 Startup probe error: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused... 39m Warning ProbeError pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-jrrrp Startup probe error: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-2dmhf Startup probe failed: Get "http://10.132.0.105:15021/healthz/ready": dial tcp 10.132.0.105:15021: connect: connection refused 39m Warning ProbeError pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-2dmhf Startup probe error: Get "http://10.132.0.105:15021/healthz/ready": dial tcp 10.132.0.105:15021: connect: connection refused... 39m Warning ProbeError pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-d6kpm Startup probe error: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused... 39m Normal SuccessfulCreate replicaset/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 Created pod: e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-75rps 39m Normal SuccessfulCreate replicaset/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 Created pod: e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-fmf6b 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-ad96a4-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 39m Normal ScalingReplicaSet deployment/e2e-worker-w4-ad96a4-openshift-default Scaled up replica set e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 from 4 to 6 39m Normal AddedInterface pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-75rps Add eth0 [10.133.0.65/23] from ovn-kubernetes 39m Normal Pulled pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-75rps Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-75rps Container created 39m Normal Started pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-75rps Container started 39m Normal AddedInterface pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-fmf6b Add eth0 [10.134.0.78/23] from ovn-kubernetes 39m Normal Pulled pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-fmf6b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-fmf6b Container created 39m Normal Started pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-fmf6b Container started 39m Warning Unhealthy pod/payload-pre-processing-7b98bb678c-v5vhh Readiness probe failed: timeout: failed to connect service "10.134.0.37:50052" within 1s: context deadline exceeded 39m Warning Unhealthy pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-75rps Startup probe failed: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused 39m Warning ProbeError pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-75rps Startup probe error: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-fmf6b Startup probe failed: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused 39m Warning ProbeError pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-fmf6b Startup probe error: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused... 39m Normal SuccessfulCreate replicaset/e2e-worker-w2-b623ca-openshift-default-68847758ff (combined from similar events): Created pod: e2e-worker-w2-b623ca-openshift-default-68847758ff-smclj 39m Normal SuccessfulCreate replicaset/e2e-worker-w2-b623ca-openshift-default-68847758ff Created pod: e2e-worker-w2-b623ca-openshift-default-68847758ff-rbvpb 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-b623ca-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 39m Normal ScalingReplicaSet deployment/e2e-worker-w2-b623ca-openshift-default Scaled up replica set e2e-worker-w2-b623ca-openshift-default-68847758ff from 7 to 10 39m Normal SuccessfulCreate replicaset/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77 Created pod: e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-9z5vs 39m Normal SuccessfulCreate replicaset/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77 Created pod: e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-gsk6q 39m Normal SuccessfulCreate replicaset/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77 (combined from similar events): Created pod: e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-vz4wm 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-01b24e-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 39m Normal SuccessfulCreate replicaset/e2e-worker-w2-b623ca-openshift-default-68847758ff Created pod: e2e-worker-w2-b623ca-openshift-default-68847758ff-5frq8 39m Normal ScalingReplicaSet deployment/e2e-worker-w6-01b24e-openshift-default Scaled up replica set e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77 from 7 to 10 39m Normal Started pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-rbvpb Container started 39m Normal ScalingReplicaSet deployment/e2e-worker-w3-ce69f3-openshift-default Scaled up replica set e2e-worker-w3-ce69f3-openshift-default-6f98547647 from 4 to 6 39m Normal Started pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-vz4wm Container started 39m Normal Pulled pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-5frq8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-5frq8 Add eth0 [10.133.0.67/23] from ovn-kubernetes 39m Normal Created pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-9z5vs Container created 39m Normal Pulled pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-9z5vs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-9z5vs Add eth0 [10.134.0.79/23] from ovn-kubernetes 39m Normal Created pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-qxlwv Container created 39m Normal Started pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-smclj Container started 39m Normal Created pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-smclj Container created 39m Normal Pulled pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-smclj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-smclj Add eth0 [10.134.0.80/23] from ovn-kubernetes 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-ce69f3-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 39m Normal AddedInterface pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-qxlwv Add eth0 [10.132.0.117/23] from ovn-kubernetes 39m Normal Created pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-vz4wm Container created 39m Normal Created pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-rbvpb Container created 39m Normal Pulled pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-rbvpb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-rbvpb Add eth0 [10.132.0.116/23] from ovn-kubernetes 39m Normal SuccessfulCreate replicaset/e2e-worker-w3-ce69f3-openshift-default-6f98547647 Created pod: e2e-worker-w3-ce69f3-openshift-default-6f98547647-lkn8z 39m Normal Pulled pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-vz4wm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Started pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-9z5vs Container started 39m Normal AddedInterface pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-gsk6q Add eth0 [10.132.0.115/23] from ovn-kubernetes 39m Normal Pulled pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-gsk6q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-gsk6q Container created 39m Normal Started pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-gsk6q Container started 39m Normal SuccessfulCreate replicaset/e2e-worker-w3-ce69f3-openshift-default-6f98547647 Created pod: e2e-worker-w3-ce69f3-openshift-default-6f98547647-qxlwv 39m Normal Pulled pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-qxlwv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-vz4wm Add eth0 [10.133.0.66/23] from ovn-kubernetes 39m Normal Started pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-lkn8z Container started 39m Normal Created pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-5frq8 Container created 39m Normal Started pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-5frq8 Container started 39m Normal Created pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-lkn8z Container created 39m Normal AddedInterface pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-lkn8z Add eth0 [10.133.0.68/23] from ovn-kubernetes 39m Normal Pulled pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-lkn8z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Started pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-qxlwv Container started 38m Warning ProbeError pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-5frq8 Startup probe error: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-9z5vs Startup probe failed: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused 38m Warning Unhealthy pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-5frq8 Startup probe failed: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused 38m Warning Unhealthy pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-vz4wm Startup probe failed: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused 38m Warning ProbeError pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-vz4wm Startup probe error: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused... 38m Warning ProbeError pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-9z5vs Startup probe error: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused... 38m Warning ProbeError pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-lkn8z Startup probe error: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-smclj Startup probe failed: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused 38m Warning ProbeError pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-gsk6q Startup probe error: Get "http://10.132.0.115:15021/healthz/ready": dial tcp 10.132.0.115:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-gsk6q Startup probe failed: Get "http://10.132.0.115:15021/healthz/ready": dial tcp 10.132.0.115:15021: connect: connection refused 38m Warning ProbeError pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-smclj Startup probe error: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-rbvpb Startup probe failed: Get "http://10.132.0.116:15021/healthz/ready": dial tcp 10.132.0.116:15021: connect: connection refused 38m Warning ProbeError pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-rbvpb Startup probe error: Get "http://10.132.0.116:15021/healthz/ready": dial tcp 10.132.0.116:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-lkn8z Startup probe failed: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused 38m Warning ProbeError pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-qxlwv Startup probe error: Get "http://10.132.0.117:15021/healthz/ready": dial tcp 10.132.0.117:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-qxlwv Startup probe failed: Get "http://10.132.0.117:15021/healthz/ready": dial tcp 10.132.0.117:15021: connect: connection refused 38m Normal SuccessfulCreate replicaset/e2e-worker-w0-ae63bf-openshift-default-556d46cdff Created pod: e2e-worker-w0-ae63bf-openshift-default-556d46cdff-6smj6 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ae63bf-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 38m Normal ScalingReplicaSet deployment/e2e-worker-w0-ae63bf-openshift-default Scaled up replica set e2e-worker-w0-ae63bf-openshift-default-556d46cdff from 5 to 6 38m Normal AddedInterface pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-6smj6 Add eth0 [10.133.0.69/23] from ovn-kubernetes 38m Normal Pulled pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-6smj6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-6smj6 Container created 38m Warning Unhealthy pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-8pztb Readiness probe failed: Get "http://10.133.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning Unhealthy pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-q8kz9 Readiness probe failed: Get "http://10.132.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-8pztb Readiness probe error: Get "http://10.133.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning ProbeError pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-q8kz9 Readiness probe error: Get "http://10.132.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Normal Started pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-6smj6 Container started 38m Warning ProbeError pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-6smj6 Startup probe error: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-6smj6 Startup probe failed: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-60577a-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 38m Normal SuccessfulCreate replicaset/e2e-worker-w5-60577a-openshift-default-7c8cb87747 (combined from similar events): Created pod: e2e-worker-w5-60577a-openshift-default-7c8cb87747-6lg8j 38m Normal ScalingReplicaSet deployment/e2e-worker-w5-60577a-openshift-default Scaled up replica set e2e-worker-w5-60577a-openshift-default-7c8cb87747 from 9 to 10 38m Normal ScalingReplicaSet deployment/e2e-worker-w0-ae63bf-openshift-default Scaled up replica set e2e-worker-w0-ae63bf-openshift-default-556d46cdff from 6 to 10 38m Normal Pulled pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-6lg8j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal SuccessfulCreate replicaset/e2e-worker-w0-ae63bf-openshift-default-556d46cdff (combined from similar events): Created pod: e2e-worker-w0-ae63bf-openshift-default-556d46cdff-t8psh 38m Normal SuccessfulCreate replicaset/e2e-worker-w0-ae63bf-openshift-default-556d46cdff Created pod: e2e-worker-w0-ae63bf-openshift-default-556d46cdff-qj2fm 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ae63bf-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-ce69f3-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 38m Normal SuccessfulCreate replicaset/e2e-worker-w0-ae63bf-openshift-default-556d46cdff Created pod: e2e-worker-w0-ae63bf-openshift-default-556d46cdff-t2llg 38m Normal SuccessfulCreate replicaset/e2e-worker-w0-ae63bf-openshift-default-556d46cdff Created pod: e2e-worker-w0-ae63bf-openshift-default-556d46cdff-gspwm 38m Normal AddedInterface pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-6lg8j Add eth0 [10.134.0.86/23] from ovn-kubernetes 38m Normal AddedInterface pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-dxnzj Add eth0 [10.133.0.73/23] from ovn-kubernetes 38m Normal ScalingReplicaSet deployment/e2e-worker-w3-ce69f3-openshift-default Scaled up replica set e2e-worker-w3-ce69f3-openshift-default-6f98547647 from 6 to 10 38m Normal SuccessfulCreate replicaset/e2e-worker-w1-419d0a-openshift-default-55766cf857 (combined from similar events): Created pod: e2e-worker-w1-419d0a-openshift-default-55766cf857-mkgcz 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-419d0a-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 38m Normal SuccessfulCreate replicaset/e2e-worker-w3-ce69f3-openshift-default-6f98547647 Created pod: e2e-worker-w3-ce69f3-openshift-default-6f98547647-kzwqv 38m Normal SuccessfulCreate replicaset/e2e-worker-w3-ce69f3-openshift-default-6f98547647 Created pod: e2e-worker-w3-ce69f3-openshift-default-6f98547647-dwtpl 38m Normal SuccessfulCreate replicaset/e2e-worker-w3-ce69f3-openshift-default-6f98547647 Created pod: e2e-worker-w3-ce69f3-openshift-default-6f98547647-dxnzj 38m Normal SuccessfulCreate replicaset/e2e-worker-w3-ce69f3-openshift-default-6f98547647 (combined from similar events): Created pod: e2e-worker-w3-ce69f3-openshift-default-6f98547647-jgbnq 38m Normal ScalingReplicaSet deployment/e2e-worker-w1-419d0a-openshift-default Scaled up replica set e2e-worker-w1-419d0a-openshift-default-55766cf857 from 9 to 10 38m Normal Pulled pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-mkgcz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Pulled pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-t2llg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-dxnzj Container created 38m Normal Started pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-dxnzj Container started 38m Normal Pulled pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-dxnzj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-kzwqv Add eth0 [10.132.0.131/23] from ovn-kubernetes 38m Normal AddedInterface pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-mkgcz Add eth0 [10.134.0.87/23] from ovn-kubernetes 38m Normal AddedInterface pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-jgbnq Add eth0 [10.132.0.132/23] from ovn-kubernetes 38m Normal AddedInterface pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-t2llg Add eth0 [10.133.0.74/23] from ovn-kubernetes 38m Normal Started pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-t2llg Container started 38m Normal Created pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-t2llg Container created 38m Normal ScalingReplicaSet deployment/e2e-ait-77930c58-openshift-default Scaled up replica set e2e-ait-77930c58-openshift-default-5d78dcfbdf from 0 to 1 38m Normal Pulled pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-kzwqv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal SuccessfulCreate replicaset/e2e-ait-77930c58-openshift-default-5d78dcfbdf Created pod: e2e-ait-77930c58-openshift-default-5d78dcfbdf-hcqvq 38m Normal AddedLabel gateway/e2e-ait-77930c58 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-77930c58 38m Normal Pulled pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-jgbnq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-mkgcz Container created 38m Normal AddedInterface pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-dwtpl Add eth0 [10.134.0.88/23] from ovn-kubernetes 38m Normal Pulled pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-dwtpl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-qj2fm Add eth0 [10.132.0.133/23] from ovn-kubernetes 38m Normal Started pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-mkgcz Container started 38m Normal Created pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-6lg8j Container created 38m Normal AddedInterface pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-gspwm Add eth0 [10.134.0.89/23] from ovn-kubernetes 38m Normal Created pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-dwtpl Container created 38m Warning Unhealthy pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-jggxr Readiness probe failed: Get "http://10.132.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-jggxr Readiness probe error: Get "http://10.132.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Normal Pulled pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-gspwm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Started pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-6lg8j Container started 38m Normal AddedInterface pod/e2e-ait-77930c58-openshift-default-5d78dcfbdf-hcqvq Add eth0 [10.133.0.75/23] from ovn-kubernetes 38m Normal Created pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-kzwqv Container created 38m Normal Created pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-jgbnq Container created 38m Normal Pulled pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-qj2fm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Warning ProbeError pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-bdpxx Readiness probe error: Get "http://10.133.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Normal Started pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-kzwqv Container started 38m Normal Created pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-qj2fm Container created 38m Warning Unhealthy pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-bdpxx Readiness probe failed: Get "http://10.133.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Normal Created pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-gspwm Container created 38m Normal Started pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-jgbnq Container started 38m Normal Pulled pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-t8psh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-t8psh Add eth0 [10.132.0.134/23] from ovn-kubernetes 38m Normal Started pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-dwtpl Container started 38m Normal Created pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-t8psh Container created 38m Normal Started pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-gspwm Container started 38m Normal Started pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-qj2fm Container started 38m Normal Started pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-t8psh Container started 38m Normal AddedLabel gateway/e2e-ait-2ae6628f Added label istio.io/rev=openshift-gateway to gateway e2e-ait-2ae6628f 38m Normal SuccessfulCreate replicaset/e2e-ait-2ae6628f-openshift-default-856f6c7c6f Created pod: e2e-ait-2ae6628f-openshift-default-856f6c7c6f-q5n4b 38m Normal ScalingReplicaSet deployment/e2e-ait-2ae6628f-openshift-default Scaled up replica set e2e-ait-2ae6628f-openshift-default-856f6c7c6f from 0 to 1 38m Warning Unhealthy pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-t2llg Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused 38m Warning ProbeError pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-t2llg Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused... 38m Normal AddedInterface pod/e2e-ait-2ae6628f-openshift-default-856f6c7c6f-q5n4b Add eth0 [10.134.0.90/23] from ovn-kubernetes 38m Warning ProbeError pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-dxnzj Startup probe error: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused... 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-77930c58 Scaled up replica set payload-processing-e2e-ait-77930c58-f86d9d8fd from 0 to 1 38m Warning Unhealthy pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-dxnzj Startup probe failed: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused 38m Warning FailedCreate replicaset/payload-processing-e2e-ait-77930c58-f86d9d8fd Error creating: pods "payload-processing-e2e-ait-77930c58-f86d9d8fd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-77930c58: serviceaccount "payload-processing-e2e-ait-77930c58" not found 38m Normal Pulled pod/e2e-ait-2ae6628f-openshift-default-856f6c7c6f-q5n4b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-77930c58 Scaled up replica set payload-pre-processing-e2e-ait-77930c58-75d6fb56d7 from 0 to 1 38m Normal Created pod/e2e-ait-2ae6628f-openshift-default-856f6c7c6f-q5n4b Container created 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-77930c58-75d6fb56d7 Error creating: pods "payload-pre-processing-e2e-ait-77930c58-75d6fb56d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-77930c58: serviceaccount "payload-processing-e2e-ait-77930c58" not found 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-77930c58-f86d9d8fd Created pod: payload-processing-e2e-ait-77930c58-f86d9d8fd-gg9bz 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-77930c58-75d6fb56d7 Created pod: payload-pre-processing-e2e-ait-77930c58-75d6fb56d7-s8tgk 38m Normal Started pod/e2e-ait-2ae6628f-openshift-default-856f6c7c6f-q5n4b Container started 38m Warning Unhealthy pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-mkgcz Startup probe failed: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused 38m Warning ProbeError pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-mkgcz Startup probe error: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-6lg8j Startup probe failed: Get "http://10.134.0.86:15021/healthz/ready": dial tcp 10.134.0.86:15021: connect: connection refused 38m Warning ProbeError pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-6lg8j Startup probe error: Get "http://10.134.0.86:15021/healthz/ready": dial tcp 10.134.0.86:15021: connect: connection refused... 38m Normal Created pod/payload-pre-processing-e2e-ait-77930c58-75d6fb56d7-s8tgk Container created 38m Normal AddedInterface pod/payload-pre-processing-e2e-ait-77930c58-75d6fb56d7-s8tgk Add eth0 [10.134.0.92/23] from ovn-kubernetes 38m Normal Started pod/payload-processing-e2e-ait-77930c58-f86d9d8fd-gg9bz Container started 38m Warning ProbeError pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-dwtpl Startup probe error: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-dwtpl Startup probe failed: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused 38m Warning ProbeError pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-gspwm Startup probe error: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused... 38m Normal AddedInterface pod/payload-processing-e2e-ait-77930c58-f86d9d8fd-gg9bz Add eth0 [10.134.0.91/23] from ovn-kubernetes 38m Normal Started pod/payload-pre-processing-e2e-ait-77930c58-75d6fb56d7-s8tgk Container started 38m Normal Pulled pod/payload-processing-e2e-ait-77930c58-f86d9d8fd-gg9bz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Pulled pod/payload-pre-processing-e2e-ait-77930c58-75d6fb56d7-s8tgk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Warning Unhealthy pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-gspwm Startup probe failed: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused 38m Normal Created pod/payload-processing-e2e-ait-77930c58-f86d9d8fd-gg9bz Container created 38m Warning ProbeError pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-jgbnq Startup probe error: Get "http://10.132.0.132:15021/healthz/ready": dial tcp 10.132.0.132:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-jgbnq Startup probe failed: Get "http://10.132.0.132:15021/healthz/ready": dial tcp 10.132.0.132:15021: connect: connection refused 38m Warning ProbeError pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-kzwqv Startup probe error: Get "http://10.132.0.131:15021/healthz/ready": dial tcp 10.132.0.131:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-kzwqv Startup probe failed: Get "http://10.132.0.131:15021/healthz/ready": dial tcp 10.132.0.131:15021: connect: connection refused 37m Warning Unhealthy pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-t8psh Startup probe failed: Get "http://10.132.0.134:15021/healthz/ready": dial tcp 10.132.0.134:15021: connect: connection refused 37m Warning Unhealthy pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-qj2fm Startup probe failed: Get "http://10.132.0.133:15021/healthz/ready": dial tcp 10.132.0.133:15021: connect: connection refused 37m Warning ProbeError pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-qj2fm Startup probe error: Get "http://10.132.0.133:15021/healthz/ready": dial tcp 10.132.0.133:15021: connect: connection refused... 37m Warning ProbeError pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-t8psh Startup probe error: Get "http://10.132.0.134:15021/healthz/ready": dial tcp 10.132.0.134:15021: connect: connection refused... 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-2ae6628f-54ff895c9d Created pod: payload-processing-e2e-ait-2ae6628f-54ff895c9d-4d9pk 37m Warning FailedCreate replicaset/payload-processing-e2e-ait-2ae6628f-54ff895c9d Error creating: pods "payload-processing-e2e-ait-2ae6628f-54ff895c9d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-2ae6628f: serviceaccount "payload-processing-e2e-ait-2ae6628f" not found 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-2ae6628f Scaled up replica set payload-processing-e2e-ait-2ae6628f-54ff895c9d from 0 to 1 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-2ae6628f Scaled up replica set payload-pre-processing-e2e-ait-2ae6628f-6fbdd54f79 from 0 to 1 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-2ae6628f-6fbdd54f79 Created pod: payload-pre-processing-e2e-ait-2ae6628f-6fbdd54f79-qh5cw 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-2ae6628f-6fbdd54f79 Error creating: pods "payload-pre-processing-e2e-ait-2ae6628f-6fbdd54f79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-2ae6628f: serviceaccount "payload-processing-e2e-ait-2ae6628f" not found 37m Normal Started pod/payload-pre-processing-e2e-ait-2ae6628f-6fbdd54f79-qh5cw Container started 37m Normal Created pod/payload-pre-processing-e2e-ait-2ae6628f-6fbdd54f79-qh5cw Container created 37m Normal Pulled pod/payload-pre-processing-e2e-ait-2ae6628f-6fbdd54f79-qh5cw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/payload-pre-processing-e2e-ait-2ae6628f-6fbdd54f79-qh5cw Add eth0 [10.134.0.93/23] from ovn-kubernetes 37m Normal AddedInterface pod/payload-processing-e2e-ait-2ae6628f-54ff895c9d-4d9pk Add eth0 [10.132.0.138/23] from ovn-kubernetes 37m Normal Created pod/payload-processing-e2e-ait-2ae6628f-54ff895c9d-4d9pk Container created 37m Normal Started pod/payload-processing-e2e-ait-2ae6628f-54ff895c9d-4d9pk Container started 37m Normal Pulled pod/payload-processing-e2e-ait-2ae6628f-54ff895c9d-4d9pk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal AddedLabel gateway/e2e-migrate-5e333f16-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-5e333f16-gw 37m Normal ScalingReplicaSet deployment/e2e-migrate-5e333f16-gw-openshift-default Scaled up replica set e2e-migrate-5e333f16-gw-openshift-default-7c796c78f from 0 to 1 37m Normal AddedInterface pod/e2e-migrate-5e333f16-gw-openshift-default-7c796c78f-kpx57 Add eth0 [10.132.0.142/23] from ovn-kubernetes 37m Normal SuccessfulCreate replicaset/e2e-migrate-5e333f16-gw-openshift-default-7c796c78f Created pod: e2e-migrate-5e333f16-gw-openshift-default-7c796c78f-kpx57 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-77930c58-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-77930c58-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-5e333f16 Scaled up replica set payload-pre-processing-e2e-migrate-5e333f16-f5f6bd6dc from 0 to 1 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-5e333f16-f5f6bd6dc Error creating: pods "payload-pre-processing-e2e-migrate-5e333f16-f5f6bd6dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-5e333f16: serviceaccount "payload-processing-e2e-migrate-5e333f16" not found 37m Warning FailedCreate replicaset/payload-processing-e2e-migrate-5e333f16-68c977c4bd Error creating: pods "payload-processing-e2e-migrate-5e333f16-68c977c4bd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-5e333f16: serviceaccount "payload-processing-e2e-migrate-5e333f16" not found 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-5e333f16-68c977c4bd Created pod: payload-processing-e2e-migrate-5e333f16-68c977c4bd-rhgmr 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-5e333f16 Scaled up replica set payload-processing-e2e-migrate-5e333f16-68c977c4bd from 0 to 1 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-5e333f16-f5f6bd6dc Created pod: payload-pre-processing-e2e-migrate-5e333f16-f5f6bd6dc-jfs9n 37m Normal Pulled pod/payload-pre-processing-e2e-migrate-5e333f16-f5f6bd6dc-jfs9n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Started pod/payload-processing-e2e-migrate-5e333f16-68c977c4bd-rhgmr Container started 37m Normal Started pod/payload-pre-processing-e2e-migrate-5e333f16-f5f6bd6dc-jfs9n Container started 37m Normal AddedInterface pod/payload-processing-e2e-migrate-5e333f16-68c977c4bd-rhgmr Add eth0 [10.132.0.145/23] from ovn-kubernetes 37m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-5e333f16-f5f6bd6dc-jfs9n Add eth0 [10.132.0.144/23] from ovn-kubernetes 37m Normal Pulled pod/payload-processing-e2e-migrate-5e333f16-68c977c4bd-rhgmr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Created pod/payload-processing-e2e-migrate-5e333f16-68c977c4bd-rhgmr Container created 37m Normal Created pod/payload-pre-processing-e2e-migrate-5e333f16-f5f6bd6dc-jfs9n Container created 37m Normal ScalingReplicaSet deployment/e2e-ait-229db90d-openshift-default Scaled up replica set e2e-ait-229db90d-openshift-default-8696475765 from 0 to 1 37m Normal SuccessfulCreate replicaset/e2e-ait-229db90d-openshift-default-8696475765 Created pod: e2e-ait-229db90d-openshift-default-8696475765-tkr28 37m Normal AddedLabel gateway/e2e-ait-229db90d Added label istio.io/rev=openshift-gateway to gateway e2e-ait-229db90d 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-2ae6628f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-2ae6628f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning ProbeError pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-fz2s8 Readiness probe error: Get "http://10.134.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning Unhealthy pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-fz2s8 Readiness probe failed: Get "http://10.134.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Normal AddedInterface pod/e2e-ait-229db90d-openshift-default-8696475765-tkr28 Add eth0 [10.134.0.95/23] from ovn-kubernetes 37m Warning ProbeError pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-pjznf Readiness probe error: Get "http://10.132.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning Unhealthy pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-pjznf Readiness probe failed: Get "http://10.132.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning Unhealthy pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-nkc9c Readiness probe failed: Get "http://10.132.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Normal Pulled pod/e2e-ait-229db90d-openshift-default-8696475765-tkr28 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Warning ProbeError pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-nkc9c Readiness probe error: Get "http://10.132.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning ProbeError pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-fmf6b Readiness probe error: Get "http://10.134.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning Unhealthy pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-fmf6b Readiness probe failed: Get "http://10.134.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning Unhealthy pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-npw9t Readiness probe failed: Get "http://10.132.0.94:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning ProbeError pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-npw9t Readiness probe error: Get "http://10.132.0.94:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning Unhealthy pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-npzrt Readiness probe failed: Get "http://10.132.0.92:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning ProbeError pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-npzrt Readiness probe error: Get "http://10.132.0.92:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Normal Created pod/e2e-ait-229db90d-openshift-default-8696475765-tkr28 Container created 37m Warning ProbeError pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-g8pb5 Readiness probe error: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Normal Started pod/e2e-ait-229db90d-openshift-default-8696475765-tkr28 Container started 37m Warning Unhealthy pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-g8pb5 Readiness probe failed: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning Unhealthy pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-6z6zf Readiness probe failed: Get "http://10.134.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning Unhealthy pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-2ch45 Readiness probe failed: Get "http://10.134.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning ProbeError pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-6z6zf Readiness probe error: Get "http://10.134.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning ProbeError pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-2ch45 Readiness probe error: Get "http://10.134.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning Unhealthy pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-6x6nn Readiness probe failed: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning ProbeError pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-6x6nn Readiness probe error: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning ProbeError pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-d2xbp Readiness probe error: Get "http://10.132.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning ProbeError pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-4g5wx Readiness probe error: Get "http://10.132.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning Unhealthy pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-d7zl9 Readiness probe failed: Get "http://10.134.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning Unhealthy pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-4g5wx Readiness probe failed: Get "http://10.132.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning ProbeError pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-d7zl9 Readiness probe error: Get "http://10.134.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning Unhealthy pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-d2xbp Readiness probe failed: Get "http://10.132.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning ProbeError pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-6smj6 Readiness probe error: Get "http://10.133.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning ProbeError pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-6snr9 Readiness probe error: Get "http://10.134.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning ProbeError pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-ddzt2 Readiness probe error: Get "http://10.134.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning Unhealthy pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-ddzt2 Readiness probe failed: Get "http://10.134.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning Unhealthy pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-6smj6 Readiness probe failed: Get "http://10.133.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning Unhealthy pod/e2e-ait-229db90d-openshift-default-8696475765-tkr28 Startup probe failed: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused 37m Warning Unhealthy pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-6snr9 Readiness probe failed: Get "http://10.134.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning ProbeError pod/e2e-ait-229db90d-openshift-default-8696475765-tkr28 Startup probe error: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused... 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-5e333f16-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-5e333f16-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedCreate replicaset/payload-processing-e2e-ait-229db90d-6f45bf4ddb Error creating: pods "payload-processing-e2e-ait-229db90d-6f45bf4ddb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-229db90d: serviceaccount "payload-processing-e2e-ait-229db90d" not found 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-229db90d Scaled up replica set payload-processing-e2e-ait-229db90d-6f45bf4ddb from 0 to 1 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-229db90d-6f45bf4ddb Created pod: payload-processing-e2e-ait-229db90d-6f45bf4ddb-q875f 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-229db90d-7b97c78b67 Error creating: pods "payload-pre-processing-e2e-ait-229db90d-7b97c78b67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-229db90d: serviceaccount "payload-processing-e2e-ait-229db90d" not found 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-229db90d-7b97c78b67 Created pod: payload-pre-processing-e2e-ait-229db90d-7b97c78b67-wn2bb 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-229db90d Scaled up replica set payload-pre-processing-e2e-ait-229db90d-7b97c78b67 from 0 to 1 37m Normal Pulled pod/payload-processing-e2e-ait-229db90d-6f45bf4ddb-q875f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/payload-processing-e2e-ait-229db90d-6f45bf4ddb-q875f Add eth0 [10.132.0.148/23] from ovn-kubernetes 37m Normal Created pod/payload-processing-e2e-ait-229db90d-6f45bf4ddb-q875f Container created 37m Normal Started pod/payload-processing-e2e-ait-229db90d-6f45bf4ddb-q875f Container started 37m Normal Started pod/payload-pre-processing-e2e-ait-229db90d-7b97c78b67-wn2bb Container started 37m Normal Created pod/payload-pre-processing-e2e-ait-229db90d-7b97c78b67-wn2bb Container created 37m Normal Pulled pod/payload-pre-processing-e2e-ait-229db90d-7b97c78b67-wn2bb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/payload-pre-processing-e2e-ait-229db90d-7b97c78b67-wn2bb Add eth0 [10.134.0.96/23] from ovn-kubernetes 37m Normal Pulled pod/e2e-ait-77930c58-openshift-default-5d78dcfbdf-hcqvq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Started pod/e2e-ait-77930c58-openshift-default-5d78dcfbdf-hcqvq Container started 37m Normal Created pod/e2e-ait-77930c58-openshift-default-5d78dcfbdf-hcqvq Container created 37m Normal Killing pod/payload-processing-e2e-ait-2ae6628f-54ff895c9d-4d9pk Stopping container payload-processing 37m Normal Killing pod/payload-pre-processing-e2e-ait-2ae6628f-6fbdd54f79-qh5cw Stopping container payload-pre-processing 37m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: cpu resource utilization (percentage of request) above target 37m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 3 37m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-2zxhf 37m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-2zxhf Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 37m Normal Killing pod/payload-pre-processing-e2e-ait-77930c58-75d6fb56d7-s8tgk Stopping container payload-pre-processing 37m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-2zxhf Add eth0 [10.133.0.78/23] from ovn-kubernetes 37m Normal Killing pod/payload-processing-e2e-ait-77930c58-f86d9d8fd-gg9bz Stopping container payload-processing 37m Normal Killing pod/e2e-ait-2ae6628f-openshift-default-856f6c7c6f-q5n4b Stopping container istio-proxy 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-ad96a4-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 37m Normal SuccessfulCreate replicaset/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 Created pod: e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-p58s7 37m Normal SuccessfulCreate replicaset/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 Created pod: e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-tclm9 37m Warning Unhealthy pod/e2e-ait-77930c58-openshift-default-5d78dcfbdf-hcqvq Startup probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused 37m Normal ScalingReplicaSet deployment/e2e-worker-w4-ad96a4-openshift-default Scaled up replica set e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 from 6 to 10 37m Normal SuccessfulCreate replicaset/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 (combined from similar events): Created pod: e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-8b2zb 37m Normal SuccessfulCreate replicaset/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 Created pod: e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-jrr8j 37m Warning ProbeError pod/e2e-ait-77930c58-openshift-default-5d78dcfbdf-hcqvq Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused... 37m Normal Created pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-8b2zb Container created 37m Normal AddedInterface pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-tclm9 Add eth0 [10.133.0.79/23] from ovn-kubernetes 37m Normal Started pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-tclm9 Container started 37m Normal Started pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-8b2zb Container started 37m Normal Created pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-tclm9 Container created 37m Normal Pulled pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-8b2zb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-8b2zb Add eth0 [10.134.0.99/23] from ovn-kubernetes 37m Normal Pulled pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-tclm9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal AddedLabel gateway/e2e-shared-a-w0-395ccc Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-395ccc 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-5e333f16-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Normal Started pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-jrr8j Container started 37m Normal Created pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-jrr8j Container created 37m Normal Pulled pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-jrr8j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-jrr8j Add eth0 [10.134.0.98/23] from ovn-kubernetes 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-5e333f16-gw-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-395ccc-openshift-default-5dcf9cb54c Created pod: e2e-shared-a-w0-395ccc-openshift-default-5dcf9cb54c-fnlf6 37m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-395ccc-openshift-default Scaled up replica set e2e-shared-a-w0-395ccc-openshift-default-5dcf9cb54c from 0 to 1 37m Normal AddedInterface pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-p58s7 Add eth0 [10.132.0.149/23] from ovn-kubernetes 37m Normal Pulled pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-p58s7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Created pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-p58s7 Container created 37m Normal Started pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-p58s7 Container started 37m Warning Unhealthy pod/e2e-ait-2ae6628f-openshift-default-856f6c7c6f-q5n4b Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-ait-2ae6628f-openshift-default-856f6c7c6f-q5n4b Readiness probe error: HTTP probe failed with statuscode: 503... 37m Normal AddedInterface pod/e2e-shared-a-w0-395ccc-openshift-default-5dcf9cb54c-fnlf6 Add eth0 [10.132.0.150/23] from ovn-kubernetes 37m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-2zxhf Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 5.1s (5.1s including waiting). Image size: 185476477 bytes. 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-77930c58-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-77930c58-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-2zxhf Container created 37m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-2zxhf Container started 37m Normal Killing pod/e2e-ait-77930c58-openshift-default-5d78dcfbdf-hcqvq Stopping container istio-proxy 37m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-2zxhf Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-2zxhf Readiness probe error: HTTP probe failed with statuscode: 503... 37m Normal SuccessfulCreate replicaset/e2e-derive-5cb1c5cd-openshift-default-67897f4645 Created pod: e2e-derive-5cb1c5cd-openshift-default-67897f4645-w4hvx 37m Normal AddedLabel gateway/e2e-derive-5cb1c5cd Added label istio.io/rev=openshift-gateway to gateway e2e-derive-5cb1c5cd 37m Normal ScalingReplicaSet deployment/e2e-derive-5cb1c5cd-openshift-default Scaled up replica set e2e-derive-5cb1c5cd-openshift-default-67897f4645 from 0 to 1 37m Warning ProbeError pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-tclm9 Startup probe error: Get "http://10.133.0.79:15021/healthz/ready": dial tcp 10.133.0.79:15021: connect: connection refused... 37m Normal Started pod/e2e-derive-5cb1c5cd-openshift-default-67897f4645-w4hvx Container started 37m Warning Unhealthy pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-tclm9 Startup probe failed: Get "http://10.133.0.79:15021/healthz/ready": dial tcp 10.133.0.79:15021: connect: connection refused 37m Normal AddedInterface pod/e2e-derive-5cb1c5cd-openshift-default-67897f4645-w4hvx Add eth0 [10.134.0.100/23] from ovn-kubernetes 37m Normal Pulled pod/e2e-derive-5cb1c5cd-openshift-default-67897f4645-w4hvx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Created pod/e2e-derive-5cb1c5cd-openshift-default-67897f4645-w4hvx Container created 37m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w0-395ccc-656669bc5b Error creating: pods "payload-processing-e2e-shared-a-w0-395ccc-656669bc5b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-395ccc: serviceaccount "payload-processing-e2e-shared-a-w0-395ccc" not found 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w0-395ccc-7d4b79d8f9 Error creating: pods "payload-pre-processing-e2e-shared-a-w0-395ccc-7d4b79d8f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-395ccc: serviceaccount "payload-processing-e2e-shared-a-w0-395ccc" not found 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w0-395ccc-656669bc5b Created pod: payload-processing-e2e-shared-a-w0-395ccc-656669bc5b-sg8dz 37m Warning ProbeError pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-p58s7 Startup probe error: Get "http://10.132.0.149:15021/healthz/ready": dial tcp 10.132.0.149:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-p58s7 Startup probe failed: Get "http://10.132.0.149:15021/healthz/ready": dial tcp 10.132.0.149:15021: connect: connection refused 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w0-395ccc Scaled up replica set payload-pre-processing-e2e-shared-a-w0-395ccc-7d4b79d8f9 from 0 to 1 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w0-395ccc-7d4b79d8f9 Created pod: payload-pre-processing-e2e-shared-a-w0-395ccc-7d4b79d8f9-gn47s 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w0-395ccc Scaled up replica set payload-processing-e2e-shared-a-w0-395ccc-656669bc5b from 0 to 1 37m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w0-395ccc-7d4b79d8f9-gn47s Add eth0 [10.132.0.151/23] from ovn-kubernetes 37m Normal Pulled pod/payload-processing-e2e-shared-a-w0-395ccc-656669bc5b-sg8dz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Created pod/payload-pre-processing-e2e-shared-a-w0-395ccc-7d4b79d8f9-gn47s Container created 37m Warning Unhealthy pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-8b2zb Startup probe failed: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused 37m Normal Started pod/payload-pre-processing-e2e-shared-a-w0-395ccc-7d4b79d8f9-gn47s Container started 37m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w0-395ccc-7d4b79d8f9-gn47s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/payload-processing-e2e-shared-a-w0-395ccc-656669bc5b-sg8dz Add eth0 [10.134.0.101/23] from ovn-kubernetes 37m Warning ProbeError pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-jrr8j Startup probe error: Get "http://10.134.0.98:15021/healthz/ready": dial tcp 10.134.0.98:15021: connect: connection refused... 37m Warning ProbeError pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-8b2zb Startup probe error: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused... 37m Normal Created pod/payload-processing-e2e-shared-a-w0-395ccc-656669bc5b-sg8dz Container created 37m Normal Started pod/payload-processing-e2e-shared-a-w0-395ccc-656669bc5b-sg8dz Container started 37m Warning Unhealthy pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-jrr8j Startup probe failed: Get "http://10.134.0.98:15021/healthz/ready": dial tcp 10.134.0.98:15021: connect: connection refused 37m Warning FailedCreate replicaset/payload-processing-e2e-derive-5cb1c5cd-9cc6f45d4 Error creating: pods "payload-processing-e2e-derive-5cb1c5cd-9cc6f45d4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-5cb1c5cd: serviceaccount "payload-processing-e2e-derive-5cb1c5cd" not found 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-5cb1c5cd-8565fccdcf Error creating: pods "payload-pre-processing-e2e-derive-5cb1c5cd-8565fccdcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-5cb1c5cd: serviceaccount "payload-processing-e2e-derive-5cb1c5cd" not found 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-5cb1c5cd Scaled up replica set payload-processing-e2e-derive-5cb1c5cd-9cc6f45d4 from 0 to 1 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-5cb1c5cd-9cc6f45d4 Created pod: payload-processing-e2e-derive-5cb1c5cd-9cc6f45d4-98mg5 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-5cb1c5cd Scaled up replica set payload-pre-processing-e2e-derive-5cb1c5cd-8565fccdcf from 0 to 1 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-5cb1c5cd-8565fccdcf Created pod: payload-pre-processing-e2e-derive-5cb1c5cd-8565fccdcf-s4zvb 37m Normal Created pod/payload-pre-processing-e2e-derive-5cb1c5cd-8565fccdcf-s4zvb Container created 37m Normal Created pod/payload-processing-e2e-derive-5cb1c5cd-9cc6f45d4-98mg5 Container created 37m Normal Pulled pod/payload-processing-e2e-derive-5cb1c5cd-9cc6f45d4-98mg5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Started pod/payload-processing-e2e-derive-5cb1c5cd-9cc6f45d4-98mg5 Container started 37m Normal AddedInterface pod/payload-processing-e2e-derive-5cb1c5cd-9cc6f45d4-98mg5 Add eth0 [10.134.0.102/23] from ovn-kubernetes 37m Normal Started pod/payload-pre-processing-e2e-derive-5cb1c5cd-8565fccdcf-s4zvb Container started 37m Normal Pulled pod/payload-pre-processing-e2e-derive-5cb1c5cd-8565fccdcf-s4zvb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/payload-pre-processing-e2e-derive-5cb1c5cd-8565fccdcf-s4zvb Add eth0 [10.134.0.103/23] from ovn-kubernetes 36m Normal Killing pod/payload-processing-e2e-migrate-5e333f16-68c977c4bd-rhgmr Stopping container payload-processing 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-229db90d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-229db90d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal Killing pod/payload-pre-processing-e2e-migrate-5e333f16-f5f6bd6dc-jfs9n Stopping container payload-pre-processing 36m Warning ProbeError pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-9z5vs Readiness probe error: Get "http://10.134.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 36m Warning Unhealthy pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-9z5vs Readiness probe failed: Get "http://10.134.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 36m Normal Killing pod/payload-pre-processing-e2e-ait-229db90d-7b97c78b67-wn2bb Stopping container payload-pre-processing 36m Normal Killing pod/payload-processing-e2e-ait-229db90d-6f45bf4ddb-q875f Stopping container payload-processing 36m Normal Killing pod/e2e-ait-229db90d-openshift-default-8696475765-tkr28 Stopping container istio-proxy 36m Normal ScalingReplicaSet deployment/e2e-mt-2fd9a937-openshift-default Scaled up replica set e2e-mt-2fd9a937-openshift-default-76594d45b5 from 0 to 1 36m Normal AddedLabel gateway/e2e-mt-2fd9a937 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-2fd9a937 36m Normal SuccessfulCreate replicaset/e2e-mt-2fd9a937-openshift-default-76594d45b5 Created pod: e2e-mt-2fd9a937-openshift-default-76594d45b5-55g7m 36m Warning ProbeError pod/e2e-ait-229db90d-openshift-default-8696475765-tkr28 Readiness probe error: Get "http://10.134.0.95:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 36m Warning Unhealthy pod/e2e-ait-229db90d-openshift-default-8696475765-tkr28 Readiness probe failed: Get "http://10.134.0.95:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 36m Normal AddedInterface pod/e2e-mt-2fd9a937-openshift-default-76594d45b5-55g7m Add eth0 [10.132.0.154/23] from ovn-kubernetes 36m Warning Unhealthy pod/e2e-migrate-5e333f16-gw-openshift-default-7c796c78f-kpx57 Readiness probe failed: Get "http://10.132.0.142:15021/healthz/ready": dial tcp 10.132.0.142:15021: connect: connection refused 36m Warning ProbeError pod/e2e-migrate-5e333f16-gw-openshift-default-7c796c78f-kpx57 Readiness probe error: Get "http://10.132.0.142:15021/healthz/ready": dial tcp 10.132.0.142:15021: connect: connection refused... 36m Normal Started pod/e2e-migrate-5e333f16-gw-openshift-default-7c796c78f-kpx57 Container started 36m Normal Created pod/e2e-migrate-5e333f16-gw-openshift-default-7c796c78f-kpx57 Container created 36m Normal Pulled pod/e2e-migrate-5e333f16-gw-openshift-default-7c796c78f-kpx57 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Killing pod/e2e-migrate-5e333f16-gw-openshift-default-7c796c78f-kpx57 Stopping container istio-proxy 36m Normal AddedLabel gateway/e2e-shared-b-w0-ae425f Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-ae425f 36m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-ae425f-openshift-default Scaled up replica set e2e-shared-b-w0-ae425f-openshift-default-76874f9f6c from 0 to 1 36m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-ae425f-openshift-default-76874f9f6c Created pod: e2e-shared-b-w0-ae425f-openshift-default-76874f9f6c-vszwf 36m Normal AddedInterface pod/e2e-shared-b-w0-ae425f-openshift-default-76874f9f6c-vszwf Add eth0 [10.132.0.155/23] from ovn-kubernetes 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-2fd9a937-7b6c4cdc98 Created pod: payload-processing-e2e-mt-2fd9a937-7b6c4cdc98-p4kpl 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-2fd9a937-7c795b4979 Created pod: payload-pre-processing-e2e-mt-2fd9a937-7c795b4979-8kvvc 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-2fd9a937 Scaled up replica set payload-pre-processing-e2e-mt-2fd9a937-7c795b4979 from 0 to 1 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-2fd9a937-7c795b4979 Error creating: pods "payload-pre-processing-e2e-mt-2fd9a937-7c795b4979-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-2fd9a937: serviceaccount "payload-processing-e2e-mt-2fd9a937" not found 36m Warning FailedCreate replicaset/payload-processing-e2e-mt-2fd9a937-7b6c4cdc98 Error creating: pods "payload-processing-e2e-mt-2fd9a937-7b6c4cdc98-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-2fd9a937: serviceaccount "payload-processing-e2e-mt-2fd9a937" not found 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-2fd9a937 Scaled up replica set payload-processing-e2e-mt-2fd9a937-7b6c4cdc98 from 0 to 1 36m Normal Started pod/payload-pre-processing-e2e-mt-2fd9a937-7c795b4979-8kvvc Container started 36m Normal Pulled pod/payload-processing-e2e-mt-2fd9a937-7b6c4cdc98-p4kpl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Created pod/payload-processing-e2e-mt-2fd9a937-7b6c4cdc98-p4kpl Container created 36m Normal Created pod/payload-pre-processing-e2e-mt-2fd9a937-7c795b4979-8kvvc Container created 36m Normal AddedInterface pod/payload-processing-e2e-mt-2fd9a937-7b6c4cdc98-p4kpl Add eth0 [10.134.0.106/23] from ovn-kubernetes 36m Normal Pulled pod/payload-pre-processing-e2e-mt-2fd9a937-7c795b4979-8kvvc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/payload-pre-processing-e2e-mt-2fd9a937-7c795b4979-8kvvc Add eth0 [10.132.0.157/23] from ovn-kubernetes 36m Normal Started pod/payload-processing-e2e-mt-2fd9a937-7b6c4cdc98-p4kpl Container started 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w0-ae425f-5885c86dd4 Created pod: payload-processing-e2e-shared-b-w0-ae425f-5885c86dd4-4wvh9 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w0-ae425f-79c9ff7db7 Created pod: payload-pre-processing-e2e-shared-b-w0-ae425f-79c9ff7db7-z7nf7 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w0-ae425f Scaled up replica set payload-pre-processing-e2e-shared-b-w0-ae425f-79c9ff7db7 from 0 to 1 36m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w0-ae425f-5885c86dd4 Error creating: pods "payload-processing-e2e-shared-b-w0-ae425f-5885c86dd4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-ae425f: serviceaccount "payload-processing-e2e-shared-b-w0-ae425f" not found 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w0-ae425f-79c9ff7db7 Error creating: pods "payload-pre-processing-e2e-shared-b-w0-ae425f-79c9ff7db7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-ae425f: serviceaccount "payload-processing-e2e-shared-b-w0-ae425f" not found 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w0-ae425f Scaled up replica set payload-processing-e2e-shared-b-w0-ae425f-5885c86dd4 from 0 to 1 36m Normal Created pod/payload-pre-processing-e2e-shared-b-w0-ae425f-79c9ff7db7-z7nf7 Container created 36m Normal Pulled pod/payload-processing-e2e-shared-b-w0-ae425f-5885c86dd4-4wvh9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Created pod/payload-processing-e2e-shared-b-w0-ae425f-5885c86dd4-4wvh9 Container created 36m Normal Started pod/payload-pre-processing-e2e-shared-b-w0-ae425f-79c9ff7db7-z7nf7 Container started 36m Normal AddedInterface pod/payload-processing-e2e-shared-b-w0-ae425f-5885c86dd4-4wvh9 Add eth0 [10.132.0.160/23] from ovn-kubernetes 36m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w0-ae425f-79c9ff7db7-z7nf7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w0-ae425f-79c9ff7db7-z7nf7 Add eth0 [10.132.0.159/23] from ovn-kubernetes 36m Normal Started pod/payload-processing-e2e-shared-b-w0-ae425f-5885c86dd4-4wvh9 Container started 36m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-5jm6z Container started 36m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-f5qx7 36m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-lnkcf Add eth0 [10.134.0.107/23] from ovn-kubernetes 36m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-f5qx7 Add eth0 [10.132.0.162/23] from ovn-kubernetes 36m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-5bc884dd87 from 4 to 8 36m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-lnkcf 36m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-lnkcf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-qj2g6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-5jm6z 36m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-qj2g6 Add eth0 [10.132.0.161/23] from ovn-kubernetes 36m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-qj2g6 36m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 36m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-5jm6z Add eth0 [10.134.0.108/23] from ovn-kubernetes 36m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-5jm6z Container created 36m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-5jm6z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-f5qx7 Container created 36m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-qj2g6 Container started 36m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-lnkcf Container created 36m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-f5qx7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-lnkcf Container started 36m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-qj2g6 Container created 36m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-f5qx7 Container started 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-5cb1c5cd-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-5cb1c5cd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal Killing pod/payload-pre-processing-e2e-derive-5cb1c5cd-8565fccdcf-s4zvb Stopping container payload-pre-processing 36m Normal Killing pod/payload-processing-e2e-derive-5cb1c5cd-9cc6f45d4-98mg5 Stopping container payload-processing 36m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-lnkcf Startup probe error: Get "http://10.134.0.107:15021/healthz/ready": dial tcp 10.134.0.107:15021: connect: connection refused... 36m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 4; reason: cpu resource utilization (percentage of request) above target 36m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-5jm6z Startup probe failed: Get "http://10.134.0.108:15021/healthz/ready": dial tcp 10.134.0.108:15021: connect: connection refused 36m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-5jm6z Startup probe error: Get "http://10.134.0.108:15021/healthz/ready": dial tcp 10.134.0.108:15021: connect: connection refused... 36m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-qj2g6 Startup probe failed: Get "http://10.132.0.161:15021/healthz/ready": dial tcp 10.132.0.161:15021: connect: connection refused 36m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-qj2g6 Startup probe error: Get "http://10.132.0.161:15021/healthz/ready": dial tcp 10.132.0.161:15021: connect: connection refused... 36m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 4 36m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-p5c6s 36m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-f5qx7 Startup probe failed: Get "http://10.132.0.162:15021/healthz/ready": dial tcp 10.132.0.162:15021: connect: connection refused 36m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-f5qx7 Startup probe error: Get "http://10.132.0.162:15021/healthz/ready": dial tcp 10.132.0.162:15021: connect: connection refused... 36m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-lnkcf Startup probe failed: Get "http://10.134.0.107:15021/healthz/ready": dial tcp 10.134.0.107:15021: connect: connection refused 36m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-p5c6s Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 36m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-p5c6s Container started 36m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-p5c6s Container created 36m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-p5c6s Add eth0 [10.133.0.82/23] from ovn-kubernetes 36m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-qj2g6 Startup probe error: Get "http://10.132.0.161:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 36m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-5jm6z Startup probe failed: Get "http://10.134.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 36m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-qj2g6 Startup probe failed: Get "http://10.132.0.161:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 36m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-5jm6z Startup probe error: Get "http://10.134.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 36m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-f5qx7 Startup probe error: Get "http://10.132.0.162:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 36m Normal Killing pod/e2e-derive-5cb1c5cd-openshift-default-67897f4645-w4hvx Stopping container istio-proxy 36m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-f5qx7 Startup probe failed: Get "http://10.132.0.162:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 36m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-fdb355-openshift-default Scaled up replica set e2e-shared-a-w3-fdb355-openshift-default-d67d8d799 from 0 to 1 36m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-fdb355-openshift-default-d67d8d799 Created pod: e2e-shared-a-w3-fdb355-openshift-default-d67d8d799-b6tcd 36m Normal AddedLabel gateway/e2e-shared-a-w3-fdb355 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-fdb355 36m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-5bc884dd87 from 8 to 10 36m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-zl27m 36m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: 36m Normal AddedInterface pod/e2e-shared-a-w3-fdb355-openshift-default-d67d8d799-b6tcd Add eth0 [10.134.0.109/23] from ovn-kubernetes 36m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 (combined from similar events): Created pod: maas-default-gateway-openshift-default-5bc884dd87-5cd48 36m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-zl27m Add eth0 [10.133.0.83/23] from ovn-kubernetes 36m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-zl27m Container created 36m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-zl27m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-zl27m Container started 36m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-5cd48 Add eth0 [10.134.0.110/23] from ovn-kubernetes 36m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-5cd48 Container started 36m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-5cd48 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-5cd48 Container created 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w3-fdb355-699984f479 Error creating: pods "payload-pre-processing-e2e-shared-a-w3-fdb355-699984f479-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-fdb355: serviceaccount "payload-processing-e2e-shared-a-w3-fdb355" not found 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w3-fdb355 Scaled up replica set payload-pre-processing-e2e-shared-a-w3-fdb355-699984f479 from 0 to 1 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w3-fdb355 Scaled up replica set payload-processing-e2e-shared-a-w3-fdb355-75d6685876 from 0 to 1 36m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w3-fdb355-75d6685876 Error creating: pods "payload-processing-e2e-shared-a-w3-fdb355-75d6685876-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-fdb355: serviceaccount "payload-processing-e2e-shared-a-w3-fdb355" not found 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w3-fdb355-75d6685876 Created pod: payload-processing-e2e-shared-a-w3-fdb355-75d6685876-vw9vc 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w3-fdb355-699984f479 Created pod: payload-pre-processing-e2e-shared-a-w3-fdb355-699984f479-clsd4 36m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w3-fdb355-699984f479-clsd4 Add eth0 [10.134.0.112/23] from ovn-kubernetes 36m Normal Pulled pod/payload-processing-e2e-shared-a-w3-fdb355-75d6685876-vw9vc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/payload-processing-e2e-shared-a-w3-fdb355-75d6685876-vw9vc Add eth0 [10.134.0.113/23] from ovn-kubernetes 36m Normal Created pod/payload-processing-e2e-shared-a-w3-fdb355-75d6685876-vw9vc Container created 36m Normal Started pod/payload-processing-e2e-shared-a-w3-fdb355-75d6685876-vw9vc Container started 36m Normal Started pod/payload-pre-processing-e2e-shared-a-w3-fdb355-699984f479-clsd4 Container started 36m Normal Created pod/payload-pre-processing-e2e-shared-a-w3-fdb355-699984f479-clsd4 Container created 36m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w3-fdb355-699984f479-clsd4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-zl27m Startup probe error: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused... 36m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-zl27m Startup probe failed: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused 36m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-5cd48 Startup probe failed: Get "http://10.134.0.110:15021/healthz/ready": dial tcp 10.134.0.110:15021: connect: connection refused 36m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-5cd48 Startup probe error: Get "http://10.134.0.110:15021/healthz/ready": dial tcp 10.134.0.110:15021: connect: connection refused... 36m Normal Started pod/e2e-shared-a-w0-395ccc-openshift-default-5dcf9cb54c-fnlf6 Container started 36m Normal Created pod/e2e-shared-a-w0-395ccc-openshift-default-5dcf9cb54c-fnlf6 Container created 36m Normal Pulled pod/e2e-shared-a-w0-395ccc-openshift-default-5dcf9cb54c-fnlf6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-395ccc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-395ccc-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Normal AddedLabel gateway/e2e-mt-20e66fa4 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-20e66fa4 36m Normal ScalingReplicaSet deployment/e2e-mt-20e66fa4-openshift-default Scaled up replica set e2e-mt-20e66fa4-openshift-default-6ff5b8c94f from 0 to 1 36m Normal SuccessfulCreate replicaset/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f Created pod: e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-bqssp 35m Normal AddedInterface pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-bqssp Add eth0 [10.132.0.163/23] from ovn-kubernetes 35m Warning ProbeError pod/e2e-shared-a-w0-395ccc-openshift-default-5dcf9cb54c-fnlf6 Startup probe error: Get "http://10.132.0.150:15021/healthz/ready": dial tcp 10.132.0.150:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-shared-a-w0-395ccc-openshift-default-5dcf9cb54c-fnlf6 Startup probe failed: Get "http://10.132.0.150:15021/healthz/ready": dial tcp 10.132.0.150:15021: connect: connection refused 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-20e66fa4-997965c87 Error creating: pods "payload-pre-processing-e2e-mt-20e66fa4-997965c87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-20e66fa4: serviceaccount "payload-processing-e2e-mt-20e66fa4" not found 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-20e66fa4-997965c87 Created pod: payload-pre-processing-e2e-mt-20e66fa4-997965c87-7v4rf 35m Warning FailedCreate replicaset/payload-processing-e2e-mt-20e66fa4-54557c4fd4 Error creating: pods "payload-processing-e2e-mt-20e66fa4-54557c4fd4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-20e66fa4: serviceaccount "payload-processing-e2e-mt-20e66fa4" not found 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-20e66fa4-54557c4fd4 Created pod: payload-processing-e2e-mt-20e66fa4-54557c4fd4-j8gh8 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-20e66fa4 Scaled up replica set payload-processing-e2e-mt-20e66fa4-54557c4fd4 from 0 to 1 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-20e66fa4 Scaled up replica set payload-pre-processing-e2e-mt-20e66fa4-997965c87 from 0 to 1 35m Normal Pulled pod/payload-pre-processing-e2e-mt-20e66fa4-997965c87-7v4rf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/payload-pre-processing-e2e-mt-20e66fa4-997965c87-7v4rf Add eth0 [10.132.0.164/23] from ovn-kubernetes 35m Normal Started pod/payload-pre-processing-e2e-mt-20e66fa4-997965c87-7v4rf Container started 35m Normal AddedInterface pod/payload-processing-e2e-mt-20e66fa4-54557c4fd4-j8gh8 Add eth0 [10.132.0.165/23] from ovn-kubernetes 35m Normal Created pod/payload-pre-processing-e2e-mt-20e66fa4-997965c87-7v4rf Container created 35m Normal Pulled pod/payload-processing-e2e-mt-20e66fa4-54557c4fd4-j8gh8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Started pod/payload-processing-e2e-mt-20e66fa4-54557c4fd4-j8gh8 Container started 35m Normal Created pod/payload-processing-e2e-mt-20e66fa4-54557c4fd4-j8gh8 Container created 35m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-qdh96 Readiness probe error: Get "http://10.132.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 35m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-qdh96 Readiness probe failed: Get "http://10.132.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 35m Normal AddedLabel gateway/e2e-shared-b-w3-976b80 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-976b80 35m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-976b80-openshift-default-69d4fb4fc9 Created pod: e2e-shared-b-w3-976b80-openshift-default-69d4fb4fc9-6sdwr 35m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-976b80-openshift-default Scaled up replica set e2e-shared-b-w3-976b80-openshift-default-69d4fb4fc9 from 0 to 1 35m Normal AddedInterface pod/e2e-shared-b-w3-976b80-openshift-default-69d4fb4fc9-6sdwr Add eth0 [10.132.0.166/23] from ovn-kubernetes 35m Warning ProbeError pod/e2e-mt-2fd9a937-openshift-default-76594d45b5-55g7m Readiness probe error: Get "http://10.132.0.154:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 35m Warning Unhealthy pod/e2e-mt-2fd9a937-openshift-default-76594d45b5-55g7m Readiness probe failed: Get "http://10.132.0.154:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 35m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w3-976b80-646cd76644 Error creating: pods "payload-processing-e2e-shared-b-w3-976b80-646cd76644-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-976b80: serviceaccount "payload-processing-e2e-shared-b-w3-976b80" not found 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w3-976b80 Scaled up replica set payload-pre-processing-e2e-shared-b-w3-976b80-6869f686f9 from 0 to 1 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w3-976b80 Scaled up replica set payload-processing-e2e-shared-b-w3-976b80-646cd76644 from 0 to 1 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w3-976b80-6869f686f9 Error creating: pods "payload-pre-processing-e2e-shared-b-w3-976b80-6869f686f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-976b80: serviceaccount "payload-processing-e2e-shared-b-w3-976b80" not found 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w3-976b80-646cd76644 Created pod: payload-processing-e2e-shared-b-w3-976b80-646cd76644-rlkvm 35m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w3-976b80-6869f686f9-9cs2n Add eth0 [10.134.0.119/23] from ovn-kubernetes 35m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w3-976b80-6869f686f9-9cs2n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w3-976b80-6869f686f9 Created pod: payload-pre-processing-e2e-shared-b-w3-976b80-6869f686f9-9cs2n 35m Normal Created pod/payload-pre-processing-e2e-shared-b-w3-976b80-6869f686f9-9cs2n Container created 35m Normal AddedInterface pod/payload-processing-e2e-shared-b-w3-976b80-646cd76644-rlkvm Add eth0 [10.132.0.167/23] from ovn-kubernetes 35m Warning ProbeError pod/e2e-shared-b-w0-ae425f-openshift-default-76874f9f6c-vszwf Readiness probe error: Get "http://10.132.0.155:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 35m Warning Unhealthy pod/e2e-shared-b-w0-ae425f-openshift-default-76874f9f6c-vszwf Readiness probe failed: Get "http://10.132.0.155:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 35m Normal Pulled pod/payload-processing-e2e-shared-b-w3-976b80-646cd76644-rlkvm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Created pod/payload-processing-e2e-shared-b-w3-976b80-646cd76644-rlkvm Container created 35m Normal Started pod/payload-processing-e2e-shared-b-w3-976b80-646cd76644-rlkvm Container started 35m Normal Started pod/payload-pre-processing-e2e-shared-b-w3-976b80-6869f686f9-9cs2n Container started 35m Normal Created pod/e2e-mt-2fd9a937-openshift-default-76594d45b5-55g7m Container created 35m Normal Started pod/e2e-mt-2fd9a937-openshift-default-76594d45b5-55g7m Container started 35m Normal Pulled pod/e2e-mt-2fd9a937-openshift-default-76594d45b5-55g7m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Pulled pod/e2e-shared-b-w0-ae425f-openshift-default-76874f9f6c-vszwf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Started pod/e2e-shared-b-w0-ae425f-openshift-default-76874f9f6c-vszwf Container started 35m Normal Created pod/e2e-shared-b-w0-ae425f-openshift-default-76874f9f6c-vszwf Container created 35m Warning ProbeError pod/e2e-mt-2fd9a937-openshift-default-76594d45b5-55g7m Startup probe error: Get "http://10.132.0.154:15021/healthz/ready": dial tcp 10.132.0.154:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-mt-2fd9a937-openshift-default-76594d45b5-55g7m Startup probe failed: Get "http://10.132.0.154:15021/healthz/ready": dial tcp 10.132.0.154:15021: connect: connection refused 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-ae425f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-ae425f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning ProbeError pod/e2e-shared-b-w0-ae425f-openshift-default-76874f9f6c-vszwf Startup probe error: Get "http://10.132.0.155:15021/healthz/ready": dial tcp 10.132.0.155:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-shared-b-w0-ae425f-openshift-default-76874f9f6c-vszwf Startup probe failed: Get "http://10.132.0.155:15021/healthz/ready": dial tcp 10.132.0.155:15021: connect: connection refused 35m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-59f017-openshift-default-ddc89b68c Created pod: e2e-shared-a-w2-59f017-openshift-default-ddc89b68c-lshmc 35m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-59f017-openshift-default Scaled up replica set e2e-shared-a-w2-59f017-openshift-default-ddc89b68c from 0 to 1 35m Normal AddedLabel gateway/e2e-shared-a-w2-59f017 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-59f017 35m Normal AddedInterface pod/e2e-shared-a-w2-59f017-openshift-default-ddc89b68c-lshmc Add eth0 [10.132.0.168/23] from ovn-kubernetes 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-395ccc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-2fd9a937-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-2fd9a937-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-395ccc-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-59f017 Scaled up replica set payload-pre-processing-e2e-shared-a-w2-59f017-5fb9d9fcf5 from 0 to 1 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-59f017-5fb9d9fcf5 Created pod: payload-pre-processing-e2e-shared-a-w2-59f017-5fb9d9fcf5-5xlfc 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-59f017-5fb9d9fcf5 Error creating: pods "payload-pre-processing-e2e-shared-a-w2-59f017-5fb9d9fcf5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-59f017: serviceaccount "payload-processing-e2e-shared-a-w2-59f017" not found 35m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-59f017-694c87888d Error creating: pods "payload-processing-e2e-shared-a-w2-59f017-694c87888d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-59f017: serviceaccount "payload-processing-e2e-shared-a-w2-59f017" not found 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-59f017-694c87888d Created pod: payload-processing-e2e-shared-a-w2-59f017-694c87888d-lw9vk 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-59f017 Scaled up replica set payload-processing-e2e-shared-a-w2-59f017-694c87888d from 0 to 1 35m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-59f017-5fb9d9fcf5-5xlfc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Pulled pod/payload-processing-e2e-shared-a-w2-59f017-694c87888d-lw9vk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-59f017-694c87888d-lw9vk Add eth0 [10.132.0.169/23] from ovn-kubernetes 35m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-59f017-5fb9d9fcf5-5xlfc Container started 35m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-59f017-5fb9d9fcf5-5xlfc Container created 35m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-59f017-5fb9d9fcf5-5xlfc Add eth0 [10.134.0.122/23] from ovn-kubernetes 35m Normal Started pod/payload-processing-e2e-shared-a-w2-59f017-694c87888d-lw9vk Container started 35m Normal Created pod/payload-processing-e2e-shared-a-w2-59f017-694c87888d-lw9vk Container created 35m Normal Pulled pod/e2e-shared-a-w3-fdb355-openshift-default-d67d8d799-b6tcd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Started pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-bqssp Container started 35m Normal Created pod/e2e-shared-a-w3-fdb355-openshift-default-d67d8d799-b6tcd Container created 35m Normal Started pod/e2e-shared-a-w3-fdb355-openshift-default-d67d8d799-b6tcd Container started 35m Normal Pulled pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-bqssp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-bqssp Container created 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-fdb355-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-fdb355-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-2fd9a937-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-2fd9a937-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-976b80-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-20e66fa4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-20e66fa4-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-976b80-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning Unhealthy pod/e2e-shared-a-w3-fdb355-openshift-default-d67d8d799-b6tcd Startup probe failed: Get "http://10.134.0.109:15021/healthz/ready": dial tcp 10.134.0.109:15021: connect: connection refused 34m Warning ProbeError pod/e2e-shared-a-w3-fdb355-openshift-default-d67d8d799-b6tcd Startup probe error: Get "http://10.134.0.109:15021/healthz/ready": dial tcp 10.134.0.109:15021: connect: connection refused... 34m Warning ProbeError pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-bqssp Startup probe error: Get "http://10.132.0.163:15021/healthz/ready": dial tcp 10.132.0.163:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-bqssp Startup probe failed: Get "http://10.132.0.163:15021/healthz/ready": dial tcp 10.132.0.163:15021: connect: connection refused 34m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-ca9d14-openshift-default-65f96b67bf Created pod: e2e-shared-b-w2-ca9d14-openshift-default-65f96b67bf-klhmt 34m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-ca9d14-openshift-default Scaled up replica set e2e-shared-b-w2-ca9d14-openshift-default-65f96b67bf from 0 to 1 34m Normal AddedLabel gateway/e2e-shared-b-w2-ca9d14 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-ca9d14 34m Normal AddedInterface pod/e2e-shared-b-w2-ca9d14-openshift-default-65f96b67bf-klhmt Add eth0 [10.132.0.171/23] from ovn-kubernetes 34m Normal Pulled pod/e2e-shared-b-w2-ca9d14-openshift-default-65f96b67bf-klhmt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Started pod/e2e-shared-b-w2-ca9d14-openshift-default-65f96b67bf-klhmt Container started 34m Normal Created pod/e2e-shared-b-w2-ca9d14-openshift-default-65f96b67bf-klhmt Container created 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-ae425f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-ae425f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning ProbeError pod/e2e-shared-b-w3-976b80-openshift-default-69d4fb4fc9-6sdwr Readiness probe error: Get "http://10.132.0.166:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-ca9d14 Scaled up replica set payload-processing-e2e-shared-b-w2-ca9d14-746f5b5598 from 0 to 1 34m Warning Unhealthy pod/e2e-shared-b-w3-976b80-openshift-default-69d4fb4fc9-6sdwr Readiness probe failed: Get "http://10.132.0.166:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-ca9d14 Scaled up replica set payload-pre-processing-e2e-shared-b-w2-ca9d14-5c478d7cc5 from 0 to 1 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-ca9d14-5c478d7cc5 Created pod: payload-pre-processing-e2e-shared-b-w2-ca9d14-5c478d7cc5-wtmp4 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-ca9d14-5c478d7cc5 Error creating: pods "payload-pre-processing-e2e-shared-b-w2-ca9d14-5c478d7cc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-ca9d14: serviceaccount "payload-processing-e2e-shared-b-w2-ca9d14" not found 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-ca9d14-746f5b5598 Created pod: payload-processing-e2e-shared-b-w2-ca9d14-746f5b5598-6jt2z 34m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-ca9d14-746f5b5598 Error creating: pods "payload-processing-e2e-shared-b-w2-ca9d14-746f5b5598-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-ca9d14: serviceaccount "payload-processing-e2e-shared-b-w2-ca9d14" not found 34m Normal Pulled pod/payload-processing-e2e-shared-b-w2-ca9d14-746f5b5598-6jt2z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-ca9d14-746f5b5598-6jt2z Add eth0 [10.132.0.173/23] from ovn-kubernetes 34m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-ca9d14-5c478d7cc5-wtmp4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-ca9d14-5c478d7cc5-wtmp4 Add eth0 [10.132.0.174/23] from ovn-kubernetes 34m Normal Started pod/e2e-shared-b-w3-976b80-openshift-default-69d4fb4fc9-6sdwr Container started 34m Normal Created pod/payload-processing-e2e-shared-b-w2-ca9d14-746f5b5598-6jt2z Container created 34m Normal Started pod/payload-processing-e2e-shared-b-w2-ca9d14-746f5b5598-6jt2z Container started 34m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-ca9d14-5c478d7cc5-wtmp4 Container started 34m Normal Pulled pod/e2e-shared-b-w3-976b80-openshift-default-69d4fb4fc9-6sdwr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-ca9d14-5c478d7cc5-wtmp4 Container created 34m Normal Created pod/e2e-shared-b-w3-976b80-openshift-default-69d4fb4fc9-6sdwr Container created 34m Normal Killing pod/payload-pre-processing-e2e-mt-2fd9a937-7c795b4979-8kvvc Stopping container payload-pre-processing 34m Normal Killing pod/payload-processing-e2e-mt-2fd9a937-7b6c4cdc98-p4kpl Stopping container payload-processing 34m Warning ProbeError pod/e2e-shared-b-w3-976b80-openshift-default-69d4fb4fc9-6sdwr Startup probe error: Get "http://10.132.0.166:15021/healthz/ready": dial tcp 10.132.0.166:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-shared-b-w3-976b80-openshift-default-69d4fb4fc9-6sdwr Startup probe failed: Get "http://10.132.0.166:15021/healthz/ready": dial tcp 10.132.0.166:15021: connect: connection refused 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-fdb355-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-ca9d14-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-ca9d14-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-fdb355-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-20e66fa4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-20e66fa4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-ca9d14-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-ca9d14-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Normal Killing pod/e2e-mt-2fd9a937-openshift-default-76594d45b5-55g7m Stopping container istio-proxy 34m Normal SuccessfulCreate replicaset/e2e-api-a-c7880b-openshift-default-56db9c9557 Created pod: e2e-api-a-c7880b-openshift-default-56db9c9557-9vwgs 34m Normal ScalingReplicaSet deployment/e2e-api-a-c7880b-openshift-default Scaled up replica set e2e-api-a-c7880b-openshift-default-56db9c9557 from 0 to 1 34m Normal AddedLabel gateway/e2e-api-a-c7880b Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-c7880b 34m Normal AddedInterface pod/e2e-api-a-c7880b-openshift-default-56db9c9557-9vwgs Add eth0 [10.133.0.85/23] from ovn-kubernetes 34m Warning FailedCreate replicaset/payload-processing-e2e-api-a-c7880b-d5b8b5f94 Error creating: pods "payload-processing-e2e-api-a-c7880b-d5b8b5f94-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-c7880b: serviceaccount "payload-processing-e2e-api-a-c7880b" not found 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-c7880b Scaled up replica set payload-pre-processing-e2e-api-a-c7880b-c46dcb8b7 from 0 to 1 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-c7880b Scaled up replica set payload-processing-e2e-api-a-c7880b-d5b8b5f94 from 0 to 1 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-c7880b-d5b8b5f94 Created pod: payload-processing-e2e-api-a-c7880b-d5b8b5f94-gwr52 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-c7880b-c46dcb8b7 Error creating: pods "payload-pre-processing-e2e-api-a-c7880b-c46dcb8b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-c7880b: serviceaccount "payload-processing-e2e-api-a-c7880b" not found 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-c7880b-c46dcb8b7 Created pod: payload-pre-processing-e2e-api-a-c7880b-c46dcb8b7-dkz2k 34m Normal Started pod/payload-processing-e2e-api-a-c7880b-d5b8b5f94-gwr52 Container started 34m Normal AddedInterface pod/payload-processing-e2e-api-a-c7880b-d5b8b5f94-gwr52 Add eth0 [10.132.0.178/23] from ovn-kubernetes 34m Normal Pulled pod/payload-processing-e2e-api-a-c7880b-d5b8b5f94-gwr52 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Created pod/payload-processing-e2e-api-a-c7880b-d5b8b5f94-gwr52 Container created 34m Normal Pulled pod/payload-pre-processing-e2e-api-a-c7880b-c46dcb8b7-dkz2k Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Started pod/payload-pre-processing-e2e-api-a-c7880b-c46dcb8b7-dkz2k Container started 34m Normal Created pod/payload-pre-processing-e2e-api-a-c7880b-c46dcb8b7-dkz2k Container created 34m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-c7880b-c46dcb8b7-dkz2k Add eth0 [10.132.0.177/23] from ovn-kubernetes 34m Warning Unhealthy pod/e2e-shared-a-w2-59f017-openshift-default-ddc89b68c-lshmc Readiness probe failed: Get "http://10.132.0.168:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 34m Warning ProbeError pod/e2e-shared-a-w2-59f017-openshift-default-ddc89b68c-lshmc Readiness probe error: Get "http://10.132.0.168:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 34m Normal Created pod/e2e-shared-a-w2-59f017-openshift-default-ddc89b68c-lshmc Container created 34m Normal Started pod/e2e-shared-a-w2-59f017-openshift-default-ddc89b68c-lshmc Container started 34m Normal Pulled pod/e2e-shared-a-w2-59f017-openshift-default-ddc89b68c-lshmc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal ScalingReplicaSet deployment/e2e-mt-20e66fa4-openshift-default Scaled up replica set e2e-mt-20e66fa4-openshift-default-6ff5b8c94f from 1 to 2 34m Normal SuccessfulCreate replicaset/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f Created pod: e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-pgfhg 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-976b80-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-20e66fa4-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-976b80-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Normal AddedInterface pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-pgfhg Add eth0 [10.134.0.124/23] from ovn-kubernetes 33m Normal Created pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-pgfhg Container created 33m Normal Pulled pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-pgfhg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Started pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-pgfhg Container started 33m Warning Unhealthy pod/e2e-shared-a-w2-59f017-openshift-default-ddc89b68c-lshmc Startup probe failed: Get "http://10.132.0.168:15021/healthz/ready": dial tcp 10.132.0.168:15021: connect: connection refused 33m Warning ProbeError pod/e2e-shared-a-w2-59f017-openshift-default-ddc89b68c-lshmc Startup probe error: Get "http://10.132.0.168:15021/healthz/ready": dial tcp 10.132.0.168:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-pgfhg Startup probe failed: Get "http://10.134.0.124:15021/healthz/ready": dial tcp 10.134.0.124:15021: connect: connection refused 33m Warning ProbeError pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-pgfhg Startup probe error: Get "http://10.134.0.124:15021/healthz/ready": dial tcp 10.134.0.124:15021: connect: connection refused... 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-59f017-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-59f017-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-59f017-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-59f017-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Normal ScalingReplicaSet deployment/e2e-api-b-95d2ac-openshift-default Scaled up replica set e2e-api-b-95d2ac-openshift-default-7f6cb95d55 from 0 to 1 33m Normal AddedLabel gateway/e2e-api-b-95d2ac Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-95d2ac 33m Normal SuccessfulCreate replicaset/e2e-api-b-95d2ac-openshift-default-7f6cb95d55 Created pod: e2e-api-b-95d2ac-openshift-default-7f6cb95d55-t6xhk 33m Normal AddedInterface pod/e2e-api-b-95d2ac-openshift-default-7f6cb95d55-t6xhk Add eth0 [10.132.0.179/23] from ovn-kubernetes 33m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-95d2ac-74dbfdbb99 Error creating: pods "payload-pre-processing-e2e-api-b-95d2ac-74dbfdbb99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-95d2ac: serviceaccount "payload-processing-e2e-api-b-95d2ac" not found 33m Warning FailedCreate replicaset/payload-processing-e2e-api-b-95d2ac-75f686bc4d Error creating: pods "payload-processing-e2e-api-b-95d2ac-75f686bc4d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-95d2ac: serviceaccount "payload-processing-e2e-api-b-95d2ac" not found 33m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-95d2ac Scaled up replica set payload-pre-processing-e2e-api-b-95d2ac-74dbfdbb99 from 0 to 1 33m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-95d2ac-74dbfdbb99 Created pod: payload-pre-processing-e2e-api-b-95d2ac-74dbfdbb99-fjbtf 33m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-95d2ac Scaled up replica set payload-processing-e2e-api-b-95d2ac-75f686bc4d from 0 to 1 33m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-95d2ac-75f686bc4d Created pod: payload-processing-e2e-api-b-95d2ac-75f686bc4d-v7vw5 33m Normal Created pod/payload-pre-processing-e2e-api-b-95d2ac-74dbfdbb99-fjbtf Container created 33m Normal Started pod/payload-processing-e2e-api-b-95d2ac-75f686bc4d-v7vw5 Container started 33m Normal Created pod/payload-processing-e2e-api-b-95d2ac-75f686bc4d-v7vw5 Container created 33m Normal Pulled pod/payload-processing-e2e-api-b-95d2ac-75f686bc4d-v7vw5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/payload-processing-e2e-api-b-95d2ac-75f686bc4d-v7vw5 Add eth0 [10.132.0.181/23] from ovn-kubernetes 33m Normal Started pod/payload-pre-processing-e2e-api-b-95d2ac-74dbfdbb99-fjbtf Container started 33m Normal Pulled pod/payload-pre-processing-e2e-api-b-95d2ac-74dbfdbb99-fjbtf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-95d2ac-74dbfdbb99-fjbtf Add eth0 [10.132.0.182/23] from ovn-kubernetes 33m Normal Killing pod/payload-processing-e2e-mt-20e66fa4-54557c4fd4-j8gh8 Stopping container payload-processing 33m Normal Killing pod/payload-pre-processing-e2e-mt-20e66fa4-997965c87-7v4rf Stopping container payload-pre-processing 33m Warning Unhealthy pod/payload-pre-processing-e2e-mt-20e66fa4-997965c87-7v4rf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Warning Unhealthy pod/payload-processing-e2e-mt-20e66fa4-54557c4fd4-j8gh8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-frx85 Readiness probe failed: Get "http://10.133.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 33m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-q4g9n Readiness probe failed: Get "http://10.134.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 33m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-q4g9n Readiness probe error: Get "http://10.134.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 33m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-frx85 Readiness probe error: Get "http://10.133.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-95d2ac-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Normal SuccessfulCreate replicaset/e2e-api-a-73f96d-openshift-default-66d877c755 Created pod: e2e-api-a-73f96d-openshift-default-66d877c755-9nvtp 33m Normal ScalingReplicaSet deployment/e2e-api-a-73f96d-openshift-default Scaled up replica set e2e-api-a-73f96d-openshift-default-66d877c755 from 0 to 1 33m Normal AddedLabel gateway/e2e-api-a-73f96d Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-73f96d 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-95d2ac-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Normal AddedInterface pod/e2e-api-a-73f96d-openshift-default-66d877c755-9nvtp Add eth0 [10.133.0.87/23] from ovn-kubernetes 33m Warning FailedCreate replicaset/payload-processing-e2e-api-a-73f96d-767bd9568 Error creating: pods "payload-processing-e2e-api-a-73f96d-767bd9568-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-73f96d: serviceaccount "payload-processing-e2e-api-a-73f96d" not found 33m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-73f96d Scaled up replica set payload-pre-processing-e2e-api-a-73f96d-65c48fbd9f from 0 to 1 33m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-73f96d Scaled up replica set payload-processing-e2e-api-a-73f96d-767bd9568 from 0 to 1 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-73f96d-65c48fbd9f Created pod: payload-pre-processing-e2e-api-a-73f96d-65c48fbd9f-98p49 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-73f96d-65c48fbd9f Error creating: pods "payload-pre-processing-e2e-api-a-73f96d-65c48fbd9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-73f96d: serviceaccount "payload-processing-e2e-api-a-73f96d" not found 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-73f96d-767bd9568 Created pod: payload-processing-e2e-api-a-73f96d-767bd9568-4pjpt 32m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-73f96d-65c48fbd9f-98p49 Add eth0 [10.132.0.184/23] from ovn-kubernetes 32m Normal Created pod/payload-pre-processing-e2e-api-a-73f96d-65c48fbd9f-98p49 Container created 32m Normal Pulled pod/payload-processing-e2e-api-a-73f96d-767bd9568-4pjpt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Started pod/payload-pre-processing-e2e-api-a-73f96d-65c48fbd9f-98p49 Container started 32m Normal Created pod/payload-processing-e2e-api-a-73f96d-767bd9568-4pjpt Container created 32m Normal Started pod/payload-processing-e2e-api-a-73f96d-767bd9568-4pjpt Container started 32m Normal Pulled pod/payload-pre-processing-e2e-api-a-73f96d-65c48fbd9f-98p49 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/payload-processing-e2e-api-a-73f96d-767bd9568-4pjpt Add eth0 [10.134.0.127/23] from ovn-kubernetes 32m Normal Created pod/e2e-api-a-c7880b-openshift-default-56db9c9557-9vwgs Container created 32m Normal Started pod/e2e-api-a-c7880b-openshift-default-56db9c9557-9vwgs Container started 32m Normal Pulled pod/e2e-api-a-c7880b-openshift-default-56db9c9557-9vwgs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Warning Unhealthy pod/e2e-api-a-c7880b-openshift-default-56db9c9557-9vwgs Startup probe failed: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused 32m Warning ProbeError pod/e2e-api-a-c7880b-openshift-default-56db9c9557-9vwgs Startup probe error: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused... 32m Normal Killing pod/payload-pre-processing-e2e-api-a-c7880b-c46dcb8b7-dkz2k Stopping container payload-pre-processing 32m Normal Killing pod/payload-processing-e2e-api-a-c7880b-d5b8b5f94-gwr52 Stopping container payload-processing 32m Normal Killing pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-qxlwv Stopping container istio-proxy 32m Normal SuccessfulDelete replicaset/e2e-worker-w3-ce69f3-openshift-default-6f98547647 Deleted pod: e2e-worker-w3-ce69f3-openshift-default-6f98547647-jgbnq 32m Normal SuccessfulDelete replicaset/e2e-worker-w3-ce69f3-openshift-default-6f98547647 Deleted pod: e2e-worker-w3-ce69f3-openshift-default-6f98547647-qxlwv 32m Normal Killing pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-jgbnq Stopping container istio-proxy 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-ce69f3-openshift-default New size: 8; reason: All metrics below target 32m Normal ScalingReplicaSet deployment/e2e-worker-w3-ce69f3-openshift-default Scaled down replica set e2e-worker-w3-ce69f3-openshift-default-6f98547647 from 10 to 8 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-20e66fa4 Scaled up replica set payload-processing-e2e-mt-20e66fa4-54557c4fd4 from 0 to 1 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-20e66fa4 Scaled up replica set payload-pre-processing-e2e-mt-20e66fa4-997965c87 from 0 to 1 32m Warning ProbeError pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-jgbnq Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning Unhealthy pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-jgbnq Readiness probe failed: HTTP probe failed with statuscode: 503 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-20e66fa4-997965c87 Created pod: payload-pre-processing-e2e-mt-20e66fa4-997965c87-hsg4m 32m Warning FailedCreate replicaset/payload-processing-e2e-mt-20e66fa4-54557c4fd4 Error creating: pods "payload-processing-e2e-mt-20e66fa4-54557c4fd4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-20e66fa4: serviceaccount "payload-processing-e2e-mt-20e66fa4" not found 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-20e66fa4-54557c4fd4 Created pod: payload-processing-e2e-mt-20e66fa4-54557c4fd4-8ffbj 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-20e66fa4-997965c87 Error creating: pods "payload-pre-processing-e2e-mt-20e66fa4-997965c87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-20e66fa4: serviceaccount "payload-processing-e2e-mt-20e66fa4" not found 32m Normal Pulled pod/payload-processing-e2e-mt-20e66fa4-54557c4fd4-8ffbj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Created pod/payload-pre-processing-e2e-mt-20e66fa4-997965c87-hsg4m Container created 32m Normal Started pod/payload-pre-processing-e2e-mt-20e66fa4-997965c87-hsg4m Container started 32m Normal Created pod/payload-processing-e2e-mt-20e66fa4-54557c4fd4-8ffbj Container created 32m Normal AddedInterface pod/payload-pre-processing-e2e-mt-20e66fa4-997965c87-hsg4m Add eth0 [10.132.0.186/23] from ovn-kubernetes 32m Normal AddedInterface pod/payload-processing-e2e-mt-20e66fa4-54557c4fd4-8ffbj Add eth0 [10.132.0.187/23] from ovn-kubernetes 32m Normal Pulled pod/payload-pre-processing-e2e-mt-20e66fa4-997965c87-hsg4m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Started pod/payload-processing-e2e-mt-20e66fa4-54557c4fd4-8ffbj Container started 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-c7880b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-c7880b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-73f96d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-73f96d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-c7880b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-c7880b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Normal ScalingReplicaSet deployment/e2e-api-b-6a6348-openshift-default Scaled up replica set e2e-api-b-6a6348-openshift-default-7bd9c5bc47 from 0 to 1 32m Normal AddedLabel gateway/e2e-api-b-6a6348 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-6a6348 32m Normal SuccessfulCreate replicaset/e2e-api-b-6a6348-openshift-default-7bd9c5bc47 Created pod: e2e-api-b-6a6348-openshift-default-7bd9c5bc47-cmtk5 32m Normal AddedInterface pod/e2e-api-b-6a6348-openshift-default-7bd9c5bc47-cmtk5 Add eth0 [10.132.0.189/23] from ovn-kubernetes 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-6a6348 Scaled up replica set payload-pre-processing-e2e-api-b-6a6348-79bcfb7655 from 0 to 1 32m Warning FailedCreate replicaset/payload-processing-e2e-api-b-6a6348-677c897664 Error creating: pods "payload-processing-e2e-api-b-6a6348-677c897664-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-6a6348: serviceaccount "payload-processing-e2e-api-b-6a6348" not found 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-6a6348-677c897664 Created pod: payload-processing-e2e-api-b-6a6348-677c897664-5q8vf 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-6a6348-79bcfb7655 Created pod: payload-pre-processing-e2e-api-b-6a6348-79bcfb7655-cpxz9 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-6a6348-79bcfb7655 Error creating: pods "payload-pre-processing-e2e-api-b-6a6348-79bcfb7655-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-6a6348: serviceaccount "payload-processing-e2e-api-b-6a6348" not found 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-6a6348 Scaled up replica set payload-processing-e2e-api-b-6a6348-677c897664 from 0 to 1 32m Normal AddedInterface pod/payload-processing-e2e-api-b-6a6348-677c897664-5q8vf Add eth0 [10.132.0.191/23] from ovn-kubernetes 32m Normal Created pod/payload-pre-processing-e2e-api-b-6a6348-79bcfb7655-cpxz9 Container created 32m Normal Pulled pod/payload-pre-processing-e2e-api-b-6a6348-79bcfb7655-cpxz9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Pulled pod/payload-processing-e2e-api-b-6a6348-677c897664-5q8vf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Created pod/payload-processing-e2e-api-b-6a6348-677c897664-5q8vf Container created 32m Normal Started pod/payload-processing-e2e-api-b-6a6348-677c897664-5q8vf Container started 32m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-6a6348-79bcfb7655-cpxz9 Add eth0 [10.132.0.190/23] from ovn-kubernetes 32m Normal Started pod/payload-pre-processing-e2e-api-b-6a6348-79bcfb7655-cpxz9 Container started 32m Warning Unhealthy pod/e2e-api-b-95d2ac-openshift-default-7f6cb95d55-t6xhk Readiness probe failed: Get "http://10.132.0.179:15021/healthz/ready": dial tcp 10.132.0.179:15021: connect: connection refused 32m Warning ProbeError pod/e2e-api-b-95d2ac-openshift-default-7f6cb95d55-t6xhk Readiness probe error: Get "http://10.132.0.179:15021/healthz/ready": dial tcp 10.132.0.179:15021: connect: connection refused... 32m Normal Pulled pod/e2e-api-b-95d2ac-openshift-default-7f6cb95d55-t6xhk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-api-b-95d2ac-openshift-default-7f6cb95d55-t6xhk Container created 32m Normal Started pod/e2e-api-b-95d2ac-openshift-default-7f6cb95d55-t6xhk Container started 32m Normal Killing pod/e2e-api-a-c7880b-openshift-default-56db9c9557-9vwgs Stopping container istio-proxy 32m Warning ProbeError pod/e2e-api-b-95d2ac-openshift-default-7f6cb95d55-t6xhk Startup probe error: Get "http://10.132.0.179:15021/healthz/ready": dial tcp 10.132.0.179:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-api-b-95d2ac-openshift-default-7f6cb95d55-t6xhk Startup probe failed: Get "http://10.132.0.179:15021/healthz/ready": dial tcp 10.132.0.179:15021: connect: connection refused 32m Warning Unhealthy pod/e2e-api-a-c7880b-openshift-default-56db9c9557-9vwgs Readiness probe failed: HTTP probe failed with statuscode: 503 32m Warning ProbeError pod/e2e-api-a-c7880b-openshift-default-56db9c9557-9vwgs Readiness probe error: HTTP probe failed with statuscode: 503... 32m Normal Killing pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-kqc8x Stopping container istio-proxy 32m Normal SuccessfulDelete replicaset/e2e-worker-w1-419d0a-openshift-default-55766cf857 Deleted pod: e2e-worker-w1-419d0a-openshift-default-55766cf857-mkgcz 32m Normal SuccessfulDelete replicaset/e2e-worker-w5-60577a-openshift-default-7c8cb87747 Deleted pod: e2e-worker-w5-60577a-openshift-default-7c8cb87747-fz2s8 32m Normal Killing pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-g8pb5 Stopping container istio-proxy 32m Normal Killing pod/payload-pre-processing-e2e-mt-20e66fa4-997965c87-hsg4m Stopping container payload-pre-processing 32m Normal SuccessfulDelete replicaset/e2e-worker-w2-b623ca-openshift-default-68847758ff Deleted pod: e2e-worker-w2-b623ca-openshift-default-68847758ff-g8pb5 32m Normal SuccessfulDelete replicaset/e2e-worker-w5-60577a-openshift-default-7c8cb87747 Deleted pod: e2e-worker-w5-60577a-openshift-default-7c8cb87747-6lg8j 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-60577a-openshift-default New size: 8; reason: All metrics below target 32m Normal Killing pod/payload-processing-e2e-mt-20e66fa4-54557c4fd4-8ffbj Stopping container payload-processing 32m Normal ScalingReplicaSet deployment/e2e-worker-w5-60577a-openshift-default Scaled down replica set e2e-worker-w5-60577a-openshift-default-7c8cb87747 from 10 to 8 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-419d0a-openshift-default New size: 9; reason: All metrics below target 32m Normal SuccessfulDelete replicaset/e2e-worker-w2-b623ca-openshift-default-68847758ff Deleted pod: e2e-worker-w2-b623ca-openshift-default-68847758ff-kqc8x 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-b623ca-openshift-default New size: 8; reason: All metrics below target 32m Normal ScalingReplicaSet deployment/e2e-worker-w2-b623ca-openshift-default Scaled down replica set e2e-worker-w2-b623ca-openshift-default-68847758ff from 10 to 8 32m Normal ScalingReplicaSet deployment/e2e-worker-w1-419d0a-openshift-default Scaled down replica set e2e-worker-w1-419d0a-openshift-default-55766cf857 from 10 to 9 32m Normal SuccessfulDelete replicaset/e2e-worker-w0-ae63bf-openshift-default-556d46cdff Deleted pod: e2e-worker-w0-ae63bf-openshift-default-556d46cdff-t8psh 32m Normal ScalingReplicaSet deployment/e2e-worker-w0-ae63bf-openshift-default Scaled down replica set e2e-worker-w0-ae63bf-openshift-default-556d46cdff from 10 to 8 32m Normal Killing pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-t8psh Stopping container istio-proxy 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ae63bf-openshift-default New size: 8; reason: All metrics below target 32m Normal Killing pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-qj2fm Stopping container istio-proxy 32m Normal Killing pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-mkgcz Stopping container istio-proxy 32m Normal Killing pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-fz2s8 Stopping container istio-proxy 32m Normal Killing pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-6lg8j Stopping container istio-proxy 32m Normal SuccessfulDelete replicaset/e2e-worker-w0-ae63bf-openshift-default-556d46cdff Deleted pod: e2e-worker-w0-ae63bf-openshift-default-556d46cdff-qj2fm 32m Warning ProbeError pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-6lg8j Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning Unhealthy pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-mkgcz Readiness probe failed: HTTP probe failed with statuscode: 503 32m Warning ProbeError pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-mkgcz Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning Unhealthy pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-6lg8j Readiness probe failed: HTTP probe failed with statuscode: 503 31m Normal Killing pod/payload-processing-e2e-api-b-95d2ac-75f686bc4d-v7vw5 Stopping container payload-processing 31m Normal Killing pod/payload-pre-processing-e2e-api-b-95d2ac-74dbfdbb99-fjbtf Stopping container payload-pre-processing 31m Warning ProbeError pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-t8psh Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning Unhealthy pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-t8psh Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning Unhealthy pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-qj2fm Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning ProbeError pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-qj2fm Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-95d2ac-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-95d2ac-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Normal SuccessfulCreate replicaset/e2e-api-a-82012b-openshift-default-67457b4d8c Created pod: e2e-api-a-82012b-openshift-default-67457b4d8c-fx2xx 31m Normal ScalingReplicaSet deployment/e2e-api-a-82012b-openshift-default Scaled up replica set e2e-api-a-82012b-openshift-default-67457b4d8c from 0 to 1 31m Normal AddedLabel gateway/e2e-api-a-82012b Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-82012b 31m Normal AddedInterface pod/e2e-api-a-82012b-openshift-default-67457b4d8c-fx2xx Add eth0 [10.134.0.131/23] from ovn-kubernetes 31m Normal SuccessfulCreate replicaset/e2e-inf-a-89a3f6-openshift-default-547f79fc89 Created pod: e2e-inf-a-89a3f6-openshift-default-547f79fc89-njwhz 31m Normal ScalingReplicaSet deployment/e2e-inf-a-89a3f6-openshift-default Scaled up replica set e2e-inf-a-89a3f6-openshift-default-547f79fc89 from 0 to 1 31m Normal AddedLabel gateway/e2e-inf-a-89a3f6 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-89a3f6 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-419d0a-openshift-default New size: 7; reason: All metrics below target 31m Normal ScalingReplicaSet deployment/e2e-worker-w1-419d0a-openshift-default Scaled down replica set e2e-worker-w1-419d0a-openshift-default-55766cf857 from 9 to 7 31m Normal SuccessfulDelete replicaset/e2e-worker-w1-419d0a-openshift-default-55766cf857 Deleted pod: e2e-worker-w1-419d0a-openshift-default-55766cf857-nkc9c 31m Normal AddedInterface pod/e2e-inf-a-89a3f6-openshift-default-547f79fc89-njwhz Add eth0 [10.134.0.132/23] from ovn-kubernetes 31m Normal SuccessfulDelete replicaset/e2e-worker-w1-419d0a-openshift-default-55766cf857 Deleted pod: e2e-worker-w1-419d0a-openshift-default-55766cf857-tmlkg 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-ad96a4-openshift-default New size: 9; reason: All metrics below target 31m Normal Killing pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-tmlkg Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-nkc9c Stopping container istio-proxy 31m Normal ScalingReplicaSet deployment/e2e-worker-w4-ad96a4-openshift-default Scaled down replica set e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 from 10 to 9 31m Normal Killing pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-fmf6b Stopping container istio-proxy 31m Normal SuccessfulDelete replicaset/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 Deleted pod: e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-fmf6b 31m Warning Unhealthy pod/e2e-api-a-73f96d-openshift-default-66d877c755-9nvtp Readiness probe failed: Get "http://10.133.0.87:15021/healthz/ready": read tcp 10.133.0.2:39900->10.133.0.87:15021: read: connection reset by peer 31m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-82012b Scaled up replica set payload-processing-e2e-api-a-82012b-64d6c8f676 from 0 to 1 31m Warning ProbeError pod/e2e-api-a-73f96d-openshift-default-66d877c755-9nvtp Readiness probe error: Get "http://10.133.0.87:15021/healthz/ready": read tcp 10.133.0.2:39900->10.133.0.87:15021: read: connection reset by peer... 31m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-82012b-64d6c8f676 Created pod: payload-processing-e2e-api-a-82012b-64d6c8f676-wv8kj 31m Warning FailedCreate replicaset/payload-processing-e2e-api-a-82012b-64d6c8f676 Error creating: pods "payload-processing-e2e-api-a-82012b-64d6c8f676-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-82012b: serviceaccount "payload-processing-e2e-api-a-82012b" not found 31m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-82012b-5987f89c59 Error creating: pods "payload-pre-processing-e2e-api-a-82012b-5987f89c59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-82012b: serviceaccount "payload-processing-e2e-api-a-82012b" not found 31m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-82012b-5987f89c59 Created pod: payload-pre-processing-e2e-api-a-82012b-5987f89c59-xqtgs 31m Normal Pulled pod/e2e-api-a-73f96d-openshift-default-66d877c755-9nvtp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Started pod/e2e-api-a-73f96d-openshift-default-66d877c755-9nvtp Container started 31m Normal Created pod/e2e-api-a-73f96d-openshift-default-66d877c755-9nvtp Container created 31m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-82012b Scaled up replica set payload-pre-processing-e2e-api-a-82012b-5987f89c59 from 0 to 1 31m Normal Started pod/payload-pre-processing-e2e-api-a-82012b-5987f89c59-xqtgs Container started 31m Normal Started pod/payload-processing-e2e-api-a-82012b-64d6c8f676-wv8kj Container started 31m Normal Pulled pod/payload-processing-e2e-api-a-82012b-64d6c8f676-wv8kj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/payload-processing-e2e-api-a-82012b-64d6c8f676-wv8kj Add eth0 [10.134.0.133/23] from ovn-kubernetes 31m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-82012b-5987f89c59-xqtgs Add eth0 [10.132.0.195/23] from ovn-kubernetes 31m Normal Pulled pod/payload-pre-processing-e2e-api-a-82012b-5987f89c59-xqtgs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal Created pod/payload-pre-processing-e2e-api-a-82012b-5987f89c59-xqtgs Container created 31m Normal Created pod/payload-processing-e2e-api-a-82012b-64d6c8f676-wv8kj Container created 31m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-89a3f6-db46ffd87 Created pod: payload-pre-processing-e2e-inf-a-89a3f6-db46ffd87-8l4s5 31m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-89a3f6-7b8cb866f4 Created pod: payload-processing-e2e-inf-a-89a3f6-7b8cb866f4-5zqx4 31m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-89a3f6-db46ffd87 Error creating: pods "payload-pre-processing-e2e-inf-a-89a3f6-db46ffd87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-89a3f6: serviceaccount "payload-processing-e2e-inf-a-89a3f6" not found 31m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-89a3f6-7b8cb866f4 Error creating: pods "payload-processing-e2e-inf-a-89a3f6-7b8cb866f4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-89a3f6: serviceaccount "payload-processing-e2e-inf-a-89a3f6" not found 31m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-89a3f6 Scaled up replica set payload-pre-processing-e2e-inf-a-89a3f6-db46ffd87 from 0 to 1 31m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-89a3f6 Scaled up replica set payload-processing-e2e-inf-a-89a3f6-7b8cb866f4 from 0 to 1 31m Normal Pulled pod/payload-processing-e2e-inf-a-89a3f6-7b8cb866f4-5zqx4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal Created pod/payload-processing-e2e-inf-a-89a3f6-7b8cb866f4-5zqx4 Container created 31m Normal Created pod/payload-pre-processing-e2e-inf-a-89a3f6-db46ffd87-8l4s5 Container created 31m Normal Started pod/payload-pre-processing-e2e-inf-a-89a3f6-db46ffd87-8l4s5 Container started 31m Normal Pulled pod/payload-pre-processing-e2e-inf-a-89a3f6-db46ffd87-8l4s5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-89a3f6-db46ffd87-8l4s5 Add eth0 [10.132.0.196/23] from ovn-kubernetes 31m Normal AddedInterface pod/payload-processing-e2e-inf-a-89a3f6-7b8cb866f4-5zqx4 Add eth0 [10.134.0.134/23] from ovn-kubernetes 31m Normal Started pod/payload-processing-e2e-inf-a-89a3f6-7b8cb866f4-5zqx4 Container started 31m Normal Killing pod/payload-processing-e2e-api-a-73f96d-767bd9568-4pjpt Stopping container payload-processing 31m Normal Started pod/e2e-api-b-6a6348-openshift-default-7bd9c5bc47-cmtk5 Container started 31m Normal Created pod/e2e-api-b-6a6348-openshift-default-7bd9c5bc47-cmtk5 Container created 31m Normal Pulled pod/e2e-api-b-6a6348-openshift-default-7bd9c5bc47-cmtk5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Killing pod/payload-pre-processing-e2e-api-a-73f96d-65c48fbd9f-98p49 Stopping container payload-pre-processing 31m Warning Unhealthy pod/e2e-api-a-73f96d-openshift-default-66d877c755-9nvtp Startup probe failed: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused 31m Warning ProbeError pod/e2e-api-a-73f96d-openshift-default-66d877c755-9nvtp Startup probe error: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused... 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-6a6348-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-6a6348-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning ProbeError pod/e2e-api-b-6a6348-openshift-default-7bd9c5bc47-cmtk5 Startup probe error: Get "http://10.132.0.189:15021/healthz/ready": dial tcp 10.132.0.189:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-api-b-6a6348-openshift-default-7bd9c5bc47-cmtk5 Startup probe failed: Get "http://10.132.0.189:15021/healthz/ready": dial tcp 10.132.0.189:15021: connect: connection refused 31m Normal SuccessfulDelete replicaset/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77 Deleted pod: e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-gsk6q 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-419d0a-openshift-default New size: 6; reason: All metrics below target 31m Normal ScalingReplicaSet deployment/e2e-worker-w6-01b24e-openshift-default Scaled down replica set e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77 from 10 to 6 31m Normal Killing pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-npzrt Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-w5hjp Stopping container istio-proxy 31m Normal ScalingReplicaSet deployment/e2e-worker-w5-60577a-openshift-default Scaled down replica set e2e-worker-w5-60577a-openshift-default-7c8cb87747 from 8 to 6 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-60577a-openshift-default New size: 6; reason: All metrics below target 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-01b24e-openshift-default New size: 6; reason: All metrics below target 31m Normal SuccessfulDelete replicaset/e2e-worker-w2-b623ca-openshift-default-68847758ff Deleted pod: e2e-worker-w2-b623ca-openshift-default-68847758ff-5frq8 31m Normal SuccessfulDelete replicaset/e2e-worker-w2-b623ca-openshift-default-68847758ff Deleted pod: e2e-worker-w2-b623ca-openshift-default-68847758ff-2d82j 31m Normal Killing pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-gsk6q Stopping container istio-proxy 31m Normal SuccessfulDelete replicaset/e2e-worker-w1-419d0a-openshift-default-55766cf857 Deleted pod: e2e-worker-w1-419d0a-openshift-default-55766cf857-2ch45 31m Normal SuccessfulDelete replicaset/e2e-worker-w5-60577a-openshift-default-7c8cb87747 Deleted pod: e2e-worker-w5-60577a-openshift-default-7c8cb87747-npzrt 31m Normal SuccessfulDelete replicaset/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77 Deleted pod: e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-q8kz9 31m Normal SuccessfulDelete replicaset/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77 Deleted pod: e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-pjznf 31m Normal SuccessfulDelete replicaset/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77 Deleted pod: e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-lf4jg 31m Normal SuccessfulDelete replicaset/e2e-worker-w5-60577a-openshift-default-7c8cb87747 Deleted pod: e2e-worker-w5-60577a-openshift-default-7c8cb87747-w5hjp 31m Normal Killing pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-pjznf Stopping container istio-proxy 31m Normal ScalingReplicaSet deployment/e2e-worker-w2-b623ca-openshift-default Scaled down replica set e2e-worker-w2-b623ca-openshift-default-68847758ff from 8 to 6 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-b623ca-openshift-default New size: 6; reason: All metrics below target 31m Normal ScalingReplicaSet deployment/e2e-worker-w1-419d0a-openshift-default Scaled down replica set e2e-worker-w1-419d0a-openshift-default-55766cf857 from 7 to 6 31m Normal Killing pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-lf4jg Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-q8kz9 Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-5frq8 Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-2d82j Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-2ch45 Stopping container istio-proxy 31m Normal SuccessfulDelete replicaset/e2e-worker-w0-ae63bf-openshift-default-556d46cdff Deleted pod: e2e-worker-w0-ae63bf-openshift-default-556d46cdff-t2llg 31m Normal SuccessfulDelete replicaset/e2e-worker-w3-ce69f3-openshift-default-6f98547647 Deleted pod: e2e-worker-w3-ce69f3-openshift-default-6f98547647-dwtpl 31m Normal ScalingReplicaSet deployment/e2e-worker-w0-ae63bf-openshift-default Scaled down replica set e2e-worker-w0-ae63bf-openshift-default-556d46cdff from 8 to 6 31m Normal SuccessfulDelete replicaset/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 Deleted pod: e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-75rps 31m Normal SuccessfulDelete replicaset/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 Deleted pod: e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-p58s7 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ae63bf-openshift-default New size: 6; reason: All metrics below target 31m Normal Killing pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-t2llg Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-tclm9 Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-dwtpl Stopping container istio-proxy 31m Normal SuccessfulDelete replicaset/e2e-worker-w3-ce69f3-openshift-default-6f98547647 Deleted pod: e2e-worker-w3-ce69f3-openshift-default-6f98547647-lkn8z 31m Normal SuccessfulDelete replicaset/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 Deleted pod: e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-tclm9 31m Normal SuccessfulDelete replicaset/e2e-worker-w0-ae63bf-openshift-default-556d46cdff Deleted pod: e2e-worker-w0-ae63bf-openshift-default-556d46cdff-d6kpm 31m Normal ScalingReplicaSet deployment/e2e-worker-w4-ad96a4-openshift-default Scaled down replica set e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 from 9 to 6 31m Normal Killing pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-lkn8z Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-75rps Stopping container istio-proxy 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-ad96a4-openshift-default New size: 6; reason: All metrics below target 31m Normal Killing pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-d6kpm Stopping container istio-proxy 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-ce69f3-openshift-default New size: 6; reason: All metrics below target 31m Normal Killing pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-p58s7 Stopping container istio-proxy 31m Normal ScalingReplicaSet deployment/e2e-worker-w3-ce69f3-openshift-default Scaled down replica set e2e-worker-w3-ce69f3-openshift-default-6f98547647 from 8 to 6 31m Normal Killing pod/payload-processing-e2e-worker-w5-60577a-646d79c448-2w78j Stopping container payload-processing 31m Normal Killing pod/payload-pre-processing-e2e-worker-w5-60577a-785b689d99-wh2hs Stopping container payload-pre-processing 31m Warning Unhealthy pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-dwtpl Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning ProbeError pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-dwtpl Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning ProbeError pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-q8kz9 Readiness probe error: HTTP probe failed with statuscode: 503... 31m Normal Created pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-zcdmk Container created 31m Normal AddedInterface pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-c6gjl Add eth0 [10.133.0.89/23] from ovn-kubernetes 31m Warning Unhealthy pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-q8kz9 Readiness probe failed: HTTP probe failed with statuscode: 503 31m Normal Started pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-c6gjl Container started 31m Normal SuccessfulCreate replicaset/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f Created pod: e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-zcdmk 31m Normal SuccessfulCreate replicaset/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f Created pod: e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-c6gjl 31m Normal Pulled pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-c6gjl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-20e66fa4-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 31m Normal ScalingReplicaSet deployment/e2e-mt-20e66fa4-openshift-default Scaled up replica set e2e-mt-20e66fa4-openshift-default-6ff5b8c94f from 2 to 4 31m Normal Created pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-c6gjl Container created 31m Normal AddedInterface pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-zcdmk Add eth0 [10.132.0.198/23] from ovn-kubernetes 31m Normal Started pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-zcdmk Container started 31m Normal Pulled pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-zcdmk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Killing pod/e2e-api-b-95d2ac-openshift-default-7f6cb95d55-t6xhk Stopping container istio-proxy 31m Normal ScalingReplicaSet deployment/e2e-inf-a-579ddb-openshift-default Scaled up replica set e2e-inf-a-579ddb-openshift-default-677f4b6845 from 0 to 1 31m Normal AddedLabel gateway/e2e-inf-a-579ddb Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-579ddb 31m Normal AddedInterface pod/e2e-inf-a-579ddb-openshift-default-677f4b6845-mwmnb Add eth0 [10.132.0.201/23] from ovn-kubernetes 31m Normal SuccessfulCreate replicaset/e2e-inf-a-579ddb-openshift-default-677f4b6845 Created pod: e2e-inf-a-579ddb-openshift-default-677f4b6845-mwmnb 31m Warning ProbeError pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-c6gjl Startup probe error: Get "http://10.133.0.89:15021/healthz/ready": dial tcp 10.133.0.89:15021: connect: connection refused... 31m Warning ProbeError pod/e2e-api-b-95d2ac-openshift-default-7f6cb95d55-t6xhk Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning Unhealthy pod/e2e-api-b-95d2ac-openshift-default-7f6cb95d55-t6xhk Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning Unhealthy pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-c6gjl Startup probe failed: Get "http://10.133.0.89:15021/healthz/ready": dial tcp 10.133.0.89:15021: connect: connection refused 31m Warning ProbeError pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-zcdmk Startup probe error: Get "http://10.132.0.198:15021/healthz/ready": dial tcp 10.132.0.198:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-zcdmk Startup probe failed: Get "http://10.132.0.198:15021/healthz/ready": dial tcp 10.132.0.198:15021: connect: connection refused 31m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-579ddb-6756775486 Created pod: payload-processing-e2e-inf-a-579ddb-6756775486-ghzlj 31m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-579ddb-6756775486 Error creating: pods "payload-processing-e2e-inf-a-579ddb-6756775486-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-579ddb: serviceaccount "payload-processing-e2e-inf-a-579ddb" not found 31m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-579ddb-9b6756cf9 Error creating: pods "payload-pre-processing-e2e-inf-a-579ddb-9b6756cf9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-579ddb: serviceaccount "payload-processing-e2e-inf-a-579ddb" not found 31m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-579ddb Scaled up replica set payload-processing-e2e-inf-a-579ddb-6756775486 from 0 to 1 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-73f96d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-579ddb-9b6756cf9 Created pod: payload-pre-processing-e2e-inf-a-579ddb-9b6756cf9-gfdfg 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-73f96d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-579ddb Scaled up replica set payload-pre-processing-e2e-inf-a-579ddb-9b6756cf9 from 0 to 1 31m Normal Started pod/payload-pre-processing-e2e-inf-a-579ddb-9b6756cf9-gfdfg Container started 31m Normal Started pod/payload-processing-e2e-inf-a-579ddb-6756775486-ghzlj Container started 31m Normal Created pod/payload-pre-processing-e2e-inf-a-579ddb-9b6756cf9-gfdfg Container created 31m Normal AddedInterface pod/payload-processing-e2e-inf-a-579ddb-6756775486-ghzlj Add eth0 [10.132.0.203/23] from ovn-kubernetes 31m Normal Pulled pod/payload-pre-processing-e2e-inf-a-579ddb-9b6756cf9-gfdfg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal Pulled pod/payload-processing-e2e-inf-a-579ddb-6756775486-ghzlj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal Created pod/payload-processing-e2e-inf-a-579ddb-6756775486-ghzlj Container created 31m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-579ddb-9b6756cf9-gfdfg Add eth0 [10.132.0.202/23] from ovn-kubernetes 31m Warning Unhealthy pod/payload-processing-e2e-worker-w5-60577a-646d79c448-2w78j Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Normal SuccessfulCreate replicaset/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f Created pod: e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-7f784 30m Normal AddedInterface pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-vm6lq Add eth0 [10.133.0.90/23] from ovn-kubernetes 30m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w5-60577a-785b689d99-wh2hs Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Normal AddedInterface pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-xbjth Add eth0 [10.134.0.138/23] from ovn-kubernetes 30m Normal Started pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-vm6lq Container started 30m Normal Started pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-jx2g9 Container started 30m Normal Created pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-xbjth Container created 30m Normal AddedInterface pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-7f784 Add eth0 [10.134.0.137/23] from ovn-kubernetes 30m Normal SuccessfulCreate replicaset/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f Created pod: e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-xbjth 30m Normal SuccessfulCreate replicaset/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f Created pod: e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-jx2g9 30m Normal Pulled pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-7f784 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Started pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-xbjth Container started 30m Normal SuccessfulCreate replicaset/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f Created pod: e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-vm6lq 30m Normal Pulled pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-xbjth Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Pulled pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-vm6lq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Created pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-vm6lq Container created 30m Normal Created pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-jx2g9 Container created 30m Normal ScalingReplicaSet deployment/e2e-mt-20e66fa4-openshift-default Scaled up replica set e2e-mt-20e66fa4-openshift-default-6ff5b8c94f from 4 to 8 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-20e66fa4-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 30m Normal Started pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-7f784 Container started 30m Normal AddedInterface pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-jx2g9 Add eth0 [10.132.0.204/23] from ovn-kubernetes 30m Normal Created pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-7f784 Container created 30m Normal Pulled pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-jx2g9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal AddedLabel gateway/e2e-api-b-156838 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-156838 30m Normal ScalingReplicaSet deployment/e2e-api-b-156838-openshift-default Scaled up replica set e2e-api-b-156838-openshift-default-6cc4b565b5 from 0 to 1 30m Normal SuccessfulCreate replicaset/e2e-api-b-156838-openshift-default-6cc4b565b5 Created pod: e2e-api-b-156838-openshift-default-6cc4b565b5-ljxwd 30m Normal Killing pod/e2e-api-a-73f96d-openshift-default-66d877c755-9nvtp Stopping container istio-proxy 30m Normal AddedInterface pod/e2e-api-b-156838-openshift-default-6cc4b565b5-ljxwd Add eth0 [10.132.0.205/23] from ovn-kubernetes 30m Warning ProbeError pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-xbjth Startup probe error: Get "http://10.134.0.138:15021/healthz/ready": dial tcp 10.134.0.138:15021: connect: connection refused... 30m Warning ProbeError pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-vm6lq Startup probe error: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused... 30m Warning ProbeError pod/e2e-api-a-73f96d-openshift-default-66d877c755-9nvtp Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning Unhealthy pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-7f784 Startup probe failed: Get "http://10.134.0.137:15021/healthz/ready": dial tcp 10.134.0.137:15021: connect: connection refused 30m Normal Killing pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-vm6lq Stopping container istio-proxy 30m Warning Unhealthy pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-xbjth Startup probe failed: Get "http://10.134.0.138:15021/healthz/ready": dial tcp 10.134.0.138:15021: connect: connection refused 30m Normal Killing pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-pgfhg Stopping container istio-proxy 30m Warning ProbeError pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-jx2g9 Startup probe error: Get "http://10.132.0.204:15021/healthz/ready": dial tcp 10.132.0.204:15021: connect: connection refused... 30m Warning ProbeError pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-7f784 Startup probe error: Get "http://10.134.0.137:15021/healthz/ready": dial tcp 10.134.0.137:15021: connect: connection refused... 30m Normal Killing pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-c6gjl Stopping container istio-proxy 30m Warning Unhealthy pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-jx2g9 Startup probe failed: Get "http://10.132.0.204:15021/healthz/ready": dial tcp 10.132.0.204:15021: connect: connection refused 30m Warning Unhealthy pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-vm6lq Startup probe failed: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused 30m Normal Killing pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-7f784 Stopping container istio-proxy 30m Warning Unhealthy pod/e2e-api-a-73f96d-openshift-default-66d877c755-9nvtp Readiness probe failed: HTTP probe failed with statuscode: 503 30m Normal Killing pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-jx2g9 Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-zcdmk Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-bqssp Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-xbjth Stopping container istio-proxy 30m Warning ProbeError pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-c6gjl Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning Unhealthy pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-c6gjl Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning ProbeError pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-pgfhg Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning ProbeError pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-zcdmk Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning Unhealthy pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-pgfhg Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning Unhealthy pod/e2e-mt-20e66fa4-openshift-default-6ff5b8c94f-zcdmk Readiness probe failed: HTTP probe failed with statuscode: 503 30m Normal AddedLabel gateway/e2e-api-a-d1d256 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-d1d256 30m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-156838 Scaled up replica set payload-pre-processing-e2e-api-b-156838-6749758ddc from 0 to 1 30m Normal ScalingReplicaSet deployment/e2e-api-a-d1d256-openshift-default Scaled up replica set e2e-api-a-d1d256-openshift-default-979b4ccbf from 0 to 1 30m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-156838-79f475dcd Created pod: payload-processing-e2e-api-b-156838-79f475dcd-x2qg9 30m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-156838 Scaled up replica set payload-processing-e2e-api-b-156838-79f475dcd from 0 to 1 30m Normal SuccessfulCreate replicaset/e2e-api-a-d1d256-openshift-default-979b4ccbf Created pod: e2e-api-a-d1d256-openshift-default-979b4ccbf-lm6hp 30m Warning FailedCreate replicaset/payload-processing-e2e-api-b-156838-79f475dcd Error creating: pods "payload-processing-e2e-api-b-156838-79f475dcd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-156838: serviceaccount "payload-processing-e2e-api-b-156838" not found 30m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-156838-6749758ddc Error creating: pods "payload-pre-processing-e2e-api-b-156838-6749758ddc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-156838: serviceaccount "payload-processing-e2e-api-b-156838" not found 30m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-156838-6749758ddc Created pod: payload-pre-processing-e2e-api-b-156838-6749758ddc-tnv4d 30m Warning Unhealthy pod/payload-processing-e2e-worker-w5-60577a-646d79c448-2w78j Readiness probe failed: timeout: failed to connect service "10.134.0.45:50052" within 1s: context deadline exceeded 30m Normal AddedInterface pod/payload-processing-e2e-api-b-156838-79f475dcd-x2qg9 Add eth0 [10.132.0.207/23] from ovn-kubernetes 30m Normal Started pod/payload-processing-e2e-api-b-156838-79f475dcd-x2qg9 Container started 30m Normal Pulled pod/payload-processing-e2e-api-b-156838-79f475dcd-x2qg9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-156838-6749758ddc-tnv4d Add eth0 [10.132.0.208/23] from ovn-kubernetes 30m Normal Created pod/payload-processing-e2e-api-b-156838-79f475dcd-x2qg9 Container created 30m Normal AddedInterface pod/e2e-api-a-d1d256-openshift-default-979b4ccbf-lm6hp Add eth0 [10.132.0.209/23] from ovn-kubernetes 30m Normal ScalingReplicaSet deployment/e2e-inf-b-d651c4-openshift-default Scaled up replica set e2e-inf-b-d651c4-openshift-default-775869444c from 0 to 1 30m Normal SuccessfulCreate replicaset/e2e-inf-b-d651c4-openshift-default-775869444c Created pod: e2e-inf-b-d651c4-openshift-default-775869444c-9wh89 30m Normal Pulled pod/payload-pre-processing-e2e-api-b-156838-6749758ddc-tnv4d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal Created pod/payload-pre-processing-e2e-api-b-156838-6749758ddc-tnv4d Container created 30m Normal Started pod/payload-pre-processing-e2e-api-b-156838-6749758ddc-tnv4d Container started 30m Normal AddedLabel gateway/e2e-inf-b-d651c4 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-d651c4 30m Normal AddedInterface pod/e2e-inf-b-d651c4-openshift-default-775869444c-9wh89 Add eth0 [10.134.0.139/23] from ovn-kubernetes 30m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-d1d256 Scaled up replica set payload-processing-e2e-api-a-d1d256-5b65b66d9d from 0 to 1 30m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-d1d256-5b65b66d9d Created pod: payload-processing-e2e-api-a-d1d256-5b65b66d9d-2djrn 30m Warning FailedCreate replicaset/payload-processing-e2e-api-a-d1d256-5b65b66d9d Error creating: pods "payload-processing-e2e-api-a-d1d256-5b65b66d9d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-d1d256: serviceaccount "payload-processing-e2e-api-a-d1d256" not found 30m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-d1d256 Scaled up replica set payload-pre-processing-e2e-api-a-d1d256-5b889bbcc5 from 0 to 1 30m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-d1d256-5b889bbcc5 Created pod: payload-pre-processing-e2e-api-a-d1d256-5b889bbcc5-qb7k8 30m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-d1d256-5b889bbcc5 Error creating: pods "payload-pre-processing-e2e-api-a-d1d256-5b889bbcc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-d1d256: serviceaccount "payload-processing-e2e-api-a-d1d256" not found 30m Normal Started pod/payload-processing-e2e-api-a-d1d256-5b65b66d9d-2djrn Container started 30m Normal AddedInterface pod/payload-processing-e2e-api-a-d1d256-5b65b66d9d-2djrn Add eth0 [10.132.0.212/23] from ovn-kubernetes 30m Normal Created pod/payload-pre-processing-e2e-api-a-d1d256-5b889bbcc5-qb7k8 Container created 30m Normal Pulled pod/payload-pre-processing-e2e-api-a-d1d256-5b889bbcc5-qb7k8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal Killing pod/payload-pre-processing-e2e-api-b-6a6348-79bcfb7655-cpxz9 Stopping container payload-pre-processing 30m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-d1d256-5b889bbcc5-qb7k8 Add eth0 [10.132.0.211/23] from ovn-kubernetes 30m Normal Started pod/payload-pre-processing-e2e-api-a-d1d256-5b889bbcc5-qb7k8 Container started 30m Normal Pulled pod/payload-processing-e2e-api-a-d1d256-5b65b66d9d-2djrn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal Killing pod/payload-processing-e2e-api-b-6a6348-677c897664-5q8vf Stopping container payload-processing 30m Normal Created pod/payload-processing-e2e-api-a-d1d256-5b65b66d9d-2djrn Container created 30m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-d651c4 Scaled up replica set payload-processing-e2e-inf-b-d651c4-7f667dbdfb from 0 to 1 30m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-d651c4 Scaled up replica set payload-pre-processing-e2e-inf-b-d651c4-75cbc576bc from 0 to 1 30m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-d651c4-7f667dbdfb Created pod: payload-processing-e2e-inf-b-d651c4-7f667dbdfb-48tkv 30m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-d651c4-7f667dbdfb Error creating: pods "payload-processing-e2e-inf-b-d651c4-7f667dbdfb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-d651c4: serviceaccount "payload-processing-e2e-inf-b-d651c4" not found 30m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-d651c4-75cbc576bc Error creating: pods "payload-pre-processing-e2e-inf-b-d651c4-75cbc576bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-d651c4: serviceaccount "payload-processing-e2e-inf-b-d651c4" not found 30m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-d651c4-75cbc576bc Created pod: payload-pre-processing-e2e-inf-b-d651c4-75cbc576bc-tp8q9 30m Normal Created pod/payload-processing-e2e-inf-b-d651c4-7f667dbdfb-48tkv Container created 30m Normal AddedInterface pod/payload-processing-e2e-inf-b-d651c4-7f667dbdfb-48tkv Add eth0 [10.134.0.140/23] from ovn-kubernetes 30m Normal Pulled pod/payload-processing-e2e-inf-b-d651c4-7f667dbdfb-48tkv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal Started pod/payload-pre-processing-e2e-inf-b-d651c4-75cbc576bc-tp8q9 Container started 30m Normal Created pod/payload-pre-processing-e2e-inf-b-d651c4-75cbc576bc-tp8q9 Container created 30m Normal Pulled pod/payload-pre-processing-e2e-inf-b-d651c4-75cbc576bc-tp8q9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal Started pod/payload-processing-e2e-inf-b-d651c4-7f667dbdfb-48tkv Container started 30m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-d651c4-75cbc576bc-tp8q9 Add eth0 [10.134.0.141/23] from ovn-kubernetes 30m Normal Pulled pod/e2e-inf-a-89a3f6-openshift-default-547f79fc89-njwhz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Started pod/e2e-inf-a-89a3f6-openshift-default-547f79fc89-njwhz Container started 30m Normal Created pod/e2e-inf-a-89a3f6-openshift-default-547f79fc89-njwhz Container created 30m Normal Killing pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-sbl2p Stopping container istio-proxy 30m Normal Killing pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-jggxr Stopping container istio-proxy 30m Normal Killing pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-m9f79 Stopping container istio-proxy 30m Normal Killing pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-ncnjw Stopping container istio-proxy 30m Normal Killing pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-2dmhf Stopping container istio-proxy 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-89a3f6-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Normal Killing pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-jrrrp Stopping container istio-proxy 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-89a3f6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Warning ProbeError pod/e2e-inf-a-89a3f6-openshift-default-547f79fc89-njwhz Startup probe error: Get "http://10.134.0.132:15021/healthz/ready": dial tcp 10.134.0.132:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-inf-a-89a3f6-openshift-default-547f79fc89-njwhz Startup probe failed: Get "http://10.134.0.132:15021/healthz/ready": dial tcp 10.134.0.132:15021: connect: connection refused 30m Warning Unhealthy pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-jggxr Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning ProbeError pod/e2e-worker-w5-60577a-openshift-default-7c8cb87747-jggxr Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning FailedToUpdateEndpoint endpoints/payload-pre-processing-e2e-shared-a-w0-395ccc Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-shared-a-w0-395ccc: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-shared-a-w0-395ccc": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-shared-a-w0-395ccc, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 8799b1dd-df1b-4962-a2b2-a7ce6d787323, UID in object meta: 30m Normal Killing pod/payload-processing-e2e-shared-a-w0-395ccc-656669bc5b-sg8dz Stopping container payload-processing 30m Normal Killing pod/payload-pre-processing-e2e-shared-a-w0-395ccc-7d4b79d8f9-gn47s Stopping container payload-pre-processing 30m Normal AddedLabel gateway/e2e-inf-b-1e13df Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-1e13df 30m Normal SuccessfulCreate replicaset/e2e-inf-b-1e13df-openshift-default-58bb9b8669 Created pod: e2e-inf-b-1e13df-openshift-default-58bb9b8669-c6zvx 30m Normal ScalingReplicaSet deployment/e2e-inf-b-1e13df-openshift-default Scaled up replica set e2e-inf-b-1e13df-openshift-default-58bb9b8669 from 0 to 1 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-d651c4-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Normal AddedInterface pod/e2e-inf-b-1e13df-openshift-default-58bb9b8669-c6zvx Add eth0 [10.132.0.214/23] from ovn-kubernetes 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-d651c4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-1e13df Scaled up replica set payload-pre-processing-e2e-inf-b-1e13df-78b7ccf4bc from 0 to 1 30m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-1e13df Scaled up replica set payload-processing-e2e-inf-b-1e13df-68d544b494 from 0 to 1 30m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-1e13df-78b7ccf4bc Error creating: pods "payload-pre-processing-e2e-inf-b-1e13df-78b7ccf4bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-1e13df: serviceaccount "payload-processing-e2e-inf-b-1e13df" not found 30m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-1e13df-68d544b494 Error creating: pods "payload-processing-e2e-inf-b-1e13df-68d544b494-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-1e13df: serviceaccount "payload-processing-e2e-inf-b-1e13df" not found 30m Normal Pulled pod/payload-processing-e2e-inf-b-1e13df-68d544b494-c828d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/payload-processing-e2e-inf-b-1e13df-68d544b494-c828d Add eth0 [10.134.0.144/23] from ovn-kubernetes 30m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-1e13df-78b7ccf4bc-8ldn2 Add eth0 [10.132.0.215/23] from ovn-kubernetes 30m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-1e13df-68d544b494 Created pod: payload-processing-e2e-inf-b-1e13df-68d544b494-c828d 30m Normal Pulled pod/payload-pre-processing-e2e-inf-b-1e13df-78b7ccf4bc-8ldn2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-1e13df-78b7ccf4bc Created pod: payload-pre-processing-e2e-inf-b-1e13df-78b7ccf4bc-8ldn2 30m Normal Started pod/payload-pre-processing-e2e-inf-b-1e13df-78b7ccf4bc-8ldn2 Container started 30m Normal Created pod/payload-processing-e2e-inf-b-1e13df-68d544b494-c828d Container created 30m Normal Started pod/payload-processing-e2e-inf-b-1e13df-68d544b494-c828d Container started 30m Normal Created pod/payload-pre-processing-e2e-inf-b-1e13df-78b7ccf4bc-8ldn2 Container created 30m Normal Killing pod/e2e-shared-a-w0-395ccc-openshift-default-5dcf9cb54c-fnlf6 Stopping container istio-proxy 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-89a3f6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Normal AddedLabel gateway/e2e-api-b-22af66 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-22af66 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-89a3f6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Normal SuccessfulCreate replicaset/e2e-api-b-22af66-openshift-default-5f7c494769 Created pod: e2e-api-b-22af66-openshift-default-5f7c494769-gqftx 30m Normal ScalingReplicaSet deployment/e2e-api-b-22af66-openshift-default Scaled up replica set e2e-api-b-22af66-openshift-default-5f7c494769 from 0 to 1 30m Normal AddedInterface pod/e2e-api-b-22af66-openshift-default-5f7c494769-gqftx Add eth0 [10.134.0.145/23] from ovn-kubernetes 29m Normal Killing pod/payload-processing-e2e-shared-b-w0-ae425f-5885c86dd4-4wvh9 Stopping container payload-processing 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-22af66 Scaled up replica set payload-processing-e2e-api-b-22af66-7b5b988746 from 0 to 1 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-22af66-5bf7f46cd7 Created pod: payload-pre-processing-e2e-api-b-22af66-5bf7f46cd7-ffx8p 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-22af66 Scaled up replica set payload-pre-processing-e2e-api-b-22af66-5bf7f46cd7 from 0 to 1 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-22af66-5bf7f46cd7 Error creating: pods "payload-pre-processing-e2e-api-b-22af66-5bf7f46cd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-22af66: serviceaccount "payload-processing-e2e-api-b-22af66" not found 29m Normal Killing pod/payload-pre-processing-e2e-shared-b-w0-ae425f-79c9ff7db7-z7nf7 Stopping container payload-pre-processing 29m Warning FailedCreate replicaset/payload-processing-e2e-api-b-22af66-7b5b988746 Error creating: pods "payload-processing-e2e-api-b-22af66-7b5b988746-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-22af66: serviceaccount "payload-processing-e2e-api-b-22af66" not found 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-22af66-7b5b988746 Created pod: payload-processing-e2e-api-b-22af66-7b5b988746-mm9zl 29m Normal Created pod/payload-pre-processing-e2e-api-b-22af66-5bf7f46cd7-ffx8p Container created 29m Normal Started pod/payload-pre-processing-e2e-api-b-22af66-5bf7f46cd7-ffx8p Container started 29m Normal Pulled pod/payload-pre-processing-e2e-api-b-22af66-5bf7f46cd7-ffx8p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-22af66-5bf7f46cd7-ffx8p Add eth0 [10.132.0.219/23] from ovn-kubernetes 29m Normal Started pod/payload-processing-e2e-api-b-22af66-7b5b988746-mm9zl Container started 29m Normal Created pod/payload-processing-e2e-api-b-22af66-7b5b988746-mm9zl Container created 29m Normal Pulled pod/payload-processing-e2e-api-b-22af66-7b5b988746-mm9zl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/payload-processing-e2e-api-b-22af66-7b5b988746-mm9zl Add eth0 [10.132.0.218/23] from ovn-kubernetes 29m Normal Killing pod/payload-processing-e2e-api-a-82012b-64d6c8f676-wv8kj Stopping container payload-processing 29m Normal Killing pod/payload-pre-processing-e2e-api-a-82012b-5987f89c59-xqtgs Stopping container payload-pre-processing 29m Normal Killing pod/e2e-api-b-6a6348-openshift-default-7bd9c5bc47-cmtk5 Stopping container istio-proxy 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-6a6348-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-6a6348-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning FailedUpdateStatus horizontalpodautoscaler/e2e-api-b-6a6348-openshift-default horizontalpodautoscalers.autoscaling "e2e-api-b-6a6348-openshift-default" not found 29m Normal AddedLabel gateway/e2e-inf-a-744b57 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-744b57 29m Normal SuccessfulCreate replicaset/e2e-inf-a-744b57-openshift-default-6c5cbf6bb9 Created pod: e2e-inf-a-744b57-openshift-default-6c5cbf6bb9-zctmd 29m Normal ScalingReplicaSet deployment/e2e-inf-a-744b57-openshift-default Scaled up replica set e2e-inf-a-744b57-openshift-default-6c5cbf6bb9 from 0 to 1 29m Normal AddedInterface pod/e2e-inf-a-744b57-openshift-default-6c5cbf6bb9-zctmd Add eth0 [10.132.0.220/23] from ovn-kubernetes 29m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w0-ae425f-79c9ff7db7-z7nf7 Readiness probe failed: timeout: failed to connect service "10.132.0.159:50052" within 1s: context deadline exceeded 29m Warning Unhealthy pod/payload-processing-e2e-shared-b-w0-ae425f-5885c86dd4-4wvh9 Readiness probe failed: timeout: failed to connect service "10.132.0.160:50052" within 1s: context deadline exceeded 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-744b57 Scaled up replica set payload-processing-e2e-inf-a-744b57-597d6c8986 from 0 to 1 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-744b57 Scaled up replica set payload-pre-processing-e2e-inf-a-744b57-77cc66bfc9 from 0 to 1 29m Warning Unhealthy pod/e2e-api-b-6a6348-openshift-default-7bd9c5bc47-cmtk5 Readiness probe failed: Get "http://10.132.0.189:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 29m Warning ProbeError pod/e2e-api-b-6a6348-openshift-default-7bd9c5bc47-cmtk5 Readiness probe error: Get "http://10.132.0.189:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-744b57-77cc66bfc9 Error creating: pods "payload-pre-processing-e2e-inf-a-744b57-77cc66bfc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-744b57: serviceaccount "payload-processing-e2e-inf-a-744b57" not found 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-744b57-77cc66bfc9 Created pod: payload-pre-processing-e2e-inf-a-744b57-77cc66bfc9-2fcz9 29m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-744b57-597d6c8986 Error creating: pods "payload-processing-e2e-inf-a-744b57-597d6c8986-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-744b57: serviceaccount "payload-processing-e2e-inf-a-744b57" not found 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-744b57-597d6c8986 Created pod: payload-processing-e2e-inf-a-744b57-597d6c8986-4v2zh 29m Normal AddedInterface pod/payload-processing-e2e-inf-a-744b57-597d6c8986-4v2zh Add eth0 [10.132.0.222/23] from ovn-kubernetes 29m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-744b57-77cc66bfc9-2fcz9 Add eth0 [10.132.0.221/23] from ovn-kubernetes 29m Normal Started pod/payload-pre-processing-e2e-inf-a-744b57-77cc66bfc9-2fcz9 Container started 29m Normal Pulled pod/payload-pre-processing-e2e-inf-a-744b57-77cc66bfc9-2fcz9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal Created pod/payload-pre-processing-e2e-inf-a-744b57-77cc66bfc9-2fcz9 Container created 29m Normal Pulled pod/payload-processing-e2e-inf-a-744b57-597d6c8986-4v2zh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal Started pod/payload-processing-e2e-inf-a-744b57-597d6c8986-4v2zh Container started 29m Normal Created pod/payload-processing-e2e-inf-a-744b57-597d6c8986-4v2zh Container created 29m Normal Pulled pod/e2e-api-a-82012b-openshift-default-67457b4d8c-fx2xx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Started pod/e2e-api-a-82012b-openshift-default-67457b4d8c-fx2xx Container started 29m Normal Created pod/e2e-api-a-82012b-openshift-default-67457b4d8c-fx2xx Container created 29m Normal Pulled pod/e2e-inf-b-d651c4-openshift-default-775869444c-9wh89 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Started pod/e2e-inf-b-d651c4-openshift-default-775869444c-9wh89 Container started 29m Normal Created pod/e2e-inf-b-d651c4-openshift-default-775869444c-9wh89 Container created 29m Normal Killing pod/e2e-shared-b-w0-ae425f-openshift-default-76874f9f6c-vszwf Stopping container istio-proxy 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-82012b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Normal Pulled pod/e2e-api-b-156838-openshift-default-6cc4b565b5-ljxwd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Pulled pod/e2e-api-a-d1d256-openshift-default-979b4ccbf-lm6hp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-82012b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Normal Created pod/e2e-api-b-156838-openshift-default-6cc4b565b5-ljxwd Container created 29m Normal Created pod/e2e-api-a-d1d256-openshift-default-979b4ccbf-lm6hp Container created 29m Normal Started pod/e2e-api-a-d1d256-openshift-default-979b4ccbf-lm6hp Container started 29m Warning Unhealthy pod/e2e-shared-b-w0-ae425f-openshift-default-76874f9f6c-vszwf Readiness probe failed: HTTP probe failed with statuscode: 503 29m Normal Started pod/e2e-api-b-156838-openshift-default-6cc4b565b5-ljxwd Container started 29m Warning ProbeError pod/e2e-shared-b-w0-ae425f-openshift-default-76874f9f6c-vszwf Readiness probe error: HTTP probe failed with statuscode: 503... 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-d1d256-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning Unhealthy pod/e2e-api-a-82012b-openshift-default-67457b4d8c-fx2xx Startup probe failed: Get "http://10.134.0.131:15021/healthz/ready": dial tcp 10.134.0.131:15021: connect: connection refused 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-d1d256-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning ProbeError pod/e2e-api-a-82012b-openshift-default-67457b4d8c-fx2xx Startup probe error: Get "http://10.134.0.131:15021/healthz/ready": dial tcp 10.134.0.131:15021: connect: connection refused... 29m Warning ProbeError pod/e2e-inf-b-d651c4-openshift-default-775869444c-9wh89 Startup probe error: Get "http://10.134.0.139:15021/healthz/ready": dial tcp 10.134.0.139:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-inf-b-d651c4-openshift-default-775869444c-9wh89 Startup probe failed: Get "http://10.134.0.139:15021/healthz/ready": dial tcp 10.134.0.139:15021: connect: connection refused 29m Warning Unhealthy pod/e2e-api-a-d1d256-openshift-default-979b4ccbf-lm6hp Startup probe failed: Get "http://10.132.0.209:15021/healthz/ready": dial tcp 10.132.0.209:15021: connect: connection refused 29m Warning ProbeError pod/e2e-api-b-156838-openshift-default-6cc4b565b5-ljxwd Startup probe error: Get "http://10.132.0.205:15021/healthz/ready": dial tcp 10.132.0.205:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-api-b-156838-openshift-default-6cc4b565b5-ljxwd Startup probe failed: Get "http://10.132.0.205:15021/healthz/ready": dial tcp 10.132.0.205:15021: connect: connection refused 29m Warning ProbeError pod/e2e-api-a-d1d256-openshift-default-979b4ccbf-lm6hp Startup probe error: Get "http://10.132.0.209:15021/healthz/ready": dial tcp 10.132.0.209:15021: connect: connection refused... 29m Normal Killing pod/payload-processing-e2e-worker-w0-ae63bf-56c67d7d7b-mgtt5 Stopping container payload-processing 29m Normal Killing pod/payload-pre-processing-e2e-worker-w0-ae63bf-76d99cc4d7-fdtz4 Stopping container payload-pre-processing 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-156838-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-156838-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-82012b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-82012b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Normal Killing pod/payload-processing-e2e-api-a-d1d256-5b65b66d9d-2djrn Stopping container payload-processing 29m Normal Killing pod/payload-pre-processing-e2e-api-a-d1d256-5b889bbcc5-qb7k8 Stopping container payload-pre-processing 29m Normal SuccessfulCreate replicaset/e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c Created pod: e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c-qj56r 29m Normal AddedLabel gateway/e2e-inf-b-ac37c8 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-ac37c8 29m Normal ScalingReplicaSet deployment/e2e-inf-b-ac37c8-openshift-default Scaled up replica set e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c from 0 to 1 29m Warning Unhealthy pod/payload-processing-e2e-worker-w0-ae63bf-56c67d7d7b-mgtt5 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Normal Killing pod/e2e-api-a-82012b-openshift-default-67457b4d8c-fx2xx Stopping container istio-proxy 29m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-ae63bf-76d99cc4d7-fdtz4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Normal AddedInterface pod/e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c-qj56r Add eth0 [10.132.0.229/23] from ovn-kubernetes 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-d1d256-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-d1d256-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-d651c4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-d651c4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Warning Unhealthy pod/e2e-inf-a-744b57-openshift-default-6c5cbf6bb9-zctmd Readiness probe failed: Get "http://10.132.0.220:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 29m Warning ProbeError pod/e2e-inf-a-744b57-openshift-default-6c5cbf6bb9-zctmd Readiness probe error: Get "http://10.132.0.220:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 29m Warning Unhealthy pod/e2e-api-a-82012b-openshift-default-67457b4d8c-fx2xx Readiness probe failed: HTTP probe failed with statuscode: 503 29m Warning ProbeError pod/e2e-api-a-82012b-openshift-default-67457b4d8c-fx2xx Readiness probe error: HTTP probe failed with statuscode: 503... 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-ac37c8 Scaled up replica set payload-processing-e2e-inf-b-ac37c8-69dbd9fb6d from 0 to 1 29m Normal Started pod/e2e-api-b-22af66-openshift-default-5f7c494769-gqftx Container started 29m Normal Created pod/e2e-api-b-22af66-openshift-default-5f7c494769-gqftx Container created 29m Normal Pulled pod/e2e-api-b-22af66-openshift-default-5f7c494769-gqftx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Warning ProbeError pod/e2e-inf-b-1e13df-openshift-default-58bb9b8669-c6zvx Readiness probe error: Get "http://10.132.0.214:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-ac37c8 Scaled up replica set payload-pre-processing-e2e-inf-b-ac37c8-57cb47d8c from 0 to 1 28m Normal Pulled pod/payload-pre-processing-e2e-inf-b-ac37c8-57cb47d8c-4cprd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-ac37c8-69dbd9fb6d Error creating: pods "payload-processing-e2e-inf-b-ac37c8-69dbd9fb6d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-ac37c8: serviceaccount "payload-processing-e2e-inf-b-ac37c8" not found 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-ac37c8-69dbd9fb6d Created pod: payload-processing-e2e-inf-b-ac37c8-69dbd9fb6d-xjzdf 28m Warning Unhealthy pod/e2e-inf-b-1e13df-openshift-default-58bb9b8669-c6zvx Readiness probe failed: Get "http://10.132.0.214:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 28m Normal AddedInterface pod/payload-processing-e2e-inf-b-ac37c8-69dbd9fb6d-xjzdf Add eth0 [10.132.0.232/23] from ovn-kubernetes 28m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-ac37c8-57cb47d8c-4cprd Add eth0 [10.132.0.231/23] from ovn-kubernetes 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-ac37c8-57cb47d8c Created pod: payload-pre-processing-e2e-inf-b-ac37c8-57cb47d8c-4cprd 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-ac37c8-57cb47d8c Error creating: pods "payload-pre-processing-e2e-inf-b-ac37c8-57cb47d8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-ac37c8: serviceaccount "payload-processing-e2e-inf-b-ac37c8" not found 28m Normal Created pod/payload-processing-e2e-inf-b-ac37c8-69dbd9fb6d-xjzdf Container created 28m Normal Pulled pod/payload-processing-e2e-inf-b-ac37c8-69dbd9fb6d-xjzdf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Started pod/payload-pre-processing-e2e-inf-b-ac37c8-57cb47d8c-4cprd Container started 28m Normal Created pod/payload-pre-processing-e2e-inf-b-ac37c8-57cb47d8c-4cprd Container created 28m Normal Started pod/payload-processing-e2e-inf-b-ac37c8-69dbd9fb6d-xjzdf Container started 28m Normal Created pod/e2e-inf-a-744b57-openshift-default-6c5cbf6bb9-zctmd Container created 28m Normal Created pod/e2e-inf-b-1e13df-openshift-default-58bb9b8669-c6zvx Container created 28m Normal Started pod/e2e-inf-b-1e13df-openshift-default-58bb9b8669-c6zvx Container started 28m Normal Pulled pod/e2e-inf-a-744b57-openshift-default-6c5cbf6bb9-zctmd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Pulled pod/e2e-inf-b-1e13df-openshift-default-58bb9b8669-c6zvx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Started pod/e2e-inf-a-744b57-openshift-default-6c5cbf6bb9-zctmd Container started 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-156838-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-156838-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Normal Killing pod/payload-processing-e2e-api-b-156838-79f475dcd-x2qg9 Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-api-b-156838-6749758ddc-tnv4d Stopping container payload-pre-processing 28m Warning Unhealthy pod/e2e-api-b-22af66-openshift-default-5f7c494769-gqftx Startup probe failed: Get "http://10.134.0.145:15021/healthz/ready": dial tcp 10.134.0.145:15021: connect: connection refused 28m Warning ProbeError pod/e2e-api-b-22af66-openshift-default-5f7c494769-gqftx Startup probe error: Get "http://10.134.0.145:15021/healthz/ready": dial tcp 10.134.0.145:15021: connect: connection refused... 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-744b57-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-744b57-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Normal Killing pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-6smj6 Stopping container istio-proxy 28m Normal Killing pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-gspwm Stopping container istio-proxy 28m Warning ProbeError pod/e2e-inf-a-744b57-openshift-default-6c5cbf6bb9-zctmd Startup probe error: Get "http://10.132.0.220:15021/healthz/ready": dial tcp 10.132.0.220:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-inf-a-744b57-openshift-default-6c5cbf6bb9-zctmd Startup probe failed: Get "http://10.132.0.220:15021/healthz/ready": dial tcp 10.132.0.220:15021: connect: connection refused 28m Normal Killing pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-vwbfz Stopping container istio-proxy 28m Normal Killing pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-8pztb Stopping container istio-proxy 28m Normal Killing pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-npw9t Stopping container istio-proxy 28m Normal Killing pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-28g9t Stopping container istio-proxy 28m Warning ProbeError pod/e2e-inf-b-1e13df-openshift-default-58bb9b8669-c6zvx Startup probe error: Get "http://10.132.0.214:15021/healthz/ready": dial tcp 10.132.0.214:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-inf-b-1e13df-openshift-default-58bb9b8669-c6zvx Startup probe failed: Get "http://10.132.0.214:15021/healthz/ready": dial tcp 10.132.0.214:15021: connect: connection refused 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-1e13df-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Normal SuccessfulCreate replicaset/e2e-inf-a-89a3f6-openshift-default-547f79fc89 Created pod: e2e-inf-a-89a3f6-openshift-default-547f79fc89-d75jj 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-89a3f6-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 28m Normal ScalingReplicaSet deployment/e2e-inf-a-89a3f6-openshift-default Scaled up replica set e2e-inf-a-89a3f6-openshift-default-547f79fc89 from 1 to 2 28m Warning Unhealthy pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-6smj6 Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning ProbeError pod/e2e-worker-w0-ae63bf-openshift-default-556d46cdff-6smj6 Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-1e13df-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Normal Pulled pod/e2e-inf-a-89a3f6-openshift-default-547f79fc89-d75jj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/e2e-inf-a-89a3f6-openshift-default-547f79fc89-d75jj Add eth0 [10.132.0.233/23] from ovn-kubernetes 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-22af66-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Normal Created pod/e2e-inf-a-89a3f6-openshift-default-547f79fc89-d75jj Container created 28m Normal Started pod/e2e-inf-a-89a3f6-openshift-default-547f79fc89-d75jj Container started 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-22af66-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Normal Killing pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-5bwkh Stopping container istio-proxy 28m Normal SuccessfulDelete replicaset/e2e-worker-w3-ce69f3-openshift-default-6f98547647 Deleted pod: e2e-worker-w3-ce69f3-openshift-default-6f98547647-5bwkh 28m Normal ScalingReplicaSet deployment/e2e-worker-w3-ce69f3-openshift-default Scaled down replica set e2e-worker-w3-ce69f3-openshift-default-6f98547647 from 6 to 5 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-ce69f3-openshift-default New size: 5; reason: All metrics below target 28m Warning Unhealthy pod/e2e-inf-a-89a3f6-openshift-default-547f79fc89-d75jj Startup probe failed: Get "http://10.132.0.233:15021/healthz/ready": dial tcp 10.132.0.233:15021: connect: connection refused 28m Warning ProbeError pod/e2e-inf-a-89a3f6-openshift-default-547f79fc89-d75jj Startup probe error: Get "http://10.132.0.233:15021/healthz/ready": dial tcp 10.132.0.233:15021: connect: connection refused... 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-1e13df-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-1e13df-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-22af66-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-22af66-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Normal Killing pod/e2e-api-b-156838-openshift-default-6cc4b565b5-ljxwd Stopping container istio-proxy 28m Normal Killing pod/e2e-api-a-d1d256-openshift-default-979b4ccbf-lm6hp Stopping container istio-proxy 28m Normal ScalingReplicaSet deployment/e2e-inf-a-3e35df-openshift-default Scaled up replica set e2e-inf-a-3e35df-openshift-default-67646bf575 from 0 to 1 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-744b57-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-744b57-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Normal SuccessfulCreate replicaset/e2e-inf-a-3e35df-openshift-default-67646bf575 Created pod: e2e-inf-a-3e35df-openshift-default-67646bf575-7psjn 28m Normal AddedInterface pod/e2e-inf-a-3e35df-openshift-default-67646bf575-7psjn Add eth0 [10.133.0.96/23] from ovn-kubernetes 28m Normal AddedLabel gateway/e2e-inf-a-3e35df Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-3e35df 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-3e35df Scaled up replica set payload-processing-e2e-inf-a-3e35df-7f747b7fb from 0 to 1 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-3e35df-7874fb7595 Created pod: payload-pre-processing-e2e-inf-a-3e35df-7874fb7595-2v6xk 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-3e35df-7f747b7fb Created pod: payload-processing-e2e-inf-a-3e35df-7f747b7fb-gdjfs 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-3e35df-7874fb7595 Error creating: pods "payload-pre-processing-e2e-inf-a-3e35df-7874fb7595-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-3e35df: serviceaccount "payload-processing-e2e-inf-a-3e35df" not found 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-3e35df Scaled up replica set payload-pre-processing-e2e-inf-a-3e35df-7874fb7595 from 0 to 1 28m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-3e35df-7f747b7fb Error creating: pods "payload-processing-e2e-inf-a-3e35df-7f747b7fb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-3e35df: serviceaccount "payload-processing-e2e-inf-a-3e35df" not found 28m Normal Started pod/payload-processing-e2e-inf-a-3e35df-7f747b7fb-gdjfs Container started 28m Normal Pulled pod/payload-pre-processing-e2e-inf-a-3e35df-7874fb7595-2v6xk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Started pod/payload-pre-processing-e2e-inf-a-3e35df-7874fb7595-2v6xk Container started 28m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-3e35df-7874fb7595-2v6xk Add eth0 [10.132.0.236/23] from ovn-kubernetes 28m Normal Created pod/payload-processing-e2e-inf-a-3e35df-7f747b7fb-gdjfs Container created 28m Normal Created pod/payload-pre-processing-e2e-inf-a-3e35df-7874fb7595-2v6xk Container created 28m Normal Pulled pod/payload-processing-e2e-inf-a-3e35df-7f747b7fb-gdjfs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/payload-processing-e2e-inf-a-3e35df-7f747b7fb-gdjfs Add eth0 [10.132.0.235/23] from ovn-kubernetes 28m Normal Created pod/e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c-qj56r Container created 28m Normal Started pod/e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c-qj56r Container started 28m Normal Created pod/e2e-inf-a-579ddb-openshift-default-677f4b6845-mwmnb Container created 28m Normal Pulled pod/e2e-inf-a-579ddb-openshift-default-677f4b6845-mwmnb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Pulled pod/e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c-qj56r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Started pod/e2e-inf-a-579ddb-openshift-default-677f4b6845-mwmnb Container started 28m Normal Killing pod/payload-processing-e2e-api-b-22af66-7b5b988746-mm9zl Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-api-b-22af66-5bf7f46cd7-ffx8p Stopping container payload-pre-processing 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-579ddb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-579ddb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning ProbeError pod/e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c-qj56r Startup probe error: Get "http://10.132.0.229:15021/healthz/ready": dial tcp 10.132.0.229:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c-qj56r Startup probe failed: Get "http://10.132.0.229:15021/healthz/ready": dial tcp 10.132.0.229:15021: connect: connection refused 28m Warning ProbeError pod/e2e-inf-a-579ddb-openshift-default-677f4b6845-mwmnb Startup probe error: Get "http://10.132.0.201:15021/healthz/ready": dial tcp 10.132.0.201:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-inf-a-579ddb-openshift-default-677f4b6845-mwmnb Startup probe failed: Get "http://10.132.0.201:15021/healthz/ready": dial tcp 10.132.0.201:15021: connect: connection refused 28m Normal AddedInterface pod/e2e-inf-b-1e13df-openshift-default-58bb9b8669-cf9fq Add eth0 [10.133.0.98/23] from ovn-kubernetes 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-1e13df-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 28m Normal Pulled pod/e2e-inf-b-1e13df-openshift-default-58bb9b8669-cf9fq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal ScalingReplicaSet deployment/e2e-inf-b-1e13df-openshift-default Scaled up replica set e2e-inf-b-1e13df-openshift-default-58bb9b8669 from 1 to 2 28m Normal Started pod/e2e-inf-b-1e13df-openshift-default-58bb9b8669-cf9fq Container started 28m Normal SuccessfulCreate replicaset/e2e-inf-b-1e13df-openshift-default-58bb9b8669 Created pod: e2e-inf-b-1e13df-openshift-default-58bb9b8669-cf9fq 28m Normal Created pod/e2e-inf-b-1e13df-openshift-default-58bb9b8669-cf9fq Container created 27m Warning ProbeError pod/e2e-inf-b-1e13df-openshift-default-58bb9b8669-cf9fq Startup probe error: Get "http://10.133.0.98:15021/healthz/ready": dial tcp 10.133.0.98:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-inf-b-1e13df-openshift-default-58bb9b8669-cf9fq Startup probe failed: Get "http://10.133.0.98:15021/healthz/ready": dial tcp 10.133.0.98:15021: connect: connection refused 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-579ddb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-579ddb-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-ac37c8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-ac37c8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Normal SuccessfulCreate replicaset/e2e-inf-b-47110f-openshift-default-6dc75dc96c Created pod: e2e-inf-b-47110f-openshift-default-6dc75dc96c-jtp46 27m Normal ScalingReplicaSet deployment/e2e-inf-b-47110f-openshift-default Scaled up replica set e2e-inf-b-47110f-openshift-default-6dc75dc96c from 0 to 1 27m Normal AddedLabel gateway/e2e-inf-b-47110f Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-47110f 27m Normal AddedInterface pod/e2e-inf-b-47110f-openshift-default-6dc75dc96c-jtp46 Add eth0 [10.132.0.238/23] from ovn-kubernetes 27m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-47110f-65f4f5b6b4 Error creating: pods "payload-processing-e2e-inf-b-47110f-65f4f5b6b4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-47110f: serviceaccount "payload-processing-e2e-inf-b-47110f" not found 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-47110f Scaled up replica set payload-pre-processing-e2e-inf-b-47110f-7cdc4b47f5 from 0 to 1 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-47110f-7cdc4b47f5 Error creating: pods "payload-pre-processing-e2e-inf-b-47110f-7cdc4b47f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-47110f: serviceaccount "payload-processing-e2e-inf-b-47110f" not found 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-47110f Scaled up replica set payload-processing-e2e-inf-b-47110f-65f4f5b6b4 from 0 to 1 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-47110f-7cdc4b47f5 Created pod: payload-pre-processing-e2e-inf-b-47110f-7cdc4b47f5-2brw4 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-47110f-65f4f5b6b4 Created pod: payload-processing-e2e-inf-b-47110f-65f4f5b6b4-5528c 27m Normal Started pod/payload-pre-processing-e2e-inf-b-47110f-7cdc4b47f5-2brw4 Container started 27m Normal Pulled pod/payload-processing-e2e-inf-b-47110f-65f4f5b6b4-5528c Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal Created pod/payload-processing-e2e-inf-b-47110f-65f4f5b6b4-5528c Container created 27m Normal Started pod/payload-processing-e2e-inf-b-47110f-65f4f5b6b4-5528c Container started 27m Normal AddedInterface pod/payload-processing-e2e-inf-b-47110f-65f4f5b6b4-5528c Add eth0 [10.132.0.239/23] from ovn-kubernetes 27m Normal Created pod/payload-pre-processing-e2e-inf-b-47110f-7cdc4b47f5-2brw4 Container created 27m Normal Pulled pod/payload-pre-processing-e2e-inf-b-47110f-7cdc4b47f5-2brw4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-47110f-7cdc4b47f5-2brw4 Add eth0 [10.132.0.240/23] from ovn-kubernetes 27m Normal Created pod/e2e-inf-a-3e35df-openshift-default-67646bf575-7psjn Container created 27m Normal Pulled pod/e2e-inf-a-3e35df-openshift-default-67646bf575-7psjn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal Started pod/e2e-inf-a-3e35df-openshift-default-67646bf575-7psjn Container started 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-3e35df-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-ac37c8-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 27m Normal ScalingReplicaSet deployment/e2e-inf-b-ac37c8-openshift-default Scaled up replica set e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c from 1 to 2 27m Normal SuccessfulCreate replicaset/e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c Created pod: e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c-xmzml 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-3e35df-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Normal Pulled pod/e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c-xmzml Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c-xmzml Add eth0 [10.133.0.99/23] from ovn-kubernetes 27m Normal Created pod/e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c-xmzml Container created 27m Normal Started pod/e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c-xmzml Container started 27m Warning ProbeError pod/e2e-inf-a-3e35df-openshift-default-67646bf575-7psjn Startup probe error: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-inf-a-3e35df-openshift-default-67646bf575-7psjn Startup probe failed: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused 27m Warning ProbeError pod/e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c-xmzml Startup probe error: Get "http://10.133.0.99:15021/healthz/ready": dial tcp 10.133.0.99:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c-xmzml Startup probe failed: Get "http://10.133.0.99:15021/healthz/ready": dial tcp 10.133.0.99:15021: connect: connection refused 27m Normal Killing pod/e2e-api-b-22af66-openshift-default-5f7c494769-gqftx Stopping container istio-proxy 27m Warning ProbeError pod/e2e-api-b-22af66-openshift-default-5f7c494769-gqftx Readiness probe error: HTTP probe failed with statuscode: 503... 27m Warning Unhealthy pod/e2e-api-b-22af66-openshift-default-5f7c494769-gqftx Readiness probe failed: HTTP probe failed with statuscode: 503 27m Normal Pulled pod/e2e-inf-a-744b57-openshift-default-6c5cbf6bb9-b74k6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/e2e-inf-a-744b57-openshift-default-6c5cbf6bb9-b74k6 Add eth0 [10.134.0.150/23] from ovn-kubernetes 27m Normal ScalingReplicaSet deployment/e2e-inf-a-744b57-openshift-default Scaled up replica set e2e-inf-a-744b57-openshift-default-6c5cbf6bb9 from 1 to 2 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-744b57-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 27m Normal SuccessfulCreate replicaset/e2e-inf-a-744b57-openshift-default-6c5cbf6bb9 Created pod: e2e-inf-a-744b57-openshift-default-6c5cbf6bb9-b74k6 27m Normal Created pod/e2e-inf-a-744b57-openshift-default-6c5cbf6bb9-b74k6 Container created 27m Normal Started pod/e2e-inf-a-744b57-openshift-default-6c5cbf6bb9-b74k6 Container started 27m Normal Killing pod/payload-processing-e2e-shared-a-w2-59f017-694c87888d-lw9vk Stopping container payload-processing 27m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-59f017-5fb9d9fcf5-5xlfc Stopping container payload-pre-processing 27m Warning Unhealthy pod/e2e-inf-a-744b57-openshift-default-6c5cbf6bb9-b74k6 Startup probe failed: Get "http://10.134.0.150:15021/healthz/ready": dial tcp 10.134.0.150:15021: connect: connection refused 27m Warning ProbeError pod/e2e-inf-a-744b57-openshift-default-6c5cbf6bb9-b74k6 Startup probe error: Get "http://10.134.0.150:15021/healthz/ready": dial tcp 10.134.0.150:15021: connect: connection refused... 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-3e35df-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-3e35df-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 27m Normal Killing pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-jssx7 Stopping container istio-proxy 27m Normal Killing pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-d2xbp Stopping container istio-proxy 27m Normal ScalingReplicaSet deployment/e2e-worker-w2-b623ca-openshift-default Scaled down replica set e2e-worker-w2-b623ca-openshift-default-68847758ff from 6 to 5 27m Normal SuccessfulDelete replicaset/e2e-worker-w1-419d0a-openshift-default-55766cf857 Deleted pod: e2e-worker-w1-419d0a-openshift-default-55766cf857-jssx7 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-419d0a-openshift-default New size: 5; reason: All metrics below target 27m Normal ScalingReplicaSet deployment/e2e-worker-w1-419d0a-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w1-419d0a-openshift-default-55766cf857 from 6 to 5 27m Normal Killing pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-ddzt2 Stopping container istio-proxy 27m Normal ScalingReplicaSet deployment/e2e-worker-w4-ad96a4-openshift-default Scaled down replica set e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 from 6 to 5 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-b623ca-openshift-default New size: 5; reason: All metrics below target 27m Normal SuccessfulDelete replicaset/e2e-worker-w2-b623ca-openshift-default-68847758ff Deleted pod: e2e-worker-w2-b623ca-openshift-default-68847758ff-d2xbp 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-ad96a4-openshift-default New size: 5; reason: All metrics below target 27m Normal SuccessfulDelete replicaset/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 Deleted pod: e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-ddzt2 26m Normal Killing pod/e2e-shared-a-w2-59f017-openshift-default-ddc89b68c-lshmc Stopping container istio-proxy 26m Normal Killing pod/payload-processing-e2e-shared-b-w2-ca9d14-746f5b5598-6jt2z Stopping container payload-processing 26m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-ca9d14-5c478d7cc5-wtmp4 Stopping container payload-pre-processing 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-47110f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-47110f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Normal Created pod/e2e-inf-b-47110f-openshift-default-6dc75dc96c-jtp46 Container created 26m Normal Pulled pod/e2e-inf-b-47110f-openshift-default-6dc75dc96c-jtp46 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Started pod/e2e-inf-b-47110f-openshift-default-6dc75dc96c-jtp46 Container started 26m Warning Unhealthy pod/e2e-inf-b-47110f-openshift-default-6dc75dc96c-jtp46 Startup probe failed: Get "http://10.132.0.238:15021/healthz/ready": dial tcp 10.132.0.238:15021: connect: connection refused 26m Warning ProbeError pod/e2e-inf-b-47110f-openshift-default-6dc75dc96c-jtp46 Startup probe error: Get "http://10.132.0.238:15021/healthz/ready": dial tcp 10.132.0.238:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-shared-b-w2-ca9d14-openshift-default-65f96b67bf-klhmt Readiness probe failed: Get "http://10.132.0.171:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 26m Warning ProbeError pod/e2e-shared-b-w2-ca9d14-openshift-default-65f96b67bf-klhmt Readiness probe error: Get "http://10.132.0.171:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 26m Normal Killing pod/e2e-shared-b-w2-ca9d14-openshift-default-65f96b67bf-klhmt Stopping container istio-proxy 26m Normal Killing pod/payload-processing-e2e-worker-w2-b623ca-867779ffc6-6dg44 Stopping container payload-processing 26m Normal Killing pod/payload-pre-processing-e2e-worker-w2-b623ca-5c58d97f4c-p6qfh Stopping container payload-pre-processing 26m Normal ScalingReplicaSet deployment/e2e-worker-w6-01b24e-openshift-default Scaled down replica set e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77 from 6 to 5 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-01b24e-openshift-default New size: 5; reason: All metrics below target 26m Normal SuccessfulDelete replicaset/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77 Deleted pod: e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-bdpxx 26m Normal Killing pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-bdpxx Stopping container istio-proxy 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-47110f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-47110f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning Unhealthy pod/payload-processing-e2e-worker-w2-b623ca-867779ffc6-6dg44 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-b623ca-5c58d97f4c-p6qfh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Normal ScalingReplicaSet deployment/e2e-inf-a-3e35df-openshift-default Scaled up replica set e2e-inf-a-3e35df-openshift-default-67646bf575 from 1 to 2 26m Normal SuccessfulCreate replicaset/e2e-inf-a-3e35df-openshift-default-67646bf575 Created pod: e2e-inf-a-3e35df-openshift-default-67646bf575-jmxjq 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-3e35df-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 26m Normal Created pod/e2e-inf-a-3e35df-openshift-default-67646bf575-jmxjq Container created 26m Normal Pulled pod/e2e-inf-a-3e35df-openshift-default-67646bf575-jmxjq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/e2e-inf-a-3e35df-openshift-default-67646bf575-jmxjq Add eth0 [10.132.0.247/23] from ovn-kubernetes 26m Normal Started pod/e2e-inf-a-3e35df-openshift-default-67646bf575-jmxjq Container started 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-47110f-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 26m Normal Started pod/e2e-inf-b-47110f-openshift-default-6dc75dc96c-dcf9r Container started 26m Normal SuccessfulCreate replicaset/e2e-inf-b-47110f-openshift-default-6dc75dc96c Created pod: e2e-inf-b-47110f-openshift-default-6dc75dc96c-dcf9r 26m Normal AddedInterface pod/e2e-inf-b-47110f-openshift-default-6dc75dc96c-dcf9r Add eth0 [10.133.0.101/23] from ovn-kubernetes 26m Normal Pulled pod/e2e-inf-b-47110f-openshift-default-6dc75dc96c-dcf9r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Created pod/e2e-inf-b-47110f-openshift-default-6dc75dc96c-dcf9r Container created 26m Normal ScalingReplicaSet deployment/e2e-inf-b-47110f-openshift-default Scaled up replica set e2e-inf-b-47110f-openshift-default-6dc75dc96c from 1 to 2 26m Warning ProbeError pod/e2e-inf-a-3e35df-openshift-default-67646bf575-jmxjq Startup probe error: Get "http://10.132.0.247:15021/healthz/ready": dial tcp 10.132.0.247:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-inf-a-3e35df-openshift-default-67646bf575-jmxjq Startup probe failed: Get "http://10.132.0.247:15021/healthz/ready": dial tcp 10.132.0.247:15021: connect: connection refused 25m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 9; reason: All metrics below target 25m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5bc884dd87 Deleted pod: maas-default-gateway-openshift-default-5bc884dd87-5cd48 25m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-5bc884dd87 from 10 to 9 25m Normal Killing pod/maas-default-gateway-openshift-default-5bc884dd87-5cd48 Stopping container istio-proxy 25m Warning Unhealthy pod/e2e-inf-b-47110f-openshift-default-6dc75dc96c-dcf9r Startup probe failed: Get "http://10.133.0.101:15021/healthz/ready": dial tcp 10.133.0.101:15021: connect: connection refused 25m Warning ProbeError pod/e2e-inf-b-47110f-openshift-default-6dc75dc96c-dcf9r Startup probe error: Get "http://10.133.0.101:15021/healthz/ready": dial tcp 10.133.0.101:15021: connect: connection refused... 25m Normal Killing pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-6z6zf Stopping container istio-proxy 25m Normal Killing pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-smclj Stopping container istio-proxy 25m Normal Killing pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-bsbrr Stopping container istio-proxy 25m Normal Killing pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-rbvpb Stopping container istio-proxy 25m Normal Killing pod/e2e-worker-w2-b623ca-openshift-default-68847758ff-5dpgx Stopping container istio-proxy 25m Normal Killing pod/payload-processing-e2e-inf-a-579ddb-6756775486-ghzlj Stopping container payload-processing 25m Normal Killing pod/payload-pre-processing-e2e-inf-a-579ddb-9b6756cf9-gfdfg Stopping container payload-pre-processing 24m Normal Killing pod/e2e-inf-a-579ddb-openshift-default-677f4b6845-mwmnb Stopping container istio-proxy 24m Warning Unhealthy pod/payload-processing-e2e-inf-a-579ddb-6756775486-ghzlj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Warning Unhealthy pod/e2e-inf-a-579ddb-openshift-default-677f4b6845-mwmnb Readiness probe failed: HTTP probe failed with statuscode: 503 24m Warning ProbeError pod/e2e-inf-a-579ddb-openshift-default-677f4b6845-mwmnb Readiness probe error: HTTP probe failed with statuscode: 503... 24m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-579ddb-9b6756cf9-gfdfg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Normal Killing pod/payload-processing-e2e-inf-b-1e13df-68d544b494-c828d Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-inf-b-1e13df-78b7ccf4bc-8ldn2 Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-inf-a-744b57-597d6c8986-4v2zh Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-inf-a-744b57-77cc66bfc9-2fcz9 Stopping container payload-pre-processing 24m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: All metrics below target 24m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-2zxhf 24m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-2zxhf Stopping container discovery 24m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 4 to 3 24m Normal Killing pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-fpzgc Stopping container istio-proxy 24m Normal SuccessfulDelete replicaset/e2e-worker-w3-ce69f3-openshift-default-6f98547647 Deleted pod: e2e-worker-w3-ce69f3-openshift-default-6f98547647-fpzgc 24m Normal ScalingReplicaSet deployment/e2e-worker-w3-ce69f3-openshift-default Scaled down replica set e2e-worker-w3-ce69f3-openshift-default-6f98547647 from 5 to 4 24m Normal SuccessfulDelete replicaset/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 Deleted pod: e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-gqmnf 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-ce69f3-openshift-default New size: 4; reason: All metrics below target 24m Normal Killing pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-gqmnf Stopping container istio-proxy 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-ad96a4-openshift-default New size: 4; reason: All metrics below target 24m Normal ScalingReplicaSet deployment/e2e-worker-w4-ad96a4-openshift-default Scaled down replica set e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 from 5 to 4 24m Normal Killing pod/payload-pre-processing-e2e-inf-a-89a3f6-db46ffd87-8l4s5 Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-inf-a-89a3f6-7b8cb866f4-5zqx4 Stopping container payload-processing 24m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: All metrics below target 24m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5bc884dd87 Deleted pod: maas-default-gateway-openshift-default-5bc884dd87-qdh96 24m Normal Killing pod/maas-default-gateway-openshift-default-5bc884dd87-qdh96 Stopping container istio-proxy 24m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-5bc884dd87 from 9 to 8 24m Normal Killing pod/e2e-inf-a-744b57-openshift-default-6c5cbf6bb9-b74k6 Stopping container istio-proxy 24m Normal Killing pod/e2e-inf-a-744b57-openshift-default-6c5cbf6bb9-zctmd Stopping container istio-proxy 24m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-744b57-77cc66bfc9-2fcz9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-1e13df-78b7ccf4bc-8ldn2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Normal Killing pod/e2e-inf-b-1e13df-openshift-default-58bb9b8669-cf9fq Stopping container istio-proxy 24m Normal Killing pod/e2e-inf-b-1e13df-openshift-default-58bb9b8669-c6zvx Stopping container istio-proxy 24m Warning Unhealthy pod/e2e-inf-b-1e13df-openshift-default-58bb9b8669-c6zvx Readiness probe failed: HTTP probe failed with statuscode: 503 24m Warning ProbeError pod/e2e-inf-b-1e13df-openshift-default-58bb9b8669-c6zvx Readiness probe error: HTTP probe failed with statuscode: 503... 24m Normal Killing pod/payload-pre-processing-e2e-inf-b-ac37c8-57cb47d8c-4cprd Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-inf-b-ac37c8-69dbd9fb6d-xjzdf Stopping container payload-processing 24m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-89a3f6-db46ffd87-8l4s5 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Normal Killing pod/e2e-inf-a-89a3f6-openshift-default-547f79fc89-d75jj Stopping container istio-proxy 24m Normal Killing pod/e2e-inf-a-89a3f6-openshift-default-547f79fc89-njwhz Stopping container istio-proxy 24m Warning ProbeError pod/e2e-inf-a-89a3f6-openshift-default-547f79fc89-njwhz Readiness probe error: HTTP probe failed with statuscode: 503... 24m Warning Unhealthy pod/e2e-inf-a-89a3f6-openshift-default-547f79fc89-njwhz Readiness probe failed: HTTP probe failed with statuscode: 503 24m Warning Unhealthy pod/payload-processing-e2e-inf-a-89a3f6-7b8cb866f4-5zqx4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Warning Unhealthy pod/e2e-inf-a-89a3f6-openshift-default-547f79fc89-d75jj Readiness probe failed: HTTP probe failed with statuscode: 503 24m Warning ProbeError pod/e2e-inf-a-89a3f6-openshift-default-547f79fc89-d75jj Readiness probe error: HTTP probe failed with statuscode: 503... 24m Normal Killing pod/payload-processing-e2e-worker-w6-01b24e-7b4dc88878-4fbph Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-worker-w6-01b24e-f7447dc8c-z5lr6 Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-inf-b-d651c4-7f667dbdfb-48tkv Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-inf-b-d651c4-75cbc576bc-tp8q9 Stopping container payload-pre-processing 24m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-89a3f6-db46ffd87-8l4s5 Readiness probe failed: timeout: failed to connect service "10.132.0.196:50052" within 1s: context deadline exceeded 23m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-ac37c8-57cb47d8c-4cprd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Normal Killing pod/e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c-xmzml Stopping container istio-proxy 23m Normal Killing pod/e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c-qj56r Stopping container istio-proxy 23m Normal Killing pod/e2e-inf-b-d651c4-openshift-default-775869444c-9wh89 Stopping container istio-proxy 23m Warning ProbeError pod/e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c-qj56r Readiness probe error: Get "http://10.132.0.229:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 23m Warning Unhealthy pod/e2e-inf-b-ac37c8-openshift-default-7c4d5bbb8c-qj56r Readiness probe failed: Get "http://10.132.0.229:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 23m Warning Unhealthy pod/payload-processing-e2e-worker-w6-01b24e-7b4dc88878-4fbph Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w6-01b24e-f7447dc8c-z5lr6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-d651c4-75cbc576bc-tp8q9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning ProbeError pod/e2e-inf-b-d651c4-openshift-default-775869444c-9wh89 Readiness probe error: Get "http://10.134.0.139:15021/healthz/ready": read tcp 10.134.0.2:60430->10.134.0.139:15021: read: connection reset by peer... 23m Warning Unhealthy pod/payload-processing-e2e-inf-b-d651c4-7f667dbdfb-48tkv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning Unhealthy pod/e2e-inf-b-d651c4-openshift-default-775869444c-9wh89 Readiness probe failed: Get "http://10.134.0.139:15021/healthz/ready": read tcp 10.134.0.2:60430->10.134.0.139:15021: read: connection reset by peer 23m Normal Killing pod/payload-processing-e2e-shared-a-w3-fdb355-75d6685876-vw9vc Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-shared-a-w3-fdb355-699984f479-clsd4 Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-worker-w1-419d0a-688c6dfd6-8qmrl Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-worker-w1-419d0a-57688785ff-fjn4j Stopping container payload-pre-processing 23m Normal Killing pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-d7zl9 Stopping container istio-proxy 23m Normal Killing pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-vz4wm Stopping container istio-proxy 23m Normal Killing pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-t8wnk Stopping container istio-proxy 23m Normal Killing pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-9z5vs Stopping container istio-proxy 23m Normal Killing pod/payload-pre-processing-e2e-inf-a-3e35df-7874fb7595-2v6xk Stopping container payload-pre-processing 23m Normal Killing pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-57r8q Stopping container istio-proxy 23m Normal Killing pod/payload-processing-e2e-inf-a-3e35df-7f747b7fb-gdjfs Stopping container payload-processing 23m Warning Unhealthy pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-d7zl9 Readiness probe failed: HTTP probe failed with statuscode: 503 23m Warning ProbeError pod/e2e-worker-w6-01b24e-openshift-default-6f9bfc6d77-d7zl9 Readiness probe error: HTTP probe failed with statuscode: 503... 23m Normal Killing pod/e2e-shared-a-w3-fdb355-openshift-default-d67d8d799-b6tcd Stopping container istio-proxy 23m Normal Killing pod/payload-pre-processing-e2e-shared-b-w3-976b80-6869f686f9-9cs2n Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-shared-b-w3-976b80-646cd76644-rlkvm Stopping container payload-processing 23m Normal Killing pod/e2e-inf-a-3e35df-openshift-default-67646bf575-7psjn Stopping container istio-proxy 23m Normal Killing pod/e2e-inf-a-3e35df-openshift-default-67646bf575-jmxjq Stopping container istio-proxy 23m Warning Unhealthy pod/payload-processing-e2e-worker-w1-419d0a-688c6dfd6-8qmrl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w1-419d0a-57688785ff-fjn4j Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning ProbeError pod/e2e-inf-a-3e35df-openshift-default-67646bf575-7psjn Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning Unhealthy pod/e2e-inf-a-3e35df-openshift-default-67646bf575-7psjn Readiness probe failed: HTTP probe failed with statuscode: 503 23m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-3e35df-7874fb7595-2v6xk Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Normal Killing pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-rkbbc Stopping container istio-proxy 23m Normal Killing pod/payload-processing-e2e-inf-b-47110f-65f4f5b6b4-5528c Stopping container payload-processing 23m Normal Killing pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-6snr9 Stopping container istio-proxy 23m Normal Killing pod/payload-pre-processing-e2e-inf-b-47110f-7cdc4b47f5-2brw4 Stopping container payload-pre-processing 23m Warning ProbeError pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-rkbbc Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning Unhealthy pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-rkbbc Readiness probe failed: HTTP probe failed with statuscode: 503 23m Normal Killing pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-lr27f Stopping container istio-proxy 23m Normal Killing pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-6x6nn Stopping container istio-proxy 23m Normal Killing pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-fqcxb Stopping container istio-proxy 23m Normal Killing pod/e2e-shared-b-w3-976b80-openshift-default-69d4fb4fc9-6sdwr Stopping container istio-proxy 23m Warning ProbeError pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-6x6nn Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning Unhealthy pod/e2e-worker-w1-419d0a-openshift-default-55766cf857-6x6nn Readiness probe failed: HTTP probe failed with statuscode: 503 22m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5bc884dd87 Deleted pod: maas-default-gateway-openshift-default-5bc884dd87-q4g9n 22m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: All metrics below target 22m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-5bc884dd87 from 8 to 7 22m Normal Killing pod/maas-default-gateway-openshift-default-5bc884dd87-q4g9n Stopping container istio-proxy 22m Normal Killing pod/e2e-inf-b-47110f-openshift-default-6dc75dc96c-dcf9r Stopping container istio-proxy 22m Normal Killing pod/e2e-inf-b-47110f-openshift-default-6dc75dc96c-jtp46 Stopping container istio-proxy 22m Warning Unhealthy pod/e2e-inf-b-47110f-openshift-default-6dc75dc96c-jtp46 Readiness probe failed: HTTP probe failed with statuscode: 503 22m Normal Killing pod/payload-processing-e2e-worker-w3-ce69f3-598446f65b-z7qlw Stopping container payload-processing 22m Warning ProbeError pod/e2e-inf-b-47110f-openshift-default-6dc75dc96c-jtp46 Readiness probe error: HTTP probe failed with statuscode: 503... 22m Normal Killing pod/payload-pre-processing-e2e-worker-w3-ce69f3-f897b54f5-hv6zc Stopping container payload-pre-processing 22m Warning ProbeError pod/e2e-inf-b-47110f-openshift-default-6dc75dc96c-dcf9r Readiness probe error: HTTP probe failed with statuscode: 503... 22m Warning Unhealthy pod/e2e-inf-b-47110f-openshift-default-6dc75dc96c-dcf9r Readiness probe failed: HTTP probe failed with statuscode: 503 22m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-47110f-7cdc4b47f5-2brw4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Normal Killing pod/payload-pre-processing-e2e-worker-w4-ad96a4-746867c7d7-lnlgf Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-worker-w4-ad96a4-7988fc87b-bpvc9 Stopping container payload-processing 22m Normal Killing pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-kzwqv Stopping container istio-proxy 22m Normal Killing pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-mn4mp Stopping container istio-proxy 22m Normal Killing pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-whfvm Stopping container istio-proxy 22m Normal Killing pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-dxnzj Stopping container istio-proxy 22m Warning ProbeError pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-dxnzj Readiness probe error: HTTP probe failed with statuscode: 503... 22m Warning Unhealthy pod/e2e-worker-w3-ce69f3-openshift-default-6f98547647-dxnzj Readiness probe failed: HTTP probe failed with statuscode: 503 22m Warning Unhealthy pod/payload-processing-e2e-worker-w3-ce69f3-598446f65b-z7qlw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w3-ce69f3-f897b54f5-hv6zc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Normal ScalingReplicaSet deployment/e2e-worker-w4-ad96a4-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 from 4 to 3 22m Normal SuccessfulDelete replicaset/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7 Deleted pod: e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-jrr8j 22m Normal Killing pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-jrr8j Stopping container istio-proxy 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-ad96a4-openshift-default New size: 3; reason: All metrics below target 22m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-5bc884dd87 from 7 to 6 22m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5bc884dd87 Deleted pod: maas-default-gateway-openshift-default-5bc884dd87-frx85 22m Normal Killing pod/maas-default-gateway-openshift-default-5bc884dd87-frx85 Stopping container istio-proxy 22m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: All metrics below target 22m Warning Unhealthy pod/payload-processing-e2e-worker-w4-ad96a4-7988fc87b-bpvc9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w4-ad96a4-746867c7d7-lnlgf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Normal Killing pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-8b2zb Stopping container istio-proxy 22m Normal Killing pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-4g5wx Stopping container istio-proxy 22m Normal Killing pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-wkhn2 Stopping container istio-proxy 22m Warning ProbeError pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-8b2zb Readiness probe error: HTTP probe failed with statuscode: 503... 22m Warning Unhealthy pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-8b2zb Readiness probe failed: HTTP probe failed with statuscode: 503 22m Warning ProbeError pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-4g5wx Readiness probe error: HTTP probe failed with statuscode: 503... 22m Warning Unhealthy pod/e2e-worker-w4-ad96a4-openshift-default-5b7bbc6b7-4g5wx Readiness probe failed: HTTP probe failed with statuscode: 503 21m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-742kv 21m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 21m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-s6ldn 21m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-q4492 21m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-ndqct 21m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-s6ldn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-q4492 Container started 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-742kv Add eth0 [10.134.0.157/23] from ovn-kubernetes 21m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-742kv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-ndqct Container created 21m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-s6ldn Container started 21m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-742kv Container created 21m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-s6ldn Container created 21m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-742kv Container started 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-ndqct Add eth0 [10.133.0.104/23] from ovn-kubernetes 21m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-ndqct Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-q4492 Container created 21m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-q4492 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-q4492 Add eth0 [10.132.1.26/23] from ovn-kubernetes 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-s6ldn Add eth0 [10.134.0.156/23] from ovn-kubernetes 21m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-ndqct Container started 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-q4492 Startup probe failed: Get "http://10.132.1.26:15021/healthz/ready": dial tcp 10.132.1.26:15021: connect: connection refused 21m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-q4492 Startup probe error: Get "http://10.132.1.26:15021/healthz/ready": dial tcp 10.132.1.26:15021: connect: connection refused... 21m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-ndqct Startup probe error: Get "http://10.133.0.104:15021/healthz/ready": dial tcp 10.133.0.104:15021: connect: connection refused... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-ndqct Startup probe failed: Get "http://10.133.0.104:15021/healthz/ready": dial tcp 10.133.0.104:15021: connect: connection refused 21m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-s6ldn Startup probe error: Get "http://10.134.0.156:15021/healthz/ready": dial tcp 10.134.0.156:15021: connect: connection refused... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-742kv Startup probe failed: Get "http://10.134.0.157:15021/healthz/ready": dial tcp 10.134.0.157:15021: connect: connection refused 21m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-742kv Startup probe error: Get "http://10.134.0.157:15021/healthz/ready": dial tcp 10.134.0.157:15021: connect: connection refused... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-s6ldn Startup probe failed: Get "http://10.134.0.156:15021/healthz/ready": dial tcp 10.134.0.156:15021: connect: connection refused 21m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-p5c6s Stopping container discovery 21m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 21m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 21m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-p5c6s 20m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-5jm6z Readiness probe failed: Get "http://10.134.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 20m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-5jm6z Readiness probe error: Get "http://10.134.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-f5qx7 Readiness probe failed: Get "http://10.132.0.162:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 18m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-qj2g6 Readiness probe error: Get "http://10.132.0.161:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-qj2g6 Readiness probe failed: Get "http://10.132.0.161:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 18m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-f5qx7 Readiness probe error: Get "http://10.132.0.162:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 18m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 18m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 18m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-mfsgz Stopping container discovery 18m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-mfsgz 17m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-s6ldn Readiness probe error: Get "http://10.134.0.156:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 17m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-s6ldn Readiness probe failed: Get "http://10.134.0.156:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 17m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-742kv Readiness probe failed: Get "http://10.134.0.157:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 17m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-lnkcf Readiness probe failed: Get "http://10.134.0.107:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 17m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-lnkcf Readiness probe error: Get "http://10.134.0.107:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 17m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-742kv Readiness probe error: Get "http://10.134.0.157:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 11m Normal SuccessfulCreate replicaset/e2e-worker-single-main-98b568-openshift-default-96865bb7c Created pod: e2e-worker-single-main-98b568-openshift-default-96865bb7c-bxslr 11m Normal ScalingReplicaSet deployment/e2e-worker-single-main-98b568-openshift-default Scaled up replica set e2e-worker-single-main-98b568-openshift-default-96865bb7c from 0 to 1 11m Normal AddedLabel gateway/e2e-worker-single-main-98b568 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-98b568 11m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-5bc884dd87 from 10 to 1 11m Normal Killing pod/maas-default-gateway-openshift-default-5bc884dd87-ndqct Stopping container istio-proxy 11m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 1; reason: All metrics below target 11m Normal Killing pod/maas-default-gateway-openshift-default-5bc884dd87-5jm6z Stopping container istio-proxy 11m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5bc884dd87 Deleted pod: maas-default-gateway-openshift-default-5bc884dd87-5jm6z 11m Normal Killing pod/maas-default-gateway-openshift-default-5bc884dd87-zl27m Stopping container istio-proxy 11m Normal Killing pod/maas-default-gateway-openshift-default-5bc884dd87-s6ldn Stopping container istio-proxy 11m Normal Killing pod/maas-default-gateway-openshift-default-5bc884dd87-q4492 Stopping container istio-proxy 11m Normal AddedInterface pod/e2e-worker-single-main-98b568-openshift-default-96865bb7c-bxslr Add eth0 [10.134.0.173/23] from ovn-kubernetes 11m Normal Killing pod/maas-default-gateway-openshift-default-5bc884dd87-qj2g6 Stopping container istio-proxy 11m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5bc884dd87 Deleted pod: maas-default-gateway-openshift-default-5bc884dd87-f5qx7 11m Normal Killing pod/maas-default-gateway-openshift-default-5bc884dd87-742kv Stopping container istio-proxy 11m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5bc884dd87 Deleted pod: maas-default-gateway-openshift-default-5bc884dd87-zl27m 11m Normal Killing pod/maas-default-gateway-openshift-default-5bc884dd87-f5qx7 Stopping container istio-proxy 11m Normal Killing pod/maas-default-gateway-openshift-default-5bc884dd87-lnkcf Stopping container istio-proxy 11m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5bc884dd87 Deleted pod: maas-default-gateway-openshift-default-5bc884dd87-s6ldn 11m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5bc884dd87 Deleted pod: maas-default-gateway-openshift-default-5bc884dd87-ndqct 11m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5bc884dd87 Deleted pod: maas-default-gateway-openshift-default-5bc884dd87-742kv 11m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5bc884dd87 Deleted pod: maas-default-gateway-openshift-default-5bc884dd87-q4492 11m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5bc884dd87 Deleted pod: maas-default-gateway-openshift-default-5bc884dd87-qj2g6 11m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5bc884dd87 Deleted pod: maas-default-gateway-openshift-default-5bc884dd87-lnkcf 11m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-98b568-58446m98kd Add eth0 [10.134.0.175/23] from ovn-kubernetes 11m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-98b568 Scaled up replica set payload-pre-processing-e2e-worker-single-main-98b568-584467994c from 0 to 1 11m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-98b568 Scaled up replica set payload-processing-e2e-worker-single-main-98b568-75c89f78c6 from 0 to 1 11m Normal Pulled pod/payload-processing-e2e-worker-single-main-98b568-75c89f78cgkxbl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 11m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-98b568-584467994c Error creating: pods "payload-pre-processing-e2e-worker-single-main-98b568-584467994c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-98b568: serviceaccount "payload-processing-e2e-worker-single-main-98b568" not found 11m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-98b568-584467994c Created pod: payload-pre-processing-e2e-worker-single-main-98b568-58446m98kd 11m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-98b568-75c89f78cgkxbl Add eth0 [10.134.0.176/23] from ovn-kubernetes 11m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-98b568-75c89f78c6 Error creating: pods "payload-processing-e2e-worker-single-main-98b568-75c89f78c6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-98b568: serviceaccount "payload-processing-e2e-worker-single-main-98b568" not found 11m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-98b568-58446m98kd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 11m Normal Created pod/payload-pre-processing-e2e-worker-single-main-98b568-58446m98kd Container created 11m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-98b568-75c89f78c6 Created pod: payload-processing-e2e-worker-single-main-98b568-75c89f78cgkxbl 11m Normal Started pod/payload-pre-processing-e2e-worker-single-main-98b568-58446m98kd Container started 11m Normal Created pod/payload-processing-e2e-worker-single-main-98b568-75c89f78cgkxbl Container created 11m Normal Started pod/payload-processing-e2e-worker-single-main-98b568-75c89f78cgkxbl Container started 11m Normal Created pod/e2e-worker-single-main-98b568-openshift-default-96865bb7c-bxslr Container created 11m Normal Pulled pod/e2e-worker-single-main-98b568-openshift-default-96865bb7c-bxslr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 11m Normal Started pod/e2e-worker-single-main-98b568-openshift-default-96865bb7c-bxslr Container started 11m Warning Unhealthy pod/e2e-worker-single-main-98b568-openshift-default-96865bb7c-bxslr Startup probe failed: Get "http://10.134.0.173:15021/healthz/ready": dial tcp 10.134.0.173:15021: connect: connection refused 11m Warning ProbeError pod/e2e-worker-single-main-98b568-openshift-default-96865bb7c-bxslr Startup probe error: Get "http://10.134.0.173:15021/healthz/ready": dial tcp 10.134.0.173:15021: connect: connection refused... 11m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-98b568-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 11m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-98b568-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 11m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-qbqj6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 11m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-5bc884dd87 from 1 to 3 11m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-qbqj6 11m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-zgd28 11m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-qbqj6 Add eth0 [10.134.0.178/23] from ovn-kubernetes 11m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-zgd28 Add eth0 [10.132.1.27/23] from ovn-kubernetes 11m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 11m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-zgd28 Container started 11m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-qbqj6 Container created 11m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-zgd28 Container created 11m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-qbqj6 Container started 11m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-zgd28 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-qbqj6 Startup probe failed: Get "http://10.134.0.178:15021/healthz/ready": dial tcp 10.134.0.178:15021: connect: connection refused 11m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-qbqj6 Startup probe error: Get "http://10.134.0.178:15021/healthz/ready": dial tcp 10.134.0.178:15021: connect: connection refused... 11m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-zgd28 Startup probe error: Get "http://10.132.1.27:15021/healthz/ready": dial tcp 10.132.1.27:15021: connect: connection refused... 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-zgd28 Startup probe failed: Get "http://10.132.1.27:15021/healthz/ready": dial tcp 10.132.1.27:15021: connect: connection refused 11m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-98b568-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 11m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-98b568-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 10m Normal Killing pod/payload-processing-e2e-worker-single-main-98b568-75c89f78cgkxbl Stopping container payload-processing 10m Normal Killing pod/payload-pre-processing-e2e-worker-single-main-98b568-58446m98kd Stopping container payload-pre-processing 10m Normal Killing pod/e2e-worker-single-main-98b568-openshift-default-96865bb7c-bxslr Stopping container istio-proxy 10m Warning ProbeError pod/e2e-worker-single-main-98b568-openshift-default-96865bb7c-bxslr Readiness probe error: HTTP probe failed with statuscode: 503... 10m Warning Unhealthy pod/e2e-worker-single-main-98b568-openshift-default-96865bb7c-bxslr Readiness probe failed: HTTP probe failed with statuscode: 503 10m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-98b568-58446m98kd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 10m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-98b568-75c89f78cgkxbl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 9m58s Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 9m58s Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-s7gc9 9m58s Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-82vrf 9m58s Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-jrm57 9m58s Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-5bc884dd87 from 3 to 6 9m57s Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-s7gc9 Add eth0 [10.133.0.105/23] from ovn-kubernetes 9m57s Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-jrm57 Add eth0 [10.132.1.30/23] from ovn-kubernetes 9m57s Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-82vrf Add eth0 [10.134.0.180/23] from ovn-kubernetes 9m43s Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-5bc884dd87 from 6 to 10 9m43s Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: All metrics below target 9m42s Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-bjr2f Add eth0 [10.134.0.182/23] from ovn-kubernetes 9m42s Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-ctqxf Add eth0 [10.133.0.106/23] from ovn-kubernetes 9m42s Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-ndnrw Add eth0 [10.134.0.181/23] from ovn-kubernetes 9m42s Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-ftjzp Add eth0 [10.132.1.31/23] from ovn-kubernetes 8m44s Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-s7gc9 Container created 8m44s Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-ctqxf Container created 8m44s Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-s7gc9 Container started 8m44s Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-s7gc9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 8m44s Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-ctqxf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 8m44s Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-ctqxf Container started 8m43s Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-ftjzp Container started 8m43s Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-ftjzp Container created 8m43s Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-ftjzp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 8m42s Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-82vrf Container started 8m42s Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-ndnrw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 8m42s Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-jrm57 Container started 8m42s Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-jrm57 Container created 8m42s Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-82vrf Container created 8m42s Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-82vrf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 8m42s Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-bjr2f Container created 8m42s Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-bjr2f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 8m42s Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-jrm57 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 8m42s Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-bjr2f Container started 8m42s Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-ndnrw Container started 8m42s Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-ndnrw Container created 8m37s Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-s7gc9 Startup probe failed: Get "http://10.133.0.105:15021/healthz/ready": dial tcp 10.133.0.105:15021: connect: connection refused 8m37s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-ctqxf Startup probe error: Get "http://10.133.0.106:15021/healthz/ready": dial tcp 10.133.0.106:15021: connect: connection refused... 8m37s Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-ctqxf Startup probe failed: Get "http://10.133.0.106:15021/healthz/ready": dial tcp 10.133.0.106:15021: connect: connection refused 8m37s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-s7gc9 Startup probe error: Get "http://10.133.0.105:15021/healthz/ready": dial tcp 10.133.0.105:15021: connect: connection refused... 8m36s Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-ftjzp Startup probe failed: Get "http://10.132.1.31:15021/healthz/ready": dial tcp 10.132.1.31:15021: connect: connection refused 8m36s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-ftjzp Startup probe error: Get "http://10.132.1.31:15021/healthz/ready": dial tcp 10.132.1.31:15021: connect: connection refused... 8m35s Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-82vrf Startup probe failed: Get "http://10.134.0.180:15021/healthz/ready": dial tcp 10.134.0.180:15021: connect: connection refused 8m35s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-bjr2f Startup probe error: Get "http://10.134.0.182:15021/healthz/ready": dial tcp 10.134.0.182:15021: connect: connection refused... 8m35s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-ndnrw Startup probe error: Get "http://10.134.0.181:15021/healthz/ready": dial tcp 10.134.0.181:15021: connect: connection refused... 8m35s Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-ndnrw Startup probe failed: Get "http://10.134.0.181:15021/healthz/ready": dial tcp 10.134.0.181:15021: connect: connection refused 8m35s Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-jrm57 Startup probe failed: Get "http://10.132.1.30:15021/healthz/ready": dial tcp 10.132.1.30:15021: connect: connection refused 8m35s Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-bjr2f Startup probe failed: Get "http://10.134.0.182:15021/healthz/ready": dial tcp 10.134.0.182:15021: connect: connection refused 8m35s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-jrm57 Startup probe error: Get "http://10.132.1.30:15021/healthz/ready": dial tcp 10.132.1.30:15021: connect: connection refused... 8m35s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-82vrf Startup probe error: Get "http://10.134.0.180:15021/healthz/ready": dial tcp 10.134.0.180:15021: connect: connection refused... 8m32s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-s7gc9 Startup probe error: Get "http://10.133.0.105:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 8m32s Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-s7gc9 Startup probe failed: Get "http://10.133.0.105:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 8m30s Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-ctqxf Startup probe failed: Get "http://10.133.0.106:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 8m30s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-ftjzp Startup probe error: Get "http://10.132.1.31:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 8m30s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-ctqxf Startup probe error: Get "http://10.133.0.106:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 8m30s Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-ftjzp Startup probe failed: Get "http://10.132.1.31:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 8m28s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-jrm57 Startup probe error: Get "http://10.132.1.30:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 8m28s Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-bjr2f Startup probe failed: Get "http://10.134.0.182:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 8m28s Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-jrm57 Startup probe failed: Get "http://10.132.1.30:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 8m27s Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-ndnrw Startup probe failed: Get "http://10.134.0.181:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 8m27s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-ndnrw Startup probe error: Get "http://10.134.0.181:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 8m27s Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-82vrf Startup probe failed: Get "http://10.134.0.180:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 8m27s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-82vrf Startup probe error: Get "http://10.134.0.180:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 8m27s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-bjr2f Startup probe error: Get "http://10.134.0.182:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 8m16s Normal AddedInterface pod/e2e-worker-single-main-cb7041-openshift-default-776964d669mzkbp Add eth0 [10.134.0.186/23] from ovn-kubernetes 8m16s Normal SuccessfulCreate replicaset/e2e-worker-single-main-cb7041-openshift-default-776964d669 Created pod: e2e-worker-single-main-cb7041-openshift-default-776964d669mzkbp 8m16s Normal ScalingReplicaSet deployment/e2e-worker-single-main-cb7041-openshift-default Scaled up replica set e2e-worker-single-main-cb7041-openshift-default-776964d669 from 0 to 1 8m16s Normal AddedLabel gateway/e2e-worker-single-main-cb7041 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-cb7041 8m12s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-cb7041 Scaled up replica set payload-pre-processing-e2e-worker-single-main-cb7041-68c6dc6fb7 from 0 to 1 8m12s Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-cb7041-9c74d8c9d Error creating: pods "payload-processing-e2e-worker-single-main-cb7041-9c74d8c9d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-cb7041: serviceaccount "payload-processing-e2e-worker-single-main-cb7041" not found 8m12s Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-cb7041-68c6dc6fb7 Error creating: pods "payload-pre-processing-e2e-worker-single-main-cb7041-68c6dc6fb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-cb7041: serviceaccount "payload-processing-e2e-worker-single-main-cb7041" not found 8m12s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-cb7041-68c6dc6fb7 Created pod: payload-pre-processing-e2e-worker-single-main-cb7041-68c6d7lt67 8m12s Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-cb7041 Scaled up replica set payload-processing-e2e-worker-single-main-cb7041-9c74d8c9d from 0 to 1 8m12s Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-cb7041-9c74d8c9d Created pod: payload-processing-e2e-worker-single-main-cb7041-9c74d8c9dzmn2j 8m11s Normal Created pod/payload-pre-processing-e2e-worker-single-main-cb7041-68c6d7lt67 Container created 8m11s Normal Started pod/payload-processing-e2e-worker-single-main-cb7041-9c74d8c9dzmn2j Container started 8m11s Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-cb7041-68c6d7lt67 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m11s Normal Started pod/payload-pre-processing-e2e-worker-single-main-cb7041-68c6d7lt67 Container started 8m11s Normal Created pod/payload-processing-e2e-worker-single-main-cb7041-9c74d8c9dzmn2j Container created 8m11s Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-cb7041-68c6d7lt67 Add eth0 [10.134.0.188/23] from ovn-kubernetes 8m11s Normal AddedInterface pod/payload-processing-e2e-worker-single-main-cb7041-9c74d8c9dzmn2j Add eth0 [10.134.0.189/23] from ovn-kubernetes 8m11s Normal Pulled pod/payload-processing-e2e-worker-single-main-cb7041-9c74d8c9dzmn2j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 7m59s Normal Pulled pod/e2e-worker-single-main-cb7041-openshift-default-776964d669mzkbp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 7m59s Warning Unhealthy pod/e2e-worker-single-main-cb7041-openshift-default-776964d669mzkbp Readiness probe failed: Get "http://10.134.0.186:15021/healthz/ready": dial tcp 10.134.0.186:15021: connect: connection refused 7m59s Normal Created pod/e2e-worker-single-main-cb7041-openshift-default-776964d669mzkbp Container created 7m59s Normal Started pod/e2e-worker-single-main-cb7041-openshift-default-776964d669mzkbp Container started 7m59s Warning ProbeError pod/e2e-worker-single-main-cb7041-openshift-default-776964d669mzkbp Readiness probe error: Get "http://10.134.0.186:15021/healthz/ready": dial tcp 10.134.0.186:15021: connect: connection refused... 7m54s Warning Unhealthy pod/e2e-worker-single-main-cb7041-openshift-default-776964d669mzkbp Startup probe failed: Get "http://10.134.0.186:15021/healthz/ready": dial tcp 10.134.0.186:15021: connect: connection refused 7m54s Warning ProbeError pod/e2e-worker-single-main-cb7041-openshift-default-776964d669mzkbp Startup probe error: Get "http://10.134.0.186:15021/healthz/ready": dial tcp 10.134.0.186:15021: connect: connection refused... 7m46s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-cb7041-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 7m46s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-cb7041-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 7m31s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-cb7041-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 7m31s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-cb7041-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 7m5s Normal Killing pod/payload-processing-e2e-worker-single-main-cb7041-9c74d8c9dzmn2j Stopping container payload-processing 7m5s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-cb7041-68c6d7lt67 Stopping container payload-pre-processing 6m56s Normal Killing pod/e2e-worker-single-main-cb7041-openshift-default-776964d669mzkbp Stopping container istio-proxy 6m53s Warning Unhealthy pod/e2e-worker-single-main-cb7041-openshift-default-776964d669mzkbp Readiness probe failed: HTTP probe failed with statuscode: 503 6m53s Warning ProbeError pod/e2e-worker-single-main-cb7041-openshift-default-776964d669mzkbp Readiness probe error: HTTP probe failed with statuscode: 503... 6m49s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-cb7041-68c6d7lt67 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 6m49s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-cb7041-9c74d8c9dzmn2j Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 6m30s Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-s7gc9 Readiness probe failed: Get "http://10.133.0.105:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 6m30s Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-x2wnb Readiness probe failed: Get "http://10.133.0.18:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 6m30s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-x2wnb Readiness probe error: Get "http://10.133.0.18:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 6m15s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-s7gc9 Readiness probe error: Get "http://10.133.0.105:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 6m14s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-ctqxf Readiness probe error: Get "http://10.133.0.106:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 6m3s Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-zgd28 Readiness probe failed: Get "http://10.132.1.27:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 6m3s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-zgd28 Readiness probe error: Get "http://10.132.1.27:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 6m3s Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-qbqj6 Readiness probe failed: Get "http://10.134.0.178:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 6m3s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-qbqj6 Readiness probe error: Get "http://10.134.0.178:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 5m58s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-ftjzp Readiness probe error: Get "http://10.132.1.31:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 5m24s Normal AddedLabel gateway/e2e-ait-58847295 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-58847295 5m23s Normal AddedInterface pod/e2e-ait-58847295-openshift-default-8f844cdfc-m4s26 Add eth0 [10.134.0.198/23] from ovn-kubernetes 5m23s Normal SuccessfulCreate replicaset/e2e-ait-58847295-openshift-default-8f844cdfc Created pod: e2e-ait-58847295-openshift-default-8f844cdfc-m4s26 5m23s Normal ScalingReplicaSet deployment/e2e-ait-58847295-openshift-default Scaled up replica set e2e-ait-58847295-openshift-default-8f844cdfc from 0 to 1 5m19s Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-58847295-7cbc7fcfc5 Error creating: pods "payload-pre-processing-e2e-ait-58847295-7cbc7fcfc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-58847295: serviceaccount "payload-processing-e2e-ait-58847295" not found 5m19s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-58847295-7cbc7fcfc5 Created pod: payload-pre-processing-e2e-ait-58847295-7cbc7fcfc5-7srtc 5m19s Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-58847295 Scaled up replica set payload-processing-e2e-ait-58847295-865b6f598b from 0 to 1 5m19s Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-58847295-865b6f598b Created pod: payload-processing-e2e-ait-58847295-865b6f598b-q29b7 5m19s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-58847295 Scaled up replica set payload-pre-processing-e2e-ait-58847295-7cbc7fcfc5 from 0 to 1 5m19s Warning FailedCreate replicaset/payload-processing-e2e-ait-58847295-865b6f598b Error creating: pods "payload-processing-e2e-ait-58847295-865b6f598b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-58847295: serviceaccount "payload-processing-e2e-ait-58847295" not found 5m18s Normal Created pod/payload-pre-processing-e2e-ait-58847295-7cbc7fcfc5-7srtc Container created 5m18s Normal Started pod/payload-pre-processing-e2e-ait-58847295-7cbc7fcfc5-7srtc Container started 5m18s Normal Started pod/payload-processing-e2e-ait-58847295-865b6f598b-q29b7 Container started 5m18s Normal AddedInterface pod/payload-processing-e2e-ait-58847295-865b6f598b-q29b7 Add eth0 [10.134.0.199/23] from ovn-kubernetes 5m18s Normal Pulled pod/payload-pre-processing-e2e-ait-58847295-7cbc7fcfc5-7srtc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m18s Normal Pulled pod/payload-processing-e2e-ait-58847295-865b6f598b-q29b7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m18s Normal Created pod/payload-processing-e2e-ait-58847295-865b6f598b-q29b7 Container created 5m18s Normal AddedInterface pod/payload-pre-processing-e2e-ait-58847295-7cbc7fcfc5-7srtc Add eth0 [10.134.0.200/23] from ovn-kubernetes 5m8s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-58847295-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 5m8s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-58847295-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 5m6s Warning Unhealthy pod/e2e-ait-58847295-openshift-default-8f844cdfc-m4s26 Readiness probe failed: Get "http://10.134.0.198:15021/healthz/ready": dial tcp 10.134.0.198:15021: connect: connection refused 5m6s Normal Pulled pod/e2e-ait-58847295-openshift-default-8f844cdfc-m4s26 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 5m6s Normal Created pod/e2e-ait-58847295-openshift-default-8f844cdfc-m4s26 Container created 5m6s Normal Started pod/e2e-ait-58847295-openshift-default-8f844cdfc-m4s26 Container started 5m6s Warning ProbeError pod/e2e-ait-58847295-openshift-default-8f844cdfc-m4s26 Readiness probe error: Get "http://10.134.0.198:15021/healthz/ready": dial tcp 10.134.0.198:15021: connect: connection refused... 5m Warning ProbeError pod/e2e-ait-58847295-openshift-default-8f844cdfc-m4s26 Startup probe error: Get "http://10.134.0.198:15021/healthz/ready": dial tcp 10.134.0.198:15021: connect: connection refused... 5m Warning Unhealthy pod/e2e-ait-58847295-openshift-default-8f844cdfc-m4s26 Startup probe failed: Get "http://10.134.0.198:15021/healthz/ready": dial tcp 10.134.0.198:15021: connect: connection refused 4m54s Normal Killing pod/payload-pre-processing-e2e-ait-58847295-7cbc7fcfc5-7srtc Stopping container payload-pre-processing 4m54s Normal Killing pod/payload-processing-e2e-ait-58847295-865b6f598b-q29b7 Stopping container payload-processing 4m53s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-58847295-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m53s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-58847295-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m48s Normal Killing pod/e2e-ait-58847295-openshift-default-8f844cdfc-m4s26 Stopping container istio-proxy 4m47s Normal AddedLabel gateway/e2e-authcfg-http-64f2bb Added label istio.io/rev=openshift-gateway to gateway e2e-authcfg-http-64f2bb 4m46s Normal SuccessfulCreate replicaset/e2e-authcfg-http-64f2bb-openshift-default-6c68c554d9 Created pod: e2e-authcfg-http-64f2bb-openshift-default-6c68c554d9-dwc8m 4m46s Normal AddedInterface pod/e2e-authcfg-http-64f2bb-openshift-default-6c68c554d9-dwc8m Add eth0 [10.134.0.202/23] from ovn-kubernetes 4m46s Normal ScalingReplicaSet deployment/e2e-authcfg-http-64f2bb-openshift-default Scaled up replica set e2e-authcfg-http-64f2bb-openshift-default-6c68c554d9 from 0 to 1 4m44s Warning Unhealthy pod/e2e-ait-58847295-openshift-default-8f844cdfc-m4s26 Readiness probe failed: HTTP probe failed with statuscode: 503 4m44s Warning ProbeError pod/e2e-ait-58847295-openshift-default-8f844cdfc-m4s26 Readiness probe error: HTTP probe failed with statuscode: 503... 4m42s Normal Started pod/payload-pre-processing-e2e-authcfg-http-64f2bb-7dbf5f4555-gns4h Container started 4m42s Normal SuccessfulCreate replicaset/payload-processing-e2e-authcfg-http-64f2bb-5cbb5f6456 Created pod: payload-processing-e2e-authcfg-http-64f2bb-5cbb5f6456-x96sh 4m42s Normal Pulled pod/payload-processing-e2e-authcfg-http-64f2bb-5cbb5f6456-x96sh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 4m42s Normal Created pod/payload-processing-e2e-authcfg-http-64f2bb-5cbb5f6456-x96sh Container created 4m42s Normal Started pod/payload-processing-e2e-authcfg-http-64f2bb-5cbb5f6456-x96sh Container started 4m42s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-authcfg-http-64f2bb Scaled up replica set payload-pre-processing-e2e-authcfg-http-64f2bb-7dbf5f4555 from 0 to 1 4m42s Warning FailedCreate replicaset/payload-processing-e2e-authcfg-http-64f2bb-5cbb5f6456 Error creating: pods "payload-processing-e2e-authcfg-http-64f2bb-5cbb5f6456-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-64f2bb: serviceaccount "payload-processing-e2e-authcfg-http-64f2bb" not found 4m42s Normal Pulled pod/payload-pre-processing-e2e-authcfg-http-64f2bb-7dbf5f4555-gns4h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 4m42s Normal ScalingReplicaSet deployment/payload-processing-e2e-authcfg-http-64f2bb Scaled up replica set payload-processing-e2e-authcfg-http-64f2bb-5cbb5f6456 from 0 to 1 4m42s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-authcfg-http-64f2bb-7dbf5f4555 Created pod: payload-pre-processing-e2e-authcfg-http-64f2bb-7dbf5f4555-gns4h 4m42s Normal AddedInterface pod/payload-pre-processing-e2e-authcfg-http-64f2bb-7dbf5f4555-gns4h Add eth0 [10.134.0.204/23] from ovn-kubernetes 4m42s Warning FailedCreate replicaset/payload-pre-processing-e2e-authcfg-http-64f2bb-7dbf5f4555 Error creating: pods "payload-pre-processing-e2e-authcfg-http-64f2bb-7dbf5f4555-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-64f2bb: serviceaccount "payload-processing-e2e-authcfg-http-64f2bb" not found 4m42s Normal AddedInterface pod/payload-processing-e2e-authcfg-http-64f2bb-5cbb5f6456-x96sh Add eth0 [10.134.0.205/23] from ovn-kubernetes 4m42s Normal Created pod/payload-pre-processing-e2e-authcfg-http-64f2bb-7dbf5f4555-gns4h Container created 4m31s Normal Pulled pod/e2e-authcfg-http-64f2bb-openshift-default-6c68c554d9-dwc8m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 4m31s Normal Started pod/e2e-authcfg-http-64f2bb-openshift-default-6c68c554d9-dwc8m Container started 4m31s Normal Created pod/e2e-authcfg-http-64f2bb-openshift-default-6c68c554d9-dwc8m Container created 4m25s Warning Unhealthy pod/e2e-authcfg-http-64f2bb-openshift-default-6c68c554d9-dwc8m Startup probe failed: Get "http://10.134.0.202:15021/healthz/ready": dial tcp 10.134.0.202:15021: connect: connection refused 4m25s Warning ProbeError pod/e2e-authcfg-http-64f2bb-openshift-default-6c68c554d9-dwc8m Startup probe error: Get "http://10.134.0.202:15021/healthz/ready": dial tcp 10.134.0.202:15021: connect: connection refused... 4m1s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-64f2bb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m1s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-64f2bb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 2m58s Normal Killing pod/payload-pre-processing-e2e-authcfg-http-64f2bb-7dbf5f4555-gns4h Stopping container payload-pre-processing 2m58s Normal Killing pod/payload-processing-e2e-authcfg-http-64f2bb-5cbb5f6456-x96sh Stopping container payload-processing 2m45s Normal Killing pod/e2e-authcfg-http-64f2bb-openshift-default-6c68c554d9-dwc8m Stopping container istio-proxy 2m41s Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-ndnrw Readiness probe failed: Get "http://10.134.0.181:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m41s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-ndnrw Readiness probe error: Get "http://10.134.0.181:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 2m41s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-bjr2f Readiness probe error: Get "http://10.134.0.182:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)...