=== events in openshift-ingress (2026-11-02T16:07:01Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 60m Normal Scheduled pod/payload-processing-55fbbbf9dc-j4tg5 Successfully assigned openshift-ingress/payload-processing-55fbbbf9dc-j4tg5 to ip-10-0-135-103.us-west-1.compute.internal 50m Normal Scheduled pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-tbbpc Successfully assigned openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-tbbpc to ip-10-0-135-103.us-west-1.compute.internal 32m Normal Scheduled pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-j2s2j Successfully assigned openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-j2s2j to ip-10-0-135-103.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j Successfully assigned openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j to ip-10-0-139-197.us-west-1.compute.internal 32m Normal Scheduled pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k Successfully assigned openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k to ip-10-0-139-197.us-west-1.compute.internal 59m Normal Scheduled pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-ntwpq Successfully assigned openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-ntwpq to ip-10-0-135-103.us-west-1.compute.internal 32m Normal Scheduled pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-s87nf Successfully assigned openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-s87nf to ip-10-0-132-27.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-9rhdc Successfully assigned openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-9rhdc to ip-10-0-135-103.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-6tjz5 Successfully assigned openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-6tjz5 to ip-10-0-132-27.us-west-1.compute.internal 6m31s Normal Scheduled pod/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n Successfully assigned openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n to ip-10-0-139-197.us-west-1.compute.internal 66m Normal Scheduled pod/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-kgs4p Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-kgs4p to ip-10-0-132-27.us-west-1.compute.internal 32m Normal Scheduled pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-fszfw Successfully assigned openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-fszfw to ip-10-0-135-103.us-west-1.compute.internal 43m Normal Scheduled pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-fj6qk Successfully assigned openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-fj6qk to ip-10-0-135-103.us-west-1.compute.internal 60m Normal Scheduled pod/payload-pre-processing-7b98bb678c-74wfk Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-74wfk to ip-10-0-135-103.us-west-1.compute.internal 55m Normal Scheduled pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7 Successfully assigned openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7 to ip-10-0-139-197.us-west-1.compute.internal 32m Normal Scheduled pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt Successfully assigned openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt to ip-10-0-139-197.us-west-1.compute.internal 46m Normal Scheduled pod/maas-default-gateway-openshift-default-75fd9f5d47-zzbkt Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-zzbkt to ip-10-0-132-27.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp Successfully assigned openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp to ip-10-0-139-197.us-west-1.compute.internal 44m Normal Scheduled pod/e2e-shared-a-w1-895ac8-openshift-default-756985bc89-9bq5g Successfully assigned openshift-ingress/e2e-shared-a-w1-895ac8-openshift-default-756985bc89-9bq5g to ip-10-0-135-103.us-west-1.compute.internal 54m Normal Scheduled pod/maas-default-gateway-openshift-default-75fd9f5d47-xv4h4 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-xv4h4 to ip-10-0-135-103.us-west-1.compute.internal 6m23s Normal Scheduled pod/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq to ip-10-0-139-197.us-west-1.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-ait-5cbda1cc-6cd64c495-85bwn Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-5cbda1cc-6cd64c495-85bwn to ip-10-0-135-103.us-west-1.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-ait-748f2e23-5dd9db5667-tsk5t Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-748f2e23-5dd9db5667-tsk5t to ip-10-0-135-103.us-west-1.compute.internal 32m Normal Scheduled pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-2sdrk Successfully assigned openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-2sdrk to ip-10-0-132-27.us-west-1.compute.internal 45m Normal Scheduled pod/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2 Successfully assigned openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2 to ip-10-0-139-197.us-west-1.compute.internal 70m Normal Scheduled pod/router-default-684675bd5f-zwwbp Successfully assigned openshift-ingress/router-default-684675bd5f-zwwbp to ip-10-0-132-27.us-west-1.compute.internal 71m Warning FailedScheduling pod/router-default-684675bd5f-zwwbp no nodes available to schedule pods 76m Warning FailedScheduling pod/router-default-684675bd5f-zwwbp no nodes available to schedule pods 49m Normal Scheduled pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw Successfully assigned openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw to ip-10-0-139-197.us-west-1.compute.internal 59m Normal Scheduled pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-tlwnb Successfully assigned openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-tlwnb to ip-10-0-135-103.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-rvvqb Successfully assigned openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-rvvqb to ip-10-0-132-27.us-west-1.compute.internal 50m Normal Scheduled pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-b9xj5 Successfully assigned openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-b9xj5 to ip-10-0-132-27.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-njrmp Successfully assigned openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-njrmp to ip-10-0-135-103.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq Successfully assigned openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq to ip-10-0-139-197.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-5sfdr Successfully assigned openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-5sfdr to ip-10-0-135-103.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kpchw Successfully assigned openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kpchw to ip-10-0-132-27.us-west-1.compute.internal 59m Normal Scheduled pod/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4 to ip-10-0-139-197.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-cs984 Successfully assigned openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-cs984 to ip-10-0-135-103.us-west-1.compute.internal 54m Normal Scheduled pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kj2h7 Successfully assigned openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kj2h7 to ip-10-0-132-27.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h Successfully assigned openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h to ip-10-0-139-197.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-bnsmj Successfully assigned openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-bnsmj to ip-10-0-135-103.us-west-1.compute.internal 45m Normal Scheduled pod/e2e-ait-748f2e23-openshift-default-588f97c5cf-gqphg Successfully assigned openshift-ingress/e2e-ait-748f2e23-openshift-default-588f97c5cf-gqphg to ip-10-0-135-103.us-west-1.compute.internal 59m Normal Scheduled pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md Successfully assigned openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md to ip-10-0-139-197.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7 Successfully assigned openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7 to ip-10-0-139-197.us-west-1.compute.internal 54m Normal Scheduled pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7 Successfully assigned openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7 to ip-10-0-139-197.us-west-1.compute.internal 59m Normal Scheduled pod/payload-processing-e2e-worker-w5-db3931-587478dc8f-gbx6h Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-db3931-587478dc8f-gbx6h to ip-10-0-135-103.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-kzwzk Successfully assigned openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-kzwzk to ip-10-0-132-27.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-bq2wn Successfully assigned openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-bq2wn to ip-10-0-135-103.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq Successfully assigned openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq to ip-10-0-139-197.us-west-1.compute.internal 47m Normal Scheduled pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-njj69 Successfully assigned openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-njj69 to ip-10-0-132-27.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f Successfully assigned openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f to ip-10-0-139-197.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lrjs8 Successfully assigned openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lrjs8 to ip-10-0-135-103.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf Successfully assigned openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf to ip-10-0-139-197.us-west-1.compute.internal 59m Normal Scheduled pod/payload-processing-e2e-worker-w4-6dc69b-5fd6bbbf9f-gdvfc Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-6dc69b-5fd6bbbf9f-gdvfc to ip-10-0-132-27.us-west-1.compute.internal 54m Normal Scheduled pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-x5xvr Successfully assigned openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-x5xvr to ip-10-0-132-27.us-west-1.compute.internal 50m Normal Scheduled pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7 Successfully assigned openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7 to ip-10-0-139-197.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7 Successfully assigned openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7 to ip-10-0-139-197.us-west-1.compute.internal 59m Normal Scheduled pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb Successfully assigned openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb to ip-10-0-139-197.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb Successfully assigned openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb to ip-10-0-139-197.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-tcvnj Successfully assigned openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-tcvnj to ip-10-0-132-27.us-west-1.compute.internal 50m Normal Scheduled pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-x75hl Successfully assigned openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-x75hl to ip-10-0-132-27.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-vdwrl Successfully assigned openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-vdwrl to ip-10-0-132-27.us-west-1.compute.internal 50m Normal Scheduled pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527 Successfully assigned openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527 to ip-10-0-139-197.us-west-1.compute.internal 59m Normal Scheduled pod/payload-processing-e2e-worker-w3-42555d-7c658fbd6c-st4wb Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-42555d-7c658fbd6c-st4wb to ip-10-0-135-103.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-tfln7 Successfully assigned openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-tfln7 to ip-10-0-132-27.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt Successfully assigned openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt to ip-10-0-139-197.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b Successfully assigned openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b to ip-10-0-139-197.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7krbm Successfully assigned openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7krbm to ip-10-0-132-27.us-west-1.compute.internal 54m Normal Scheduled pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-c6252 Successfully assigned openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-c6252 to ip-10-0-135-103.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-derive-5ccdeb27-openshift-default-6b98454f65-tdjwb Successfully assigned openshift-ingress/e2e-derive-5ccdeb27-openshift-default-6b98454f65-tdjwb to ip-10-0-135-103.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-zntdc Successfully assigned openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-zntdc to ip-10-0-135-103.us-west-1.compute.internal 50m Normal Scheduled pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-bj5xt Successfully assigned openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-bj5xt to ip-10-0-135-103.us-west-1.compute.internal 50m Normal Scheduled pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-6dk9z Successfully assigned openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-6dk9z to ip-10-0-135-103.us-west-1.compute.internal 39m Normal Scheduled pod/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2 Successfully assigned openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2 to ip-10-0-139-197.us-west-1.compute.internal 50m Normal Scheduled pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-rnfds Successfully assigned openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-rnfds to ip-10-0-135-103.us-west-1.compute.internal 59m Normal Scheduled pod/payload-processing-e2e-worker-w2-de35cd-c8f875df5-fj5qn Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-de35cd-c8f875df5-fj5qn to ip-10-0-135-103.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-zr4vt Successfully assigned openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-zr4vt to ip-10-0-132-27.us-west-1.compute.internal 59m Normal Scheduled pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6 Successfully assigned openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6 to ip-10-0-139-197.us-west-1.compute.internal 66m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-klqrq Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq to ip-10-0-139-197.us-west-1.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-ait-ad0b4b5e-76f7f4795f-nkmnn Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-ad0b4b5e-76f7f4795f-nkmnn to ip-10-0-135-103.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-fzgw4 Successfully assigned openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-fzgw4 to ip-10-0-135-103.us-west-1.compute.internal 50m Normal Scheduled pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7shxl Successfully assigned openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7shxl to ip-10-0-132-27.us-west-1.compute.internal 60m Normal Scheduled pod/maas-default-gateway-openshift-default-75fd9f5d47-x44q9 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-x44q9 to ip-10-0-132-27.us-west-1.compute.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b to ip-10-0-139-197.us-west-1.compute.internal 59m Normal Scheduled pod/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5 to ip-10-0-139-197.us-west-1.compute.internal 50m Normal Scheduled pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-zsvr9 Successfully assigned openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-zsvr9 to ip-10-0-135-103.us-west-1.compute.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284 to ip-10-0-139-197.us-west-1.compute.internal 50m Normal Scheduled pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv Successfully assigned openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv to ip-10-0-139-197.us-west-1.compute.internal 40m Normal Scheduled pod/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk Successfully assigned openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk to ip-10-0-139-197.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f Successfully assigned openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f to ip-10-0-139-197.us-west-1.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg to ip-10-0-139-197.us-west-1.compute.internal 5m24s Normal Scheduled pod/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49 Successfully assigned openshift-ingress/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49 to ip-10-0-139-197.us-west-1.compute.internal 59m Normal Scheduled pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-sv6lk Successfully assigned openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-sv6lk to ip-10-0-135-103.us-west-1.compute.internal 59m Normal Scheduled pod/payload-processing-e2e-worker-w0-ba49d0-c49b65df5-qvfjh Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-ba49d0-c49b65df5-qvfjh to ip-10-0-135-103.us-west-1.compute.internal 16m Normal Scheduled pod/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv to ip-10-0-139-197.us-west-1.compute.internal 11m Normal Scheduled pod/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh to ip-10-0-139-197.us-west-1.compute.internal 40m Normal Scheduled pod/payload-processing-e2e-shared-b-w5-885c69-74b4b487c9-7nfnb Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w5-885c69-74b4b487c9-7nfnb to ip-10-0-135-103.us-west-1.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt to ip-10-0-139-197.us-west-1.compute.internal 40m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl to ip-10-0-139-197.us-west-1.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc to ip-10-0-139-197.us-west-1.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-shared-a-w5-6b2f61-78b8fcdb9c-ttp26 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w5-6b2f61-78b8fcdb9c-ttp26 to ip-10-0-135-103.us-west-1.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-shared-a-w3-b87e90-68f96cc4c7-fx9v5 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-b87e90-68f96cc4c7-fx9v5 to ip-10-0-132-27.us-west-1.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-559973-7b9b654d6f-pptpc Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-559973-7b9b654d6f-pptpc to ip-10-0-135-103.us-west-1.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-shared-a-w1-895ac8-69555fcc47-6n97d Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w1-895ac8-69555fcc47-6n97d to ip-10-0-135-103.us-west-1.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-mt-de9c4ba4-768fdffc87-hzb6r Successfully assigned openshift-ingress/payload-processing-e2e-mt-de9c4ba4-768fdffc87-hzb6r to ip-10-0-132-27.us-west-1.compute.internal 41m Normal Scheduled pod/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l Successfully assigned openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l to ip-10-0-139-197.us-west-1.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-mt-de9c4ba4-768fdffc87-dmd48 Successfully assigned openshift-ingress/payload-processing-e2e-mt-de9c4ba4-768fdffc87-dmd48 to ip-10-0-132-27.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 Successfully assigned openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 to ip-10-0-139-197.us-west-1.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-migrate-c82fc808-559f785f49-gcs86 Successfully assigned openshift-ingress/payload-processing-e2e-migrate-c82fc808-559f785f49-gcs86 to ip-10-0-135-103.us-west-1.compute.internal 33m Normal Scheduled pod/payload-processing-e2e-inf-b-4e4683-7784dc776c-n6ch5 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-4e4683-7784dc776c-n6ch5 to ip-10-0-135-103.us-west-1.compute.internal 39m Normal Scheduled pod/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss to ip-10-0-139-197.us-west-1.compute.internal 35m Normal Scheduled pod/payload-processing-e2e-inf-a-39098c-7dfd88f45c-5mh26 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-39098c-7dfd88f45c-5mh26 to ip-10-0-132-27.us-west-1.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-derive-5ccdeb27-86b6749bc7-wxdps Successfully assigned openshift-ingress/payload-processing-e2e-derive-5ccdeb27-86b6749bc7-wxdps to ip-10-0-135-103.us-west-1.compute.internal 5m24s Normal Scheduled pod/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x Successfully assigned openshift-ingress/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x to ip-10-0-139-197.us-west-1.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9 to ip-10-0-139-197.us-west-1.compute.internal 40m Normal Scheduled pod/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m Successfully assigned openshift-ingress/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m to ip-10-0-139-197.us-west-1.compute.internal 39m Normal Scheduled pod/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2 to ip-10-0-139-197.us-west-1.compute.internal 45m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-9d6dj Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj to ip-10-0-139-197.us-west-1.compute.internal 63m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-g5twx Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx to ip-10-0-139-197.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-rm6ws Successfully assigned openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-rm6ws to ip-10-0-132-27.us-west-1.compute.internal 5m32s Normal Scheduled pod/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7 Successfully assigned openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7 to ip-10-0-139-197.us-west-1.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-ait-ad0b4b5e-5b66d56bdf-l2twq Successfully assigned openshift-ingress/payload-processing-e2e-ait-ad0b4b5e-5b66d56bdf-l2twq to ip-10-0-135-103.us-west-1.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-ait-748f2e23-7597598dd5-27kfh Successfully assigned openshift-ingress/payload-processing-e2e-ait-748f2e23-7597598dd5-27kfh to ip-10-0-135-103.us-west-1.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-ait-5cbda1cc-7dbc678c5f-9mtq7 Successfully assigned openshift-ingress/payload-processing-e2e-ait-5cbda1cc-7dbc678c5f-9mtq7 to ip-10-0-135-103.us-west-1.compute.internal 6m23s Normal Scheduled pod/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh Successfully assigned openshift-ingress/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh to ip-10-0-139-197.us-west-1.compute.internal 54m Normal Scheduled pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft Successfully assigned openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft to ip-10-0-139-197.us-west-1.compute.internal 50m Normal Scheduled pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh Successfully assigned openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh to ip-10-0-139-197.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4 Successfully assigned openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4 to ip-10-0-139-197.us-west-1.compute.internal 55m Normal Scheduled pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-mm2v8 Successfully assigned openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-mm2v8 to ip-10-0-135-103.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-k62kg Successfully assigned openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-k62kg to ip-10-0-132-27.us-west-1.compute.internal 59m Normal Scheduled pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24 Successfully assigned openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24 to ip-10-0-139-197.us-west-1.compute.internal 54m Normal Scheduled pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-22679 Successfully assigned openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-22679 to ip-10-0-132-27.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-dzdcx Successfully assigned openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-dzdcx to ip-10-0-132-27.us-west-1.compute.internal 65m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-6b748b65857h8ws Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-6b748b65857h8ws to ip-10-0-132-27.us-west-1.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-mt-be5d503e-7cddf876cc-c2skm Successfully assigned openshift-ingress/payload-processing-e2e-mt-be5d503e-7cddf876cc-c2skm to ip-10-0-135-103.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-gn4z7 Successfully assigned openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-gn4z7 to ip-10-0-135-103.us-west-1.compute.internal 45m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-hh82k Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hh82k to ip-10-0-132-27.us-west-1.compute.internal 49m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-rm9wv Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rm9wv to ip-10-0-135-103.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6 Successfully assigned openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6 to ip-10-0-139-197.us-west-1.compute.internal 50m Normal Scheduled pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-2qxwf Successfully assigned openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-2qxwf to ip-10-0-132-27.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-6hqmk Successfully assigned openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-6hqmk to ip-10-0-132-27.us-west-1.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-derive-5ccdeb27-7b96689bd7-n76hv Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-5ccdeb27-7b96689bd7-n76hv to ip-10-0-132-27.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-d9msj Successfully assigned openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-d9msj to ip-10-0-135-103.us-west-1.compute.internal 65m Normal Scheduled pod/kube-auth-proxy-d9588f984-q7dr4 Successfully assigned openshift-ingress/kube-auth-proxy-d9588f984-q7dr4 to ip-10-0-135-103.us-west-1.compute.internal 59m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-c18e79-5fcfd69bc9-dmk26 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-c18e79-5fcfd69bc9-dmk26 to ip-10-0-132-27.us-west-1.compute.internal 16m Normal Scheduled pod/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl Successfully assigned openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl to ip-10-0-139-197.us-west-1.compute.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-39098c-85df849f9f-7vp5x Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-39098c-85df849f9f-7vp5x to ip-10-0-132-27.us-west-1.compute.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q to ip-10-0-139-197.us-west-1.compute.internal 33m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-4e4683-66d784d5c5-9b5lb Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-4e4683-66d784d5c5-9b5lb to ip-10-0-132-27.us-west-1.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-migrate-c82fc808-759c665f7f-79x9p Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-c82fc808-759c665f7f-79x9p to ip-10-0-135-103.us-west-1.compute.internal 44m Normal Scheduled pod/e2e-mt-be5d503e-openshift-default-f868fdd8f-8ql8s Successfully assigned openshift-ingress/e2e-mt-be5d503e-openshift-default-f868fdd8f-8ql8s to ip-10-0-135-103.us-west-1.compute.internal 65m Normal Scheduled pod/kube-auth-proxy-d9588f984-vxkqn Successfully assigned openshift-ingress/kube-auth-proxy-d9588f984-vxkqn to ip-10-0-132-27.us-west-1.compute.internal 36m Normal Scheduled pod/e2e-inf-a-39098c-openshift-default-76bff56b5f-b9c7h Successfully assigned openshift-ingress/e2e-inf-a-39098c-openshift-default-76bff56b5f-b9c7h to ip-10-0-132-27.us-west-1.compute.internal 42m Normal Scheduled pod/e2e-shared-a-w2-559973-openshift-default-5974f778dc-frvnh Successfully assigned openshift-ingress/e2e-shared-a-w2-559973-openshift-default-5974f778dc-frvnh to ip-10-0-135-103.us-west-1.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-mt-be5d503e-7dfdd45c7f-pvx7g Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-be5d503e-7dfdd45c7f-pvx7g to ip-10-0-135-103.us-west-1.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-mt-de9c4ba4-6564f5d4dc-866q8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-de9c4ba4-6564f5d4dc-866q8 to ip-10-0-132-27.us-west-1.compute.internal 59m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-db3931-5b4bd8c8c5-g8gwx Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-db3931-5b4bd8c8c5-g8gwx to ip-10-0-135-103.us-west-1.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-mt-de9c4ba4-6564f5d4dc-s8wg7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-de9c4ba4-6564f5d4dc-s8wg7 to ip-10-0-132-27.us-west-1.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w1-895ac8-65fcccbb9f-m5hp2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w1-895ac8-65fcccbb9f-m5hp2 to ip-10-0-132-27.us-west-1.compute.internal 61m Normal Scheduled pod/maas-default-gateway-openshift-default-75fd9f5d47-vdl45 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45 to ip-10-0-139-197.us-west-1.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-559973-c967685f-ls5fw Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-559973-c967685f-ls5fw to ip-10-0-135-103.us-west-1.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w3-b87e90-76d87d9fb7-n5vf8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b87e90-76d87d9fb7-n5vf8 to ip-10-0-132-27.us-west-1.compute.internal 11m Normal Scheduled pod/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk Successfully assigned openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk to ip-10-0-139-197.us-west-1.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w5-6b2f61-798968dcb7-2rkdk Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w5-6b2f61-798968dcb7-2rkdk to ip-10-0-135-103.us-west-1.compute.internal 59m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-6dc69b-68d55b7995-kl9q4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-6dc69b-68d55b7995-kl9q4 to ip-10-0-135-103.us-west-1.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr to ip-10-0-139-197.us-west-1.compute.internal 33m Normal Scheduled pod/maas-default-gateway-openshift-default-75fd9f5d47-4lr42 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-4lr42 to ip-10-0-132-27.us-west-1.compute.internal 54m Normal Scheduled pod/maas-default-gateway-openshift-default-75fd9f5d47-4qvql Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-4qvql to ip-10-0-135-103.us-west-1.compute.internal 40m Normal Scheduled pod/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg Successfully assigned openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg to ip-10-0-139-197.us-west-1.compute.internal 39m Normal Scheduled pod/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz Successfully assigned openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz to ip-10-0-139-197.us-west-1.compute.internal 54m Normal Scheduled pod/maas-default-gateway-openshift-default-75fd9f5d47-56p7g Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-56p7g to ip-10-0-135-103.us-west-1.compute.internal 33m Normal Scheduled pod/maas-default-gateway-openshift-default-75fd9f5d47-5j55g Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g to ip-10-0-139-197.us-west-1.compute.internal 46m Normal Scheduled pod/maas-default-gateway-openshift-default-75fd9f5d47-5l5bb Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5l5bb to ip-10-0-135-103.us-west-1.compute.internal 59m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-42555d-756d9fcdc-mt9h6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-42555d-756d9fcdc-mt9h6 to ip-10-0-135-103.us-west-1.compute.internal 46m Normal Scheduled pod/maas-default-gateway-openshift-default-75fd9f5d47-6qcng Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-6qcng to ip-10-0-132-27.us-west-1.compute.internal 47m Normal Scheduled pod/maas-default-gateway-openshift-default-75fd9f5d47-6ql9n Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-6ql9n to ip-10-0-135-103.us-west-1.compute.internal 61m Normal Scheduled pod/maas-default-gateway-openshift-default-75fd9f5d47-6rhn6 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-6rhn6 to ip-10-0-135-103.us-west-1.compute.internal 42m Normal Scheduled pod/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8 Successfully assigned openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8 to ip-10-0-139-197.us-west-1.compute.internal 60m Normal Scheduled pod/maas-default-gateway-openshift-default-75fd9f5d47-7n2bx Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-7n2bx to ip-10-0-135-103.us-west-1.compute.internal 40m Normal Scheduled pod/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p Successfully assigned openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p to ip-10-0-139-197.us-west-1.compute.internal 61m Normal Scheduled pod/maas-default-gateway-openshift-default-75fd9f5d47-9znhx Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-9znhx to ip-10-0-135-103.us-west-1.compute.internal 59m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-de35cd-7dddfd879-8kn45 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-de35cd-7dddfd879-8kn45 to ip-10-0-135-103.us-west-1.compute.internal 47m Normal Scheduled pod/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6 to ip-10-0-139-197.us-west-1.compute.internal 60m Normal Scheduled pod/maas-default-gateway-openshift-default-75fd9f5d47-f9td2 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2 to ip-10-0-139-197.us-west-1.compute.internal 42m Normal Scheduled pod/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999 Successfully assigned openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999 to ip-10-0-139-197.us-west-1.compute.internal 46m Normal Scheduled pod/maas-default-gateway-openshift-default-75fd9f5d47-fxb56 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56 to ip-10-0-139-197.us-west-1.compute.internal 33m Normal Scheduled pod/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g Successfully assigned openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g to ip-10-0-139-197.us-west-1.compute.internal 33m Normal Scheduled pod/maas-default-gateway-openshift-default-75fd9f5d47-gv6hc Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-gv6hc to ip-10-0-132-27.us-west-1.compute.internal 33m Normal Scheduled pod/maas-default-gateway-openshift-default-75fd9f5d47-j8c7c Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-j8c7c to ip-10-0-132-27.us-west-1.compute.internal 47m Normal Scheduled pod/maas-default-gateway-openshift-default-75fd9f5d47-lcdkf Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-lcdkf to ip-10-0-132-27.us-west-1.compute.internal 54m Normal Scheduled pod/maas-default-gateway-openshift-default-75fd9f5d47-lwjt6 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-lwjt6 to ip-10-0-135-103.us-west-1.compute.internal 59m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz to ip-10-0-139-197.us-west-1.compute.internal 42m Normal Scheduled pod/e2e-shared-a-w5-6b2f61-openshift-default-6466884b9f-5pxf2 Successfully assigned openshift-ingress/e2e-shared-a-w5-6b2f61-openshift-default-6466884b9f-5pxf2 to ip-10-0-135-103.us-west-1.compute.internal 54m Normal Scheduled pod/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn to ip-10-0-139-197.us-west-1.compute.internal 54m Normal Scheduled pod/maas-default-gateway-openshift-default-75fd9f5d47-qg548 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-qg548 to ip-10-0-132-27.us-west-1.compute.internal 44m Normal Scheduled pod/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm Successfully assigned openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm to ip-10-0-139-197.us-west-1.compute.internal 66m Normal Scheduled pod/maas-default-gateway-openshift-default-75fd9f5d47-qwd8j Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-qwd8j to ip-10-0-132-27.us-west-1.compute.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv to ip-10-0-139-197.us-west-1.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6 to ip-10-0-139-197.us-west-1.compute.internal 59m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-ba49d0-79cdb5449f-m7srf Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-ba49d0-79cdb5449f-m7srf to ip-10-0-135-103.us-west-1.compute.internal 16m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz to ip-10-0-139-197.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6 Successfully assigned openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6 to ip-10-0-139-197.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rftjh Successfully assigned openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rftjh to ip-10-0-132-27.us-west-1.compute.internal 11m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz to ip-10-0-139-197.us-west-1.compute.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w5-885c69-784df4fff-gn9bz Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w5-885c69-784df4fff-gn9bz to ip-10-0-132-27.us-west-1.compute.internal 77m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-684675bd5f from 0 to 1 76m Normal SuccessfulCreate replicaset/router-default-684675bd5f Created pod: router-default-684675bd5f-zwwbp 76m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 76m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 76m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 71m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 70m Warning FailedMount pod/router-default-684675bd5f-zwwbp MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 70m Warning FailedMount pod/router-default-684675bd5f-zwwbp MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 70m Warning FailedMount pod/router-default-684675bd5f-zwwbp MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 70m Warning FailedMount pod/router-default-684675bd5f-zwwbp MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 70m Normal Pulled pod/router-default-684675bd5f-zwwbp Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 70m Normal Started pod/router-default-684675bd5f-zwwbp Container started 70m Normal AddedInterface pod/router-default-684675bd5f-zwwbp Add eth0 [10.132.0.7/23] from ovn-kubernetes 70m Normal Created pod/router-default-684675bd5f-zwwbp Container created 66m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 66m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-klqrq Add eth0 [10.133.0.23/23] from ovn-kubernetes 66m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 66m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-klqrq 66m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-klqrq Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 66m Normal Started pod/istiod-openshift-gateway-7876bfcb74-klqrq Container started 66m Normal Created pod/istiod-openshift-gateway-7876bfcb74-klqrq Container created 66m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-klqrq Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.049s (4.049s including waiting). Image size: 185476477 bytes. 66m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-klqrq Readiness probe failed: HTTP probe failed with statuscode: 503 66m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-klqrq Readiness probe error: HTTP probe failed with statuscode: 503... 66m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75fd9f5d47 Created pod: maas-default-gateway-openshift-default-75fd9f5d47-qwd8j 66m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-75fd9f5d47 from 0 to 1 66m Normal EnsuringLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensuring load balancer 66m Normal ScalingReplicaSet deployment/openshift-ai-inference-openshift-ai-inference Scaled up replica set openshift-ai-inference-openshift-ai-inference-6b78c66f8c from 0 to 1 66m Normal AddedInterface pod/maas-default-gateway-openshift-default-75fd9f5d47-qwd8j Add eth0 [10.132.0.34/23] from ovn-kubernetes 66m Normal Pulling pod/maas-default-gateway-openshift-default-75fd9f5d47-qwd8j Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 66m Normal Pulling pod/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-kgs4p Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 66m Normal AddedInterface pod/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-kgs4p Add eth0 [10.132.0.35/23] from ovn-kubernetes 66m Normal SuccessfulCreate replicaset/openshift-ai-inference-openshift-ai-inference-6b78c66f8c Created pod: openshift-ai-inference-openshift-ai-inference-6b78c66f8c-kgs4p 66m Normal EnsuredLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensured load balancer 66m Normal Pulled pod/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-kgs4p Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 10.148s (10.148s including waiting). Image size: 183555333 bytes. 66m Normal Created pod/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-kgs4p Container created 66m Normal Started pod/openshift-ai-inference-openshift-ai-inference-6b78c66f8c-kgs4p Container started 66m Normal Pulled pod/maas-default-gateway-openshift-default-75fd9f5d47-qwd8j Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 10.683s (10.683s including waiting). Image size: 183555333 bytes. 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-d9588f984 from 0 to 2 65m Normal SuccessfulCreate replicaset/kube-auth-proxy-d9588f984 Created pod: kube-auth-proxy-d9588f984-vxkqn 65m Normal SuccessfulCreate replicaset/kube-auth-proxy-d9588f984 Created pod: kube-auth-proxy-d9588f984-q7dr4 65m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-6b748b6585 from 0 to 1 65m Normal AddedInterface pod/kube-auth-proxy-d9588f984-q7dr4 Add eth0 [10.134.0.23/23] from ovn-kubernetes 65m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-6b748b6585 Created pod: data-science-gateway-data-science-gateway-class-6b748b65857h8ws 65m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-6b748b65857h8ws Add eth0 [10.132.0.45/23] from ovn-kubernetes 65m Normal Pulling pod/kube-auth-proxy-d9588f984-vxkqn Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 65m Normal AddedInterface pod/kube-auth-proxy-d9588f984-vxkqn Add eth0 [10.132.0.44/23] from ovn-kubernetes 65m Normal Pulling pod/kube-auth-proxy-d9588f984-q7dr4 Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 65m Normal Pulled pod/data-science-gateway-data-science-gateway-class-6b748b65857h8ws Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal Created pod/data-science-gateway-data-science-gateway-class-6b748b65857h8ws Container created 65m Normal Started pod/data-science-gateway-data-science-gateway-class-6b748b65857h8ws Container started 65m Normal Started pod/kube-auth-proxy-d9588f984-q7dr4 Container started 65m Normal Started pod/kube-auth-proxy-d9588f984-vxkqn Container started 65m Normal Created pod/kube-auth-proxy-d9588f984-vxkqn Container created 65m Normal Pulled pod/kube-auth-proxy-d9588f984-vxkqn Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.704s (2.704s including waiting). Image size: 188904677 bytes. 65m Normal Pulled pod/kube-auth-proxy-d9588f984-q7dr4 Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 3.056s (3.056s including waiting). Image size: 188904677 bytes. 65m Normal Created pod/kube-auth-proxy-d9588f984-q7dr4 Container created 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 64m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-g5twx Add eth0 [10.133.0.33/23] from ovn-kubernetes 63m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-g5twx 63m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 63m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-g5twx Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 63m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-g5twx Container started 63m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-g5twx Container created 63m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-g5twx Readiness probe failed: HTTP probe failed with statuscode: 503 63m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-g5twx Readiness probe error: HTTP probe failed with statuscode: 503... 63m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 63m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-klqrq Stopping container discovery 63m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-klqrq 63m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Normal Created pod/maas-default-gateway-openshift-default-75fd9f5d47-qwd8j Container created 63m Normal Started pod/maas-default-gateway-openshift-default-75fd9f5d47-qwd8j Container started 63m Normal Pulled pod/maas-default-gateway-openshift-default-75fd9f5d47-qwd8j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-qwd8j Startup probe failed: Get "http://10.132.0.34:15021/healthz/ready": dial tcp 10.132.0.34:15021: connect: connection refused 63m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-qwd8j Startup probe error: Get "http://10.132.0.34:15021/healthz/ready": dial tcp 10.132.0.34:15021: connect: connection refused... 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 63m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 63m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 61m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-75fd9f5d47 from 1 to 4 61m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75fd9f5d47 Created pod: maas-default-gateway-openshift-default-75fd9f5d47-vdl45 61m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75fd9f5d47 Created pod: maas-default-gateway-openshift-default-75fd9f5d47-6rhn6 61m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 61m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75fd9f5d47 Created pod: maas-default-gateway-openshift-default-75fd9f5d47-9znhx 61m Normal Pulling pod/maas-default-gateway-openshift-default-75fd9f5d47-vdl45 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 61m Normal AddedInterface pod/maas-default-gateway-openshift-default-75fd9f5d47-vdl45 Add eth0 [10.133.0.41/23] from ovn-kubernetes 61m Normal AddedInterface pod/maas-default-gateway-openshift-default-75fd9f5d47-9znhx Add eth0 [10.134.0.31/23] from ovn-kubernetes 61m Normal Pulling pod/maas-default-gateway-openshift-default-75fd9f5d47-9znhx Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 61m Normal Pulling pod/maas-default-gateway-openshift-default-75fd9f5d47-6rhn6 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 61m Normal AddedInterface pod/maas-default-gateway-openshift-default-75fd9f5d47-6rhn6 Add eth0 [10.134.0.32/23] from ovn-kubernetes 61m Normal Pulled pod/maas-default-gateway-openshift-default-75fd9f5d47-vdl45 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.13s (4.13s including waiting). Image size: 183555333 bytes. 61m Normal Started pod/maas-default-gateway-openshift-default-75fd9f5d47-vdl45 Container started 61m Normal Pulled pod/maas-default-gateway-openshift-default-75fd9f5d47-6rhn6 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.633s (4.633s including waiting). Image size: 183555333 bytes. 61m Normal Created pod/maas-default-gateway-openshift-default-75fd9f5d47-vdl45 Container created 61m Normal Created pod/maas-default-gateway-openshift-default-75fd9f5d47-6rhn6 Container created 61m Normal Created pod/maas-default-gateway-openshift-default-75fd9f5d47-9znhx Container created 61m Normal Pulled pod/maas-default-gateway-openshift-default-75fd9f5d47-9znhx Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.837s (4.837s including waiting). Image size: 183555333 bytes. 61m Normal Started pod/maas-default-gateway-openshift-default-75fd9f5d47-6rhn6 Container started 61m Normal Started pod/maas-default-gateway-openshift-default-75fd9f5d47-9znhx Container started 61m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-vdl45 Startup probe error: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused... 61m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-vdl45 Startup probe failed: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused 61m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-6rhn6 Startup probe failed: Get "http://10.134.0.32:15021/healthz/ready": dial tcp 10.134.0.32:15021: connect: connection refused 61m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-9znhx Startup probe error: Get "http://10.134.0.31:15021/healthz/ready": dial tcp 10.134.0.31:15021: connect: connection refused... 61m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-9znhx Startup probe failed: Get "http://10.134.0.31:15021/healthz/ready": dial tcp 10.134.0.31:15021: connect: connection refused 61m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-6rhn6 Startup probe error: Get "http://10.134.0.32:15021/healthz/ready": dial tcp 10.134.0.32:15021: connect: connection refused... 61m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-vdl45 Startup probe failed: Get "http://10.133.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-vdl45 Startup probe error: Get "http://10.133.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-6rhn6 Startup probe error: Get "http://10.134.0.32:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-6rhn6 Startup probe failed: Get "http://10.134.0.32:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75fd9f5d47 Created pod: maas-default-gateway-openshift-default-75fd9f5d47-7n2bx 60m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75fd9f5d47 Created pod: maas-default-gateway-openshift-default-75fd9f5d47-x44q9 60m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75fd9f5d47 Created pod: maas-default-gateway-openshift-default-75fd9f5d47-f9td2 60m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 60m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-75fd9f5d47 from 4 to 7 60m Normal Created pod/maas-default-gateway-openshift-default-75fd9f5d47-x44q9 Container created 60m Normal Started pod/maas-default-gateway-openshift-default-75fd9f5d47-f9td2 Container started 60m Normal AddedInterface pod/maas-default-gateway-openshift-default-75fd9f5d47-7n2bx Add eth0 [10.134.0.34/23] from ovn-kubernetes 60m Normal Pulled pod/maas-default-gateway-openshift-default-75fd9f5d47-f9td2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal AddedInterface pod/maas-default-gateway-openshift-default-75fd9f5d47-x44q9 Add eth0 [10.132.0.51/23] from ovn-kubernetes 60m Normal Pulled pod/maas-default-gateway-openshift-default-75fd9f5d47-x44q9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Started pod/maas-default-gateway-openshift-default-75fd9f5d47-x44q9 Container started 60m Normal Pulled pod/maas-default-gateway-openshift-default-75fd9f5d47-7n2bx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Created pod/maas-default-gateway-openshift-default-75fd9f5d47-7n2bx Container created 60m Normal Started pod/maas-default-gateway-openshift-default-75fd9f5d47-7n2bx Container started 60m Normal AddedInterface pod/maas-default-gateway-openshift-default-75fd9f5d47-f9td2 Add eth0 [10.133.0.45/23] from ovn-kubernetes 60m Normal Created pod/maas-default-gateway-openshift-default-75fd9f5d47-f9td2 Container created 60m Normal SuccessfulCreate replicaset/payload-processing-55fbbbf9dc Created pod: payload-processing-55fbbbf9dc-j4tg5 60m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-55fbbbf9dc from 0 to 1 60m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 60m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-74wfk 60m Warning FailedCreate replicaset/payload-processing-55fbbbf9dc Error creating: pods "payload-processing-55fbbbf9dc-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 60m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 60m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-74wfk Add eth0 [10.134.0.35/23] from ovn-kubernetes 60m Normal Pulling pod/payload-pre-processing-7b98bb678c-74wfk Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 60m Normal AddedInterface pod/payload-processing-55fbbbf9dc-j4tg5 Add eth0 [10.134.0.36/23] from ovn-kubernetes 60m Normal Pulling pod/payload-processing-55fbbbf9dc-j4tg5 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 60m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-f9td2 Startup probe failed: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused 60m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-f9td2 Startup probe error: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused... 60m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-x44q9 Startup probe failed: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused 60m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-x44q9 Startup probe error: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused... 60m Normal Pulled pod/payload-pre-processing-7b98bb678c-74wfk Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.811s (2.811s including waiting). Image size: 125353963 bytes. 60m Normal Started pod/payload-processing-55fbbbf9dc-j4tg5 Container started 60m Normal Created pod/payload-processing-55fbbbf9dc-j4tg5 Container created 60m Normal Started pod/payload-pre-processing-7b98bb678c-74wfk Container started 60m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-7n2bx Startup probe error: Get "http://10.134.0.34:15021/healthz/ready": dial tcp 10.134.0.34:15021: connect: connection refused... 60m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-7n2bx Startup probe failed: Get "http://10.134.0.34:15021/healthz/ready": dial tcp 10.134.0.34:15021: connect: connection refused 60m Normal Created pod/payload-pre-processing-7b98bb678c-74wfk Container created 60m Normal Pulled pod/payload-processing-55fbbbf9dc-j4tg5 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.858s (2.858s including waiting). Image size: 125353963 bytes. 59m Normal AddedLabel gateway/e2e-worker-w3-42555d Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-42555d 59m Normal AddedLabel gateway/e2e-worker-w1-019efe Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-019efe 59m Normal AddedLabel gateway/e2e-worker-w5-db3931 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-db3931 59m Normal AddedLabel gateway/e2e-worker-w2-de35cd Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-de35cd 59m Normal AddedLabel gateway/e2e-worker-w4-6dc69b Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-6dc69b 59m Normal AddedLabel gateway/e2e-worker-w0-ba49d0 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-ba49d0 59m Normal ScalingReplicaSet deployment/e2e-worker-w2-de35cd-openshift-default Scaled up replica set e2e-worker-w2-de35cd-openshift-default-684fbf96fc from 0 to 1 59m Normal ScalingReplicaSet deployment/e2e-worker-w5-db3931-openshift-default Scaled up replica set e2e-worker-w5-db3931-openshift-default-5f47d69f9f from 0 to 1 59m Normal AddedLabel gateway/e2e-worker-w6-c18e79 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-c18e79 59m Normal AddedInterface pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-tlwnb Add eth0 [10.134.0.37/23] from ovn-kubernetes 59m Normal SuccessfulCreate replicaset/e2e-worker-w0-ba49d0-openshift-default-6b978b7557 Created pod: e2e-worker-w0-ba49d0-openshift-default-6b978b7557-sv6lk 59m Normal SuccessfulCreate replicaset/e2e-worker-w5-db3931-openshift-default-5f47d69f9f Created pod: e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24 59m Normal SuccessfulCreate replicaset/e2e-worker-w2-de35cd-openshift-default-684fbf96fc Created pod: e2e-worker-w2-de35cd-openshift-default-684fbf96fc-tlwnb 59m Normal ScalingReplicaSet deployment/e2e-worker-w0-ba49d0-openshift-default Scaled up replica set e2e-worker-w0-ba49d0-openshift-default-6b978b7557 from 0 to 1 59m Normal AddedInterface pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-sv6lk Add eth0 [10.134.0.38/23] from ovn-kubernetes 59m Normal ScalingReplicaSet deployment/e2e-worker-w4-6dc69b-openshift-default Scaled up replica set e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 from 0 to 1 59m Normal SuccessfulCreate replicaset/e2e-worker-w1-019efe-openshift-default-5d87cb7859 Created pod: e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6 59m Normal AddedInterface pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md Add eth0 [10.133.0.48/23] from ovn-kubernetes 59m Normal AddedInterface pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24 Add eth0 [10.133.0.47/23] from ovn-kubernetes 59m Normal SuccessfulCreate replicaset/e2e-worker-w3-42555d-openshift-default-6c8c6d89c Created pod: e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md 59m Normal ScalingReplicaSet deployment/e2e-worker-w1-019efe-openshift-default Scaled up replica set e2e-worker-w1-019efe-openshift-default-5d87cb7859 from 0 to 1 59m Normal SuccessfulCreate replicaset/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 Created pod: e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-ntwpq 59m Normal ScalingReplicaSet deployment/e2e-worker-w3-42555d-openshift-default Scaled up replica set e2e-worker-w3-42555d-openshift-default-6c8c6d89c from 0 to 1 59m Normal AddedInterface pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb Add eth0 [10.133.0.50/23] from ovn-kubernetes 59m Normal AddedInterface pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6 Add eth0 [10.133.0.49/23] from ovn-kubernetes 59m Normal SuccessfulCreate replicaset/e2e-worker-w6-c18e79-openshift-default-59544557d7 Created pod: e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb 59m Normal AddedInterface pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-ntwpq Add eth0 [10.134.0.39/23] from ovn-kubernetes 59m Normal ScalingReplicaSet deployment/e2e-worker-w6-c18e79-openshift-default Scaled up replica set e2e-worker-w6-c18e79-openshift-default-59544557d7 from 0 to 1 59m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-de35cd Scaled up replica set payload-processing-e2e-worker-w2-de35cd-c8f875df5 from 0 to 1 59m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-de35cd-c8f875df5 Created pod: payload-processing-e2e-worker-w2-de35cd-c8f875df5-fj5qn 59m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-de35cd-7dddfd879 Error creating: pods "payload-pre-processing-e2e-worker-w2-de35cd-7dddfd879-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-de35cd: serviceaccount "payload-processing-e2e-worker-w2-de35cd" not found 59m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-de35cd-c8f875df5 Error creating: pods "payload-processing-e2e-worker-w2-de35cd-c8f875df5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-de35cd: serviceaccount "payload-processing-e2e-worker-w2-de35cd" not found 59m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-de35cd-7dddfd879 Created pod: payload-pre-processing-e2e-worker-w2-de35cd-7dddfd879-8kn45 59m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-de35cd Scaled up replica set payload-pre-processing-e2e-worker-w2-de35cd-7dddfd879 from 0 to 1 59m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-de35cd-7dddfd879-8kn45 Add eth0 [10.134.0.40/23] from ovn-kubernetes 59m Normal Created pod/payload-processing-e2e-worker-w2-de35cd-c8f875df5-fj5qn Container created 59m Normal AddedInterface pod/payload-processing-e2e-worker-w2-de35cd-c8f875df5-fj5qn Add eth0 [10.134.0.41/23] from ovn-kubernetes 59m Normal Started pod/payload-pre-processing-e2e-worker-w2-de35cd-7dddfd879-8kn45 Container started 59m Normal Pulled pod/payload-processing-e2e-worker-w2-de35cd-c8f875df5-fj5qn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-de35cd-7dddfd879-8kn45 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal Created pod/payload-pre-processing-e2e-worker-w2-de35cd-7dddfd879-8kn45 Container created 59m Normal Started pod/payload-processing-e2e-worker-w2-de35cd-c8f875df5-fj5qn Container started 59m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-db3931 Scaled up replica set payload-processing-e2e-worker-w5-db3931-587478dc8f from 0 to 1 59m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-db3931-587478dc8f Error creating: pods "payload-processing-e2e-worker-w5-db3931-587478dc8f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-db3931: serviceaccount "payload-processing-e2e-worker-w5-db3931" not found 59m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-db3931 Scaled up replica set payload-pre-processing-e2e-worker-w5-db3931-5b4bd8c8c5 from 0 to 1 59m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-db3931-5b4bd8c8c5 Created pod: payload-pre-processing-e2e-worker-w5-db3931-5b4bd8c8c5-g8gwx 59m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-db3931-5b4bd8c8c5 Error creating: pods "payload-pre-processing-e2e-worker-w5-db3931-5b4bd8c8c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-db3931: serviceaccount "payload-processing-e2e-worker-w5-db3931" not found 59m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-db3931-587478dc8f Created pod: payload-processing-e2e-worker-w5-db3931-587478dc8f-gbx6h 59m Normal Created pod/payload-processing-e2e-worker-w5-db3931-587478dc8f-gbx6h Container created 59m Normal Started pod/payload-pre-processing-e2e-worker-w5-db3931-5b4bd8c8c5-g8gwx Container started 59m Normal Started pod/payload-processing-e2e-worker-w5-db3931-587478dc8f-gbx6h Container started 59m Normal Created pod/payload-pre-processing-e2e-worker-w5-db3931-5b4bd8c8c5-g8gwx Container created 59m Normal AddedInterface pod/payload-processing-e2e-worker-w5-db3931-587478dc8f-gbx6h Add eth0 [10.134.0.43/23] from ovn-kubernetes 59m Normal Pulled pod/payload-processing-e2e-worker-w5-db3931-587478dc8f-gbx6h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-db3931-5b4bd8c8c5-g8gwx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-db3931-5b4bd8c8c5-g8gwx Add eth0 [10.134.0.42/23] from ovn-kubernetes 59m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-ba49d0-79cdb5449f Created pod: payload-pre-processing-e2e-worker-w0-ba49d0-79cdb5449f-m7srf 59m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-ba49d0 Scaled up replica set payload-processing-e2e-worker-w0-ba49d0-c49b65df5 from 0 to 1 59m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-ba49d0 Scaled up replica set payload-pre-processing-e2e-worker-w0-ba49d0-79cdb5449f from 0 to 1 59m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-ba49d0-79cdb5449f Error creating: pods "payload-pre-processing-e2e-worker-w0-ba49d0-79cdb5449f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-ba49d0: serviceaccount "payload-processing-e2e-worker-w0-ba49d0" not found 59m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-ba49d0-c49b65df5 Error creating: pods "payload-processing-e2e-worker-w0-ba49d0-c49b65df5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-ba49d0: serviceaccount "payload-processing-e2e-worker-w0-ba49d0" not found 59m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-ba49d0-79cdb5449f-m7srf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal Pulled pod/payload-processing-e2e-worker-w0-ba49d0-c49b65df5-qvfjh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal Started pod/payload-pre-processing-e2e-worker-w0-ba49d0-79cdb5449f-m7srf Container started 59m Normal Started pod/payload-processing-e2e-worker-w0-ba49d0-c49b65df5-qvfjh Container started 59m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-ba49d0-79cdb5449f-m7srf Add eth0 [10.134.0.45/23] from ovn-kubernetes 59m Normal AddedInterface pod/payload-processing-e2e-worker-w0-ba49d0-c49b65df5-qvfjh Add eth0 [10.134.0.46/23] from ovn-kubernetes 59m Normal Created pod/payload-processing-e2e-worker-w0-ba49d0-c49b65df5-qvfjh Container created 59m Normal Created pod/payload-pre-processing-e2e-worker-w0-ba49d0-79cdb5449f-m7srf Container created 59m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-ba49d0-c49b65df5 Created pod: payload-processing-e2e-worker-w0-ba49d0-c49b65df5-qvfjh 59m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-019efe Scaled up replica set payload-pre-processing-e2e-worker-w1-019efe-8544574bbc from 0 to 1 59m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc Created pod: payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz 59m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-019efe-997bbb677 Created pod: payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5 59m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-019efe-997bbb677 Error creating: pods "payload-processing-e2e-worker-w1-019efe-997bbb677-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-019efe: serviceaccount "payload-processing-e2e-worker-w1-019efe" not found 59m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc Error creating: pods "payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-019efe: serviceaccount "payload-processing-e2e-worker-w1-019efe" not found 59m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-019efe Scaled up replica set payload-processing-e2e-worker-w1-019efe-997bbb677 from 0 to 1 59m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz Add eth0 [10.133.0.52/23] from ovn-kubernetes 59m Normal Pulling pod/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 59m Normal AddedInterface pod/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5 Add eth0 [10.133.0.53/23] from ovn-kubernetes 59m Normal Pulling pod/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 59m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-42555d Scaled up replica set payload-processing-e2e-worker-w3-42555d-7c658fbd6c from 0 to 1 59m Normal Started pod/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz Container started 59m Normal Started pod/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5 Container started 59m Normal Created pod/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz Container created 59m Normal Created pod/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5 Container created 59m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.098s (3.098s including waiting). Image size: 125353963 bytes. 59m Normal Pulled pod/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.891s (2.891s including waiting). Image size: 125353963 bytes. 59m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-42555d-7c658fbd6c Created pod: payload-processing-e2e-worker-w3-42555d-7c658fbd6c-st4wb 59m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-42555d-756d9fcdc-mt9h6 Add eth0 [10.134.0.47/23] from ovn-kubernetes 59m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-42555d-756d9fcdc Created pod: payload-pre-processing-e2e-worker-w3-42555d-756d9fcdc-mt9h6 59m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-42555d Scaled up replica set payload-pre-processing-e2e-worker-w3-42555d-756d9fcdc from 0 to 1 59m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-42555d-7c658fbd6c Error creating: pods "payload-processing-e2e-worker-w3-42555d-7c658fbd6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-42555d: serviceaccount "payload-processing-e2e-worker-w3-42555d" not found 59m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-42555d-756d9fcdc Error creating: pods "payload-pre-processing-e2e-worker-w3-42555d-756d9fcdc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-42555d: serviceaccount "payload-processing-e2e-worker-w3-42555d" not found 59m Normal Created pod/payload-processing-e2e-worker-w3-42555d-7c658fbd6c-st4wb Container created 59m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-42555d-756d9fcdc-mt9h6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/payload-processing-e2e-worker-w3-42555d-7c658fbd6c-st4wb Add eth0 [10.134.0.48/23] from ovn-kubernetes 59m Normal Created pod/payload-pre-processing-e2e-worker-w3-42555d-756d9fcdc-mt9h6 Container created 59m Normal Started pod/payload-pre-processing-e2e-worker-w3-42555d-756d9fcdc-mt9h6 Container started 59m Normal Pulled pod/payload-processing-e2e-worker-w3-42555d-7c658fbd6c-st4wb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal Started pod/payload-processing-e2e-worker-w3-42555d-7c658fbd6c-st4wb Container started 59m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-c18e79-5fcfd69bc9 Created pod: payload-pre-processing-e2e-worker-w6-c18e79-5fcfd69bc9-dmk26 59m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-c18e79 Scaled up replica set payload-pre-processing-e2e-worker-w6-c18e79-5fcfd69bc9 from 0 to 1 59m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-c18e79-688b5c688f Error creating: pods "payload-processing-e2e-worker-w6-c18e79-688b5c688f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-c18e79: serviceaccount "payload-processing-e2e-worker-w6-c18e79" not found 59m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-c18e79 Scaled up replica set payload-processing-e2e-worker-w6-c18e79-688b5c688f from 0 to 1 59m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-c18e79-5fcfd69bc9 Error creating: pods "payload-pre-processing-e2e-worker-w6-c18e79-5fcfd69bc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-c18e79: serviceaccount "payload-processing-e2e-worker-w6-c18e79" not found 59m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-c18e79-688b5c688f Created pod: payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4 59m Normal AddedInterface pod/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4 Add eth0 [10.133.0.55/23] from ovn-kubernetes 59m Normal Pulling pod/payload-pre-processing-e2e-worker-w6-c18e79-5fcfd69bc9-dmk26 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 59m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-c18e79-5fcfd69bc9-dmk26 Add eth0 [10.132.0.52/23] from ovn-kubernetes 59m Normal Pulled pod/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal Started pod/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4 Container started 59m Normal Created pod/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4 Container created 59m Normal Created pod/payload-pre-processing-e2e-worker-w6-c18e79-5fcfd69bc9-dmk26 Container created 59m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-c18e79-5fcfd69bc9-dmk26 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.972s (2.972s including waiting). Image size: 125353963 bytes. 59m Normal Started pod/payload-pre-processing-e2e-worker-w6-c18e79-5fcfd69bc9-dmk26 Container started 59m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-6dc69b Scaled up replica set payload-pre-processing-e2e-worker-w4-6dc69b-68d55b7995 from 0 to 1 59m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-6dc69b-5fd6bbbf9f Error creating: pods "payload-processing-e2e-worker-w4-6dc69b-5fd6bbbf9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-6dc69b: serviceaccount "payload-processing-e2e-worker-w4-6dc69b" not found 59m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-6dc69b Scaled up replica set payload-processing-e2e-worker-w4-6dc69b-5fd6bbbf9f from 0 to 1 59m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-6dc69b-68d55b7995 Error creating: pods "payload-pre-processing-e2e-worker-w4-6dc69b-68d55b7995-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-6dc69b: serviceaccount "payload-processing-e2e-worker-w4-6dc69b" not found 59m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-6dc69b-68d55b7995 Created pod: payload-pre-processing-e2e-worker-w4-6dc69b-68d55b7995-kl9q4 59m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-6dc69b-5fd6bbbf9f Created pod: payload-processing-e2e-worker-w4-6dc69b-5fd6bbbf9f-gdvfc 59m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-6dc69b-68d55b7995-kl9q4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal Created pod/payload-pre-processing-e2e-worker-w4-6dc69b-68d55b7995-kl9q4 Container created 59m Normal AddedInterface pod/payload-processing-e2e-worker-w4-6dc69b-5fd6bbbf9f-gdvfc Add eth0 [10.132.0.53/23] from ovn-kubernetes 59m Normal Pulled pod/payload-processing-e2e-worker-w4-6dc69b-5fd6bbbf9f-gdvfc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-6dc69b-68d55b7995-kl9q4 Add eth0 [10.134.0.49/23] from ovn-kubernetes 59m Normal Started pod/payload-pre-processing-e2e-worker-w4-6dc69b-68d55b7995-kl9q4 Container started 59m Normal Created pod/payload-processing-e2e-worker-w4-6dc69b-5fd6bbbf9f-gdvfc Container created 59m Normal Started pod/payload-processing-e2e-worker-w4-6dc69b-5fd6bbbf9f-gdvfc Container started 59m Normal Pulled pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md Container created 59m Normal Started pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md Container started 59m Normal Created pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-sv6lk Container created 59m Normal Pulled pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-sv6lk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Started pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-sv6lk Container started 59m Warning ProbeError pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md Startup probe error: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md Startup probe failed: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-sv6lk Startup probe error: Get "http://10.134.0.38:15021/healthz/ready": dial tcp 10.134.0.38:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-sv6lk Startup probe failed: Get "http://10.134.0.38:15021/healthz/ready": dial tcp 10.134.0.38:15021: connect: connection refused 59m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-ba49d0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-ba49d0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-42555d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-42555d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning ProbeError pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-tlwnb Readiness probe error: Get "http://10.134.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning Unhealthy pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-tlwnb Readiness probe failed: Get "http://10.134.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Normal Pulled pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-tlwnb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Started pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-tlwnb Container started 58m Normal Created pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-tlwnb Container created 58m Warning Unhealthy pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-tlwnb Startup probe failed: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-tlwnb Startup probe error: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused... 58m Normal Pulled pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Pulled pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Started pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6 Container started 58m Normal Created pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6 Container created 58m Normal Created pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24 Container created 58m Normal Started pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24 Container started 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-de35cd-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-de35cd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-42555d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-ba49d0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-ba49d0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-42555d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning ProbeError pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24 Startup probe error: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24 Startup probe failed: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6 Startup probe error: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6 Startup probe failed: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused 58m Normal Pulled pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-ntwpq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-ntwpq Container created 58m Normal Started pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-ntwpq Container started 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-de35cd-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-de35cd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-db3931-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-db3931-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning Unhealthy pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-ntwpq Startup probe failed: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-ntwpq Startup probe error: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused... 58m Normal Created pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb Container created 58m Normal Started pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb Container started 58m Normal Pulled pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Warning Unhealthy pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb Startup probe failed: Get "http://10.133.0.50:15021/healthz/ready": dial tcp 10.133.0.50:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb Startup probe error: Get "http://10.133.0.50:15021/healthz/ready": dial tcp 10.133.0.50:15021: connect: connection refused... 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-6dc69b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-6dc69b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-019efe-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-019efe-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-c18e79-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-c18e79-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Normal SuccessfulCreate replicaset/e2e-worker-w5-db3931-openshift-default-5f47d69f9f Created pod: e2e-worker-w5-db3931-openshift-default-5f47d69f9f-mm2v8 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-db3931-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 56m Normal ScalingReplicaSet deployment/e2e-worker-w5-db3931-openshift-default Scaled up replica set e2e-worker-w5-db3931-openshift-default-5f47d69f9f from 1 to 2 55m Normal Started pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-mm2v8 Container started 55m Normal AddedInterface pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-mm2v8 Add eth0 [10.134.0.53/23] from ovn-kubernetes 55m Normal Pulled pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-mm2v8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-mm2v8 Container created 55m Warning Unhealthy pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-mm2v8 Startup probe failed: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-mm2v8 Startup probe error: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused... 55m Normal ScalingReplicaSet deployment/e2e-worker-w4-6dc69b-openshift-default Scaled up replica set e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 from 1 to 2 55m Normal SuccessfulCreate replicaset/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 Created pod: e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-6dc69b-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 55m Normal Pulled pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7 Container created 55m Normal AddedInterface pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7 Add eth0 [10.133.0.60/23] from ovn-kubernetes 55m Normal Started pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7 Container started 55m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75fd9f5d47 Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-9znhx 55m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-75fd9f5d47 from 7 to 4 55m Normal Killing pod/maas-default-gateway-openshift-default-75fd9f5d47-6rhn6 Stopping container istio-proxy 55m Normal Killing pod/maas-default-gateway-openshift-default-75fd9f5d47-9znhx Stopping container istio-proxy 55m Normal Killing pod/maas-default-gateway-openshift-default-75fd9f5d47-7n2bx Stopping container istio-proxy 55m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75fd9f5d47 Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-7n2bx 55m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75fd9f5d47 Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-6rhn6 55m Warning ProbeError pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7 Startup probe error: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7 Startup probe failed: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-db3931-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 55m Normal SuccessfulCreate replicaset/e2e-worker-w5-db3931-openshift-default-5f47d69f9f Created pod: e2e-worker-w5-db3931-openshift-default-5f47d69f9f-x5xvr 55m Normal ScalingReplicaSet deployment/e2e-worker-w5-db3931-openshift-default Scaled up replica set e2e-worker-w5-db3931-openshift-default-5f47d69f9f from 2 to 3 54m Normal AddedInterface pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-x5xvr Add eth0 [10.132.0.72/23] from ovn-kubernetes 54m Normal Created pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-x5xvr Container created 54m Normal Pulled pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-x5xvr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Started pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-x5xvr Container started 54m Warning ProbeError pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-x5xvr Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-x5xvr Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-6dc69b-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 54m Normal ScalingReplicaSet deployment/e2e-worker-w4-6dc69b-openshift-default Scaled up replica set e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 from 2 to 3 54m Normal SuccessfulCreate replicaset/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 Created pod: e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-22679 54m Normal Started pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-22679 Container started 54m Normal Created pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-22679 Container created 54m Normal Pulled pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-22679 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-22679 Add eth0 [10.132.0.79/23] from ovn-kubernetes 54m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75fd9f5d47 Created pod: maas-default-gateway-openshift-default-75fd9f5d47-4qvql 54m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75fd9f5d47 Created pod: maas-default-gateway-openshift-default-75fd9f5d47-lwjt6 54m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-75fd9f5d47 from 4 to 8 54m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 54m Normal AddedInterface pod/maas-default-gateway-openshift-default-75fd9f5d47-xv4h4 Add eth0 [10.134.0.56/23] from ovn-kubernetes 54m Normal Started pod/maas-default-gateway-openshift-default-75fd9f5d47-4qvql Container started 54m Normal Created pod/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn Container created 54m Normal Created pod/maas-default-gateway-openshift-default-75fd9f5d47-4qvql Container created 54m Normal Started pod/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn Container started 54m Normal Pulled pod/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/maas-default-gateway-openshift-default-75fd9f5d47-lwjt6 Add eth0 [10.134.0.57/23] from ovn-kubernetes 54m Normal Started pod/maas-default-gateway-openshift-default-75fd9f5d47-xv4h4 Container started 54m Normal AddedInterface pod/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn Add eth0 [10.133.0.61/23] from ovn-kubernetes 54m Normal AddedInterface pod/maas-default-gateway-openshift-default-75fd9f5d47-4qvql Add eth0 [10.134.0.55/23] from ovn-kubernetes 54m Normal Created pod/maas-default-gateway-openshift-default-75fd9f5d47-xv4h4 Container created 54m Normal Pulled pod/maas-default-gateway-openshift-default-75fd9f5d47-xv4h4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Started pod/maas-default-gateway-openshift-default-75fd9f5d47-lwjt6 Container started 54m Normal Created pod/maas-default-gateway-openshift-default-75fd9f5d47-lwjt6 Container created 54m Normal Pulled pod/maas-default-gateway-openshift-default-75fd9f5d47-lwjt6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Pulled pod/maas-default-gateway-openshift-default-75fd9f5d47-4qvql Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Warning ProbeError pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-22679 Startup probe error: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-22679 Startup probe failed: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-4qvql Startup probe failed: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused 54m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-4qvql Startup probe error: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused... 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-lwjt6 Startup probe failed: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused 54m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-lwjt6 Startup probe error: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused... 54m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn Startup probe error: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused... 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn Startup probe failed: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused 54m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-xv4h4 Startup probe error: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused... 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-xv4h4 Startup probe failed: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused 54m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-4qvql Startup probe error: Get "http://10.134.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-lwjt6 Startup probe error: Get "http://10.134.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-lwjt6 Startup probe failed: Get "http://10.134.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-4qvql Startup probe failed: Get "http://10.134.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-75fd9f5d47 from 8 to 10 54m Normal AddedInterface pod/maas-default-gateway-openshift-default-75fd9f5d47-56p7g Add eth0 [10.134.0.58/23] from ovn-kubernetes 54m Normal AddedInterface pod/maas-default-gateway-openshift-default-75fd9f5d47-qg548 Add eth0 [10.132.0.82/23] from ovn-kubernetes 54m Normal Created pod/maas-default-gateway-openshift-default-75fd9f5d47-qg548 Container created 54m Normal Started pod/maas-default-gateway-openshift-default-75fd9f5d47-56p7g Container started 54m Normal Pulled pod/maas-default-gateway-openshift-default-75fd9f5d47-qg548 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Started pod/maas-default-gateway-openshift-default-75fd9f5d47-qg548 Container started 54m Normal Pulled pod/maas-default-gateway-openshift-default-75fd9f5d47-56p7g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/maas-default-gateway-openshift-default-75fd9f5d47-56p7g Container created 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-56p7g Startup probe failed: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused 54m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-56p7g Startup probe error: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused... 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-qg548 Startup probe failed: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused 54m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-qg548 Startup probe error: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused... 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-de35cd-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulCreate replicaset/e2e-worker-w2-de35cd-openshift-default-684fbf96fc Created pod: e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7 54m Normal SuccessfulCreate replicaset/e2e-worker-w2-de35cd-openshift-default-684fbf96fc Created pod: e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kj2h7 54m Normal ScalingReplicaSet deployment/e2e-worker-w2-de35cd-openshift-default Scaled up replica set e2e-worker-w2-de35cd-openshift-default-684fbf96fc from 1 to 3 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-db3931-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 54m Normal Started pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7 Container started 54m Normal AddedInterface pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7 Add eth0 [10.133.0.62/23] from ovn-kubernetes 54m Normal AddedInterface pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft Add eth0 [10.133.0.63/23] from ovn-kubernetes 54m Normal Started pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kj2h7 Container started 54m Normal Created pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kj2h7 Container created 54m Normal Pulled pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Pulled pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kj2h7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kj2h7 Add eth0 [10.132.0.85/23] from ovn-kubernetes 54m Normal Created pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7 Container created 54m Normal ScalingReplicaSet deployment/e2e-worker-w5-db3931-openshift-default Scaled up replica set e2e-worker-w5-db3931-openshift-default-5f47d69f9f from 3 to 4 54m Normal SuccessfulCreate replicaset/e2e-worker-w5-db3931-openshift-default-5f47d69f9f Created pod: e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft 54m Normal SuccessfulCreate replicaset/e2e-worker-w1-019efe-openshift-default-5d87cb7859 Created pod: e2e-worker-w1-019efe-openshift-default-5d87cb7859-c6252 54m Normal Pulled pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft Container created 54m Normal Started pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft Container started 54m Normal ScalingReplicaSet deployment/e2e-worker-w1-019efe-openshift-default Scaled up replica set e2e-worker-w1-019efe-openshift-default-5d87cb7859 from 1 to 2 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-019efe-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal Created pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-c6252 Container created 54m Normal Started pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-c6252 Container started 54m Normal AddedInterface pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-c6252 Add eth0 [10.134.0.60/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-c6252 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Warning Unhealthy pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kj2h7 Startup probe failed: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kj2h7 Startup probe error: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused... 54m Warning ProbeError pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft Startup probe error: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused... 54m Warning ProbeError pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-c6252 Startup probe error: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-c6252 Startup probe failed: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused 54m Warning Unhealthy pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7 Startup probe failed: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7 Startup probe error: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft Startup probe failed: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused 53m Normal SuccessfulCreate replicaset/e2e-worker-w0-ba49d0-openshift-default-6b978b7557 Created pod: e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ba49d0-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w0-ba49d0-openshift-default Scaled up replica set e2e-worker-w0-ba49d0-openshift-default-6b978b7557 from 1 to 2 53m Normal Started pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6 Container started 53m Normal Created pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6 Container created 53m Normal AddedInterface pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6 Add eth0 [10.133.0.64/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6 Startup probe error: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6 Startup probe failed: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused 53m Normal AddedInterface pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq Add eth0 [10.133.0.65/23] from ovn-kubernetes 53m Normal ScalingReplicaSet deployment/e2e-worker-w2-de35cd-openshift-default Scaled up replica set e2e-worker-w2-de35cd-openshift-default-684fbf96fc from 3 to 5 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-de35cd-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 53m Normal SuccessfulCreate replicaset/e2e-worker-w2-de35cd-openshift-default-684fbf96fc Created pod: e2e-worker-w2-de35cd-openshift-default-684fbf96fc-njrmp 53m Normal SuccessfulCreate replicaset/e2e-worker-w2-de35cd-openshift-default-684fbf96fc Created pod: e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq 53m Normal AddedInterface pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-njrmp Add eth0 [10.134.0.62/23] from ovn-kubernetes 53m Normal Started pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-njrmp Container started 53m Normal Pulled pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq Container created 53m Normal Created pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-njrmp Container created 53m Normal Pulled pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-njrmp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq Container started 53m Warning ProbeError pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-njrmp Startup probe error: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-njrmp Startup probe failed: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq Startup probe error: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq Startup probe failed: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused 52m Normal SuccessfulCreate replicaset/e2e-worker-w6-c18e79-openshift-default-59544557d7 Created pod: e2e-worker-w6-c18e79-openshift-default-59544557d7-9rhdc 52m Normal ScalingReplicaSet deployment/e2e-worker-w6-c18e79-openshift-default Scaled up replica set e2e-worker-w6-c18e79-openshift-default-59544557d7 from 1 to 2 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-c18e79-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 52m Normal Started pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-9rhdc Container started 52m Normal AddedInterface pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-9rhdc Add eth0 [10.134.0.64/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-9rhdc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-9rhdc Container created 52m Warning ProbeError pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-9rhdc Startup probe error: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-9rhdc Startup probe failed: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-019efe-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 52m Normal SuccessfulCreate replicaset/e2e-worker-w1-019efe-openshift-default-5d87cb7859 Created pod: e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt 52m Normal ScalingReplicaSet deployment/e2e-worker-w1-019efe-openshift-default Scaled up replica set e2e-worker-w1-019efe-openshift-default-5d87cb7859 from 2 to 4 52m Normal SuccessfulCreate replicaset/e2e-worker-w1-019efe-openshift-default-5d87cb7859 Created pod: e2e-worker-w1-019efe-openshift-default-5d87cb7859-vdwrl 52m Normal AddedInterface pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-vdwrl Add eth0 [10.132.0.92/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt Container started 52m Normal Pulled pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-vdwrl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt Container created 52m Normal AddedInterface pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt Add eth0 [10.133.0.67/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Started pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-vdwrl Container started 52m Normal Created pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-vdwrl Container created 52m Warning ProbeError pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt Startup probe error: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-vdwrl Startup probe error: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-vdwrl Startup probe failed: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt Startup probe failed: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused 52m Normal ScalingReplicaSet deployment/e2e-worker-w0-ba49d0-openshift-default Scaled up replica set e2e-worker-w0-ba49d0-openshift-default-6b978b7557 from 2 to 3 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ba49d0-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 52m Normal SuccessfulCreate replicaset/e2e-worker-w0-ba49d0-openshift-default-6b978b7557 Created pod: e2e-worker-w0-ba49d0-openshift-default-6b978b7557-rm6ws 52m Normal Created pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-rm6ws Container created 52m Normal Pulled pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-rm6ws Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-rm6ws Add eth0 [10.132.0.93/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-rm6ws Container started 52m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-rm6ws Startup probe failed: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-rm6ws Startup probe error: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused... 51m Normal AddedInterface pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-d9msj Add eth0 [10.134.0.68/23] from ovn-kubernetes 51m Normal SuccessfulCreate replicaset/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 Created pod: e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-d9msj 51m Normal ScalingReplicaSet deployment/e2e-worker-w4-6dc69b-openshift-default Scaled up replica set e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 from 3 to 4 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-6dc69b-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 51m Normal Started pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-d9msj Container started 51m Normal Created pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-d9msj Container created 51m Normal Pulled pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-d9msj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Warning ProbeError pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-d9msj Startup probe error: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-d9msj Startup probe failed: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused 51m Normal AddedInterface pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-5sfdr Add eth0 [10.134.0.71/23] from ovn-kubernetes 51m Normal SuccessfulCreate replicaset/e2e-worker-w3-42555d-openshift-default-6c8c6d89c Created pod: e2e-worker-w3-42555d-openshift-default-6c8c6d89c-5sfdr 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-42555d-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 51m Normal ScalingReplicaSet deployment/e2e-worker-w3-42555d-openshift-default Scaled up replica set e2e-worker-w3-42555d-openshift-default-6c8c6d89c from 1 to 2 51m Normal Started pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-5sfdr Container started 51m Normal Created pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-5sfdr Container created 51m Normal Pulled pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-5sfdr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Warning Unhealthy pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-5sfdr Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-5sfdr Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused... 50m Normal SuccessfulCreate replicaset/e2e-worker-w5-db3931-openshift-default-5f47d69f9f Created pod: e2e-worker-w5-db3931-openshift-default-5f47d69f9f-rnfds 50m Normal ScalingReplicaSet deployment/e2e-worker-w5-db3931-openshift-default Scaled up replica set e2e-worker-w5-db3931-openshift-default-5f47d69f9f from 4 to 8 50m Normal SuccessfulCreate replicaset/e2e-worker-w5-db3931-openshift-default-5f47d69f9f Created pod: e2e-worker-w5-db3931-openshift-default-5f47d69f9f-tbbpc 50m Normal SuccessfulCreate replicaset/e2e-worker-w5-db3931-openshift-default-5f47d69f9f Created pod: e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv 50m Normal SuccessfulCreate replicaset/e2e-worker-w5-db3931-openshift-default-5f47d69f9f Created pod: e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7shxl 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-db3931-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 50m Normal AddedInterface pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv Add eth0 [10.133.0.71/23] from ovn-kubernetes 50m Normal Started pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv Container started 50m Normal Created pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-tbbpc Container created 50m Normal Pulled pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-tbbpc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ba49d0-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 50m Normal AddedInterface pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-2qxwf Add eth0 [10.132.0.104/23] from ovn-kubernetes 50m Normal AddedInterface pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-zsvr9 Add eth0 [10.134.0.74/23] from ovn-kubernetes 50m Normal AddedInterface pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7shxl Add eth0 [10.132.0.103/23] from ovn-kubernetes 50m Normal Created pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv Container created 50m Normal AddedInterface pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh Add eth0 [10.133.0.72/23] from ovn-kubernetes 50m Normal ScalingReplicaSet deployment/e2e-worker-w0-ba49d0-openshift-default Scaled up replica set e2e-worker-w0-ba49d0-openshift-default-6b978b7557 from 3 to 6 50m Normal Pulled pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-tbbpc Container started 50m Normal SuccessfulCreate replicaset/e2e-worker-w0-ba49d0-openshift-default-6b978b7557 Created pod: e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh 50m Normal SuccessfulCreate replicaset/e2e-worker-w0-ba49d0-openshift-default-6b978b7557 Created pod: e2e-worker-w0-ba49d0-openshift-default-6b978b7557-2qxwf 50m Normal Pulled pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7shxl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-tbbpc Add eth0 [10.134.0.72/23] from ovn-kubernetes 50m Normal AddedInterface pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-rnfds Add eth0 [10.134.0.73/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-rnfds Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-rnfds Container started 50m Normal Created pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-rnfds Container created 50m Normal Created pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7shxl Container created 50m Normal Started pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7shxl Container started 50m Normal SuccessfulCreate replicaset/e2e-worker-w0-ba49d0-openshift-default-6b978b7557 Created pod: e2e-worker-w0-ba49d0-openshift-default-6b978b7557-zsvr9 50m Normal Started pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-2qxwf Container started 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-c18e79-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 50m Normal ScalingReplicaSet deployment/e2e-worker-w6-c18e79-openshift-default Scaled up replica set e2e-worker-w6-c18e79-openshift-default-59544557d7 from 2 to 4 50m Normal SuccessfulCreate replicaset/e2e-worker-w6-c18e79-openshift-default-59544557d7 Created pod: e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7 50m Normal SuccessfulCreate replicaset/e2e-worker-w6-c18e79-openshift-default-59544557d7 Created pod: e2e-worker-w6-c18e79-openshift-default-59544557d7-b9xj5 50m Normal Started pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh Container started 50m Normal Created pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh Container created 50m Normal Pulled pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-zsvr9 Container started 50m Normal Created pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-zsvr9 Container created 50m Normal Created pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-2qxwf Container created 50m Normal Pulled pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-zsvr9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Pulled pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-2qxwf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-b9xj5 Add eth0 [10.132.0.106/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-b9xj5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Pulled pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7 Add eth0 [10.133.0.73/23] from ovn-kubernetes 50m Normal Started pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-b9xj5 Container started 50m Normal Started pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7 Container started 50m Normal Created pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-b9xj5 Container created 50m Normal Created pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7 Container created 50m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-2qxwf Startup probe error: Get "http://10.132.0.104:15021/healthz/ready": dial tcp 10.132.0.104:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-rnfds Startup probe error: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-zsvr9 Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh Startup probe error: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-rnfds Startup probe failed: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-zsvr9 Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-2qxwf Startup probe failed: Get "http://10.132.0.104:15021/healthz/ready": dial tcp 10.132.0.104:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-tbbpc Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh Startup probe failed: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7shxl Startup probe error: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-tbbpc Startup probe failed: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7shxl Startup probe failed: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-zsvr9 Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-2qxwf Startup probe error: Get "http://10.132.0.104:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-b9xj5 Startup probe failed: Get "http://10.132.0.106:15021/healthz/ready": dial tcp 10.132.0.106:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-b9xj5 Startup probe error: Get "http://10.132.0.106:15021/healthz/ready": dial tcp 10.132.0.106:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-zsvr9 Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning Unhealthy pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7 Startup probe failed: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh Startup probe failed: Get "http://10.133.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh Startup probe error: Get "http://10.133.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-2qxwf Startup probe failed: Get "http://10.132.0.104:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7 Startup probe error: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused... 50m Normal SuccessfulCreate replicaset/e2e-worker-w1-019efe-openshift-default-5d87cb7859 Created pod: e2e-worker-w1-019efe-openshift-default-5d87cb7859-bj5xt 50m Normal SuccessfulCreate replicaset/e2e-worker-w1-019efe-openshift-default-5d87cb7859 Created pod: e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527 50m Normal ScalingReplicaSet deployment/e2e-worker-w1-019efe-openshift-default Scaled up replica set e2e-worker-w1-019efe-openshift-default-5d87cb7859 from 4 to 8 50m Normal SuccessfulCreate replicaset/e2e-worker-w1-019efe-openshift-default-5d87cb7859 Created pod: e2e-worker-w1-019efe-openshift-default-5d87cb7859-x75hl 50m Normal SuccessfulCreate replicaset/e2e-worker-w1-019efe-openshift-default-5d87cb7859 Created pod: e2e-worker-w1-019efe-openshift-default-5d87cb7859-6dk9z 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-019efe-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 50m Normal Pulled pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-bj5xt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-bj5xt Container created 50m Normal Created pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-6dk9z Container created 50m Normal Started pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-6dk9z Container started 50m Normal Started pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527 Container started 50m Normal AddedInterface pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-6dk9z Add eth0 [10.134.0.75/23] from ovn-kubernetes 50m Normal Started pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-bj5xt Container started 50m Normal AddedInterface pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-bj5xt Add eth0 [10.134.0.76/23] from ovn-kubernetes 50m Normal Started pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-x75hl Container started 50m Normal Created pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-x75hl Container created 50m Normal Pulled pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-x75hl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-x75hl Add eth0 [10.132.0.110/23] from ovn-kubernetes 50m Normal AddedInterface pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527 Add eth0 [10.133.0.75/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Pulled pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-6dk9z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527 Container created 50m Warning Unhealthy pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-x75hl Startup probe failed: Get "http://10.132.0.110:15021/healthz/ready": dial tcp 10.132.0.110:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-bj5xt Startup probe failed: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527 Startup probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-6dk9z Startup probe error: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-6dk9z Startup probe failed: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527 Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-x75hl Startup probe error: Get "http://10.132.0.110:15021/healthz/ready": dial tcp 10.132.0.110:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-bj5xt Startup probe error: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused... 49m Normal ScalingReplicaSet deployment/e2e-worker-w1-019efe-openshift-default Scaled up replica set e2e-worker-w1-019efe-openshift-default-5d87cb7859 from 8 to 9 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-019efe-openshift-default New size: 9; reason: All metrics below target 49m Normal SuccessfulCreate replicaset/e2e-worker-w1-019efe-openshift-default-5d87cb7859 Created pod: e2e-worker-w1-019efe-openshift-default-5d87cb7859-dzdcx 49m Normal Pulled pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-dzdcx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-dzdcx Container created 49m Normal Started pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-dzdcx Container started 49m Normal AddedInterface pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-dzdcx Add eth0 [10.132.0.114/23] from ovn-kubernetes 49m Warning ProbeError pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-dzdcx Startup probe error: Get "http://10.132.0.114:15021/healthz/ready": dial tcp 10.132.0.114:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-dzdcx Startup probe failed: Get "http://10.132.0.114:15021/healthz/ready": dial tcp 10.132.0.114:15021: connect: connection refused 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-de35cd-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-ba49d0-openshift-default-6b978b7557 Created pod: e2e-worker-w0-ba49d0-openshift-default-6b978b7557-gn4z7 49m Normal Created pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw Container created 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-de35cd-openshift-default-684fbf96fc Created pod: e2e-worker-w2-de35cd-openshift-default-684fbf96fc-bnsmj 49m Normal ScalingReplicaSet deployment/e2e-worker-w2-de35cd-openshift-default Scaled up replica set e2e-worker-w2-de35cd-openshift-default-684fbf96fc from 5 to 10 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-de35cd-openshift-default-684fbf96fc (combined from similar events): Created pod: e2e-worker-w2-de35cd-openshift-default-684fbf96fc-rvvqb 49m Normal AddedInterface pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-rvvqb Add eth0 [10.132.0.119/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-rvvqb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal ScalingReplicaSet deployment/e2e-worker-w0-ba49d0-openshift-default Scaled up replica set e2e-worker-w0-ba49d0-openshift-default-6b978b7557 from 6 to 10 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-de35cd-openshift-default-684fbf96fc Created pod: e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-de35cd-openshift-default-684fbf96fc Created pod: e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kpchw 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-de35cd-openshift-default-684fbf96fc Created pod: e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-ba49d0-openshift-default-6b978b7557 Created pod: e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f 49m Normal Pulled pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Pulled pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kpchw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw Add eth0 [10.133.0.77/23] from ovn-kubernetes 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ba49d0-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 49m Normal AddedInterface pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kpchw Add eth0 [10.132.0.118/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-ba49d0-openshift-default-6b978b7557 Created pod: e2e-worker-w0-ba49d0-openshift-default-6b978b7557-k62kg 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-ba49d0-openshift-default-6b978b7557 (combined from similar events): Created pod: e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4 49m Normal Started pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f Container started 49m Normal Started pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-bnsmj Container started 49m Normal Pulled pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-gn4z7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-gn4z7 Add eth0 [10.134.0.78/23] from ovn-kubernetes 49m Normal Created pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f Container created 49m Normal AddedInterface pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7 Add eth0 [10.133.0.76/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7 Container created 49m Normal Started pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7 Container started 49m Normal Pulled pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-k62kg Add eth0 [10.132.0.120/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw Container started 49m Normal AddedInterface pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-bnsmj Add eth0 [10.134.0.77/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-bnsmj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-rvvqb Container started 49m Normal Created pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-rvvqb Container created 49m Normal Pulled pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-k62kg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-k62kg Container created 49m Normal AddedInterface pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f Add eth0 [10.133.0.78/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kpchw Container started 49m Normal Created pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kpchw Container created 49m Normal AddedInterface pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4 Add eth0 [10.133.0.79/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-k62kg Container started 49m Normal Created pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-bnsmj Container created 49m Normal SuccessfulCreate replicaset/e2e-worker-w6-c18e79-openshift-default-59544557d7 Created pod: e2e-worker-w6-c18e79-openshift-default-59544557d7-lrjs8 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-c18e79-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 49m Normal ScalingReplicaSet deployment/e2e-worker-w6-c18e79-openshift-default Scaled up replica set e2e-worker-w6-c18e79-openshift-default-59544557d7 from 4 to 8 49m Normal Killing pod/maas-default-gateway-openshift-default-75fd9f5d47-4qvql Stopping container istio-proxy 49m Normal Created pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-gn4z7 Container created 49m Normal Killing pod/maas-default-gateway-openshift-default-75fd9f5d47-lwjt6 Stopping container istio-proxy 49m Normal SuccessfulCreate replicaset/e2e-worker-w6-c18e79-openshift-default-59544557d7 Created pod: e2e-worker-w6-c18e79-openshift-default-59544557d7-6tjz5 49m Normal SuccessfulCreate replicaset/e2e-worker-w6-c18e79-openshift-default-59544557d7 Created pod: e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq 49m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: All metrics below target 49m Normal Created pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4 Container created 49m Normal Started pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4 Container started 49m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-75fd9f5d47 from 10 to 8 49m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75fd9f5d47 Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-lwjt6 49m Normal SuccessfulCreate replicaset/e2e-worker-w6-c18e79-openshift-default-59544557d7 Created pod: e2e-worker-w6-c18e79-openshift-default-59544557d7-cs984 49m Normal Started pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-gn4z7 Container started 49m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75fd9f5d47 Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-4qvql 49m Normal AddedInterface pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-cs984 Add eth0 [10.134.0.79/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-cs984 Container started 49m Normal Started pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lrjs8 Container started 49m Normal AddedInterface pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lrjs8 Add eth0 [10.134.0.80/23] from ovn-kubernetes 49m Normal AddedInterface pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-6tjz5 Add eth0 [10.132.0.122/23] from ovn-kubernetes 49m Normal Created pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lrjs8 Container created 49m Normal Pulled pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-cs984 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-cs984 Container created 49m Normal Pulled pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lrjs8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Pulled pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-6tjz5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq Add eth0 [10.133.0.80/23] from ovn-kubernetes 49m Normal Created pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-6tjz5 Container created 49m Normal Started pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-6tjz5 Container started 49m Normal Pulled pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq Container started 49m Normal Created pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq Container created 49m Warning ProbeError pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-rvvqb Startup probe error: Get "http://10.132.0.119:15021/healthz/ready": dial tcp 10.132.0.119:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-rvvqb Startup probe failed: Get "http://10.132.0.119:15021/healthz/ready": dial tcp 10.132.0.119:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kpchw Startup probe failed: Get "http://10.132.0.118:15021/healthz/ready": dial tcp 10.132.0.118:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kpchw Startup probe error: Get "http://10.132.0.118:15021/healthz/ready": dial tcp 10.132.0.118:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-bnsmj Startup probe error: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7 Startup probe failed: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7 Startup probe error: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-k62kg Startup probe failed: Get "http://10.132.0.120:15021/healthz/ready": dial tcp 10.132.0.120:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-k62kg Startup probe error: Get "http://10.132.0.120:15021/healthz/ready": dial tcp 10.132.0.120:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw Startup probe error: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw Startup probe failed: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-bnsmj Startup probe failed: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lrjs8 Startup probe error: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4 Startup probe failed: Get "http://10.133.0.79:15021/healthz/ready": dial tcp 10.133.0.79:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f Startup probe error: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-cs984 Startup probe error: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-cs984 Startup probe failed: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f Startup probe failed: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-gn4z7 Startup probe error: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-gn4z7 Startup probe failed: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lrjs8 Startup probe failed: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4 Startup probe error: Get "http://10.133.0.79:15021/healthz/ready": dial tcp 10.133.0.79:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-6tjz5 Startup probe failed: Get "http://10.132.0.122:15021/healthz/ready": dial tcp 10.132.0.122:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-6tjz5 Startup probe error: Get "http://10.132.0.122:15021/healthz/ready": dial tcp 10.132.0.122:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq Startup probe failed: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq Startup probe error: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused... 49m Normal ScalingReplicaSet deployment/e2e-worker-w6-c18e79-openshift-default Scaled up replica set e2e-worker-w6-c18e79-openshift-default-59544557d7 from 8 to 9 49m Normal SuccessfulCreate replicaset/e2e-worker-w6-c18e79-openshift-default-59544557d7 Created pod: e2e-worker-w6-c18e79-openshift-default-59544557d7-kzwzk 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-c18e79-openshift-default New size: 9; reason: All metrics below target 49m Normal AddedInterface pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-kzwzk Add eth0 [10.132.0.126/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-kzwzk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-kzwzk Container created 49m Normal Started pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-kzwzk Container started 49m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-rm9wv 49m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 49m Warning Unhealthy pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-kzwzk Startup probe failed: Get "http://10.132.0.126:15021/healthz/ready": dial tcp 10.132.0.126:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-kzwzk Startup probe error: Get "http://10.132.0.126:15021/healthz/ready": dial tcp 10.132.0.126:15021: connect: connection refused... 49m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 49m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-rm9wv Add eth0 [10.134.0.81/23] from ovn-kubernetes 49m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-rm9wv Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 49m Normal ScalingReplicaSet deployment/e2e-worker-w3-42555d-openshift-default Scaled up replica set e2e-worker-w3-42555d-openshift-default-6c8c6d89c from 2 to 4 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-42555d-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 49m Normal AddedInterface pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h Add eth0 [10.133.0.81/23] from ovn-kubernetes 49m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-rm9wv Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.845s (3.845s including waiting). Image size: 185476477 bytes. 49m Normal SuccessfulCreate replicaset/e2e-worker-w3-42555d-openshift-default-6c8c6d89c Created pod: e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h 49m Normal SuccessfulCreate replicaset/e2e-worker-w3-42555d-openshift-default-6c8c6d89c Created pod: e2e-worker-w3-42555d-openshift-default-6c8c6d89c-tcvnj 49m Normal AddedInterface pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-tcvnj Add eth0 [10.132.0.129/23] from ovn-kubernetes 49m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-rm9wv Container created 49m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-rm9wv Container started 49m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75fd9f5d47 Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-x44q9 49m Normal Pulled pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-tcvnj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Pulled pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Killing pod/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn Stopping container istio-proxy 49m Normal Killing pod/maas-default-gateway-openshift-default-75fd9f5d47-qg548 Stopping container istio-proxy 49m Normal Created pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h Container created 49m Normal Killing pod/maas-default-gateway-openshift-default-75fd9f5d47-f9td2 Stopping container istio-proxy 49m Normal Started pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h Container started 49m Normal Killing pod/maas-default-gateway-openshift-default-75fd9f5d47-x44q9 Stopping container istio-proxy 49m Normal Started pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-tcvnj Container started 49m Normal Created pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-tcvnj Container created 49m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75fd9f5d47 Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-mzmxn 49m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: All metrics below target 49m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-75fd9f5d47 from 8 to 4 49m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-rm9wv Readiness probe error: HTTP probe failed with statuscode: 503... 49m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-rm9wv Readiness probe failed: HTTP probe failed with statuscode: 503 49m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75fd9f5d47 Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-f9td2 49m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75fd9f5d47 Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-qg548 49m Warning ProbeError pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-tcvnj Startup probe error: Get "http://10.132.0.129:15021/healthz/ready": dial tcp 10.132.0.129:15021: connect: connection refused... 49m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-f9td2 Readiness probe failed: Get "http://10.133.0.45:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-x44q9 Readiness probe failed: Get "http://10.132.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning Unhealthy pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-tcvnj Startup probe failed: Get "http://10.132.0.129:15021/healthz/ready": dial tcp 10.132.0.129:15021: connect: connection refused 49m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-x44q9 Readiness probe error: Get "http://10.132.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h Startup probe failed: Get "http://10.133.0.81:15021/healthz/ready": dial tcp 10.133.0.81:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h Startup probe error: Get "http://10.133.0.81:15021/healthz/ready": dial tcp 10.133.0.81:15021: connect: connection refused... 49m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-f9td2 Readiness probe error: Get "http://10.133.0.45:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Normal SuccessfulCreate replicaset/e2e-worker-w5-db3931-openshift-default-5f47d69f9f (combined from similar events): Created pod: e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-db3931-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 48m Normal ScalingReplicaSet deployment/e2e-worker-w5-db3931-openshift-default Scaled up replica set e2e-worker-w5-db3931-openshift-default-5f47d69f9f from 8 to 10 48m Normal SuccessfulCreate replicaset/e2e-worker-w5-db3931-openshift-default-5f47d69f9f Created pod: e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7krbm 48m Normal Started pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7krbm Container started 48m Normal SuccessfulCreate replicaset/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 Created pod: e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 48m Normal Pulled pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7krbm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal SuccessfulCreate replicaset/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 Created pod: e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-6hqmk 48m Normal SuccessfulCreate replicaset/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 Created pod: e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-fzgw4 48m Normal Created pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7krbm Container created 48m Normal SuccessfulCreate replicaset/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 Created pod: e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp 48m Normal AddedInterface pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7krbm Add eth0 [10.132.0.135/23] from ovn-kubernetes 48m Normal ScalingReplicaSet deployment/e2e-worker-w4-6dc69b-openshift-default Scaled up replica set e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 from 4 to 8 48m Normal Started pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f Container started 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-6dc69b-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 48m Normal Created pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f Container created 48m Normal Pulled pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f Add eth0 [10.133.0.84/23] from ovn-kubernetes 48m Normal AddedInterface pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp Add eth0 [10.133.0.85/23] from ovn-kubernetes 48m Normal Started pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp Container started 48m Normal Pulled pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 Container created 48m Normal Started pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 Container started 48m Normal Pulled pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-fzgw4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-fzgw4 Container created 48m Normal Started pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-fzgw4 Container started 48m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-75fd9f5d47 from 4 to 3 48m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: All metrics below target 48m Normal AddedInterface pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 Add eth0 [10.133.0.86/23] from ovn-kubernetes 48m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75fd9f5d47 (combined from similar events): Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-xv4h4 48m Normal AddedInterface pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-6hqmk Add eth0 [10.132.0.136/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-6hqmk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-6hqmk Container created 48m Normal Started pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-6hqmk Container started 48m Normal AddedInterface pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-fzgw4 Add eth0 [10.134.0.82/23] from ovn-kubernetes 48m Normal Created pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp Container created 48m Normal Pulled pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Killing pod/maas-default-gateway-openshift-default-75fd9f5d47-xv4h4 Stopping container istio-proxy 48m Normal SuccessfulCreate replicaset/e2e-ait-ad0b4b5e-openshift-default-66df65f85f Created pod: e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb 48m Normal ScalingReplicaSet deployment/e2e-ait-ad0b4b5e-openshift-default Scaled up replica set e2e-ait-ad0b4b5e-openshift-default-66df65f85f from 0 to 1 48m Normal AddedLabel gateway/e2e-ait-ad0b4b5e Added label istio.io/rev=openshift-gateway to gateway e2e-ait-ad0b4b5e 48m Warning Unhealthy pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-9rhdc Readiness probe failed: Get "http://10.134.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Normal AddedInterface pod/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb Add eth0 [10.133.0.87/23] from ovn-kubernetes 48m Warning ProbeError pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7shxl Readiness probe error: Get "http://10.132.0.103:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-9rhdc Readiness probe error: Get "http://10.134.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7shxl Readiness probe failed: Get "http://10.132.0.103:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-tbbpc Readiness probe failed: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-tbbpc Readiness probe error: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-tcvnj Readiness probe error: Get "http://10.132.0.129:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-tcvnj Readiness probe failed: Get "http://10.132.0.129:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-rnfds Readiness probe failed: Get "http://10.134.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-rnfds Readiness probe error: Get "http://10.134.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md Readiness probe error: Get "http://10.133.0.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-6dk9z Readiness probe failed: Get "http://10.134.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-6dk9z Readiness probe error: Get "http://10.134.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h Readiness probe failed: Get "http://10.133.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-bj5xt Readiness probe failed: Get "http://10.134.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-sv6lk Readiness probe failed: Get "http://10.134.0.38:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv Readiness probe failed: Get "http://10.133.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md Readiness probe failed: Get "http://10.133.0.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h Readiness probe error: Get "http://10.133.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-bj5xt Readiness probe error: Get "http://10.134.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-x75hl Readiness probe failed: Get "http://10.132.0.110:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv Readiness probe error: Get "http://10.133.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-zsvr9 Readiness probe error: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-dzdcx Readiness probe error: Get "http://10.132.0.114:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-dzdcx Readiness probe failed: Get "http://10.132.0.114:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb Readiness probe failed: Get "http://10.133.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb Readiness probe error: Get "http://10.133.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-x75hl Readiness probe error: Get "http://10.132.0.110:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-sv6lk Readiness probe error: Get "http://10.134.0.38:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-zsvr9 Readiness probe failed: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-2qxwf Readiness probe error: Get "http://10.132.0.104:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh Readiness probe failed: Get "http://10.133.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh Readiness probe error: Get "http://10.133.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-2qxwf Readiness probe failed: Get "http://10.132.0.104:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-gn4z7 Readiness probe error: Get "http://10.134.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-gn4z7 Readiness probe failed: Get "http://10.134.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f Readiness probe failed: Get "http://10.133.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Normal AddedLabel gateway/e2e-migrate-c82fc808-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-c82fc808-gw 48m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f Readiness probe error: Get "http://10.133.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-ntwpq Readiness probe failed: Get "http://10.134.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Normal SuccessfulCreate replicaset/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597 Created pod: e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6 48m Warning Unhealthy pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7krbm Startup probe failed: Get "http://10.132.0.135:15021/healthz/ready": dial tcp 10.132.0.135:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f Startup probe failed: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp Startup probe failed: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4 Readiness probe failed: Get "http://10.133.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4 Readiness probe error: Get "http://10.133.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Normal ScalingReplicaSet deployment/e2e-migrate-c82fc808-gw-openshift-default Scaled up replica set e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597 from 0 to 1 48m Warning ProbeError pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-ntwpq Readiness probe error: Get "http://10.134.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-fzgw4 Startup probe failed: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-6hqmk Startup probe failed: Get "http://10.132.0.136:15021/healthz/ready": dial tcp 10.132.0.136:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f Startup probe error: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 Startup probe failed: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp Startup probe error: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7krbm Startup probe error: Get "http://10.132.0.135:15021/healthz/ready": dial tcp 10.132.0.135:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-fzgw4 Startup probe error: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6 Readiness probe error: Get "http://10.133.0.49:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 Startup probe error: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused... 48m Normal ScalingReplicaSet deployment/e2e-worker-w4-6dc69b-openshift-default Scaled up replica set e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 from 8 to 9 48m Warning Unhealthy pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6 Readiness probe failed: Get "http://10.133.0.49:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Normal SuccessfulCreate replicaset/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 Created pod: e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rftjh 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-6dc69b-openshift-default New size: 9; reason: All metrics below target 48m Warning ProbeError pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-6hqmk Startup probe error: Get "http://10.132.0.136:15021/healthz/ready": dial tcp 10.132.0.136:15021: connect: connection refused... 48m Normal AddedInterface pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rftjh Add eth0 [10.132.0.139/23] from ovn-kubernetes 48m Normal Started pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rftjh Container started 48m Normal Pulled pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rftjh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rftjh Container created 48m Normal AddedLabel gateway/e2e-derive-5ccdeb27 Added label istio.io/rev=openshift-gateway to gateway e2e-derive-5ccdeb27 48m Normal AddedInterface pod/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6 Add eth0 [10.133.0.88/23] from ovn-kubernetes 48m Normal ScalingReplicaSet deployment/e2e-derive-5ccdeb27-openshift-default Scaled up replica set e2e-derive-5ccdeb27-openshift-default-6b98454f65 from 0 to 1 48m Normal SuccessfulCreate replicaset/e2e-derive-5ccdeb27-openshift-default-6b98454f65 Created pod: e2e-derive-5ccdeb27-openshift-default-6b98454f65-tdjwb 48m Normal AddedInterface pod/e2e-derive-5ccdeb27-openshift-default-6b98454f65-tdjwb Add eth0 [10.134.0.83/23] from ovn-kubernetes 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-ad0b4b5e Scaled up replica set payload-processing-e2e-ait-ad0b4b5e-5b66d56bdf from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-ad0b4b5e-5b66d56bdf Created pod: payload-processing-e2e-ait-ad0b4b5e-5b66d56bdf-l2twq 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-ad0b4b5e-76f7f4795f Error creating: pods "payload-pre-processing-e2e-ait-ad0b4b5e-76f7f4795f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-ad0b4b5e: serviceaccount "payload-processing-e2e-ait-ad0b4b5e" not found 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-ad0b4b5e-76f7f4795f Created pod: payload-pre-processing-e2e-ait-ad0b4b5e-76f7f4795f-nkmnn 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-ad0b4b5e Scaled up replica set payload-pre-processing-e2e-ait-ad0b4b5e-76f7f4795f from 0 to 1 48m Warning FailedCreate replicaset/payload-processing-e2e-ait-ad0b4b5e-5b66d56bdf Error creating: pods "payload-processing-e2e-ait-ad0b4b5e-5b66d56bdf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-ad0b4b5e: serviceaccount "payload-processing-e2e-ait-ad0b4b5e" not found 48m Normal AddedInterface pod/payload-pre-processing-e2e-ait-ad0b4b5e-76f7f4795f-nkmnn Add eth0 [10.134.0.85/23] from ovn-kubernetes 48m Warning Unhealthy pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rftjh Startup probe failed: Get "http://10.132.0.139:15021/healthz/ready": dial tcp 10.132.0.139:15021: connect: connection refused 48m Normal AddedInterface pod/payload-processing-e2e-ait-ad0b4b5e-5b66d56bdf-l2twq Add eth0 [10.134.0.84/23] from ovn-kubernetes 48m Normal Pulled pod/payload-processing-e2e-ait-ad0b4b5e-5b66d56bdf-l2twq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/payload-processing-e2e-ait-ad0b4b5e-5b66d56bdf-l2twq Container created 48m Normal Started pod/payload-processing-e2e-ait-ad0b4b5e-5b66d56bdf-l2twq Container started 48m Normal Pulled pod/payload-pre-processing-e2e-ait-ad0b4b5e-76f7f4795f-nkmnn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/payload-pre-processing-e2e-ait-ad0b4b5e-76f7f4795f-nkmnn Container created 48m Normal Started pod/payload-pre-processing-e2e-ait-ad0b4b5e-76f7f4795f-nkmnn Container started 48m Warning ProbeError pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rftjh Startup probe error: Get "http://10.132.0.139:15021/healthz/ready": dial tcp 10.132.0.139:15021: connect: connection refused... 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-c82fc808-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-c82fc808-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Normal SuccessfulCreate replicaset/e2e-worker-w3-42555d-openshift-default-6c8c6d89c Created pod: e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf 48m Normal SuccessfulCreate replicaset/e2e-worker-w3-42555d-openshift-default-6c8c6d89c Created pod: e2e-worker-w3-42555d-openshift-default-6c8c6d89c-bq2wn 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-42555d-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 48m Normal ScalingReplicaSet deployment/e2e-worker-w1-019efe-openshift-default Scaled up replica set e2e-worker-w1-019efe-openshift-default-5d87cb7859 from 9 to 10 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-019efe-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 48m Normal SuccessfulCreate replicaset/e2e-worker-w1-019efe-openshift-default-5d87cb7859 (combined from similar events): Created pod: e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7 48m Normal SuccessfulCreate replicaset/e2e-worker-w3-42555d-openshift-default-6c8c6d89c Created pod: e2e-worker-w3-42555d-openshift-default-6c8c6d89c-tfln7 48m Normal SuccessfulCreate replicaset/e2e-worker-w3-42555d-openshift-default-6c8c6d89c Created pod: e2e-worker-w3-42555d-openshift-default-6c8c6d89c-zr4vt 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-6dc69b-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 48m Normal ScalingReplicaSet deployment/e2e-worker-w4-6dc69b-openshift-default Scaled up replica set e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 from 9 to 10 48m Normal ScalingReplicaSet deployment/e2e-worker-w3-42555d-openshift-default Scaled up replica set e2e-worker-w3-42555d-openshift-default-6c8c6d89c from 4 to 8 48m Normal SuccessfulCreate replicaset/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 (combined from similar events): Created pod: e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j 48m Normal Pulled pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j Container started 48m Normal Pulled pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j Add eth0 [10.133.0.90/23] from ovn-kubernetes 48m Normal Started pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-bq2wn Container started 48m Normal Created pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-bq2wn Container created 48m Normal Pulled pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-bq2wn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-bq2wn Add eth0 [10.134.0.86/23] from ovn-kubernetes 48m Normal ScalingReplicaSet deployment/e2e-worker-w6-c18e79-openshift-default Scaled up replica set e2e-worker-w6-c18e79-openshift-default-59544557d7 from 9 to 10 48m Normal AddedInterface pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-zr4vt Add eth0 [10.132.0.142/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-zr4vt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-zr4vt Container created 48m Normal Started pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-zr4vt Container started 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-c18e79-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-c82fc808 Scaled up replica set payload-processing-e2e-migrate-c82fc808-559f785f49 from 0 to 1 48m Normal Created pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf Container created 48m Normal Started pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf Container started 48m Normal SuccessfulCreate replicaset/e2e-worker-w6-c18e79-openshift-default-59544557d7 (combined from similar events): Created pod: e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b 48m Normal AddedInterface pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7 Add eth0 [10.133.0.91/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7 Container created 48m Normal Started pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7 Container started 48m Normal Created pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j Container created 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-c82fc808 Scaled up replica set payload-pre-processing-e2e-migrate-c82fc808-759c665f7f from 0 to 1 48m Normal AddedInterface pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-tfln7 Add eth0 [10.132.0.143/23] from ovn-kubernetes 48m Normal AddedInterface pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf Add eth0 [10.133.0.89/23] from ovn-kubernetes 48m Normal Started pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-tfln7 Container started 48m Normal Created pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-tfln7 Container created 48m Normal Pulled pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-tfln7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-c82fc808-759c665f7f Created pod: payload-pre-processing-e2e-migrate-c82fc808-759c665f7f-79x9p 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-c82fc808-559f785f49 Created pod: payload-processing-e2e-migrate-c82fc808-559f785f49-gcs86 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-c82fc808-759c665f7f Error creating: pods "payload-pre-processing-e2e-migrate-c82fc808-759c665f7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-c82fc808: serviceaccount "payload-processing-e2e-migrate-c82fc808" not found 48m Warning FailedCreate replicaset/payload-processing-e2e-migrate-c82fc808-559f785f49 Error creating: pods "payload-processing-e2e-migrate-c82fc808-559f785f49-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-c82fc808: serviceaccount "payload-processing-e2e-migrate-c82fc808" not found 48m Normal Started pod/payload-processing-e2e-migrate-c82fc808-559f785f49-gcs86 Container started 48m Normal Created pod/payload-processing-e2e-migrate-c82fc808-559f785f49-gcs86 Container created 48m Normal AddedInterface pod/payload-processing-e2e-migrate-c82fc808-559f785f49-gcs86 Add eth0 [10.134.0.88/23] from ovn-kubernetes 48m Normal Started pod/payload-pre-processing-e2e-migrate-c82fc808-759c665f7f-79x9p Container started 48m Normal Created pod/payload-pre-processing-e2e-migrate-c82fc808-759c665f7f-79x9p Container created 48m Normal Pulled pod/payload-processing-e2e-migrate-c82fc808-559f785f49-gcs86 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-c82fc808-759c665f7f-79x9p Add eth0 [10.134.0.87/23] from ovn-kubernetes 48m Normal Pulled pod/payload-pre-processing-e2e-migrate-c82fc808-759c665f7f-79x9p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b Container created 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-5ccdeb27 Scaled up replica set payload-processing-e2e-derive-5ccdeb27-86b6749bc7 from 0 to 1 48m Normal AddedInterface pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b Add eth0 [10.133.0.92/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Warning FailedCreate replicaset/payload-processing-e2e-derive-5ccdeb27-86b6749bc7 Error creating: pods "payload-processing-e2e-derive-5ccdeb27-86b6749bc7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-5ccdeb27: serviceaccount "payload-processing-e2e-derive-5ccdeb27" not found 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-5ccdeb27-86b6749bc7 Created pod: payload-processing-e2e-derive-5ccdeb27-86b6749bc7-wxdps 48m Normal Started pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b Container started 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-5ccdeb27 Scaled up replica set payload-pre-processing-e2e-derive-5ccdeb27-7b96689bd7 from 0 to 1 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-5ccdeb27-7b96689bd7 Error creating: pods "payload-pre-processing-e2e-derive-5ccdeb27-7b96689bd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-5ccdeb27: serviceaccount "payload-processing-e2e-derive-5ccdeb27" not found 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-5ccdeb27-7b96689bd7 Created pod: payload-pre-processing-e2e-derive-5ccdeb27-7b96689bd7-n76hv 48m Normal AddedInterface pod/payload-processing-e2e-derive-5ccdeb27-86b6749bc7-wxdps Add eth0 [10.134.0.89/23] from ovn-kubernetes 48m Normal Created pod/payload-processing-e2e-derive-5ccdeb27-86b6749bc7-wxdps Container created 48m Normal Pulled pod/payload-processing-e2e-derive-5ccdeb27-86b6749bc7-wxdps Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Started pod/payload-processing-e2e-derive-5ccdeb27-86b6749bc7-wxdps Container started 48m Normal AddedInterface pod/payload-pre-processing-e2e-derive-5ccdeb27-7b96689bd7-n76hv Add eth0 [10.132.0.145/23] from ovn-kubernetes 48m Normal Pulled pod/payload-pre-processing-e2e-derive-5ccdeb27-7b96689bd7-n76hv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/payload-pre-processing-e2e-derive-5ccdeb27-7b96689bd7-n76hv Container created 48m Normal Started pod/payload-pre-processing-e2e-derive-5ccdeb27-7b96689bd7-n76hv Container started 48m Warning ProbeError pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-bq2wn Startup probe error: Get "http://10.134.0.86:15021/healthz/ready": dial tcp 10.134.0.86:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-tfln7 Startup probe failed: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-zr4vt Startup probe failed: Get "http://10.132.0.142:15021/healthz/ready": dial tcp 10.132.0.142:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-zr4vt Startup probe error: Get "http://10.132.0.142:15021/healthz/ready": dial tcp 10.132.0.142:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-bq2wn Startup probe failed: Get "http://10.134.0.86:15021/healthz/ready": dial tcp 10.134.0.86:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-tfln7 Startup probe error: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf Startup probe failed: Get "http://10.133.0.89:15021/healthz/ready": dial tcp 10.133.0.89:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7 Startup probe error: Get "http://10.133.0.91:15021/healthz/ready": dial tcp 10.133.0.91:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j Startup probe failed: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j Startup probe error: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7 Startup probe failed: Get "http://10.133.0.91:15021/healthz/ready": dial tcp 10.133.0.91:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf Startup probe error: Get "http://10.133.0.89:15021/healthz/ready": dial tcp 10.133.0.89:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b Startup probe failed: Get "http://10.133.0.92:15021/healthz/ready": dial tcp 10.133.0.92:15021: connect: connection refused 48m Normal ScalingReplicaSet deployment/e2e-worker-w3-42555d-openshift-default Scaled up replica set e2e-worker-w3-42555d-openshift-default-6c8c6d89c from 8 to 9 48m Normal SuccessfulCreate replicaset/e2e-worker-w3-42555d-openshift-default-6c8c6d89c Created pod: e2e-worker-w3-42555d-openshift-default-6c8c6d89c-zntdc 48m Warning ProbeError pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b Startup probe error: Get "http://10.133.0.92:15021/healthz/ready": dial tcp 10.133.0.92:15021: connect: connection refused... 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-42555d-openshift-default New size: 9; reason: 48m Warning Unhealthy pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b Startup probe failed: Get "http://10.133.0.92:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b Startup probe error: Get "http://10.133.0.92:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-mm2v8 Readiness probe failed: Get "http://10.134.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-mm2v8 Readiness probe error: Get "http://10.134.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Normal Created pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-zntdc Container created 48m Normal Started pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-zntdc Container started 48m Warning Unhealthy pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft Readiness probe failed: Get "http://10.133.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft Readiness probe error: Get "http://10.133.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Normal Pulled pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-zntdc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-ad0b4b5e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-ad0b4b5e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Normal AddedInterface pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-zntdc Add eth0 [10.134.0.90/23] from ovn-kubernetes 48m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6 Readiness probe error: Get "http://10.133.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6 Readiness probe failed: Get "http://10.133.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-zntdc Startup probe failed: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-zntdc Startup probe error: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused... 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-5ccdeb27-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-5ccdeb27-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Normal SuccessfulCreate replicaset/e2e-worker-w3-42555d-openshift-default-6c8c6d89c (combined from similar events): Created pod: e2e-worker-w3-42555d-openshift-default-6c8c6d89c-njj69 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-42555d-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 47m Normal ScalingReplicaSet deployment/e2e-worker-w3-42555d-openshift-default Scaled up replica set e2e-worker-w3-42555d-openshift-default-6c8c6d89c from 9 to 10 47m Normal Started pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-njj69 Container started 47m Normal Created pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-njj69 Container created 47m Normal Pulled pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-njj69 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-njj69 Add eth0 [10.132.0.149/23] from ovn-kubernetes 47m Warning Unhealthy pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-njj69 Startup probe failed: Get "http://10.132.0.149:15021/healthz/ready": dial tcp 10.132.0.149:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-njj69 Startup probe error: Get "http://10.132.0.149:15021/healthz/ready": dial tcp 10.132.0.149:15021: connect: connection refused... 47m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 47m Normal Started pod/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6 Container started 47m Normal Pulled pod/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/maas-default-gateway-openshift-default-75fd9f5d47-lcdkf Add eth0 [10.132.0.150/23] from ovn-kubernetes 47m Normal Created pod/maas-default-gateway-openshift-default-75fd9f5d47-6ql9n Container created 47m Normal Pulled pod/maas-default-gateway-openshift-default-75fd9f5d47-6ql9n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Pulled pod/maas-default-gateway-openshift-default-75fd9f5d47-lcdkf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/maas-default-gateway-openshift-default-75fd9f5d47-6ql9n Add eth0 [10.134.0.92/23] from ovn-kubernetes 47m Normal Created pod/maas-default-gateway-openshift-default-75fd9f5d47-lcdkf Container created 47m Normal Started pod/maas-default-gateway-openshift-default-75fd9f5d47-lcdkf Container started 47m Normal Created pod/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6 Container created 47m Normal AddedInterface pod/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6 Add eth0 [10.133.0.95/23] from ovn-kubernetes 47m Normal Started pod/maas-default-gateway-openshift-default-75fd9f5d47-6ql9n Container started 47m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-lcdkf Startup probe failed: Get "http://10.132.0.150:15021/healthz/ready": dial tcp 10.132.0.150:15021: connect: connection refused 47m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-lcdkf Startup probe error: Get "http://10.132.0.150:15021/healthz/ready": dial tcp 10.132.0.150:15021: connect: connection refused... 47m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-6ql9n Startup probe error: Get "http://10.134.0.92:15021/healthz/ready": dial tcp 10.134.0.92:15021: connect: connection refused... 47m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-6ql9n Startup probe failed: Get "http://10.134.0.92:15021/healthz/ready": dial tcp 10.134.0.92:15021: connect: connection refused 47m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6 Startup probe error: Get "http://10.133.0.95:15021/healthz/ready": dial tcp 10.133.0.95:15021: connect: connection refused... 47m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6 Startup probe failed: Get "http://10.133.0.95:15021/healthz/ready": dial tcp 10.133.0.95:15021: connect: connection refused 46m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75fd9f5d47 (combined from similar events): Created pod: maas-default-gateway-openshift-default-75fd9f5d47-5l5bb 46m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default (combined from similar events): Scaled up replica set maas-default-gateway-openshift-default-75fd9f5d47 from 6 to 10 46m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: 46m Normal AddedInterface pod/maas-default-gateway-openshift-default-75fd9f5d47-fxb56 Add eth0 [10.133.0.96/23] from ovn-kubernetes 46m Normal Started pod/maas-default-gateway-openshift-default-75fd9f5d47-5l5bb Container started 46m Normal AddedInterface pod/maas-default-gateway-openshift-default-75fd9f5d47-5l5bb Add eth0 [10.134.0.93/23] from ovn-kubernetes 46m Normal Created pod/maas-default-gateway-openshift-default-75fd9f5d47-6qcng Container created 46m Normal Started pod/maas-default-gateway-openshift-default-75fd9f5d47-6qcng Container started 46m Normal Pulled pod/maas-default-gateway-openshift-default-75fd9f5d47-5l5bb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/maas-default-gateway-openshift-default-75fd9f5d47-5l5bb Container created 46m Normal Started pod/maas-default-gateway-openshift-default-75fd9f5d47-zzbkt Container started 46m Normal Started pod/maas-default-gateway-openshift-default-75fd9f5d47-fxb56 Container started 46m Normal Created pod/maas-default-gateway-openshift-default-75fd9f5d47-fxb56 Container created 46m Normal Pulled pod/maas-default-gateway-openshift-default-75fd9f5d47-fxb56 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/maas-default-gateway-openshift-default-75fd9f5d47-zzbkt Add eth0 [10.132.0.151/23] from ovn-kubernetes 46m Normal Pulled pod/maas-default-gateway-openshift-default-75fd9f5d47-zzbkt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Pulled pod/maas-default-gateway-openshift-default-75fd9f5d47-6qcng Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/maas-default-gateway-openshift-default-75fd9f5d47-zzbkt Container created 46m Normal AddedInterface pod/maas-default-gateway-openshift-default-75fd9f5d47-6qcng Add eth0 [10.132.0.152/23] from ovn-kubernetes 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-5l5bb Startup probe failed: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-6qcng Startup probe failed: Get "http://10.132.0.152:15021/healthz/ready": dial tcp 10.132.0.152:15021: connect: connection refused 46m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-5l5bb Startup probe error: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused... 46m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-6qcng Startup probe error: Get "http://10.132.0.152:15021/healthz/ready": dial tcp 10.132.0.152:15021: connect: connection refused... 46m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-zzbkt Startup probe error: Get "http://10.132.0.151:15021/healthz/ready": dial tcp 10.132.0.151:15021: connect: connection refused... 46m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-fxb56 Startup probe error: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused... 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-fxb56 Startup probe failed: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-zzbkt Startup probe failed: Get "http://10.132.0.151:15021/healthz/ready": dial tcp 10.132.0.151:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6 Readiness probe failed: Get "http://10.133.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6 Readiness probe error: Get "http://10.133.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-derive-5ccdeb27-openshift-default-6b98454f65-tdjwb Readiness probe failed: Get "http://10.134.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-derive-5ccdeb27-openshift-default-6b98454f65-tdjwb Readiness probe error: Get "http://10.134.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Normal Created pod/e2e-derive-5ccdeb27-openshift-default-6b98454f65-tdjwb Container created 46m Normal Started pod/e2e-derive-5ccdeb27-openshift-default-6b98454f65-tdjwb Container started 46m Normal Pulled pod/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Pulled pod/e2e-derive-5ccdeb27-openshift-default-6b98454f65-tdjwb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6 Container created 46m Normal Started pod/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6 Container started 46m Normal Started pod/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb Container started 46m Normal Pulled pod/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb Container created 46m Normal Killing pod/payload-processing-e2e-ait-ad0b4b5e-5b66d56bdf-l2twq Stopping container payload-processing 46m Normal Killing pod/payload-pre-processing-e2e-ait-ad0b4b5e-76f7f4795f-nkmnn Stopping container payload-pre-processing 46m Warning ProbeError pod/e2e-derive-5ccdeb27-openshift-default-6b98454f65-tdjwb Startup probe error: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-derive-5ccdeb27-openshift-default-6b98454f65-tdjwb Startup probe failed: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused 46m Warning ProbeError pod/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6 Startup probe error: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused... 46m Normal Killing pod/payload-pre-processing-e2e-migrate-c82fc808-759c665f7f-79x9p Stopping container payload-pre-processing 46m Warning Unhealthy pod/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6 Startup probe failed: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused 46m Normal Killing pod/payload-processing-e2e-migrate-c82fc808-559f785f49-gcs86 Stopping container payload-processing 46m Warning Unhealthy pod/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb Startup probe failed: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused 46m Warning ProbeError pod/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb Startup probe error: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused... 46m Normal Killing pod/payload-processing-e2e-derive-5ccdeb27-86b6749bc7-wxdps Stopping container payload-processing 46m Warning Unhealthy pod/payload-pre-processing-e2e-migrate-c82fc808-759c665f7f-79x9p Readiness probe failed: timeout: failed to connect service "10.134.0.87:50052" within 1s: context deadline exceeded 46m Normal Killing pod/payload-pre-processing-e2e-derive-5ccdeb27-7b96689bd7-n76hv Stopping container payload-pre-processing 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-c82fc808-gw-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-c82fc808-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-5ccdeb27-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-5ccdeb27-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-ad0b4b5e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-ad0b4b5e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal Killing pod/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb Stopping container istio-proxy 45m Normal AddedLabel gateway/e2e-ait-5cbda1cc Added label istio.io/rev=openshift-gateway to gateway e2e-ait-5cbda1cc 45m Normal Killing pod/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6 Stopping container istio-proxy 45m Normal SuccessfulCreate replicaset/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c Created pod: e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2 45m Normal ScalingReplicaSet deployment/e2e-ait-5cbda1cc-openshift-default Scaled up replica set e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c from 0 to 1 45m Normal AddedLabel gateway/e2e-ait-748f2e23 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-748f2e23 45m Normal AddedInterface pod/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2 Add eth0 [10.133.0.97/23] from ovn-kubernetes 45m Normal ScalingReplicaSet deployment/e2e-ait-748f2e23-openshift-default Scaled up replica set e2e-ait-748f2e23-openshift-default-588f97c5cf from 0 to 1 45m Normal SuccessfulCreate replicaset/e2e-ait-748f2e23-openshift-default-588f97c5cf Created pod: e2e-ait-748f2e23-openshift-default-588f97c5cf-gqphg 45m Normal AddedInterface pod/e2e-ait-748f2e23-openshift-default-588f97c5cf-gqphg Add eth0 [10.134.0.94/23] from ovn-kubernetes 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-5cbda1cc Scaled up replica set payload-processing-e2e-ait-5cbda1cc-7dbc678c5f from 0 to 1 45m Warning FailedCreate replicaset/payload-processing-e2e-ait-5cbda1cc-7dbc678c5f Error creating: pods "payload-processing-e2e-ait-5cbda1cc-7dbc678c5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-5cbda1cc: serviceaccount "payload-processing-e2e-ait-5cbda1cc" not found 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-5cbda1cc-6cd64c495 Error creating: pods "payload-pre-processing-e2e-ait-5cbda1cc-6cd64c495-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-5cbda1cc: serviceaccount "payload-processing-e2e-ait-5cbda1cc" not found 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-5cbda1cc Scaled up replica set payload-pre-processing-e2e-ait-5cbda1cc-6cd64c495 from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-5cbda1cc-6cd64c495 Created pod: payload-pre-processing-e2e-ait-5cbda1cc-6cd64c495-85bwn 45m Normal Killing pod/e2e-derive-5ccdeb27-openshift-default-6b98454f65-tdjwb Stopping container istio-proxy 45m Normal AddedInterface pod/payload-pre-processing-e2e-ait-5cbda1cc-6cd64c495-85bwn Add eth0 [10.134.0.95/23] from ovn-kubernetes 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-5cbda1cc-7dbc678c5f Created pod: payload-processing-e2e-ait-5cbda1cc-7dbc678c5f-9mtq7 45m Warning Unhealthy pod/e2e-derive-5ccdeb27-openshift-default-6b98454f65-tdjwb Readiness probe failed: HTTP probe failed with statuscode: 503 45m Normal Started pod/payload-processing-e2e-ait-5cbda1cc-7dbc678c5f-9mtq7 Container started 45m Normal Pulled pod/payload-pre-processing-e2e-ait-5cbda1cc-6cd64c495-85bwn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-pre-processing-e2e-ait-5cbda1cc-6cd64c495-85bwn Container created 45m Normal Started pod/payload-pre-processing-e2e-ait-5cbda1cc-6cd64c495-85bwn Container started 45m Normal Pulled pod/payload-processing-e2e-ait-5cbda1cc-7dbc678c5f-9mtq7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-processing-e2e-ait-5cbda1cc-7dbc678c5f-9mtq7 Container created 45m Normal AddedInterface pod/payload-processing-e2e-ait-5cbda1cc-7dbc678c5f-9mtq7 Add eth0 [10.134.0.96/23] from ovn-kubernetes 45m Warning ProbeError pod/e2e-derive-5ccdeb27-openshift-default-6b98454f65-tdjwb Readiness probe error: HTTP probe failed with statuscode: 503... 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-748f2e23 Scaled up replica set payload-processing-e2e-ait-748f2e23-7597598dd5 from 0 to 1 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-748f2e23 Scaled up replica set payload-pre-processing-e2e-ait-748f2e23-5dd9db5667 from 0 to 1 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-748f2e23-5dd9db5667 Error creating: pods "payload-pre-processing-e2e-ait-748f2e23-5dd9db5667-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-748f2e23: serviceaccount "payload-processing-e2e-ait-748f2e23" not found 45m Warning FailedCreate replicaset/payload-processing-e2e-ait-748f2e23-7597598dd5 Error creating: pods "payload-processing-e2e-ait-748f2e23-7597598dd5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-748f2e23: serviceaccount "payload-processing-e2e-ait-748f2e23" not found 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-748f2e23-7597598dd5 Created pod: payload-processing-e2e-ait-748f2e23-7597598dd5-27kfh 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-748f2e23-5dd9db5667 Created pod: payload-pre-processing-e2e-ait-748f2e23-5dd9db5667-tsk5t 45m Normal Pulled pod/payload-pre-processing-e2e-ait-748f2e23-5dd9db5667-tsk5t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Pulled pod/payload-processing-e2e-ait-748f2e23-7597598dd5-27kfh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-pre-processing-e2e-ait-748f2e23-5dd9db5667-tsk5t Container created 45m Normal AddedInterface pod/payload-processing-e2e-ait-748f2e23-7597598dd5-27kfh Add eth0 [10.134.0.98/23] from ovn-kubernetes 45m Normal Started pod/payload-pre-processing-e2e-ait-748f2e23-5dd9db5667-tsk5t Container started 45m Normal Created pod/payload-processing-e2e-ait-748f2e23-7597598dd5-27kfh Container created 45m Normal AddedInterface pod/payload-pre-processing-e2e-ait-748f2e23-5dd9db5667-tsk5t Add eth0 [10.134.0.97/23] from ovn-kubernetes 45m Normal Started pod/payload-processing-e2e-ait-748f2e23-7597598dd5-27kfh Container started 45m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 4; reason: cpu resource utilization (percentage of request) above target 45m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-hh82k 45m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 4 45m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-9d6dj 45m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-hh82k Add eth0 [10.132.0.157/23] from ovn-kubernetes 45m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-9d6dj Container started 45m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-9d6dj Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-9d6dj Add eth0 [10.133.0.98/23] from ovn-kubernetes 45m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-hh82k Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 45m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-9d6dj Container created 45m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-9d6dj Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-9d6dj Readiness probe failed: HTTP probe failed with statuscode: 503 45m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-hh82k Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.727s (3.727s including waiting). Image size: 185476477 bytes. 45m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-hh82k Container created 45m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-hh82k Container started 45m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-hh82k Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-hh82k Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-748f2e23-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-748f2e23-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal AddedLabel gateway/e2e-mt-be5d503e Added label istio.io/rev=openshift-gateway to gateway e2e-mt-be5d503e 44m Normal SuccessfulCreate replicaset/e2e-mt-be5d503e-openshift-default-f868fdd8f Created pod: e2e-mt-be5d503e-openshift-default-f868fdd8f-8ql8s 44m Normal ScalingReplicaSet deployment/e2e-mt-be5d503e-openshift-default Scaled up replica set e2e-mt-be5d503e-openshift-default-f868fdd8f from 0 to 1 44m Normal AddedInterface pod/e2e-mt-be5d503e-openshift-default-f868fdd8f-8ql8s Add eth0 [10.134.0.101/23] from ovn-kubernetes 44m Normal AddedLabel gateway/e2e-shared-a-w1-895ac8 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w1-895ac8 44m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-895ac8-openshift-default-756985bc89 Created pod: e2e-shared-a-w1-895ac8-openshift-default-756985bc89-9bq5g 44m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-895ac8-openshift-default Scaled up replica set e2e-shared-a-w1-895ac8-openshift-default-756985bc89 from 0 to 1 44m Normal AddedInterface pod/e2e-shared-a-w1-895ac8-openshift-default-756985bc89-9bq5g Add eth0 [10.134.0.102/23] from ovn-kubernetes 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-be5d503e Scaled up replica set payload-processing-e2e-mt-be5d503e-7cddf876cc from 0 to 1 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-be5d503e Scaled up replica set payload-pre-processing-e2e-mt-be5d503e-7dfdd45c7f from 0 to 1 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-be5d503e-7dfdd45c7f Error creating: pods "payload-pre-processing-e2e-mt-be5d503e-7dfdd45c7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-be5d503e: serviceaccount "payload-processing-e2e-mt-be5d503e" not found 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-be5d503e-7dfdd45c7f Created pod: payload-pre-processing-e2e-mt-be5d503e-7dfdd45c7f-pvx7g 44m Warning FailedCreate replicaset/payload-processing-e2e-mt-be5d503e-7cddf876cc Error creating: pods "payload-processing-e2e-mt-be5d503e-7cddf876cc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-be5d503e: serviceaccount "payload-processing-e2e-mt-be5d503e" not found 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-be5d503e-7cddf876cc Created pod: payload-processing-e2e-mt-be5d503e-7cddf876cc-c2skm 44m Normal AddedInterface pod/payload-processing-e2e-mt-be5d503e-7cddf876cc-c2skm Add eth0 [10.134.0.104/23] from ovn-kubernetes 44m Normal AddedInterface pod/payload-pre-processing-e2e-mt-be5d503e-7dfdd45c7f-pvx7g Add eth0 [10.134.0.103/23] from ovn-kubernetes 44m Normal Started pod/payload-processing-e2e-mt-be5d503e-7cddf876cc-c2skm Container started 44m Normal Created pod/payload-processing-e2e-mt-be5d503e-7cddf876cc-c2skm Container created 44m Normal Pulled pod/payload-processing-e2e-mt-be5d503e-7cddf876cc-c2skm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Pulled pod/payload-pre-processing-e2e-mt-be5d503e-7dfdd45c7f-pvx7g Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-pre-processing-e2e-mt-be5d503e-7dfdd45c7f-pvx7g Container created 44m Normal Started pod/payload-pre-processing-e2e-mt-be5d503e-7dfdd45c7f-pvx7g Container started 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w1-895ac8 Scaled up replica set payload-processing-e2e-shared-a-w1-895ac8-69555fcc47 from 0 to 1 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w1-895ac8-65fcccbb9f Error creating: pods "payload-pre-processing-e2e-shared-a-w1-895ac8-65fcccbb9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-895ac8: serviceaccount "payload-processing-e2e-shared-a-w1-895ac8" not found 44m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w1-895ac8-69555fcc47 Error creating: pods "payload-processing-e2e-shared-a-w1-895ac8-69555fcc47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-895ac8: serviceaccount "payload-processing-e2e-shared-a-w1-895ac8" not found 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w1-895ac8 Scaled up replica set payload-pre-processing-e2e-shared-a-w1-895ac8-65fcccbb9f from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w1-895ac8-65fcccbb9f Created pod: payload-pre-processing-e2e-shared-a-w1-895ac8-65fcccbb9f-m5hp2 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w1-895ac8-69555fcc47 Created pod: payload-processing-e2e-shared-a-w1-895ac8-69555fcc47-6n97d 44m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w1-895ac8-65fcccbb9f-m5hp2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7 Created pod: e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm 44m Normal Created pod/payload-processing-e2e-shared-a-w1-895ac8-69555fcc47-6n97d Container created 44m Normal Started pod/payload-processing-e2e-shared-a-w1-895ac8-69555fcc47-6n97d Container started 44m Normal AddedInterface pod/payload-processing-e2e-shared-a-w1-895ac8-69555fcc47-6n97d Add eth0 [10.134.0.105/23] from ovn-kubernetes 44m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w1-895ac8-65fcccbb9f-m5hp2 Add eth0 [10.132.0.158/23] from ovn-kubernetes 44m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-b87e90-openshift-default Scaled up replica set e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7 from 0 to 1 44m Normal AddedLabel gateway/e2e-shared-a-w3-b87e90 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-b87e90 44m Normal Pulled pod/payload-processing-e2e-shared-a-w1-895ac8-69555fcc47-6n97d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Started pod/payload-pre-processing-e2e-shared-a-w1-895ac8-65fcccbb9f-m5hp2 Container started 44m Normal Created pod/payload-pre-processing-e2e-shared-a-w1-895ac8-65fcccbb9f-m5hp2 Container created 44m Normal AddedInterface pod/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm Add eth0 [10.133.0.99/23] from ovn-kubernetes 44m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w3-b87e90-68f96cc4c7 Error creating: pods "payload-processing-e2e-shared-a-w3-b87e90-68f96cc4c7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-b87e90: serviceaccount "payload-processing-e2e-shared-a-w3-b87e90" not found 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w3-b87e90 Scaled up replica set payload-pre-processing-e2e-shared-a-w3-b87e90-76d87d9fb7 from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w3-b87e90-76d87d9fb7 Created pod: payload-pre-processing-e2e-shared-a-w3-b87e90-76d87d9fb7-n5vf8 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w3-b87e90-76d87d9fb7 Error creating: pods "payload-pre-processing-e2e-shared-a-w3-b87e90-76d87d9fb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-b87e90: serviceaccount "payload-processing-e2e-shared-a-w3-b87e90" not found 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w3-b87e90 Scaled up replica set payload-processing-e2e-shared-a-w3-b87e90-68f96cc4c7 from 0 to 1 44m Normal Pulled pod/payload-processing-e2e-shared-a-w3-b87e90-68f96cc4c7-fx9v5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-processing-e2e-shared-a-w3-b87e90-68f96cc4c7-fx9v5 Container created 44m Normal Started pod/payload-processing-e2e-shared-a-w3-b87e90-68f96cc4c7-fx9v5 Container started 44m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w3-b87e90-76d87d9fb7-n5vf8 Add eth0 [10.132.0.159/23] from ovn-kubernetes 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w3-b87e90-68f96cc4c7 Created pod: payload-processing-e2e-shared-a-w3-b87e90-68f96cc4c7-fx9v5 44m Normal AddedInterface pod/payload-processing-e2e-shared-a-w3-b87e90-68f96cc4c7-fx9v5 Add eth0 [10.132.0.160/23] from ovn-kubernetes 44m Normal Started pod/payload-pre-processing-e2e-shared-a-w3-b87e90-76d87d9fb7-n5vf8 Container started 44m Normal Created pod/payload-pre-processing-e2e-shared-a-w3-b87e90-76d87d9fb7-n5vf8 Container created 44m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w3-b87e90-76d87d9fb7-n5vf8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-de35cd-openshift-default New size: 8; reason: All metrics below target 44m Normal ScalingReplicaSet deployment/e2e-worker-w2-de35cd-openshift-default Scaled down replica set e2e-worker-w2-de35cd-openshift-default-684fbf96fc from 10 to 8 44m Normal Killing pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw Stopping container istio-proxy 44m Normal SuccessfulDelete replicaset/e2e-worker-w2-de35cd-openshift-default-684fbf96fc Deleted pod: e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7 44m Normal SuccessfulDelete replicaset/e2e-worker-w2-de35cd-openshift-default-684fbf96fc Deleted pod: e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw 44m Normal Killing pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7 Stopping container istio-proxy 44m Normal ScalingReplicaSet deployment/e2e-worker-w2-de35cd-openshift-default Scaled down replica set e2e-worker-w2-de35cd-openshift-default-684fbf96fc from 8 to 6 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-de35cd-openshift-default New size: 6; reason: All metrics below target 44m Normal Killing pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-bnsmj Stopping container istio-proxy 44m Normal Killing pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-rvvqb Stopping container istio-proxy 44m Normal SuccessfulDelete replicaset/e2e-worker-w2-de35cd-openshift-default-684fbf96fc Deleted pod: e2e-worker-w2-de35cd-openshift-default-684fbf96fc-rvvqb 44m Normal SuccessfulDelete replicaset/e2e-worker-w2-de35cd-openshift-default-684fbf96fc Deleted pod: e2e-worker-w2-de35cd-openshift-default-684fbf96fc-bnsmj 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-895ac8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-895ac8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning ProbeError pod/e2e-ait-748f2e23-openshift-default-588f97c5cf-gqphg Readiness probe error: Get "http://10.134.0.94:15021/healthz/ready": dial tcp 10.134.0.94:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2 Readiness probe failed: Get "http://10.133.0.97:15021/healthz/ready": read tcp 10.133.0.2:43786->10.133.0.97:15021: read: connection reset by peer 44m Warning ProbeError pod/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2 Readiness probe error: Get "http://10.133.0.97:15021/healthz/ready": read tcp 10.133.0.2:43786->10.133.0.97:15021: read: connection reset by peer... 44m Normal Started pod/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2 Container started 44m Normal Created pod/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2 Container created 44m Normal Pulled pod/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Warning Unhealthy pod/e2e-ait-748f2e23-openshift-default-588f97c5cf-gqphg Readiness probe failed: Get "http://10.134.0.94:15021/healthz/ready": dial tcp 10.134.0.94:15021: connect: connection refused 44m Normal Started pod/e2e-ait-748f2e23-openshift-default-588f97c5cf-gqphg Container started 44m Normal Pulled pod/e2e-ait-748f2e23-openshift-default-588f97c5cf-gqphg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/e2e-ait-748f2e23-openshift-default-588f97c5cf-gqphg Container created 43m Warning Unhealthy pod/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2 Startup probe failed: Get "http://10.133.0.97:15021/healthz/ready": dial tcp 10.133.0.97:15021: connect: connection refused 43m Warning ProbeError pod/e2e-ait-748f2e23-openshift-default-588f97c5cf-gqphg Startup probe error: Get "http://10.134.0.94:15021/healthz/ready": dial tcp 10.134.0.94:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-ait-748f2e23-openshift-default-588f97c5cf-gqphg Startup probe failed: Get "http://10.134.0.94:15021/healthz/ready": dial tcp 10.134.0.94:15021: connect: connection refused 43m Warning ProbeError pod/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2 Startup probe error: Get "http://10.133.0.97:15021/healthz/ready": dial tcp 10.133.0.97:15021: connect: connection refused... 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-5cbda1cc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-5cbda1cc-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal Killing pod/payload-processing-e2e-ait-5cbda1cc-7dbc678c5f-9mtq7 Stopping container payload-processing 43m Normal Killing pod/payload-pre-processing-e2e-ait-5cbda1cc-6cd64c495-85bwn Stopping container payload-pre-processing 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-748f2e23-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-748f2e23-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal Killing pod/payload-processing-e2e-ait-748f2e23-7597598dd5-27kfh Stopping container payload-processing 43m Normal Killing pod/payload-pre-processing-e2e-ait-748f2e23-5dd9db5667-tsk5t Stopping container payload-pre-processing 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-5cbda1cc-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-5cbda1cc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning Unhealthy pod/e2e-shared-a-w1-895ac8-openshift-default-756985bc89-9bq5g Readiness probe failed: Get "http://10.134.0.102:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Warning ProbeError pod/e2e-shared-a-w1-895ac8-openshift-default-756985bc89-9bq5g Readiness probe error: Get "http://10.134.0.102:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Normal Created pod/e2e-mt-be5d503e-openshift-default-f868fdd8f-8ql8s Container created 43m Normal Started pod/e2e-mt-be5d503e-openshift-default-f868fdd8f-8ql8s Container started 43m Normal Started pod/e2e-shared-a-w1-895ac8-openshift-default-756985bc89-9bq5g Container started 43m Normal Created pod/e2e-shared-a-w1-895ac8-openshift-default-756985bc89-9bq5g Container created 43m Normal Pulled pod/e2e-shared-a-w1-895ac8-openshift-default-756985bc89-9bq5g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Pulled pod/e2e-mt-be5d503e-openshift-default-f868fdd8f-8ql8s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-be5d503e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-be5d503e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning Unhealthy pod/e2e-mt-be5d503e-openshift-default-f868fdd8f-8ql8s Startup probe failed: Get "http://10.134.0.101:15021/healthz/ready": dial tcp 10.134.0.101:15021: connect: connection refused 43m Warning ProbeError pod/e2e-mt-be5d503e-openshift-default-f868fdd8f-8ql8s Startup probe error: Get "http://10.134.0.101:15021/healthz/ready": dial tcp 10.134.0.101:15021: connect: connection refused... 43m Warning ProbeError pod/e2e-shared-a-w1-895ac8-openshift-default-756985bc89-9bq5g Startup probe error: Get "http://10.134.0.102:15021/healthz/ready": dial tcp 10.134.0.102:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-shared-a-w1-895ac8-openshift-default-756985bc89-9bq5g Startup probe failed: Get "http://10.134.0.102:15021/healthz/ready": dial tcp 10.134.0.102:15021: connect: connection refused 43m Normal AddedLabel gateway/e2e-mt-de9c4ba4 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-de9c4ba4 43m Normal SuccessfulCreate replicaset/e2e-mt-de9c4ba4-openshift-default-854f5f846f Created pod: e2e-mt-de9c4ba4-openshift-default-854f5f846f-fj6qk 43m Normal ScalingReplicaSet deployment/e2e-mt-de9c4ba4-openshift-default Scaled up replica set e2e-mt-de9c4ba4-openshift-default-854f5f846f from 0 to 1 43m Normal AddedInterface pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-fj6qk Add eth0 [10.134.0.107/23] from ovn-kubernetes 43m Normal Killing pod/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2 Stopping container istio-proxy 42m Normal Killing pod/e2e-ait-748f2e23-openshift-default-588f97c5cf-gqphg Stopping container istio-proxy 42m Normal AddedLabel gateway/e2e-shared-a-w5-6b2f61 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w5-6b2f61 42m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-6b2f61-openshift-default-6466884b9f Created pod: e2e-shared-a-w5-6b2f61-openshift-default-6466884b9f-5pxf2 42m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-6b2f61-openshift-default Scaled up replica set e2e-shared-a-w5-6b2f61-openshift-default-6466884b9f from 0 to 1 42m Normal AddedLabel gateway/e2e-shared-a-w2-559973 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-559973 42m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-559973-openshift-default-5974f778dc Created pod: e2e-shared-a-w2-559973-openshift-default-5974f778dc-frvnh 42m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-559973-openshift-default Scaled up replica set e2e-shared-a-w2-559973-openshift-default-5974f778dc from 0 to 1 42m Normal AddedInterface pod/e2e-shared-a-w5-6b2f61-openshift-default-6466884b9f-5pxf2 Add eth0 [10.134.0.108/23] from ovn-kubernetes 42m Normal AddedInterface pod/e2e-shared-a-w2-559973-openshift-default-5974f778dc-frvnh Add eth0 [10.134.0.109/23] from ovn-kubernetes 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-895ac8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-895ac8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-559973-7b9b654d6f Error creating: pods "payload-processing-e2e-shared-a-w2-559973-7b9b654d6f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-559973: serviceaccount "payload-processing-e2e-shared-a-w2-559973" not found 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-559973 Scaled up replica set payload-processing-e2e-shared-a-w2-559973-7b9b654d6f from 0 to 1 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-559973 Scaled up replica set payload-pre-processing-e2e-shared-a-w2-559973-c967685f from 0 to 1 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-559973-c967685f Error creating: pods "payload-pre-processing-e2e-shared-a-w2-559973-c967685f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-559973: serviceaccount "payload-processing-e2e-shared-a-w2-559973" not found 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-559973-c967685f Created pod: payload-pre-processing-e2e-shared-a-w2-559973-c967685f-ls5fw 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-559973-7b9b654d6f Created pod: payload-processing-e2e-shared-a-w2-559973-7b9b654d6f-pptpc 42m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-559973-c967685f-ls5fw Add eth0 [10.134.0.110/23] from ovn-kubernetes 42m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-559973-c967685f-ls5fw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Created pod/payload-processing-e2e-shared-a-w2-559973-7b9b654d6f-pptpc Container created 42m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-559973-c967685f-ls5fw Container started 42m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-559973-7b9b654d6f-pptpc Add eth0 [10.134.0.111/23] from ovn-kubernetes 42m Normal Pulled pod/payload-processing-e2e-shared-a-w2-559973-7b9b654d6f-pptpc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-559973-c967685f-ls5fw Container created 42m Normal Started pod/payload-processing-e2e-shared-a-w2-559973-7b9b654d6f-pptpc Container started 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-be5d503e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-be5d503e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Normal ScalingReplicaSet deployment/e2e-worker-w4-6dc69b-openshift-default Scaled down replica set e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 from 10 to 6 42m Normal Killing pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6 Stopping container istio-proxy 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ba49d0-openshift-default New size: 6; reason: All metrics below target 42m Normal SuccessfulDelete replicaset/e2e-worker-w5-db3931-openshift-default-5f47d69f9f Deleted pod: e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft 42m Normal SuccessfulDelete replicaset/e2e-worker-w1-019efe-openshift-default-5d87cb7859 Deleted pod: e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6 42m Normal ScalingReplicaSet deployment/e2e-worker-w5-db3931-openshift-default Scaled down replica set e2e-worker-w5-db3931-openshift-default-5f47d69f9f from 10 to 6 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-db3931-openshift-default New size: 6; reason: All metrics below target 42m Normal SuccessfulDelete replicaset/e2e-worker-w1-019efe-openshift-default-5d87cb7859 Deleted pod: e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527 42m Normal SuccessfulDelete replicaset/e2e-worker-w5-db3931-openshift-default-5f47d69f9f Deleted pod: e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24 42m Normal SuccessfulDelete replicaset/e2e-worker-w1-019efe-openshift-default-5d87cb7859 Deleted pod: e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7 42m Normal SuccessfulDelete replicaset/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 Deleted pod: e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-019efe-openshift-default New size: 6; reason: All metrics below target 42m Normal ScalingReplicaSet deployment/e2e-worker-w0-ba49d0-openshift-default Scaled down replica set e2e-worker-w0-ba49d0-openshift-default-6b978b7557 from 10 to 6 42m Normal ScalingReplicaSet deployment/e2e-worker-w1-019efe-openshift-default Scaled down replica set e2e-worker-w1-019efe-openshift-default-5d87cb7859 from 10 to 6 42m Normal Killing pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 Deleted pod: e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7 42m Normal SuccessfulDelete replicaset/e2e-worker-w5-db3931-openshift-default-5f47d69f9f Deleted pod: e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv 42m Normal Killing pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w5-db3931-openshift-default-5f47d69f9f Deleted pod: e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f 42m Normal SuccessfulDelete replicaset/e2e-worker-w1-019efe-openshift-default-5d87cb7859 Deleted pod: e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-6dc69b-openshift-default New size: 6; reason: All metrics below target 42m Normal Killing pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w0-ba49d0-openshift-default-6b978b7557 Deleted pod: e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6 42m Normal SuccessfulDelete replicaset/e2e-worker-w0-ba49d0-openshift-default-6b978b7557 Deleted pod: e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4 42m Normal SuccessfulDelete replicaset/e2e-worker-w0-ba49d0-openshift-default-6b978b7557 Deleted pod: e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f 42m Normal Killing pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j Stopping container istio-proxy 42m Normal Killing pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w0-ba49d0-openshift-default-6b978b7557 Deleted pod: e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh 42m Normal SuccessfulDelete replicaset/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 Deleted pod: e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j 42m Normal SuccessfulDelete replicaset/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 Deleted pod: e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp 42m Normal Killing pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6 Stopping container istio-proxy 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w5-6b2f61-798968dcb7 Created pod: payload-pre-processing-e2e-shared-a-w5-6b2f61-798968dcb7-2rkdk 42m Normal Killing pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv Stopping container istio-proxy 42m Normal Killing pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f Stopping container istio-proxy 42m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w5-6b2f61-78b8fcdb9c Error creating: pods "payload-processing-e2e-shared-a-w5-6b2f61-78b8fcdb9c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-6b2f61: serviceaccount "payload-processing-e2e-shared-a-w5-6b2f61" not found 42m Normal Killing pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7 Stopping container istio-proxy 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w5-6b2f61 Scaled up replica set payload-processing-e2e-shared-a-w5-6b2f61-78b8fcdb9c from 0 to 1 42m Normal Killing pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7 Stopping container istio-proxy 42m Normal Killing pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4 Stopping container istio-proxy 42m Normal Killing pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt Stopping container istio-proxy 42m Normal Killing pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft Stopping container istio-proxy 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w5-6b2f61 Scaled up replica set payload-pre-processing-e2e-shared-a-w5-6b2f61-798968dcb7 from 0 to 1 42m Normal Killing pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527 Stopping container istio-proxy 42m Normal Killing pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24 Stopping container istio-proxy 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w5-6b2f61-798968dcb7 Error creating: pods "payload-pre-processing-e2e-shared-a-w5-6b2f61-798968dcb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-6b2f61: serviceaccount "payload-processing-e2e-shared-a-w5-6b2f61" not found 42m Normal AddedInterface pod/payload-processing-e2e-shared-a-w5-6b2f61-78b8fcdb9c-ttp26 Add eth0 [10.134.0.113/23] from ovn-kubernetes 42m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w5-6b2f61-798968dcb7-2rkdk Add eth0 [10.134.0.112/23] from ovn-kubernetes 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w5-6b2f61-78b8fcdb9c Created pod: payload-processing-e2e-shared-a-w5-6b2f61-78b8fcdb9c-ttp26 42m Normal ScalingReplicaSet deployment/e2e-worker-w6-c18e79-openshift-default Scaled down replica set e2e-worker-w6-c18e79-openshift-default-59544557d7 from 10 to 6 42m Normal Started pod/payload-processing-e2e-shared-a-w5-6b2f61-78b8fcdb9c-ttp26 Container started 42m Normal Created pod/payload-processing-e2e-shared-a-w5-6b2f61-78b8fcdb9c-ttp26 Container created 42m Normal Killing pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7 Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w6-c18e79-openshift-default-59544557d7 Deleted pod: e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq 42m Normal Killing pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w6-c18e79-openshift-default-59544557d7 Deleted pod: e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b 42m Normal SuccessfulDelete replicaset/e2e-worker-w6-c18e79-openshift-default-59544557d7 Deleted pod: e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb 42m Normal Started pod/payload-pre-processing-e2e-shared-a-w5-6b2f61-798968dcb7-2rkdk Container started 42m Normal Created pod/payload-pre-processing-e2e-shared-a-w5-6b2f61-798968dcb7-2rkdk Container created 42m Normal Killing pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb Stopping container istio-proxy 42m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w5-6b2f61-798968dcb7-2rkdk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-c18e79-openshift-default New size: 6; reason: All metrics below target 42m Normal SuccessfulDelete replicaset/e2e-worker-w6-c18e79-openshift-default-59544557d7 Deleted pod: e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7 42m Normal Pulled pod/payload-processing-e2e-shared-a-w5-6b2f61-78b8fcdb9c-ttp26 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Killing pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b Stopping container istio-proxy 42m Normal AddedLabel gateway/e2e-shared-b-w1-fd1339 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w1-fd1339 42m Warning ProbeError pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft Readiness probe error: HTTP probe failed with statuscode: 503... 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-de9c4ba4 Scaled up replica set payload-pre-processing-e2e-mt-de9c4ba4-6564f5d4dc from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-de9c4ba4-768fdffc87 Created pod: payload-processing-e2e-mt-de9c4ba4-768fdffc87-hzb6r 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-de9c4ba4-6564f5d4dc Error creating: pods "payload-pre-processing-e2e-mt-de9c4ba4-6564f5d4dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-de9c4ba4: serviceaccount "payload-processing-e2e-mt-de9c4ba4" not found 42m Warning FailedCreate replicaset/payload-processing-e2e-mt-de9c4ba4-768fdffc87 Error creating: pods "payload-processing-e2e-mt-de9c4ba4-768fdffc87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-de9c4ba4: serviceaccount "payload-processing-e2e-mt-de9c4ba4" not found 42m Warning Unhealthy pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft Readiness probe failed: HTTP probe failed with statuscode: 503 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-de9c4ba4 Scaled up replica set payload-processing-e2e-mt-de9c4ba4-768fdffc87 from 0 to 1 42m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-fd1339-openshift-default Scaled up replica set e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c from 0 to 1 42m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c Created pod: e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999 42m Warning ProbeError pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp Readiness probe error: HTTP probe failed with statuscode: 503... 42m Warning ProbeError pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7 Readiness probe error: HTTP probe failed with statuscode: 503... 42m Warning Unhealthy pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7 Readiness probe failed: HTTP probe failed with statuscode: 503 42m Warning ProbeError pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 Readiness probe error: HTTP probe failed with statuscode: 503... 42m Normal AddedInterface pod/payload-pre-processing-e2e-mt-de9c4ba4-6564f5d4dc-s8wg7 Add eth0 [10.132.0.166/23] from ovn-kubernetes 42m Normal Created pod/payload-processing-e2e-mt-de9c4ba4-768fdffc87-hzb6r Container created 42m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6 Readiness probe error: HTTP probe failed with statuscode: 503... 42m Warning ProbeError pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f Readiness probe error: HTTP probe failed with statuscode: 503... 42m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6 Readiness probe failed: HTTP probe failed with statuscode: 503 42m Normal Pulled pod/payload-pre-processing-e2e-mt-de9c4ba4-6564f5d4dc-s8wg7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-de9c4ba4-6564f5d4dc Created pod: payload-pre-processing-e2e-mt-de9c4ba4-6564f5d4dc-s8wg7 42m Normal AddedInterface pod/payload-processing-e2e-mt-de9c4ba4-768fdffc87-hzb6r Add eth0 [10.132.0.165/23] from ovn-kubernetes 42m Normal AddedInterface pod/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999 Add eth0 [10.133.0.102/23] from ovn-kubernetes 42m Normal Started pod/payload-processing-e2e-mt-de9c4ba4-768fdffc87-hzb6r Container started 42m Normal Pulled pod/payload-processing-e2e-mt-de9c4ba4-768fdffc87-hzb6r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Created pod/payload-pre-processing-e2e-mt-de9c4ba4-6564f5d4dc-s8wg7 Container created 42m Normal Started pod/payload-pre-processing-e2e-mt-de9c4ba4-6564f5d4dc-s8wg7 Container started 42m Warning Unhealthy pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt Readiness probe failed: Get "http://10.133.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Warning ProbeError pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt Readiness probe error: Get "http://10.133.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Warning Unhealthy pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb Readiness probe failed: HTTP probe failed with statuscode: 503 42m Warning ProbeError pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb Readiness probe error: HTTP probe failed with statuscode: 503... 42m Warning ProbeError pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7 Readiness probe error: Get "http://10.133.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Warning Unhealthy pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7 Readiness probe failed: Get "http://10.133.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-559973-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-559973-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal Killing pod/payload-pre-processing-e2e-mt-be5d503e-7dfdd45c7f-pvx7g Stopping container payload-pre-processing 42m Normal Killing pod/payload-processing-e2e-mt-be5d503e-7cddf876cc-c2skm Stopping container payload-processing 42m Warning Unhealthy pod/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm Readiness probe failed: Get "http://10.133.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-4c5b4e-openshift-default Scaled up replica set e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5 from 0 to 1 42m Normal AddedLabel gateway/e2e-shared-b-w3-4c5b4e Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-4c5b4e 42m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5 Created pod: e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8 42m Warning ProbeError pod/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm Readiness probe error: Get "http://10.133.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Normal AddedInterface pod/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8 Add eth0 [10.133.0.103/23] from ovn-kubernetes 42m Normal Pulled pod/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm Container created 42m Normal Started pod/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm Container started 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w1-fd1339 Scaled up replica set payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc Created pod: payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w1-fd1339 Scaled up replica set payload-processing-e2e-shared-b-w1-fd1339-7456979db9 from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w1-fd1339-7456979db9 Created pod: payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc 42m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w1-fd1339-7456979db9 Error creating: pods "payload-processing-e2e-shared-b-w1-fd1339-7456979db9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339: serviceaccount "payload-processing-e2e-shared-b-w1-fd1339" not found 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc Error creating: pods "payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339: serviceaccount "payload-processing-e2e-shared-b-w1-fd1339" not found 42m Normal AddedInterface pod/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc Add eth0 [10.133.0.105/23] from ovn-kubernetes 42m Normal Started pod/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr Container started 42m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Created pod/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr Container created 42m Normal Created pod/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc Container created 42m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr Add eth0 [10.133.0.104/23] from ovn-kubernetes 42m Normal Pulled pod/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Started pod/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc Container started 42m Warning Unhealthy pod/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm Startup probe failed: Get "http://10.133.0.99:15021/healthz/ready": dial tcp 10.133.0.99:15021: connect: connection refused 42m Warning ProbeError pod/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm Startup probe error: Get "http://10.133.0.99:15021/healthz/ready": dial tcp 10.133.0.99:15021: connect: connection refused... 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w3-4c5b4e Scaled up replica set payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f from 0 to 1 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f Error creating: pods "payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e: serviceaccount "payload-processing-e2e-shared-b-w3-4c5b4e" not found 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f Created pod: payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6 42m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f Error creating: pods "payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e: serviceaccount "payload-processing-e2e-shared-b-w3-4c5b4e" not found 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w3-4c5b4e Scaled up replica set payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f Created pod: payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt 42m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6 Add eth0 [10.133.0.106/23] from ovn-kubernetes 42m Normal Pulled pod/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt Add eth0 [10.133.0.107/23] from ovn-kubernetes 42m Normal Created pod/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6 Container created 42m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Started pod/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt Container started 42m Normal Started pod/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6 Container started 42m Normal Created pod/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt Container created 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-b87e90-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-b87e90-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-fxb56 Readiness probe error: Get "http://10.133.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-fxb56 Readiness probe failed: Get "http://10.133.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Normal ScalingReplicaSet deployment/e2e-worker-w3-42555d-openshift-default Scaled down replica set e2e-worker-w3-42555d-openshift-default-6c8c6d89c from 10 to 6 41m Normal Killing pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-zr4vt Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-njj69 Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-tcvnj Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w3-42555d-openshift-default-6c8c6d89c Deleted pod: e2e-worker-w3-42555d-openshift-default-6c8c6d89c-tfln7 41m Normal SuccessfulDelete replicaset/e2e-worker-w3-42555d-openshift-default-6c8c6d89c Deleted pod: e2e-worker-w3-42555d-openshift-default-6c8c6d89c-zr4vt 41m Normal SuccessfulDelete replicaset/e2e-worker-w3-42555d-openshift-default-6c8c6d89c Deleted pod: e2e-worker-w3-42555d-openshift-default-6c8c6d89c-tcvnj 41m Normal Killing pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-tfln7 Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w3-42555d-openshift-default-6c8c6d89c Deleted pod: e2e-worker-w3-42555d-openshift-default-6c8c6d89c-njj69 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-42555d-openshift-default New size: 6; reason: All metrics below target 41m Normal Killing pod/e2e-mt-be5d503e-openshift-default-f868fdd8f-8ql8s Stopping container istio-proxy 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-b87e90-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-b87e90-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal AddedLabel gateway/e2e-api-a-eafc0d Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-eafc0d 41m Normal ScalingReplicaSet deployment/e2e-api-a-eafc0d-openshift-default Scaled up replica set e2e-api-a-eafc0d-openshift-default-78865f7d8c from 0 to 1 41m Normal SuccessfulCreate replicaset/e2e-api-a-eafc0d-openshift-default-78865f7d8c Created pod: e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l 41m Normal AddedInterface pod/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l Add eth0 [10.133.0.111/23] from ovn-kubernetes 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-eafc0d Scaled up replica set payload-processing-e2e-api-a-eafc0d-5588fc499c from 0 to 1 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-eafc0d Scaled up replica set payload-pre-processing-e2e-api-a-eafc0d-75789b677f from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-eafc0d-5588fc499c Created pod: payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9 41m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg Add eth0 [10.133.0.112/23] from ovn-kubernetes 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-eafc0d-75789b677f Error creating: pods "payload-pre-processing-e2e-api-a-eafc0d-75789b677f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-eafc0d: serviceaccount "payload-processing-e2e-api-a-eafc0d" not found 41m Warning FailedCreate replicaset/payload-processing-e2e-api-a-eafc0d-5588fc499c Error creating: pods "payload-processing-e2e-api-a-eafc0d-5588fc499c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-eafc0d: serviceaccount "payload-processing-e2e-api-a-eafc0d" not found 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-eafc0d-75789b677f Created pod: payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg 41m Normal Pulled pod/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9 Add eth0 [10.133.0.113/23] from ovn-kubernetes 41m Normal Created pod/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg Container created 41m Normal Created pod/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9 Container created 41m Normal Pulled pod/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Started pod/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9 Container started 41m Normal Started pod/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg Container started 41m Warning ProbeError pod/e2e-shared-a-w5-6b2f61-openshift-default-6466884b9f-5pxf2 Readiness probe error: Get "http://10.134.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Warning Unhealthy pod/e2e-shared-a-w5-6b2f61-openshift-default-6466884b9f-5pxf2 Readiness probe failed: Get "http://10.134.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Warning Unhealthy pod/e2e-shared-a-w2-559973-openshift-default-5974f778dc-frvnh Readiness probe failed: Get "http://10.134.0.109:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Warning ProbeError pod/e2e-shared-a-w2-559973-openshift-default-5974f778dc-frvnh Readiness probe error: Get "http://10.134.0.109:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Normal Pulled pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-fj6qk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Pulled pod/e2e-shared-a-w5-6b2f61-openshift-default-6466884b9f-5pxf2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Pulled pod/e2e-shared-a-w2-559973-openshift-default-5974f778dc-frvnh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-fj6qk Container created 41m Normal Started pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-fj6qk Container started 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-de9c4ba4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal Created pod/e2e-shared-a-w2-559973-openshift-default-5974f778dc-frvnh Container created 41m Normal Started pod/e2e-shared-a-w5-6b2f61-openshift-default-6466884b9f-5pxf2 Container started 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-de9c4ba4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal Created pod/e2e-shared-a-w5-6b2f61-openshift-default-6466884b9f-5pxf2 Container created 41m Normal Started pod/e2e-shared-a-w2-559973-openshift-default-5974f778dc-frvnh Container started 40m Warning ProbeError pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-fj6qk Startup probe error: Get "http://10.134.0.107:15021/healthz/ready": dial tcp 10.134.0.107:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-fj6qk Startup probe failed: Get "http://10.134.0.107:15021/healthz/ready": dial tcp 10.134.0.107:15021: connect: connection refused 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-6b2f61-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-6b2f61-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning ProbeError pod/e2e-shared-a-w5-6b2f61-openshift-default-6466884b9f-5pxf2 Startup probe error: Get "http://10.134.0.108:15021/healthz/ready": dial tcp 10.134.0.108:15021: connect: connection refused... 40m Warning ProbeError pod/e2e-shared-a-w2-559973-openshift-default-5974f778dc-frvnh Startup probe error: Get "http://10.134.0.109:15021/healthz/ready": dial tcp 10.134.0.109:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-shared-a-w5-6b2f61-openshift-default-6466884b9f-5pxf2 Startup probe failed: Get "http://10.134.0.108:15021/healthz/ready": dial tcp 10.134.0.108:15021: connect: connection refused 40m Warning Unhealthy pod/e2e-shared-a-w2-559973-openshift-default-5974f778dc-frvnh Startup probe failed: Get "http://10.134.0.109:15021/healthz/ready": dial tcp 10.134.0.109:15021: connect: connection refused 40m Normal AddedLabel gateway/e2e-api-a-53fd96 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-53fd96 40m Normal ScalingReplicaSet deployment/e2e-api-a-53fd96-openshift-default Scaled up replica set e2e-api-a-53fd96-openshift-default-677ff6848c from 0 to 1 40m Normal SuccessfulCreate replicaset/e2e-api-a-53fd96-openshift-default-677ff6848c Created pod: e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk 40m Normal AddedInterface pod/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk Add eth0 [10.133.0.116/23] from ovn-kubernetes 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-53fd96 Scaled up replica set payload-processing-e2e-api-a-53fd96-c945bcf7 from 0 to 1 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-53fd96 Scaled up replica set payload-pre-processing-e2e-api-a-53fd96-69848f4575 from 0 to 1 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-53fd96-69848f4575 Error creating: pods "payload-pre-processing-e2e-api-a-53fd96-69848f4575-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-53fd96: serviceaccount "payload-processing-e2e-api-a-53fd96" not found 40m Warning FailedCreate replicaset/payload-processing-e2e-api-a-53fd96-c945bcf7 Error creating: pods "payload-processing-e2e-api-a-53fd96-c945bcf7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-53fd96: serviceaccount "payload-processing-e2e-api-a-53fd96" not found 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-53fd96-c945bcf7 Created pod: payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-53fd96-69848f4575 Created pod: payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-de9c4ba4-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-de9c4ba4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284 Add eth0 [10.133.0.117/23] from ovn-kubernetes 40m Normal Started pod/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m Container started 40m Normal Started pod/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284 Container started 40m Normal Pulled pod/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Created pod/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284 Container created 40m Normal AddedInterface pod/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m Add eth0 [10.133.0.118/23] from ovn-kubernetes 40m Normal Pulled pod/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Created pod/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m Container created 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-6b2f61-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-6b2f61-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-559973-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-559973-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal SuccessfulDelete replicaset/e2e-worker-w0-ba49d0-openshift-default-6b978b7557 Deleted pod: e2e-worker-w0-ba49d0-openshift-default-6b978b7557-gn4z7 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ba49d0-openshift-default New size: 5; reason: All metrics below target 40m Normal ScalingReplicaSet deployment/e2e-worker-w0-ba49d0-openshift-default Scaled down replica set e2e-worker-w0-ba49d0-openshift-default-6b978b7557 from 6 to 5 40m Normal ScalingReplicaSet deployment/e2e-worker-w2-de35cd-openshift-default Scaled down replica set e2e-worker-w2-de35cd-openshift-default-684fbf96fc from 6 to 5 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-de35cd-openshift-default New size: 5; reason: All metrics below target 40m Normal Killing pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-gn4z7 Stopping container istio-proxy 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-db3931-openshift-default New size: 5; reason: All metrics below target 40m Normal SuccessfulDelete replicaset/e2e-worker-w5-db3931-openshift-default-5f47d69f9f Deleted pod: e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7krbm 40m Normal Pulled pod/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal ScalingReplicaSet deployment/e2e-worker-w5-db3931-openshift-default Scaled down replica set e2e-worker-w5-db3931-openshift-default-5f47d69f9f from 6 to 5 40m Normal Created pod/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999 Container created 40m Normal Started pod/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999 Container started 40m Normal SuccessfulDelete replicaset/e2e-worker-w2-de35cd-openshift-default-684fbf96fc Deleted pod: e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kj2h7 40m Normal AddedLabel gateway/e2e-shared-b-w2-d6b9c6 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-d6b9c6 40m Normal Pulled pod/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8 Container created 40m Normal Started pod/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8 Container started 40m Normal Killing pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7krbm Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kj2h7 Stopping container istio-proxy 40m Normal AddedLabel gateway/e2e-shared-b-w5-885c69 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w5-885c69 40m Warning ProbeError pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kj2h7 Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kj2h7 Readiness probe failed: HTTP probe failed with statuscode: 503 40m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f Created pod: e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p 40m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-885c69-openshift-default Scaled up replica set e2e-shared-b-w5-885c69-openshift-default-85b7c65d57 from 0 to 1 40m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-d6b9c6-openshift-default Scaled up replica set e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f from 0 to 1 40m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57 Created pod: e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg 40m Normal AddedInterface pod/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p Add eth0 [10.133.0.119/23] from ovn-kubernetes 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-fd1339-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-fd1339-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal AddedInterface pod/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg Add eth0 [10.133.0.120/23] from ovn-kubernetes 40m Warning ProbeError pod/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999 Startup probe error: Get "http://10.133.0.102:15021/healthz/ready": dial tcp 10.133.0.102:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999 Startup probe failed: Get "http://10.133.0.102:15021/healthz/ready": dial tcp 10.133.0.102:15021: connect: connection refused 40m Warning Unhealthy pod/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8 Startup probe failed: Get "http://10.133.0.103:15021/healthz/ready": dial tcp 10.133.0.103:15021: connect: connection refused 40m Warning ProbeError pod/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8 Startup probe error: Get "http://10.133.0.103:15021/healthz/ready": dial tcp 10.133.0.103:15021: connect: connection refused... 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-d6b9c6 Scaled up replica set payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7 from 0 to 1 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c Created pod: payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv 40m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7 Error creating: pods "payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6: serviceaccount "payload-processing-e2e-shared-b-w2-d6b9c6" not found 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7 Created pod: payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c Error creating: pods "payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6: serviceaccount "payload-processing-e2e-shared-b-w2-d6b9c6" not found 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-d6b9c6 Scaled up replica set payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c from 0 to 1 40m Normal Pulled pod/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv Add eth0 [10.133.0.122/23] from ovn-kubernetes 40m Normal Created pod/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl Container created 40m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl Add eth0 [10.133.0.123/23] from ovn-kubernetes 40m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv Container created 40m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv Container started 40m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Started pod/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl Container started 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-4c5b4e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-4c5b4e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w5-885c69-784df4fff Created pod: payload-pre-processing-e2e-shared-b-w5-885c69-784df4fff-gn9bz 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w5-885c69 Scaled up replica set payload-processing-e2e-shared-b-w5-885c69-74b4b487c9 from 0 to 1 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w5-885c69-784df4fff Error creating: pods "payload-pre-processing-e2e-shared-b-w5-885c69-784df4fff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-885c69: serviceaccount "payload-processing-e2e-shared-b-w5-885c69" not found 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w5-885c69 Scaled up replica set payload-pre-processing-e2e-shared-b-w5-885c69-784df4fff from 0 to 1 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w5-885c69-74b4b487c9 Created pod: payload-processing-e2e-shared-b-w5-885c69-74b4b487c9-7nfnb 40m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w5-885c69-74b4b487c9 Error creating: pods "payload-processing-e2e-shared-b-w5-885c69-74b4b487c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-885c69: serviceaccount "payload-processing-e2e-shared-b-w5-885c69" not found 40m Normal AddedInterface pod/payload-processing-e2e-shared-b-w5-885c69-74b4b487c9-7nfnb Add eth0 [10.134.0.114/23] from ovn-kubernetes 40m Normal Pulled pod/payload-processing-e2e-shared-b-w5-885c69-74b4b487c9-7nfnb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w5-885c69-784df4fff-gn9bz Add eth0 [10.132.0.167/23] from ovn-kubernetes 40m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w5-885c69-784df4fff-gn9bz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Started pod/payload-pre-processing-e2e-shared-b-w5-885c69-784df4fff-gn9bz Container started 40m Normal Created pod/payload-pre-processing-e2e-shared-b-w5-885c69-784df4fff-gn9bz Container created 40m Normal Started pod/payload-processing-e2e-shared-b-w5-885c69-74b4b487c9-7nfnb Container started 40m Normal Created pod/payload-processing-e2e-shared-b-w5-885c69-74b4b487c9-7nfnb Container created 39m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 4 to 3 39m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-9d6dj 39m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-9d6dj Stopping container discovery 39m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: All metrics below target 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-4c5b4e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-4c5b4e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-fd1339-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-fd1339-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal SuccessfulCreate replicaset/e2e-api-a-303ba8-openshift-default-68d7456995 Created pod: e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2 39m Normal AddedLabel gateway/e2e-api-a-303ba8 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-303ba8 39m Normal ScalingReplicaSet deployment/e2e-api-a-303ba8-openshift-default Scaled up replica set e2e-api-a-303ba8-openshift-default-68d7456995 from 0 to 1 39m Normal AddedInterface pod/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2 Add eth0 [10.133.0.125/23] from ovn-kubernetes 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499 Error creating: pods "payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-303ba8: serviceaccount "payload-processing-e2e-api-a-303ba8" not found 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-303ba8 Scaled up replica set payload-processing-e2e-api-a-303ba8-7fb5f66b89 from 0 to 1 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-303ba8-7fb5f66b89 Created pod: payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2 39m Warning FailedCreate replicaset/payload-processing-e2e-api-a-303ba8-7fb5f66b89 Error creating: pods "payload-processing-e2e-api-a-303ba8-7fb5f66b89-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-303ba8: serviceaccount "payload-processing-e2e-api-a-303ba8" not found 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-303ba8 Scaled up replica set payload-pre-processing-e2e-api-a-303ba8-5dc89d9499 from 0 to 1 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499 Created pod: payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b 39m Normal Pulled pod/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Started pod/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b Container started 39m Normal Created pod/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b Container created 39m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b Add eth0 [10.133.0.126/23] from ovn-kubernetes 39m Normal Started pod/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2 Container started 39m Normal Created pod/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2 Container created 39m Normal Pulled pod/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2 Add eth0 [10.133.0.127/23] from ovn-kubernetes 39m Normal AddedLabel gateway/e2e-inf-a-468af4 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-468af4 39m Normal ScalingReplicaSet deployment/e2e-inf-a-468af4-openshift-default Scaled up replica set e2e-inf-a-468af4-openshift-default-dd578d549 from 0 to 1 39m Normal SuccessfulCreate replicaset/e2e-inf-a-468af4-openshift-default-dd578d549 Created pod: e2e-inf-a-468af4-openshift-default-dd578d549-dbppz 39m Normal AddedInterface pod/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz Add eth0 [10.133.0.128/23] from ovn-kubernetes 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-de35cd-openshift-default New size: 4; reason: All metrics below target 39m Normal ScalingReplicaSet deployment/e2e-worker-w2-de35cd-openshift-default Scaled down replica set e2e-worker-w2-de35cd-openshift-default-684fbf96fc from 5 to 4 39m Normal SuccessfulDelete replicaset/e2e-worker-w2-de35cd-openshift-default-684fbf96fc Deleted pod: e2e-worker-w2-de35cd-openshift-default-684fbf96fc-njrmp 39m Normal Killing pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-njrmp Stopping container istio-proxy 39m Warning Unhealthy pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-njrmp Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning ProbeError pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-njrmp Readiness probe error: HTTP probe failed with statuscode: 503... 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-468af4-6b5fb57cd7 Created pod: payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9 Created pod: payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-468af4 Scaled up replica set payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9 from 0 to 1 39m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-468af4-6b5fb57cd7 Error creating: pods "payload-processing-e2e-inf-a-468af4-6b5fb57cd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-468af4: serviceaccount "payload-processing-e2e-inf-a-468af4" not found 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-468af4 Scaled up replica set payload-processing-e2e-inf-a-468af4-6b5fb57cd7 from 0 to 1 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9 Error creating: pods "payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-468af4: serviceaccount "payload-processing-e2e-inf-a-468af4" not found 39m Normal Started pod/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q Container started 39m Normal AddedInterface pod/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss Add eth0 [10.133.0.131/23] from ovn-kubernetes 39m Normal Created pod/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q Container created 39m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q Add eth0 [10.133.0.130/23] from ovn-kubernetes 39m Normal Pulled pod/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Started pod/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss Container started 39m Normal Created pod/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss Container created 39m Normal Pulled pod/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Pulled pod/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Started pod/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l Container started 39m Normal Created pod/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l Container created 38m Warning Unhealthy pod/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l Startup probe failed: Get "http://10.133.0.111:15021/healthz/ready": dial tcp 10.133.0.111:15021: connect: connection refused 38m Warning ProbeError pod/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l Startup probe error: Get "http://10.133.0.111:15021/healthz/ready": dial tcp 10.133.0.111:15021: connect: connection refused... 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-468af4-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-468af4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-eafc0d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-eafc0d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Normal Started pod/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk Container started 38m Normal Pulled pod/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk Container created 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-53fd96-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-53fd96-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning ProbeError pod/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk Startup probe error: Get "http://10.133.0.116:15021/healthz/ready": dial tcp 10.133.0.116:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk Startup probe failed: Get "http://10.133.0.116:15021/healthz/ready": dial tcp 10.133.0.116:15021: connect: connection refused 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-eafc0d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-eafc0d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Normal Killing pod/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9 Stopping container payload-processing 38m Normal Killing pod/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg Stopping container payload-pre-processing 38m Normal Started pod/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p Container started 38m Normal Pulled pod/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p Container created 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-d6b9c6-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-d6b9c6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning Unhealthy pod/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p Startup probe failed: Get "http://10.133.0.119:15021/healthz/ready": dial tcp 10.133.0.119:15021: connect: connection refused 38m Warning ProbeError pod/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p Startup probe error: Get "http://10.133.0.119:15021/healthz/ready": dial tcp 10.133.0.119:15021: connect: connection refused... 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-53fd96-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-53fd96-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ba49d0-openshift-default New size: 3; reason: All metrics below target 37m Normal Killing pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-6hqmk Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/e2e-worker-w1-019efe-openshift-default-5d87cb7859 Deleted pod: e2e-worker-w1-019efe-openshift-default-5d87cb7859-c6252 37m Normal SuccessfulDelete replicaset/e2e-worker-w1-019efe-openshift-default-5d87cb7859 Deleted pod: e2e-worker-w1-019efe-openshift-default-5d87cb7859-vdwrl 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-019efe-openshift-default New size: 3; reason: All metrics below target 37m Normal Killing pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-22679 Stopping container istio-proxy 37m Normal ScalingReplicaSet deployment/e2e-worker-w1-019efe-openshift-default Scaled down replica set e2e-worker-w1-019efe-openshift-default-5d87cb7859 from 6 to 3 37m Normal Killing pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-vdwrl Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/e2e-worker-w0-ba49d0-openshift-default-6b978b7557 Deleted pod: e2e-worker-w0-ba49d0-openshift-default-6b978b7557-k62kg 37m Normal Killing pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-tbbpc Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 Deleted pod: e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-22679 37m Normal SuccessfulDelete replicaset/e2e-worker-w0-ba49d0-openshift-default-6b978b7557 Deleted pod: e2e-worker-w0-ba49d0-openshift-default-6b978b7557-rm6ws 37m Normal SuccessfulDelete replicaset/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 Deleted pod: e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-6hqmk 37m Normal SuccessfulDelete replicaset/e2e-worker-w1-019efe-openshift-default-5d87cb7859 Deleted pod: e2e-worker-w1-019efe-openshift-default-5d87cb7859-6dk9z 37m Normal ScalingReplicaSet deployment/e2e-worker-w4-6dc69b-openshift-default Scaled down replica set e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 from 6 to 3 37m Normal SuccessfulDelete replicaset/e2e-worker-w5-db3931-openshift-default-5f47d69f9f Deleted pod: e2e-worker-w5-db3931-openshift-default-5f47d69f9f-tbbpc 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-6dc69b-openshift-default New size: 3; reason: All metrics below target 37m Normal Killing pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rftjh Stopping container istio-proxy 37m Normal Killing pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-c6252 Stopping container istio-proxy 37m Normal ScalingReplicaSet deployment/e2e-worker-w5-db3931-openshift-default Scaled down replica set e2e-worker-w5-db3931-openshift-default-5f47d69f9f from 5 to 4 37m Normal SuccessfulDelete replicaset/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7 Deleted pod: e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rftjh 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-db3931-openshift-default New size: 4; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/e2e-worker-w0-ba49d0-openshift-default Scaled down replica set e2e-worker-w0-ba49d0-openshift-default-6b978b7557 from 5 to 3 37m Normal Killing pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-6dk9z Stopping container istio-proxy 37m Normal Killing pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-rm6ws Stopping container istio-proxy 37m Normal Killing pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-k62kg Stopping container istio-proxy 37m Normal Killing pod/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284 Stopping container payload-pre-processing 37m Normal Killing pod/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m Stopping container payload-processing 37m Warning ProbeError pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-c6252 Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-d6b9c6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning Unhealthy pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-c6252 Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning Unhealthy pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-22679 Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-22679 Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-rm6ws Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-rm6ws Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-d6b9c6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Normal ScalingReplicaSet deployment/e2e-worker-w6-c18e79-openshift-default Scaled down replica set e2e-worker-w6-c18e79-openshift-default-59544557d7 from 6 to 4 37m Normal SuccessfulDelete replicaset/e2e-worker-w6-c18e79-openshift-default-59544557d7 Deleted pod: e2e-worker-w6-c18e79-openshift-default-59544557d7-cs984 37m Normal Killing pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-6tjz5 Stopping container istio-proxy 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-c18e79-openshift-default New size: 4; reason: All metrics below target 37m Warning Unhealthy pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-vdwrl Readiness probe failed: HTTP probe failed with statuscode: 503 37m Normal Killing pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-cs984 Stopping container istio-proxy 37m Warning ProbeError pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-vdwrl Readiness probe error: HTTP probe failed with statuscode: 503... 37m Normal SuccessfulDelete replicaset/e2e-worker-w6-c18e79-openshift-default-59544557d7 Deleted pod: e2e-worker-w6-c18e79-openshift-default-59544557d7-6tjz5 37m Warning Unhealthy pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-tbbpc Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-tbbpc Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-k62kg Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-6dk9z Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-6dk9z Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-k62kg Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning ProbeError pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-cs984 Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-cs984 Readiness probe failed: HTTP probe failed with statuscode: 503 37m Normal Killing pod/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk Stopping container istio-proxy 37m Warning ProbeError pod/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2 Readiness probe error: Get "http://10.133.0.125:15021/healthz/ready": read tcp 10.133.0.2:33720->10.133.0.125:15021: read: connection reset by peer... 37m Normal Created pod/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg Container created 37m Normal Created pod/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2 Container created 37m Warning Unhealthy pod/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2 Readiness probe failed: Get "http://10.133.0.125:15021/healthz/ready": read tcp 10.133.0.2:33720->10.133.0.125:15021: read: connection reset by peer 37m Normal Pulled pod/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Pulled pod/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Started pod/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2 Container started 37m Normal Started pod/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg Container started 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-885c69-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-885c69-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning Unhealthy pod/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg Startup probe failed: Get "http://10.133.0.120:15021/healthz/ready": dial tcp 10.133.0.120:15021: connect: connection refused 37m Warning ProbeError pod/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg Startup probe error: Get "http://10.133.0.120:15021/healthz/ready": dial tcp 10.133.0.120:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2 Startup probe failed: Get "http://10.133.0.125:15021/healthz/ready": dial tcp 10.133.0.125:15021: connect: connection refused 37m Warning ProbeError pod/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2 Startup probe error: Get "http://10.133.0.125:15021/healthz/ready": dial tcp 10.133.0.125:15021: connect: connection refused... 37m Normal Killing pod/payload-processing-e2e-worker-w0-ba49d0-c49b65df5-qvfjh Stopping container payload-processing 37m Normal Killing pod/payload-pre-processing-e2e-worker-w0-ba49d0-79cdb5449f-m7srf Stopping container payload-pre-processing 37m Normal SuccessfulDelete replicaset/e2e-worker-w5-db3931-openshift-default-5f47d69f9f Deleted pod: e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7shxl 37m Normal Killing pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/e2e-worker-w2-de35cd-openshift-default-684fbf96fc Deleted pod: e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-de35cd-openshift-default New size: 3; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/e2e-worker-w2-de35cd-openshift-default Scaled down replica set e2e-worker-w2-de35cd-openshift-default-684fbf96fc from 4 to 3 37m Normal SuccessfulDelete replicaset/e2e-worker-w3-42555d-openshift-default-6c8c6d89c Deleted pod: e2e-worker-w3-42555d-openshift-default-6c8c6d89c-5sfdr 37m Normal Killing pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7shxl Stopping container istio-proxy 37m Normal Killing pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-5sfdr Stopping container istio-proxy 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-db3931-openshift-default New size: 3; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/e2e-worker-w3-42555d-openshift-default Scaled down replica set e2e-worker-w3-42555d-openshift-default-6c8c6d89c from 6 to 5 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-42555d-openshift-default New size: 5; reason: All metrics below target 37m Warning Unhealthy pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq Readiness probe error: HTTP probe failed with statuscode: 503... 37m Normal Killing pod/payload-processing-e2e-shared-a-w2-559973-7b9b654d6f-pptpc Stopping container payload-processing 37m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-559973-c967685f-ls5fw Stopping container payload-pre-processing 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-c18e79-openshift-default New size: 3; reason: All metrics below target 37m Warning ProbeError pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-5sfdr Readiness probe error: HTTP probe failed with statuscode: 503... 37m Normal SuccessfulDelete replicaset/e2e-worker-w6-c18e79-openshift-default-59544557d7 Deleted pod: e2e-worker-w6-c18e79-openshift-default-59544557d7-kzwzk 37m Warning Unhealthy pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-5sfdr Readiness probe failed: HTTP probe failed with statuscode: 503 37m Normal Killing pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-kzwzk Stopping container istio-proxy 37m Normal ScalingReplicaSet deployment/e2e-worker-w6-c18e79-openshift-default Scaled down replica set e2e-worker-w6-c18e79-openshift-default-59544557d7 from 4 to 3 37m Warning Unhealthy pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7shxl Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-7shxl Readiness probe error: HTTP probe failed with statuscode: 503... 37m Normal Killing pod/payload-pre-processing-e2e-shared-a-w5-6b2f61-798968dcb7-2rkdk Stopping container payload-pre-processing 37m Normal Killing pod/payload-processing-e2e-shared-a-w5-6b2f61-78b8fcdb9c-ttp26 Stopping container payload-processing 37m Normal Killing pod/payload-pre-processing-e2e-mt-de9c4ba4-6564f5d4dc-s8wg7 Stopping container payload-pre-processing 37m Normal Killing pod/payload-processing-e2e-mt-de9c4ba4-768fdffc87-hzb6r Stopping container payload-processing 37m Warning Unhealthy pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-kzwzk Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-kzwzk Readiness probe error: HTTP probe failed with statuscode: 503... 36m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 36m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 36m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-hh82k 36m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-hh82k Stopping container discovery 36m Normal ScalingReplicaSet deployment/e2e-worker-w5-db3931-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w5-db3931-openshift-default-5f47d69f9f from 3 to 2 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-db3931-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 36m Normal Killing pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf Stopping container istio-proxy 36m Normal SuccessfulDelete replicaset/e2e-worker-w3-42555d-openshift-default-6c8c6d89c Deleted pod: e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf 36m Normal Killing pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h Stopping container istio-proxy 36m Normal SuccessfulDelete replicaset/e2e-worker-w5-db3931-openshift-default-5f47d69f9f Deleted pod: e2e-worker-w5-db3931-openshift-default-5f47d69f9f-rnfds 36m Normal Killing pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-rnfds Stopping container istio-proxy 36m Normal SuccessfulDelete replicaset/e2e-worker-w3-42555d-openshift-default-6c8c6d89c Deleted pod: e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-42555d-openshift-default New size: 3; reason: All metrics below target 36m Normal ScalingReplicaSet deployment/e2e-worker-w3-42555d-openshift-default Scaled down replica set e2e-worker-w3-42555d-openshift-default-6c8c6d89c from 5 to 3 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-885c69-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-885c69-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning Unhealthy pod/payload-processing-e2e-worker-w0-ba49d0-c49b65df5-qvfjh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Warning Unhealthy pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning ProbeError pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning Unhealthy pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-rnfds Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning ProbeError pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-rnfds Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-ba49d0-79cdb5449f-m7srf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Warning Unhealthy pod/payload-processing-e2e-mt-de9c4ba4-768fdffc87-hzb6r Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Warning Unhealthy pod/payload-pre-processing-e2e-mt-de9c4ba4-6564f5d4dc-s8wg7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-303ba8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-303ba8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal Killing pod/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l Stopping container istio-proxy 36m Normal Killing pod/maas-default-gateway-openshift-default-75fd9f5d47-zzbkt Stopping container istio-proxy 36m Normal Killing pod/maas-default-gateway-openshift-default-75fd9f5d47-qwd8j Stopping container istio-proxy 36m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-75fd9f5d47 from 10 to 6 36m Normal Killing pod/maas-default-gateway-openshift-default-75fd9f5d47-6qcng Stopping container istio-proxy 36m Normal Killing pod/maas-default-gateway-openshift-default-75fd9f5d47-lcdkf Stopping container istio-proxy 36m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: All metrics below target 36m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75fd9f5d47 Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-6qcng 36m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-75fd9f5d47 Deleted pod: maas-default-gateway-openshift-default-75fd9f5d47-lcdkf 36m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-lcdkf Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-6qcng Readiness probe error: Get "http://10.132.0.152:15021/healthz/ready": dial tcp 10.132.0.152:15021: connect: connection refused... 36m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-lcdkf Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-6qcng Readiness probe failed: Get "http://10.132.0.152:15021/healthz/ready": dial tcp 10.132.0.152:15021: connect: connection refused 36m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-qwd8j Readiness probe failed: Get "http://10.132.0.34:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 36m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-qwd8j Readiness probe error: Get "http://10.132.0.34:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 36m Normal Killing pod/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2 Stopping container payload-processing 36m Normal Killing pod/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b Stopping container payload-pre-processing 36m Normal Killing pod/payload-processing-e2e-worker-w4-6dc69b-5fd6bbbf9f-gdvfc Stopping container payload-processing 36m Normal Killing pod/payload-pre-processing-e2e-worker-w4-6dc69b-68d55b7995-kl9q4 Stopping container payload-pre-processing 36m Warning Unhealthy pod/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b Readiness probe failed: timeout: failed to connect service "10.133.0.126:50052" within 1s: context deadline exceeded 36m Normal Killing pod/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2 Stopping container istio-proxy 36m Normal SuccessfulCreate replicaset/e2e-inf-a-39098c-openshift-default-76bff56b5f Created pod: e2e-inf-a-39098c-openshift-default-76bff56b5f-b9c7h 36m Normal AddedLabel gateway/e2e-inf-a-39098c Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-39098c 36m Normal ScalingReplicaSet deployment/e2e-inf-a-39098c-openshift-default Scaled up replica set e2e-inf-a-39098c-openshift-default-76bff56b5f from 0 to 1 36m Normal AddedInterface pod/e2e-inf-a-39098c-openshift-default-76bff56b5f-b9c7h Add eth0 [10.132.0.192/23] from ovn-kubernetes 36m Normal Killing pod/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss Stopping container payload-processing 36m Normal Killing pod/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q Stopping container payload-pre-processing 36m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w4-6dc69b-68d55b7995-kl9q4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Warning Unhealthy pod/payload-processing-e2e-worker-w4-6dc69b-5fd6bbbf9f-gdvfc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-39098c Scaled up replica set payload-processing-e2e-inf-a-39098c-7dfd88f45c from 0 to 1 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-39098c Scaled up replica set payload-pre-processing-e2e-inf-a-39098c-85df849f9f from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-39098c-85df849f9f Created pod: payload-pre-processing-e2e-inf-a-39098c-85df849f9f-7vp5x 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-39098c-7dfd88f45c Created pod: payload-processing-e2e-inf-a-39098c-7dfd88f45c-5mh26 36m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-39098c-7dfd88f45c Error creating: pods "payload-processing-e2e-inf-a-39098c-7dfd88f45c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-39098c: serviceaccount "payload-processing-e2e-inf-a-39098c" not found 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-39098c-85df849f9f Error creating: pods "payload-pre-processing-e2e-inf-a-39098c-85df849f9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-39098c: serviceaccount "payload-processing-e2e-inf-a-39098c" not found 35m Normal Pulled pod/payload-pre-processing-e2e-inf-a-39098c-85df849f9f-7vp5x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Started pod/payload-pre-processing-e2e-inf-a-39098c-85df849f9f-7vp5x Container started 35m Normal Created pod/payload-pre-processing-e2e-inf-a-39098c-85df849f9f-7vp5x Container created 35m Normal AddedInterface pod/payload-processing-e2e-inf-a-39098c-7dfd88f45c-5mh26 Add eth0 [10.132.0.194/23] from ovn-kubernetes 35m Normal Started pod/payload-processing-e2e-inf-a-39098c-7dfd88f45c-5mh26 Container started 35m Normal Pulled pod/payload-processing-e2e-inf-a-39098c-7dfd88f45c-5mh26 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Created pod/payload-processing-e2e-inf-a-39098c-7dfd88f45c-5mh26 Container created 35m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-39098c-85df849f9f-7vp5x Add eth0 [10.132.0.193/23] from ovn-kubernetes 35m Normal Started pod/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz Container started 35m Normal Created pod/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz Container created 35m Normal Pulled pod/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal ScalingReplicaSet deployment/e2e-worker-w2-de35cd-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w2-de35cd-openshift-default-684fbf96fc from 3 to 2 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-de35cd-openshift-default New size: 2; reason: All metrics below target 35m Normal Killing pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kpchw Stopping container istio-proxy 35m Normal SuccessfulDelete replicaset/e2e-worker-w2-de35cd-openshift-default-684fbf96fc Deleted pod: e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kpchw 35m Warning Unhealthy pod/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz Startup probe failed: Get "http://10.133.0.128:15021/healthz/ready": dial tcp 10.133.0.128:15021: connect: connection refused 35m Warning ProbeError pod/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz Startup probe error: Get "http://10.133.0.128:15021/healthz/ready": dial tcp 10.133.0.128:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kpchw Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-kpchw Readiness probe error: HTTP probe failed with statuscode: 503... 35m Normal Killing pod/e2e-shared-a-w2-559973-openshift-default-5974f778dc-frvnh Stopping container istio-proxy 35m Normal Killing pod/e2e-shared-a-w5-6b2f61-openshift-default-6466884b9f-5pxf2 Stopping container istio-proxy 35m Warning Unhealthy pod/e2e-shared-a-w5-6b2f61-openshift-default-6466884b9f-5pxf2 Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning Unhealthy pod/e2e-shared-a-w2-559973-openshift-default-5974f778dc-frvnh Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-shared-a-w5-6b2f61-openshift-default-6466884b9f-5pxf2 Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning ProbeError pod/e2e-shared-a-w2-559973-openshift-default-5974f778dc-frvnh Readiness probe error: HTTP probe failed with statuscode: 503... 35m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl Stopping container payload-processing 35m Normal Killing pod/payload-processing-e2e-shared-b-w5-885c69-74b4b487c9-7nfnb Stopping container payload-processing 35m Normal Killing pod/payload-pre-processing-e2e-shared-b-w5-885c69-784df4fff-gn9bz Stopping container payload-pre-processing 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-468af4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-468af4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Normal Killing pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-zsvr9 Stopping container istio-proxy 35m Normal Killing pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-2qxwf Stopping container istio-proxy 35m Normal Killing pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-sv6lk Stopping container istio-proxy 35m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-sv6lk Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-sv6lk Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-zsvr9 Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-zsvr9 Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning Unhealthy pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-2qxwf Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-2qxwf Readiness probe error: HTTP probe failed with statuscode: 503... 34m Normal Killing pod/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz Stopping container istio-proxy 34m Warning Unhealthy pod/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz Readiness probe failed: Get "http://10.133.0.128:15021/healthz/ready": dial tcp 10.133.0.128:15021: connect: connection refused 34m Warning ProbeError pod/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz Readiness probe error: Get "http://10.133.0.128:15021/healthz/ready": dial tcp 10.133.0.128:15021: connect: connection refused... 34m Normal Killing pod/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p Stopping container istio-proxy 34m Normal Killing pod/payload-pre-processing-e2e-shared-a-w3-b87e90-76d87d9fb7-n5vf8 Stopping container payload-pre-processing 34m Normal Killing pod/payload-processing-e2e-shared-a-w3-b87e90-68f96cc4c7-fx9v5 Stopping container payload-processing 34m Normal Killing pod/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg Stopping container istio-proxy 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-de9c4ba4 Scaled up replica set payload-processing-e2e-mt-de9c4ba4-768fdffc87 from 0 to 1 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-de9c4ba4 Scaled up replica set payload-pre-processing-e2e-mt-de9c4ba4-6564f5d4dc from 0 to 1 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-de9c4ba4-6564f5d4dc Error creating: pods "payload-pre-processing-e2e-mt-de9c4ba4-6564f5d4dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-de9c4ba4: serviceaccount "payload-processing-e2e-mt-de9c4ba4" not found 34m Normal Started pod/payload-pre-processing-e2e-mt-de9c4ba4-6564f5d4dc-866q8 Container started 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-de9c4ba4-768fdffc87 Created pod: payload-processing-e2e-mt-de9c4ba4-768fdffc87-dmd48 34m Normal Pulled pod/payload-pre-processing-e2e-mt-de9c4ba4-6564f5d4dc-866q8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Created pod/payload-pre-processing-e2e-mt-de9c4ba4-6564f5d4dc-866q8 Container created 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-de9c4ba4-6564f5d4dc Created pod: payload-pre-processing-e2e-mt-de9c4ba4-6564f5d4dc-866q8 34m Normal AddedInterface pod/payload-pre-processing-e2e-mt-de9c4ba4-6564f5d4dc-866q8 Add eth0 [10.132.0.202/23] from ovn-kubernetes 34m Warning FailedCreate replicaset/payload-processing-e2e-mt-de9c4ba4-768fdffc87 Error creating: pods "payload-processing-e2e-mt-de9c4ba4-768fdffc87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-de9c4ba4: serviceaccount "payload-processing-e2e-mt-de9c4ba4" not found 34m Normal Pulled pod/payload-processing-e2e-mt-de9c4ba4-768fdffc87-dmd48 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Started pod/payload-processing-e2e-mt-de9c4ba4-768fdffc87-dmd48 Container started 34m Normal AddedInterface pod/payload-processing-e2e-mt-de9c4ba4-768fdffc87-dmd48 Add eth0 [10.132.0.203/23] from ovn-kubernetes 34m Normal Created pod/payload-processing-e2e-mt-de9c4ba4-768fdffc87-dmd48 Container created 34m Normal Killing pod/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm Stopping container istio-proxy 34m Normal Killing pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-fzgw4 Stopping container istio-proxy 34m Warning Unhealthy pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-d9msj Readiness probe failed: HTTP probe failed with statuscode: 503 34m Normal Killing pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-ntwpq Stopping container istio-proxy 34m Warning ProbeError pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-d9msj Readiness probe error: HTTP probe failed with statuscode: 503... 34m Normal Killing pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-d9msj Stopping container istio-proxy 34m Normal Pulled pod/e2e-inf-a-39098c-openshift-default-76bff56b5f-b9c7h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-inf-a-39098c-openshift-default-76bff56b5f-b9c7h Container created 33m Normal Killing pod/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6 Stopping container payload-pre-processing 33m Normal Killing pod/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt Stopping container payload-processing 33m Normal Started pod/e2e-inf-a-39098c-openshift-default-76bff56b5f-b9c7h Container started 33m Warning Unhealthy pod/e2e-inf-a-39098c-openshift-default-76bff56b5f-b9c7h Startup probe failed: Get "http://10.132.0.192:15021/healthz/ready": dial tcp 10.132.0.192:15021: connect: connection refused 33m Warning ProbeError pod/e2e-inf-a-39098c-openshift-default-76bff56b5f-b9c7h Startup probe error: Get "http://10.132.0.192:15021/healthz/ready": dial tcp 10.132.0.192:15021: connect: connection refused... 33m Normal AddedLabel gateway/e2e-inf-b-4e4683 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-4e4683 33m Normal AddedInterface pod/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g Add eth0 [10.133.0.136/23] from ovn-kubernetes 33m Normal SuccessfulCreate replicaset/e2e-inf-b-4e4683-openshift-default-5d7577cbc7 Created pod: e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g 33m Normal ScalingReplicaSet deployment/e2e-inf-b-4e4683-openshift-default Scaled up replica set e2e-inf-b-4e4683-openshift-default-5d7577cbc7 from 0 to 1 33m Normal Killing pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lrjs8 Stopping container istio-proxy 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-c18e79-openshift-default New size: 2; reason: All metrics below target 33m Normal SuccessfulDelete replicaset/e2e-worker-w6-c18e79-openshift-default-59544557d7 Deleted pod: e2e-worker-w6-c18e79-openshift-default-59544557d7-lrjs8 33m Warning Unhealthy pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lrjs8 Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lrjs8 Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-4e4683-66d784d5c5 Error creating: pods "payload-pre-processing-e2e-inf-b-4e4683-66d784d5c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-4e4683: serviceaccount "payload-processing-e2e-inf-b-4e4683" not found 33m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-4e4683 Scaled up replica set payload-pre-processing-e2e-inf-b-4e4683-66d784d5c5 from 0 to 1 33m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-4e4683 Scaled up replica set payload-processing-e2e-inf-b-4e4683-7784dc776c from 0 to 1 33m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-4e4683-66d784d5c5 Created pod: payload-pre-processing-e2e-inf-b-4e4683-66d784d5c5-9b5lb 33m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-4e4683-7784dc776c Error creating: pods "payload-processing-e2e-inf-b-4e4683-7784dc776c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-4e4683: serviceaccount "payload-processing-e2e-inf-b-4e4683" not found 33m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-4e4683-7784dc776c Created pod: payload-processing-e2e-inf-b-4e4683-7784dc776c-n6ch5 33m Normal Started pod/payload-processing-e2e-inf-b-4e4683-7784dc776c-n6ch5 Container started 33m Normal AddedInterface pod/payload-processing-e2e-inf-b-4e4683-7784dc776c-n6ch5 Add eth0 [10.134.0.123/23] from ovn-kubernetes 33m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-4e4683-66d784d5c5-9b5lb Add eth0 [10.132.0.206/23] from ovn-kubernetes 33m Normal Started pod/payload-pre-processing-e2e-inf-b-4e4683-66d784d5c5-9b5lb Container started 33m Normal Pulled pod/payload-pre-processing-e2e-inf-b-4e4683-66d784d5c5-9b5lb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal Created pod/payload-pre-processing-e2e-inf-b-4e4683-66d784d5c5-9b5lb Container created 33m Normal Created pod/payload-processing-e2e-inf-b-4e4683-7784dc776c-n6ch5 Container created 33m Normal Pulled pod/payload-processing-e2e-inf-b-4e4683-7784dc776c-n6ch5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-39098c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-39098c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-39098c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-39098c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Normal Killing pod/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8 Stopping container istio-proxy 33m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-75fd9f5d47 from 6 to 10 33m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-75fd9f5d47 Created pod: maas-default-gateway-openshift-default-75fd9f5d47-4lr42 33m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 33m Normal AddedInterface pod/maas-default-gateway-openshift-default-75fd9f5d47-gv6hc Add eth0 [10.132.0.209/23] from ovn-kubernetes 33m Normal Started pod/maas-default-gateway-openshift-default-75fd9f5d47-5j55g Container started 33m Normal AddedInterface pod/maas-default-gateway-openshift-default-75fd9f5d47-4lr42 Add eth0 [10.132.0.208/23] from ovn-kubernetes 33m Normal Created pod/maas-default-gateway-openshift-default-75fd9f5d47-j8c7c Container created 33m Normal AddedInterface pod/maas-default-gateway-openshift-default-75fd9f5d47-5j55g Add eth0 [10.133.0.138/23] from ovn-kubernetes 33m Normal Pulled pod/maas-default-gateway-openshift-default-75fd9f5d47-gv6hc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Pulled pod/maas-default-gateway-openshift-default-75fd9f5d47-5j55g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/maas-default-gateway-openshift-default-75fd9f5d47-5j55g Container created 33m Normal Pulled pod/maas-default-gateway-openshift-default-75fd9f5d47-4lr42 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Started pod/maas-default-gateway-openshift-default-75fd9f5d47-j8c7c Container started 33m Normal Pulled pod/maas-default-gateway-openshift-default-75fd9f5d47-j8c7c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Started pod/maas-default-gateway-openshift-default-75fd9f5d47-gv6hc Container started 33m Normal AddedInterface pod/maas-default-gateway-openshift-default-75fd9f5d47-j8c7c Add eth0 [10.132.0.210/23] from ovn-kubernetes 33m Normal Created pod/maas-default-gateway-openshift-default-75fd9f5d47-4lr42 Container created 33m Normal Started pod/maas-default-gateway-openshift-default-75fd9f5d47-4lr42 Container started 33m Normal Created pod/maas-default-gateway-openshift-default-75fd9f5d47-gv6hc Container created 32m Normal ScalingReplicaSet deployment/e2e-mt-de9c4ba4-openshift-default Scaled up replica set e2e-mt-de9c4ba4-openshift-default-854f5f846f from 1 to 3 32m Normal SuccessfulCreate replicaset/e2e-mt-de9c4ba4-openshift-default-854f5f846f Created pod: e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-de9c4ba4-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 32m Normal SuccessfulCreate replicaset/e2e-mt-de9c4ba4-openshift-default-854f5f846f Created pod: e2e-mt-de9c4ba4-openshift-default-854f5f846f-s87nf 32m Normal AddedInterface pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k Add eth0 [10.133.0.139/23] from ovn-kubernetes 32m Normal Pulled pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k Container created 32m Normal Started pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k Container started 32m Normal AddedInterface pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-s87nf Add eth0 [10.132.0.211/23] from ovn-kubernetes 32m Normal Killing pod/payload-pre-processing-e2e-worker-w3-42555d-756d9fcdc-mt9h6 Stopping container payload-pre-processing 32m Normal Killing pod/payload-processing-e2e-worker-w3-42555d-7c658fbd6c-st4wb Stopping container payload-processing 32m Normal Pulled pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-s87nf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-s87nf Container created 32m Normal Started pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-s87nf Container started 32m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-5j55g Startup probe failed: Get "http://10.133.0.138:15021/healthz/ready": dial tcp 10.133.0.138:15021: connect: connection refused 32m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-gv6hc Startup probe failed: Get "http://10.132.0.209:15021/healthz/ready": dial tcp 10.132.0.209:15021: connect: connection refused 32m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-5j55g Startup probe error: Get "http://10.133.0.138:15021/healthz/ready": dial tcp 10.133.0.138:15021: connect: connection refused... 32m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-j8c7c Startup probe failed: Get "http://10.132.0.210:15021/healthz/ready": dial tcp 10.132.0.210:15021: connect: connection refused 32m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-j8c7c Startup probe error: Get "http://10.132.0.210:15021/healthz/ready": dial tcp 10.132.0.210:15021: connect: connection refused... 32m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-4lr42 Startup probe failed: Get "http://10.132.0.208:15021/healthz/ready": dial tcp 10.132.0.208:15021: connect: connection refused 32m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-4lr42 Startup probe error: Get "http://10.132.0.208:15021/healthz/ready": dial tcp 10.132.0.208:15021: connect: connection refused... 32m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-gv6hc Startup probe error: Get "http://10.132.0.209:15021/healthz/ready": dial tcp 10.132.0.209:15021: connect: connection refused... 32m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-j8c7c Startup probe failed: Get "http://10.132.0.210:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 32m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-gv6hc Startup probe error: Get "http://10.132.0.209:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 32m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-gv6hc Startup probe failed: Get "http://10.132.0.209:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 32m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-4lr42 Startup probe error: Get "http://10.132.0.208:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 32m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-4lr42 Startup probe failed: Get "http://10.132.0.208:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 32m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-j8c7c Startup probe error: Get "http://10.132.0.210:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 32m Warning ProbeError pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k Startup probe error: Get "http://10.133.0.139:15021/healthz/ready": dial tcp 10.133.0.139:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k Startup probe failed: Get "http://10.133.0.139:15021/healthz/ready": dial tcp 10.133.0.139:15021: connect: connection refused 32m Normal SuccessfulDelete replicaset/e2e-worker-w1-019efe-openshift-default-5d87cb7859 Deleted pod: e2e-worker-w1-019efe-openshift-default-5d87cb7859-x75hl 32m Normal Killing pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-x75hl Stopping container istio-proxy 32m Normal ScalingReplicaSet deployment/e2e-worker-w1-019efe-openshift-default Scaled down replica set e2e-worker-w1-019efe-openshift-default-5d87cb7859 from 3 to 2 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-019efe-openshift-default New size: 2; reason: All metrics below target 32m Warning ProbeError pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-s87nf Startup probe error: Get "http://10.132.0.211:15021/healthz/ready": dial tcp 10.132.0.211:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-s87nf Startup probe failed: Get "http://10.132.0.211:15021/healthz/ready": dial tcp 10.132.0.211:15021: connect: connection refused 32m Warning ProbeError pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-x75hl Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning Unhealthy pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-x75hl Readiness probe failed: HTTP probe failed with statuscode: 503 32m Normal SuccessfulCreate replicaset/e2e-mt-de9c4ba4-openshift-default-854f5f846f Created pod: e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt 32m Normal SuccessfulCreate replicaset/e2e-mt-de9c4ba4-openshift-default-854f5f846f Created pod: e2e-mt-de9c4ba4-openshift-default-854f5f846f-2sdrk 32m Normal ScalingReplicaSet deployment/e2e-mt-de9c4ba4-openshift-default Scaled up replica set e2e-mt-de9c4ba4-openshift-default-854f5f846f from 3 to 6 32m Normal SuccessfulCreate replicaset/e2e-mt-de9c4ba4-openshift-default-854f5f846f Created pod: e2e-mt-de9c4ba4-openshift-default-854f5f846f-fszfw 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-de9c4ba4-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 32m Normal Pulled pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt Container created 32m Normal AddedInterface pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-2sdrk Add eth0 [10.132.0.212/23] from ovn-kubernetes 32m Normal AddedInterface pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt Add eth0 [10.133.0.141/23] from ovn-kubernetes 32m Normal AddedInterface pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-fszfw Add eth0 [10.134.0.124/23] from ovn-kubernetes 32m Normal Started pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt Container started 32m Normal Started pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-2sdrk Container started 32m Normal Created pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-2sdrk Container created 32m Normal Pulled pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-2sdrk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Pulled pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-fszfw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-fszfw Container created 32m Normal Started pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-fszfw Container started 32m Warning Unhealthy pod/payload-processing-e2e-worker-w3-42555d-7c658fbd6c-st4wb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Warning Unhealthy pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-fszfw Startup probe failed: Get "http://10.134.0.124:15021/healthz/ready": dial tcp 10.134.0.124:15021: connect: connection refused 32m Warning ProbeError pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-2sdrk Startup probe error: Get "http://10.132.0.212:15021/healthz/ready": dial tcp 10.132.0.212:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-2sdrk Startup probe failed: Get "http://10.132.0.212:15021/healthz/ready": dial tcp 10.132.0.212:15021: connect: connection refused 32m Warning ProbeError pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-fszfw Startup probe error: Get "http://10.134.0.124:15021/healthz/ready": dial tcp 10.134.0.124:15021: connect: connection refused... 32m Warning ProbeError pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt Startup probe error: Get "http://10.133.0.141:15021/healthz/ready": dial tcp 10.133.0.141:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt Startup probe failed: Get "http://10.133.0.141:15021/healthz/ready": dial tcp 10.133.0.141:15021: connect: connection refused 32m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w3-42555d-756d9fcdc-mt9h6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-de9c4ba4-openshift-default New size: 7; reason: 32m Normal SuccessfulCreate replicaset/e2e-mt-de9c4ba4-openshift-default-854f5f846f Created pod: e2e-mt-de9c4ba4-openshift-default-854f5f846f-j2s2j 32m Normal ScalingReplicaSet deployment/e2e-mt-de9c4ba4-openshift-default Scaled up replica set e2e-mt-de9c4ba4-openshift-default-854f5f846f from 6 to 7 32m Normal Pulled pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-j2s2j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-j2s2j Container created 32m Normal AddedInterface pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-j2s2j Add eth0 [10.134.0.126/23] from ovn-kubernetes 32m Normal Started pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-j2s2j Container started 32m Normal Killing pod/payload-processing-e2e-mt-de9c4ba4-768fdffc87-dmd48 Stopping container payload-processing 32m Normal Killing pod/payload-pre-processing-e2e-mt-de9c4ba4-6564f5d4dc-866q8 Stopping container payload-pre-processing 32m Warning ProbeError pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-j2s2j Startup probe error: Get "http://10.134.0.126:15021/healthz/ready": dial tcp 10.134.0.126:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-j2s2j Startup probe failed: Get "http://10.134.0.126:15021/healthz/ready": dial tcp 10.134.0.126:15021: connect: connection refused 32m Normal Started pod/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g Container started 32m Normal Pulled pod/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g Container created 32m Warning Unhealthy pod/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g Startup probe failed: Get "http://10.133.0.136:15021/healthz/ready": dial tcp 10.133.0.136:15021: connect: connection refused 32m Warning ProbeError pod/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g Startup probe error: Get "http://10.133.0.136:15021/healthz/ready": dial tcp 10.133.0.136:15021: connect: connection refused... 32m Normal Killing pod/payload-processing-e2e-worker-w2-de35cd-c8f875df5-fj5qn Stopping container payload-processing 32m Normal Killing pod/payload-pre-processing-e2e-worker-w2-de35cd-7dddfd879-8kn45 Stopping container payload-pre-processing 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-4e4683-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-4e4683-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-4e4683-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-4e4683-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning Unhealthy pod/payload-processing-e2e-worker-w2-de35cd-c8f875df5-fj5qn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-de35cd-7dddfd879-8kn45 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Normal Killing pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-zntdc Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-bq2wn Stopping container istio-proxy 31m Warning ProbeError pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-bq2wn Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning Unhealthy pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md Readiness probe failed: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused 31m Warning ProbeError pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md Readiness probe error: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-bq2wn Readiness probe failed: HTTP probe failed with statuscode: 503 31m Normal Killing pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-tlwnb Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7 Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-s87nf Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-fj6qk Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-fszfw Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-2sdrk Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-j2s2j Stopping container istio-proxy 30m Normal Killing pod/payload-pre-processing-e2e-worker-w5-db3931-5b4bd8c8c5-g8gwx Stopping container payload-pre-processing 30m Normal Killing pod/payload-processing-e2e-worker-w5-db3931-587478dc8f-gbx6h Stopping container payload-processing 30m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w5-db3931-5b4bd8c8c5-g8gwx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Warning Unhealthy pod/payload-processing-e2e-worker-w5-db3931-587478dc8f-gbx6h Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Normal Killing pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-x5xvr Stopping container istio-proxy 29m Normal Killing pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-mm2v8 Stopping container istio-proxy 29m Normal Killing pod/payload-pre-processing-e2e-worker-w6-c18e79-5fcfd69bc9-dmk26 Stopping container payload-pre-processing 29m Normal Killing pod/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4 Stopping container payload-processing 29m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w6-c18e79-5fcfd69bc9-dmk26 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 28m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-g5twx Stopping container discovery 28m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-g5twx 28m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-c18e79-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 28m Normal ScalingReplicaSet deployment/e2e-worker-w6-c18e79-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w6-c18e79-openshift-default-59544557d7 from 2 to 1 28m Normal SuccessfulDelete replicaset/e2e-worker-w6-c18e79-openshift-default-59544557d7 Deleted pod: e2e-worker-w6-c18e79-openshift-default-59544557d7-9rhdc 28m Normal Killing pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-9rhdc Stopping container istio-proxy 28m Normal ScalingReplicaSet deployment/e2e-worker-w1-019efe-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w1-019efe-openshift-default-5d87cb7859 from 2 to 1 28m Normal Killing pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-bj5xt Stopping container istio-proxy 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-019efe-openshift-default New size: 1; reason: All metrics below target 28m Normal SuccessfulDelete replicaset/e2e-worker-w1-019efe-openshift-default-5d87cb7859 Deleted pod: e2e-worker-w1-019efe-openshift-default-5d87cb7859-bj5xt 28m Warning Unhealthy pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-bj5xt Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning ProbeError pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-bj5xt Readiness probe error: HTTP probe failed with statuscode: 503... 28m Normal Killing pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-b9xj5 Stopping container istio-proxy 27m Normal Killing pod/payload-processing-e2e-inf-a-39098c-7dfd88f45c-5mh26 Stopping container payload-processing 27m Normal Killing pod/payload-pre-processing-e2e-inf-a-39098c-85df849f9f-7vp5x Stopping container payload-pre-processing 26m Normal Killing pod/e2e-inf-a-39098c-openshift-default-76bff56b5f-b9c7h Stopping container istio-proxy 26m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-39098c-85df849f9f-7vp5x Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Warning ProbeError pod/e2e-inf-a-39098c-openshift-default-76bff56b5f-b9c7h Readiness probe error: Get "http://10.132.0.192:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 26m Warning Unhealthy pod/e2e-inf-a-39098c-openshift-default-76bff56b5f-b9c7h Readiness probe failed: Get "http://10.132.0.192:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 26m Normal Killing pod/payload-pre-processing-e2e-inf-b-4e4683-66d784d5c5-9b5lb Stopping container payload-pre-processing 26m Normal Killing pod/payload-processing-e2e-inf-b-4e4683-7784dc776c-n6ch5 Stopping container payload-processing 26m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-4e4683-66d784d5c5-9b5lb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Normal Killing pod/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g Stopping container istio-proxy 26m Normal Killing pod/payload-processing-e2e-shared-a-w1-895ac8-69555fcc47-6n97d Stopping container payload-processing 26m Normal Killing pod/payload-pre-processing-e2e-shared-a-w1-895ac8-65fcccbb9f-m5hp2 Stopping container payload-pre-processing 25m Normal Killing pod/e2e-shared-a-w1-895ac8-openshift-default-756985bc89-9bq5g Stopping container istio-proxy 25m Warning Unhealthy pod/e2e-shared-a-w1-895ac8-openshift-default-756985bc89-9bq5g Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning ProbeError pod/e2e-shared-a-w1-895ac8-openshift-default-756985bc89-9bq5g Readiness probe error: HTTP probe failed with statuscode: 503... 25m Normal Killing pod/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc Stopping container payload-processing 25m Normal Killing pod/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr Stopping container payload-pre-processing 25m Normal Killing pod/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999 Stopping container istio-proxy 25m Normal Killing pod/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz Stopping container payload-pre-processing 25m Normal Killing pod/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5 Stopping container payload-processing 24m Normal Killing pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-dzdcx Stopping container istio-proxy 24m Warning ProbeError pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-dzdcx Readiness probe error: HTTP probe failed with statuscode: 503... 24m Warning Unhealthy pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-dzdcx Readiness probe failed: HTTP probe failed with statuscode: 503 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-j8c7c Readiness probe failed: Get "http://10.132.0.210:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 18m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-j8c7c Readiness probe error: Get "http://10.132.0.210:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 18m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-gv6hc Readiness probe error: Get "http://10.132.0.209:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-gv6hc Readiness probe failed: Get "http://10.132.0.209:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 16m Normal AddedLabel gateway/e2e-worker-single-main-b72a1f Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-b72a1f 16m Normal ScalingReplicaSet deployment/e2e-worker-single-main-b72a1f-openshift-default Scaled up replica set e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7 from 0 to 1 16m Normal SuccessfulCreate replicaset/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7 Created pod: e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl 16m Normal AddedInterface pod/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl Add eth0 [10.133.0.146/23] from ovn-kubernetes 16m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-b72a1f Scaled up replica set payload-processing-e2e-worker-single-main-b72a1f-56b68d7949 from 0 to 1 16m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bd6b79 Created pod: payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz 16m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bd6b79 Error creating: pods "payload-pre-processing-e2e-worker-single-main-b72a1f-5449bd6b79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f: serviceaccount "payload-processing-e2e-worker-single-main-b72a1f" not found 16m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-b72a1f-56b68d7949 Error creating: pods "payload-processing-e2e-worker-single-main-b72a1f-56b68d7949-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f: serviceaccount "payload-processing-e2e-worker-single-main-b72a1f" not found 16m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-b72a1f-56b68d7949 Created pod: payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv 16m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-b72a1f Scaled up replica set payload-pre-processing-e2e-worker-single-main-b72a1f-5449bd6b79 from 0 to 1 16m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 16m Normal Started pod/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv Container started 16m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz Add eth0 [10.133.0.147/23] from ovn-kubernetes 16m Normal Created pod/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz Container created 16m Normal Created pod/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv Container created 16m Normal Pulled pod/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 16m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv Add eth0 [10.133.0.148/23] from ovn-kubernetes 16m Normal Started pod/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz Container started 15m Normal Created pod/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl Container created 15m Normal Started pod/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl Container started 15m Normal Pulled pod/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 15m Warning ProbeError pod/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl Startup probe error: Get "http://10.133.0.146:15021/healthz/ready": dial tcp 10.133.0.146:15021: connect: connection refused... 15m Warning Unhealthy pod/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl Startup probe failed: Get "http://10.133.0.146:15021/healthz/ready": dial tcp 10.133.0.146:15021: connect: connection refused 15m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-b72a1f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 15m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-b72a1f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 15m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-b72a1f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 15m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-b72a1f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 14m Normal Killing pod/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv Stopping container payload-processing 14m Normal Killing pod/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz Stopping container payload-pre-processing 14m Normal Killing pod/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl Stopping container istio-proxy 14m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6 Readiness probe failed: Get "http://10.133.0.95:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 12m Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6 Readiness probe error: Get "http://10.133.0.95:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 11m Normal AddedLabel gateway/e2e-worker-single-main-a60cf3 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-a60cf3 11m Normal SuccessfulCreate replicaset/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfc Created pod: e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk 11m Normal ScalingReplicaSet deployment/e2e-worker-single-main-a60cf3-openshift-default Scaled up replica set e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfc from 0 to 1 11m Normal AddedInterface pod/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk Add eth0 [10.133.0.151/23] from ovn-kubernetes 11m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-a60cf3 Scaled up replica set payload-processing-e2e-worker-single-main-a60cf3-7c644dd865 from 0 to 1 11m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-a60cf3 Scaled up replica set payload-pre-processing-e2e-worker-single-main-a60cf3-5686bd4fdc from 0 to 1 11m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-a60cf3-5686bd4fdc Created pod: payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz 11m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-a60cf3-5686bd4fdc Error creating: pods "payload-pre-processing-e2e-worker-single-main-a60cf3-5686bd4fdc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3: serviceaccount "payload-processing-e2e-worker-single-main-a60cf3" not found 11m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-a60cf3-7c644dd865 Created pod: payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh 11m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-a60cf3-7c644dd865 Error creating: pods "payload-processing-e2e-worker-single-main-a60cf3-7c644dd865-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3: serviceaccount "payload-processing-e2e-worker-single-main-a60cf3" not found 11m Normal Created pod/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz Container created 11m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz Add eth0 [10.133.0.153/23] from ovn-kubernetes 11m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 11m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh Add eth0 [10.133.0.152/23] from ovn-kubernetes 11m Normal Started pod/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz Container started 11m Normal Started pod/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh Container started 11m Normal Created pod/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh Container created 11m Normal Pulled pod/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 11m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-a60cf3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 11m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-a60cf3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 11m Normal Started pod/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk Container started 11m Normal Created pod/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk Container created 11m Normal Pulled pod/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 10m Warning Unhealthy pod/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk Startup probe failed: Get "http://10.133.0.151:15021/healthz/ready": dial tcp 10.133.0.151:15021: connect: connection refused 10m Warning ProbeError pod/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk Startup probe error: Get "http://10.133.0.151:15021/healthz/ready": dial tcp 10.133.0.151:15021: connect: connection refused... 10m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-a60cf3-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 10m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-a60cf3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 9m37s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz Stopping container payload-pre-processing 9m37s Normal Killing pod/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh Stopping container payload-processing 9m22s Normal Killing pod/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk Stopping container istio-proxy 9m19s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 9m19s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 6m32s Normal SuccessfulCreate replicaset/e2e-ait-5315ece5-openshift-default-6f776d87df Created pod: e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n 6m32s Normal ScalingReplicaSet deployment/e2e-ait-5315ece5-openshift-default Scaled up replica set e2e-ait-5315ece5-openshift-default-6f776d87df from 0 to 1 6m32s Normal AddedLabel gateway/e2e-ait-5315ece5 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-5315ece5 6m31s Normal AddedInterface pod/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n Add eth0 [10.133.0.159/23] from ovn-kubernetes 6m25s Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-5315ece5 Scaled up replica set payload-processing-e2e-ait-5315ece5-778b49bd7f from 0 to 1 6m24s Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-5315ece5-778b49bd7f Created pod: payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh 6m24s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc Created pod: payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq 6m24s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-5315ece5 Scaled up replica set payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc from 0 to 1 6m24s Warning FailedCreate replicaset/payload-processing-e2e-ait-5315ece5-778b49bd7f Error creating: pods "payload-processing-e2e-ait-5315ece5-778b49bd7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-5315ece5: serviceaccount "payload-processing-e2e-ait-5315ece5" not found 6m24s Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc Error creating: pods "payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-5315ece5: serviceaccount "payload-processing-e2e-ait-5315ece5" not found 6m23s Normal Created pod/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh Container created 6m23s Normal Created pod/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq Container created 6m23s Normal AddedInterface pod/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq Add eth0 [10.133.0.160/23] from ovn-kubernetes 6m23s Normal Pulled pod/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 6m23s Normal AddedInterface pod/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh Add eth0 [10.133.0.162/23] from ovn-kubernetes 6m23s Normal Pulled pod/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 6m23s Normal Started pod/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh Container started 6m23s Normal Started pod/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq Container started 6m16s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-5315ece5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 6m16s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-5315ece5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 6m6s Normal Started pod/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n Container started 6m6s Normal Created pod/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n Container created 6m6s Normal Pulled pod/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 6m Warning Unhealthy pod/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n Startup probe failed: Get "http://10.133.0.159:15021/healthz/ready": dial tcp 10.133.0.159:15021: connect: connection refused 6m Warning ProbeError pod/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n Startup probe error: Get "http://10.133.0.159:15021/healthz/ready": dial tcp 10.133.0.159:15021: connect: connection refused... 5m46s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-5315ece5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 5m46s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-5315ece5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 5m45s Normal Killing pod/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq Stopping container payload-pre-processing 5m45s Normal Killing pod/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh Stopping container payload-processing 5m35s Normal Killing pod/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n Stopping container istio-proxy 5m33s Normal SuccessfulCreate replicaset/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949 Created pod: e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7 5m33s Normal ScalingReplicaSet deployment/e2e-authcfg-http-2afef9-openshift-default Scaled up replica set e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949 from 0 to 1 5m33s Normal AddedLabel gateway/e2e-authcfg-http-2afef9 Added label istio.io/rev=openshift-gateway to gateway e2e-authcfg-http-2afef9 5m32s Normal AddedInterface pod/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7 Add eth0 [10.133.0.163/23] from ovn-kubernetes 5m26s Normal ScalingReplicaSet deployment/payload-processing-e2e-authcfg-http-2afef9 Scaled up replica set payload-processing-e2e-authcfg-http-2afef9-7b86cc8865 from 0 to 1 5m25s Warning FailedCreate replicaset/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c Error creating: pods "payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-2afef9: serviceaccount "payload-processing-e2e-authcfg-http-2afef9" not found 5m25s Normal SuccessfulCreate replicaset/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865 Created pod: payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x 5m25s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-authcfg-http-2afef9 Scaled up replica set payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c from 0 to 1 5m25s Warning FailedCreate replicaset/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865 Error creating: pods "payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-2afef9: serviceaccount "payload-processing-e2e-authcfg-http-2afef9" not found 5m25s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c Created pod: payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49 5m24s Normal Pulled pod/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m24s Normal AddedInterface pod/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x Add eth0 [10.133.0.164/23] from ovn-kubernetes 5m24s Normal Started pod/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49 Container started 5m24s Normal Pulled pod/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m24s Normal AddedInterface pod/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49 Add eth0 [10.133.0.165/23] from ovn-kubernetes 5m24s Normal Started pod/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x Container started 5m24s Normal Created pod/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49 Container created 5m24s Normal Created pod/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x Container created 5m5s Normal Pulled pod/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 5m4s Normal Created pod/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7 Container created 5m4s Normal Started pod/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7 Container started 4m59s Warning ProbeError pod/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7 Startup probe error: Get "http://10.133.0.163:15021/healthz/ready": dial tcp 10.133.0.163:15021: connect: connection refused... 4m59s Warning Unhealthy pod/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7 Startup probe failed: Get "http://10.133.0.163:15021/healthz/ready": dial tcp 10.133.0.163:15021: connect: connection refused 4m32s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-2afef9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m32s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-2afef9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 3m31s Normal Killing pod/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x Stopping container payload-processing 3m31s Normal Killing pod/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49 Stopping container payload-pre-processing 3m13s Normal Killing pod/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7 Stopping container istio-proxy 2m50s Warning Unhealthy pod/maas-default-gateway-openshift-default-75fd9f5d47-5j55g Readiness probe failed: Get "http://10.133.0.138:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m50s Warning ProbeError pod/maas-default-gateway-openshift-default-75fd9f5d47-5j55g Readiness probe error: Get "http://10.133.0.138:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)...