=== events in openshift-ingress (2026-11-03T23:38:00Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 46m Normal Scheduled pod/e2e-shared-b-w6-0189ef-openshift-default-cd86cdd47-8h87f Successfully assigned openshift-ingress/e2e-shared-b-w6-0189ef-openshift-default-cd86cdd47-8h87f to ip-10-0-139-173.us-west-2.compute.internal 50m Normal Scheduled pod/payload-processing-e2e-mt-12450a06-5c4cc76cd6-c9zgk Successfully assigned openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-c9zgk to ip-10-0-128-222.us-west-2.compute.internal 60m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-mm95n Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-mm95n to ip-10-0-128-222.us-west-2.compute.internal 25m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-l9zcx Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l9zcx to ip-10-0-139-173.us-west-2.compute.internal 60m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-l2qpx Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-l2qpx to ip-10-0-143-0.us-west-2.compute.internal 50m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-k2jmm Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-k2jmm to ip-10-0-139-173.us-west-2.compute.internal 38m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-jlhrc Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-jlhrc to ip-10-0-139-173.us-west-2.compute.internal 40m Normal Scheduled pod/e2e-mt-12450a06-openshift-default-7576d969f7-zp9gc Successfully assigned openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-zp9gc to ip-10-0-139-173.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc Successfully assigned openshift-ingress/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc to ip-10-0-143-0.us-west-2.compute.internal 66m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-n2sn6 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-n2sn6 to ip-10-0-128-222.us-west-2.compute.internal 40m Normal Scheduled pod/e2e-mt-12450a06-openshift-default-7576d969f7-vrdvc Successfully assigned openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-vrdvc to ip-10-0-128-222.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-ait-bb95c832-openshift-default-997798bd5-x5tlh Successfully assigned openshift-ingress/e2e-ait-bb95c832-openshift-default-997798bd5-x5tlh to ip-10-0-128-222.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm Successfully assigned openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm to ip-10-0-143-0.us-west-2.compute.internal 50m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-n5gnt Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-n5gnt to ip-10-0-139-173.us-west-2.compute.internal 38m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-ndnzf Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-ndnzf to ip-10-0-128-222.us-west-2.compute.internal 40m Normal Scheduled pod/e2e-mt-12450a06-openshift-default-7576d969f7-t67gd Successfully assigned openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t67gd to ip-10-0-128-222.us-west-2.compute.internal 38m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-nr7zh Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-nr7zh to ip-10-0-139-173.us-west-2.compute.internal 66m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-p59kr Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-p59kr to ip-10-0-143-0.us-west-2.compute.internal 40m Normal Scheduled pod/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69 Successfully assigned openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69 to ip-10-0-143-0.us-west-2.compute.internal 66m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-q2jqm Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-q2jqm to ip-10-0-143-0.us-west-2.compute.internal 25m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-qrmft Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-qrmft to ip-10-0-128-222.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-8kw9p Successfully assigned openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-8kw9p to ip-10-0-139-173.us-west-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-dn59n Successfully assigned openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-dn59n to ip-10-0-139-173.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-mt-12450a06-openshift-default-7576d969f7-kh8pc Successfully assigned openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-kh8pc to ip-10-0-128-222.us-west-2.compute.internal 66m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-rlkdl Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-rlkdl to ip-10-0-128-222.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-5qmh2 Successfully assigned openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-5qmh2 to ip-10-0-128-222.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-ait-c082ab23-openshift-default-7f9b898b47-fpps2 Successfully assigned openshift-ingress/e2e-ait-c082ab23-openshift-default-7f9b898b47-fpps2 to ip-10-0-128-222.us-west-2.compute.internal 40m Normal Scheduled pod/e2e-mt-12450a06-openshift-default-7576d969f7-6zght Successfully assigned openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-6zght to ip-10-0-139-173.us-west-2.compute.internal 38m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-std4j Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-std4j to ip-10-0-139-173.us-west-2.compute.internal 50m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-vhkwc Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-vhkwc to ip-10-0-128-222.us-west-2.compute.internal 23m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-wkwkn Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-wkwkn to ip-10-0-128-222.us-west-2.compute.internal 49m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-xmfvp Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-xmfvp to ip-10-0-128-222.us-west-2.compute.internal 75m Normal Scheduled pod/router-default-74d647c79-nsnqv Successfully assigned openshift-ingress/router-default-74d647c79-nsnqv to ip-10-0-139-173.us-west-2.compute.internal 75m Warning FailedScheduling pod/router-default-74d647c79-nsnqv no nodes available to schedule pods 81m Warning FailedScheduling pod/router-default-74d647c79-nsnqv no nodes available to schedule pods 40m Normal Scheduled pod/e2e-mt-12450a06-openshift-default-7576d969f7-5l758 Successfully assigned openshift-ingress/e2e-mt-12450a06-openshift-default-7576d969f7-5l758 to ip-10-0-143-0.us-west-2.compute.internal 66m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-j72ct Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-j72ct to ip-10-0-139-173.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-ff7t5 Successfully assigned openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-ff7t5 to ip-10-0-139-173.us-west-2.compute.internal 38m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-j6q6l Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-j6q6l to ip-10-0-139-173.us-west-2.compute.internal 65m Normal Scheduled pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7 Successfully assigned openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7 to ip-10-0-143-0.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-w8xsc Successfully assigned openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-w8xsc to ip-10-0-128-222.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-vw6ft Successfully assigned openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-vw6ft to ip-10-0-128-222.us-west-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-k5m5d Successfully assigned openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-k5m5d to ip-10-0-128-222.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9 Successfully assigned openshift-ingress/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9 to ip-10-0-143-0.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-gpbv6 Successfully assigned openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-gpbv6 to ip-10-0-139-173.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr Successfully assigned openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr to ip-10-0-143-0.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-b69rq Successfully assigned openshift-ingress/e2e-worker-w3-961054-openshift-default-5d5665c66f-b69rq to ip-10-0-139-173.us-west-2.compute.internal 60m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-ftxcj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-ftxcj to ip-10-0-139-173.us-west-2.compute.internal 65m Normal Scheduled pod/payload-processing-e2e-worker-w6-5e72ad-69d4b7cbb-r4xrh Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-5e72ad-69d4b7cbb-r4xrh to ip-10-0-128-222.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr Successfully assigned openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr to ip-10-0-143-0.us-west-2.compute.internal 65m Normal Scheduled pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b Successfully assigned openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b to ip-10-0-143-0.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r69n5 Successfully assigned openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r69n5 to ip-10-0-139-173.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz Successfully assigned openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz to ip-10-0-143-0.us-west-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-sq629 Successfully assigned openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-sq629 to ip-10-0-139-173.us-west-2.compute.internal 60m Normal Scheduled pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-rj8x7 Successfully assigned openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-rj8x7 to ip-10-0-128-222.us-west-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r7xzt Successfully assigned openshift-ingress/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r7xzt to ip-10-0-128-222.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-qnfhf Successfully assigned openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-qnfhf to ip-10-0-128-222.us-west-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-qkcnq Successfully assigned openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-qkcnq to ip-10-0-128-222.us-west-2.compute.internal 65m Normal Scheduled pod/payload-processing-e2e-worker-w5-6f1190-c8f5948fd-sd9d7 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-6f1190-c8f5948fd-sd9d7 to ip-10-0-128-222.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-k46v6 Successfully assigned openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-k46v6 to ip-10-0-128-222.us-west-2.compute.internal 8m8s Normal Scheduled pod/e2e-ait-cc1c5661-openshift-default-65b64fd645-v2pmz Successfully assigned openshift-ingress/e2e-ait-cc1c5661-openshift-default-65b64fd645-v2pmz to ip-10-0-128-222.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-fwck9 Successfully assigned openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-fwck9 to ip-10-0-139-173.us-west-2.compute.internal 35m Normal Scheduled pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-dwx77 Successfully assigned openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-dwx77 to ip-10-0-128-222.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4 Successfully assigned openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4 to ip-10-0-143-0.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74 Successfully assigned openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74 to ip-10-0-143-0.us-west-2.compute.internal 65m Normal Scheduled pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-b7xdq Successfully assigned openshift-ingress/e2e-worker-w2-ee74cb-openshift-default-9758876f-b7xdq to ip-10-0-139-173.us-west-2.compute.internal 23m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-lch84 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-lch84 to ip-10-0-139-173.us-west-2.compute.internal 49m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-fkwnw Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-fkwnw to ip-10-0-139-173.us-west-2.compute.internal 64m Normal Scheduled pod/payload-processing-e2e-worker-w4-18c03a-85c44654fd-b8t9v Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-18c03a-85c44654fd-b8t9v to ip-10-0-128-222.us-west-2.compute.internal 51m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-xkzbs Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xkzbs to ip-10-0-143-0.us-west-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc Successfully assigned openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc to ip-10-0-143-0.us-west-2.compute.internal 49m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-f64cj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-f64cj to ip-10-0-143-0.us-west-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm Successfully assigned openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm to ip-10-0-143-0.us-west-2.compute.internal 34m Normal Scheduled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6 Successfully assigned openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6 to ip-10-0-143-0.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-jxgb9 Successfully assigned openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-jxgb9 to ip-10-0-139-173.us-west-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-kp9cz Successfully assigned openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-kp9cz to ip-10-0-139-173.us-west-2.compute.internal 64m Normal Scheduled pod/payload-processing-e2e-worker-w3-961054-66867965c8-8zxjh Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-961054-66867965c8-8zxjh to ip-10-0-128-222.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-api-a-0c21c7-openshift-default-8476d54f65-ph5gh Successfully assigned openshift-ingress/e2e-api-a-0c21c7-openshift-default-8476d54f65-ph5gh to ip-10-0-128-222.us-west-2.compute.internal 35m Normal Scheduled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zc5t5 Successfully assigned openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zc5t5 to ip-10-0-128-222.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-wfg5p Successfully assigned openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-wfg5p to ip-10-0-128-222.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vpkm5 Successfully assigned openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vpkm5 to ip-10-0-139-173.us-west-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7 Successfully assigned openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7 to ip-10-0-143-0.us-west-2.compute.internal 65m Normal Scheduled pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-mzhsz Successfully assigned openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-mzhsz to ip-10-0-128-222.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-shared-a-w2-fb4f0a-openshift-default-7b7c76784f-ldhbt Successfully assigned openshift-ingress/e2e-shared-a-w2-fb4f0a-openshift-default-7b7c76784f-ldhbt to ip-10-0-128-222.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-th2cc Successfully assigned openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-th2cc to ip-10-0-128-222.us-west-2.compute.internal 65m Normal Scheduled pod/payload-processing-e2e-worker-w2-ee74cb-6479cc6f4b-5qpss Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-ee74cb-6479cc6f4b-5qpss to ip-10-0-128-222.us-west-2.compute.internal 61m Normal Scheduled pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-n7vdt Successfully assigned openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-n7vdt to ip-10-0-139-173.us-west-2.compute.internal 35m Normal Scheduled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-t6mjs Successfully assigned openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-t6mjs to ip-10-0-139-173.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-pfkck Successfully assigned openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-pfkck to ip-10-0-128-222.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl Successfully assigned openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl to ip-10-0-143-0.us-west-2.compute.internal 60m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-dvtvl Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-dvtvl to ip-10-0-128-222.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-sxzfr Successfully assigned openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-sxzfr to ip-10-0-128-222.us-west-2.compute.internal 47m Normal Scheduled pod/e2e-api-a-2627c5-openshift-default-6c84fbdb55-5g5zm Successfully assigned openshift-ingress/e2e-api-a-2627c5-openshift-default-6c84fbdb55-5g5zm to ip-10-0-139-173.us-west-2.compute.internal 26m Normal Scheduled pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr Successfully assigned openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr to ip-10-0-143-0.us-west-2.compute.internal 64m Normal Scheduled pod/payload-processing-e2e-worker-w1-545b5d-6ff8c4558-75gnb Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-545b5d-6ff8c4558-75gnb to ip-10-0-128-222.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7 Successfully assigned openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7 to ip-10-0-143-0.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-bpdgg Successfully assigned openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-bpdgg to ip-10-0-139-173.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-migrate-027a6fbe-gw-openshift-default-ccd69c4b5-stjc4 Successfully assigned openshift-ingress/e2e-migrate-027a6fbe-gw-openshift-default-ccd69c4b5-stjc4 to ip-10-0-139-173.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-72tpd Successfully assigned openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-72tpd to ip-10-0-139-173.us-west-2.compute.internal 60m Normal Scheduled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5z5wk Successfully assigned openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5z5wk to ip-10-0-139-173.us-west-2.compute.internal 65m Normal Scheduled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5ctxt Successfully assigned openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5ctxt to ip-10-0-128-222.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc Successfully assigned openshift-ingress/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc to ip-10-0-143-0.us-west-2.compute.internal 64m Normal Scheduled pod/payload-processing-e2e-worker-w0-68fbd1-6d979f99cb-65n5z Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-68fbd1-6d979f99cb-65n5z to ip-10-0-128-222.us-west-2.compute.internal 12m Normal Scheduled pod/payload-processing-e2e-worker-single-main-e433fd-7c6897ff52rsr5 Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-e433fd-7c6897ff52rsr5 to ip-10-0-128-222.us-west-2.compute.internal 17m Normal Scheduled pod/payload-processing-e2e-worker-single-main-1376dc-6f575447fb7tb8 Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-1376dc-6f575447fb7tb8 to ip-10-0-128-222.us-west-2.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-shared-b-w6-0189ef-64d6f8fb8-r5l4j Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w6-0189ef-64d6f8fb8-r5l4j to ip-10-0-128-222.us-west-2.compute.internal 50m Normal Scheduled pod/payload-processing-e2e-shared-b-w4-a03fee-5d9bb89966-kfsbz Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-a03fee-5d9bb89966-kfsbz to ip-10-0-128-222.us-west-2.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-65913e-75fdbc7fd8-vgd45 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-65913e-75fdbc7fd8-vgd45 to ip-10-0-128-222.us-west-2.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-shared-a-w6-6a3169-866cf6cb6d-t7spd Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w6-6a3169-866cf6cb6d-t7spd to ip-10-0-128-222.us-west-2.compute.internal 52m Normal Scheduled pod/payload-processing-e2e-shared-a-w4-65cac0-7bbdd5876-c484q Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-65cac0-7bbdd5876-c484q to ip-10-0-128-222.us-west-2.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-fb4f0a-6fdc7bbbfd-m9229 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-fb4f0a-6fdc7bbbfd-m9229 to ip-10-0-128-222.us-west-2.compute.internal 39m Normal Scheduled pod/e2e-api-a-31ad7b-openshift-default-65dcf6d7f7-d6fz2 Successfully assigned openshift-ingress/e2e-api-a-31ad7b-openshift-default-65dcf6d7f7-d6fz2 to ip-10-0-128-222.us-west-2.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-mt-d18a5b05-95bc9bb4-9tn94 Successfully assigned openshift-ingress/payload-processing-e2e-mt-d18a5b05-95bc9bb4-9tn94 to ip-10-0-128-222.us-west-2.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw Successfully assigned openshift-ingress/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw to ip-10-0-143-0.us-west-2.compute.internal 49m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-ddv45 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-ddv45 to ip-10-0-139-173.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55 Successfully assigned openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55 to ip-10-0-143-0.us-west-2.compute.internal 34m Normal Scheduled pod/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v Successfully assigned openshift-ingress/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v to ip-10-0-143-0.us-west-2.compute.internal 72m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-d5brv Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-d5brv to ip-10-0-139-173.us-west-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd Successfully assigned openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd to ip-10-0-143-0.us-west-2.compute.internal 72m Normal Scheduled pod/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5 Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5 to ip-10-0-143-0.us-west-2.compute.internal 65m Normal Scheduled pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm Successfully assigned openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm to ip-10-0-143-0.us-west-2.compute.internal 44m Normal Scheduled pod/e2e-inf-b-9674f5-openshift-default-7b45c9f895-w26vx Successfully assigned openshift-ingress/e2e-inf-b-9674f5-openshift-default-7b45c9f895-w26vx to ip-10-0-139-173.us-west-2.compute.internal 53m Normal Scheduled pod/payload-processing-e2e-migrate-027a6fbe-d6957d7c4-rprgn Successfully assigned openshift-ingress/payload-processing-e2e-migrate-027a6fbe-d6957d7c4-rprgn to ip-10-0-128-222.us-west-2.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-inf-b-ac6633-64d9fbd554-vtdtd Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-ac6633-64d9fbd554-vtdtd to ip-10-0-128-222.us-west-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl to ip-10-0-143-0.us-west-2.compute.internal 38m Normal Scheduled pod/payload-processing-e2e-inf-b-2cea18-6fbbf7d74b-wknm7 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-2cea18-6fbbf7d74b-wknm7 to ip-10-0-128-222.us-west-2.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-inf-a-9b4383-5b9cf487f8-qlxxp Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-9b4383-5b9cf487f8-qlxxp to ip-10-0-128-222.us-west-2.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-inf-a-2420f6-b68c89c6-shx8b Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-2420f6-b68c89c6-shx8b to ip-10-0-128-222.us-west-2.compute.internal 46m Normal Scheduled pod/e2e-api-a-922795-openshift-default-59b46784df-6bc47 Successfully assigned openshift-ingress/e2e-api-a-922795-openshift-default-59b46784df-6bc47 to ip-10-0-139-173.us-west-2.compute.internal 36m Normal Scheduled pod/payload-processing-e2e-inf-a-0bf80c-cd68887bb-m4r47 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-0bf80c-cd68887bb-m4r47 to ip-10-0-128-222.us-west-2.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x Successfully assigned openshift-ingress/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x to ip-10-0-143-0.us-west-2.compute.internal 6m57s Normal Scheduled pod/payload-processing-e2e-authcfg-http-05b91b-66bb9db55d-gcw9p Successfully assigned openshift-ingress/payload-processing-e2e-authcfg-http-05b91b-66bb9db55d-gcw9p to ip-10-0-128-222.us-west-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-api-b-d6296b-95dc94cbb-k2wv7 Successfully assigned openshift-ingress/payload-processing-e2e-api-b-d6296b-95dc94cbb-k2wv7 to ip-10-0-128-222.us-west-2.compute.internal 36m Normal Scheduled pod/payload-processing-e2e-api-b-656ee6-7d7cf8d498-9blpz Successfully assigned openshift-ingress/payload-processing-e2e-api-b-656ee6-7d7cf8d498-9blpz to ip-10-0-128-222.us-west-2.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-api-b-2769cb-85cd775f46-hxl4x Successfully assigned openshift-ingress/payload-processing-e2e-api-b-2769cb-85cd775f46-hxl4x to ip-10-0-139-173.us-west-2.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-api-a-922795-56d4c4f7db-hjv42 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-922795-56d4c4f7db-hjv42 to ip-10-0-128-222.us-west-2.compute.internal 39m Normal Scheduled pod/payload-processing-e2e-api-a-31ad7b-74bb7bcdf8-fdhjn Successfully assigned openshift-ingress/payload-processing-e2e-api-a-31ad7b-74bb7bcdf8-fdhjn to ip-10-0-139-173.us-west-2.compute.internal 47m Normal Scheduled pod/payload-processing-e2e-api-a-2627c5-56b96dd67b-vqv5f Successfully assigned openshift-ingress/payload-processing-e2e-api-a-2627c5-56b96dd67b-vqv5f to ip-10-0-128-222.us-west-2.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-api-a-0c21c7-884dbcbfd-vcsmv Successfully assigned openshift-ingress/payload-processing-e2e-api-a-0c21c7-884dbcbfd-vcsmv to ip-10-0-128-222.us-west-2.compute.internal 8m Normal Scheduled pod/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv Successfully assigned openshift-ingress/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv to ip-10-0-143-0.us-west-2.compute.internal 53m Normal Scheduled pod/payload-processing-e2e-ait-c6905007-65d4cdd474-s4nr2 Successfully assigned openshift-ingress/payload-processing-e2e-ait-c6905007-65d4cdd474-s4nr2 to ip-10-0-128-222.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-p8vxh Successfully assigned openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-p8vxh to ip-10-0-128-222.us-west-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-nnsbl Successfully assigned openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-nnsbl to ip-10-0-128-222.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-mvhjt Successfully assigned openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-mvhjt to ip-10-0-128-222.us-west-2.compute.internal 53m Normal Scheduled pod/payload-processing-e2e-ait-c082ab23-876b49bfb-6f94j Successfully assigned openshift-ingress/payload-processing-e2e-ait-c082ab23-876b49bfb-6f94j to ip-10-0-128-222.us-west-2.compute.internal 45m Normal Scheduled pod/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q Successfully assigned openshift-ingress/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q to ip-10-0-143-0.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-lk9lz Successfully assigned openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-lk9lz to ip-10-0-139-173.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-krp2k Successfully assigned openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-krp2k to ip-10-0-139-173.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-hjqrh Successfully assigned openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-hjqrh to ip-10-0-139-173.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t Successfully assigned openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t to ip-10-0-143-0.us-west-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-q27rd Successfully assigned openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-q27rd to ip-10-0-139-173.us-west-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-c6fst Successfully assigned openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-c6fst to ip-10-0-139-173.us-west-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-8t4p6 Successfully assigned openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-8t4p6 to ip-10-0-128-222.us-west-2.compute.internal 53m Normal Scheduled pod/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h Successfully assigned openshift-ingress/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h to ip-10-0-143-0.us-west-2.compute.internal 66m Normal Scheduled pod/payload-pre-processing-7b98bb678c-wlsbd Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-wlsbd to ip-10-0-128-222.us-west-2.compute.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q to ip-10-0-143-0.us-west-2.compute.internal 38m Normal Scheduled pod/e2e-inf-b-2cea18-openshift-default-794d55d96f-rmqhf Successfully assigned openshift-ingress/e2e-inf-b-2cea18-openshift-default-794d55d96f-rmqhf to ip-10-0-139-173.us-west-2.compute.internal 25m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-chpk6 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-chpk6 to ip-10-0-143-0.us-west-2.compute.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-ait-c082ab23-5df95f8c-dvxqg Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-c082ab23-5df95f8c-dvxqg to ip-10-0-128-222.us-west-2.compute.internal 65m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-8mmql Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-8mmql to ip-10-0-128-222.us-west-2.compute.internal 36m Normal Scheduled pod/e2e-api-b-656ee6-openshift-default-7bc94fbdf7-lf2vj Successfully assigned openshift-ingress/e2e-api-b-656ee6-openshift-default-7bc94fbdf7-lf2vj to ip-10-0-139-173.us-west-2.compute.internal 25m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-79h22 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-79h22 to ip-10-0-128-222.us-west-2.compute.internal 66m Normal Scheduled pod/payload-processing-7bb9489676-6xcbp Successfully assigned openshift-ingress/payload-processing-7bb9489676-6xcbp to ip-10-0-128-222.us-west-2.compute.internal 25m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-66p28 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-66p28 to ip-10-0-143-0.us-west-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p Successfully assigned openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p to ip-10-0-143-0.us-west-2.compute.internal 65m Normal Scheduled pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d Successfully assigned openshift-ingress/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d to ip-10-0-143-0.us-west-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-x5fmj Successfully assigned openshift-ingress/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-x5fmj to ip-10-0-139-173.us-west-2.compute.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-ait-c6905007-5d7cf6b467-8hdjj Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-c6905007-5d7cf6b467-8hdjj to ip-10-0-128-222.us-west-2.compute.internal 8m Normal Scheduled pod/payload-pre-processing-e2e-ait-cc1c5661-585757c4f9-hqp2b Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-cc1c5661-585757c4f9-hqp2b to ip-10-0-128-222.us-west-2.compute.internal 12m Normal Scheduled pod/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l Successfully assigned openshift-ingress/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l to ip-10-0-143-0.us-west-2.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-api-a-0c21c7-dd7b5dd55-qzcqr Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-0c21c7-dd7b5dd55-qzcqr to ip-10-0-128-222.us-west-2.compute.internal 65m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-5e72ad-65d5d94fc9-6r57s Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-5e72ad-65d5d94fc9-6r57s to ip-10-0-128-222.us-west-2.compute.internal 17m Normal Scheduled pod/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g Successfully assigned openshift-ingress/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g to ip-10-0-143-0.us-west-2.compute.internal 72m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-tbw6j Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-tbw6j to ip-10-0-128-222.us-west-2.compute.internal 41m Normal Scheduled pod/e2e-inf-a-9b4383-openshift-default-64f46d78c7-lblz9 Successfully assigned openshift-ingress/e2e-inf-a-9b4383-openshift-default-64f46d78c7-lblz9 to ip-10-0-128-222.us-west-2.compute.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-api-a-2627c5-86d7c64d59-2l69t Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-2627c5-86d7c64d59-2l69t to ip-10-0-128-222.us-west-2.compute.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-api-a-31ad7b-5c4dcf55-2fmdz Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-31ad7b-5c4dcf55-2fmdz to ip-10-0-139-173.us-west-2.compute.internal 44m Normal Scheduled pod/e2e-api-b-d6296b-openshift-default-7f98f9c4d5-p9g4x Successfully assigned openshift-ingress/e2e-api-b-d6296b-openshift-default-7f98f9c4d5-p9g4x to ip-10-0-139-173.us-west-2.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-api-a-922795-559bb85d47-l8qpq Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-922795-559bb85d47-l8qpq to ip-10-0-128-222.us-west-2.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-api-b-2769cb-7668b55d7-b9vqz Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-2769cb-7668b55d7-b9vqz to ip-10-0-139-173.us-west-2.compute.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-api-b-656ee6-6f576464c-vsgw4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-656ee6-6f576464c-vsgw4 to ip-10-0-128-222.us-west-2.compute.internal 65m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-6f1190-7568b9dd79-jvzsd Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-6f1190-7568b9dd79-jvzsd to ip-10-0-139-173.us-west-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-api-b-d6296b-b54fff875-sktff Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-d6296b-b54fff875-sktff to ip-10-0-128-222.us-west-2.compute.internal 6m58s Normal Scheduled pod/payload-pre-processing-e2e-authcfg-http-05b91b-559cc4c5bc-pmj9m Successfully assigned openshift-ingress/payload-pre-processing-e2e-authcfg-http-05b91b-559cc4c5bc-pmj9m to ip-10-0-128-222.us-west-2.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-derive-e7a91ca4-65b8c9ccf-h87z2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-e7a91ca4-65b8c9ccf-h87z2 to ip-10-0-128-222.us-west-2.compute.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-0bf80c-5dc5c7cfc9-cfdqp Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-0bf80c-5dc5c7cfc9-cfdqp to ip-10-0-128-222.us-west-2.compute.internal 69m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-8kwp6 Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8kwp6 to ip-10-0-128-222.us-west-2.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-2420f6-cb5bf5df5-9td4b Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-2420f6-cb5bf5df5-9td4b to ip-10-0-128-222.us-west-2.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-9b4383-7c7d96654c-wj9bn Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-9b4383-7c7d96654c-wj9bn to ip-10-0-128-222.us-west-2.compute.internal 64m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-18c03a-d7d874c9-bn495 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-18c03a-d7d874c9-bn495 to ip-10-0-139-173.us-west-2.compute.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-2cea18-5bcdcd4dff-bczwf Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-2cea18-5bcdcd4dff-bczwf to ip-10-0-128-222.us-west-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx to ip-10-0-143-0.us-west-2.compute.internal 7m6s Normal Scheduled pod/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp Successfully assigned openshift-ingress/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp to ip-10-0-143-0.us-west-2.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-ac6633-7bf6856fbc-5nf4g Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-ac6633-7bf6856fbc-5nf4g to ip-10-0-128-222.us-west-2.compute.internal 47m Normal Scheduled pod/e2e-inf-a-2420f6-openshift-default-75c86ff8ff-ltdfl Successfully assigned openshift-ingress/e2e-inf-a-2420f6-openshift-default-75c86ff8ff-ltdfl to ip-10-0-139-173.us-west-2.compute.internal 54m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-hmb27 Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-hmb27 to ip-10-0-139-173.us-west-2.compute.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-migrate-027a6fbe-658459cfb7-qg795 Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-027a6fbe-658459cfb7-qg795 to ip-10-0-128-222.us-west-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-2qsk9 Successfully assigned openshift-ingress/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-2qsk9 to ip-10-0-128-222.us-west-2.compute.internal 70m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4 Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4 to ip-10-0-143-0.us-west-2.compute.internal 64m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr to ip-10-0-143-0.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-shared-a-w6-6a3169-openshift-default-58d9474d87-gztml Successfully assigned openshift-ingress/e2e-shared-a-w6-6a3169-openshift-default-58d9474d87-gztml to ip-10-0-128-222.us-west-2.compute.internal 60m Normal Scheduled pod/maas-default-gateway-openshift-default-59f957fdd7-4vg6x Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-59f957fdd7-4vg6x to ip-10-0-139-173.us-west-2.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n to ip-10-0-143-0.us-west-2.compute.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-5tt6k Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-5tt6k to ip-10-0-128-222.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-shared-b-w4-a03fee-openshift-default-5f8f5cbff7-pzmcj Successfully assigned openshift-ingress/e2e-shared-b-w4-a03fee-openshift-default-5f8f5cbff7-pzmcj to ip-10-0-128-222.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-derive-e7a91ca4-openshift-default-59b8db5485-7fm8v Successfully assigned openshift-ingress/e2e-derive-e7a91ca4-openshift-default-59b8db5485-7fm8v to ip-10-0-128-222.us-west-2.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-mt-d18a5b05-6bc8f78cd7-xbnbd Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-d18a5b05-6bc8f78cd7-xbnbd to ip-10-0-128-222.us-west-2.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-fb4f0a-7f6d59c45f-gpbjr Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fb4f0a-7f6d59c45f-gpbjr to ip-10-0-128-222.us-west-2.compute.internal 65m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-ee74cb-5786ddf6ff-lb7f5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-ee74cb-5786ddf6ff-lb7f5 to ip-10-0-128-222.us-west-2.compute.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w4-65cac0-85d8c6fb67-ln66p Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-65cac0-85d8c6fb67-ln66p to ip-10-0-128-222.us-west-2.compute.internal 70m Normal Scheduled pod/kube-auth-proxy-84cff947f5-5xh27 Successfully assigned openshift-ingress/kube-auth-proxy-84cff947f5-5xh27 to ip-10-0-143-0.us-west-2.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w6-6a3169-67b8ff4bb7-k6wb9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w6-6a3169-67b8ff4bb7-k6wb9 to ip-10-0-128-222.us-west-2.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-65913e-568db8dd5f-msdbs Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-65913e-568db8dd5f-msdbs to ip-10-0-128-222.us-west-2.compute.internal 30m Normal Scheduled pod/e2e-inf-a-0bf80c-openshift-default-6566586789-zwtht Successfully assigned openshift-ingress/e2e-inf-a-0bf80c-openshift-default-6566586789-zwtht to ip-10-0-139-173.us-west-2.compute.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w4-a03fee-5595ccdb5f-xgxjk Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-a03fee-5595ccdb5f-xgxjk to ip-10-0-128-222.us-west-2.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w6-0189ef-676855966c-2p5wb Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0189ef-676855966c-2p5wb to ip-10-0-128-222.us-west-2.compute.internal 64m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-545b5d-5c89cdd65f-w2cnc Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-545b5d-5c89cdd65f-w2cnc to ip-10-0-128-222.us-west-2.compute.internal 70m Normal Scheduled pod/kube-auth-proxy-84cff947f5-6bglq Successfully assigned openshift-ingress/kube-auth-proxy-84cff947f5-6bglq to ip-10-0-128-222.us-west-2.compute.internal 17m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-1376dc-5c8ddlnpd6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-1376dc-5c8ddlnpd6 to ip-10-0-128-222.us-west-2.compute.internal 12m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-e433fd-5df9f8pxkd Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-e433fd-5df9f8pxkd to ip-10-0-128-222.us-west-2.compute.internal 36m Normal Scheduled pod/e2e-inf-a-0bf80c-openshift-default-6566586789-wsnl8 Successfully assigned openshift-ingress/e2e-inf-a-0bf80c-openshift-default-6566586789-wsnl8 to ip-10-0-128-222.us-west-2.compute.internal 64m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-68fbd1-76d7f5c887-p9lsw Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-68fbd1-76d7f5c887-p9lsw to ip-10-0-128-222.us-west-2.compute.internal 46m Normal Scheduled pod/e2e-shared-b-w2-65913e-openshift-default-65986b75f9-s5cnk Successfully assigned openshift-ingress/e2e-shared-b-w2-65913e-openshift-default-65986b75f9-s5cnk to ip-10-0-139-173.us-west-2.compute.internal 81m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 81m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 81m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-74d647c79 from 0 to 1 81m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 81m Normal SuccessfulCreate replicaset/router-default-74d647c79 Created pod: router-default-74d647c79-nsnqv 75m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 75m Warning FailedMount pod/router-default-74d647c79-nsnqv MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 75m Warning FailedMount pod/router-default-74d647c79-nsnqv MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 75m Warning FailedMount pod/router-default-74d647c79-nsnqv MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 75m Warning FailedMount pod/router-default-74d647c79-nsnqv MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 75m Normal Pulled pod/router-default-74d647c79-nsnqv Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 75m Normal Started pod/router-default-74d647c79-nsnqv Container started 75m Normal Created pod/router-default-74d647c79-nsnqv Container created 75m Normal AddedInterface pod/router-default-74d647c79-nsnqv Add eth0 [10.132.0.13/23] from ovn-kubernetes 72m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 72m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-tbw6j 72m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-tbw6j Add eth0 [10.133.0.14/23] from ovn-kubernetes 72m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-tbw6j Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 72m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 72m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-tbw6j Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.813s (4.813s including waiting). Image size: 185476477 bytes. 72m Normal Created pod/istiod-openshift-gateway-7876bfcb74-tbw6j Container created 72m Normal Started pod/istiod-openshift-gateway-7876bfcb74-tbw6j Container started 72m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-tbw6j Readiness probe failed: HTTP probe failed with statuscode: 503 72m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-tbw6j Readiness probe error: HTTP probe failed with statuscode: 503... 72m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-59f957fdd7 from 0 to 1 72m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-59f957fdd7 Created pod: maas-default-gateway-openshift-default-59f957fdd7-d5brv 72m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-d5brv Add eth0 [10.132.0.32/23] from ovn-kubernetes 72m Normal SuccessfulCreate replicaset/openshift-ai-inference-openshift-ai-inference-7cd79c685f Created pod: openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5 72m Normal EnsuringLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensuring load balancer 72m Normal ScalingReplicaSet deployment/openshift-ai-inference-openshift-ai-inference Scaled up replica set openshift-ai-inference-openshift-ai-inference-7cd79c685f from 0 to 1 72m Normal Pulling pod/maas-default-gateway-openshift-default-59f957fdd7-d5brv Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 72m Normal Pulling pod/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 72m Normal AddedInterface pod/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5 Add eth0 [10.134.0.27/23] from ovn-kubernetes 71m Normal EnsuredLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensured load balancer 71m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-d5brv Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.138s (4.138s including waiting). Image size: 183555333 bytes. 71m Normal Created pod/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5 Container created 71m Normal Pulled pod/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 7.562s (7.562s including waiting). Image size: 183555333 bytes. 71m Normal Started pod/openshift-ai-inference-openshift-ai-inference-7cd79c685f-2lqt5 Container started 71m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 71m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 71m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 71m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 70m Normal Pulling pod/kube-auth-proxy-84cff947f5-6bglq Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 70m Normal AddedInterface pod/kube-auth-proxy-84cff947f5-5xh27 Add eth0 [10.134.0.33/23] from ovn-kubernetes 70m Normal SuccessfulCreate replicaset/kube-auth-proxy-84cff947f5 Created pod: kube-auth-proxy-84cff947f5-6bglq 70m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-76f56c5945 from 0 to 1 70m Normal SuccessfulCreate replicaset/kube-auth-proxy-84cff947f5 Created pod: kube-auth-proxy-84cff947f5-5xh27 70m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-84cff947f5 from 0 to 2 70m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-76f56c5945 Created pod: data-science-gateway-data-science-gateway-class-76f56c5945rhvx4 70m Normal AddedInterface pod/kube-auth-proxy-84cff947f5-6bglq Add eth0 [10.133.0.16/23] from ovn-kubernetes 70m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4 Add eth0 [10.134.0.34/23] from ovn-kubernetes 70m Normal Created pod/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4 Container created 70m Normal Pulled pod/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 70m Normal Pulling pod/kube-auth-proxy-84cff947f5-5xh27 Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 70m Normal Started pod/data-science-gateway-data-science-gateway-class-76f56c5945rhvx4 Container started 70m Normal Started pod/kube-auth-proxy-84cff947f5-5xh27 Container started 70m Normal Pulled pod/kube-auth-proxy-84cff947f5-5xh27 Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 3.316s (3.316s including waiting). Image size: 188904677 bytes. 70m Normal Started pod/kube-auth-proxy-84cff947f5-6bglq Container started 70m Normal Created pod/kube-auth-proxy-84cff947f5-5xh27 Container created 70m Normal Created pod/kube-auth-proxy-84cff947f5-6bglq Container created 70m Normal Pulled pod/kube-auth-proxy-84cff947f5-6bglq Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 3.447s (3.447s including waiting). Image size: 188904677 bytes. 70m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 70m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 70m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 70m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 70m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 70m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 69m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 69m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-8kwp6 Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 69m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-8kwp6 Container created 69m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-8kwp6 Container started 69m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-8kwp6 Add eth0 [10.133.0.22/23] from ovn-kubernetes 69m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-8kwp6 69m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-8kwp6 Readiness probe error: HTTP probe failed with statuscode: 503... 69m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-8kwp6 Readiness probe failed: HTTP probe failed with statuscode: 503 69m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 69m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-tbw6j 69m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-tbw6j Stopping container discovery 68m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 68m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 68m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-d5brv Container started 68m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-d5brv Container created 68m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-d5brv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 68m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 68m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 68m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-d5brv Startup probe error: Get "http://10.132.0.32:15021/healthz/ready": dial tcp 10.132.0.32:15021: connect: connection refused... 68m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-d5brv Startup probe failed: Get "http://10.132.0.32:15021/healthz/ready": dial tcp 10.132.0.32:15021: connect: connection refused 68m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 68m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-59f957fdd7 from 1 to 4 66m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 66m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-p59kr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-q2jqm Container created 66m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-q2jqm Container started 66m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-p59kr Add eth0 [10.134.0.43/23] from ovn-kubernetes 66m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-p59kr Container created 66m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-n2sn6 Add eth0 [10.133.0.32/23] from ovn-kubernetes 66m Normal Pulling pod/maas-default-gateway-openshift-default-59f957fdd7-n2sn6 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 66m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-p59kr Container started 66m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-59f957fdd7 Created pod: maas-default-gateway-openshift-default-59f957fdd7-q2jqm 66m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-59f957fdd7 Created pod: maas-default-gateway-openshift-default-59f957fdd7-p59kr 66m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-59f957fdd7 Created pod: maas-default-gateway-openshift-default-59f957fdd7-n2sn6 66m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-q2jqm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-q2jqm Add eth0 [10.134.0.42/23] from ovn-kubernetes 66m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-n2sn6 Container created 66m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-n2sn6 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.619s (4.619s including waiting). Image size: 183555333 bytes. 66m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-n2sn6 Container started 66m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-p59kr Startup probe error: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused... 66m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-p59kr Startup probe failed: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused 66m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-q2jqm Startup probe failed: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused 66m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-q2jqm Startup probe error: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused... 66m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-n2sn6 Startup probe error: Get "http://10.133.0.32:15021/healthz/ready": dial tcp 10.133.0.32:15021: connect: connection refused... 66m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-n2sn6 Startup probe failed: Get "http://10.133.0.32:15021/healthz/ready": dial tcp 10.133.0.32:15021: connect: connection refused 66m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-j72ct Container created 66m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-59f957fdd7 Created pod: maas-default-gateway-openshift-default-59f957fdd7-j72ct 66m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-rlkdl Add eth0 [10.133.0.35/23] from ovn-kubernetes 66m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-59f957fdd7 Created pod: maas-default-gateway-openshift-default-59f957fdd7-rlkdl 66m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-j72ct Container started 66m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-j72ct Add eth0 [10.132.0.49/23] from ovn-kubernetes 66m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-j72ct Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-rlkdl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-59f957fdd7 from 4 to 6 66m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-rlkdl Container created 66m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-rlkdl Container started 66m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-wlsbd 66m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 66m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-7bb9489676 from 0 to 1 66m Warning FailedCreate replicaset/payload-processing-7bb9489676 Error creating: pods "payload-processing-7bb9489676-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 66m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 66m Normal AddedInterface pod/payload-processing-7bb9489676-6xcbp Add eth0 [10.133.0.37/23] from ovn-kubernetes 66m Normal SuccessfulCreate replicaset/payload-processing-7bb9489676 Created pod: payload-processing-7bb9489676-6xcbp 66m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-wlsbd Add eth0 [10.133.0.36/23] from ovn-kubernetes 66m Normal Pulling pod/payload-pre-processing-7b98bb678c-wlsbd Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 66m Normal Pulling pod/payload-processing-7bb9489676-6xcbp Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 66m Normal Started pod/payload-processing-7bb9489676-6xcbp Container started 66m Normal Created pod/payload-processing-7bb9489676-6xcbp Container created 66m Normal Pulled pod/payload-pre-processing-7b98bb678c-wlsbd Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 4.144s (4.144s including waiting). Image size: 125353963 bytes. 66m Normal Created pod/payload-pre-processing-7b98bb678c-wlsbd Container created 66m Normal Pulled pod/payload-processing-7bb9489676-6xcbp Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.698s (3.698s including waiting). Image size: 125353963 bytes. 66m Normal Started pod/payload-pre-processing-7b98bb678c-wlsbd Container started 66m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-rlkdl Startup probe error: Get "http://10.133.0.35:15021/healthz/ready": dial tcp 10.133.0.35:15021: connect: connection refused... 66m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-j72ct Startup probe failed: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused 66m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-j72ct Startup probe error: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused... 66m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-rlkdl Startup probe failed: Get "http://10.133.0.35:15021/healthz/ready": dial tcp 10.133.0.35:15021: connect: connection refused 66m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-rlkdl Startup probe failed: Get "http://10.133.0.35:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 66m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-rlkdl Startup probe error: Get "http://10.133.0.35:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 66m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-8mmql Add eth0 [10.133.0.39/23] from ovn-kubernetes 66m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-59f957fdd7 from 6 to 7 66m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-59f957fdd7 Created pod: maas-default-gateway-openshift-default-59f957fdd7-8mmql 65m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-8mmql Container started 65m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-8mmql Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-8mmql Container created 65m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-8mmql Startup probe error: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused... 65m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-8mmql Startup probe failed: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused 65m Normal ScalingReplicaSet deployment/e2e-worker-w3-961054-openshift-default Scaled up replica set e2e-worker-w3-961054-openshift-default-5d5665c66f from 0 to 1 65m Normal AddedLabel gateway/e2e-worker-w2-ee74cb Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-ee74cb 65m Normal AddedLabel gateway/e2e-worker-w0-68fbd1 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-68fbd1 65m Normal SuccessfulCreate replicaset/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 Created pod: e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-mzhsz 65m Normal ScalingReplicaSet deployment/e2e-worker-w5-6f1190-openshift-default Scaled up replica set e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 from 0 to 1 65m Normal AddedLabel gateway/e2e-worker-w5-6f1190 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-6f1190 65m Normal AddedLabel gateway/e2e-worker-w1-545b5d Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-545b5d 65m Normal AddedLabel gateway/e2e-worker-w6-5e72ad Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-5e72ad 65m Normal AddedLabel gateway/e2e-worker-w3-961054 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-961054 65m Normal SuccessfulCreate replicaset/e2e-worker-w3-961054-openshift-default-5d5665c66f Created pod: e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7 65m Normal ScalingReplicaSet deployment/e2e-worker-w1-545b5d-openshift-default Scaled up replica set e2e-worker-w1-545b5d-openshift-default-5c8c66f65c from 0 to 1 65m Normal SuccessfulCreate replicaset/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 Created pod: e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d 65m Normal SuccessfulCreate replicaset/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c Created pod: e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5ctxt 65m Normal SuccessfulCreate replicaset/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c Created pod: e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm 65m Normal ScalingReplicaSet deployment/e2e-worker-w6-5e72ad-openshift-default Scaled up replica set e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c from 0 to 1 65m Normal ScalingReplicaSet deployment/e2e-worker-w0-68fbd1-openshift-default Scaled up replica set e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 from 0 to 1 65m Normal AddedInterface pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-mzhsz Add eth0 [10.133.0.40/23] from ovn-kubernetes 65m Normal AddedInterface pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d Add eth0 [10.134.0.44/23] from ovn-kubernetes 65m Normal ScalingReplicaSet deployment/e2e-worker-w2-ee74cb-openshift-default Scaled up replica set e2e-worker-w2-ee74cb-openshift-default-9758876f from 0 to 1 65m Normal AddedInterface pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7 Add eth0 [10.134.0.45/23] from ovn-kubernetes 65m Normal AddedInterface pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm Add eth0 [10.134.0.46/23] from ovn-kubernetes 65m Normal AddedInterface pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-b7xdq Add eth0 [10.132.0.50/23] from ovn-kubernetes 65m Normal SuccessfulCreate replicaset/e2e-worker-w2-ee74cb-openshift-default-9758876f Created pod: e2e-worker-w2-ee74cb-openshift-default-9758876f-b7xdq 65m Normal AddedInterface pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5ctxt Add eth0 [10.133.0.41/23] from ovn-kubernetes 65m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-5e72ad Scaled up replica set payload-processing-e2e-worker-w6-5e72ad-69d4b7cbb from 0 to 1 65m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-5e72ad Scaled up replica set payload-pre-processing-e2e-worker-w6-5e72ad-65d5d94fc9 from 0 to 1 65m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-5e72ad-65d5d94fc9 Error creating: pods "payload-pre-processing-e2e-worker-w6-5e72ad-65d5d94fc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-5e72ad: serviceaccount "payload-processing-e2e-worker-w6-5e72ad" not found 65m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-5e72ad-69d4b7cbb Created pod: payload-processing-e2e-worker-w6-5e72ad-69d4b7cbb-r4xrh 65m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-5e72ad-65d5d94fc9 Created pod: payload-pre-processing-e2e-worker-w6-5e72ad-65d5d94fc9-6r57s 65m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-5e72ad-69d4b7cbb Error creating: pods "payload-processing-e2e-worker-w6-5e72ad-69d4b7cbb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-5e72ad: serviceaccount "payload-processing-e2e-worker-w6-5e72ad" not found 65m Normal Created pod/payload-processing-e2e-worker-w6-5e72ad-69d4b7cbb-r4xrh Container created 65m Normal Started pod/payload-processing-e2e-worker-w6-5e72ad-69d4b7cbb-r4xrh Container started 65m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-5e72ad-65d5d94fc9-6r57s Add eth0 [10.133.0.42/23] from ovn-kubernetes 65m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-5e72ad-65d5d94fc9-6r57s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 65m Normal ScalingReplicaSet deployment/e2e-worker-w4-18c03a-openshift-default Scaled up replica set e2e-worker-w4-18c03a-openshift-default-589bbc88df from 0 to 1 65m Normal SuccessfulCreate replicaset/e2e-worker-w4-18c03a-openshift-default-589bbc88df Created pod: e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b 65m Normal AddedLabel gateway/e2e-worker-w4-18c03a Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-18c03a 65m Normal Started pod/payload-pre-processing-e2e-worker-w6-5e72ad-65d5d94fc9-6r57s Container started 65m Normal Created pod/payload-pre-processing-e2e-worker-w6-5e72ad-65d5d94fc9-6r57s Container created 65m Normal Pulled pod/payload-processing-e2e-worker-w6-5e72ad-69d4b7cbb-r4xrh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 65m Normal AddedInterface pod/payload-processing-e2e-worker-w6-5e72ad-69d4b7cbb-r4xrh Add eth0 [10.133.0.43/23] from ovn-kubernetes 65m Normal AddedInterface pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b Add eth0 [10.134.0.47/23] from ovn-kubernetes 65m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-6f1190 Scaled up replica set payload-pre-processing-e2e-worker-w5-6f1190-7568b9dd79 from 0 to 1 65m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-6f1190-7568b9dd79 Error creating: pods "payload-pre-processing-e2e-worker-w5-6f1190-7568b9dd79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-6f1190: serviceaccount "payload-processing-e2e-worker-w5-6f1190" not found 65m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-6f1190-c8f5948fd Error creating: pods "payload-processing-e2e-worker-w5-6f1190-c8f5948fd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-6f1190: serviceaccount "payload-processing-e2e-worker-w5-6f1190" not found 65m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-6f1190-c8f5948fd Created pod: payload-processing-e2e-worker-w5-6f1190-c8f5948fd-sd9d7 65m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-6f1190 Scaled up replica set payload-processing-e2e-worker-w5-6f1190-c8f5948fd from 0 to 1 65m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-6f1190-7568b9dd79 Created pod: payload-pre-processing-e2e-worker-w5-6f1190-7568b9dd79-jvzsd 65m Normal Pulled pod/payload-processing-e2e-worker-w5-6f1190-c8f5948fd-sd9d7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 65m Normal Created pod/payload-processing-e2e-worker-w5-6f1190-c8f5948fd-sd9d7 Container created 65m Normal AddedInterface pod/payload-processing-e2e-worker-w5-6f1190-c8f5948fd-sd9d7 Add eth0 [10.133.0.44/23] from ovn-kubernetes 65m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-6f1190-7568b9dd79-jvzsd Add eth0 [10.132.0.51/23] from ovn-kubernetes 65m Normal Pulling pod/payload-pre-processing-e2e-worker-w5-6f1190-7568b9dd79-jvzsd Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 65m Normal Started pod/payload-processing-e2e-worker-w5-6f1190-c8f5948fd-sd9d7 Container started 65m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-545b5d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-545b5d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-ee74cb Scaled up replica set payload-processing-e2e-worker-w2-ee74cb-6479cc6f4b from 0 to 1 65m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-ee74cb Scaled up replica set payload-pre-processing-e2e-worker-w2-ee74cb-5786ddf6ff from 0 to 1 65m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-ee74cb-5786ddf6ff-lb7f5 Add eth0 [10.133.0.45/23] from ovn-kubernetes 65m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-ee74cb-5786ddf6ff-lb7f5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 65m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-ee74cb-6479cc6f4b Created pod: payload-processing-e2e-worker-w2-ee74cb-6479cc6f4b-5qpss 65m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-ee74cb-6479cc6f4b Error creating: pods "payload-processing-e2e-worker-w2-ee74cb-6479cc6f4b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-ee74cb: serviceaccount "payload-processing-e2e-worker-w2-ee74cb" not found 65m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-ee74cb-5786ddf6ff Error creating: pods "payload-pre-processing-e2e-worker-w2-ee74cb-5786ddf6ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-ee74cb: serviceaccount "payload-processing-e2e-worker-w2-ee74cb" not found 65m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-ee74cb-5786ddf6ff Created pod: payload-pre-processing-e2e-worker-w2-ee74cb-5786ddf6ff-lb7f5 65m Normal AddedInterface pod/payload-processing-e2e-worker-w2-ee74cb-6479cc6f4b-5qpss Add eth0 [10.133.0.46/23] from ovn-kubernetes 65m Normal Created pod/payload-pre-processing-e2e-worker-w2-ee74cb-5786ddf6ff-lb7f5 Container created 65m Normal Started pod/payload-pre-processing-e2e-worker-w2-ee74cb-5786ddf6ff-lb7f5 Container started 65m Normal Started pod/payload-pre-processing-e2e-worker-w5-6f1190-7568b9dd79-jvzsd Container started 65m Normal Created pod/payload-processing-e2e-worker-w2-ee74cb-6479cc6f4b-5qpss Container created 65m Normal Started pod/payload-processing-e2e-worker-w2-ee74cb-6479cc6f4b-5qpss Container started 65m Normal Created pod/payload-pre-processing-e2e-worker-w5-6f1190-7568b9dd79-jvzsd Container created 65m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-6f1190-7568b9dd79-jvzsd Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.864s (2.864s including waiting). Image size: 125353963 bytes. 65m Normal Pulled pod/payload-processing-e2e-worker-w2-ee74cb-6479cc6f4b-5qpss Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 64m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-68fbd1 Scaled up replica set payload-pre-processing-e2e-worker-w0-68fbd1-76d7f5c887 from 0 to 1 64m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-68fbd1 Scaled up replica set payload-processing-e2e-worker-w0-68fbd1-6d979f99cb from 0 to 1 64m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-68fbd1-76d7f5c887 Created pod: payload-pre-processing-e2e-worker-w0-68fbd1-76d7f5c887-p9lsw 64m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-68fbd1-76d7f5c887 Error creating: pods "payload-pre-processing-e2e-worker-w0-68fbd1-76d7f5c887-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-68fbd1: serviceaccount "payload-processing-e2e-worker-w0-68fbd1" not found 64m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-68fbd1-6d979f99cb Created pod: payload-processing-e2e-worker-w0-68fbd1-6d979f99cb-65n5z 64m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-68fbd1-6d979f99cb Error creating: pods "payload-processing-e2e-worker-w0-68fbd1-6d979f99cb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-68fbd1: serviceaccount "payload-processing-e2e-worker-w0-68fbd1" not found 64m Normal Pulled pod/payload-processing-e2e-worker-w0-68fbd1-6d979f99cb-65n5z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 64m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-68fbd1-76d7f5c887-p9lsw Add eth0 [10.133.0.47/23] from ovn-kubernetes 64m Normal AddedInterface pod/payload-processing-e2e-worker-w0-68fbd1-6d979f99cb-65n5z Add eth0 [10.133.0.48/23] from ovn-kubernetes 64m Normal Started pod/payload-pre-processing-e2e-worker-w0-68fbd1-76d7f5c887-p9lsw Container started 64m Normal Created pod/payload-pre-processing-e2e-worker-w0-68fbd1-76d7f5c887-p9lsw Container created 64m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-68fbd1-76d7f5c887-p9lsw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 64m Normal Started pod/payload-processing-e2e-worker-w0-68fbd1-6d979f99cb-65n5z Container started 64m Normal Created pod/payload-processing-e2e-worker-w0-68fbd1-6d979f99cb-65n5z Container created 64m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-961054-66867965c8 Error creating: pods "payload-processing-e2e-worker-w3-961054-66867965c8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-961054: serviceaccount "payload-processing-e2e-worker-w3-961054" not found 64m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7 Error creating: pods "payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-961054: serviceaccount "payload-processing-e2e-worker-w3-961054" not found 64m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-961054-66867965c8 Created pod: payload-processing-e2e-worker-w3-961054-66867965c8-8zxjh 64m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-961054 Scaled up replica set payload-processing-e2e-worker-w3-961054-66867965c8 from 0 to 1 64m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7 Created pod: payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr 64m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-961054 Scaled up replica set payload-pre-processing-e2e-worker-w3-961054-64cfb455d7 from 0 to 1 64m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr Add eth0 [10.134.0.49/23] from ovn-kubernetes 64m Normal AddedInterface pod/payload-processing-e2e-worker-w3-961054-66867965c8-8zxjh Add eth0 [10.133.0.49/23] from ovn-kubernetes 64m Normal Pulled pod/payload-processing-e2e-worker-w3-961054-66867965c8-8zxjh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 64m Normal Started pod/payload-processing-e2e-worker-w3-961054-66867965c8-8zxjh Container started 64m Normal Created pod/payload-processing-e2e-worker-w3-961054-66867965c8-8zxjh Container created 64m Normal Pulling pod/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 64m Normal Started pod/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr Container started 64m Normal Created pod/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr Container created 64m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.013s (3.013s including waiting). Image size: 125353963 bytes. 64m Normal Pulled pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-mzhsz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-18c03a-85c44654fd Error creating: pods "payload-processing-e2e-worker-w4-18c03a-85c44654fd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-18c03a: serviceaccount "payload-processing-e2e-worker-w4-18c03a" not found 64m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-18c03a Scaled up replica set payload-pre-processing-e2e-worker-w4-18c03a-d7d874c9 from 0 to 1 64m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-18c03a Scaled up replica set payload-processing-e2e-worker-w4-18c03a-85c44654fd from 0 to 1 64m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-18c03a-d7d874c9-bn495 Add eth0 [10.132.0.54/23] from ovn-kubernetes 64m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-18c03a-d7d874c9 Error creating: pods "payload-pre-processing-e2e-worker-w4-18c03a-d7d874c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-18c03a: serviceaccount "payload-processing-e2e-worker-w4-18c03a" not found 64m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-18c03a-d7d874c9-bn495 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 64m Normal Created pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-mzhsz Container created 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-ee74cb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Normal AddedInterface pod/payload-processing-e2e-worker-w4-18c03a-85c44654fd-b8t9v Add eth0 [10.133.0.50/23] from ovn-kubernetes 64m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-ee74cb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-18c03a-85c44654fd Created pod: payload-processing-e2e-worker-w4-18c03a-85c44654fd-b8t9v 64m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-18c03a-d7d874c9 Created pod: payload-pre-processing-e2e-worker-w4-18c03a-d7d874c9-bn495 64m Normal Started pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-mzhsz Container started 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-68fbd1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-68fbd1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Normal Started pod/payload-processing-e2e-worker-w4-18c03a-85c44654fd-b8t9v Container started 64m Normal Started pod/payload-pre-processing-e2e-worker-w4-18c03a-d7d874c9-bn495 Container started 64m Normal Pulled pod/payload-processing-e2e-worker-w4-18c03a-85c44654fd-b8t9v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 64m Normal Created pod/payload-pre-processing-e2e-worker-w4-18c03a-d7d874c9-bn495 Container created 64m Normal Created pod/payload-processing-e2e-worker-w4-18c03a-85c44654fd-b8t9v Container created 64m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-545b5d-5c89cdd65f Error creating: pods "payload-pre-processing-e2e-worker-w1-545b5d-5c89cdd65f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-545b5d: serviceaccount "payload-processing-e2e-worker-w1-545b5d" not found 64m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-545b5d-6ff8c4558 Created pod: payload-processing-e2e-worker-w1-545b5d-6ff8c4558-75gnb 64m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-545b5d-6ff8c4558 Error creating: pods "payload-processing-e2e-worker-w1-545b5d-6ff8c4558-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-545b5d: serviceaccount "payload-processing-e2e-worker-w1-545b5d" not found 64m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-545b5d Scaled up replica set payload-pre-processing-e2e-worker-w1-545b5d-5c89cdd65f from 0 to 1 64m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-545b5d-5c89cdd65f Created pod: payload-pre-processing-e2e-worker-w1-545b5d-5c89cdd65f-w2cnc 64m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-545b5d Scaled up replica set payload-processing-e2e-worker-w1-545b5d-6ff8c4558 from 0 to 1 64m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-545b5d-5c89cdd65f-w2cnc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 64m Normal Created pod/payload-pre-processing-e2e-worker-w1-545b5d-5c89cdd65f-w2cnc Container created 64m Normal Started pod/payload-processing-e2e-worker-w1-545b5d-6ff8c4558-75gnb Container started 64m Normal Created pod/payload-processing-e2e-worker-w1-545b5d-6ff8c4558-75gnb Container created 64m Normal Pulled pod/payload-processing-e2e-worker-w1-545b5d-6ff8c4558-75gnb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 64m Normal AddedInterface pod/payload-processing-e2e-worker-w1-545b5d-6ff8c4558-75gnb Add eth0 [10.133.0.52/23] from ovn-kubernetes 64m Normal Started pod/payload-pre-processing-e2e-worker-w1-545b5d-5c89cdd65f-w2cnc Container started 64m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-545b5d-5c89cdd65f-w2cnc Add eth0 [10.133.0.51/23] from ovn-kubernetes 64m Warning ProbeError pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-mzhsz Startup probe error: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused... 64m Warning Unhealthy pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-mzhsz Startup probe failed: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused 64m Normal Pulled pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Created pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d Container created 64m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-18c03a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-18c03a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Normal Started pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d Container started 64m Normal Pulled pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-b7xdq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Started pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-b7xdq Container started 64m Normal Created pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-b7xdq Container created 64m Warning ProbeError pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d Startup probe error: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused... 64m Warning Unhealthy pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d Startup probe failed: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused 64m Warning ProbeError pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-b7xdq Startup probe error: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused... 64m Warning Unhealthy pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-b7xdq Startup probe failed: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused 64m Warning ProbeError pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm Readiness probe error: Get "http://10.134.0.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 64m Warning Unhealthy pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm Readiness probe failed: Get "http://10.134.0.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 64m Normal Created pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7 Container created 64m Warning Unhealthy pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7 Readiness probe failed: Get "http://10.134.0.45:15021/healthz/ready": read tcp 10.134.0.2:58250->10.134.0.45:15021: read: connection reset by peer 64m Normal Pulled pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Started pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7 Container started 64m Warning ProbeError pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7 Readiness probe error: Get "http://10.134.0.45:15021/healthz/ready": read tcp 10.134.0.2:58250->10.134.0.45:15021: read: connection reset by peer... 64m Normal Pulled pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Created pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm Container created 64m Normal Started pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm Container started 64m Warning ProbeError pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7 Startup probe error: Get "http://10.134.0.45:15021/healthz/ready": dial tcp 10.134.0.45:15021: connect: connection refused... 64m Warning Unhealthy pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7 Startup probe failed: Get "http://10.134.0.45:15021/healthz/ready": dial tcp 10.134.0.45:15021: connect: connection refused 64m Warning Unhealthy pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm Startup probe failed: Get "http://10.134.0.46:15021/healthz/ready": dial tcp 10.134.0.46:15021: connect: connection refused 64m Warning ProbeError pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm Startup probe error: Get "http://10.134.0.46:15021/healthz/ready": dial tcp 10.134.0.46:15021: connect: connection refused... 64m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-961054-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-961054-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-5e72ad-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-5e72ad-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-ee74cb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 64m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-ee74cb-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 64m Normal Started pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5ctxt Container started 64m Normal Created pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5ctxt Container created 64m Normal Pulled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5ctxt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Created pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b Container created 64m Normal Pulled pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Started pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b Container started 64m Warning Unhealthy pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5ctxt Startup probe failed: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused 64m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5ctxt Startup probe error: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused... 64m Warning Unhealthy pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b Startup probe failed: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused 64m Warning ProbeError pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b Startup probe error: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused... 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-5e72ad-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-6f1190-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-6f1190-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-5e72ad-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 64m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-961054-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-961054-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-18c03a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 63m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-18c03a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-545b5d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 63m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-545b5d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 61m Normal SuccessfulCreate replicaset/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 Created pod: e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-n7vdt 61m Normal Created pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-n7vdt Container created 61m Normal AddedInterface pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-n7vdt Add eth0 [10.132.0.59/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-n7vdt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Started pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-n7vdt Container started 61m Warning Unhealthy pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-n7vdt Startup probe failed: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-n7vdt Startup probe error: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused... 60m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-ee74cb-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 60m Normal ScalingReplicaSet deployment/e2e-worker-w2-ee74cb-openshift-default Scaled up replica set e2e-worker-w2-ee74cb-openshift-default-9758876f from 1 to 2 60m Normal AddedInterface pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-rj8x7 Add eth0 [10.133.0.64/23] from ovn-kubernetes 60m Normal Pulled pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-rj8x7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal SuccessfulCreate replicaset/e2e-worker-w2-ee74cb-openshift-default-9758876f Created pod: e2e-worker-w2-ee74cb-openshift-default-9758876f-rj8x7 60m Normal Started pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-rj8x7 Container started 60m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-59f957fdd7 from 7 to 5 60m Normal Killing pod/maas-default-gateway-openshift-default-59f957fdd7-n2sn6 Stopping container istio-proxy 60m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-59f957fdd7 Deleted pod: maas-default-gateway-openshift-default-59f957fdd7-8mmql 60m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-59f957fdd7 Deleted pod: maas-default-gateway-openshift-default-59f957fdd7-n2sn6 60m Normal Killing pod/maas-default-gateway-openshift-default-59f957fdd7-8mmql Stopping container istio-proxy 60m Normal Created pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-rj8x7 Container created 60m Warning ProbeError pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-rj8x7 Startup probe error: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-rj8x7 Startup probe failed: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused 60m Normal SuccessfulCreate replicaset/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c Created pod: e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5z5wk 60m Normal ScalingReplicaSet deployment/e2e-worker-w1-545b5d-openshift-default Scaled up replica set e2e-worker-w1-545b5d-openshift-default-5c8c66f65c from 1 to 2 60m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-545b5d-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 60m Normal Created pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5z5wk Container created 60m Normal Pulled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5z5wk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal AddedInterface pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5z5wk Add eth0 [10.132.0.60/23] from ovn-kubernetes 60m Normal Started pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5z5wk Container started 60m Warning Unhealthy pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5z5wk Startup probe failed: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5z5wk Startup probe error: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused... 60m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-59f957fdd7 Created pod: maas-default-gateway-openshift-default-59f957fdd7-mm95n 60m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-59f957fdd7 Created pod: maas-default-gateway-openshift-default-59f957fdd7-dvtvl 60m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-59f957fdd7 from 5 to 10 60m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-ftxcj Container created 60m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-l2qpx Container started 60m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-4vg6x Add eth0 [10.132.0.61/23] from ovn-kubernetes 60m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-4vg6x Container created 60m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-4vg6x Container started 60m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-l2qpx Container created 60m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-ftxcj Container started 60m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-4vg6x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-ftxcj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-ftxcj Add eth0 [10.132.0.62/23] from ovn-kubernetes 60m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-mm95n Container started 60m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-mm95n Container created 60m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-mm95n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-l2qpx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-mm95n Add eth0 [10.133.0.75/23] from ovn-kubernetes 60m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-l2qpx Add eth0 [10.134.0.59/23] from ovn-kubernetes 60m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-dvtvl Container started 60m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-dvtvl Add eth0 [10.133.0.76/23] from ovn-kubernetes 60m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-dvtvl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-dvtvl Container created 60m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-l2qpx Startup probe error: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused... 60m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-dvtvl Startup probe error: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused... 60m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-dvtvl Startup probe failed: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused 60m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-ftxcj Startup probe error: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused... 60m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-mm95n Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused... 60m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-l2qpx Startup probe failed: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused 60m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-4vg6x Startup probe failed: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused 60m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-4vg6x Startup probe error: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused... 60m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-ftxcj Startup probe failed: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused 60m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-mm95n Startup probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6f1190-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 59m Normal ScalingReplicaSet deployment/e2e-worker-w5-6f1190-openshift-default Scaled up replica set e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 from 2 to 3 59m Normal SuccessfulCreate replicaset/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 Created pod: e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm 59m Normal Started pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm Container started 59m Normal Created pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm Container created 59m Normal Pulled pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm Add eth0 [10.134.0.60/23] from ovn-kubernetes 59m Warning Unhealthy pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm Startup probe failed: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm Startup probe error: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused... 59m Normal Pulled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal SuccessfulCreate replicaset/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c Created pod: e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7 59m Normal SuccessfulCreate replicaset/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 Created pod: e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-nnsbl 59m Normal ScalingReplicaSet deployment/e2e-worker-w1-545b5d-openshift-default Scaled up replica set e2e-worker-w1-545b5d-openshift-default-5c8c66f65c from 2 to 3 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-68fbd1-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 59m Normal ScalingReplicaSet deployment/e2e-worker-w0-68fbd1-openshift-default Scaled up replica set e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 from 1 to 2 59m Normal Created pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7 Container created 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-545b5d-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 59m Normal AddedInterface pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7 Add eth0 [10.134.0.61/23] from ovn-kubernetes 59m Normal AddedInterface pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-nnsbl Add eth0 [10.133.0.83/23] from ovn-kubernetes 59m Normal Started pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7 Container started 59m Normal Started pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-nnsbl Container started 59m Normal Created pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-nnsbl Container created 59m Normal Pulled pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-nnsbl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Warning Unhealthy pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7 Startup probe failed: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7 Startup probe error: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-nnsbl Startup probe failed: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-nnsbl Startup probe error: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused... 58m Normal SuccessfulCreate replicaset/e2e-worker-w3-961054-openshift-default-5d5665c66f Created pod: e2e-worker-w3-961054-openshift-default-5d5665c66f-b69rq 58m Normal ScalingReplicaSet deployment/e2e-worker-w3-961054-openshift-default Scaled up replica set e2e-worker-w3-961054-openshift-default-5d5665c66f from 1 to 2 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-961054-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 58m Normal Started pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-b69rq Container started 58m Normal Created pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-b69rq Container created 58m Normal Pulled pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-b69rq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-b69rq Add eth0 [10.132.0.64/23] from ovn-kubernetes 58m Normal SuccessfulCreate replicaset/e2e-worker-w2-ee74cb-openshift-default-9758876f Created pod: e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4 58m Normal AddedInterface pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4 Add eth0 [10.134.0.63/23] from ovn-kubernetes 58m Normal Pulled pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4 Container created 58m Normal Started pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4 Container started 58m Warning ProbeError pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-b69rq Startup probe error: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-b69rq Startup probe failed: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused 58m Warning Unhealthy pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4 Startup probe failed: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4 Startup probe error: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused... 58m Normal ScalingReplicaSet deployment/e2e-worker-w6-5e72ad-openshift-default Scaled up replica set e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c from 1 to 2 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-5e72ad-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 58m Normal SuccessfulCreate replicaset/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c Created pod: e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-mvhjt 58m Normal Pulled pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-mvhjt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-mvhjt Add eth0 [10.133.0.87/23] from ovn-kubernetes 58m Normal Started pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-mvhjt Container started 58m Normal Created pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-mvhjt Container created 58m Warning ProbeError pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-mvhjt Startup probe error: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-mvhjt Startup probe failed: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused 58m Normal SuccessfulCreate replicaset/e2e-worker-w4-18c03a-openshift-default-589bbc88df Created pod: e2e-worker-w4-18c03a-openshift-default-589bbc88df-r69n5 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-18c03a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 58m Normal ScalingReplicaSet deployment/e2e-worker-w4-18c03a-openshift-default Scaled up replica set e2e-worker-w4-18c03a-openshift-default-589bbc88df from 1 to 2 58m Normal Started pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r69n5 Container started 58m Normal AddedInterface pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r69n5 Add eth0 [10.132.0.66/23] from ovn-kubernetes 58m Normal Created pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r69n5 Container created 58m Normal Pulled pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r69n5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Warning Unhealthy pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r69n5 Startup probe failed: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r69n5 Startup probe error: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused... 57m Normal SuccessfulCreate replicaset/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c Created pod: e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-bpdgg 57m Normal SuccessfulCreate replicaset/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c Created pod: e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-th2cc 57m Normal ScalingReplicaSet deployment/e2e-worker-w1-545b5d-openshift-default Scaled up replica set e2e-worker-w1-545b5d-openshift-default-5c8c66f65c from 3 to 5 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-545b5d-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 57m Normal Pulled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-bpdgg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-bpdgg Add eth0 [10.132.0.67/23] from ovn-kubernetes 57m Normal Created pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-th2cc Container created 57m Normal Pulled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-th2cc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-th2cc Add eth0 [10.133.0.88/23] from ovn-kubernetes 57m Normal Started pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-th2cc Container started 57m Normal Started pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-bpdgg Container started 57m Normal Created pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-bpdgg Container created 57m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-th2cc Startup probe error: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-th2cc Startup probe failed: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-bpdgg Startup probe error: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-bpdgg Startup probe failed: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-961054-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 56m Normal SuccessfulCreate replicaset/e2e-worker-w3-961054-openshift-default-5d5665c66f Created pod: e2e-worker-w3-961054-openshift-default-5d5665c66f-gpbv6 56m Normal ScalingReplicaSet deployment/e2e-worker-w3-961054-openshift-default Scaled up replica set e2e-worker-w3-961054-openshift-default-5d5665c66f from 2 to 4 56m Normal SuccessfulCreate replicaset/e2e-worker-w3-961054-openshift-default-5d5665c66f Created pod: e2e-worker-w3-961054-openshift-default-5d5665c66f-w8xsc 56m Normal Started pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-gpbv6 Container started 56m Normal Pulled pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-w8xsc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal SuccessfulCreate replicaset/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 Created pod: e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-hjqrh 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-68fbd1-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 56m Normal SuccessfulCreate replicaset/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 Created pod: e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t 56m Normal AddedInterface pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-w8xsc Add eth0 [10.133.0.93/23] from ovn-kubernetes 56m Normal ScalingReplicaSet deployment/e2e-worker-w0-68fbd1-openshift-default Scaled up replica set e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 from 2 to 4 56m Normal Started pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-w8xsc Container started 56m Normal Created pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-w8xsc Container created 56m Normal AddedInterface pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-gpbv6 Add eth0 [10.132.0.68/23] from ovn-kubernetes 56m Normal Created pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-gpbv6 Container created 56m Normal Pulled pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-gpbv6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t Container started 56m Normal Created pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t Container created 56m Normal Pulled pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t Add eth0 [10.134.0.67/23] from ovn-kubernetes 56m Normal Started pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-hjqrh Container started 56m Normal Pulled pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-hjqrh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-hjqrh Container created 56m Normal AddedInterface pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-hjqrh Add eth0 [10.132.0.69/23] from ovn-kubernetes 56m Warning Unhealthy pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-gpbv6 Startup probe failed: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-gpbv6 Startup probe error: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-w8xsc Startup probe failed: Get "http://10.133.0.93:15021/healthz/ready": dial tcp 10.133.0.93:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-w8xsc Startup probe error: Get "http://10.133.0.93:15021/healthz/ready": dial tcp 10.133.0.93:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-hjqrh Startup probe failed: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t Startup probe error: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t Startup probe failed: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-hjqrh Startup probe error: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-hjqrh Startup probe failed: Get "http://10.132.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-hjqrh Startup probe error: Get "http://10.132.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Normal AddedInterface pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-qnfhf Add eth0 [10.133.0.96/23] from ovn-kubernetes 56m Normal AddedInterface pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-fwck9 Add eth0 [10.132.0.70/23] from ovn-kubernetes 56m Normal ScalingReplicaSet deployment/e2e-worker-w2-ee74cb-openshift-default Scaled up replica set e2e-worker-w2-ee74cb-openshift-default-9758876f from 3 to 6 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-ee74cb-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 56m Normal SuccessfulCreate replicaset/e2e-worker-w2-ee74cb-openshift-default-9758876f Created pod: e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz 56m Normal SuccessfulCreate replicaset/e2e-worker-w2-ee74cb-openshift-default-9758876f Created pod: e2e-worker-w2-ee74cb-openshift-default-9758876f-qnfhf 56m Normal SuccessfulCreate replicaset/e2e-worker-w2-ee74cb-openshift-default-9758876f Created pod: e2e-worker-w2-ee74cb-openshift-default-9758876f-fwck9 56m Normal AddedInterface pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz Add eth0 [10.134.0.70/23] from ovn-kubernetes 56m Normal Created pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz Container created 56m Normal Pulled pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-qnfhf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-qnfhf Container created 56m Normal Pulled pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-qnfhf Container started 56m Normal Pulled pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-fwck9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-fwck9 Container created 56m Normal Started pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-fwck9 Container started 56m Normal Started pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz Container started 56m Warning ProbeError pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-fwck9 Startup probe error: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-fwck9 Startup probe failed: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused 56m Warning Unhealthy pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-qnfhf Startup probe failed: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-qnfhf Startup probe error: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz Startup probe failed: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz Startup probe error: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused... 55m Normal SuccessfulCreate replicaset/e2e-worker-w2-ee74cb-openshift-default-9758876f Created pod: e2e-worker-w2-ee74cb-openshift-default-9758876f-sq629 55m Normal SuccessfulCreate replicaset/e2e-worker-w2-ee74cb-openshift-default-9758876f Created pod: e2e-worker-w2-ee74cb-openshift-default-9758876f-qkcnq 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-ee74cb-openshift-default New size: 8; reason: All metrics below target 55m Normal ScalingReplicaSet deployment/e2e-worker-w2-ee74cb-openshift-default Scaled up replica set e2e-worker-w2-ee74cb-openshift-default-9758876f from 6 to 8 55m Normal Created pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-sq629 Container created 55m Normal Started pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-qkcnq Container started 55m Normal Created pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-qkcnq Container created 55m Normal Pulled pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-qkcnq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-qkcnq Add eth0 [10.133.0.99/23] from ovn-kubernetes 55m Normal AddedInterface pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-sq629 Add eth0 [10.132.0.72/23] from ovn-kubernetes 55m Normal Pulled pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-sq629 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Started pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-sq629 Container started 55m Warning Unhealthy pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-qkcnq Startup probe failed: Get "http://10.133.0.99:15021/healthz/ready": dial tcp 10.133.0.99:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-qkcnq Startup probe error: Get "http://10.133.0.99:15021/healthz/ready": dial tcp 10.133.0.99:15021: connect: connection refused... 55m Warning ProbeError pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-sq629 Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-sq629 Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-18c03a-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 55m Normal ScalingReplicaSet deployment/e2e-worker-w4-18c03a-openshift-default Scaled up replica set e2e-worker-w4-18c03a-openshift-default-589bbc88df from 2 to 4 55m Normal SuccessfulCreate replicaset/e2e-worker-w4-18c03a-openshift-default-589bbc88df Created pod: e2e-worker-w4-18c03a-openshift-default-589bbc88df-r7xzt 55m Normal SuccessfulCreate replicaset/e2e-worker-w4-18c03a-openshift-default-589bbc88df Created pod: e2e-worker-w4-18c03a-openshift-default-589bbc88df-dn59n 55m Normal Created pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r7xzt Container created 55m Normal AddedInterface pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r7xzt Add eth0 [10.133.0.100/23] from ovn-kubernetes 55m Normal Started pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-dn59n Container started 55m Normal Created pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-dn59n Container created 55m Normal Pulled pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r7xzt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Pulled pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-dn59n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-dn59n Add eth0 [10.132.0.73/23] from ovn-kubernetes 55m Normal Started pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r7xzt Container started 55m Normal SuccessfulCreate replicaset/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c Created pod: e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd 55m Normal AddedInterface pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc Add eth0 [10.134.0.71/23] from ovn-kubernetes 55m Normal SuccessfulCreate replicaset/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c Created pod: e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-q27rd 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-5e72ad-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 55m Normal ScalingReplicaSet deployment/e2e-worker-w6-5e72ad-openshift-default Scaled up replica set e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c from 2 to 4 55m Normal ScalingReplicaSet deployment/e2e-worker-w5-6f1190-openshift-default Scaled up replica set e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 from 3 to 6 55m Normal SuccessfulCreate replicaset/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 Created pod: e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-kp9cz 55m Normal AddedInterface pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-kp9cz Add eth0 [10.132.0.74/23] from ovn-kubernetes 55m Normal AddedInterface pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-2qsk9 Add eth0 [10.133.0.102/23] from ovn-kubernetes 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6f1190-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 55m Normal SuccessfulCreate replicaset/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 Created pod: e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-2qsk9 55m Normal SuccessfulCreate replicaset/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 Created pod: e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc 55m Normal Created pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-kp9cz Container created 55m Normal SuccessfulCreate replicaset/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 Created pod: e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-c6fst 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-68fbd1-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 55m Normal AddedInterface pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd Add eth0 [10.134.0.72/23] from ovn-kubernetes 55m Normal Pulled pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd Container created 55m Normal Started pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd Container started 55m Normal AddedInterface pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-c6fst Add eth0 [10.132.0.76/23] from ovn-kubernetes 55m Normal Started pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-kp9cz Container started 55m Normal Pulled pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-kp9cz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Pulled pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-q27rd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Pulled pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-2qsk9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-2qsk9 Container created 55m Normal ScalingReplicaSet deployment/e2e-worker-w0-68fbd1-openshift-default Scaled up replica set e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 from 4 to 7 55m Normal SuccessfulCreate replicaset/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 Created pod: e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p 55m Normal SuccessfulCreate replicaset/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 Created pod: e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-8t4p6 55m Normal Started pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-2qsk9 Container started 55m Normal AddedInterface pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-q27rd Add eth0 [10.132.0.75/23] from ovn-kubernetes 55m Normal Started pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc Container started 55m Normal Created pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc Container created 55m Normal Pulled pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p Add eth0 [10.134.0.73/23] from ovn-kubernetes 55m Normal Pulled pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-q27rd Container created 55m Normal Started pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-q27rd Container started 55m Normal Created pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-8t4p6 Container created 55m Normal Pulled pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-c6fst Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Warning Unhealthy pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-dn59n Startup probe failed: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused 55m Normal Started pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-c6fst Container started 55m Normal Started pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p Container started 55m Normal Created pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p Container created 55m Normal Started pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-8t4p6 Container started 55m Normal AddedInterface pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-8t4p6 Add eth0 [10.133.0.103/23] from ovn-kubernetes 55m Normal Created pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-c6fst Container created 55m Warning ProbeError pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r7xzt Startup probe error: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused... 55m Normal Pulled pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-8t4p6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Warning Unhealthy pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r7xzt Startup probe failed: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-dn59n Startup probe error: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused... 55m Warning ProbeError pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-2qsk9 Startup probe error: Get "http://10.133.0.102:15021/healthz/ready": dial tcp 10.133.0.102:15021: connect: connection refused... 55m Warning ProbeError pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd Startup probe failed: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused 55m Warning Unhealthy pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused 55m Warning Unhealthy pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-2qsk9 Startup probe failed: Get "http://10.133.0.102:15021/healthz/ready": dial tcp 10.133.0.102:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused... 55m Warning ProbeError pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p Startup probe error: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused... 55m Warning ProbeError pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-q27rd Startup probe error: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-q27rd Startup probe failed: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-kp9cz Startup probe error: Get "http://10.132.0.74:15021/healthz/ready": dial tcp 10.132.0.74:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-8t4p6 Startup probe failed: Get "http://10.133.0.103:15021/healthz/ready": dial tcp 10.133.0.103:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-8t4p6 Startup probe error: Get "http://10.133.0.103:15021/healthz/ready": dial tcp 10.133.0.103:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-kp9cz Startup probe failed: Get "http://10.132.0.74:15021/healthz/ready": dial tcp 10.132.0.74:15021: connect: connection refused 55m Warning Unhealthy pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p Startup probe failed: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused 55m Normal ScalingReplicaSet deployment/e2e-worker-w4-18c03a-openshift-default Scaled up replica set e2e-worker-w4-18c03a-openshift-default-589bbc88df from 4 to 5 55m Normal SuccessfulCreate replicaset/e2e-worker-w4-18c03a-openshift-default-589bbc88df Created pod: e2e-worker-w4-18c03a-openshift-default-589bbc88df-k5m5d 55m Normal Pulled pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-k5m5d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-k5m5d Add eth0 [10.133.0.105/23] from ovn-kubernetes 55m Normal Started pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-k5m5d Container started 55m Normal Created pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-k5m5d Container created 55m Warning ProbeError pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-c6fst Startup probe error: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-c6fst Startup probe failed: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused 55m Normal ScalingReplicaSet deployment/e2e-worker-w6-5e72ad-openshift-default Scaled up replica set e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c from 4 to 5 55m Normal SuccessfulCreate replicaset/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c Created pod: e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-x5fmj 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-5e72ad-openshift-default New size: 5; reason: All metrics below target 55m Normal Created pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-x5fmj Container created 55m Normal Started pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-x5fmj Container started 55m Normal Pulled pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-x5fmj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-x5fmj Add eth0 [10.132.0.77/23] from ovn-kubernetes 55m Warning ProbeError pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-k5m5d Startup probe error: Get "http://10.133.0.105:15021/healthz/ready": dial tcp 10.133.0.105:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-k5m5d Startup probe failed: Get "http://10.133.0.105:15021/healthz/ready": dial tcp 10.133.0.105:15021: connect: connection refused 55m Warning Unhealthy pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-x5fmj Startup probe failed: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-x5fmj Startup probe error: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused... 54m Normal ScalingReplicaSet deployment/e2e-worker-w3-961054-openshift-default Scaled up replica set e2e-worker-w3-961054-openshift-default-5d5665c66f from 4 to 6 54m Normal SuccessfulCreate replicaset/e2e-worker-w3-961054-openshift-default-5d5665c66f Created pod: e2e-worker-w3-961054-openshift-default-5d5665c66f-vw6ft 54m Normal SuccessfulCreate replicaset/e2e-worker-w3-961054-openshift-default-5d5665c66f Created pod: e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-961054-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulCreate replicaset/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c Created pod: e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-wfg5p 54m Normal AddedInterface pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr Add eth0 [10.134.0.74/23] from ovn-kubernetes 54m Normal SuccessfulCreate replicaset/e2e-worker-w2-ee74cb-openshift-default-9758876f (combined from similar events): Created pod: e2e-worker-w2-ee74cb-openshift-default-9758876f-k46v6 54m Normal SuccessfulCreate replicaset/e2e-worker-w2-ee74cb-openshift-default-9758876f Created pod: e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74 54m Normal ScalingReplicaSet deployment/e2e-worker-w2-ee74cb-openshift-default Scaled up replica set e2e-worker-w2-ee74cb-openshift-default-9758876f from 8 to 10 54m Normal Started pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr Container started 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-545b5d-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 54m Normal AddedInterface pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-72tpd Add eth0 [10.132.0.78/23] from ovn-kubernetes 54m Normal SuccessfulCreate replicaset/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c Created pod: e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc 54m Normal SuccessfulCreate replicaset/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c Created pod: e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-72tpd 54m Normal AddedInterface pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-vw6ft Add eth0 [10.133.0.116/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-vw6ft Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-vw6ft Container created 54m Normal ScalingReplicaSet deployment/e2e-worker-w1-545b5d-openshift-default Scaled up replica set e2e-worker-w1-545b5d-openshift-default-5c8c66f65c from 5 to 8 54m Normal Pulled pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc Add eth0 [10.134.0.75/23] from ovn-kubernetes 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-ee74cb-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 54m Normal Started pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-vw6ft Container started 54m Normal Created pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr Container created 54m Normal Pulled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc Container created 54m Normal Started pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc Container started 54m Normal Pulled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-72tpd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-72tpd Container created 54m Normal Started pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-72tpd Container started 54m Normal Killing pod/maas-default-gateway-openshift-default-59f957fdd7-ftxcj Stopping container istio-proxy 54m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-59f957fdd7 Deleted pod: maas-default-gateway-openshift-default-59f957fdd7-ftxcj 54m Normal Killing pod/maas-default-gateway-openshift-default-59f957fdd7-j72ct Stopping container istio-proxy 54m Normal Killing pod/maas-default-gateway-openshift-default-59f957fdd7-mm95n Stopping container istio-proxy 54m Normal Pulled pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-wfg5p Add eth0 [10.133.0.117/23] from ovn-kubernetes 54m Normal AddedInterface pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74 Add eth0 [10.134.0.76/23] from ovn-kubernetes 54m Normal Killing pod/maas-default-gateway-openshift-default-59f957fdd7-dvtvl Stopping container istio-proxy 54m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-59f957fdd7 Deleted pod: maas-default-gateway-openshift-default-59f957fdd7-mm95n 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-d5brv Readiness probe failed: HTTP probe failed with statuscode: 503 54m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-d5brv Readiness probe error: HTTP probe failed with statuscode: 503... 54m Normal Killing pod/maas-default-gateway-openshift-default-59f957fdd7-d5brv Stopping container istio-proxy 54m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-59f957fdd7 Deleted pod: maas-default-gateway-openshift-default-59f957fdd7-d5brv 54m Normal Killing pod/maas-default-gateway-openshift-default-59f957fdd7-4vg6x Stopping container istio-proxy 54m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-59f957fdd7 Deleted pod: maas-default-gateway-openshift-default-59f957fdd7-j72ct 54m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-59f957fdd7 Deleted pod: maas-default-gateway-openshift-default-59f957fdd7-4vg6x 54m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-59f957fdd7 Deleted pod: maas-default-gateway-openshift-default-59f957fdd7-dvtvl 54m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-59f957fdd7 from 10 to 4 54m Normal Created pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74 Container created 54m Warning Unhealthy pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r7xzt Readiness probe failed: Get "http://10.133.0.100:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning ProbeError pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r7xzt Readiness probe error: Get "http://10.133.0.100:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Normal Pulled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-wfg5p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Started pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74 Container started 54m Warning Unhealthy pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-dn59n Readiness probe failed: Get "http://10.132.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Normal AddedInterface pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-k46v6 Add eth0 [10.133.0.118/23] from ovn-kubernetes 54m Warning ProbeError pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-dn59n Readiness probe error: Get "http://10.132.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Normal Pulled pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-k46v6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Warning ProbeError pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-k5m5d Readiness probe error: Get "http://10.133.0.105:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Normal Created pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-k46v6 Container created 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-j72ct Readiness probe failed: HTTP probe failed with statuscode: 503 54m Warning Unhealthy pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5z5wk Readiness probe failed: Get "http://10.132.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5z5wk Readiness probe error: Get "http://10.132.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Normal Started pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-k46v6 Container started 54m Normal Started pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-wfg5p Container started 54m Normal Created pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-wfg5p Container created 54m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-j72ct Readiness probe error: HTTP probe failed with statuscode: 503... 54m Warning Unhealthy pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-k5m5d Readiness probe failed: Get "http://10.133.0.105:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning ProbeError pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused... 54m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-mm95n Readiness probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused... 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-mm95n Readiness probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused 54m Warning Unhealthy pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused 54m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-dvtvl Readiness probe error: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused... 54m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc Startup probe error: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc Startup probe failed: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-dvtvl Readiness probe failed: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused 54m Warning Unhealthy pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74 Startup probe failed: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74 Startup probe error: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-vw6ft Startup probe failed: Get "http://10.133.0.116:15021/healthz/ready": dial tcp 10.133.0.116:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-vw6ft Startup probe error: Get "http://10.133.0.116:15021/healthz/ready": dial tcp 10.133.0.116:15021: connect: connection refused... 54m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-72tpd Startup probe error: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-72tpd Startup probe failed: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused 54m Warning Unhealthy pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-wfg5p Startup probe failed: Get "http://10.133.0.117:15021/healthz/ready": dial tcp 10.133.0.117:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-wfg5p Startup probe error: Get "http://10.133.0.117:15021/healthz/ready": dial tcp 10.133.0.117:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-k46v6 Startup probe failed: Get "http://10.133.0.118:15021/healthz/ready": dial tcp 10.133.0.118:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-k46v6 Startup probe error: Get "http://10.133.0.118:15021/healthz/ready": dial tcp 10.133.0.118:15021: connect: connection refused... 54m Normal AddedInterface pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-8kw9p Add eth0 [10.132.0.81/23] from ovn-kubernetes 54m Normal SuccessfulCreate replicaset/e2e-worker-w4-18c03a-openshift-default-589bbc88df Created pod: e2e-worker-w4-18c03a-openshift-default-589bbc88df-8kw9p 54m Normal SuccessfulCreate replicaset/e2e-worker-w4-18c03a-openshift-default-589bbc88df Created pod: e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr 54m Normal AddedInterface pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-ff7t5 Add eth0 [10.132.0.82/23] from ovn-kubernetes 54m Normal ScalingReplicaSet deployment/e2e-worker-w4-18c03a-openshift-default Scaled up replica set e2e-worker-w4-18c03a-openshift-default-589bbc88df from 5 to 10 54m Normal SuccessfulCreate replicaset/e2e-worker-w4-18c03a-openshift-default-589bbc88df Created pod: e2e-worker-w4-18c03a-openshift-default-589bbc88df-5qmh2 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-18c03a-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulCreate replicaset/e2e-worker-w4-18c03a-openshift-default-589bbc88df (combined from similar events): Created pod: e2e-worker-w4-18c03a-openshift-default-589bbc88df-ff7t5 54m Normal AddedInterface pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr Add eth0 [10.134.0.77/23] from ovn-kubernetes 54m Normal AddedInterface pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm Add eth0 [10.134.0.78/23] from ovn-kubernetes 54m Normal SuccessfulCreate replicaset/e2e-worker-w4-18c03a-openshift-default-589bbc88df Created pod: e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm 54m Normal Created pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-8kw9p Container created 54m Normal Created pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr Container created 54m Normal Pulled pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Started pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-5qmh2 Container started 54m Normal Created pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm Container created 54m Normal Created pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-5qmh2 Container created 54m Normal Started pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-8kw9p Container started 54m Normal Started pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr Container started 54m Normal Pulled pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-ff7t5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-ff7t5 Container created 54m Normal Started pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-ff7t5 Container started 54m Normal Pulled pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Pulled pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-5qmh2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Pulled pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-8kw9p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-5qmh2 Add eth0 [10.133.0.124/23] from ovn-kubernetes 54m Normal Started pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm Container started 54m Normal SuccessfulCreate replicaset/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c (combined from similar events): Created pod: e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vpkm5 54m Normal ScalingReplicaSet deployment/e2e-worker-w1-545b5d-openshift-default Scaled up replica set e2e-worker-w1-545b5d-openshift-default-5c8c66f65c from 8 to 10 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-545b5d-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulCreate replicaset/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c Created pod: e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl 54m Normal SuccessfulCreate replicaset/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 Created pod: e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-lk9lz 54m Normal SuccessfulCreate replicaset/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 Created pod: e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-p8vxh 54m Normal SuccessfulCreate replicaset/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 (combined from similar events): Created pod: e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-krp2k 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-68fbd1-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 54m Normal ScalingReplicaSet deployment/e2e-worker-w0-68fbd1-openshift-default Scaled up replica set e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 from 7 to 10 54m Warning Unhealthy pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm Startup probe failed: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused 54m Normal Created pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-p8vxh Container created 54m Warning ProbeError pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr Startup probe error: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-5qmh2 Startup probe failed: Get "http://10.133.0.124:15021/healthz/ready": dial tcp 10.133.0.124:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-5qmh2 Startup probe error: Get "http://10.133.0.124:15021/healthz/ready": dial tcp 10.133.0.124:15021: connect: connection refused... 54m Warning ProbeError pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-8kw9p Startup probe error: Get "http://10.132.0.81:15021/healthz/ready": dial tcp 10.132.0.81:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-8kw9p Startup probe failed: Get "http://10.132.0.81:15021/healthz/ready": dial tcp 10.132.0.81:15021: connect: connection refused 54m Normal AddedInterface pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vpkm5 Add eth0 [10.132.0.83/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vpkm5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vpkm5 Container created 54m Normal Started pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vpkm5 Container started 54m Warning ProbeError pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm Startup probe error: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused... 54m Normal Created pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl Container created 54m Normal Pulled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Started pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-p8vxh Container started 54m Normal Started pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl Container started 54m Normal AddedInterface pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-krp2k Add eth0 [10.132.0.85/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-krp2k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-krp2k Container created 54m Warning Unhealthy pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr Startup probe failed: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused 54m Normal Pulled pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-p8vxh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-p8vxh Add eth0 [10.133.0.126/23] from ovn-kubernetes 54m Warning ProbeError pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-ff7t5 Startup probe error: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused... 54m Normal Started pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-lk9lz Container started 54m Normal Started pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-krp2k Container started 54m Normal Created pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-lk9lz Container created 54m Warning Unhealthy pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-ff7t5 Startup probe failed: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused 54m Normal Pulled pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-lk9lz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-lk9lz Add eth0 [10.132.0.84/23] from ovn-kubernetes 54m Normal AddedInterface pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl Add eth0 [10.134.0.79/23] from ovn-kubernetes 54m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-59f957fdd7 Deleted pod: maas-default-gateway-openshift-default-59f957fdd7-l2qpx 54m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-59f957fdd7 from 4 to 3 54m Normal Killing pod/maas-default-gateway-openshift-default-59f957fdd7-l2qpx Stopping container istio-proxy 54m Normal AddedLabel gateway/e2e-ait-bb95c832 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-bb95c832 54m Normal SuccessfulCreate replicaset/e2e-ait-bb95c832-openshift-default-997798bd5 Created pod: e2e-ait-bb95c832-openshift-default-997798bd5-x5tlh 54m Normal ScalingReplicaSet deployment/e2e-ait-bb95c832-openshift-default Scaled up replica set e2e-ait-bb95c832-openshift-default-997798bd5 from 0 to 1 54m Normal ScalingReplicaSet deployment/e2e-migrate-027a6fbe-gw-openshift-default Scaled up replica set e2e-migrate-027a6fbe-gw-openshift-default-ccd69c4b5 from 0 to 1 54m Normal SuccessfulCreate replicaset/e2e-migrate-027a6fbe-gw-openshift-default-ccd69c4b5 Created pod: e2e-migrate-027a6fbe-gw-openshift-default-ccd69c4b5-stjc4 54m Normal AddedInterface pod/e2e-ait-bb95c832-openshift-default-997798bd5-x5tlh Add eth0 [10.133.0.128/23] from ovn-kubernetes 54m Normal AddedLabel gateway/e2e-migrate-027a6fbe-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-027a6fbe-gw 54m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl Startup probe error: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl Startup probe failed: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused 54m Normal AddedInterface pod/e2e-migrate-027a6fbe-gw-openshift-default-ccd69c4b5-stjc4 Add eth0 [10.132.0.86/23] from ovn-kubernetes 54m Warning ProbeError pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-p8vxh Startup probe error: Get "http://10.133.0.126:15021/healthz/ready": dial tcp 10.133.0.126:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-p8vxh Startup probe failed: Get "http://10.133.0.126:15021/healthz/ready": dial tcp 10.133.0.126:15021: connect: connection refused 54m Warning Unhealthy pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vpkm5 Startup probe failed: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vpkm5 Startup probe error: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused... 54m Warning ProbeError pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-krp2k Startup probe error: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-krp2k Startup probe failed: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-lk9lz Startup probe error: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-lk9lz Startup probe failed: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused 54m Warning Unhealthy pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-lk9lz Startup probe failed: Get "http://10.132.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning Unhealthy pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-krp2k Startup probe failed: Get "http://10.132.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning Unhealthy pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vpkm5 Startup probe failed: Get "http://10.132.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vpkm5 Startup probe error: Get "http://10.132.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning Unhealthy pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-wfg5p Readiness probe failed: Get "http://10.133.0.117:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-wfg5p Readiness probe error: Get "http://10.133.0.117:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning ProbeError pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-krp2k Startup probe error: Get "http://10.132.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning ProbeError pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-lk9lz Startup probe error: Get "http://10.132.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning ProbeError pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r69n5 Readiness probe error: Get "http://10.132.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning Unhealthy pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r69n5 Readiness probe failed: Get "http://10.132.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning ProbeError pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-mzhsz Readiness probe error: Get "http://10.133.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning Unhealthy pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-mzhsz Readiness probe failed: Get "http://10.133.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 54m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-hmb27 Add eth0 [10.132.0.87/23] from ovn-kubernetes 54m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-hmb27 54m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulCreate replicaset/e2e-ait-c082ab23-openshift-default-7f9b898b47 Created pod: e2e-ait-c082ab23-openshift-default-7f9b898b47-fpps2 54m Normal AddedLabel gateway/e2e-ait-c082ab23 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-c082ab23 54m Normal ScalingReplicaSet deployment/e2e-ait-c082ab23-openshift-default Scaled up replica set e2e-ait-c082ab23-openshift-default-7f9b898b47 from 0 to 1 54m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-hmb27 Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 54m Warning FailedCreate replicaset/payload-processing-e2e-ait-bb95c832-584887d746 Error creating: pods "payload-processing-e2e-ait-bb95c832-584887d746-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-bb95c832: serviceaccount "payload-processing-e2e-ait-bb95c832" not found 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-bb95c832-584887d746 Created pod: payload-processing-e2e-ait-bb95c832-584887d746-5cw9h 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-bb95c832 Scaled up replica set payload-processing-e2e-ait-bb95c832-584887d746 from 0 to 1 54m Normal ScalingReplicaSet deployment/e2e-ait-c6905007-openshift-default Scaled up replica set e2e-ait-c6905007-openshift-default-56c9f74765 from 0 to 1 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-bb95c832-766789c6f9 Error creating: pods "payload-pre-processing-e2e-ait-bb95c832-766789c6f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-bb95c832: serviceaccount "payload-processing-e2e-ait-bb95c832" not found 54m Normal AddedInterface pod/e2e-ait-c082ab23-openshift-default-7f9b898b47-fpps2 Add eth0 [10.133.0.132/23] from ovn-kubernetes 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-bb95c832 Scaled up replica set payload-pre-processing-e2e-ait-bb95c832-766789c6f9 from 0 to 1 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-bb95c832-766789c6f9 Created pod: payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q 54m Normal SuccessfulCreate replicaset/e2e-ait-c6905007-openshift-default-56c9f74765 Created pod: e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9 54m Normal AddedLabel gateway/e2e-ait-c6905007 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-c6905007 53m Normal AddedInterface pod/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q Add eth0 [10.134.0.81/23] from ovn-kubernetes 53m Normal Pulled pod/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Pulled pod/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Created pod/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h Container created 53m Normal Started pod/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h Container started 53m Normal AddedInterface pod/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h Add eth0 [10.134.0.80/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9 Add eth0 [10.134.0.82/23] from ovn-kubernetes 53m Normal Started pod/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q Container started 53m Normal Created pod/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q Container created 53m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-hmb27 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 5.473s (5.473s including waiting). Image size: 185476477 bytes. 53m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-hmb27 Container created 53m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-hmb27 Container started 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-027a6fbe Scaled up replica set payload-processing-e2e-migrate-027a6fbe-d6957d7c4 from 0 to 1 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-027a6fbe Scaled up replica set payload-pre-processing-e2e-migrate-027a6fbe-658459cfb7 from 0 to 1 53m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-hmb27 Readiness probe error: HTTP probe failed with statuscode: 503... 53m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-hmb27 Readiness probe failed: HTTP probe failed with statuscode: 503 53m Warning FailedCreate replicaset/payload-processing-e2e-migrate-027a6fbe-d6957d7c4 Error creating: pods "payload-processing-e2e-migrate-027a6fbe-d6957d7c4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-027a6fbe: serviceaccount "payload-processing-e2e-migrate-027a6fbe" not found 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-027a6fbe-658459cfb7 Created pod: payload-pre-processing-e2e-migrate-027a6fbe-658459cfb7-qg795 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-027a6fbe-658459cfb7 Error creating: pods "payload-pre-processing-e2e-migrate-027a6fbe-658459cfb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-027a6fbe: serviceaccount "payload-processing-e2e-migrate-027a6fbe" not found 53m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-027a6fbe-658459cfb7-qg795 Add eth0 [10.133.0.134/23] from ovn-kubernetes 53m Normal AddedInterface pod/payload-processing-e2e-migrate-027a6fbe-d6957d7c4-rprgn Add eth0 [10.133.0.135/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-027a6fbe-d6957d7c4 Created pod: payload-processing-e2e-migrate-027a6fbe-d6957d7c4-rprgn 53m Normal Started pod/payload-processing-e2e-migrate-027a6fbe-d6957d7c4-rprgn Container started 53m Normal Created pod/payload-pre-processing-e2e-migrate-027a6fbe-658459cfb7-qg795 Container created 53m Normal Pulled pod/payload-pre-processing-e2e-migrate-027a6fbe-658459cfb7-qg795 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Created pod/payload-processing-e2e-migrate-027a6fbe-d6957d7c4-rprgn Container created 53m Normal Started pod/payload-pre-processing-e2e-migrate-027a6fbe-658459cfb7-qg795 Container started 53m Normal Pulled pod/payload-processing-e2e-migrate-027a6fbe-d6957d7c4-rprgn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-c082ab23 Scaled up replica set payload-processing-e2e-ait-c082ab23-876b49bfb from 0 to 1 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-c082ab23 Scaled up replica set payload-pre-processing-e2e-ait-c082ab23-5df95f8c from 0 to 1 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-c082ab23-876b49bfb Created pod: payload-processing-e2e-ait-c082ab23-876b49bfb-6f94j 53m Normal AddedInterface pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7 Add eth0 [10.134.0.85/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 Created pod: e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-c082ab23-5df95f8c Error creating: pods "payload-pre-processing-e2e-ait-c082ab23-5df95f8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-c082ab23: serviceaccount "payload-processing-e2e-ait-c082ab23" not found 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 Created pod: e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-pfkck 53m Warning FailedCreate replicaset/payload-processing-e2e-ait-c082ab23-876b49bfb Error creating: pods "payload-processing-e2e-ait-c082ab23-876b49bfb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-c082ab23: serviceaccount "payload-processing-e2e-ait-c082ab23" not found 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 (combined from similar events): Created pod: e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-sxzfr 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-c082ab23-5df95f8c Created pod: payload-pre-processing-e2e-ait-c082ab23-5df95f8c-dvxqg 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 Created pod: e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-jxgb9 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6f1190-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w5-6f1190-openshift-default Scaled up replica set e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 from 6 to 10 53m Normal AddedInterface pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-pfkck Add eth0 [10.133.0.138/23] from ovn-kubernetes 53m Normal Created pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-pfkck Container created 53m Normal AddedInterface pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-sxzfr Add eth0 [10.133.0.139/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-sxzfr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-pfkck Container started 53m Normal Created pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-sxzfr Container created 53m Normal Started pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-sxzfr Container started 53m Normal Pulled pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-pfkck Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Pulled pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7 Container created 53m Normal Started pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7 Container started 53m Normal Started pod/payload-processing-e2e-ait-c082ab23-876b49bfb-6f94j Container started 53m Normal Created pod/payload-processing-e2e-ait-c082ab23-876b49bfb-6f94j Container created 53m Normal Pulled pod/payload-processing-e2e-ait-c082ab23-876b49bfb-6f94j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/payload-processing-e2e-ait-c082ab23-876b49bfb-6f94j Add eth0 [10.133.0.140/23] from ovn-kubernetes 53m Normal Started pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-jxgb9 Container started 53m Normal AddedInterface pod/payload-pre-processing-e2e-ait-c082ab23-5df95f8c-dvxqg Add eth0 [10.133.0.137/23] from ovn-kubernetes 53m Normal Created pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-jxgb9 Container created 53m Normal Pulled pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-jxgb9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Pulled pod/payload-pre-processing-e2e-ait-c082ab23-5df95f8c-dvxqg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Created pod/payload-pre-processing-e2e-ait-c082ab23-5df95f8c-dvxqg Container created 53m Normal Started pod/payload-pre-processing-e2e-ait-c082ab23-5df95f8c-dvxqg Container started 53m Normal AddedInterface pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-jxgb9 Add eth0 [10.132.0.88/23] from ovn-kubernetes 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-c6905007 Scaled up replica set payload-processing-e2e-ait-c6905007-65d4cdd474 from 0 to 1 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-c6905007-5d7cf6b467 Error creating: pods "payload-pre-processing-e2e-ait-c6905007-5d7cf6b467-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-c6905007: serviceaccount "payload-processing-e2e-ait-c6905007" not found 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-c6905007-5d7cf6b467 Created pod: payload-pre-processing-e2e-ait-c6905007-5d7cf6b467-8hdjj 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-c6905007 Scaled up replica set payload-pre-processing-e2e-ait-c6905007-5d7cf6b467 from 0 to 1 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-c6905007-65d4cdd474 Created pod: payload-processing-e2e-ait-c6905007-65d4cdd474-s4nr2 53m Warning FailedCreate replicaset/payload-processing-e2e-ait-c6905007-65d4cdd474 Error creating: pods "payload-processing-e2e-ait-c6905007-65d4cdd474-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-c6905007: serviceaccount "payload-processing-e2e-ait-c6905007" not found 53m Normal AddedInterface pod/payload-processing-e2e-ait-c6905007-65d4cdd474-s4nr2 Add eth0 [10.133.0.141/23] from ovn-kubernetes 53m Normal Pulled pod/payload-processing-e2e-ait-c6905007-65d4cdd474-s4nr2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Started pod/payload-processing-e2e-ait-c6905007-65d4cdd474-s4nr2 Container started 53m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-th2cc Readiness probe error: Get "http://10.133.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 53m Warning Unhealthy pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7 Startup probe failed: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7 Startup probe error: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-th2cc Readiness probe failed: Get "http://10.133.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 53m Warning Unhealthy pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-bpdgg Readiness probe failed: Get "http://10.132.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 53m Normal Created pod/payload-processing-e2e-ait-c6905007-65d4cdd474-s4nr2 Container created 53m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-bpdgg Readiness probe error: Get "http://10.132.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 53m Normal AddedInterface pod/payload-pre-processing-e2e-ait-c6905007-5d7cf6b467-8hdjj Add eth0 [10.133.0.142/23] from ovn-kubernetes 53m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl Readiness probe error: Get "http://10.134.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 53m Normal Pulled pod/payload-pre-processing-e2e-ait-c6905007-5d7cf6b467-8hdjj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Warning Unhealthy pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-jxgb9 Startup probe failed: Get "http://10.132.0.88:15021/healthz/ready": dial tcp 10.132.0.88:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl Readiness probe failed: Get "http://10.134.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 53m Warning ProbeError pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-jxgb9 Startup probe error: Get "http://10.132.0.88:15021/healthz/ready": dial tcp 10.132.0.88:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-jxgb9 Startup probe error: Get "http://10.132.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 53m Warning Unhealthy pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-jxgb9 Startup probe failed: Get "http://10.132.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 53m Normal Created pod/payload-pre-processing-e2e-ait-c6905007-5d7cf6b467-8hdjj Container created 53m Warning Unhealthy pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-sxzfr Startup probe failed: Get "http://10.133.0.139:15021/healthz/ready": dial tcp 10.133.0.139:15021: connect: connection refused 53m Normal Started pod/payload-pre-processing-e2e-ait-c6905007-5d7cf6b467-8hdjj Container started 53m Warning ProbeError pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-sxzfr Startup probe error: Get "http://10.133.0.139:15021/healthz/ready": dial tcp 10.133.0.139:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-pfkck Startup probe error: Get "http://10.133.0.138:15021/healthz/ready": dial tcp 10.133.0.138:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-pfkck Startup probe failed: Get "http://10.133.0.138:15021/healthz/ready": dial tcp 10.133.0.138:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-sxzfr Startup probe error: Get "http://10.133.0.139:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 53m Warning Unhealthy pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-sxzfr Startup probe failed: Get "http://10.133.0.139:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 53m Warning ProbeError pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-pfkck Startup probe error: Get "http://10.133.0.138:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 53m Normal Pulled pod/e2e-ait-bb95c832-openshift-default-997798bd5-x5tlh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-ait-bb95c832-openshift-default-997798bd5-x5tlh Container created 53m Normal Started pod/e2e-ait-bb95c832-openshift-default-997798bd5-x5tlh Container started 53m Warning ProbeError pod/e2e-ait-bb95c832-openshift-default-997798bd5-x5tlh Startup probe error: Get "http://10.133.0.128:15021/healthz/ready": dial tcp 10.133.0.128:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-ait-bb95c832-openshift-default-997798bd5-x5tlh Startup probe failed: Get "http://10.133.0.128:15021/healthz/ready": dial tcp 10.133.0.128:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-sxzfr Readiness probe error: Get "http://10.133.0.139:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-bb95c832-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-bb95c832-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Normal AddedLabel gateway/e2e-shared-a-w4-65cac0 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-65cac0 52m Normal Started pod/e2e-migrate-027a6fbe-gw-openshift-default-ccd69c4b5-stjc4 Container started 52m Normal Created pod/e2e-migrate-027a6fbe-gw-openshift-default-ccd69c4b5-stjc4 Container created 52m Normal Pulled pod/e2e-migrate-027a6fbe-gw-openshift-default-ccd69c4b5-stjc4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55 Add eth0 [10.134.0.87/23] from ovn-kubernetes 52m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-65cac0-openshift-default Scaled up replica set e2e-shared-a-w4-65cac0-openshift-default-575894bb79 from 0 to 1 52m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-65cac0-openshift-default-575894bb79 Created pod: e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55 52m Warning Unhealthy pod/e2e-migrate-027a6fbe-gw-openshift-default-ccd69c4b5-stjc4 Startup probe failed: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused 52m Warning ProbeError pod/e2e-migrate-027a6fbe-gw-openshift-default-ccd69c4b5-stjc4 Startup probe error: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused... 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w4-65cac0 Scaled up replica set payload-processing-e2e-shared-a-w4-65cac0-7bbdd5876 from 0 to 1 52m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w4-65cac0-7bbdd5876 Error creating: pods "payload-processing-e2e-shared-a-w4-65cac0-7bbdd5876-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-65cac0: serviceaccount "payload-processing-e2e-shared-a-w4-65cac0" not found 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w4-65cac0-7bbdd5876 Created pod: payload-processing-e2e-shared-a-w4-65cac0-7bbdd5876-c484q 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w4-65cac0 Scaled up replica set payload-pre-processing-e2e-shared-a-w4-65cac0-85d8c6fb67 from 0 to 1 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w4-65cac0-85d8c6fb67 Created pod: payload-pre-processing-e2e-shared-a-w4-65cac0-85d8c6fb67-ln66p 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w4-65cac0-85d8c6fb67 Error creating: pods "payload-pre-processing-e2e-shared-a-w4-65cac0-85d8c6fb67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-65cac0: serviceaccount "payload-processing-e2e-shared-a-w4-65cac0" not found 52m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w4-65cac0-85d8c6fb67-ln66p Add eth0 [10.133.0.152/23] from ovn-kubernetes 52m Normal AddedInterface pod/payload-processing-e2e-shared-a-w4-65cac0-7bbdd5876-c484q Add eth0 [10.133.0.151/23] from ovn-kubernetes 52m Normal Pulled pod/payload-processing-e2e-shared-a-w4-65cac0-7bbdd5876-c484q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Created pod/payload-processing-e2e-shared-a-w4-65cac0-7bbdd5876-c484q Container created 52m Normal Started pod/payload-processing-e2e-shared-a-w4-65cac0-7bbdd5876-c484q Container started 52m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w4-65cac0-85d8c6fb67-ln66p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Created pod/payload-pre-processing-e2e-shared-a-w4-65cac0-85d8c6fb67-ln66p Container created 52m Normal Started pod/payload-pre-processing-e2e-shared-a-w4-65cac0-85d8c6fb67-ln66p Container started 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-027a6fbe-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-027a6fbe-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Normal Killing pod/payload-pre-processing-e2e-ait-bb95c832-766789c6f9-7k98q Stopping container payload-pre-processing 52m Normal Killing pod/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h Stopping container payload-processing 52m Warning Unhealthy pod/e2e-ait-c082ab23-openshift-default-7f9b898b47-fpps2 Readiness probe failed: Get "http://10.133.0.132:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-ait-c082ab23-openshift-default-7f9b898b47-fpps2 Readiness probe error: Get "http://10.133.0.132:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/payload-processing-e2e-ait-bb95c832-584887d746-5cw9h Readiness probe failed: timeout: failed to connect service "10.134.0.80:50052" within 1s: context deadline exceeded 52m Warning ProbeError pod/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9 Readiness probe error: Get "http://10.134.0.82:15021/healthz/ready": read tcp 10.134.0.2:45700->10.134.0.82:15021: read: connection reset by peer... 52m Warning Unhealthy pod/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9 Readiness probe failed: Get "http://10.134.0.82:15021/healthz/ready": read tcp 10.134.0.2:45700->10.134.0.82:15021: read: connection reset by peer 52m Normal Pulled pod/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Started pod/e2e-ait-c082ab23-openshift-default-7f9b898b47-fpps2 Container started 52m Normal Started pod/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9 Container started 52m Normal Created pod/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9 Container created 52m Normal Pulled pod/e2e-ait-c082ab23-openshift-default-7f9b898b47-fpps2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-ait-c082ab23-openshift-default-7f9b898b47-fpps2 Container created 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-027a6fbe-gw-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-027a6fbe-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Normal Killing pod/payload-processing-e2e-migrate-027a6fbe-d6957d7c4-rprgn Stopping container payload-processing 52m Warning Unhealthy pod/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9 Startup probe failed: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused 52m Normal Killing pod/payload-pre-processing-e2e-migrate-027a6fbe-658459cfb7-qg795 Stopping container payload-pre-processing 52m Warning Unhealthy pod/e2e-ait-c082ab23-openshift-default-7f9b898b47-fpps2 Startup probe failed: Get "http://10.133.0.132:15021/healthz/ready": dial tcp 10.133.0.132:15021: connect: connection refused 52m Warning ProbeError pod/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9 Startup probe error: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-ait-c082ab23-openshift-default-7f9b898b47-fpps2 Startup probe error: Get "http://10.133.0.132:15021/healthz/ready": dial tcp 10.133.0.132:15021: connect: connection refused... 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-c082ab23-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-c082ab23-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-c6905007-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-c6905007-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-c082ab23-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-c082ab23-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-c6905007-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-c6905007-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Normal Killing pod/e2e-ait-bb95c832-openshift-default-997798bd5-x5tlh Stopping container istio-proxy 51m Normal AddedLabel gateway/e2e-derive-e7a91ca4 Added label istio.io/rev=openshift-gateway to gateway e2e-derive-e7a91ca4 51m Normal SuccessfulCreate replicaset/e2e-derive-e7a91ca4-openshift-default-59b8db5485 Created pod: e2e-derive-e7a91ca4-openshift-default-59b8db5485-7fm8v 51m Normal ScalingReplicaSet deployment/e2e-derive-e7a91ca4-openshift-default Scaled up replica set e2e-derive-e7a91ca4-openshift-default-59b8db5485 from 0 to 1 51m Normal AddedInterface pod/e2e-derive-e7a91ca4-openshift-default-59b8db5485-7fm8v Add eth0 [10.133.0.156/23] from ovn-kubernetes 51m Warning ProbeError pod/e2e-ait-bb95c832-openshift-default-997798bd5-x5tlh Readiness probe error: HTTP probe failed with statuscode: 503... 51m Normal AddedLabel gateway/e2e-mt-d18a5b05 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-d18a5b05 51m Normal SuccessfulCreate replicaset/e2e-mt-d18a5b05-openshift-default-7489c59bd9 Created pod: e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc 51m Warning Unhealthy pod/e2e-ait-bb95c832-openshift-default-997798bd5-x5tlh Readiness probe failed: HTTP probe failed with statuscode: 503 51m Normal ScalingReplicaSet deployment/e2e-mt-d18a5b05-openshift-default Scaled up replica set e2e-mt-d18a5b05-openshift-default-7489c59bd9 from 0 to 1 51m Normal AddedInterface pod/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc Add eth0 [10.134.0.89/23] from ovn-kubernetes 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-e7a91ca4 Scaled up replica set payload-pre-processing-e2e-derive-e7a91ca4-65b8c9ccf from 0 to 1 51m Warning FailedCreate replicaset/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d Error creating: pods "payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-e7a91ca4: serviceaccount "payload-processing-e2e-derive-e7a91ca4" not found 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-e7a91ca4-65b8c9ccf Error creating: pods "payload-pre-processing-e2e-derive-e7a91ca4-65b8c9ccf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-e7a91ca4: serviceaccount "payload-processing-e2e-derive-e7a91ca4" not found 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d Created pod: payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-e7a91ca4 Scaled up replica set payload-processing-e2e-derive-e7a91ca4-68b77b4f5d from 0 to 1 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-e7a91ca4-65b8c9ccf Created pod: payload-pre-processing-e2e-derive-e7a91ca4-65b8c9ccf-h87z2 51m Normal Pulled pod/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x Add eth0 [10.134.0.90/23] from ovn-kubernetes 51m Normal AddedInterface pod/payload-pre-processing-e2e-derive-e7a91ca4-65b8c9ccf-h87z2 Add eth0 [10.133.0.157/23] from ovn-kubernetes 51m Normal Started pod/payload-pre-processing-e2e-derive-e7a91ca4-65b8c9ccf-h87z2 Container started 51m Normal Pulled pod/payload-pre-processing-e2e-derive-e7a91ca4-65b8c9ccf-h87z2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Started pod/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x Container started 51m Normal Killing pod/e2e-migrate-027a6fbe-gw-openshift-default-ccd69c4b5-stjc4 Stopping container istio-proxy 51m Normal Created pod/payload-pre-processing-e2e-derive-e7a91ca4-65b8c9ccf-h87z2 Container created 51m Normal Created pod/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x Container created 51m Normal Killing pod/payload-processing-e2e-ait-c6905007-65d4cdd474-s4nr2 Stopping container payload-processing 51m Normal Killing pod/payload-pre-processing-e2e-ait-c6905007-5d7cf6b467-8hdjj Stopping container payload-pre-processing 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-d18a5b05-6bc8f78cd7 Error creating: pods "payload-pre-processing-e2e-mt-d18a5b05-6bc8f78cd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d18a5b05: serviceaccount "payload-processing-e2e-mt-d18a5b05" not found 51m Warning FailedCreate replicaset/payload-processing-e2e-mt-d18a5b05-95bc9bb4 Error creating: pods "payload-processing-e2e-mt-d18a5b05-95bc9bb4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d18a5b05: serviceaccount "payload-processing-e2e-mt-d18a5b05" not found 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-d18a5b05 Scaled up replica set payload-processing-e2e-mt-d18a5b05-95bc9bb4 from 0 to 1 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-d18a5b05 Scaled up replica set payload-pre-processing-e2e-mt-d18a5b05-6bc8f78cd7 from 0 to 1 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-d18a5b05-95bc9bb4 Created pod: payload-processing-e2e-mt-d18a5b05-95bc9bb4-9tn94 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-d18a5b05-6bc8f78cd7 Created pod: payload-pre-processing-e2e-mt-d18a5b05-6bc8f78cd7-xbnbd 51m Normal AddedInterface pod/payload-pre-processing-e2e-mt-d18a5b05-6bc8f78cd7-xbnbd Add eth0 [10.133.0.158/23] from ovn-kubernetes 51m Normal Killing pod/payload-processing-e2e-ait-c082ab23-876b49bfb-6f94j Stopping container payload-processing 51m Normal Pulled pod/payload-pre-processing-e2e-mt-d18a5b05-6bc8f78cd7-xbnbd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Killing pod/payload-pre-processing-e2e-ait-c082ab23-5df95f8c-dvxqg Stopping container payload-pre-processing 51m Normal AddedInterface pod/payload-processing-e2e-mt-d18a5b05-95bc9bb4-9tn94 Add eth0 [10.133.0.159/23] from ovn-kubernetes 51m Normal Started pod/payload-processing-e2e-mt-d18a5b05-95bc9bb4-9tn94 Container started 51m Normal Created pod/payload-pre-processing-e2e-mt-d18a5b05-6bc8f78cd7-xbnbd Container created 51m Normal Started pod/payload-pre-processing-e2e-mt-d18a5b05-6bc8f78cd7-xbnbd Container started 51m Normal Created pod/payload-processing-e2e-mt-d18a5b05-95bc9bb4-9tn94 Container created 51m Normal Pulled pod/payload-processing-e2e-mt-d18a5b05-95bc9bb4-9tn94 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 3 51m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-xkzbs 51m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: cpu resource utilization (percentage of request) above target 51m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-xkzbs Add eth0 [10.134.0.91/23] from ovn-kubernetes 51m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-xkzbs Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 51m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-xkzbs Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.796s (3.796s including waiting). Image size: 185476477 bytes. 51m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-xkzbs Container created 51m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-xkzbs Container started 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-d18a5b05-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-d18a5b05-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-xkzbs Readiness probe error: HTTP probe failed with statuscode: 503... 51m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-xkzbs Readiness probe failed: HTTP probe failed with statuscode: 503 50m Normal Killing pod/e2e-ait-c6905007-openshift-default-56c9f74765-zjsj9 Stopping container istio-proxy 50m Normal Killing pod/e2e-ait-c082ab23-openshift-default-7f9b898b47-fpps2 Stopping container istio-proxy 50m Normal SuccessfulCreate replicaset/e2e-mt-12450a06-openshift-default-7576d969f7 Created pod: e2e-mt-12450a06-openshift-default-7576d969f7-kh8pc 50m Normal AddedLabel gateway/e2e-mt-12450a06 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-12450a06 50m Normal ScalingReplicaSet deployment/e2e-mt-12450a06-openshift-default Scaled up replica set e2e-mt-12450a06-openshift-default-7576d969f7 from 0 to 1 50m Normal AddedInterface pod/e2e-mt-12450a06-openshift-default-7576d969f7-kh8pc Add eth0 [10.133.0.160/23] from ovn-kubernetes 50m Warning FailedCreate replicaset/payload-processing-e2e-mt-12450a06-5c4cc76cd6 Error creating: pods "payload-processing-e2e-mt-12450a06-5c4cc76cd6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-12450a06: serviceaccount "payload-processing-e2e-mt-12450a06" not found 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-12450a06 Scaled up replica set payload-processing-e2e-mt-12450a06-5c4cc76cd6 from 0 to 1 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-12450a06 Scaled up replica set payload-pre-processing-e2e-mt-12450a06-99d9f9f9 from 0 to 1 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-12450a06-99d9f9f9 Created pod: payload-pre-processing-e2e-mt-12450a06-99d9f9f9-5tt6k 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-12450a06-99d9f9f9 Error creating: pods "payload-pre-processing-e2e-mt-12450a06-99d9f9f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-12450a06: serviceaccount "payload-processing-e2e-mt-12450a06" not found 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-12450a06-5c4cc76cd6 Created pod: payload-processing-e2e-mt-12450a06-5c4cc76cd6-c9zgk 50m Normal Created pod/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-5tt6k Container created 50m Normal Started pod/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-5tt6k Container started 50m Normal Pulled pod/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-5tt6k Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-5tt6k Add eth0 [10.133.0.161/23] from ovn-kubernetes 50m Normal AddedInterface pod/payload-processing-e2e-mt-12450a06-5c4cc76cd6-c9zgk Add eth0 [10.133.0.162/23] from ovn-kubernetes 50m Normal Started pod/payload-processing-e2e-mt-12450a06-5c4cc76cd6-c9zgk Container started 50m Normal Pulled pod/payload-processing-e2e-mt-12450a06-5c4cc76cd6-c9zgk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Created pod/payload-processing-e2e-mt-12450a06-5c4cc76cd6-c9zgk Container created 50m Normal ScalingReplicaSet deployment/e2e-worker-w6-5e72ad-openshift-default Scaled down replica set e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c from 5 to 4 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-5e72ad-openshift-default New size: 4; reason: All metrics below target 50m Normal Killing pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-x5fmj Stopping container istio-proxy 50m Normal SuccessfulDelete replicaset/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c Deleted pod: e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-x5fmj 50m Normal AddedLabel gateway/e2e-shared-b-w4-a03fee Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-a03fee 50m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-a03fee-openshift-default Scaled up replica set e2e-shared-b-w4-a03fee-openshift-default-5f8f5cbff7 from 0 to 1 50m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-a03fee-openshift-default-5f8f5cbff7 Created pod: e2e-shared-b-w4-a03fee-openshift-default-5f8f5cbff7-pzmcj 50m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-vhkwc Container started 50m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-vhkwc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-shared-b-w4-a03fee-openshift-default-5f8f5cbff7-pzmcj Add eth0 [10.133.0.164/23] from ovn-kubernetes 50m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-k2jmm Add eth0 [10.132.0.97/23] from ovn-kubernetes 50m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-k2jmm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-k2jmm Container created 50m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-n5gnt Add eth0 [10.132.0.96/23] from ovn-kubernetes 50m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-vhkwc Add eth0 [10.133.0.163/23] from ovn-kubernetes 50m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-n5gnt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-k2jmm Container started 50m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-n5gnt Container started 50m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-n5gnt Container created 50m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-vhkwc Container created 50m Normal Pulled pod/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55 Container started 50m Normal Created pod/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55 Container created 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w4-a03fee Scaled up replica set payload-processing-e2e-shared-b-w4-a03fee-5d9bb89966 from 0 to 1 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-65cac0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w4-a03fee-5d9bb89966 Error creating: pods "payload-processing-e2e-shared-b-w4-a03fee-5d9bb89966-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-a03fee: serviceaccount "payload-processing-e2e-shared-b-w4-a03fee" not found 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-65cac0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w4-a03fee-5d9bb89966 Created pod: payload-processing-e2e-shared-b-w4-a03fee-5d9bb89966-kfsbz 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w4-a03fee-5595ccdb5f Error creating: pods "payload-pre-processing-e2e-shared-b-w4-a03fee-5595ccdb5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-a03fee: serviceaccount "payload-processing-e2e-shared-b-w4-a03fee" not found 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w4-a03fee-5595ccdb5f Created pod: payload-pre-processing-e2e-shared-b-w4-a03fee-5595ccdb5f-xgxjk 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w4-a03fee Scaled up replica set payload-pre-processing-e2e-shared-b-w4-a03fee-5595ccdb5f from 0 to 1 50m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-k2jmm Startup probe error: Get "http://10.132.0.97:15021/healthz/ready": dial tcp 10.132.0.97:15021: connect: connection refused... 50m Normal Created pod/payload-pre-processing-e2e-shared-b-w4-a03fee-5595ccdb5f-xgxjk Container created 50m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-n5gnt Startup probe failed: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused 50m Normal Pulled pod/payload-processing-e2e-shared-b-w4-a03fee-5d9bb89966-kfsbz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/payload-processing-e2e-shared-b-w4-a03fee-5d9bb89966-kfsbz Add eth0 [10.133.0.167/23] from ovn-kubernetes 50m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w4-a03fee-5595ccdb5f-xgxjk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Started pod/payload-pre-processing-e2e-shared-b-w4-a03fee-5595ccdb5f-xgxjk Container started 50m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-k2jmm Startup probe failed: Get "http://10.132.0.97:15021/healthz/ready": dial tcp 10.132.0.97:15021: connect: connection refused 50m Normal Created pod/payload-processing-e2e-shared-b-w4-a03fee-5d9bb89966-kfsbz Container created 50m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-n5gnt Startup probe error: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused... 50m Normal Started pod/payload-processing-e2e-shared-b-w4-a03fee-5d9bb89966-kfsbz Container started 50m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w4-a03fee-5595ccdb5f-xgxjk Add eth0 [10.133.0.168/23] from ovn-kubernetes 50m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-vhkwc Startup probe failed: Get "http://10.133.0.163:15021/healthz/ready": dial tcp 10.133.0.163:15021: connect: connection refused 50m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-vhkwc Startup probe error: Get "http://10.133.0.163:15021/healthz/ready": dial tcp 10.133.0.163:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55 Startup probe error: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55 Startup probe failed: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused 50m Normal Killing pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd Stopping container istio-proxy 50m Normal SuccessfulDelete replicaset/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c Deleted pod: e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-fbvvd 50m Normal ScalingReplicaSet deployment/e2e-worker-w6-5e72ad-openshift-default Scaled down replica set e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c from 4 to 3 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-5e72ad-openshift-default New size: 3; reason: All metrics below target 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-a03fee-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-a03fee-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-rlkdl Readiness probe error: Get "http://10.133.0.35:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-rlkdl Readiness probe failed: Get "http://10.133.0.35:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-65cac0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-65cac0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning ProbeError pod/e2e-derive-e7a91ca4-openshift-default-59b8db5485-7fm8v Readiness probe error: Get "http://10.133.0.156:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/e2e-derive-e7a91ca4-openshift-default-59b8db5485-7fm8v Readiness probe failed: Get "http://10.133.0.156:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc Readiness probe error: Get "http://10.134.0.89:15021/healthz/ready": read tcp 10.134.0.2:32872->10.134.0.89:15021: read: connection reset by peer... 49m Warning Unhealthy pod/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc Readiness probe failed: Get "http://10.134.0.89:15021/healthz/ready": read tcp 10.134.0.2:32872->10.134.0.89:15021: read: connection reset by peer 49m Normal Created pod/e2e-derive-e7a91ca4-openshift-default-59b8db5485-7fm8v Container created 49m Normal Started pod/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc Container started 49m Normal Pulled pod/e2e-derive-e7a91ca4-openshift-default-59b8db5485-7fm8v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Pulled pod/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-derive-e7a91ca4-openshift-default-59b8db5485-7fm8v Container started 49m Normal Created pod/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc Container created 49m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-59f957fdd7 (combined from similar events): Created pod: maas-default-gateway-openshift-default-59f957fdd7-ddv45 49m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-f64cj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-f64cj Add eth0 [10.134.0.93/23] from ovn-kubernetes 49m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-f64cj Container created 49m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-xmfvp Container created 49m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-fkwnw Container started 49m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-ddv45 Container created 49m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-ddv45 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-ddv45 Container started 49m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-f64cj Container started 49m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-xmfvp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-ddv45 Add eth0 [10.132.0.99/23] from ovn-kubernetes 49m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-xmfvp Container started 49m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-fkwnw Container created 49m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-xmfvp Add eth0 [10.133.0.169/23] from ovn-kubernetes 49m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-fkwnw Add eth0 [10.132.0.98/23] from ovn-kubernetes 49m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-fkwnw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Warning Unhealthy pod/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc Startup probe failed: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-derive-e7a91ca4-openshift-default-59b8db5485-7fm8v Startup probe failed: Get "http://10.133.0.156:15021/healthz/ready": dial tcp 10.133.0.156:15021: connect: connection refused 49m Warning ProbeError pod/e2e-derive-e7a91ca4-openshift-default-59b8db5485-7fm8v Startup probe error: Get "http://10.133.0.156:15021/healthz/ready": dial tcp 10.133.0.156:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc Startup probe error: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused... 49m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-ddv45 Startup probe error: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused... 49m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-xmfvp Startup probe failed: Get "http://10.133.0.169:15021/healthz/ready": dial tcp 10.133.0.169:15021: connect: connection refused 49m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-xmfvp Startup probe error: Get "http://10.133.0.169:15021/healthz/ready": dial tcp 10.133.0.169:15021: connect: connection refused... 49m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-ddv45 Startup probe failed: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused 49m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-f64cj Startup probe error: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused... 49m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-f64cj Startup probe failed: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused 49m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-fkwnw Startup probe failed: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused 49m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-fkwnw Startup probe error: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused... 49m Normal Killing pod/payload-pre-processing-e2e-derive-e7a91ca4-65b8c9ccf-h87z2 Stopping container payload-pre-processing 49m Normal Killing pod/payload-processing-e2e-derive-e7a91ca4-68b77b4f5d-fwj4x Stopping container payload-processing 49m Normal ScalingReplicaSet deployment/e2e-worker-w3-961054-openshift-default Scaled down replica set e2e-worker-w3-961054-openshift-default-5d5665c66f from 6 to 4 49m Normal SuccessfulDelete replicaset/e2e-worker-w3-961054-openshift-default-5d5665c66f Deleted pod: e2e-worker-w3-961054-openshift-default-5d5665c66f-w8xsc 49m Normal SuccessfulDelete replicaset/e2e-worker-w3-961054-openshift-default-5d5665c66f Deleted pod: e2e-worker-w3-961054-openshift-default-5d5665c66f-vw6ft 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-961054-openshift-default New size: 4; reason: All metrics below target 49m Normal Killing pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-vw6ft Stopping container istio-proxy 49m Normal Killing pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-w8xsc Stopping container istio-proxy 49m Warning ProbeError pod/e2e-mt-12450a06-openshift-default-7576d969f7-kh8pc Readiness probe error: Get "http://10.133.0.160:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/e2e-mt-12450a06-openshift-default-7576d969f7-kh8pc Readiness probe failed: Get "http://10.133.0.160:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Normal Started pod/e2e-mt-12450a06-openshift-default-7576d969f7-kh8pc Container started 49m Normal Created pod/e2e-mt-12450a06-openshift-default-7576d969f7-kh8pc Container created 49m Normal Pulled pod/e2e-mt-12450a06-openshift-default-7576d969f7-kh8pc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Warning Unhealthy pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-w8xsc Readiness probe failed: Get "http://10.133.0.93:15021/healthz/ready": dial tcp 10.133.0.93:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-w8xsc Readiness probe error: Get "http://10.133.0.93:15021/healthz/ready": dial tcp 10.133.0.93:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-mt-12450a06-openshift-default-7576d969f7-kh8pc Startup probe failed: Get "http://10.133.0.160:15021/healthz/ready": dial tcp 10.133.0.160:15021: connect: connection refused 49m Warning ProbeError pod/e2e-mt-12450a06-openshift-default-7576d969f7-kh8pc Startup probe error: Get "http://10.133.0.160:15021/healthz/ready": dial tcp 10.133.0.160:15021: connect: connection refused... 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-d18a5b05-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-d18a5b05-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-12450a06-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-12450a06-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-e7a91ca4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-e7a91ca4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-fb4f0a-openshift-default Scaled up replica set e2e-shared-a-w2-fb4f0a-openshift-default-7b7c76784f from 0 to 1 48m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-fb4f0a-openshift-default-7b7c76784f Created pod: e2e-shared-a-w2-fb4f0a-openshift-default-7b7c76784f-ldhbt 48m Normal AddedLabel gateway/e2e-shared-a-w2-fb4f0a Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-fb4f0a 48m Normal AddedInterface pod/e2e-shared-a-w2-fb4f0a-openshift-default-7b7c76784f-ldhbt Add eth0 [10.133.0.171/23] from ovn-kubernetes 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-fb4f0a Scaled up replica set payload-pre-processing-e2e-shared-a-w2-fb4f0a-7f6d59c45f from 0 to 1 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-fb4f0a Scaled up replica set payload-processing-e2e-shared-a-w2-fb4f0a-6fdc7bbbfd from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-fb4f0a-7f6d59c45f Created pod: payload-pre-processing-e2e-shared-a-w2-fb4f0a-7f6d59c45f-gpbjr 48m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-fb4f0a-6fdc7bbbfd Error creating: pods "payload-processing-e2e-shared-a-w2-fb4f0a-6fdc7bbbfd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-fb4f0a: serviceaccount "payload-processing-e2e-shared-a-w2-fb4f0a" not found 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-fb4f0a-6fdc7bbbfd Created pod: payload-processing-e2e-shared-a-w2-fb4f0a-6fdc7bbbfd-m9229 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-fb4f0a-7f6d59c45f Error creating: pods "payload-pre-processing-e2e-shared-a-w2-fb4f0a-7f6d59c45f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-fb4f0a: serviceaccount "payload-processing-e2e-shared-a-w2-fb4f0a" not found 48m Normal Created pod/payload-processing-e2e-shared-a-w2-fb4f0a-6fdc7bbbfd-m9229 Container created 48m Normal Started pod/payload-processing-e2e-shared-a-w2-fb4f0a-6fdc7bbbfd-m9229 Container started 48m Normal Pulled pod/payload-processing-e2e-shared-a-w2-fb4f0a-6fdc7bbbfd-m9229 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-fb4f0a-6fdc7bbbfd-m9229 Add eth0 [10.133.0.173/23] from ovn-kubernetes 48m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-fb4f0a-7f6d59c45f-gpbjr Add eth0 [10.133.0.174/23] from ovn-kubernetes 48m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-fb4f0a-7f6d59c45f-gpbjr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-fb4f0a-7f6d59c45f-gpbjr Container started 48m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-fb4f0a-7f6d59c45f-gpbjr Container created 48m Normal Killing pod/e2e-derive-e7a91ca4-openshift-default-59b8db5485-7fm8v Stopping container istio-proxy 48m Normal AddedLabel gateway/e2e-shared-a-w6-6a3169 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w6-6a3169 48m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-6a3169-openshift-default Scaled up replica set e2e-shared-a-w6-6a3169-openshift-default-58d9474d87 from 0 to 1 48m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-6a3169-openshift-default-58d9474d87 Created pod: e2e-shared-a-w6-6a3169-openshift-default-58d9474d87-gztml 48m Normal AddedInterface pod/e2e-shared-a-w6-6a3169-openshift-default-58d9474d87-gztml Add eth0 [10.133.0.175/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-shared-b-w4-a03fee-openshift-default-5f8f5cbff7-pzmcj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-shared-b-w4-a03fee-openshift-default-5f8f5cbff7-pzmcj Container started 48m Normal Created pod/e2e-shared-b-w4-a03fee-openshift-default-5f8f5cbff7-pzmcj Container created 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-12450a06-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-12450a06-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Normal SuccessfulDelete replicaset/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 Deleted pod: e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-krp2k 48m Normal Killing pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-rj8x7 Stopping container istio-proxy 48m Normal ScalingReplicaSet deployment/e2e-worker-w2-ee74cb-openshift-default Scaled down replica set e2e-worker-w2-ee74cb-openshift-default-9758876f from 10 to 6 48m Normal SuccessfulDelete replicaset/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 Deleted pod: e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-c6fst 48m Normal Killing pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-krp2k Stopping container istio-proxy 48m Normal Killing pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-lk9lz Stopping container istio-proxy 48m Normal Killing pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-qnfhf Stopping container istio-proxy 48m Normal Killing pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-qkcnq Stopping container istio-proxy 48m Normal Killing pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-c6fst Stopping container istio-proxy 48m Normal SuccessfulDelete replicaset/e2e-worker-w2-ee74cb-openshift-default-9758876f Deleted pod: e2e-worker-w2-ee74cb-openshift-default-9758876f-rj8x7 48m Normal SuccessfulDelete replicaset/e2e-worker-w2-ee74cb-openshift-default-9758876f Deleted pod: e2e-worker-w2-ee74cb-openshift-default-9758876f-qkcnq 48m Normal SuccessfulDelete replicaset/e2e-worker-w2-ee74cb-openshift-default-9758876f Deleted pod: e2e-worker-w2-ee74cb-openshift-default-9758876f-qnfhf 48m Normal SuccessfulDelete replicaset/e2e-worker-w2-ee74cb-openshift-default-9758876f Deleted pod: e2e-worker-w2-ee74cb-openshift-default-9758876f-k46v6 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-ee74cb-openshift-default New size: 6; reason: All metrics below target 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-68fbd1-openshift-default New size: 6; reason: All metrics below target 48m Normal Killing pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-hjqrh Stopping container istio-proxy 48m Normal SuccessfulDelete replicaset/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 Deleted pod: e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-hjqrh 48m Normal SuccessfulDelete replicaset/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 Deleted pod: e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-lk9lz 48m Normal Killing pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-k46v6 Stopping container istio-proxy 48m Normal ScalingReplicaSet deployment/e2e-worker-w0-68fbd1-openshift-default Scaled down replica set e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 from 10 to 6 48m Warning ProbeError pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-krp2k Readiness probe error: HTTP probe failed with statuscode: 503... 48m Warning Unhealthy pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-rj8x7 Readiness probe failed: HTTP probe failed with statuscode: 503 48m Warning ProbeError pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-rj8x7 Readiness probe error: HTTP probe failed with statuscode: 503... 48m Warning ProbeError pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-lk9lz Readiness probe error: HTTP probe failed with statuscode: 503... 48m Warning Unhealthy pod/e2e-shared-b-w4-a03fee-openshift-default-5f8f5cbff7-pzmcj Startup probe failed: Get "http://10.133.0.164:15021/healthz/ready": dial tcp 10.133.0.164:15021: connect: connection refused 48m Warning ProbeError pod/e2e-shared-b-w4-a03fee-openshift-default-5f8f5cbff7-pzmcj Startup probe error: Get "http://10.133.0.164:15021/healthz/ready": dial tcp 10.133.0.164:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-qnfhf Readiness probe failed: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-qkcnq Readiness probe error: Get "http://10.133.0.99:15021/healthz/ready": dial tcp 10.133.0.99:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-qnfhf Readiness probe error: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-qkcnq Readiness probe failed: Get "http://10.133.0.99:15021/healthz/ready": dial tcp 10.133.0.99:15021: connect: connection refused 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w6-6a3169 Scaled up replica set payload-processing-e2e-shared-a-w6-6a3169-866cf6cb6d from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w6-6a3169-866cf6cb6d Created pod: payload-processing-e2e-shared-a-w6-6a3169-866cf6cb6d-t7spd 48m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w6-6a3169-866cf6cb6d Error creating: pods "payload-processing-e2e-shared-a-w6-6a3169-866cf6cb6d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-6a3169: serviceaccount "payload-processing-e2e-shared-a-w6-6a3169" not found 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w6-6a3169-67b8ff4bb7 Error creating: pods "payload-pre-processing-e2e-shared-a-w6-6a3169-67b8ff4bb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-6a3169: serviceaccount "payload-processing-e2e-shared-a-w6-6a3169" not found 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w6-6a3169 Scaled up replica set payload-pre-processing-e2e-shared-a-w6-6a3169-67b8ff4bb7 from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w6-6a3169-67b8ff4bb7 Created pod: payload-pre-processing-e2e-shared-a-w6-6a3169-67b8ff4bb7-k6wb9 48m Normal Created pod/payload-processing-e2e-shared-a-w6-6a3169-866cf6cb6d-t7spd Container created 48m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w6-6a3169-67b8ff4bb7-k6wb9 Add eth0 [10.133.0.178/23] from ovn-kubernetes 48m Normal Pulled pod/payload-processing-e2e-shared-a-w6-6a3169-866cf6cb6d-t7spd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Started pod/payload-pre-processing-e2e-shared-a-w6-6a3169-67b8ff4bb7-k6wb9 Container started 48m Normal Started pod/payload-processing-e2e-shared-a-w6-6a3169-866cf6cb6d-t7spd Container started 48m Normal AddedInterface pod/payload-processing-e2e-shared-a-w6-6a3169-866cf6cb6d-t7spd Add eth0 [10.133.0.177/23] from ovn-kubernetes 48m Normal Created pod/payload-pre-processing-e2e-shared-a-w6-6a3169-67b8ff4bb7-k6wb9 Container created 48m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w6-6a3169-67b8ff4bb7-k6wb9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal ScalingReplicaSet deployment/e2e-api-a-0c21c7-openshift-default Scaled up replica set e2e-api-a-0c21c7-openshift-default-8476d54f65 from 0 to 1 48m Normal SuccessfulCreate replicaset/e2e-api-a-0c21c7-openshift-default-8476d54f65 Created pod: e2e-api-a-0c21c7-openshift-default-8476d54f65-ph5gh 48m Normal AddedLabel gateway/e2e-api-a-0c21c7 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-0c21c7 48m Normal AddedInterface pod/e2e-api-a-0c21c7-openshift-default-8476d54f65-ph5gh Add eth0 [10.133.0.179/23] from ovn-kubernetes 48m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-fkwnw Readiness probe failed: Get "http://10.132.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-ddv45 Readiness probe failed: Get "http://10.132.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-ddv45 Readiness probe error: Get "http://10.132.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-fkwnw Readiness probe error: Get "http://10.132.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-0c21c7-dd7b5dd55 Created pod: payload-pre-processing-e2e-api-a-0c21c7-dd7b5dd55-qzcqr 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-0c21c7 Scaled up replica set payload-processing-e2e-api-a-0c21c7-884dbcbfd from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-0c21c7-884dbcbfd Created pod: payload-processing-e2e-api-a-0c21c7-884dbcbfd-vcsmv 48m Warning FailedCreate replicaset/payload-processing-e2e-api-a-0c21c7-884dbcbfd Error creating: pods "payload-processing-e2e-api-a-0c21c7-884dbcbfd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-0c21c7: serviceaccount "payload-processing-e2e-api-a-0c21c7" not found 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-0c21c7-dd7b5dd55 Error creating: pods "payload-pre-processing-e2e-api-a-0c21c7-dd7b5dd55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-0c21c7: serviceaccount "payload-processing-e2e-api-a-0c21c7" not found 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-0c21c7 Scaled up replica set payload-pre-processing-e2e-api-a-0c21c7-dd7b5dd55 from 0 to 1 48m Normal Created pod/payload-processing-e2e-api-a-0c21c7-884dbcbfd-vcsmv Container created 48m Normal Started pod/payload-processing-e2e-api-a-0c21c7-884dbcbfd-vcsmv Container started 48m Normal Pulled pod/payload-processing-e2e-api-a-0c21c7-884dbcbfd-vcsmv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-processing-e2e-api-a-0c21c7-884dbcbfd-vcsmv Add eth0 [10.133.0.180/23] from ovn-kubernetes 48m Normal Pulled pod/payload-pre-processing-e2e-api-a-0c21c7-dd7b5dd55-qzcqr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-0c21c7-dd7b5dd55-qzcqr Add eth0 [10.133.0.181/23] from ovn-kubernetes 48m Normal Started pod/payload-pre-processing-e2e-api-a-0c21c7-dd7b5dd55-qzcqr Container started 48m Normal Created pod/payload-pre-processing-e2e-api-a-0c21c7-dd7b5dd55-qzcqr Container created 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-a03fee-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-a03fee-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Normal SuccessfulDelete replicaset/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c Deleted pod: e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vpkm5 47m Normal ScalingReplicaSet deployment/e2e-worker-w1-545b5d-openshift-default Scaled down replica set e2e-worker-w1-545b5d-openshift-default-5c8c66f65c from 10 to 6 47m Normal Killing pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5z5wk Stopping container istio-proxy 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-545b5d-openshift-default New size: 6; reason: All metrics below target 47m Normal SuccessfulDelete replicaset/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c Deleted pod: e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-72tpd 47m Normal Killing pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-72tpd Stopping container istio-proxy 47m Normal Killing pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-bpdgg Stopping container istio-proxy 47m Normal SuccessfulDelete replicaset/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c Deleted pod: e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5z5wk 47m Normal SuccessfulDelete replicaset/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c Deleted pod: e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-bpdgg 47m Normal Killing pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vpkm5 Stopping container istio-proxy 47m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5z5wk Readiness probe error: HTTP probe failed with statuscode: 503... 47m Warning Unhealthy pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5z5wk Readiness probe failed: HTTP probe failed with statuscode: 503 47m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vpkm5 Readiness probe error: HTTP probe failed with statuscode: 503... 47m Warning Unhealthy pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-bpdgg Readiness probe failed: HTTP probe failed with statuscode: 503 47m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-bpdgg Readiness probe error: HTTP probe failed with statuscode: 503... 47m Normal Killing pod/payload-pre-processing-e2e-mt-d18a5b05-6bc8f78cd7-xbnbd Stopping container payload-pre-processing 47m Normal Killing pod/payload-processing-e2e-mt-d18a5b05-95bc9bb4-9tn94 Stopping container payload-processing 47m Normal Killing pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-dn59n Stopping container istio-proxy 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-18c03a-openshift-default New size: 6; reason: All metrics below target 47m Normal SuccessfulDelete replicaset/e2e-worker-w4-18c03a-openshift-default-589bbc88df Deleted pod: e2e-worker-w4-18c03a-openshift-default-589bbc88df-ff7t5 47m Normal Killing pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-8kw9p Stopping container istio-proxy 47m Normal SuccessfulDelete replicaset/e2e-worker-w4-18c03a-openshift-default-589bbc88df Deleted pod: e2e-worker-w4-18c03a-openshift-default-589bbc88df-dn59n 47m Normal SuccessfulDelete replicaset/e2e-worker-w4-18c03a-openshift-default-589bbc88df Deleted pod: e2e-worker-w4-18c03a-openshift-default-589bbc88df-r69n5 47m Normal ScalingReplicaSet deployment/e2e-worker-w4-18c03a-openshift-default Scaled down replica set e2e-worker-w4-18c03a-openshift-default-589bbc88df from 10 to 6 47m Normal SuccessfulDelete replicaset/e2e-worker-w4-18c03a-openshift-default-589bbc88df Deleted pod: e2e-worker-w4-18c03a-openshift-default-589bbc88df-8kw9p 47m Normal Killing pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r69n5 Stopping container istio-proxy 47m Normal Killing pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-ff7t5 Stopping container istio-proxy 47m Warning ProbeError pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r69n5 Readiness probe error: HTTP probe failed with statuscode: 503... 47m Warning Unhealthy pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r69n5 Readiness probe failed: HTTP probe failed with statuscode: 503 47m Normal SuccessfulDelete replicaset/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 Deleted pod: e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-2qsk9 47m Normal Killing pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-2qsk9 Stopping container istio-proxy 47m Normal SuccessfulDelete replicaset/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 Deleted pod: e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-sxzfr 47m Normal Killing pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-sxzfr Stopping container istio-proxy 47m Normal SuccessfulDelete replicaset/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 Deleted pod: e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-pfkck 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6f1190-openshift-default New size: 7; reason: All metrics below target 47m Normal ScalingReplicaSet deployment/e2e-worker-w5-6f1190-openshift-default Scaled down replica set e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 from 10 to 7 47m Normal Killing pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-pfkck Stopping container istio-proxy 47m Normal Killing pod/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc Stopping container istio-proxy 47m Normal Started pod/e2e-shared-a-w2-fb4f0a-openshift-default-7b7c76784f-ldhbt Container started 47m Normal Pulled pod/e2e-shared-a-w2-fb4f0a-openshift-default-7b7c76784f-ldhbt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Warning ProbeError pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-2qsk9 Readiness probe error: HTTP probe failed with statuscode: 503... 47m Warning Unhealthy pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-2qsk9 Readiness probe failed: HTTP probe failed with statuscode: 503 47m Normal Created pod/e2e-shared-a-w2-fb4f0a-openshift-default-7b7c76784f-ldhbt Container created 47m Normal AddedLabel gateway/e2e-api-a-2627c5 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-2627c5 47m Normal ScalingReplicaSet deployment/e2e-api-a-2627c5-openshift-default Scaled up replica set e2e-api-a-2627c5-openshift-default-6c84fbdb55 from 0 to 1 47m Warning Unhealthy pod/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc Readiness probe failed: HTTP probe failed with statuscode: 503 47m Warning ProbeError pod/e2e-mt-d18a5b05-openshift-default-7489c59bd9-7ssjc Readiness probe error: HTTP probe failed with statuscode: 503... 47m Normal SuccessfulCreate replicaset/e2e-api-a-2627c5-openshift-default-6c84fbdb55 Created pod: e2e-api-a-2627c5-openshift-default-6c84fbdb55-5g5zm 47m Normal AddedInterface pod/e2e-api-a-2627c5-openshift-default-6c84fbdb55-5g5zm Add eth0 [10.132.0.102/23] from ovn-kubernetes 47m Warning Unhealthy pod/e2e-shared-a-w2-fb4f0a-openshift-default-7b7c76784f-ldhbt Startup probe failed: Get "http://10.133.0.171:15021/healthz/ready": dial tcp 10.133.0.171:15021: connect: connection refused 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-fb4f0a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-fb4f0a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning ProbeError pod/e2e-shared-a-w2-fb4f0a-openshift-default-7b7c76784f-ldhbt Startup probe error: Get "http://10.133.0.171:15021/healthz/ready": dial tcp 10.133.0.171:15021: connect: connection refused... 47m Normal ScalingReplicaSet deployment/e2e-inf-a-2420f6-openshift-default Scaled up replica set e2e-inf-a-2420f6-openshift-default-75c86ff8ff from 0 to 1 47m Normal AddedLabel gateway/e2e-inf-a-2420f6 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-2420f6 47m Normal Killing pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc Stopping container istio-proxy 47m Normal SuccessfulCreate replicaset/e2e-inf-a-2420f6-openshift-default-75c86ff8ff Created pod: e2e-inf-a-2420f6-openshift-default-75c86ff8ff-ltdfl 47m Normal ScalingReplicaSet deployment/e2e-worker-w5-6f1190-openshift-default Scaled down replica set e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 from 7 to 6 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-2627c5 Scaled up replica set payload-pre-processing-e2e-api-a-2627c5-86d7c64d59 from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-2627c5-86d7c64d59 Created pod: payload-pre-processing-e2e-api-a-2627c5-86d7c64d59-2l69t 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-2627c5 Scaled up replica set payload-processing-e2e-api-a-2627c5-56b96dd67b from 0 to 1 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-2627c5-86d7c64d59 Error creating: pods "payload-pre-processing-e2e-api-a-2627c5-86d7c64d59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-2627c5: serviceaccount "payload-processing-e2e-api-a-2627c5" not found 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-2627c5-56b96dd67b Created pod: payload-processing-e2e-api-a-2627c5-56b96dd67b-vqv5f 47m Warning FailedCreate replicaset/payload-processing-e2e-api-a-2627c5-56b96dd67b Error creating: pods "payload-processing-e2e-api-a-2627c5-56b96dd67b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-2627c5: serviceaccount "payload-processing-e2e-api-a-2627c5" not found 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6f1190-openshift-default New size: 6; reason: All metrics below target 47m Normal SuccessfulDelete replicaset/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 Deleted pod: e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc 47m Normal Created pod/payload-processing-e2e-api-a-2627c5-56b96dd67b-vqv5f Container created 47m Normal Pulled pod/payload-processing-e2e-api-a-2627c5-56b96dd67b-vqv5f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Pulled pod/payload-pre-processing-e2e-api-a-2627c5-86d7c64d59-2l69t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/payload-processing-e2e-api-a-2627c5-56b96dd67b-vqv5f Add eth0 [10.133.0.184/23] from ovn-kubernetes 47m Normal Started pod/payload-processing-e2e-api-a-2627c5-56b96dd67b-vqv5f Container started 47m Normal Created pod/payload-pre-processing-e2e-api-a-2627c5-86d7c64d59-2l69t Container created 47m Normal Started pod/payload-pre-processing-e2e-api-a-2627c5-86d7c64d59-2l69t Container started 47m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-2627c5-86d7c64d59-2l69t Add eth0 [10.133.0.183/23] from ovn-kubernetes 47m Normal AddedInterface pod/e2e-inf-a-2420f6-openshift-default-75c86ff8ff-ltdfl Add eth0 [10.132.0.103/23] from ovn-kubernetes 47m Warning Unhealthy pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc Readiness probe failed: HTTP probe failed with statuscode: 503 47m Warning ProbeError pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-4pqhc Readiness probe error: HTTP probe failed with statuscode: 503... 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-2420f6 Scaled up replica set payload-processing-e2e-inf-a-2420f6-b68c89c6 from 0 to 1 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-2420f6 Scaled up replica set payload-pre-processing-e2e-inf-a-2420f6-cb5bf5df5 from 0 to 1 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-2420f6-cb5bf5df5 Error creating: pods "payload-pre-processing-e2e-inf-a-2420f6-cb5bf5df5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-2420f6: serviceaccount "payload-processing-e2e-inf-a-2420f6" not found 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-2420f6-b68c89c6 Created pod: payload-processing-e2e-inf-a-2420f6-b68c89c6-shx8b 46m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-2420f6-b68c89c6 Error creating: pods "payload-processing-e2e-inf-a-2420f6-b68c89c6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-2420f6: serviceaccount "payload-processing-e2e-inf-a-2420f6" not found 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-2420f6-cb5bf5df5 Created pod: payload-pre-processing-e2e-inf-a-2420f6-cb5bf5df5-9td4b 46m Normal Created pod/payload-processing-e2e-inf-a-2420f6-b68c89c6-shx8b Container created 46m Normal Pulled pod/payload-processing-e2e-inf-a-2420f6-b68c89c6-shx8b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Started pod/payload-processing-e2e-inf-a-2420f6-b68c89c6-shx8b Container started 46m Normal AddedInterface pod/payload-processing-e2e-inf-a-2420f6-b68c89c6-shx8b Add eth0 [10.133.0.185/23] from ovn-kubernetes 46m Normal Pulled pod/payload-pre-processing-e2e-inf-a-2420f6-cb5bf5df5-9td4b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-2420f6-cb5bf5df5-9td4b Add eth0 [10.133.0.186/23] from ovn-kubernetes 46m Normal Created pod/payload-pre-processing-e2e-inf-a-2420f6-cb5bf5df5-9td4b Container created 46m Normal Started pod/payload-pre-processing-e2e-inf-a-2420f6-cb5bf5df5-9td4b Container started 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-2420f6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-2420f6-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-2627c5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-2627c5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal ScalingReplicaSet deployment/e2e-api-a-922795-openshift-default Scaled up replica set e2e-api-a-922795-openshift-default-59b46784df from 0 to 1 46m Normal SuccessfulCreate replicaset/e2e-api-a-922795-openshift-default-59b46784df Created pod: e2e-api-a-922795-openshift-default-59b46784df-6bc47 46m Normal AddedLabel gateway/e2e-api-a-922795 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-922795 46m Normal AddedInterface pod/e2e-api-a-922795-openshift-default-59b46784df-6bc47 Add eth0 [10.132.0.105/23] from ovn-kubernetes 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-fb4f0a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-fb4f0a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal Created pod/e2e-shared-a-w6-6a3169-openshift-default-58d9474d87-gztml Container created 46m Normal Created pod/e2e-api-a-0c21c7-openshift-default-8476d54f65-ph5gh Container created 46m Normal Started pod/e2e-shared-a-w6-6a3169-openshift-default-58d9474d87-gztml Container started 46m Normal Started pod/e2e-api-a-0c21c7-openshift-default-8476d54f65-ph5gh Container started 46m Normal Pulled pod/e2e-shared-a-w6-6a3169-openshift-default-58d9474d87-gztml Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Pulled pod/e2e-api-a-0c21c7-openshift-default-8476d54f65-ph5gh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-922795-559bb85d47 Error creating: pods "payload-pre-processing-e2e-api-a-922795-559bb85d47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-922795: serviceaccount "payload-processing-e2e-api-a-922795" not found 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-922795 Scaled up replica set payload-pre-processing-e2e-api-a-922795-559bb85d47 from 0 to 1 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-922795 Scaled up replica set payload-processing-e2e-api-a-922795-56d4c4f7db from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-922795-56d4c4f7db Created pod: payload-processing-e2e-api-a-922795-56d4c4f7db-hjv42 46m Warning FailedCreate replicaset/payload-processing-e2e-api-a-922795-56d4c4f7db Error creating: pods "payload-processing-e2e-api-a-922795-56d4c4f7db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-922795: serviceaccount "payload-processing-e2e-api-a-922795" not found 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-922795-559bb85d47 Created pod: payload-pre-processing-e2e-api-a-922795-559bb85d47-l8qpq 46m Normal Pulled pod/payload-pre-processing-e2e-api-a-922795-559bb85d47-l8qpq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-922795-559bb85d47-l8qpq Add eth0 [10.133.0.188/23] from ovn-kubernetes 46m Normal Started pod/payload-processing-e2e-api-a-922795-56d4c4f7db-hjv42 Container started 46m Normal Created pod/payload-processing-e2e-api-a-922795-56d4c4f7db-hjv42 Container created 46m Normal Pulled pod/payload-processing-e2e-api-a-922795-56d4c4f7db-hjv42 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/payload-processing-e2e-api-a-922795-56d4c4f7db-hjv42 Add eth0 [10.133.0.187/23] from ovn-kubernetes 46m Normal Started pod/payload-pre-processing-e2e-api-a-922795-559bb85d47-l8qpq Container started 46m Normal Created pod/payload-pre-processing-e2e-api-a-922795-559bb85d47-l8qpq Container created 46m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-65913e-openshift-default Scaled up replica set e2e-shared-b-w2-65913e-openshift-default-65986b75f9 from 0 to 1 46m Normal AddedLabel gateway/e2e-shared-b-w2-65913e Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-65913e 46m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-65913e-openshift-default-65986b75f9 Created pod: e2e-shared-b-w2-65913e-openshift-default-65986b75f9-s5cnk 46m Warning ProbeError pod/e2e-api-a-0c21c7-openshift-default-8476d54f65-ph5gh Startup probe error: Get "http://10.133.0.179:15021/healthz/ready": dial tcp 10.133.0.179:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-shared-a-w6-6a3169-openshift-default-58d9474d87-gztml Startup probe failed: Get "http://10.133.0.175:15021/healthz/ready": dial tcp 10.133.0.175:15021: connect: connection refused 46m Warning ProbeError pod/e2e-shared-a-w6-6a3169-openshift-default-58d9474d87-gztml Startup probe error: Get "http://10.133.0.175:15021/healthz/ready": dial tcp 10.133.0.175:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-api-a-0c21c7-openshift-default-8476d54f65-ph5gh Startup probe failed: Get "http://10.133.0.179:15021/healthz/ready": dial tcp 10.133.0.179:15021: connect: connection refused 46m Normal AddedInterface pod/e2e-shared-b-w2-65913e-openshift-default-65986b75f9-s5cnk Add eth0 [10.132.0.106/23] from ovn-kubernetes 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-6a3169-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-6a3169-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-65913e Scaled up replica set payload-processing-e2e-shared-b-w2-65913e-75fdbc7fd8 from 0 to 1 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-65913e Scaled up replica set payload-pre-processing-e2e-shared-b-w2-65913e-568db8dd5f from 0 to 1 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-0c21c7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-0c21c7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-65913e-75fdbc7fd8 Error creating: pods "payload-processing-e2e-shared-b-w2-65913e-75fdbc7fd8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-65913e: serviceaccount "payload-processing-e2e-shared-b-w2-65913e" not found 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-65913e-568db8dd5f Created pod: payload-pre-processing-e2e-shared-b-w2-65913e-568db8dd5f-msdbs 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-65913e-568db8dd5f Error creating: pods "payload-pre-processing-e2e-shared-b-w2-65913e-568db8dd5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-65913e: serviceaccount "payload-processing-e2e-shared-b-w2-65913e" not found 46m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-65913e-568db8dd5f-msdbs Add eth0 [10.133.0.189/23] from ovn-kubernetes 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-65913e-75fdbc7fd8 Created pod: payload-processing-e2e-shared-b-w2-65913e-75fdbc7fd8-vgd45 46m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-65913e-75fdbc7fd8-vgd45 Add eth0 [10.133.0.190/23] from ovn-kubernetes 46m Normal Created pod/payload-processing-e2e-shared-b-w2-65913e-75fdbc7fd8-vgd45 Container created 46m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-65913e-568db8dd5f-msdbs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-65913e-568db8dd5f-msdbs Container created 46m Normal ScalingReplicaSet deployment/e2e-worker-w4-18c03a-openshift-default Scaled down replica set e2e-worker-w4-18c03a-openshift-default-589bbc88df from 6 to 5 46m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-65913e-568db8dd5f-msdbs Container started 46m Normal Pulled pod/payload-processing-e2e-shared-b-w2-65913e-75fdbc7fd8-vgd45 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-18c03a-openshift-default New size: 5; reason: All metrics below target 46m Normal Started pod/payload-processing-e2e-shared-b-w2-65913e-75fdbc7fd8-vgd45 Container started 46m Normal Killing pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-k5m5d Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/e2e-worker-w4-18c03a-openshift-default-589bbc88df Deleted pod: e2e-worker-w4-18c03a-openshift-default-589bbc88df-k5m5d 46m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-0189ef-openshift-default Scaled up replica set e2e-shared-b-w6-0189ef-openshift-default-cd86cdd47 from 0 to 1 46m Normal AddedLabel gateway/e2e-shared-b-w6-0189ef Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w6-0189ef 46m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-0189ef-openshift-default-cd86cdd47 Created pod: e2e-shared-b-w6-0189ef-openshift-default-cd86cdd47-8h87f 46m Normal AddedInterface pod/e2e-shared-b-w6-0189ef-openshift-default-cd86cdd47-8h87f Add eth0 [10.132.0.107/23] from ovn-kubernetes 46m Normal SuccessfulDelete replicaset/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 Deleted pod: e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-jxgb9 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6f1190-openshift-default New size: 5; reason: All metrics below target 46m Normal SuccessfulDelete replicaset/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c Deleted pod: e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-5e72ad-openshift-default New size: 2; reason: All metrics below target 46m Normal Killing pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-jxgb9 Stopping container istio-proxy 46m Normal ScalingReplicaSet deployment/e2e-worker-w6-5e72ad-openshift-default Scaled down replica set e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c from 3 to 2 46m Normal Killing pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm Stopping container istio-proxy 46m Normal ScalingReplicaSet deployment/e2e-worker-w5-6f1190-openshift-default Scaled down replica set e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 from 6 to 5 46m Normal SuccessfulDelete replicaset/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 Deleted pod: e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-p8vxh 46m Normal Killing pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc Stopping container istio-proxy 46m Normal ScalingReplicaSet deployment/e2e-worker-w1-545b5d-openshift-default Scaled down replica set e2e-worker-w1-545b5d-openshift-default-5c8c66f65c from 6 to 5 46m Normal Killing pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74 Stopping container istio-proxy 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-ee74cb-openshift-default New size: 5; reason: All metrics below target 46m Normal Killing pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-p8vxh Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/e2e-worker-w2-ee74cb-openshift-default-9758876f Deleted pod: e2e-worker-w2-ee74cb-openshift-default-9758876f-bfk74 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-68fbd1-openshift-default New size: 5; reason: All metrics below target 46m Normal ScalingReplicaSet deployment/e2e-worker-w0-68fbd1-openshift-default Scaled down replica set e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 from 6 to 5 46m Normal ScalingReplicaSet deployment/e2e-worker-w2-ee74cb-openshift-default Scaled down replica set e2e-worker-w2-ee74cb-openshift-default-9758876f from 6 to 5 46m Normal SuccessfulDelete replicaset/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c Deleted pod: e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-42mkc 46m Warning ProbeError pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm Readiness probe error: HTTP probe failed with statuscode: 503... 46m Warning Unhealthy pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-kf4xm Readiness probe failed: HTTP probe failed with statuscode: 503 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w6-0189ef Scaled up replica set payload-pre-processing-e2e-shared-b-w6-0189ef-676855966c from 0 to 1 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w6-0189ef Scaled up replica set payload-processing-e2e-shared-b-w6-0189ef-64d6f8fb8 from 0 to 1 46m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w6-0189ef-64d6f8fb8 Error creating: pods "payload-processing-e2e-shared-b-w6-0189ef-64d6f8fb8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-0189ef: serviceaccount "payload-processing-e2e-shared-b-w6-0189ef" not found 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w6-0189ef-676855966c Error creating: pods "payload-pre-processing-e2e-shared-b-w6-0189ef-676855966c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-0189ef: serviceaccount "payload-processing-e2e-shared-b-w6-0189ef" not found 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w6-0189ef-676855966c Created pod: payload-pre-processing-e2e-shared-b-w6-0189ef-676855966c-2p5wb 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-0c21c7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-0c21c7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w6-0189ef-64d6f8fb8 Created pod: payload-processing-e2e-shared-b-w6-0189ef-64d6f8fb8-r5l4j 46m Normal Pulled pod/payload-processing-e2e-shared-b-w6-0189ef-64d6f8fb8-r5l4j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/payload-processing-e2e-shared-b-w6-0189ef-64d6f8fb8-r5l4j Add eth0 [10.133.0.192/23] from ovn-kubernetes 46m Normal Started pod/payload-pre-processing-e2e-shared-b-w6-0189ef-676855966c-2p5wb Container started 46m Normal Created pod/payload-pre-processing-e2e-shared-b-w6-0189ef-676855966c-2p5wb Container created 46m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w6-0189ef-676855966c-2p5wb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w6-0189ef-676855966c-2p5wb Add eth0 [10.133.0.191/23] from ovn-kubernetes 46m Normal Created pod/payload-processing-e2e-shared-b-w6-0189ef-64d6f8fb8-r5l4j Container created 46m Normal Started pod/payload-processing-e2e-shared-b-w6-0189ef-64d6f8fb8-r5l4j Container started 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-6a3169-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-6a3169-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Normal ScalingReplicaSet deployment/e2e-api-b-2769cb-openshift-default Scaled up replica set e2e-api-b-2769cb-openshift-default-554f5c6bf from 0 to 1 45m Normal AddedLabel gateway/e2e-api-b-2769cb Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-2769cb 45m Normal SuccessfulCreate replicaset/e2e-api-b-2769cb-openshift-default-554f5c6bf Created pod: e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q 45m Normal Killing pod/payload-processing-e2e-mt-12450a06-5c4cc76cd6-c9zgk Stopping container payload-processing 45m Normal AddedInterface pod/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q Add eth0 [10.134.0.95/23] from ovn-kubernetes 45m Normal Killing pod/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-5tt6k Stopping container payload-pre-processing 45m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 45m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 45m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-hmb27 45m Warning ProbeError pod/e2e-api-a-2627c5-openshift-default-6c84fbdb55-5g5zm Readiness probe error: Get "http://10.132.0.102:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-hmb27 Stopping container discovery 45m Warning Unhealthy pod/e2e-api-a-2627c5-openshift-default-6c84fbdb55-5g5zm Readiness probe failed: Get "http://10.132.0.102:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Normal Pulled pod/e2e-api-a-2627c5-openshift-default-6c84fbdb55-5g5zm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/e2e-api-a-2627c5-openshift-default-6c84fbdb55-5g5zm Container started 45m Normal Created pod/e2e-api-a-2627c5-openshift-default-6c84fbdb55-5g5zm Container created 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-2769cb Scaled up replica set payload-processing-e2e-api-b-2769cb-85cd775f46 from 0 to 1 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-2769cb Scaled up replica set payload-pre-processing-e2e-api-b-2769cb-7668b55d7 from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-2769cb-7668b55d7 Created pod: payload-pre-processing-e2e-api-b-2769cb-7668b55d7-b9vqz 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-2769cb-7668b55d7 Error creating: pods "payload-pre-processing-e2e-api-b-2769cb-7668b55d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-2769cb: serviceaccount "payload-processing-e2e-api-b-2769cb" not found 45m Warning FailedCreate replicaset/payload-processing-e2e-api-b-2769cb-85cd775f46 Error creating: pods "payload-processing-e2e-api-b-2769cb-85cd775f46-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-2769cb: serviceaccount "payload-processing-e2e-api-b-2769cb" not found 45m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-2769cb-7668b55d7-b9vqz Add eth0 [10.132.0.111/23] from ovn-kubernetes 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-2769cb-85cd775f46 Created pod: payload-processing-e2e-api-b-2769cb-85cd775f46-hxl4x 45m Normal AddedInterface pod/payload-processing-e2e-api-b-2769cb-85cd775f46-hxl4x Add eth0 [10.132.0.112/23] from ovn-kubernetes 45m Normal Pulled pod/payload-processing-e2e-api-b-2769cb-85cd775f46-hxl4x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-processing-e2e-api-b-2769cb-85cd775f46-hxl4x Container created 45m Normal Pulled pod/payload-pre-processing-e2e-api-b-2769cb-7668b55d7-b9vqz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Started pod/payload-processing-e2e-api-b-2769cb-85cd775f46-hxl4x Container started 45m Normal Started pod/payload-pre-processing-e2e-api-b-2769cb-7668b55d7-b9vqz Container started 45m Normal Created pod/payload-pre-processing-e2e-api-b-2769cb-7668b55d7-b9vqz Container created 45m Warning ProbeError pod/e2e-api-a-2627c5-openshift-default-6c84fbdb55-5g5zm Startup probe error: Get "http://10.132.0.102:15021/healthz/ready": dial tcp 10.132.0.102:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-api-a-2627c5-openshift-default-6c84fbdb55-5g5zm Startup probe failed: Get "http://10.132.0.102:15021/healthz/ready": dial tcp 10.132.0.102:15021: connect: connection refused 45m Normal ScalingReplicaSet deployment/e2e-worker-w3-961054-openshift-default Scaled down replica set e2e-worker-w3-961054-openshift-default-5d5665c66f from 4 to 2 45m Normal SuccessfulDelete replicaset/e2e-worker-w3-961054-openshift-default-5d5665c66f Deleted pod: e2e-worker-w3-961054-openshift-default-5d5665c66f-gpbv6 45m Normal SuccessfulDelete replicaset/e2e-worker-w3-961054-openshift-default-5d5665c66f Deleted pod: e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-961054-openshift-default New size: 2; reason: All metrics below target 45m Normal Killing pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-bqcnr Stopping container istio-proxy 45m Normal Killing pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-gpbv6 Stopping container istio-proxy 45m Normal ScalingReplicaSet deployment/e2e-worker-w0-68fbd1-openshift-default Scaled down replica set e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 from 5 to 4 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-68fbd1-openshift-default New size: 4; reason: All metrics below target 45m Normal Killing pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-th2cc Stopping container istio-proxy 45m Normal SuccessfulDelete replicaset/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 Deleted pod: e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-545b5d-openshift-default New size: 4; reason: All metrics below target 45m Normal ScalingReplicaSet deployment/e2e-worker-w1-545b5d-openshift-default Scaled down replica set e2e-worker-w1-545b5d-openshift-default-5c8c66f65c from 5 to 4 45m Normal SuccessfulDelete replicaset/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c Deleted pod: e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-th2cc 45m Normal SuccessfulDelete replicaset/e2e-worker-w2-ee74cb-openshift-default-9758876f Deleted pod: e2e-worker-w2-ee74cb-openshift-default-9758876f-fwck9 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-ee74cb-openshift-default New size: 4; reason: All metrics below target 45m Normal Killing pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p Stopping container istio-proxy 45m Normal Killing pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-fwck9 Stopping container istio-proxy 45m Normal ScalingReplicaSet deployment/e2e-worker-w2-ee74cb-openshift-default Scaled down replica set e2e-worker-w2-ee74cb-openshift-default-9758876f from 5 to 4 45m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-th2cc Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning Unhealthy pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-th2cc Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning Unhealthy pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-gpbv6 Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-gpbv6 Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning ProbeError pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning Unhealthy pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-6lm8p Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-fwck9 Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning Unhealthy pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-fwck9 Readiness probe failed: HTTP probe failed with statuscode: 503 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-2627c5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-2627c5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal Pulled pod/e2e-inf-a-2420f6-openshift-default-75c86ff8ff-ltdfl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/e2e-inf-a-2420f6-openshift-default-75c86ff8ff-ltdfl Container created 44m Normal Started pod/e2e-inf-a-2420f6-openshift-default-75c86ff8ff-ltdfl Container started 44m Warning ProbeError pod/e2e-inf-a-2420f6-openshift-default-75c86ff8ff-ltdfl Startup probe error: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-inf-a-2420f6-openshift-default-75c86ff8ff-ltdfl Startup probe failed: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused 44m Normal SuccessfulCreate replicaset/e2e-api-b-d6296b-openshift-default-7f98f9c4d5 Created pod: e2e-api-b-d6296b-openshift-default-7f98f9c4d5-p9g4x 44m Normal ScalingReplicaSet deployment/e2e-api-b-d6296b-openshift-default Scaled up replica set e2e-api-b-d6296b-openshift-default-7f98f9c4d5 from 0 to 1 44m Normal AddedLabel gateway/e2e-api-b-d6296b Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-d6296b 44m Normal AddedInterface pod/e2e-api-b-d6296b-openshift-default-7f98f9c4d5-p9g4x Add eth0 [10.132.0.116/23] from ovn-kubernetes 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-d6296b-b54fff875 Error creating: pods "payload-pre-processing-e2e-api-b-d6296b-b54fff875-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-d6296b: serviceaccount "payload-processing-e2e-api-b-d6296b" not found 44m Warning FailedCreate replicaset/payload-processing-e2e-api-b-d6296b-95dc94cbb Error creating: pods "payload-processing-e2e-api-b-d6296b-95dc94cbb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-d6296b: serviceaccount "payload-processing-e2e-api-b-d6296b" not found 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-d6296b Scaled up replica set payload-processing-e2e-api-b-d6296b-95dc94cbb from 0 to 1 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-d6296b Scaled up replica set payload-pre-processing-e2e-api-b-d6296b-b54fff875 from 0 to 1 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-2420f6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-2420f6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal AddedInterface pod/payload-processing-e2e-api-b-d6296b-95dc94cbb-k2wv7 Add eth0 [10.133.0.196/23] from ovn-kubernetes 44m Normal Pulled pod/payload-pre-processing-e2e-api-b-d6296b-b54fff875-sktff Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-d6296b-b54fff875-sktff Add eth0 [10.133.0.195/23] from ovn-kubernetes 44m Normal Created pod/payload-pre-processing-e2e-api-b-d6296b-b54fff875-sktff Container created 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-d6296b-95dc94cbb Created pod: payload-processing-e2e-api-b-d6296b-95dc94cbb-k2wv7 44m Normal Started pod/payload-pre-processing-e2e-api-b-d6296b-b54fff875-sktff Container started 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-d6296b-b54fff875 Created pod: payload-pre-processing-e2e-api-b-d6296b-b54fff875-sktff 44m Normal Created pod/payload-processing-e2e-api-b-d6296b-95dc94cbb-k2wv7 Container created 44m Normal Started pod/payload-processing-e2e-api-b-d6296b-95dc94cbb-k2wv7 Container started 44m Normal Pulled pod/payload-processing-e2e-api-b-d6296b-95dc94cbb-k2wv7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal SuccessfulCreate replicaset/e2e-inf-b-9674f5-openshift-default-7b45c9f895 Created pod: e2e-inf-b-9674f5-openshift-default-7b45c9f895-w26vx 44m Normal ScalingReplicaSet deployment/e2e-inf-b-9674f5-openshift-default Scaled up replica set e2e-inf-b-9674f5-openshift-default-7b45c9f895 from 0 to 1 44m Normal AddedLabel gateway/e2e-inf-b-9674f5 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-9674f5 44m Normal AddedInterface pod/e2e-inf-b-9674f5-openshift-default-7b45c9f895-w26vx Add eth0 [10.132.0.117/23] from ovn-kubernetes 43m Normal Started pod/e2e-shared-b-w2-65913e-openshift-default-65986b75f9-s5cnk Container started 43m Warning Unhealthy pod/e2e-shared-b-w2-65913e-openshift-default-65986b75f9-s5cnk Readiness probe failed: Get "http://10.132.0.106:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Warning ProbeError pod/e2e-shared-b-w2-65913e-openshift-default-65986b75f9-s5cnk Readiness probe error: Get "http://10.132.0.106:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Warning Unhealthy pod/e2e-shared-b-w6-0189ef-openshift-default-cd86cdd47-8h87f Readiness probe failed: Get "http://10.132.0.107:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Warning ProbeError pod/e2e-shared-b-w6-0189ef-openshift-default-cd86cdd47-8h87f Readiness probe error: Get "http://10.132.0.107:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Normal Created pod/e2e-shared-b-w2-65913e-openshift-default-65986b75f9-s5cnk Container created 43m Normal Pulled pod/e2e-shared-b-w2-65913e-openshift-default-65986b75f9-s5cnk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Pulled pod/e2e-shared-b-w6-0189ef-openshift-default-cd86cdd47-8h87f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-shared-b-w6-0189ef-openshift-default-cd86cdd47-8h87f Container started 43m Normal Created pod/e2e-shared-b-w6-0189ef-openshift-default-cd86cdd47-8h87f Container created 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-9674f5-699cd9d748 Created pod: payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67 Error creating: pods "payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-9674f5: serviceaccount "payload-processing-e2e-inf-b-9674f5" not found 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67 Created pod: payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx 43m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-9674f5-699cd9d748 Error creating: pods "payload-processing-e2e-inf-b-9674f5-699cd9d748-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-9674f5: serviceaccount "payload-processing-e2e-inf-b-9674f5" not found 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-9674f5 Scaled up replica set payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67 from 0 to 1 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-9674f5 Scaled up replica set payload-processing-e2e-inf-b-9674f5-699cd9d748 from 0 to 1 43m Normal Started pod/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx Container started 43m Normal Created pod/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx Container created 43m Normal AddedInterface pod/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl Add eth0 [10.134.0.97/23] from ovn-kubernetes 43m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx Add eth0 [10.134.0.96/23] from ovn-kubernetes 43m Normal Pulled pod/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Pulled pod/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl Container created 43m Normal Started pod/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl Container started 43m Warning Unhealthy pod/e2e-shared-b-w2-65913e-openshift-default-65986b75f9-s5cnk Startup probe failed: Get "http://10.132.0.106:15021/healthz/ready": dial tcp 10.132.0.106:15021: connect: connection refused 43m Warning ProbeError pod/e2e-shared-b-w2-65913e-openshift-default-65986b75f9-s5cnk Startup probe error: Get "http://10.132.0.106:15021/healthz/ready": dial tcp 10.132.0.106:15021: connect: connection refused... 43m Warning ProbeError pod/e2e-shared-b-w6-0189ef-openshift-default-cd86cdd47-8h87f Startup probe error: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-shared-b-w6-0189ef-openshift-default-cd86cdd47-8h87f Startup probe failed: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-0189ef-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-0189ef-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal SuccessfulDelete replicaset/e2e-worker-w2-ee74cb-openshift-default-9758876f Deleted pod: e2e-worker-w2-ee74cb-openshift-default-9758876f-sq629 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-ee74cb-openshift-default New size: 3; reason: All metrics below target 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-68fbd1-openshift-default New size: 3; reason: All metrics below target 43m Normal SuccessfulDelete replicaset/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 Deleted pod: e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-nnsbl 43m Normal ScalingReplicaSet deployment/e2e-worker-w0-68fbd1-openshift-default Scaled down replica set e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 from 4 to 3 43m Normal Killing pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-sq629 Stopping container istio-proxy 43m Normal Killing pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-nnsbl Stopping container istio-proxy 43m Warning ProbeError pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-nnsbl Readiness probe error: HTTP probe failed with statuscode: 503... 43m Warning Unhealthy pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-nnsbl Readiness probe failed: HTTP probe failed with statuscode: 503 43m Warning Unhealthy pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-sq629 Readiness probe failed: HTTP probe failed with statuscode: 503 43m Warning ProbeError pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-sq629 Readiness probe error: HTTP probe failed with statuscode: 503... 43m Normal Pulled pod/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q Container started 43m Normal Created pod/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q Container created 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-2769cb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-2769cb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal Killing pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7 Stopping container istio-proxy 43m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7 Readiness probe error: HTTP probe failed with statuscode: 503... 43m Warning Unhealthy pod/e2e-api-a-922795-openshift-default-59b46784df-6bc47 Readiness probe failed: Get "http://10.132.0.105:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Warning ProbeError pod/e2e-api-a-922795-openshift-default-59b46784df-6bc47 Readiness probe error: Get "http://10.132.0.105:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Warning Unhealthy pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7 Readiness probe failed: HTTP probe failed with statuscode: 503 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-545b5d-openshift-default New size: 3; reason: All metrics below target 43m Normal SuccessfulDelete replicaset/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c Deleted pod: e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-vl6l7 43m Warning ProbeError pod/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q Startup probe error: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q Startup probe failed: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused 43m Normal Created pod/e2e-api-a-922795-openshift-default-59b46784df-6bc47 Container created 43m Normal Started pod/e2e-api-a-922795-openshift-default-59b46784df-6bc47 Container started 43m Normal Pulled pod/e2e-api-a-922795-openshift-default-59b46784df-6bc47 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Killing pod/payload-processing-e2e-api-a-922795-56d4c4f7db-hjv42 Stopping container payload-processing 43m Normal Killing pod/payload-pre-processing-e2e-api-a-922795-559bb85d47-l8qpq Stopping container payload-pre-processing 43m Normal Killing pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm Stopping container istio-proxy 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-18c03a-openshift-default New size: 3; reason: All metrics below target 43m Normal SuccessfulDelete replicaset/e2e-worker-w4-18c03a-openshift-default-589bbc88df Deleted pod: e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm 43m Normal SuccessfulDelete replicaset/e2e-worker-w4-18c03a-openshift-default-589bbc88df Deleted pod: e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr 43m Normal ScalingReplicaSet deployment/e2e-worker-w4-18c03a-openshift-default Scaled down replica set e2e-worker-w4-18c03a-openshift-default-589bbc88df from 5 to 3 43m Normal Killing pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr Stopping container istio-proxy 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-0189ef-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning ProbeError pod/e2e-api-a-922795-openshift-default-59b46784df-6bc47 Startup probe error: Get "http://10.132.0.105:15021/healthz/ready": dial tcp 10.132.0.105:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-api-a-922795-openshift-default-59b46784df-6bc47 Startup probe failed: Get "http://10.132.0.105:15021/healthz/ready": dial tcp 10.132.0.105:15021: connect: connection refused 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-0189ef-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-65913e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-65913e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-fb4f0a-7f6d59c45f-gpbjr Stopping container payload-pre-processing 43m Warning ProbeError pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr Readiness probe error: HTTP probe failed with statuscode: 503... 43m Warning Unhealthy pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-lctzr Readiness probe failed: HTTP probe failed with statuscode: 503 43m Warning ProbeError pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm Readiness probe error: HTTP probe failed with statuscode: 503... 43m Normal Killing pod/payload-processing-e2e-shared-a-w2-fb4f0a-6fdc7bbbfd-m9229 Stopping container payload-processing 43m Warning Unhealthy pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-9d5fm Readiness probe failed: HTTP probe failed with statuscode: 503 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-2769cb-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-2769cb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal ScalingReplicaSet deployment/e2e-worker-w5-6f1190-openshift-default Scaled down replica set e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 from 5 to 3 43m Normal SuccessfulDelete replicaset/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 Deleted pod: e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-n7vdt 43m Normal Killing pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7 Stopping container istio-proxy 43m Normal Killing pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-n7vdt Stopping container istio-proxy 43m Normal SuccessfulDelete replicaset/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 Deleted pod: e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-xfgh7 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6f1190-openshift-default New size: 3; reason: All metrics below target 42m Normal Killing pod/payload-processing-e2e-shared-a-w6-6a3169-866cf6cb6d-t7spd Stopping container payload-processing 42m Normal Killing pod/payload-pre-processing-e2e-shared-a-w6-6a3169-67b8ff4bb7-k6wb9 Stopping container payload-pre-processing 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-922795-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-922795-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Normal Killing pod/maas-default-gateway-openshift-default-59f957fdd7-ddv45 Stopping container istio-proxy 42m Normal Killing pod/maas-default-gateway-openshift-default-59f957fdd7-n5gnt Stopping container istio-proxy 42m Normal Killing pod/maas-default-gateway-openshift-default-59f957fdd7-vhkwc Stopping container istio-proxy 42m Normal Killing pod/maas-default-gateway-openshift-default-59f957fdd7-k2jmm Stopping container istio-proxy 42m Normal Killing pod/maas-default-gateway-openshift-default-59f957fdd7-fkwnw Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-59f957fdd7 Deleted pod: maas-default-gateway-openshift-default-59f957fdd7-vhkwc 42m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-59f957fdd7 Deleted pod: maas-default-gateway-openshift-default-59f957fdd7-n5gnt 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-12450a06 Scaled up replica set payload-processing-e2e-mt-12450a06-5c4cc76cd6 from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-12450a06-5c4cc76cd6 Created pod: payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw 42m Warning FailedCreate replicaset/payload-processing-e2e-mt-12450a06-5c4cc76cd6 Error creating: pods "payload-processing-e2e-mt-12450a06-5c4cc76cd6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-12450a06: serviceaccount "payload-processing-e2e-mt-12450a06" not found 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-12450a06-99d9f9f9 Error creating: pods "payload-pre-processing-e2e-mt-12450a06-99d9f9f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-12450a06: serviceaccount "payload-processing-e2e-mt-12450a06" not found 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-12450a06-99d9f9f9 Created pod: payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-12450a06 Scaled up replica set payload-pre-processing-e2e-mt-12450a06-99d9f9f9 from 0 to 1 42m Normal Created pod/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n Container created 42m Normal Started pod/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw Container started 42m Normal Started pod/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n Container started 42m Normal AddedInterface pod/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw Add eth0 [10.134.0.99/23] from ovn-kubernetes 42m Normal Pulled pod/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Pulled pod/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n Add eth0 [10.134.0.98/23] from ovn-kubernetes 42m Normal Created pod/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw Container created 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-922795-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-922795-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal Killing pod/payload-pre-processing-e2e-api-a-0c21c7-dd7b5dd55-qzcqr Stopping container payload-pre-processing 42m Normal Killing pod/payload-processing-e2e-api-a-0c21c7-884dbcbfd-vcsmv Stopping container payload-processing 41m Normal Pulled pod/e2e-api-b-d6296b-openshift-default-7f98f9c4d5-p9g4x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Started pod/e2e-api-b-d6296b-openshift-default-7f98f9c4d5-p9g4x Container started 41m Normal Created pod/e2e-api-b-d6296b-openshift-default-7f98f9c4d5-p9g4x Container created 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-d6296b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-d6296b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning Unhealthy pod/e2e-api-b-d6296b-openshift-default-7f98f9c4d5-p9g4x Startup probe failed: Get "http://10.132.0.116:15021/healthz/ready": dial tcp 10.132.0.116:15021: connect: connection refused 41m Warning ProbeError pod/e2e-api-b-d6296b-openshift-default-7f98f9c4d5-p9g4x Startup probe error: Get "http://10.132.0.116:15021/healthz/ready": dial tcp 10.132.0.116:15021: connect: connection refused... 41m Normal Killing pod/e2e-api-a-922795-openshift-default-59b46784df-6bc47 Stopping container istio-proxy 41m Normal Killing pod/e2e-shared-a-w2-fb4f0a-openshift-default-7b7c76784f-ldhbt Stopping container istio-proxy 41m Normal AddedLabel gateway/e2e-inf-a-9b4383 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-9b4383 41m Normal SuccessfulCreate replicaset/e2e-inf-a-9b4383-openshift-default-64f46d78c7 Created pod: e2e-inf-a-9b4383-openshift-default-64f46d78c7-lblz9 41m Normal AddedInterface pod/e2e-inf-a-9b4383-openshift-default-64f46d78c7-lblz9 Add eth0 [10.133.0.202/23] from ovn-kubernetes 41m Normal ScalingReplicaSet deployment/e2e-inf-a-9b4383-openshift-default Scaled up replica set e2e-inf-a-9b4383-openshift-default-64f46d78c7 from 0 to 1 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-9b4383 Scaled up replica set payload-pre-processing-e2e-inf-a-9b4383-7c7d96654c from 0 to 1 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-9b4383 Scaled up replica set payload-processing-e2e-inf-a-9b4383-5b9cf487f8 from 0 to 1 41m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-9b4383-5b9cf487f8 Error creating: pods "payload-processing-e2e-inf-a-9b4383-5b9cf487f8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-9b4383: serviceaccount "payload-processing-e2e-inf-a-9b4383" not found 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-9b4383-7c7d96654c Error creating: pods "payload-pre-processing-e2e-inf-a-9b4383-7c7d96654c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-9b4383: serviceaccount "payload-processing-e2e-inf-a-9b4383" not found 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-9b4383-7c7d96654c Created pod: payload-pre-processing-e2e-inf-a-9b4383-7c7d96654c-wj9bn 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-9b4383-5b9cf487f8 Created pod: payload-processing-e2e-inf-a-9b4383-5b9cf487f8-qlxxp 41m Normal Pulled pod/payload-pre-processing-e2e-inf-a-9b4383-7c7d96654c-wj9bn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-9b4383-7c7d96654c-wj9bn Add eth0 [10.133.0.204/23] from ovn-kubernetes 41m Normal Started pod/payload-pre-processing-e2e-inf-a-9b4383-7c7d96654c-wj9bn Container started 41m Normal Created pod/payload-pre-processing-e2e-inf-a-9b4383-7c7d96654c-wj9bn Container created 41m Normal Started pod/payload-processing-e2e-inf-a-9b4383-5b9cf487f8-qlxxp Container started 41m Normal Created pod/payload-processing-e2e-inf-a-9b4383-5b9cf487f8-qlxxp Container created 41m Normal Pulled pod/payload-processing-e2e-inf-a-9b4383-5b9cf487f8-qlxxp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Pulled pod/e2e-inf-b-9674f5-openshift-default-7b45c9f895-w26vx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/payload-processing-e2e-inf-a-9b4383-5b9cf487f8-qlxxp Add eth0 [10.133.0.203/23] from ovn-kubernetes 41m Normal Created pod/e2e-inf-b-9674f5-openshift-default-7b45c9f895-w26vx Container created 41m Normal Started pod/e2e-inf-b-9674f5-openshift-default-7b45c9f895-w26vx Container started 41m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-65913e-568db8dd5f-msdbs Stopping container payload-pre-processing 41m Normal Killing pod/payload-processing-e2e-shared-b-w2-65913e-75fdbc7fd8-vgd45 Stopping container payload-processing 41m Warning Unhealthy pod/e2e-inf-b-9674f5-openshift-default-7b45c9f895-w26vx Startup probe failed: Get "http://10.132.0.117:15021/healthz/ready": dial tcp 10.132.0.117:15021: connect: connection refused 41m Warning ProbeError pod/e2e-inf-b-9674f5-openshift-default-7b45c9f895-w26vx Startup probe error: Get "http://10.132.0.117:15021/healthz/ready": dial tcp 10.132.0.117:15021: connect: connection refused... 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-d6296b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-d6296b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal Killing pod/e2e-shared-a-w6-6a3169-openshift-default-58d9474d87-gztml Stopping container istio-proxy 41m Normal Killing pod/payload-pre-processing-e2e-api-a-2627c5-86d7c64d59-2l69t Stopping container payload-pre-processing 41m Normal Killing pod/payload-processing-e2e-api-a-2627c5-56b96dd67b-vqv5f Stopping container payload-processing 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-9674f5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-9674f5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal Killing pod/payload-pre-processing-e2e-shared-b-w6-0189ef-676855966c-2p5wb Stopping container payload-pre-processing 40m Normal Killing pod/payload-processing-e2e-shared-b-w6-0189ef-64d6f8fb8-r5l4j Stopping container payload-processing 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-12450a06-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 40m Normal ScalingReplicaSet deployment/e2e-mt-12450a06-openshift-default Scaled up replica set e2e-mt-12450a06-openshift-default-7576d969f7 from 1 to 2 40m Normal SuccessfulCreate replicaset/e2e-mt-12450a06-openshift-default-7576d969f7 Created pod: e2e-mt-12450a06-openshift-default-7576d969f7-6zght 40m Normal Created pod/e2e-mt-12450a06-openshift-default-7576d969f7-6zght Container created 40m Normal AddedInterface pod/e2e-mt-12450a06-openshift-default-7576d969f7-6zght Add eth0 [10.132.0.122/23] from ovn-kubernetes 40m Normal Started pod/e2e-mt-12450a06-openshift-default-7576d969f7-6zght Container started 40m Normal Pulled pod/e2e-mt-12450a06-openshift-default-7576d969f7-6zght Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Killing pod/payload-processing-e2e-inf-a-2420f6-b68c89c6-shx8b Stopping container payload-processing 40m Normal Killing pod/payload-pre-processing-e2e-inf-a-2420f6-cb5bf5df5-9td4b Stopping container payload-pre-processing 40m Warning Unhealthy pod/e2e-mt-12450a06-openshift-default-7576d969f7-6zght Startup probe failed: Get "http://10.132.0.122:15021/healthz/ready": dial tcp 10.132.0.122:15021: connect: connection refused 40m Warning ProbeError pod/e2e-mt-12450a06-openshift-default-7576d969f7-6zght Startup probe error: Get "http://10.132.0.122:15021/healthz/ready": dial tcp 10.132.0.122:15021: connect: connection refused... 40m Normal SuccessfulCreate replicaset/e2e-mt-12450a06-openshift-default-7576d969f7 Created pod: e2e-mt-12450a06-openshift-default-7576d969f7-5l758 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-12450a06-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 40m Normal SuccessfulCreate replicaset/e2e-mt-12450a06-openshift-default-7576d969f7 Created pod: e2e-mt-12450a06-openshift-default-7576d969f7-t67gd 40m Normal ScalingReplicaSet deployment/e2e-mt-12450a06-openshift-default Scaled up replica set e2e-mt-12450a06-openshift-default-7576d969f7 from 2 to 4 40m Normal Pulled pod/e2e-mt-12450a06-openshift-default-7576d969f7-t67gd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/e2e-mt-12450a06-openshift-default-7576d969f7-5l758 Add eth0 [10.134.0.100/23] from ovn-kubernetes 40m Normal Created pod/e2e-mt-12450a06-openshift-default-7576d969f7-5l758 Container created 40m Normal Started pod/e2e-mt-12450a06-openshift-default-7576d969f7-5l758 Container started 40m Normal AddedInterface pod/e2e-mt-12450a06-openshift-default-7576d969f7-t67gd Add eth0 [10.133.0.208/23] from ovn-kubernetes 40m Normal Pulled pod/e2e-mt-12450a06-openshift-default-7576d969f7-5l758 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-mt-12450a06-openshift-default-7576d969f7-t67gd Container created 40m Normal Started pod/e2e-mt-12450a06-openshift-default-7576d969f7-t67gd Container started 40m Warning ProbeError pod/e2e-mt-12450a06-openshift-default-7576d969f7-5l758 Startup probe error: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused... 40m Warning ProbeError pod/e2e-mt-12450a06-openshift-default-7576d969f7-t67gd Startup probe error: Get "http://10.133.0.208:15021/healthz/ready": dial tcp 10.133.0.208:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-mt-12450a06-openshift-default-7576d969f7-5l758 Startup probe failed: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused 40m Warning Unhealthy pod/e2e-mt-12450a06-openshift-default-7576d969f7-t67gd Startup probe failed: Get "http://10.133.0.208:15021/healthz/ready": dial tcp 10.133.0.208:15021: connect: connection refused 40m Normal SuccessfulCreate replicaset/e2e-mt-12450a06-openshift-default-7576d969f7 Created pod: e2e-mt-12450a06-openshift-default-7576d969f7-t5c69 40m Normal SuccessfulCreate replicaset/e2e-mt-12450a06-openshift-default-7576d969f7 Created pod: e2e-mt-12450a06-openshift-default-7576d969f7-vrdvc 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6f1190-openshift-default New size: 2; reason: All metrics below target 40m Normal Killing pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-kp9cz Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c Deleted pod: e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-wfg5p 40m Normal ScalingReplicaSet deployment/e2e-worker-w5-6f1190-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 from 3 to 2 40m Normal SuccessfulDelete replicaset/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 Deleted pod: e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-kp9cz 40m Normal SuccessfulCreate replicaset/e2e-mt-12450a06-openshift-default-7576d969f7 Created pod: e2e-mt-12450a06-openshift-default-7576d969f7-zp9gc 40m Normal ScalingReplicaSet deployment/e2e-mt-12450a06-openshift-default Scaled up replica set e2e-mt-12450a06-openshift-default-7576d969f7 from 4 to 7 40m Normal Killing pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-wfg5p Stopping container istio-proxy 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-12450a06-openshift-default New size: 7; reason: 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-68fbd1-openshift-default New size: 2; reason: All metrics below target 40m Normal AddedInterface pod/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69 Add eth0 [10.134.0.101/23] from ovn-kubernetes 40m Normal AddedInterface pod/e2e-mt-12450a06-openshift-default-7576d969f7-vrdvc Add eth0 [10.133.0.209/23] from ovn-kubernetes 40m Normal Started pod/e2e-mt-12450a06-openshift-default-7576d969f7-zp9gc Container started 40m Normal Created pod/e2e-mt-12450a06-openshift-default-7576d969f7-zp9gc Container created 40m Warning Unhealthy pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4 Readiness probe failed: HTTP probe failed with statuscode: 503 40m Normal Pulled pod/e2e-mt-12450a06-openshift-default-7576d969f7-zp9gc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Pulled pod/e2e-mt-12450a06-openshift-default-7576d969f7-vrdvc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/e2e-mt-12450a06-openshift-default-7576d969f7-zp9gc Add eth0 [10.132.0.123/23] from ovn-kubernetes 40m Normal Created pod/e2e-mt-12450a06-openshift-default-7576d969f7-vrdvc Container created 40m Normal Started pod/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69 Container started 40m Normal Created pod/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69 Container created 40m Normal Pulled pod/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Warning ProbeError pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4 Readiness probe error: HTTP probe failed with statuscode: 503... 40m Normal Started pod/e2e-mt-12450a06-openshift-default-7576d969f7-vrdvc Container started 40m Normal Killing pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 Deleted pod: e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t 40m Normal SuccessfulDelete replicaset/e2e-worker-w2-ee74cb-openshift-default-9758876f Deleted pod: e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4 40m Normal Killing pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-crqt4 Stopping container istio-proxy 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-ee74cb-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 40m Normal ScalingReplicaSet deployment/e2e-worker-w2-ee74cb-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w2-ee74cb-openshift-default-9758876f from 3 to 2 40m Warning ProbeError pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-kp9cz Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-kp9cz Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning Unhealthy pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning ProbeError pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-gz42t Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/e2e-mt-12450a06-openshift-default-7576d969f7-zp9gc Startup probe failed: Get "http://10.132.0.123:15021/healthz/ready": dial tcp 10.132.0.123:15021: connect: connection refused 40m Warning ProbeError pod/e2e-mt-12450a06-openshift-default-7576d969f7-vrdvc Startup probe error: Get "http://10.133.0.209:15021/healthz/ready": dial tcp 10.133.0.209:15021: connect: connection refused... 40m Warning ProbeError pod/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69 Startup probe error: Get "http://10.134.0.101:15021/healthz/ready": dial tcp 10.134.0.101:15021: connect: connection refused... 40m Warning ProbeError pod/e2e-mt-12450a06-openshift-default-7576d969f7-zp9gc Startup probe error: Get "http://10.132.0.123:15021/healthz/ready": dial tcp 10.132.0.123:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69 Startup probe failed: Get "http://10.134.0.101:15021/healthz/ready": dial tcp 10.134.0.101:15021: connect: connection refused 40m Warning Unhealthy pod/e2e-mt-12450a06-openshift-default-7576d969f7-vrdvc Startup probe failed: Get "http://10.133.0.209:15021/healthz/ready": dial tcp 10.133.0.209:15021: connect: connection refused 39m Normal Killing pod/e2e-shared-b-w2-65913e-openshift-default-65986b75f9-s5cnk Stopping container istio-proxy 39m Warning ProbeError pod/e2e-shared-b-w2-65913e-openshift-default-65986b75f9-s5cnk Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-shared-b-w2-65913e-openshift-default-65986b75f9-s5cnk Readiness probe failed: HTTP probe failed with statuscode: 503 39m Normal Killing pod/payload-processing-e2e-mt-12450a06-5c4cc76cd6-fl5mw Stopping container payload-processing 39m Normal Killing pod/payload-pre-processing-e2e-mt-12450a06-99d9f9f9-2584n Stopping container payload-pre-processing 39m Normal Killing pod/e2e-shared-b-w6-0189ef-openshift-default-cd86cdd47-8h87f Stopping container istio-proxy 39m Normal ScalingReplicaSet deployment/e2e-api-a-31ad7b-openshift-default Scaled up replica set e2e-api-a-31ad7b-openshift-default-65dcf6d7f7 from 0 to 1 39m Normal SuccessfulCreate replicaset/e2e-api-a-31ad7b-openshift-default-65dcf6d7f7 Created pod: e2e-api-a-31ad7b-openshift-default-65dcf6d7f7-d6fz2 39m Warning ProbeError pod/e2e-shared-b-w6-0189ef-openshift-default-cd86cdd47-8h87f Readiness probe error: HTTP probe failed with statuscode: 503... 39m Normal AddedLabel gateway/e2e-api-a-31ad7b Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-31ad7b 39m Warning Unhealthy pod/e2e-shared-b-w6-0189ef-openshift-default-cd86cdd47-8h87f Readiness probe failed: HTTP probe failed with statuscode: 503 39m Normal AddedInterface pod/e2e-api-a-31ad7b-openshift-default-65dcf6d7f7-d6fz2 Add eth0 [10.133.0.212/23] from ovn-kubernetes 39m Normal Killing pod/e2e-api-a-0c21c7-openshift-default-8476d54f65-ph5gh Stopping container istio-proxy 39m Warning FailedCreate replicaset/payload-processing-e2e-api-a-31ad7b-74bb7bcdf8 Error creating: pods "payload-processing-e2e-api-a-31ad7b-74bb7bcdf8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-31ad7b: serviceaccount "payload-processing-e2e-api-a-31ad7b" not found 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-31ad7b Scaled up replica set payload-processing-e2e-api-a-31ad7b-74bb7bcdf8 from 0 to 1 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-31ad7b Scaled up replica set payload-pre-processing-e2e-api-a-31ad7b-5c4dcf55 from 0 to 1 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-31ad7b-5c4dcf55 Error creating: pods "payload-pre-processing-e2e-api-a-31ad7b-5c4dcf55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-31ad7b: serviceaccount "payload-processing-e2e-api-a-31ad7b" not found 39m Normal AddedInterface pod/payload-processing-e2e-api-a-31ad7b-74bb7bcdf8-fdhjn Add eth0 [10.132.0.127/23] from ovn-kubernetes 39m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-31ad7b-5c4dcf55-2fmdz Add eth0 [10.132.0.126/23] from ovn-kubernetes 39m Normal Created pod/payload-pre-processing-e2e-api-a-31ad7b-5c4dcf55-2fmdz Container created 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-31ad7b-74bb7bcdf8 Created pod: payload-processing-e2e-api-a-31ad7b-74bb7bcdf8-fdhjn 39m Normal Pulled pod/payload-pre-processing-e2e-api-a-31ad7b-5c4dcf55-2fmdz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-31ad7b-5c4dcf55 Created pod: payload-pre-processing-e2e-api-a-31ad7b-5c4dcf55-2fmdz 39m Normal Started pod/payload-pre-processing-e2e-api-a-31ad7b-5c4dcf55-2fmdz Container started 39m Normal Created pod/payload-processing-e2e-api-a-31ad7b-74bb7bcdf8-fdhjn Container created 39m Normal Started pod/payload-processing-e2e-api-a-31ad7b-74bb7bcdf8-fdhjn Container started 39m Normal Pulled pod/payload-processing-e2e-api-a-31ad7b-74bb7bcdf8-fdhjn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Started pod/e2e-inf-a-9b4383-openshift-default-64f46d78c7-lblz9 Container started 39m Normal Pulled pod/e2e-inf-a-9b4383-openshift-default-64f46d78c7-lblz9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-inf-a-9b4383-openshift-default-64f46d78c7-lblz9 Container created 39m Normal Killing pod/e2e-inf-a-2420f6-openshift-default-75c86ff8ff-ltdfl Stopping container istio-proxy 39m Warning ProbeError pod/e2e-inf-a-9b4383-openshift-default-64f46d78c7-lblz9 Startup probe error: Get "http://10.133.0.202:15021/healthz/ready": dial tcp 10.133.0.202:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-inf-a-9b4383-openshift-default-64f46d78c7-lblz9 Startup probe failed: Get "http://10.133.0.202:15021/healthz/ready": dial tcp 10.133.0.202:15021: connect: connection refused 39m Normal Killing pod/payload-pre-processing-e2e-api-b-2769cb-7668b55d7-b9vqz Stopping container payload-pre-processing 39m Normal Killing pod/payload-processing-e2e-api-b-2769cb-85cd775f46-hxl4x Stopping container payload-processing 39m Normal Killing pod/payload-processing-e2e-inf-b-9674f5-699cd9d748-v2bnl Stopping container payload-processing 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-9b4383-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal Killing pod/payload-pre-processing-e2e-inf-b-9674f5-8cdb85d67-dpjcx Stopping container payload-pre-processing 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-9b4383-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-9b4383-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-9b4383-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-31ad7b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-31ad7b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-18c03a-openshift-default New size: 2; reason: All metrics below target 38m Normal Killing pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r7xzt Stopping container istio-proxy 38m Normal SuccessfulDelete replicaset/e2e-worker-w4-18c03a-openshift-default-589bbc88df Deleted pod: e2e-worker-w4-18c03a-openshift-default-589bbc88df-r7xzt 38m Normal ScalingReplicaSet deployment/e2e-worker-w4-18c03a-openshift-default Scaled down replica set e2e-worker-w4-18c03a-openshift-default-589bbc88df from 3 to 2 38m Warning ProbeError pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r7xzt Readiness probe error: HTTP probe failed with statuscode: 503... 38m Warning Unhealthy pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-r7xzt Readiness probe failed: HTTP probe failed with statuscode: 503 38m Normal AddedLabel gateway/e2e-inf-b-2cea18 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-2cea18 38m Normal SuccessfulCreate replicaset/e2e-inf-b-2cea18-openshift-default-794d55d96f Created pod: e2e-inf-b-2cea18-openshift-default-794d55d96f-rmqhf 38m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-59f957fdd7 Created pod: maas-default-gateway-openshift-default-59f957fdd7-j6q6l 38m Normal AddedInterface pod/e2e-inf-b-2cea18-openshift-default-794d55d96f-rmqhf Add eth0 [10.132.0.128/23] from ovn-kubernetes 38m Normal ScalingReplicaSet deployment/e2e-inf-b-2cea18-openshift-default Scaled up replica set e2e-inf-b-2cea18-openshift-default-794d55d96f from 0 to 1 38m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 38m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-j6q6l Container started 38m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-jlhrc Add eth0 [10.132.0.130/23] from ovn-kubernetes 38m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-jlhrc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-jlhrc Container created 38m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-j6q6l Container created 38m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-j6q6l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-jlhrc Container started 38m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-j6q6l Add eth0 [10.132.0.129/23] from ovn-kubernetes 38m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-jlhrc Startup probe error: Get "http://10.132.0.130:15021/healthz/ready": dial tcp 10.132.0.130:15021: connect: connection refused... 38m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-jlhrc Startup probe failed: Get "http://10.132.0.130:15021/healthz/ready": dial tcp 10.132.0.130:15021: connect: connection refused 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-2cea18-5bcdcd4dff Created pod: payload-pre-processing-e2e-inf-b-2cea18-5bcdcd4dff-bczwf 38m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-j6q6l Startup probe failed: Get "http://10.132.0.129:15021/healthz/ready": dial tcp 10.132.0.129:15021: connect: connection refused 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-2cea18 Scaled up replica set payload-processing-e2e-inf-b-2cea18-6fbbf7d74b from 0 to 1 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-2cea18 Scaled up replica set payload-pre-processing-e2e-inf-b-2cea18-5bcdcd4dff from 0 to 1 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-2cea18-6fbbf7d74b Created pod: payload-processing-e2e-inf-b-2cea18-6fbbf7d74b-wknm7 38m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-2cea18-6fbbf7d74b Error creating: pods "payload-processing-e2e-inf-b-2cea18-6fbbf7d74b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-2cea18: serviceaccount "payload-processing-e2e-inf-b-2cea18" not found 38m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-j6q6l Startup probe error: Get "http://10.132.0.129:15021/healthz/ready": dial tcp 10.132.0.129:15021: connect: connection refused... 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-2cea18-5bcdcd4dff Error creating: pods "payload-pre-processing-e2e-inf-b-2cea18-5bcdcd4dff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-2cea18: serviceaccount "payload-processing-e2e-inf-b-2cea18" not found 38m Normal Created pod/payload-pre-processing-e2e-inf-b-2cea18-5bcdcd4dff-bczwf Container created 38m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-2cea18-5bcdcd4dff-bczwf Add eth0 [10.133.0.215/23] from ovn-kubernetes 38m Normal AddedInterface pod/payload-processing-e2e-inf-b-2cea18-6fbbf7d74b-wknm7 Add eth0 [10.133.0.216/23] from ovn-kubernetes 38m Normal Started pod/payload-pre-processing-e2e-inf-b-2cea18-5bcdcd4dff-bczwf Container started 38m Normal Pulled pod/payload-pre-processing-e2e-inf-b-2cea18-5bcdcd4dff-bczwf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Started pod/payload-processing-e2e-inf-b-2cea18-6fbbf7d74b-wknm7 Container started 38m Normal Pulled pod/payload-processing-e2e-inf-b-2cea18-6fbbf7d74b-wknm7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Created pod/payload-processing-e2e-inf-b-2cea18-6fbbf7d74b-wknm7 Container created 38m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-ndnzf Container created 38m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-nr7zh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-ndnzf Add eth0 [10.133.0.217/23] from ovn-kubernetes 38m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-std4j Container started 38m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-std4j Container created 38m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-std4j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-std4j Add eth0 [10.132.0.131/23] from ovn-kubernetes 38m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-ndnzf Container started 38m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-nr7zh Container created 38m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-nr7zh Add eth0 [10.132.0.132/23] from ovn-kubernetes 38m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-nr7zh Container started 38m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-ndnzf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-nr7zh Startup probe failed: Get "http://10.132.0.132:15021/healthz/ready": dial tcp 10.132.0.132:15021: connect: connection refused 38m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-ndnzf Startup probe error: Get "http://10.133.0.217:15021/healthz/ready": dial tcp 10.133.0.217:15021: connect: connection refused... 38m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-ndnzf Startup probe failed: Get "http://10.133.0.217:15021/healthz/ready": dial tcp 10.133.0.217:15021: connect: connection refused 38m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-nr7zh Startup probe error: Get "http://10.132.0.132:15021/healthz/ready": dial tcp 10.132.0.132:15021: connect: connection refused... 38m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-std4j Startup probe error: Get "http://10.132.0.131:15021/healthz/ready": dial tcp 10.132.0.131:15021: connect: connection refused... 38m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-std4j Startup probe failed: Get "http://10.132.0.131:15021/healthz/ready": dial tcp 10.132.0.131:15021: connect: connection refused 38m Normal Killing pod/e2e-api-a-2627c5-openshift-default-6c84fbdb55-5g5zm Stopping container istio-proxy 38m Warning Unhealthy pod/e2e-api-a-2627c5-openshift-default-6c84fbdb55-5g5zm Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning ProbeError pod/e2e-api-a-2627c5-openshift-default-6c84fbdb55-5g5zm Readiness probe error: HTTP probe failed with statuscode: 503... 37m Normal Killing pod/payload-pre-processing-e2e-api-b-d6296b-b54fff875-sktff Stopping container payload-pre-processing 37m Normal Killing pod/payload-processing-e2e-api-b-d6296b-95dc94cbb-k2wv7 Stopping container payload-processing 37m Normal Killing pod/e2e-mt-12450a06-openshift-default-7576d969f7-zp9gc Stopping container istio-proxy 37m Normal Killing pod/e2e-mt-12450a06-openshift-default-7576d969f7-6zght Stopping container istio-proxy 37m Normal Killing pod/e2e-mt-12450a06-openshift-default-7576d969f7-t5c69 Stopping container istio-proxy 37m Normal Killing pod/e2e-mt-12450a06-openshift-default-7576d969f7-t67gd Stopping container istio-proxy 37m Normal Killing pod/e2e-mt-12450a06-openshift-default-7576d969f7-5l758 Stopping container istio-proxy 37m Normal Killing pod/e2e-mt-12450a06-openshift-default-7576d969f7-vrdvc Stopping container istio-proxy 37m Normal Killing pod/e2e-mt-12450a06-openshift-default-7576d969f7-kh8pc Stopping container istio-proxy 37m Normal Killing pod/e2e-api-b-2769cb-openshift-default-554f5c6bf-9zk2q Stopping container istio-proxy 37m Normal Killing pod/e2e-inf-b-9674f5-openshift-default-7b45c9f895-w26vx Stopping container istio-proxy 37m Normal Killing pod/payload-pre-processing-e2e-shared-a-w4-65cac0-85d8c6fb67-ln66p Stopping container payload-pre-processing 37m Normal Killing pod/payload-processing-e2e-shared-a-w4-65cac0-7bbdd5876-c484q Stopping container payload-processing 37m Normal Started pod/e2e-api-a-31ad7b-openshift-default-65dcf6d7f7-d6fz2 Container started 37m Normal Created pod/e2e-api-a-31ad7b-openshift-default-65dcf6d7f7-d6fz2 Container created 37m Normal Pulled pod/e2e-api-a-31ad7b-openshift-default-65dcf6d7f7-d6fz2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Killing pod/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr Stopping container payload-pre-processing 37m Normal Killing pod/payload-processing-e2e-worker-w3-961054-66867965c8-8zxjh Stopping container payload-processing 37m Warning ProbeError pod/e2e-api-a-31ad7b-openshift-default-65dcf6d7f7-d6fz2 Startup probe error: Get "http://10.133.0.212:15021/healthz/ready": dial tcp 10.133.0.212:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-api-a-31ad7b-openshift-default-65dcf6d7f7-d6fz2 Startup probe failed: Get "http://10.133.0.212:15021/healthz/ready": dial tcp 10.133.0.212:15021: connect: connection refused 37m Warning Unhealthy pod/payload-processing-e2e-worker-w3-961054-66867965c8-8zxjh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w3-961054-64cfb455d7-vmwbr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Normal Killing pod/e2e-api-b-d6296b-openshift-default-7f98f9c4d5-p9g4x Stopping container istio-proxy 36m Normal ScalingReplicaSet deployment/e2e-inf-a-0bf80c-openshift-default Scaled up replica set e2e-inf-a-0bf80c-openshift-default-6566586789 from 0 to 1 36m Normal SuccessfulCreate replicaset/e2e-inf-a-0bf80c-openshift-default-6566586789 Created pod: e2e-inf-a-0bf80c-openshift-default-6566586789-wsnl8 36m Normal AddedLabel gateway/e2e-api-b-656ee6 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-656ee6 36m Normal AddedLabel gateway/e2e-inf-a-0bf80c Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-0bf80c 36m Normal ScalingReplicaSet deployment/e2e-api-b-656ee6-openshift-default Scaled up replica set e2e-api-b-656ee6-openshift-default-7bc94fbdf7 from 0 to 1 36m Normal AddedInterface pod/e2e-inf-a-0bf80c-openshift-default-6566586789-wsnl8 Add eth0 [10.133.0.230/23] from ovn-kubernetes 36m Normal SuccessfulCreate replicaset/e2e-api-b-656ee6-openshift-default-7bc94fbdf7 Created pod: e2e-api-b-656ee6-openshift-default-7bc94fbdf7-lf2vj 36m Normal AddedInterface pod/e2e-api-b-656ee6-openshift-default-7bc94fbdf7-lf2vj Add eth0 [10.132.0.137/23] from ovn-kubernetes 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-31ad7b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-31ad7b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-0bf80c Scaled up replica set payload-processing-e2e-inf-a-0bf80c-cd68887bb from 0 to 1 36m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-0bf80c-cd68887bb Error creating: pods "payload-processing-e2e-inf-a-0bf80c-cd68887bb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-0bf80c: serviceaccount "payload-processing-e2e-inf-a-0bf80c" not found 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-0bf80c-5dc5c7cfc9 Created pod: payload-pre-processing-e2e-inf-a-0bf80c-5dc5c7cfc9-cfdqp 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-0bf80c Scaled up replica set payload-pre-processing-e2e-inf-a-0bf80c-5dc5c7cfc9 from 0 to 1 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-0bf80c-5dc5c7cfc9 Error creating: pods "payload-pre-processing-e2e-inf-a-0bf80c-5dc5c7cfc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-0bf80c: serviceaccount "payload-processing-e2e-inf-a-0bf80c" not found 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-0bf80c-cd68887bb Created pod: payload-processing-e2e-inf-a-0bf80c-cd68887bb-m4r47 36m Normal Created pod/payload-processing-e2e-inf-a-0bf80c-cd68887bb-m4r47 Container created 36m Normal Started pod/payload-processing-e2e-inf-a-0bf80c-cd68887bb-m4r47 Container started 36m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-0bf80c-5dc5c7cfc9-cfdqp Add eth0 [10.133.0.232/23] from ovn-kubernetes 36m Normal Pulled pod/payload-pre-processing-e2e-inf-a-0bf80c-5dc5c7cfc9-cfdqp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Created pod/payload-pre-processing-e2e-inf-a-0bf80c-5dc5c7cfc9-cfdqp Container created 36m Normal Pulled pod/payload-processing-e2e-inf-a-0bf80c-cd68887bb-m4r47 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/payload-processing-e2e-inf-a-0bf80c-cd68887bb-m4r47 Add eth0 [10.133.0.231/23] from ovn-kubernetes 36m Normal Started pod/payload-pre-processing-e2e-inf-a-0bf80c-5dc5c7cfc9-cfdqp Container started 36m Warning FailedCreate replicaset/payload-processing-e2e-api-b-656ee6-7d7cf8d498 Error creating: pods "payload-processing-e2e-api-b-656ee6-7d7cf8d498-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-656ee6: serviceaccount "payload-processing-e2e-api-b-656ee6" not found 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-656ee6 Scaled up replica set payload-processing-e2e-api-b-656ee6-7d7cf8d498 from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-656ee6-6f576464c Created pod: payload-pre-processing-e2e-api-b-656ee6-6f576464c-vsgw4 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-656ee6-6f576464c Error creating: pods "payload-pre-processing-e2e-api-b-656ee6-6f576464c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-656ee6: serviceaccount "payload-processing-e2e-api-b-656ee6" not found 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-656ee6 Scaled up replica set payload-pre-processing-e2e-api-b-656ee6-6f576464c from 0 to 1 36m Normal Started pod/payload-processing-e2e-api-b-656ee6-7d7cf8d498-9blpz Container started 36m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-656ee6-6f576464c-vsgw4 Add eth0 [10.133.0.235/23] from ovn-kubernetes 36m Normal AddedInterface pod/payload-processing-e2e-api-b-656ee6-7d7cf8d498-9blpz Add eth0 [10.133.0.236/23] from ovn-kubernetes 36m Normal Pulled pod/payload-processing-e2e-api-b-656ee6-7d7cf8d498-9blpz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Created pod/payload-processing-e2e-api-b-656ee6-7d7cf8d498-9blpz Container created 36m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-std4j Readiness probe failed: Get "http://10.132.0.131:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-656ee6-7d7cf8d498 Created pod: payload-processing-e2e-api-b-656ee6-7d7cf8d498-9blpz 36m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-nr7zh Readiness probe failed: Get "http://10.132.0.132:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 36m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-std4j Readiness probe error: Get "http://10.132.0.131:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 36m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-nr7zh Readiness probe error: Get "http://10.132.0.132:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 36m Normal Created pod/payload-pre-processing-e2e-api-b-656ee6-6f576464c-vsgw4 Container created 36m Normal Pulled pod/payload-pre-processing-e2e-api-b-656ee6-6f576464c-vsgw4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Started pod/payload-pre-processing-e2e-api-b-656ee6-6f576464c-vsgw4 Container started 36m Warning ProbeError pod/e2e-inf-b-2cea18-openshift-default-794d55d96f-rmqhf Readiness probe error: Get "http://10.132.0.128:15021/healthz/ready": dial tcp 10.132.0.128:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-inf-b-2cea18-openshift-default-794d55d96f-rmqhf Readiness probe failed: Get "http://10.132.0.128:15021/healthz/ready": dial tcp 10.132.0.128:15021: connect: connection refused 36m Normal Pulled pod/e2e-inf-b-2cea18-openshift-default-794d55d96f-rmqhf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/e2e-inf-b-2cea18-openshift-default-794d55d96f-rmqhf Container started 36m Normal Created pod/e2e-inf-b-2cea18-openshift-default-794d55d96f-rmqhf Container created 36m Warning Unhealthy pod/e2e-inf-b-2cea18-openshift-default-794d55d96f-rmqhf Startup probe failed: Get "http://10.132.0.128:15021/healthz/ready": dial tcp 10.132.0.128:15021: connect: connection refused 36m Warning ProbeError pod/e2e-inf-b-2cea18-openshift-default-794d55d96f-rmqhf Startup probe error: Get "http://10.132.0.128:15021/healthz/ready": dial tcp 10.132.0.128:15021: connect: connection refused... 36m Normal Killing pod/e2e-shared-a-w4-65cac0-openshift-default-575894bb79-48g55 Stopping container istio-proxy 36m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-65cac0-openshift-default Scaled up replica set e2e-shared-a-w4-65cac0-openshift-default-575894bb79 from 0 to 1 36m Warning FailedCreate replicaset/e2e-shared-a-w4-65cac0-openshift-default-575894bb79 Error creating: pods "e2e-shared-a-w4-65cac0-openshift-default-575894bb79-" is forbidden: error looking up service account openshift-ingress/e2e-shared-a-w4-65cac0-openshift-default: serviceaccount "e2e-shared-a-w4-65cac0-openshift-default" not found 36m Normal Killing pod/payload-processing-e2e-shared-b-w4-a03fee-5d9bb89966-kfsbz Stopping container payload-processing 36m Normal Killing pod/payload-pre-processing-e2e-shared-b-w4-a03fee-5595ccdb5f-xgxjk Stopping container payload-pre-processing 36m Normal Killing pod/payload-processing-e2e-worker-w1-545b5d-6ff8c4558-75gnb Stopping container payload-processing 36m Normal Killing pod/payload-pre-processing-e2e-worker-w1-545b5d-5c89cdd65f-w2cnc Stopping container payload-pre-processing 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-2cea18-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-2cea18-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal Killing pod/payload-processing-e2e-worker-w2-ee74cb-6479cc6f4b-5qpss Stopping container payload-processing 36m Normal Killing pod/payload-pre-processing-e2e-worker-w2-ee74cb-5786ddf6ff-lb7f5 Stopping container payload-pre-processing 35m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-ee74cb-5786ddf6ff-lb7f5 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 35m Warning Unhealthy pod/payload-processing-e2e-worker-w2-ee74cb-6479cc6f4b-5qpss Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 35m Normal Killing pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-b69rq Stopping container istio-proxy 35m Normal Killing pod/e2e-worker-w3-961054-openshift-default-5d5665c66f-zr6k7 Stopping container istio-proxy 35m Warning Unhealthy pod/e2e-inf-a-0bf80c-openshift-default-6566586789-wsnl8 Readiness probe failed: Get "http://10.133.0.230:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 35m Warning ProbeError pod/e2e-inf-a-0bf80c-openshift-default-6566586789-wsnl8 Readiness probe error: Get "http://10.133.0.230:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 35m Normal Pulled pod/e2e-api-b-656ee6-openshift-default-7bc94fbdf7-lf2vj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Warning Unhealthy pod/e2e-api-b-656ee6-openshift-default-7bc94fbdf7-lf2vj Readiness probe failed: Get "http://10.132.0.137:15021/healthz/ready": dial tcp 10.132.0.137:15021: connect: connection refused 35m Warning ProbeError pod/e2e-api-b-656ee6-openshift-default-7bc94fbdf7-lf2vj Readiness probe error: Get "http://10.132.0.137:15021/healthz/ready": dial tcp 10.132.0.137:15021: connect: connection refused... 35m Normal Started pod/e2e-api-b-656ee6-openshift-default-7bc94fbdf7-lf2vj Container started 35m Normal Created pod/e2e-api-b-656ee6-openshift-default-7bc94fbdf7-lf2vj Container created 35m Normal Started pod/e2e-inf-a-0bf80c-openshift-default-6566586789-wsnl8 Container started 35m Normal Created pod/e2e-inf-a-0bf80c-openshift-default-6566586789-wsnl8 Container created 35m Normal Pulled pod/e2e-inf-a-0bf80c-openshift-default-6566586789-wsnl8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Warning Unhealthy pod/e2e-api-b-656ee6-openshift-default-7bc94fbdf7-lf2vj Startup probe failed: Get "http://10.132.0.137:15021/healthz/ready": dial tcp 10.132.0.137:15021: connect: connection refused 35m Warning ProbeError pod/e2e-api-b-656ee6-openshift-default-7bc94fbdf7-lf2vj Startup probe error: Get "http://10.132.0.137:15021/healthz/ready": dial tcp 10.132.0.137:15021: connect: connection refused... 35m Warning ProbeError pod/e2e-inf-a-0bf80c-openshift-default-6566586789-wsnl8 Startup probe error: Get "http://10.133.0.230:15021/healthz/ready": dial tcp 10.133.0.230:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-inf-a-0bf80c-openshift-default-6566586789-wsnl8 Startup probe failed: Get "http://10.133.0.230:15021/healthz/ready": dial tcp 10.133.0.230:15021: connect: connection refused 35m Normal Killing pod/e2e-shared-b-w4-a03fee-openshift-default-5f8f5cbff7-pzmcj Stopping container istio-proxy 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-656ee6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-656ee6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Normal Created pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zc5t5 Container created 35m Normal AddedInterface pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zc5t5 Add eth0 [10.133.0.247/23] from ovn-kubernetes 35m Normal ScalingReplicaSet deployment/e2e-worker-w2-ee74cb-openshift-default Scaled up replica set e2e-worker-w2-ee74cb-openshift-default-9758876f from 2 to 3 35m Normal Started pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-t6mjs Container started 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-ee74cb-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 35m Normal SuccessfulCreate replicaset/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c Created pod: e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zc5t5 35m Normal SuccessfulCreate replicaset/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c Created pod: e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-t6mjs 35m Normal SuccessfulCreate replicaset/e2e-worker-w2-ee74cb-openshift-default-9758876f Created pod: e2e-worker-w2-ee74cb-openshift-default-9758876f-dwx77 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-545b5d-openshift-default (combined from similar events): New size: 4; reason: cpu resource utilization (percentage of request) above target 35m Normal Created pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-t6mjs Container created 35m Normal Pulled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zc5t5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Pulled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-t6mjs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Started pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zc5t5 Container started 35m Normal AddedInterface pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-t6mjs Add eth0 [10.132.0.141/23] from ovn-kubernetes 35m Normal AddedInterface pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-dwx77 Add eth0 [10.133.0.248/23] from ovn-kubernetes 35m Normal Started pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-dwx77 Container started 35m Normal Pulled pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-dwx77 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-dwx77 Container created 34m Normal Killing pod/payload-pre-processing-e2e-worker-w4-18c03a-d7d874c9-bn495 Stopping container payload-pre-processing 34m Normal Killing pod/payload-processing-e2e-worker-w4-18c03a-85c44654fd-b8t9v Stopping container payload-processing 34m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zc5t5 Startup probe error: Get "http://10.133.0.247:15021/healthz/ready": dial tcp 10.133.0.247:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-t6mjs Startup probe failed: Get "http://10.132.0.141:15021/healthz/ready": dial tcp 10.132.0.141:15021: connect: connection refused 34m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-t6mjs Startup probe error: Get "http://10.132.0.141:15021/healthz/ready": dial tcp 10.132.0.141:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-dwx77 Startup probe failed: Get "http://10.133.0.248:15021/healthz/ready": dial tcp 10.133.0.248:15021: connect: connection refused 34m Warning ProbeError pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-dwx77 Startup probe error: Get "http://10.133.0.248:15021/healthz/ready": dial tcp 10.133.0.248:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zc5t5 Startup probe failed: Get "http://10.133.0.247:15021/healthz/ready": dial tcp 10.133.0.247:15021: connect: connection refused 34m Normal SuccessfulCreate replicaset/e2e-inf-b-ac6633-openshift-default-d5fcb767c Created pod: e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v 34m Normal AddedLabel gateway/e2e-inf-b-ac6633 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-ac6633 34m Normal ScalingReplicaSet deployment/e2e-inf-b-ac6633-openshift-default Scaled up replica set e2e-inf-b-ac6633-openshift-default-d5fcb767c from 0 to 1 34m Normal AddedInterface pod/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v Add eth0 [10.134.0.107/23] from ovn-kubernetes 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-0bf80c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-0bf80c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-656ee6-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-656ee6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-545b5d-openshift-default New size: 5; reason: All metrics below target 34m Normal ScalingReplicaSet deployment/e2e-worker-w1-545b5d-openshift-default (combined from similar events): Scaled up replica set e2e-worker-w1-545b5d-openshift-default-5c8c66f65c from 4 to 5 34m Normal SuccessfulCreate replicaset/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c Created pod: e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6 34m Normal AddedInterface pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6 Add eth0 [10.134.0.108/23] from ovn-kubernetes 34m Normal Pulled pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Created pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6 Container created 34m Normal Started pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6 Container started 34m Normal Killing pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-b7xdq Stopping container istio-proxy 34m Normal Killing pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-dwx77 Stopping container istio-proxy 34m Normal Killing pod/e2e-worker-w2-ee74cb-openshift-default-9758876f-t5tqz Stopping container istio-proxy 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-ac6633 Scaled up replica set payload-processing-e2e-inf-b-ac6633-64d9fbd554 from 0 to 1 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-ac6633 Scaled up replica set payload-pre-processing-e2e-inf-b-ac6633-7bf6856fbc from 0 to 1 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-ac6633-64d9fbd554 Created pod: payload-processing-e2e-inf-b-ac6633-64d9fbd554-vtdtd 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-ac6633-7bf6856fbc Created pod: payload-pre-processing-e2e-inf-b-ac6633-7bf6856fbc-5nf4g 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-ac6633-7bf6856fbc Error creating: pods "payload-pre-processing-e2e-inf-b-ac6633-7bf6856fbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-ac6633: serviceaccount "payload-processing-e2e-inf-b-ac6633" not found 34m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-ac6633-64d9fbd554 Error creating: pods "payload-processing-e2e-inf-b-ac6633-64d9fbd554-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-ac6633: serviceaccount "payload-processing-e2e-inf-b-ac6633" not found 34m Normal Created pod/payload-processing-e2e-inf-b-ac6633-64d9fbd554-vtdtd Container created 34m Normal Pulled pod/payload-processing-e2e-inf-b-ac6633-64d9fbd554-vtdtd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Started pod/payload-processing-e2e-inf-b-ac6633-64d9fbd554-vtdtd Container started 34m Normal Started pod/payload-pre-processing-e2e-inf-b-ac6633-7bf6856fbc-5nf4g Container started 34m Normal AddedInterface pod/payload-processing-e2e-inf-b-ac6633-64d9fbd554-vtdtd Add eth0 [10.133.0.251/23] from ovn-kubernetes 34m Normal Created pod/payload-pre-processing-e2e-inf-b-ac6633-7bf6856fbc-5nf4g Container created 34m Normal Pulled pod/payload-pre-processing-e2e-inf-b-ac6633-7bf6856fbc-5nf4g Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-ac6633-7bf6856fbc-5nf4g Add eth0 [10.133.0.250/23] from ovn-kubernetes 34m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6 Startup probe error: Get "http://10.134.0.108:15021/healthz/ready": dial tcp 10.134.0.108:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6 Startup probe failed: Get "http://10.134.0.108:15021/healthz/ready": dial tcp 10.134.0.108:15021: connect: connection refused 34m Normal Killing pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl Stopping container istio-proxy 34m Normal Killing pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-t6mjs Stopping container istio-proxy 34m Normal Killing pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5ctxt Stopping container istio-proxy 34m Normal Killing pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zc5t5 Stopping container istio-proxy 34m Normal Killing pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-zppv6 Stopping container istio-proxy 34m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl Readiness probe error: HTTP probe failed with statuscode: 503... 34m Warning Unhealthy pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-qcsvl Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning ProbeError pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5ctxt Readiness probe error: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-worker-w1-545b5d-openshift-default-5c8c66f65c-5ctxt Readiness probe failed: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused 34m Normal Killing pod/payload-processing-e2e-api-a-31ad7b-74bb7bcdf8-fdhjn Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-api-a-31ad7b-5c4dcf55-2fmdz Stopping container payload-pre-processing 33m Normal Pulled pod/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v Container created 33m Normal Started pod/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v Container started 33m Warning ProbeError pod/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v Startup probe error: Get "http://10.134.0.107:15021/healthz/ready": dial tcp 10.134.0.107:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v Startup probe failed: Get "http://10.134.0.107:15021/healthz/ready": dial tcp 10.134.0.107:15021: connect: connection refused 33m Normal Killing pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-5qmh2 Stopping container istio-proxy 33m Normal Killing pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-nml5b Stopping container istio-proxy 33m Warning ProbeError pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-5qmh2 Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-worker-w4-18c03a-openshift-default-589bbc88df-5qmh2 Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-ac6633-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-ac6633-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Normal Killing pod/e2e-api-a-31ad7b-openshift-default-65dcf6d7f7-d6fz2 Stopping container istio-proxy 33m Warning Unhealthy pod/e2e-api-a-31ad7b-openshift-default-65dcf6d7f7-d6fz2 Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-api-a-31ad7b-openshift-default-65dcf6d7f7-d6fz2 Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-ac6633-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-ac6633-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Normal Killing pod/payload-processing-e2e-api-b-656ee6-7d7cf8d498-9blpz Stopping container payload-processing 33m Normal Killing pod/payload-pre-processing-e2e-api-b-656ee6-6f576464c-vsgw4 Stopping container payload-pre-processing 31m Normal Killing pod/e2e-api-b-656ee6-openshift-default-7bc94fbdf7-lf2vj Stopping container istio-proxy 31m Normal Killing pod/payload-processing-e2e-worker-w6-5e72ad-69d4b7cbb-r4xrh Stopping container payload-processing 31m Normal Killing pod/payload-pre-processing-e2e-worker-w6-5e72ad-65d5d94fc9-6r57s Stopping container payload-pre-processing 31m Normal Killing pod/payload-pre-processing-e2e-inf-a-9b4383-7c7d96654c-wj9bn Stopping container payload-pre-processing 31m Normal Killing pod/payload-processing-e2e-inf-a-9b4383-5b9cf487f8-qlxxp Stopping container payload-processing 31m Normal Killing pod/e2e-inf-a-9b4383-openshift-default-64f46d78c7-lblz9 Stopping container istio-proxy 30m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-9b4383-7c7d96654c-wj9bn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Normal Killing pod/payload-processing-e2e-inf-b-2cea18-6fbbf7d74b-wknm7 Stopping container payload-processing 30m Normal Killing pod/payload-pre-processing-e2e-inf-b-2cea18-5bcdcd4dff-bczwf Stopping container payload-pre-processing 30m Normal Started pod/e2e-inf-a-0bf80c-openshift-default-6566586789-zwtht Container started 30m Normal Created pod/e2e-inf-a-0bf80c-openshift-default-6566586789-zwtht Container created 30m Normal SuccessfulCreate replicaset/e2e-inf-a-0bf80c-openshift-default-6566586789 Created pod: e2e-inf-a-0bf80c-openshift-default-6566586789-zwtht 30m Normal AddedInterface pod/e2e-inf-a-0bf80c-openshift-default-6566586789-zwtht Add eth0 [10.132.0.143/23] from ovn-kubernetes 30m Normal ScalingReplicaSet deployment/e2e-inf-a-0bf80c-openshift-default Scaled up replica set e2e-inf-a-0bf80c-openshift-default-6566586789 from 1 to 2 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-0bf80c-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 30m Normal Pulled pod/e2e-inf-a-0bf80c-openshift-default-6566586789-zwtht Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-68fbd1-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 30m Normal Killing pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-8t4p6 Stopping container istio-proxy 30m Normal ScalingReplicaSet deployment/e2e-worker-w0-68fbd1-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 from 2 to 1 30m Normal SuccessfulDelete replicaset/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67 Deleted pod: e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-8t4p6 30m Warning ProbeError pod/e2e-inf-a-0bf80c-openshift-default-6566586789-zwtht Startup probe error: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-inf-a-0bf80c-openshift-default-6566586789-zwtht Startup probe failed: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused 30m Normal Killing pod/e2e-inf-b-2cea18-openshift-default-794d55d96f-rmqhf Stopping container istio-proxy 30m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-2cea18-5bcdcd4dff-bczwf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: All metrics below target 30m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-59f957fdd7 Deleted pod: maas-default-gateway-openshift-default-59f957fdd7-nr7zh 30m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-59f957fdd7 Deleted pod: maas-default-gateway-openshift-default-59f957fdd7-jlhrc 30m Normal Killing pod/maas-default-gateway-openshift-default-59f957fdd7-nr7zh Stopping container istio-proxy 30m Normal Killing pod/maas-default-gateway-openshift-default-59f957fdd7-std4j Stopping container istio-proxy 30m Normal Killing pod/maas-default-gateway-openshift-default-59f957fdd7-jlhrc Stopping container istio-proxy 30m Normal Killing pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-q27rd Stopping container istio-proxy 30m Normal Killing pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-mvhjt Stopping container istio-proxy 30m Warning Unhealthy pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-mvhjt Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning ProbeError pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-mvhjt Readiness probe error: HTTP probe failed with statuscode: 503... 30m Normal Killing pod/payload-processing-e2e-worker-w0-68fbd1-6d979f99cb-65n5z Stopping container payload-processing 30m Normal Killing pod/payload-pre-processing-e2e-worker-w0-68fbd1-76d7f5c887-p9lsw Stopping container payload-pre-processing 30m Warning ProbeError pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-q27rd Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning Unhealthy pod/e2e-worker-w6-5e72ad-openshift-default-5fbd65f49c-q27rd Readiness probe failed: HTTP probe failed with statuscode: 503 30m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: All metrics below target 30m Normal Killing pod/maas-default-gateway-openshift-default-59f957fdd7-ndnzf Stopping container istio-proxy 29m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-68fbd1-76d7f5c887-p9lsw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Normal Killing pod/e2e-worker-w0-68fbd1-openshift-default-5f969f4f67-4ln8d Stopping container istio-proxy 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-68fbd1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-68fbd1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning FailedUpdateStatus horizontalpodautoscaler/e2e-worker-w0-68fbd1-openshift-default horizontalpodautoscalers.autoscaling "e2e-worker-w0-68fbd1-openshift-default" not found 29m Normal SuccessfulDelete replicaset/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 Deleted pod: e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm 29m Normal Killing pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-flcdm Stopping container istio-proxy 29m Normal ScalingReplicaSet deployment/e2e-worker-w5-6f1190-openshift-default Scaled down replica set e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 from 2 to 1 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6f1190-openshift-default New size: 1; reason: All metrics below target 29m Normal Killing pod/maas-default-gateway-openshift-default-59f957fdd7-f64cj Stopping container istio-proxy 29m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: All metrics below target 29m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-f64cj Readiness probe failed: HTTP probe failed with statuscode: 503 29m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-f64cj Readiness probe error: HTTP probe failed with statuscode: 503... 29m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-xkzbs 29m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 29m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 29m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-xkzbs Stopping container discovery 29m Normal Killing pod/maas-default-gateway-openshift-default-59f957fdd7-xmfvp Stopping container istio-proxy 29m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: All metrics below target 29m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-xmfvp Readiness probe failed: HTTP probe failed with statuscode: 503 29m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-xmfvp Readiness probe error: HTTP probe failed with statuscode: 503... 28m Normal Killing pod/payload-processing-e2e-inf-a-0bf80c-cd68887bb-m4r47 Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-inf-a-0bf80c-5dc5c7cfc9-cfdqp Stopping container payload-pre-processing 27m Normal Killing pod/e2e-inf-a-0bf80c-openshift-default-6566586789-wsnl8 Stopping container istio-proxy 27m Normal Killing pod/e2e-inf-a-0bf80c-openshift-default-6566586789-zwtht Stopping container istio-proxy 27m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-0bf80c-5dc5c7cfc9-cfdqp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal Killing pod/payload-pre-processing-e2e-inf-b-ac6633-7bf6856fbc-5nf4g Stopping container payload-pre-processing 27m Normal Killing pod/payload-processing-e2e-inf-b-ac6633-64d9fbd554-vtdtd Stopping container payload-processing 27m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-ac6633-7bf6856fbc-5nf4g Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal Killing pod/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v Stopping container istio-proxy 27m Warning Unhealthy pod/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v Readiness probe failed: HTTP probe failed with statuscode: 503 27m Warning ProbeError pod/e2e-inf-b-ac6633-openshift-default-d5fcb767c-p792v Readiness probe error: HTTP probe failed with statuscode: 503... 27m Normal Killing pod/payload-pre-processing-e2e-worker-w5-6f1190-7568b9dd79-jvzsd Stopping container payload-pre-processing 27m Normal Killing pod/payload-processing-e2e-worker-w5-6f1190-c8f5948fd-sd9d7 Stopping container payload-processing 26m Warning Unhealthy pod/payload-processing-e2e-worker-w5-6f1190-c8f5948fd-sd9d7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Normal SuccessfulCreate replicaset/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 Created pod: e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6f1190-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 26m Normal ScalingReplicaSet deployment/e2e-worker-w5-6f1190-openshift-default Scaled up replica set e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5 from 1 to 2 26m Normal Created pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr Container created 26m Normal Started pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr Container started 26m Normal AddedInterface pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr Add eth0 [10.134.0.110/23] from ovn-kubernetes 26m Normal Pulled pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Warning Unhealthy pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr Startup probe failed: Get "http://10.134.0.110:15021/healthz/ready": dial tcp 10.134.0.110:15021: connect: connection refused 26m Warning ProbeError pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr Startup probe error: Get "http://10.134.0.110:15021/healthz/ready": dial tcp 10.134.0.110:15021: connect: connection refused... 26m Normal Killing pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-mzhsz Stopping container istio-proxy 26m Normal Killing pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-tb7zr Stopping container istio-proxy 26m Warning Unhealthy pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-mzhsz Readiness probe failed: HTTP probe failed with statuscode: 503 26m Warning ProbeError pod/e2e-worker-w5-6f1190-openshift-default-59fcbbc8d5-mzhsz Readiness probe error: HTTP probe failed with statuscode: 503... 25m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: All metrics below target 25m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-59f957fdd7 Deleted pod: maas-default-gateway-openshift-default-59f957fdd7-q2jqm 25m Normal Killing pod/maas-default-gateway-openshift-default-59f957fdd7-q2jqm Stopping container istio-proxy 25m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-l9zcx Add eth0 [10.132.0.146/23] from ovn-kubernetes 25m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 25m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-79h22 Add eth0 [10.133.1.17/23] from ovn-kubernetes 25m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-59f957fdd7 from 3 to 6 25m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-chpk6 Add eth0 [10.134.0.111/23] from ovn-kubernetes 25m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-chpk6 Container created 25m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-chpk6 Container started 25m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-79h22 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-l9zcx Container started 25m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-l9zcx Container created 25m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-l9zcx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-79h22 Container created 25m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-79h22 Container started 25m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-chpk6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-79h22 Startup probe error: Get "http://10.133.1.17:15021/healthz/ready": dial tcp 10.133.1.17:15021: connect: connection refused... 25m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-l9zcx Startup probe error: Get "http://10.132.0.146:15021/healthz/ready": dial tcp 10.132.0.146:15021: connect: connection refused... 25m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-chpk6 Startup probe error: Get "http://10.134.0.111:15021/healthz/ready": dial tcp 10.134.0.111:15021: connect: connection refused... 25m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-chpk6 Startup probe failed: Get "http://10.134.0.111:15021/healthz/ready": dial tcp 10.134.0.111:15021: connect: connection refused 25m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-l9zcx Startup probe failed: Get "http://10.132.0.146:15021/healthz/ready": dial tcp 10.132.0.146:15021: connect: connection refused 25m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-79h22 Startup probe failed: Get "http://10.133.1.17:15021/healthz/ready": dial tcp 10.133.1.17:15021: connect: connection refused 25m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default (combined from similar events): New size: 8; reason: All metrics below target 25m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-qrmft Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-qrmft Add eth0 [10.133.1.18/23] from ovn-kubernetes 25m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-qrmft Container created 25m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-66p28 Container created 25m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-qrmft Container started 25m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-66p28 Add eth0 [10.134.0.112/23] from ovn-kubernetes 25m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-66p28 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-66p28 Container started 25m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-qrmft Startup probe error: Get "http://10.133.1.18:15021/healthz/ready": dial tcp 10.133.1.18:15021: connect: connection refused... 25m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-qrmft Startup probe failed: Get "http://10.133.1.18:15021/healthz/ready": dial tcp 10.133.1.18:15021: connect: connection refused 25m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-66p28 Startup probe error: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused... 25m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-66p28 Startup probe failed: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused 23m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 23m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-lch84 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default (combined from similar events): Scaled up replica set maas-default-gateway-openshift-default-59f957fdd7 from 8 to 10 23m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-wkwkn Container created 23m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-lch84 Add eth0 [10.132.0.148/23] from ovn-kubernetes 23m Normal AddedInterface pod/maas-default-gateway-openshift-default-59f957fdd7-wkwkn Add eth0 [10.133.1.20/23] from ovn-kubernetes 23m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-lch84 Container started 23m Normal Pulled pod/maas-default-gateway-openshift-default-59f957fdd7-wkwkn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Created pod/maas-default-gateway-openshift-default-59f957fdd7-lch84 Container created 23m Normal Started pod/maas-default-gateway-openshift-default-59f957fdd7-wkwkn Container started 23m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-wkwkn Startup probe error: Get "http://10.133.1.20:15021/healthz/ready": dial tcp 10.133.1.20:15021: connect: connection refused... 23m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-lch84 Startup probe failed: Get "http://10.132.0.148:15021/healthz/ready": dial tcp 10.132.0.148:15021: connect: connection refused 23m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-lch84 Startup probe error: Get "http://10.132.0.148:15021/healthz/ready": dial tcp 10.132.0.148:15021: connect: connection refused... 23m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-wkwkn Startup probe failed: Get "http://10.133.1.20:15021/healthz/ready": dial tcp 10.133.1.20:15021: connect: connection refused 21m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-j6q6l Readiness probe error: Get "http://10.132.0.129:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-j6q6l Readiness probe failed: Get "http://10.132.0.129:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 20m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-p59kr Readiness probe failed: Get "http://10.134.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 20m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-p59kr Readiness probe error: Get "http://10.134.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 17m Normal AddedLabel gateway/e2e-worker-single-main-1376dc Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-1376dc 17m Normal AddedInterface pod/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g Add eth0 [10.134.0.113/23] from ovn-kubernetes 17m Normal ScalingReplicaSet deployment/e2e-worker-single-main-1376dc-openshift-default Scaled up replica set e2e-worker-single-main-1376dc-openshift-default-56db457c89 from 0 to 1 17m Normal SuccessfulCreate replicaset/e2e-worker-single-main-1376dc-openshift-default-56db457c89 Created pod: e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g 17m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-1376dc Scaled up replica set payload-processing-e2e-worker-single-main-1376dc-6f575447fb from 0 to 1 17m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-1376dc-6f575447fb Error creating: pods "payload-processing-e2e-worker-single-main-1376dc-6f575447fb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-1376dc: serviceaccount "payload-processing-e2e-worker-single-main-1376dc" not found 17m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-1376dc Scaled up replica set payload-pre-processing-e2e-worker-single-main-1376dc-5c8ddd5899 from 0 to 1 17m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-1376dc-6f575447fb Created pod: payload-processing-e2e-worker-single-main-1376dc-6f575447fb7tb8 17m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-1376dc-5c8ddd5899 Created pod: payload-pre-processing-e2e-worker-single-main-1376dc-5c8ddlnpd6 17m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-1376dc-5c8ddd5899 Error creating: pods "payload-pre-processing-e2e-worker-single-main-1376dc-5c8ddd5899-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-1376dc: serviceaccount "payload-processing-e2e-worker-single-main-1376dc" not found 17m Normal Started pod/payload-pre-processing-e2e-worker-single-main-1376dc-5c8ddlnpd6 Container started 17m Normal Started pod/payload-processing-e2e-worker-single-main-1376dc-6f575447fb7tb8 Container started 17m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-1376dc-5c8ddlnpd6 Add eth0 [10.133.1.34/23] from ovn-kubernetes 17m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-1376dc-5c8ddlnpd6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 17m Normal Created pod/payload-processing-e2e-worker-single-main-1376dc-6f575447fb7tb8 Container created 17m Normal Pulled pod/payload-processing-e2e-worker-single-main-1376dc-6f575447fb7tb8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 17m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-1376dc-6f575447fb7tb8 Add eth0 [10.133.1.33/23] from ovn-kubernetes 17m Normal Created pod/payload-pre-processing-e2e-worker-single-main-1376dc-5c8ddlnpd6 Container created 17m Normal Pulled pod/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal Created pod/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g Container created 17m Normal Started pod/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g Container started 17m Warning Unhealthy pod/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g Startup probe failed: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused 17m Warning ProbeError pod/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g Startup probe error: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused... 16m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-1376dc-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 16m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-1376dc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 16m Normal Killing pod/payload-processing-e2e-worker-single-main-1376dc-6f575447fb7tb8 Stopping container payload-processing 16m Normal Killing pod/payload-pre-processing-e2e-worker-single-main-1376dc-5c8ddlnpd6 Stopping container payload-pre-processing 15m Normal Killing pod/e2e-worker-single-main-1376dc-openshift-default-56db457c895fm7g Stopping container istio-proxy 15m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-1376dc-5c8ddlnpd6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-1376dc-6f575447fb7tb8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 13m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-lch84 Readiness probe error: Get "http://10.132.0.148:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 13m Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-wkwkn Readiness probe error: Get "http://10.133.1.20:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-lch84 Readiness probe failed: Get "http://10.132.0.148:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-wkwkn Readiness probe failed: Get "http://10.133.1.20:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 12m Normal SuccessfulCreate replicaset/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5 Created pod: e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l 12m Normal ScalingReplicaSet deployment/e2e-worker-single-main-e433fd-openshift-default Scaled up replica set e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5 from 0 to 1 12m Normal AddedLabel gateway/e2e-worker-single-main-e433fd Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-e433fd 12m Normal AddedInterface pod/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l Add eth0 [10.134.0.115/23] from ovn-kubernetes 12m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-e433fd-5df9fdbc67 Error creating: pods "payload-pre-processing-e2e-worker-single-main-e433fd-5df9fdbc67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-e433fd: serviceaccount "payload-processing-e2e-worker-single-main-e433fd" not found 12m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-e433fd Scaled up replica set payload-pre-processing-e2e-worker-single-main-e433fd-5df9fdbc67 from 0 to 1 12m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-e433fd Scaled up replica set payload-processing-e2e-worker-single-main-e433fd-7c6897ff54 from 0 to 1 12m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-e433fd-7c6897ff54 Error creating: pods "payload-processing-e2e-worker-single-main-e433fd-7c6897ff54-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-e433fd: serviceaccount "payload-processing-e2e-worker-single-main-e433fd" not found 12m Normal Created pod/payload-processing-e2e-worker-single-main-e433fd-7c6897ff52rsr5 Container created 12m Normal Started pod/payload-processing-e2e-worker-single-main-e433fd-7c6897ff52rsr5 Container started 12m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-e433fd-5df9f8pxkd Add eth0 [10.133.1.41/23] from ovn-kubernetes 12m Normal Pulled pod/payload-processing-e2e-worker-single-main-e433fd-7c6897ff52rsr5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 12m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-e433fd-7c6897ff54 Created pod: payload-processing-e2e-worker-single-main-e433fd-7c6897ff52rsr5 12m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-e433fd-7c6897ff52rsr5 Add eth0 [10.133.1.40/23] from ovn-kubernetes 12m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-e433fd-5df9fdbc67 Created pod: payload-pre-processing-e2e-worker-single-main-e433fd-5df9f8pxkd 12m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-e433fd-5df9f8pxkd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 12m Normal Created pod/payload-pre-processing-e2e-worker-single-main-e433fd-5df9f8pxkd Container created 12m Normal Started pod/payload-pre-processing-e2e-worker-single-main-e433fd-5df9f8pxkd Container started 12m Normal Pulled pod/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 12m Normal Created pod/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l Container created 12m Normal Started pod/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l Container started 12m Warning Unhealthy pod/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l Startup probe failed: Get "http://10.134.0.115:15021/healthz/ready": dial tcp 10.134.0.115:15021: connect: connection refused 12m Warning ProbeError pod/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l Startup probe error: Get "http://10.134.0.115:15021/healthz/ready": dial tcp 10.134.0.115:15021: connect: connection refused... 11m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-e433fd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 11m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-e433fd-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 11m Normal Killing pod/payload-processing-e2e-worker-single-main-e433fd-7c6897ff52rsr5 Stopping container payload-processing 11m Normal Killing pod/payload-pre-processing-e2e-worker-single-main-e433fd-5df9f8pxkd Stopping container payload-pre-processing 11m Normal Killing pod/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l Stopping container istio-proxy 11m Warning ProbeError pod/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l Readiness probe error: HTTP probe failed with statuscode: 503... 11m Warning Unhealthy pod/e2e-worker-single-main-e433fd-openshift-default-d8b5bd5b5-h526l Readiness probe failed: HTTP probe failed with statuscode: 503 8m9s Normal ScalingReplicaSet deployment/e2e-ait-cc1c5661-openshift-default Scaled up replica set e2e-ait-cc1c5661-openshift-default-65b64fd645 from 0 to 1 8m9s Normal AddedLabel gateway/e2e-ait-cc1c5661 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-cc1c5661 8m9s Normal SuccessfulCreate replicaset/e2e-ait-cc1c5661-openshift-default-65b64fd645 Created pod: e2e-ait-cc1c5661-openshift-default-65b64fd645-v2pmz 8m9s Normal AddedInterface pod/e2e-ait-cc1c5661-openshift-default-65b64fd645-v2pmz Add eth0 [10.133.1.47/23] from ovn-kubernetes 8m2s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-cc1c5661 Scaled up replica set payload-pre-processing-e2e-ait-cc1c5661-585757c4f9 from 0 to 1 8m2s Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-cc1c5661 Scaled up replica set payload-processing-e2e-ait-cc1c5661-86bddd7ff4 from 0 to 1 8m2s Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-cc1c5661-585757c4f9 Error creating: pods "payload-pre-processing-e2e-ait-cc1c5661-585757c4f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-cc1c5661: serviceaccount "payload-processing-e2e-ait-cc1c5661" not found 8m2s Warning FailedCreate replicaset/payload-processing-e2e-ait-cc1c5661-86bddd7ff4 Error creating: pods "payload-processing-e2e-ait-cc1c5661-86bddd7ff4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-cc1c5661: serviceaccount "payload-processing-e2e-ait-cc1c5661" not found 8m1s Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-cc1c5661-86bddd7ff4 Created pod: payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv 8m1s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-cc1c5661-585757c4f9 Created pod: payload-pre-processing-e2e-ait-cc1c5661-585757c4f9-hqp2b 8m1s Normal AddedInterface pod/payload-pre-processing-e2e-ait-cc1c5661-585757c4f9-hqp2b Add eth0 [10.133.1.49/23] from ovn-kubernetes 8m1s Normal Started pod/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv Container started 8m1s Normal Created pod/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv Container created 8m1s Normal AddedInterface pod/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv Add eth0 [10.134.0.120/23] from ovn-kubernetes 8m1s Normal Pulled pod/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m Normal Created pod/payload-pre-processing-e2e-ait-cc1c5661-585757c4f9-hqp2b Container created 8m Normal Started pod/payload-pre-processing-e2e-ait-cc1c5661-585757c4f9-hqp2b Container started 8m Normal Pulled pod/payload-pre-processing-e2e-ait-cc1c5661-585757c4f9-hqp2b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 7m54s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-cc1c5661-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 7m54s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-cc1c5661-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 7m42s Normal Pulled pod/e2e-ait-cc1c5661-openshift-default-65b64fd645-v2pmz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 7m42s Normal Created pod/e2e-ait-cc1c5661-openshift-default-65b64fd645-v2pmz Container created 7m42s Normal Started pod/e2e-ait-cc1c5661-openshift-default-65b64fd645-v2pmz Container started 7m37s Warning Unhealthy pod/e2e-ait-cc1c5661-openshift-default-65b64fd645-v2pmz Startup probe failed: Get "http://10.133.1.47:15021/healthz/ready": dial tcp 10.133.1.47:15021: connect: connection refused 7m37s Warning ProbeError pod/e2e-ait-cc1c5661-openshift-default-65b64fd645-v2pmz Startup probe error: Get "http://10.133.1.47:15021/healthz/ready": dial tcp 10.133.1.47:15021: connect: connection refused... 7m17s Normal Killing pod/payload-pre-processing-e2e-ait-cc1c5661-585757c4f9-hqp2b Stopping container payload-pre-processing 7m17s Normal Killing pod/payload-processing-e2e-ait-cc1c5661-86bddd7ff4-24chv Stopping container payload-processing 7m9s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-cc1c5661-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 7m9s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-cc1c5661-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 7m8s Normal Killing pod/e2e-ait-cc1c5661-openshift-default-65b64fd645-v2pmz Stopping container istio-proxy 7m7s Normal AddedLabel gateway/e2e-authcfg-http-05b91b Added label istio.io/rev=openshift-gateway to gateway e2e-authcfg-http-05b91b 7m7s Normal ScalingReplicaSet deployment/e2e-authcfg-http-05b91b-openshift-default Scaled up replica set e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9 from 0 to 1 7m7s Normal SuccessfulCreate replicaset/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9 Created pod: e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp 7m6s Warning Unhealthy pod/e2e-ait-cc1c5661-openshift-default-65b64fd645-v2pmz Readiness probe failed: HTTP probe failed with statuscode: 503 7m6s Normal AddedInterface pod/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp Add eth0 [10.134.0.122/23] from ovn-kubernetes 7m6s Warning ProbeError pod/e2e-ait-cc1c5661-openshift-default-65b64fd645-v2pmz Readiness probe error: HTTP probe failed with statuscode: 503... 6m59s Warning FailedCreate replicaset/payload-pre-processing-e2e-authcfg-http-05b91b-559cc4c5bc Error creating: pods "payload-pre-processing-e2e-authcfg-http-05b91b-559cc4c5bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-05b91b: serviceaccount "payload-processing-e2e-authcfg-http-05b91b" not found 6m59s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-authcfg-http-05b91b Scaled up replica set payload-pre-processing-e2e-authcfg-http-05b91b-559cc4c5bc from 0 to 1 6m59s Normal ScalingReplicaSet deployment/payload-processing-e2e-authcfg-http-05b91b Scaled up replica set payload-processing-e2e-authcfg-http-05b91b-66bb9db55d from 0 to 1 6m59s Warning FailedCreate replicaset/payload-processing-e2e-authcfg-http-05b91b-66bb9db55d Error creating: pods "payload-processing-e2e-authcfg-http-05b91b-66bb9db55d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-05b91b: serviceaccount "payload-processing-e2e-authcfg-http-05b91b" not found 6m58s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-authcfg-http-05b91b-559cc4c5bc Created pod: payload-pre-processing-e2e-authcfg-http-05b91b-559cc4c5bc-pmj9m 6m58s Normal SuccessfulCreate replicaset/payload-processing-e2e-authcfg-http-05b91b-66bb9db55d Created pod: payload-processing-e2e-authcfg-http-05b91b-66bb9db55d-gcw9p 6m58s Normal Started pod/payload-pre-processing-e2e-authcfg-http-05b91b-559cc4c5bc-pmj9m Container started 6m58s Normal AddedInterface pod/payload-pre-processing-e2e-authcfg-http-05b91b-559cc4c5bc-pmj9m Add eth0 [10.133.1.51/23] from ovn-kubernetes 6m58s Normal Pulled pod/payload-pre-processing-e2e-authcfg-http-05b91b-559cc4c5bc-pmj9m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 6m58s Normal AddedInterface pod/payload-processing-e2e-authcfg-http-05b91b-66bb9db55d-gcw9p Add eth0 [10.133.1.52/23] from ovn-kubernetes 6m58s Normal Pulled pod/payload-processing-e2e-authcfg-http-05b91b-66bb9db55d-gcw9p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 6m58s Normal Created pod/payload-processing-e2e-authcfg-http-05b91b-66bb9db55d-gcw9p Container created 6m58s Normal Started pod/payload-processing-e2e-authcfg-http-05b91b-66bb9db55d-gcw9p Container started 6m58s Normal Created pod/payload-pre-processing-e2e-authcfg-http-05b91b-559cc4c5bc-pmj9m Container created 6m38s Normal Started pod/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp Container started 6m38s Normal Created pod/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp Container created 6m38s Normal Pulled pod/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 6m32s Warning Unhealthy pod/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp Startup probe failed: Get "http://10.134.0.122:15021/healthz/ready": dial tcp 10.134.0.122:15021: connect: connection refused 6m32s Warning ProbeError pod/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp Startup probe error: Get "http://10.134.0.122:15021/healthz/ready": dial tcp 10.134.0.122:15021: connect: connection refused... 5m51s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-05b91b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 5m51s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-05b91b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m52s Normal Killing pod/payload-processing-e2e-authcfg-http-05b91b-66bb9db55d-gcw9p Stopping container payload-processing 4m52s Normal Killing pod/payload-pre-processing-e2e-authcfg-http-05b91b-559cc4c5bc-pmj9m Stopping container payload-pre-processing 4m37s Normal Killing pod/e2e-authcfg-http-05b91b-openshift-default-5b64f7f7b9-z77sp Stopping container istio-proxy 3m17s Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-qrmft Readiness probe error: Get "http://10.133.1.18:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 3m17s Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-79h22 Readiness probe failed: Get "http://10.133.1.17:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 3m17s Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-qrmft Readiness probe failed: Get "http://10.133.1.18:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 3m17s Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-chpk6 Readiness probe error: Get "http://10.134.0.111:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 3m17s Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-chpk6 Readiness probe failed: Get "http://10.134.0.111:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 3m17s Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-79h22 Readiness probe error: Get "http://10.133.1.17:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 3m17s Warning Unhealthy pod/maas-default-gateway-openshift-default-59f957fdd7-66p28 Readiness probe failed: Get "http://10.134.0.112:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 3m17s Warning ProbeError pod/maas-default-gateway-openshift-default-59f957fdd7-66p28 Readiness probe error: Get "http://10.134.0.112:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)...