=== events in openshift-ingress (2026-11-02T22:32:14Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 52m Normal Scheduled pod/e2e-shared-a-w3-06366c-openshift-default-6cd8c77f65-8swpg Successfully assigned openshift-ingress/e2e-shared-a-w3-06366c-openshift-default-6cd8c77f65-8swpg to ip-10-0-137-225.us-west-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6 Successfully assigned openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6 to ip-10-0-130-83.us-west-2.compute.internal 61m Normal Scheduled pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x Successfully assigned openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x to ip-10-0-130-83.us-west-2.compute.internal 61m Normal Scheduled pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tzgp6 Successfully assigned openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tzgp6 to ip-10-0-137-225.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-qrspz Successfully assigned openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-qrspz to ip-10-0-133-33.us-west-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-lwg4n Successfully assigned openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-lwg4n to ip-10-0-133-33.us-west-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd Successfully assigned openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd to ip-10-0-130-83.us-west-2.compute.internal 63m Normal Scheduled pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-9bjf7 Successfully assigned openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-9bjf7 to ip-10-0-133-33.us-west-2.compute.internal 61m Normal Scheduled pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5khvm Successfully assigned openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5khvm to ip-10-0-133-33.us-west-2.compute.internal 6m16s Normal Scheduled pod/e2e-ait-0abfb3fc-openshift-default-6b6598d5c9-wjdtm Successfully assigned openshift-ingress/e2e-ait-0abfb3fc-openshift-default-6b6598d5c9-wjdtm to ip-10-0-137-225.us-west-2.compute.internal 32m Normal Scheduled pod/e2e-inf-a-c4a6b9-openshift-default-65db8fb597-2lkb7 Successfully assigned openshift-ingress/e2e-inf-a-c4a6b9-openshift-default-65db8fb597-2lkb7 to ip-10-0-137-225.us-west-2.compute.internal 54m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-zw295 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-zw295 to ip-10-0-137-225.us-west-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-zt7w9 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-zt7w9 to ip-10-0-137-225.us-west-2.compute.internal 19m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-zndhx Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-zndhx to ip-10-0-133-33.us-west-2.compute.internal 55m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw to ip-10-0-130-83.us-west-2.compute.internal 74m Normal Scheduled pod/openshift-ai-inference-openshift-ai-inference-6bd8768855-j8dth Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-6bd8768855-j8dth to ip-10-0-137-225.us-west-2.compute.internal 74m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-wkbnw Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-wkbnw to ip-10-0-137-225.us-west-2.compute.internal 29m Normal Scheduled pod/e2e-inf-b-49c06d-openshift-default-795bb8bd77-pgndv Successfully assigned openshift-ingress/e2e-inf-b-49c06d-openshift-default-795bb8bd77-pgndv to ip-10-0-137-225.us-west-2.compute.internal 69m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-tcck7 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-tcck7 to ip-10-0-130-83.us-west-2.compute.internal 55m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-sz8lg Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-sz8lg to ip-10-0-133-33.us-west-2.compute.internal 39m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-s5kwr Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s5kwr to ip-10-0-137-225.us-west-2.compute.internal 55m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 to ip-10-0-130-83.us-west-2.compute.internal 39m Normal Scheduled pod/e2e-inf-b-70d237-openshift-default-86895f855-4l98n Successfully assigned openshift-ingress/e2e-inf-b-70d237-openshift-default-86895f855-4l98n to ip-10-0-137-225.us-west-2.compute.internal 79m Normal Scheduled pod/router-default-79fb5bb755-mzvm9 Successfully assigned openshift-ingress/router-default-79fb5bb755-mzvm9 to ip-10-0-130-83.us-west-2.compute.internal 79m Warning FailedScheduling pod/router-default-79fb5bb755-mzvm9 no nodes available to schedule pods 86m Warning FailedScheduling pod/router-default-79fb5bb755-mzvm9 no nodes available to schedule pods 58m Normal Scheduled pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5kdtc Successfully assigned openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5kdtc to ip-10-0-137-225.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp Successfully assigned openshift-ingress/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp to ip-10-0-130-83.us-west-2.compute.internal 62m Normal Scheduled pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-48wgm Successfully assigned openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-48wgm to ip-10-0-133-33.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9 Successfully assigned openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9 to ip-10-0-130-83.us-west-2.compute.internal 67m Normal Scheduled pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-255tp Successfully assigned openshift-ingress/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-255tp to ip-10-0-137-225.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-5rd2m Successfully assigned openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-5rd2m to ip-10-0-133-33.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-79tql Successfully assigned openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-79tql to ip-10-0-137-225.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-7hdld Successfully assigned openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-7hdld to ip-10-0-137-225.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f Successfully assigned openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f to ip-10-0-130-83.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-dxqcj Successfully assigned openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-dxqcj to ip-10-0-133-33.us-west-2.compute.internal 67m Normal Scheduled pod/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t to ip-10-0-130-83.us-west-2.compute.internal 67m Normal Scheduled pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-lmksp Successfully assigned openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-lmksp to ip-10-0-137-225.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x Successfully assigned openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x to ip-10-0-130-83.us-west-2.compute.internal 14m Normal Scheduled pod/e2e-worker-single-main-d16d3a-openshift-default-784968dbdft7lb9 Successfully assigned openshift-ingress/e2e-worker-single-main-d16d3a-openshift-default-784968dbdft7lb9 to ip-10-0-137-225.us-west-2.compute.internal 60m Normal Scheduled pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf Successfully assigned openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf to ip-10-0-130-83.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb Successfully assigned openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb to ip-10-0-130-83.us-west-2.compute.internal 10m Normal Scheduled pod/e2e-worker-single-main-c19d19-openshift-default-56589d87bf2d6q9 Successfully assigned openshift-ingress/e2e-worker-single-main-c19d19-openshift-default-56589d87bf2d6q9 to ip-10-0-137-225.us-west-2.compute.internal 60m Normal Scheduled pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d Successfully assigned openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d to ip-10-0-130-83.us-west-2.compute.internal 67m Normal Scheduled pod/payload-processing-e2e-worker-w5-ef6b43-74fb5cfdbf-n4kdw Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-ef6b43-74fb5cfdbf-n4kdw to ip-10-0-137-225.us-west-2.compute.internal 67m Normal Scheduled pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-bbrxc Successfully assigned openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-bbrxc to ip-10-0-137-225.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh Successfully assigned openshift-ingress/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh to ip-10-0-130-83.us-west-2.compute.internal 61m Normal Scheduled pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-jltps Successfully assigned openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-jltps to ip-10-0-133-33.us-west-2.compute.internal 45m Normal Scheduled pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k Successfully assigned openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k to ip-10-0-130-83.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf Successfully assigned openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf to ip-10-0-130-83.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j Successfully assigned openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j to ip-10-0-130-83.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-tglx5 Successfully assigned openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-tglx5 to ip-10-0-133-33.us-west-2.compute.internal 60m Normal Scheduled pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-tkbbb Successfully assigned openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-tkbbb to ip-10-0-133-33.us-west-2.compute.internal 67m Normal Scheduled pod/payload-processing-e2e-worker-w4-4e2b27-5569b97fc9-q286k Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-4e2b27-5569b97fc9-q286k to ip-10-0-137-225.us-west-2.compute.internal 45m Normal Scheduled pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl Successfully assigned openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl to ip-10-0-130-83.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-wjhwc Successfully assigned openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-wjhwc to ip-10-0-137-225.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-wv79b Successfully assigned openshift-ingress/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-wv79b to ip-10-0-137-225.us-west-2.compute.internal 46m Normal Scheduled pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-vhbmj Successfully assigned openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-vhbmj to ip-10-0-137-225.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc Successfully assigned openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc to ip-10-0-130-83.us-west-2.compute.internal 62m Normal Scheduled pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-2ssdg Successfully assigned openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-2ssdg to ip-10-0-137-225.us-west-2.compute.internal 45m Normal Scheduled pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-nwvxb Successfully assigned openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-nwvxb to ip-10-0-133-33.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh Successfully assigned openshift-ingress/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh to ip-10-0-130-83.us-west-2.compute.internal 67m Normal Scheduled pod/payload-processing-e2e-worker-w3-9a6cf2-5f5bb8bd95-48467 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-9a6cf2-5f5bb8bd95-48467 to ip-10-0-133-33.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-7twrh Successfully assigned openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-7twrh to ip-10-0-133-33.us-west-2.compute.internal 45m Normal Scheduled pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-khkfr Successfully assigned openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-khkfr to ip-10-0-133-33.us-west-2.compute.internal 61m Normal Scheduled pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp Successfully assigned openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp to ip-10-0-130-83.us-west-2.compute.internal 46m Normal Scheduled pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-jq4p8 Successfully assigned openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-jq4p8 to ip-10-0-137-225.us-west-2.compute.internal 40m Normal Scheduled pod/e2e-inf-b-11537e-openshift-default-7fc77994d5-9hpxw Successfully assigned openshift-ingress/e2e-inf-b-11537e-openshift-default-7fc77994d5-9hpxw to ip-10-0-137-225.us-west-2.compute.internal 46m Normal Scheduled pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-cflq9 Successfully assigned openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-cflq9 to ip-10-0-133-33.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp Successfully assigned openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp to ip-10-0-130-83.us-west-2.compute.internal 67m Normal Scheduled pod/payload-processing-e2e-worker-w2-4d9ea0-75b8748dc7-bqkgq Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-4d9ea0-75b8748dc7-bqkgq to ip-10-0-137-225.us-west-2.compute.internal 45m Normal Scheduled pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-c2rhx Successfully assigned openshift-ingress/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-c2rhx to ip-10-0-137-225.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-m464d Successfully assigned openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-m464d to ip-10-0-137-225.us-west-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-rhzmq Successfully assigned openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-rhzmq to ip-10-0-133-33.us-west-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-sn5cj Successfully assigned openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-sn5cj to ip-10-0-137-225.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7 Successfully assigned openshift-ingress/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7 to ip-10-0-130-83.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-tvcl2 Successfully assigned openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-tvcl2 to ip-10-0-137-225.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc Successfully assigned openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc to ip-10-0-130-83.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-api-a-b7107c-openshift-default-778f76c545-8hgqg Successfully assigned openshift-ingress/e2e-api-a-b7107c-openshift-default-778f76c545-8hgqg to ip-10-0-137-225.us-west-2.compute.internal 67m Normal Scheduled pod/payload-processing-e2e-worker-w1-c2dd3b-65f5c79bd5-z22x2 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-c2dd3b-65f5c79bd5-z22x2 to ip-10-0-137-225.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg Successfully assigned openshift-ingress/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg to ip-10-0-130-83.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-shared-b-w1-66c72e-openshift-default-759cb84997-zklx9 Successfully assigned openshift-ingress/e2e-shared-b-w1-66c72e-openshift-default-759cb84997-zklx9 to ip-10-0-137-225.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-25ttn Successfully assigned openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-25ttn to ip-10-0-133-33.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-285rr Successfully assigned openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-285rr to ip-10-0-133-33.us-west-2.compute.internal 67m Normal Scheduled pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-4ttqx Successfully assigned openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-4ttqx to ip-10-0-133-33.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-7ntr6 Successfully assigned openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-7ntr6 to ip-10-0-137-225.us-west-2.compute.internal 60m Normal Scheduled pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns Successfully assigned openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns to ip-10-0-130-83.us-west-2.compute.internal 67m Normal Scheduled pod/payload-processing-e2e-worker-w0-dd6574-79864967d7-f8wcm Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-dd6574-79864967d7-f8wcm to ip-10-0-133-33.us-west-2.compute.internal 14m Normal Scheduled pod/payload-processing-e2e-worker-single-main-d16d3a-69cd7f47bb94qp Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-d16d3a-69cd7f47bb94qp to ip-10-0-137-225.us-west-2.compute.internal 10m Normal Scheduled pod/payload-processing-e2e-worker-single-main-c19d19-5bb7bc6d7ptddr Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-c19d19-5bb7bc6d7ptddr to ip-10-0-137-225.us-west-2.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-shared-b-w6-5451a5-566f6b7dbf-gmzc5 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w6-5451a5-566f6b7dbf-gmzc5 to ip-10-0-137-225.us-west-2.compute.internal 49m Normal Scheduled pod/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt to ip-10-0-130-83.us-west-2.compute.internal 50m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq to ip-10-0-130-83.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk Successfully assigned openshift-ingress/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk to ip-10-0-130-83.us-west-2.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-shared-b-w1-66c72e-66896d7d67-k7dq8 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w1-66c72e-66896d7d67-k7dq8 to ip-10-0-137-225.us-west-2.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn to ip-10-0-130-83.us-west-2.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh to ip-10-0-130-83.us-west-2.compute.internal 52m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-f4a242-66587fdf75-qghj6 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-f4a242-66587fdf75-qghj6 to ip-10-0-137-225.us-west-2.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t to ip-10-0-130-83.us-west-2.compute.internal 56m Normal Scheduled pod/payload-processing-e2e-mt-c7cd2d37-5cb657cf9c-cbzrr Successfully assigned openshift-ingress/payload-processing-e2e-mt-c7cd2d37-5cb657cf9c-cbzrr to ip-10-0-137-225.us-west-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-mt-6b983e02-66b998455c-t7x6x Successfully assigned openshift-ingress/payload-processing-e2e-mt-6b983e02-66b998455c-t7x6x to ip-10-0-137-225.us-west-2.compute.internal 52m Normal Scheduled pod/payload-processing-e2e-mt-6b983e02-66b998455c-swpb9 Successfully assigned openshift-ingress/payload-processing-e2e-mt-6b983e02-66b998455c-swpb9 to ip-10-0-133-33.us-west-2.compute.internal 56m Normal Scheduled pod/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg Successfully assigned openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg to ip-10-0-130-83.us-west-2.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-inf-b-d4f799-56b5f49845-8vdrg Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-d4f799-56b5f49845-8vdrg to ip-10-0-137-225.us-west-2.compute.internal 38m Normal Scheduled pod/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n to ip-10-0-130-83.us-west-2.compute.internal 39m Normal Scheduled pod/payload-processing-e2e-inf-b-70d237-c848cd89-hph72 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-70d237-c848cd89-hph72 to ip-10-0-137-225.us-west-2.compute.internal 29m Normal Scheduled pod/payload-processing-e2e-inf-b-49c06d-85d8b67ff5-lks6v Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-49c06d-85d8b67ff5-lks6v to ip-10-0-137-225.us-west-2.compute.internal 68m Normal Scheduled pod/payload-processing-55fbbbf9dc-jmk5x Successfully assigned openshift-ingress/payload-processing-55fbbbf9dc-jmk5x to ip-10-0-133-33.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76 Successfully assigned openshift-ingress/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76 to ip-10-0-130-83.us-west-2.compute.internal 32m Normal Scheduled pod/payload-processing-e2e-inf-a-c4a6b9-68dd84bb55-8gv8t Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-c4a6b9-68dd84bb55-8gv8t to ip-10-0-137-225.us-west-2.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv to ip-10-0-130-83.us-west-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57 to ip-10-0-130-83.us-west-2.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-inf-a-686bbf-7f776f59c7-ssgkm Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-686bbf-7f776f59c7-ssgkm to ip-10-0-133-33.us-west-2.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-inf-a-3566f9-6d8fc84c79-l9lpf Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-3566f9-6d8fc84c79-l9lpf to ip-10-0-137-225.us-west-2.compute.internal 56m Normal Scheduled pod/payload-processing-e2e-derive-4dff0429-cc8778797-59x9m Successfully assigned openshift-ingress/payload-processing-e2e-derive-4dff0429-cc8778797-59x9m to ip-10-0-137-225.us-west-2.compute.internal 5m6s Normal Scheduled pod/payload-processing-e2e-authcfg-http-d9f155-57d66cf5bc-bh2tj Successfully assigned openshift-ingress/payload-processing-e2e-authcfg-http-d9f155-57d66cf5bc-bh2tj to ip-10-0-137-225.us-west-2.compute.internal 49m Normal Scheduled pod/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk Successfully assigned openshift-ingress/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk to ip-10-0-130-83.us-west-2.compute.internal 40m Normal Scheduled pod/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67 Successfully assigned openshift-ingress/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67 to ip-10-0-130-83.us-west-2.compute.internal 49m Normal Scheduled pod/payload-processing-e2e-api-b-32de96-5ff6bbb5ff-rqwzc Successfully assigned openshift-ingress/payload-processing-e2e-api-b-32de96-5ff6bbb5ff-rqwzc to ip-10-0-133-33.us-west-2.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd Successfully assigned openshift-ingress/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd to ip-10-0-130-83.us-west-2.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-api-a-d0946d-59f4d669d9-k4wgp Successfully assigned openshift-ingress/payload-processing-e2e-api-a-d0946d-59f4d669d9-k4wgp to ip-10-0-133-33.us-west-2.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb Successfully assigned openshift-ingress/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb to ip-10-0-130-83.us-west-2.compute.internal 53m Normal Scheduled pod/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846 Successfully assigned openshift-ingress/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846 to ip-10-0-130-83.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf Successfully assigned openshift-ingress/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf to ip-10-0-130-83.us-west-2.compute.internal 56m Normal Scheduled pod/payload-processing-e2e-ait-c39c4123-78b4dd8875-hdg55 Successfully assigned openshift-ingress/payload-processing-e2e-ait-c39c4123-78b4dd8875-hdg55 to ip-10-0-137-225.us-west-2.compute.internal 54m Normal Scheduled pod/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg Successfully assigned openshift-ingress/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg to ip-10-0-130-83.us-west-2.compute.internal 6m9s Normal Scheduled pod/payload-processing-e2e-ait-0abfb3fc-d894d79c-v5882 Successfully assigned openshift-ingress/payload-processing-e2e-ait-0abfb3fc-d894d79c-v5882 to ip-10-0-137-225.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-shared-a-w6-9f77b7-openshift-default-6c9975d87-xngcd Successfully assigned openshift-ingress/e2e-shared-a-w6-9f77b7-openshift-default-6c9975d87-xngcd to ip-10-0-133-33.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-bmbxc Successfully assigned openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-bmbxc to ip-10-0-133-33.us-west-2.compute.internal 61m Normal Scheduled pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cf9jh Successfully assigned openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cf9jh to ip-10-0-137-225.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq Successfully assigned openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq to ip-10-0-130-83.us-west-2.compute.internal 60m Normal Scheduled pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-fsc7d Successfully assigned openshift-ingress/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-fsc7d to ip-10-0-137-225.us-west-2.compute.internal 67m Normal Scheduled pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gnxj5 Successfully assigned openshift-ingress/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gnxj5 to ip-10-0-133-33.us-west-2.compute.internal 73m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-68dd7b898fs5nlb Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-68dd7b898fs5nlb to ip-10-0-137-225.us-west-2.compute.internal 39m Normal Scheduled pod/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd to ip-10-0-130-83.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v Successfully assigned openshift-ingress/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v to ip-10-0-130-83.us-west-2.compute.internal 67m Normal Scheduled pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-cl9rj Successfully assigned openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-cl9rj to ip-10-0-137-225.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-gvmnx Successfully assigned openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-gvmnx to ip-10-0-133-33.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-jqdtb Successfully assigned openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-jqdtb to ip-10-0-133-33.us-west-2.compute.internal 60m Normal Scheduled pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht Successfully assigned openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht to ip-10-0-130-83.us-west-2.compute.internal 40m Normal Scheduled pod/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4 Successfully assigned openshift-ingress/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4 to ip-10-0-130-83.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz Successfully assigned openshift-ingress/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz to ip-10-0-130-83.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-nmbnd Successfully assigned openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-nmbnd to ip-10-0-137-225.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk Successfully assigned openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk to ip-10-0-130-83.us-west-2.compute.internal 67m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-284b8e-548df84bc-wznlk Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-284b8e-548df84bc-wznlk to ip-10-0-137-225.us-west-2.compute.internal 63m Normal Scheduled pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-zpmjw Successfully assigned openshift-ingress/e2e-worker-w5-ef6b43-openshift-default-679658c459-zpmjw to ip-10-0-133-33.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-mt-c7cd2d37-openshift-default-6458d58469-rkntm Successfully assigned openshift-ingress/e2e-mt-c7cd2d37-openshift-default-6458d58469-rkntm to ip-10-0-137-225.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-252cj Successfully assigned openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-252cj to ip-10-0-133-33.us-west-2.compute.internal 67m Normal Scheduled pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-64wq6 Successfully assigned openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-64wq6 to ip-10-0-133-33.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd Successfully assigned openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd to ip-10-0-130-83.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-djjtz Successfully assigned openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-djjtz to ip-10-0-137-225.us-west-2.compute.internal 62m Normal Scheduled pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-dz5hw Successfully assigned openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-dz5hw to ip-10-0-137-225.us-west-2.compute.internal 67m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-ef6b43-fb6db9dc5-8kf8j Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-ef6b43-fb6db9dc5-8kf8j to ip-10-0-133-33.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-mt-6b983e02-openshift-default-f8946f76f-p7dbl Successfully assigned openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-p7dbl to ip-10-0-137-225.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-gh4d7 Successfully assigned openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-gh4d7 to ip-10-0-133-33.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz Successfully assigned openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz to ip-10-0-130-83.us-west-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq Successfully assigned openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq to ip-10-0-130-83.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq Successfully assigned openshift-ingress/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq to ip-10-0-130-83.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-mt-6b983e02-openshift-default-f8946f76f-hgtfw Successfully assigned openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-hgtfw to ip-10-0-137-225.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-qptp7 Successfully assigned openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-qptp7 to ip-10-0-137-225.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-mt-6b983e02-openshift-default-f8946f76f-gdqtv Successfully assigned openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-gdqtv to ip-10-0-137-225.us-west-2.compute.internal 67m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-4e2b27-987d59d59-4j6sw Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-4e2b27-987d59d59-4j6sw to ip-10-0-137-225.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-qtwpp Successfully assigned openshift-ingress/e2e-worker-w6-284b8e-openshift-default-77c5d9855-qtwpp to ip-10-0-133-33.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-mt-6b983e02-openshift-default-f8946f76f-d9dcg Successfully assigned openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-d9dcg to ip-10-0-133-33.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s Successfully assigned openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s to ip-10-0-130-83.us-west-2.compute.internal 74m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-466l7 Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-466l7 to ip-10-0-130-83.us-west-2.compute.internal 69m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-rp6l2 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-rp6l2 to ip-10-0-137-225.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9 Successfully assigned openshift-ingress/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9 to ip-10-0-130-83.us-west-2.compute.internal 46m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-pjj8k Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-pjj8k to ip-10-0-133-33.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz Successfully assigned openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz to ip-10-0-130-83.us-west-2.compute.internal 18m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-ndztn Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-ndztn to ip-10-0-133-33.us-west-2.compute.internal 67m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-9a6cf2-7d98fcf679-zzqnq Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-9a6cf2-7d98fcf679-zzqnq to ip-10-0-137-225.us-west-2.compute.internal 72m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-brtmx Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-brtmx to ip-10-0-130-83.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q Successfully assigned openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q to ip-10-0-130-83.us-west-2.compute.internal 56m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-bxq4p Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bxq4p to ip-10-0-133-33.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-mt-6b983e02-openshift-default-f8946f76f-76pr7 Successfully assigned openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-76pr7 to ip-10-0-133-33.us-west-2.compute.internal 56m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-m56v7 Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m56v7 to ip-10-0-137-225.us-west-2.compute.internal 5m14s Normal Scheduled pod/e2e-authcfg-http-d9f155-openshift-default-7c8fbc7f6f-tbrw9 Successfully assigned openshift-ingress/e2e-authcfg-http-d9f155-openshift-default-7c8fbc7f6f-tbrw9 to ip-10-0-137-225.us-west-2.compute.internal 53m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-n2q6l Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n2q6l to ip-10-0-137-225.us-west-2.compute.internal 54m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-n2h8k Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-n2h8k to ip-10-0-133-33.us-west-2.compute.internal 67m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-4d9ea0-7c6896dc8c-d9bwg Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-4d9ea0-7c6896dc8c-d9bwg to ip-10-0-137-225.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-mt-6b983e02-openshift-default-f8946f76f-28fpg Successfully assigned openshift-ingress/e2e-mt-6b983e02-openshift-default-f8946f76f-28fpg to ip-10-0-133-33.us-west-2.compute.internal 73m Normal Scheduled pod/kube-auth-proxy-795b66c7fb-8b5vf Successfully assigned openshift-ingress/kube-auth-proxy-795b66c7fb-8b5vf to ip-10-0-130-83.us-west-2.compute.internal 73m Normal Scheduled pod/kube-auth-proxy-795b66c7fb-fjd9v Successfully assigned openshift-ingress/kube-auth-proxy-795b66c7fb-fjd9v to ip-10-0-133-33.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-migrate-2ccb9fb8-gw-openshift-default-bd688b445-9vzqb Successfully assigned openshift-ingress/e2e-migrate-2ccb9fb8-gw-openshift-default-bd688b445-9vzqb to ip-10-0-133-33.us-west-2.compute.internal 20m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-27kg9 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-27kg9 to ip-10-0-137-225.us-west-2.compute.internal 55m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-2kgm4 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-2kgm4 to ip-10-0-137-225.us-west-2.compute.internal 55m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz to ip-10-0-130-83.us-west-2.compute.internal 67m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-c2dd3b-576654d767-9lckv Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-c2dd3b-576654d767-9lckv to ip-10-0-133-33.us-west-2.compute.internal 39m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-662qj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-662qj to ip-10-0-137-225.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-derive-4dff0429-openshift-default-559487f9c7-22gzh Successfully assigned openshift-ingress/e2e-derive-4dff0429-openshift-default-559487f9c7-22gzh to ip-10-0-137-225.us-west-2.compute.internal 19m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt to ip-10-0-130-83.us-west-2.compute.internal 46m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-6swr2 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6swr2 to ip-10-0-137-225.us-west-2.compute.internal 19m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k to ip-10-0-130-83.us-west-2.compute.internal 55m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-7g2ql Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-7g2ql to ip-10-0-137-225.us-west-2.compute.internal 69m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-7x282 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-7x282 to ip-10-0-133-33.us-west-2.compute.internal 41m Normal Scheduled pod/e2e-inf-b-d4f799-openshift-default-76569fd959-phrdn Successfully assigned openshift-ingress/e2e-inf-b-d4f799-openshift-default-76569fd959-phrdn to ip-10-0-137-225.us-west-2.compute.internal 67m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-dd6574-5fd9c947b7-x7c6s Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-dd6574-5fd9c947b7-x7c6s to ip-10-0-137-225.us-west-2.compute.internal 14m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-d16d3a-7b844t5d5k Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-d16d3a-7b844t5d5k to ip-10-0-137-225.us-west-2.compute.internal 10m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-c19d19-6df95ch8kv Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-c19d19-6df95ch8kv to ip-10-0-137-225.us-west-2.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w6-5451a5-54b9b789c9-xs2rv Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w6-5451a5-54b9b789c9-xs2rv to ip-10-0-137-225.us-west-2.compute.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms to ip-10-0-130-83.us-west-2.compute.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh to ip-10-0-130-83.us-west-2.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w1-66c72e-7cd79f45c5-68fdp Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w1-66c72e-7cd79f45c5-68fdp to ip-10-0-137-225.us-west-2.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf to ip-10-0-130-83.us-west-2.compute.internal 41m Normal Scheduled pod/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db Successfully assigned openshift-ingress/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db to ip-10-0-130-83.us-west-2.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s to ip-10-0-130-83.us-west-2.compute.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-f4a242-6cd88bd755-lkvsn Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f4a242-6cd88bd755-lkvsn to ip-10-0-137-225.us-west-2.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r to ip-10-0-130-83.us-west-2.compute.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8 to ip-10-0-130-83.us-west-2.compute.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-mt-6b983e02-58d76c55bc-jqfd5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-6b983e02-58d76c55bc-jqfd5 to ip-10-0-133-33.us-west-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-mt-6b983e02-58d76c55bc-hvlc4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-6b983e02-58d76c55bc-hvlc4 to ip-10-0-137-225.us-west-2.compute.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-migrate-2ccb9fb8-856588d575-hbzhv Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-2ccb9fb8-856588d575-hbzhv to ip-10-0-137-225.us-west-2.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-d4f799-54b658bd4c-hkmkb Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-d4f799-54b658bd4c-hkmkb to ip-10-0-137-225.us-west-2.compute.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-bcb857-55c67bc8f9-44fjz Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-bcb857-55c67bc8f9-44fjz to ip-10-0-137-225.us-west-2.compute.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-70d237-7ddc758bff-6kzcg Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-70d237-7ddc758bff-6kzcg to ip-10-0-137-225.us-west-2.compute.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-49c06d-67bcf7b5c9-l2spf Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-49c06d-67bcf7b5c9-l2spf to ip-10-0-137-225.us-west-2.compute.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q to ip-10-0-130-83.us-west-2.compute.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-c4a6b9-fcb9b94f9-s642g Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-c4a6b9-fcb9b94f9-s642g to ip-10-0-137-225.us-west-2.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r to ip-10-0-130-83.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-inf-a-686bbf-openshift-default-65b6c69fbc-vwvdd Successfully assigned openshift-ingress/e2e-inf-a-686bbf-openshift-default-65b6c69fbc-vwvdd to ip-10-0-133-33.us-west-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv to ip-10-0-130-83.us-west-2.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-686bbf-56f69586dc-fczlv Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-686bbf-56f69586dc-fczlv to ip-10-0-133-33.us-west-2.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-3566f9-5654cbf979-7v5tp Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-3566f9-5654cbf979-7v5tp to ip-10-0-137-225.us-west-2.compute.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz to ip-10-0-130-83.us-west-2.compute.internal 5m6s Normal Scheduled pod/payload-pre-processing-e2e-authcfg-http-d9f155-595bf59c4c-7vgc7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-authcfg-http-d9f155-595bf59c4c-7vgc7 to ip-10-0-137-225.us-west-2.compute.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-api-b-a1bf7d-6955977c5-wvmp9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-a1bf7d-6955977c5-wvmp9 to ip-10-0-137-225.us-west-2.compute.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-api-b-3d2b29-7d6d7767-724mf Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-3d2b29-7d6d7767-724mf to ip-10-0-137-225.us-west-2.compute.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-api-b-32de96-5dc88f798c-hxh64 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-32de96-5dc88f798c-hxh64 to ip-10-0-133-33.us-west-2.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db to ip-10-0-130-83.us-west-2.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-api-a-d0946d-8465488fff-wnnmg Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-d0946d-8465488fff-wnnmg to ip-10-0-133-33.us-west-2.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv to ip-10-0-130-83.us-west-2.compute.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc to ip-10-0-130-83.us-west-2.compute.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-ait-c39c4123-64c6db69f9-77z5f Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-c39c4123-64c6db69f9-77z5f to ip-10-0-137-225.us-west-2.compute.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg to ip-10-0-130-83.us-west-2.compute.internal 43m Normal Scheduled pod/e2e-inf-a-7cc1b1-openshift-default-84d569b9c-pkbzx Successfully assigned openshift-ingress/e2e-inf-a-7cc1b1-openshift-default-84d569b9c-pkbzx to ip-10-0-137-225.us-west-2.compute.internal 6m9s Normal Scheduled pod/payload-pre-processing-e2e-ait-0abfb3fc-56984c987f-wd88r Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-0abfb3fc-56984c987f-wd88r to ip-10-0-137-225.us-west-2.compute.internal 46m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-cd928 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-cd928 to ip-10-0-137-225.us-west-2.compute.internal 19m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-hft2d Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-hft2d to ip-10-0-137-225.us-west-2.compute.internal 68m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-j9dd9 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-j9dd9 to ip-10-0-133-33.us-west-2.compute.internal 38m Normal Scheduled pod/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2 Successfully assigned openshift-ingress/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2 to ip-10-0-130-83.us-west-2.compute.internal 18m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-js4dr Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-js4dr to ip-10-0-137-225.us-west-2.compute.internal 39m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-kch8g Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-kch8g to ip-10-0-137-225.us-west-2.compute.internal 20m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-kqfwb Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-kqfwb to ip-10-0-133-33.us-west-2.compute.internal 68m Normal Scheduled pod/payload-pre-processing-7b98bb678c-lcj5f Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-lcj5f to ip-10-0-133-33.us-west-2.compute.internal 55m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-lq8nh Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-lq8nh to ip-10-0-133-33.us-west-2.compute.internal 46m Normal Scheduled pod/maas-default-gateway-openshift-default-77bfb58f8c-mwdzc Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-mwdzc to ip-10-0-133-33.us-west-2.compute.internal 86m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-79fb5bb755 from 0 to 1 86m Normal SuccessfulCreate replicaset/router-default-79fb5bb755 Created pod: router-default-79fb5bb755-mzvm9 86m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 86m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 86m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 80m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 79m Warning FailedMount pod/router-default-79fb5bb755-mzvm9 MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 79m Warning FailedMount pod/router-default-79fb5bb755-mzvm9 MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 79m Warning FailedMount pod/router-default-79fb5bb755-mzvm9 MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 79m Warning FailedMount pod/router-default-79fb5bb755-mzvm9 MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 78m Normal AddedInterface pod/router-default-79fb5bb755-mzvm9 Add eth0 [10.132.0.12/23] from ovn-kubernetes 78m Normal Created pod/router-default-79fb5bb755-mzvm9 Container created 78m Normal Started pod/router-default-79fb5bb755-mzvm9 Container started 78m Normal Pulled pod/router-default-79fb5bb755-mzvm9 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 74m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 74m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 74m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-466l7 Add eth0 [10.132.0.27/23] from ovn-kubernetes 74m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-466l7 74m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-466l7 Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 74m Normal Created pod/istiod-openshift-gateway-7876bfcb74-466l7 Container created 74m Normal Started pod/istiod-openshift-gateway-7876bfcb74-466l7 Container started 74m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-466l7 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.305s (4.305s including waiting). Image size: 185476477 bytes. 74m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-466l7 Readiness probe error: HTTP probe failed with statuscode: 503... 74m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-466l7 Readiness probe failed: HTTP probe failed with statuscode: 503 74m Normal EnsuringLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensuring load balancer 74m Normal ScalingReplicaSet deployment/openshift-ai-inference-openshift-ai-inference Scaled up replica set openshift-ai-inference-openshift-ai-inference-6bd8768855 from 0 to 1 74m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-77bfb58f8c from 0 to 1 74m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-77bfb58f8c Created pod: maas-default-gateway-openshift-default-77bfb58f8c-wkbnw 74m Normal SuccessfulCreate replicaset/openshift-ai-inference-openshift-ai-inference-6bd8768855 Created pod: openshift-ai-inference-openshift-ai-inference-6bd8768855-j8dth 74m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-wkbnw Add eth0 [10.133.0.25/23] from ovn-kubernetes 74m Normal Pulling pod/openshift-ai-inference-openshift-ai-inference-6bd8768855-j8dth Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 74m Normal AddedInterface pod/openshift-ai-inference-openshift-ai-inference-6bd8768855-j8dth Add eth0 [10.133.0.26/23] from ovn-kubernetes 74m Normal Pulling pod/maas-default-gateway-openshift-default-77bfb58f8c-wkbnw Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 74m Normal EnsuredLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensured load balancer 74m Normal Created pod/openshift-ai-inference-openshift-ai-inference-6bd8768855-j8dth Container created 74m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-wkbnw Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 13.41s (13.41s including waiting). Image size: 183555333 bytes. 74m Normal Pulled pod/openshift-ai-inference-openshift-ai-inference-6bd8768855-j8dth Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 13.129s (13.129s including waiting). Image size: 183555333 bytes. 74m Normal Started pod/openshift-ai-inference-openshift-ai-inference-6bd8768855-j8dth Container started 74m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 74m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 74m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 74m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 73m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 73m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 73m Normal AddedInterface pod/kube-auth-proxy-795b66c7fb-fjd9v Add eth0 [10.134.0.25/23] from ovn-kubernetes 73m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-68dd7b898f Created pod: data-science-gateway-data-science-gateway-class-68dd7b898fs5nlb 73m Normal AddedInterface pod/kube-auth-proxy-795b66c7fb-8b5vf Add eth0 [10.132.0.29/23] from ovn-kubernetes 73m Normal SuccessfulCreate replicaset/kube-auth-proxy-795b66c7fb Created pod: kube-auth-proxy-795b66c7fb-8b5vf 73m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-68dd7b898f from 0 to 1 73m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-795b66c7fb from 0 to 2 73m Normal SuccessfulCreate replicaset/kube-auth-proxy-795b66c7fb Created pod: kube-auth-proxy-795b66c7fb-fjd9v 73m Normal Pulled pod/data-science-gateway-data-science-gateway-class-68dd7b898fs5nlb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 73m Normal Pulling pod/kube-auth-proxy-795b66c7fb-fjd9v Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 73m Normal Pulling pod/kube-auth-proxy-795b66c7fb-8b5vf Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 73m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-68dd7b898fs5nlb Add eth0 [10.133.0.39/23] from ovn-kubernetes 73m Normal Created pod/data-science-gateway-data-science-gateway-class-68dd7b898fs5nlb Container created 73m Normal Started pod/data-science-gateway-data-science-gateway-class-68dd7b898fs5nlb Container started 73m Normal Started pod/kube-auth-proxy-795b66c7fb-fjd9v Container started 73m Normal Pulled pod/kube-auth-proxy-795b66c7fb-fjd9v Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.859s (2.859s including waiting). Image size: 188904677 bytes. 73m Normal Pulled pod/kube-auth-proxy-795b66c7fb-8b5vf Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.826s (2.826s including waiting). Image size: 188904677 bytes. 73m Normal Created pod/kube-auth-proxy-795b66c7fb-8b5vf Container created 73m Normal Created pod/kube-auth-proxy-795b66c7fb-fjd9v Container created 73m Normal Started pod/kube-auth-proxy-795b66c7fb-8b5vf Container started 72m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 72m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 72m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 72m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 72m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-brtmx 72m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 72m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-brtmx Add eth0 [10.132.0.33/23] from ovn-kubernetes 71m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-brtmx Container started 71m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-brtmx Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 71m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-brtmx Container created 71m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-brtmx Readiness probe failed: HTTP probe failed with statuscode: 503 71m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-brtmx Readiness probe error: HTTP probe failed with statuscode: 503... 71m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-466l7 Readiness probe failed: Get "http://10.132.0.27:8080/ready": dial tcp 10.132.0.27:8080: connect: connection refused 71m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-466l7 Stopping container discovery 71m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 71m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-466l7 71m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-466l7 Readiness probe error: Get "http://10.132.0.27:8080/ready": dial tcp 10.132.0.27:8080: connect: connection refused... 71m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 71m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 71m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 71m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 71m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-wkbnw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 71m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-wkbnw Container started 71m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-wkbnw Container created 71m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-wkbnw Startup probe error: Get "http://10.133.0.25:15021/healthz/ready": dial tcp 10.133.0.25:15021: connect: connection refused... 71m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-wkbnw Startup probe failed: Get "http://10.133.0.25:15021/healthz/ready": dial tcp 10.133.0.25:15021: connect: connection refused 71m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 71m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 69m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-77bfb58f8c Created pod: maas-default-gateway-openshift-default-77bfb58f8c-tcck7 69m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-77bfb58f8c Created pod: maas-default-gateway-openshift-default-77bfb58f8c-rp6l2 69m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-77bfb58f8c Created pod: maas-default-gateway-openshift-default-77bfb58f8c-7x282 69m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-rp6l2 Container started 69m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-rp6l2 Add eth0 [10.133.0.47/23] from ovn-kubernetes 69m Normal Pulling pod/maas-default-gateway-openshift-default-77bfb58f8c-7x282 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 69m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-rp6l2 Container created 69m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-7x282 Add eth0 [10.134.0.41/23] from ovn-kubernetes 69m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-rp6l2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 69m Normal Pulling pod/maas-default-gateway-openshift-default-77bfb58f8c-tcck7 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 69m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-tcck7 Add eth0 [10.132.0.38/23] from ovn-kubernetes 69m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-tcck7 Container started 69m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-tcck7 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.213s (4.214s including waiting). Image size: 183555333 bytes. 69m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-tcck7 Container created 69m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-7x282 Container started 69m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-7x282 Container created 69m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-7x282 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.895s (4.895s including waiting). Image size: 183555333 bytes. 68m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-rp6l2 Startup probe failed: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused 68m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-rp6l2 Startup probe error: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused... 68m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-rp6l2 Startup probe error: Get "http://10.133.0.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 68m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-rp6l2 Startup probe failed: Get "http://10.133.0.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 68m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-7x282 Startup probe error: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused... 68m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-7x282 Startup probe failed: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused 68m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-tcck7 Startup probe failed: Get "http://10.132.0.38:15021/healthz/ready": dial tcp 10.132.0.38:15021: connect: connection refused 68m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-tcck7 Startup probe error: Get "http://10.132.0.38:15021/healthz/ready": dial tcp 10.132.0.38:15021: connect: connection refused... 68m Warning FailedCreate replicaset/payload-processing-55fbbbf9dc Error creating: pods "payload-processing-55fbbbf9dc-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 68m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-55fbbbf9dc from 0 to 1 68m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 68m Normal SuccessfulCreate replicaset/payload-processing-55fbbbf9dc Created pod: payload-processing-55fbbbf9dc-jmk5x 68m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-lcj5f 68m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 68m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-lcj5f Add eth0 [10.134.0.43/23] from ovn-kubernetes 68m Normal Pulling pod/payload-pre-processing-7b98bb678c-lcj5f Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 68m Normal Pulling pod/payload-processing-55fbbbf9dc-jmk5x Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 68m Normal AddedInterface pod/payload-processing-55fbbbf9dc-jmk5x Add eth0 [10.134.0.44/23] from ovn-kubernetes 68m Normal Created pod/payload-pre-processing-7b98bb678c-lcj5f Container created 68m Normal Pulled pod/payload-processing-55fbbbf9dc-jmk5x Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.016s (3.016s including waiting). Image size: 125353963 bytes. 68m Normal Pulled pod/payload-pre-processing-7b98bb678c-lcj5f Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.816s (2.816s including waiting). Image size: 125353963 bytes. 68m Normal Started pod/payload-processing-55fbbbf9dc-jmk5x Container started 68m Normal Started pod/payload-pre-processing-7b98bb678c-lcj5f Container started 68m Normal Created pod/payload-processing-55fbbbf9dc-jmk5x Container created 68m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-j9dd9 Add eth0 [10.134.0.45/23] from ovn-kubernetes 68m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-77bfb58f8c Created pod: maas-default-gateway-openshift-default-77bfb58f8c-j9dd9 68m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 68m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-j9dd9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 68m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-j9dd9 Container started 68m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-j9dd9 Container created 68m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-j9dd9 Startup probe error: Get "http://10.134.0.45:15021/healthz/ready": dial tcp 10.134.0.45:15021: connect: connection refused... 68m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-j9dd9 Startup probe failed: Get "http://10.134.0.45:15021/healthz/ready": dial tcp 10.134.0.45:15021: connect: connection refused 67m Normal AddedLabel gateway/e2e-worker-w1-c2dd3b Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-c2dd3b 67m Normal AddedLabel gateway/e2e-worker-w3-9a6cf2 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-9a6cf2 67m Normal SuccessfulCreate replicaset/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c Created pod: e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gnxj5 67m Normal AddedLabel gateway/e2e-worker-w6-284b8e Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-284b8e 67m Normal AddedLabel gateway/e2e-worker-w4-4e2b27 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-4e2b27 67m Normal ScalingReplicaSet deployment/e2e-worker-w1-c2dd3b-openshift-default Scaled up replica set e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f from 0 to 1 67m Normal SuccessfulCreate replicaset/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f Created pod: e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-lmksp 67m Normal AddedLabel gateway/e2e-worker-w0-dd6574 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-dd6574 67m Normal ScalingReplicaSet deployment/e2e-worker-w3-9a6cf2-openshift-default Scaled up replica set e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c from 0 to 1 67m Normal AddedLabel gateway/e2e-worker-w5-ef6b43 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-ef6b43 67m Normal AddedLabel gateway/e2e-worker-w2-4d9ea0 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-4d9ea0 67m Normal SuccessfulCreate replicaset/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc Created pod: e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-bbrxc 67m Normal SuccessfulCreate replicaset/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf Created pod: e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-4ttqx 67m Normal ScalingReplicaSet deployment/e2e-worker-w4-4e2b27-openshift-default Scaled up replica set e2e-worker-w4-4e2b27-openshift-default-996dc4fdf from 0 to 1 67m Normal AddedInterface pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-bbrxc Add eth0 [10.133.0.51/23] from ovn-kubernetes 67m Normal SuccessfulCreate replicaset/e2e-worker-w5-ef6b43-openshift-default-679658c459 Created pod: e2e-worker-w5-ef6b43-openshift-default-679658c459-cl9rj 67m Normal AddedInterface pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gnxj5 Add eth0 [10.134.0.46/23] from ovn-kubernetes 67m Normal ScalingReplicaSet deployment/e2e-worker-w2-4d9ea0-openshift-default Scaled up replica set e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc from 0 to 1 67m Normal ScalingReplicaSet deployment/e2e-worker-w5-ef6b43-openshift-default Scaled up replica set e2e-worker-w5-ef6b43-openshift-default-679658c459 from 0 to 1 67m Normal AddedInterface pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-lmksp Add eth0 [10.133.0.50/23] from ovn-kubernetes 67m Normal SuccessfulCreate replicaset/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc Created pod: e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-255tp 67m Normal ScalingReplicaSet deployment/e2e-worker-w6-284b8e-openshift-default Scaled up replica set e2e-worker-w6-284b8e-openshift-default-77c5d9855 from 0 to 1 67m Normal ScalingReplicaSet deployment/e2e-worker-w0-dd6574-openshift-default Scaled up replica set e2e-worker-w0-dd6574-openshift-default-5d877f7ccc from 0 to 1 67m Normal AddedInterface pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-255tp Add eth0 [10.133.0.53/23] from ovn-kubernetes 67m Normal AddedInterface pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-cl9rj Add eth0 [10.133.0.52/23] from ovn-kubernetes 67m Normal SuccessfulCreate replicaset/e2e-worker-w6-284b8e-openshift-default-77c5d9855 Created pod: e2e-worker-w6-284b8e-openshift-default-77c5d9855-64wq6 67m Normal AddedInterface pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-4ttqx Add eth0 [10.134.0.47/23] from ovn-kubernetes 67m Normal AddedInterface pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-64wq6 Add eth0 [10.134.0.48/23] from ovn-kubernetes 67m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-9a6cf2-7d98fcf679 Error creating: pods "payload-pre-processing-e2e-worker-w3-9a6cf2-7d98fcf679-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-9a6cf2: serviceaccount "payload-processing-e2e-worker-w3-9a6cf2" not found 67m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-9a6cf2-5f5bb8bd95 Error creating: pods "payload-processing-e2e-worker-w3-9a6cf2-5f5bb8bd95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-9a6cf2: serviceaccount "payload-processing-e2e-worker-w3-9a6cf2" not found 67m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-9a6cf2 Scaled up replica set payload-pre-processing-e2e-worker-w3-9a6cf2-7d98fcf679 from 0 to 1 67m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-9a6cf2 Scaled up replica set payload-processing-e2e-worker-w3-9a6cf2-5f5bb8bd95 from 0 to 1 67m Normal Pulled pod/payload-processing-e2e-worker-w3-9a6cf2-5f5bb8bd95-48467 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Normal AddedInterface pod/payload-processing-e2e-worker-w3-9a6cf2-5f5bb8bd95-48467 Add eth0 [10.134.0.49/23] from ovn-kubernetes 67m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-9a6cf2-5f5bb8bd95 Created pod: payload-processing-e2e-worker-w3-9a6cf2-5f5bb8bd95-48467 67m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-9a6cf2-7d98fcf679 Created pod: payload-pre-processing-e2e-worker-w3-9a6cf2-7d98fcf679-zzqnq 67m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-9a6cf2-7d98fcf679-zzqnq Add eth0 [10.133.0.54/23] from ovn-kubernetes 67m Normal Pulling pod/payload-pre-processing-e2e-worker-w3-9a6cf2-7d98fcf679-zzqnq Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 67m Normal Started pod/payload-processing-e2e-worker-w3-9a6cf2-5f5bb8bd95-48467 Container started 67m Normal Created pod/payload-processing-e2e-worker-w3-9a6cf2-5f5bb8bd95-48467 Container created 67m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-9a6cf2-7d98fcf679-zzqnq Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.816s (2.816s including waiting). Image size: 125353963 bytes. 67m Normal Created pod/payload-pre-processing-e2e-worker-w3-9a6cf2-7d98fcf679-zzqnq Container created 67m Normal Started pod/payload-pre-processing-e2e-worker-w3-9a6cf2-7d98fcf679-zzqnq Container started 67m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-c2dd3b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 67m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-ef6b43 Scaled up replica set payload-processing-e2e-worker-w5-ef6b43-74fb5cfdbf from 0 to 1 67m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-c2dd3b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 67m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-ef6b43-fb6db9dc5 Created pod: payload-pre-processing-e2e-worker-w5-ef6b43-fb6db9dc5-8kf8j 67m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-ef6b43-74fb5cfdbf Created pod: payload-processing-e2e-worker-w5-ef6b43-74fb5cfdbf-n4kdw 67m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-ef6b43-fb6db9dc5 Error creating: pods "payload-pre-processing-e2e-worker-w5-ef6b43-fb6db9dc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-ef6b43: serviceaccount "payload-processing-e2e-worker-w5-ef6b43" not found 67m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-ef6b43-74fb5cfdbf Error creating: pods "payload-processing-e2e-worker-w5-ef6b43-74fb5cfdbf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-ef6b43: serviceaccount "payload-processing-e2e-worker-w5-ef6b43" not found 67m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-ef6b43 Scaled up replica set payload-pre-processing-e2e-worker-w5-ef6b43-fb6db9dc5 from 0 to 1 67m Normal Created pod/payload-processing-e2e-worker-w5-ef6b43-74fb5cfdbf-n4kdw Container created 67m Normal AddedInterface pod/payload-processing-e2e-worker-w5-ef6b43-74fb5cfdbf-n4kdw Add eth0 [10.133.0.55/23] from ovn-kubernetes 67m Normal Started pod/payload-processing-e2e-worker-w5-ef6b43-74fb5cfdbf-n4kdw Container started 67m Normal Pulled pod/payload-processing-e2e-worker-w5-ef6b43-74fb5cfdbf-n4kdw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Normal Started pod/payload-pre-processing-e2e-worker-w5-ef6b43-fb6db9dc5-8kf8j Container started 67m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-ef6b43-fb6db9dc5-8kf8j Add eth0 [10.134.0.50/23] from ovn-kubernetes 67m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-ef6b43-fb6db9dc5-8kf8j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Normal Created pod/payload-pre-processing-e2e-worker-w5-ef6b43-fb6db9dc5-8kf8j Container created 67m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-4d9ea0-7c6896dc8c Error creating: pods "payload-pre-processing-e2e-worker-w2-4d9ea0-7c6896dc8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-4d9ea0: serviceaccount "payload-processing-e2e-worker-w2-4d9ea0" not found 67m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-4d9ea0 Scaled up replica set payload-pre-processing-e2e-worker-w2-4d9ea0-7c6896dc8c from 0 to 1 67m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-4d9ea0 Scaled up replica set payload-processing-e2e-worker-w2-4d9ea0-75b8748dc7 from 0 to 1 67m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-4d9ea0-75b8748dc7 Error creating: pods "payload-processing-e2e-worker-w2-4d9ea0-75b8748dc7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-4d9ea0: serviceaccount "payload-processing-e2e-worker-w2-4d9ea0" not found 67m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-4d9ea0-7c6896dc8c-d9bwg Add eth0 [10.133.0.56/23] from ovn-kubernetes 67m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-4d9ea0-7c6896dc8c Created pod: payload-pre-processing-e2e-worker-w2-4d9ea0-7c6896dc8c-d9bwg 67m Normal Pulled pod/payload-processing-e2e-worker-w2-4d9ea0-75b8748dc7-bqkgq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Normal AddedInterface pod/payload-processing-e2e-worker-w2-4d9ea0-75b8748dc7-bqkgq Add eth0 [10.133.0.57/23] from ovn-kubernetes 67m Normal Started pod/payload-pre-processing-e2e-worker-w2-4d9ea0-7c6896dc8c-d9bwg Container started 67m Normal Created pod/payload-pre-processing-e2e-worker-w2-4d9ea0-7c6896dc8c-d9bwg Container created 67m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-4d9ea0-75b8748dc7 Created pod: payload-processing-e2e-worker-w2-4d9ea0-75b8748dc7-bqkgq 67m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-4d9ea0-7c6896dc8c-d9bwg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Normal Created pod/payload-processing-e2e-worker-w2-4d9ea0-75b8748dc7-bqkgq Container created 67m Normal Started pod/payload-processing-e2e-worker-w2-4d9ea0-75b8748dc7-bqkgq Container started 67m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-c2dd3b Scaled up replica set payload-pre-processing-e2e-worker-w1-c2dd3b-576654d767 from 0 to 1 67m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-c2dd3b Scaled up replica set payload-processing-e2e-worker-w1-c2dd3b-65f5c79bd5 from 0 to 1 67m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-c2dd3b-576654d767 Created pod: payload-pre-processing-e2e-worker-w1-c2dd3b-576654d767-9lckv 67m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-c2dd3b-65f5c79bd5 Created pod: payload-processing-e2e-worker-w1-c2dd3b-65f5c79bd5-z22x2 67m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-c2dd3b-65f5c79bd5 Error creating: pods "payload-processing-e2e-worker-w1-c2dd3b-65f5c79bd5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-c2dd3b: serviceaccount "payload-processing-e2e-worker-w1-c2dd3b" not found 67m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-c2dd3b-576654d767 Error creating: pods "payload-pre-processing-e2e-worker-w1-c2dd3b-576654d767-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-c2dd3b: serviceaccount "payload-processing-e2e-worker-w1-c2dd3b" not found 67m Normal Pulled pod/payload-processing-e2e-worker-w1-c2dd3b-65f5c79bd5-z22x2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Normal Created pod/payload-processing-e2e-worker-w1-c2dd3b-65f5c79bd5-z22x2 Container created 67m Normal AddedInterface pod/payload-processing-e2e-worker-w1-c2dd3b-65f5c79bd5-z22x2 Add eth0 [10.133.0.58/23] from ovn-kubernetes 67m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-c2dd3b-576654d767-9lckv Add eth0 [10.134.0.52/23] from ovn-kubernetes 67m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-c2dd3b-576654d767-9lckv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Normal Created pod/payload-pre-processing-e2e-worker-w1-c2dd3b-576654d767-9lckv Container created 67m Normal Started pod/payload-pre-processing-e2e-worker-w1-c2dd3b-576654d767-9lckv Container started 67m Normal Started pod/payload-processing-e2e-worker-w1-c2dd3b-65f5c79bd5-z22x2 Container started 67m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-284b8e-69d467cf85 Error creating: pods "payload-processing-e2e-worker-w6-284b8e-69d467cf85-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-284b8e: serviceaccount "payload-processing-e2e-worker-w6-284b8e" not found 67m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-284b8e Scaled up replica set payload-processing-e2e-worker-w6-284b8e-69d467cf85 from 0 to 1 67m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-284b8e-548df84bc Error creating: pods "payload-pre-processing-e2e-worker-w6-284b8e-548df84bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-284b8e: serviceaccount "payload-processing-e2e-worker-w6-284b8e" not found 67m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-284b8e Scaled up replica set payload-pre-processing-e2e-worker-w6-284b8e-548df84bc from 0 to 1 67m Normal Pulling pod/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 67m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-284b8e-548df84bc-wznlk Add eth0 [10.133.0.59/23] from ovn-kubernetes 67m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-284b8e-69d467cf85 Created pod: payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t 67m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-284b8e-548df84bc-wznlk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Normal Created pod/payload-pre-processing-e2e-worker-w6-284b8e-548df84bc-wznlk Container created 67m Normal AddedInterface pod/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t Add eth0 [10.132.0.40/23] from ovn-kubernetes 67m Normal Started pod/payload-pre-processing-e2e-worker-w6-284b8e-548df84bc-wznlk Container started 67m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-284b8e-548df84bc Created pod: payload-pre-processing-e2e-worker-w6-284b8e-548df84bc-wznlk 67m Normal Created pod/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t Container created 67m Normal Pulled pod/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.978s (2.978s including waiting). Image size: 125353963 bytes. 67m Normal Started pod/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t Container started 67m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-dd6574-5fd9c947b7 Error creating: pods "payload-pre-processing-e2e-worker-w0-dd6574-5fd9c947b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-dd6574: serviceaccount "payload-processing-e2e-worker-w0-dd6574" not found 67m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-dd6574-79864967d7 Created pod: payload-processing-e2e-worker-w0-dd6574-79864967d7-f8wcm 67m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-dd6574-79864967d7 Error creating: pods "payload-processing-e2e-worker-w0-dd6574-79864967d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-dd6574: serviceaccount "payload-processing-e2e-worker-w0-dd6574" not found 67m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-dd6574-5fd9c947b7 Created pod: payload-pre-processing-e2e-worker-w0-dd6574-5fd9c947b7-x7c6s 67m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-dd6574 Scaled up replica set payload-processing-e2e-worker-w0-dd6574-79864967d7 from 0 to 1 67m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-dd6574 Scaled up replica set payload-pre-processing-e2e-worker-w0-dd6574-5fd9c947b7 from 0 to 1 67m Normal Pulled pod/payload-processing-e2e-worker-w0-dd6574-79864967d7-f8wcm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-dd6574-5fd9c947b7-x7c6s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-dd6574-5fd9c947b7-x7c6s Add eth0 [10.133.0.60/23] from ovn-kubernetes 67m Normal Created pod/payload-pre-processing-e2e-worker-w0-dd6574-5fd9c947b7-x7c6s Container created 67m Normal Started pod/payload-pre-processing-e2e-worker-w0-dd6574-5fd9c947b7-x7c6s Container started 67m Normal Started pod/payload-processing-e2e-worker-w0-dd6574-79864967d7-f8wcm Container started 67m Normal Created pod/payload-processing-e2e-worker-w0-dd6574-79864967d7-f8wcm Container created 67m Normal AddedInterface pod/payload-processing-e2e-worker-w0-dd6574-79864967d7-f8wcm Add eth0 [10.134.0.54/23] from ovn-kubernetes 67m Normal Created pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-cl9rj Container created 67m Normal Started pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-cl9rj Container started 67m Normal Pulled pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-cl9rj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 67m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-4e2b27 Scaled up replica set payload-processing-e2e-worker-w4-4e2b27-5569b97fc9 from 0 to 1 67m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-ef6b43-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 67m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-ef6b43-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 67m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-4e2b27-5569b97fc9 Error creating: pods "payload-processing-e2e-worker-w4-4e2b27-5569b97fc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-4e2b27: serviceaccount "payload-processing-e2e-worker-w4-4e2b27" not found 67m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-4e2b27-5569b97fc9 Created pod: payload-processing-e2e-worker-w4-4e2b27-5569b97fc9-q286k 67m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-4e2b27-987d59d59 Error creating: pods "payload-pre-processing-e2e-worker-w4-4e2b27-987d59d59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-4e2b27: serviceaccount "payload-processing-e2e-worker-w4-4e2b27" not found 67m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-4e2b27-987d59d59 Created pod: payload-pre-processing-e2e-worker-w4-4e2b27-987d59d59-4j6sw 67m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-4e2b27 Scaled up replica set payload-pre-processing-e2e-worker-w4-4e2b27-987d59d59 from 0 to 1 67m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-4e2b27-987d59d59-4j6sw Add eth0 [10.133.0.61/23] from ovn-kubernetes 67m Normal AddedInterface pod/payload-processing-e2e-worker-w4-4e2b27-5569b97fc9-q286k Add eth0 [10.133.0.62/23] from ovn-kubernetes 67m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-4e2b27-987d59d59-4j6sw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Normal Created pod/payload-pre-processing-e2e-worker-w4-4e2b27-987d59d59-4j6sw Container created 67m Normal Pulled pod/payload-processing-e2e-worker-w4-4e2b27-5569b97fc9-q286k Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Normal Started pod/payload-pre-processing-e2e-worker-w4-4e2b27-987d59d59-4j6sw Container started 67m Normal Created pod/payload-processing-e2e-worker-w4-4e2b27-5569b97fc9-q286k Container created 67m Normal Started pod/payload-processing-e2e-worker-w4-4e2b27-5569b97fc9-q286k Container started 67m Warning Unhealthy pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-cl9rj Startup probe failed: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused 67m Warning ProbeError pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-cl9rj Startup probe error: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused... 67m Normal Pulled pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-bbrxc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 67m Normal Started pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-lmksp Container started 67m Normal Created pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-lmksp Container created 67m Normal Pulled pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-lmksp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 67m Normal Started pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-bbrxc Container started 67m Normal Created pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-bbrxc Container created 67m Warning Unhealthy pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-bbrxc Startup probe failed: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused 67m Warning ProbeError pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-bbrxc Startup probe error: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused... 67m Warning ProbeError pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-lmksp Startup probe error: Get "http://10.133.0.50:15021/healthz/ready": dial tcp 10.133.0.50:15021: connect: connection refused... 67m Warning Unhealthy pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-lmksp Startup probe failed: Get "http://10.133.0.50:15021/healthz/ready": dial tcp 10.133.0.50:15021: connect: connection refused 67m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-4d9ea0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 67m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-4d9ea0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Normal Started pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-4ttqx Container started 66m Warning ProbeError pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-64wq6 Readiness probe error: Get "http://10.134.0.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 66m Warning Unhealthy pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-64wq6 Readiness probe failed: Get "http://10.134.0.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 66m Normal Started pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-64wq6 Container started 66m Warning Unhealthy pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-4ttqx Readiness probe failed: Get "http://10.134.0.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 66m Warning ProbeError pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-4ttqx Readiness probe error: Get "http://10.134.0.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 66m Normal Created pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-64wq6 Container created 66m Normal Created pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-4ttqx Container created 66m Normal Pulled pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-4ttqx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Normal Pulled pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-64wq6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Warning ProbeError pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-4ttqx Startup probe error: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused... 66m Warning Unhealthy pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-4ttqx Startup probe failed: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused 66m Warning ProbeError pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-64wq6 Startup probe error: Get "http://10.134.0.48:15021/healthz/ready": dial tcp 10.134.0.48:15021: connect: connection refused... 66m Warning Unhealthy pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-64wq6 Startup probe failed: Get "http://10.134.0.48:15021/healthz/ready": dial tcp 10.134.0.48:15021: connect: connection refused 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-ef6b43-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Normal Pulled pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-255tp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-ef6b43-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Normal Started pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gnxj5 Container started 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-4e2b27-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Normal Pulled pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gnxj5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-284b8e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-dd6574-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-dd6574-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Normal Created pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gnxj5 Container created 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-4e2b27-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Normal Started pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-255tp Container started 66m Normal Created pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-255tp Container created 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-284b8e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning Unhealthy pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-255tp Startup probe failed: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused 66m Warning ProbeError pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-255tp Startup probe error: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused... 66m Warning Unhealthy pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gnxj5 Startup probe failed: Get "http://10.134.0.46:15021/healthz/ready": dial tcp 10.134.0.46:15021: connect: connection refused 66m Warning ProbeError pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gnxj5 Startup probe error: Get "http://10.134.0.46:15021/healthz/ready": dial tcp 10.134.0.46:15021: connect: connection refused... 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-c2dd3b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-4d9ea0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-4d9ea0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-9a6cf2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-9a6cf2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-c2dd3b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-284b8e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-284b8e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-9a6cf2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-9a6cf2-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-4e2b27-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-4e2b27-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-dd6574-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-dd6574-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Normal ScalingReplicaSet deployment/e2e-worker-w0-dd6574-openshift-default Scaled up replica set e2e-worker-w0-dd6574-openshift-default-5d877f7ccc from 1 to 2 63m Normal SuccessfulCreate replicaset/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc Created pod: e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-9bjf7 63m Normal Pulled pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-9bjf7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-dd6574-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 63m Normal AddedInterface pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-9bjf7 Add eth0 [10.134.0.57/23] from ovn-kubernetes 63m Normal Started pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-9bjf7 Container started 63m Normal Created pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-9bjf7 Container created 63m Warning ProbeError pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-9bjf7 Startup probe error: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-9bjf7 Startup probe failed: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused 63m Normal ScalingReplicaSet deployment/e2e-worker-w5-ef6b43-openshift-default Scaled up replica set e2e-worker-w5-ef6b43-openshift-default-679658c459 from 1 to 2 63m Normal SuccessfulCreate replicaset/e2e-worker-w5-ef6b43-openshift-default-679658c459 Created pod: e2e-worker-w5-ef6b43-openshift-default-679658c459-zpmjw 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-ef6b43-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 63m Normal Started pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-zpmjw Container started 63m Normal AddedInterface pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-zpmjw Add eth0 [10.134.0.58/23] from ovn-kubernetes 63m Normal Pulled pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-zpmjw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Created pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-zpmjw Container created 63m Warning Unhealthy pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-zpmjw Startup probe failed: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-zpmjw Startup probe error: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused... 63m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: All metrics below target 63m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-77bfb58f8c Deleted pod: maas-default-gateway-openshift-default-77bfb58f8c-wkbnw 63m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-77bfb58f8c from 5 to 4 63m Normal Killing pod/maas-default-gateway-openshift-default-77bfb58f8c-wkbnw Stopping container istio-proxy 62m Normal ScalingReplicaSet deployment/e2e-worker-w1-c2dd3b-openshift-default Scaled up replica set e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f from 1 to 2 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-c2dd3b-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 62m Normal Started pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-48wgm Container started 62m Normal Pulled pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-48wgm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal ScalingReplicaSet deployment/e2e-worker-w3-9a6cf2-openshift-default Scaled up replica set e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c from 1 to 2 62m Normal SuccessfulCreate replicaset/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c Created pod: e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-2ssdg 62m Normal Created pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-48wgm Container created 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-9a6cf2-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 62m Normal AddedInterface pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-2ssdg Add eth0 [10.133.0.87/23] from ovn-kubernetes 62m Normal AddedInterface pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-48wgm Add eth0 [10.134.0.59/23] from ovn-kubernetes 62m Normal SuccessfulCreate replicaset/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f Created pod: e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-48wgm 62m Normal Pulled pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-2ssdg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Created pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-2ssdg Container created 62m Normal Started pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-2ssdg Container started 62m Warning ProbeError pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-2ssdg Startup probe error: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused... 62m Warning ProbeError pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-48wgm Startup probe error: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-2ssdg Startup probe failed: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused 62m Warning Unhealthy pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-48wgm Startup probe failed: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused 62m Normal SuccessfulCreate replicaset/e2e-worker-w6-284b8e-openshift-default-77c5d9855 Created pod: e2e-worker-w6-284b8e-openshift-default-77c5d9855-dz5hw 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-284b8e-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 62m Normal ScalingReplicaSet deployment/e2e-worker-w6-284b8e-openshift-default Scaled up replica set e2e-worker-w6-284b8e-openshift-default-77c5d9855 from 1 to 2 62m Normal Started pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-dz5hw Container started 62m Normal AddedInterface pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-dz5hw Add eth0 [10.133.0.97/23] from ovn-kubernetes 62m Normal Created pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-dz5hw Container created 62m Normal Pulled pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-dz5hw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Warning Unhealthy pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-dz5hw Startup probe failed: Get "http://10.133.0.97:15021/healthz/ready": dial tcp 10.133.0.97:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-dz5hw Startup probe error: Get "http://10.133.0.97:15021/healthz/ready": dial tcp 10.133.0.97:15021: connect: connection refused... 62m Normal Killing pod/maas-default-gateway-openshift-default-77bfb58f8c-7x282 Stopping container istio-proxy 62m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-77bfb58f8c Deleted pod: maas-default-gateway-openshift-default-77bfb58f8c-7x282 62m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-77bfb58f8c from 4 to 3 62m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: All metrics below target 61m Normal ScalingReplicaSet deployment/e2e-worker-w3-9a6cf2-openshift-default Scaled up replica set e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c from 2 to 3 61m Normal SuccessfulCreate replicaset/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c Created pod: e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-9a6cf2-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 61m Normal Started pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp Container started 61m Normal Created pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp Container created 61m Normal Pulled pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp Add eth0 [10.132.0.52/23] from ovn-kubernetes 61m Warning Unhealthy pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp Startup probe failed: Get "http://10.132.0.52:15021/healthz/ready": dial tcp 10.132.0.52:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp Startup probe error: Get "http://10.132.0.52:15021/healthz/ready": dial tcp 10.132.0.52:15021: connect: connection refused... 61m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 2; reason: All metrics below target 61m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-77bfb58f8c Deleted pod: maas-default-gateway-openshift-default-77bfb58f8c-j9dd9 61m Normal Killing pod/maas-default-gateway-openshift-default-77bfb58f8c-j9dd9 Stopping container istio-proxy 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-4d9ea0-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 61m Normal ScalingReplicaSet deployment/e2e-worker-w2-4d9ea0-openshift-default Scaled up replica set e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc from 1 to 2 61m Normal SuccessfulCreate replicaset/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc Created pod: e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-jltps 61m Normal Created pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-jltps Container created 61m Normal AddedInterface pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-jltps Add eth0 [10.134.0.62/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-jltps Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Started pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-jltps Container started 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-4e2b27-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 61m Normal SuccessfulCreate replicaset/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf Created pod: e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cf9jh 61m Normal ScalingReplicaSet deployment/e2e-worker-w4-4e2b27-openshift-default Scaled up replica set e2e-worker-w4-4e2b27-openshift-default-996dc4fdf from 1 to 2 61m Normal AddedInterface pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cf9jh Add eth0 [10.133.0.99/23] from ovn-kubernetes 61m Normal Started pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cf9jh Container started 61m Normal Created pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cf9jh Container created 61m Normal Pulled pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cf9jh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Warning Unhealthy pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-jltps Startup probe failed: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-jltps Startup probe error: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cf9jh Startup probe failed: Get "http://10.133.0.99:15021/healthz/ready": dial tcp 10.133.0.99:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cf9jh Startup probe error: Get "http://10.133.0.99:15021/healthz/ready": dial tcp 10.133.0.99:15021: connect: connection refused... 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-dd6574-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 61m Normal AddedInterface pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5khvm Add eth0 [10.134.0.64/23] from ovn-kubernetes 61m Normal ScalingReplicaSet deployment/e2e-worker-w0-dd6574-openshift-default Scaled up replica set e2e-worker-w0-dd6574-openshift-default-5d877f7ccc from 2 to 4 61m Normal Pulled pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5khvm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x Add eth0 [10.132.0.53/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal SuccessfulCreate replicaset/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc Created pod: e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5khvm 61m Normal SuccessfulCreate replicaset/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc Created pod: e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x 61m Normal Started pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5khvm Container started 61m Normal Created pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5khvm Container created 61m Normal Created pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x Container created 61m Normal Started pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x Container started 61m Warning ProbeError pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5khvm Startup probe error: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5khvm Startup probe failed: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x Startup probe error: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x Startup probe failed: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused 61m Normal SuccessfulCreate replicaset/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc Created pod: e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tzgp6 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-dd6574-openshift-default New size: 5; reason: All metrics below target 61m Normal ScalingReplicaSet deployment/e2e-worker-w0-dd6574-openshift-default Scaled up replica set e2e-worker-w0-dd6574-openshift-default-5d877f7ccc from 4 to 5 61m Normal AddedInterface pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tzgp6 Add eth0 [10.133.0.100/23] from ovn-kubernetes 61m Normal Started pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tzgp6 Container started 61m Normal Created pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tzgp6 Container created 61m Normal Pulled pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tzgp6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Warning Unhealthy pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tzgp6 Startup probe failed: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tzgp6 Startup probe error: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused... 60m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-c2dd3b-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 60m Normal SuccessfulCreate replicaset/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f Created pod: e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf 60m Normal ScalingReplicaSet deployment/e2e-worker-w1-c2dd3b-openshift-default Scaled up replica set e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f from 2 to 3 60m Normal SuccessfulCreate replicaset/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc Created pod: e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d 60m Normal ScalingReplicaSet deployment/e2e-worker-w2-4d9ea0-openshift-default Scaled up replica set e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc from 2 to 4 60m Normal SuccessfulCreate replicaset/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc Created pod: e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-tkbbb 60m Normal AddedInterface pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf Add eth0 [10.132.0.55/23] from ovn-kubernetes 60m Normal Pulled pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Created pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf Container created 60m Normal Started pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf Container started 60m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-4d9ea0-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 60m Normal Started pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d Container started 60m Normal Started pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-tkbbb Container started 60m Normal Created pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d Container created 60m Normal Pulled pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal AddedInterface pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-tkbbb Add eth0 [10.134.0.69/23] from ovn-kubernetes 60m Normal AddedInterface pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d Add eth0 [10.132.0.56/23] from ovn-kubernetes 60m Normal Pulled pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-tkbbb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Created pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-tkbbb Container created 60m Warning Unhealthy pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d Startup probe failed: Get "http://10.132.0.56:15021/healthz/ready": dial tcp 10.132.0.56:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf Startup probe error: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-tkbbb Startup probe failed: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d Startup probe error: Get "http://10.132.0.56:15021/healthz/ready": dial tcp 10.132.0.56:15021: connect: connection refused... 60m Warning ProbeError pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-tkbbb Startup probe error: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf Startup probe failed: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused 60m Normal ScalingReplicaSet deployment/e2e-worker-w5-ef6b43-openshift-default Scaled up replica set e2e-worker-w5-ef6b43-openshift-default-679658c459 from 2 to 3 60m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-ef6b43-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 60m Normal SuccessfulCreate replicaset/e2e-worker-w5-ef6b43-openshift-default-679658c459 Created pod: e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht 60m Normal Created pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns Container created 60m Normal Pulled pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal ScalingReplicaSet deployment/e2e-worker-w4-4e2b27-openshift-default Scaled up replica set e2e-worker-w4-4e2b27-openshift-default-996dc4fdf from 2 to 4 60m Normal AddedInterface pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-fsc7d Add eth0 [10.133.0.102/23] from ovn-kubernetes 60m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-4e2b27-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 60m Normal SuccessfulCreate replicaset/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf Created pod: e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns 60m Normal Started pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns Container started 60m Normal Started pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht Container started 60m Normal AddedInterface pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht Add eth0 [10.132.0.57/23] from ovn-kubernetes 60m Normal Pulled pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-fsc7d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal SuccessfulCreate replicaset/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf Created pod: e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-fsc7d 60m Normal Pulled pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Created pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht Container created 60m Normal AddedInterface pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns Add eth0 [10.132.0.58/23] from ovn-kubernetes 60m Normal Started pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-fsc7d Container started 60m Normal Created pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-fsc7d Container created 60m Warning ProbeError pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht Startup probe error: Get "http://10.132.0.57:15021/healthz/ready": dial tcp 10.132.0.57:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht Startup probe failed: Get "http://10.132.0.57:15021/healthz/ready": dial tcp 10.132.0.57:15021: connect: connection refused 60m Warning Unhealthy pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns Startup probe failed: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns Startup probe error: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused... 60m Warning ProbeError pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-fsc7d Startup probe error: Get "http://10.133.0.102:15021/healthz/ready": dial tcp 10.133.0.102:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-fsc7d Startup probe failed: Get "http://10.133.0.102:15021/healthz/ready": dial tcp 10.133.0.102:15021: connect: connection refused 59m Normal ScalingReplicaSet deployment/e2e-worker-w3-9a6cf2-openshift-default Scaled up replica set e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c from 3 to 6 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-9a6cf2-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 59m Normal SuccessfulCreate replicaset/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c Created pod: e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-sn5cj 59m Normal SuccessfulCreate replicaset/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c Created pod: e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6 59m Normal SuccessfulCreate replicaset/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c Created pod: e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-rhzmq 59m Normal AddedInterface pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-rhzmq Add eth0 [10.134.0.70/23] from ovn-kubernetes 59m Normal Created pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6 Container created 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-dd6574-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 59m Normal AddedInterface pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-sn5cj Add eth0 [10.133.0.104/23] from ovn-kubernetes 59m Normal Created pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-rhzmq Container created 59m Normal Pulled pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal ScalingReplicaSet deployment/e2e-worker-w0-dd6574-openshift-default Scaled up replica set e2e-worker-w0-dd6574-openshift-default-5d877f7ccc from 5 to 7 59m Normal Started pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-sn5cj Container started 59m Normal Started pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-rhzmq Container started 59m Normal Pulled pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-sn5cj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal SuccessfulCreate replicaset/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc Created pod: e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd 59m Normal SuccessfulCreate replicaset/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc Created pod: e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-lwg4n 59m Normal Created pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-sn5cj Container created 59m Normal Started pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6 Container started 59m Normal AddedInterface pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6 Add eth0 [10.132.0.59/23] from ovn-kubernetes 59m Normal Pulled pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-rhzmq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-lwg4n Add eth0 [10.134.0.71/23] from ovn-kubernetes 59m Normal Pulled pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd Container created 59m Normal AddedInterface pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd Add eth0 [10.132.0.60/23] from ovn-kubernetes 59m Normal Started pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd Container started 59m Normal Pulled pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-lwg4n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-lwg4n Container created 59m Normal Started pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-lwg4n Container started 59m Warning Unhealthy pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-rhzmq Startup probe failed: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-rhzmq Startup probe error: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused... 59m Warning ProbeError pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6 Startup probe error: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6 Startup probe failed: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused 59m Warning Unhealthy pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-sn5cj Startup probe failed: Get "http://10.133.0.104:15021/healthz/ready": dial tcp 10.133.0.104:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-sn5cj Startup probe error: Get "http://10.133.0.104:15021/healthz/ready": dial tcp 10.133.0.104:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd Startup probe failed: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd Startup probe error: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-lwg4n Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-lwg4n Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused... 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-284b8e-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 59m Normal ScalingReplicaSet deployment/e2e-worker-w6-284b8e-openshift-default Scaled up replica set e2e-worker-w6-284b8e-openshift-default-77c5d9855 from 2 to 3 59m Normal SuccessfulCreate replicaset/e2e-worker-w6-284b8e-openshift-default-77c5d9855 Created pod: e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq 59m Normal Pulled pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq Container created 59m Normal Started pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq Container started 59m Normal AddedInterface pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq Add eth0 [10.132.0.61/23] from ovn-kubernetes 59m Warning ProbeError pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq Startup probe error: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq Startup probe failed: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-jltps Readiness probe error: Get "http://10.134.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning Unhealthy pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-jltps Readiness probe failed: Get "http://10.134.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-9a6cf2-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 58m Normal ScalingReplicaSet deployment/e2e-worker-w3-9a6cf2-openshift-default Scaled up replica set e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c from 6 to 8 58m Normal SuccessfulCreate replicaset/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c Created pod: e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-7twrh 58m Normal SuccessfulCreate replicaset/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c Created pod: e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-tvcl2 58m Normal Started pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-tvcl2 Container started 58m Normal Started pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-7twrh Container started 58m Normal Pulled pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-tvcl2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-tvcl2 Add eth0 [10.133.0.111/23] from ovn-kubernetes 58m Normal AddedInterface pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-7twrh Add eth0 [10.134.0.73/23] from ovn-kubernetes 58m Normal Created pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-tvcl2 Container created 58m Normal Pulled pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-7twrh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-7twrh Container created 58m Warning Unhealthy pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-tvcl2 Startup probe failed: Get "http://10.133.0.111:15021/healthz/ready": dial tcp 10.133.0.111:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-7twrh Startup probe error: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused... 58m Warning ProbeError pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-tvcl2 Startup probe error: Get "http://10.133.0.111:15021/healthz/ready": dial tcp 10.133.0.111:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-7twrh Startup probe failed: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-dd6574-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 58m Normal ScalingReplicaSet deployment/e2e-worker-w0-dd6574-openshift-default Scaled up replica set e2e-worker-w0-dd6574-openshift-default-5d877f7ccc from 7 to 10 58m Normal SuccessfulCreate replicaset/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc (combined from similar events): Created pod: e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9 58m Normal SuccessfulCreate replicaset/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc Created pod: e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5kdtc 58m Normal SuccessfulCreate replicaset/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc Created pod: e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-qrspz 58m Normal Pulled pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5kdtc Container created 58m Normal AddedInterface pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9 Add eth0 [10.132.0.62/23] from ovn-kubernetes 58m Normal Created pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9 Container created 58m Normal AddedInterface pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5kdtc Add eth0 [10.133.0.114/23] from ovn-kubernetes 58m Normal AddedInterface pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-qrspz Add eth0 [10.134.0.75/23] from ovn-kubernetes 58m Normal Started pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9 Container started 58m Normal Pulled pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5kdtc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Started pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5kdtc Container started 58m Normal Started pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-qrspz Container started 58m Normal Created pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-qrspz Container created 58m Normal Pulled pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-qrspz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Warning ProbeError pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-qrspz Startup probe error: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9 Startup probe failed: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused 58m Warning Unhealthy pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-qrspz Startup probe failed: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5kdtc Startup probe error: Get "http://10.133.0.114:15021/healthz/ready": dial tcp 10.133.0.114:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5kdtc Startup probe failed: Get "http://10.133.0.114:15021/healthz/ready": dial tcp 10.133.0.114:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9 Startup probe error: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused... 58m Warning ProbeError pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9 Startup probe error: Get "http://10.132.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning Unhealthy pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9 Startup probe failed: Get "http://10.132.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning Unhealthy pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5kdtc Startup probe failed: Get "http://10.133.0.114:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning ProbeError pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5kdtc Startup probe error: Get "http://10.133.0.114:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Normal SuccessfulCreate replicaset/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc Created pod: e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-wjhwc 58m Normal SuccessfulCreate replicaset/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc Created pod: e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf 58m Normal SuccessfulCreate replicaset/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc Created pod: e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-tglx5 58m Normal SuccessfulCreate replicaset/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc Created pod: e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-wv79b 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-4d9ea0-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 58m Normal ScalingReplicaSet deployment/e2e-worker-w2-4d9ea0-openshift-default Scaled up replica set e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc from 4 to 8 58m Normal AddedInterface pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-wjhwc Add eth0 [10.133.0.118/23] from ovn-kubernetes 58m Normal AddedInterface pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-tglx5 Add eth0 [10.134.0.76/23] from ovn-kubernetes 58m Normal Started pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-wv79b Container started 58m Normal Started pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf Container started 58m Normal AddedInterface pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf Add eth0 [10.132.0.63/23] from ovn-kubernetes 58m Normal Created pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-wv79b Container created 58m Normal Pulled pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-tglx5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Started pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-tglx5 Container started 58m Normal Created pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-tglx5 Container created 58m Normal Pulled pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-wjhwc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Pulled pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-wjhwc Container created 58m Normal Started pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-wjhwc Container started 58m Normal Created pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf Container created 58m Normal Pulled pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-wv79b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-wv79b Add eth0 [10.133.0.119/23] from ovn-kubernetes 58m Warning Unhealthy pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-wjhwc Startup probe failed: Get "http://10.133.0.118:15021/healthz/ready": dial tcp 10.133.0.118:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-wjhwc Startup probe error: Get "http://10.133.0.118:15021/healthz/ready": dial tcp 10.133.0.118:15021: connect: connection refused... 58m Warning ProbeError pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-tglx5 Startup probe error: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-tglx5 Startup probe failed: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf Startup probe error: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf Startup probe failed: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused 58m Warning Unhealthy pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-wv79b Startup probe failed: Get "http://10.133.0.119:15021/healthz/ready": dial tcp 10.133.0.119:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-wv79b Startup probe error: Get "http://10.133.0.119:15021/healthz/ready": dial tcp 10.133.0.119:15021: connect: connection refused... 57m Normal ScalingReplicaSet deployment/e2e-worker-w1-c2dd3b-openshift-default Scaled up replica set e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f from 3 to 6 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-c2dd3b-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 57m Normal Started pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-7hdld Container started 57m Normal Started pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb Container started 57m Normal AddedInterface pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb Add eth0 [10.132.0.64/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb Container created 57m Normal SuccessfulCreate replicaset/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f Created pod: e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-5rd2m 57m Normal SuccessfulCreate replicaset/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f Created pod: e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-7hdld 57m Normal SuccessfulCreate replicaset/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc Created pod: e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-4d9ea0-openshift-default New size: 9; reason: All metrics below target 57m Normal ScalingReplicaSet deployment/e2e-worker-w2-4d9ea0-openshift-default Scaled up replica set e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc from 8 to 9 57m Normal SuccessfulCreate replicaset/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f Created pod: e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb 57m Normal Created pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-7hdld Container created 57m Normal Pulled pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-7hdld Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-7hdld Add eth0 [10.133.0.122/23] from ovn-kubernetes 57m Normal SuccessfulCreate replicaset/e2e-worker-w5-ef6b43-openshift-default-679658c459 Created pod: e2e-worker-w5-ef6b43-openshift-default-679658c459-jqdtb 57m Normal AddedInterface pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-5rd2m Add eth0 [10.134.0.77/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-5rd2m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-5rd2m Container created 57m Normal Started pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-5rd2m Container started 57m Normal SuccessfulCreate replicaset/e2e-worker-w5-ef6b43-openshift-default-679658c459 Created pod: e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk 57m Normal SuccessfulCreate replicaset/e2e-worker-w5-ef6b43-openshift-default-679658c459 Created pod: e2e-worker-w5-ef6b43-openshift-default-679658c459-nmbnd 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-ef6b43-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 57m Normal ScalingReplicaSet deployment/e2e-worker-w5-ef6b43-openshift-default Scaled up replica set e2e-worker-w5-ef6b43-openshift-default-679658c459 from 3 to 6 57m Normal Started pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-jqdtb Container started 57m Normal AddedInterface pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-nmbnd Add eth0 [10.133.0.123/23] from ovn-kubernetes 57m Normal AddedInterface pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp Add eth0 [10.132.0.67/23] from ovn-kubernetes 57m Normal SuccessfulCreate replicaset/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf Created pod: e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-25ttn 57m Normal SuccessfulCreate replicaset/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf Created pod: e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp 57m Normal Started pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j Container started 57m Normal Started pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk Container started 57m Normal Created pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk Container created 57m Normal Pulled pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal SuccessfulCreate replicaset/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf Created pod: e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-7ntr6 57m Normal Pulled pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk Add eth0 [10.132.0.66/23] from ovn-kubernetes 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-4e2b27-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 57m Normal Created pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j Container created 57m Normal AddedInterface pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j Add eth0 [10.132.0.65/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-285rr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Started pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-nmbnd Container started 57m Normal Created pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-nmbnd Container created 57m Normal ScalingReplicaSet deployment/e2e-worker-w4-4e2b27-openshift-default Scaled up replica set e2e-worker-w4-4e2b27-openshift-default-996dc4fdf from 4 to 8 57m Normal Pulled pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-nmbnd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-285rr Add eth0 [10.134.0.80/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-25ttn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-25ttn Add eth0 [10.134.0.79/23] from ovn-kubernetes 57m Normal AddedInterface pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-7ntr6 Add eth0 [10.133.0.124/23] from ovn-kubernetes 57m Normal AddedInterface pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-jqdtb Add eth0 [10.134.0.78/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal SuccessfulCreate replicaset/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf Created pod: e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-285rr 57m Normal Pulled pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-jqdtb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-jqdtb Container created 57m Normal Started pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp Container started 57m Normal Pulled pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-7ntr6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-7ntr6 Container created 57m Normal Started pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-7ntr6 Container started 57m Normal Created pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-25ttn Container created 57m Normal Started pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-25ttn Container started 57m Normal Created pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp Container created 57m Normal Started pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-285rr Container started 57m Normal Created pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-285rr Container created 57m Warning ProbeError pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-7hdld Startup probe error: Get "http://10.133.0.122:15021/healthz/ready": dial tcp 10.133.0.122:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-7hdld Startup probe failed: Get "http://10.133.0.122:15021/healthz/ready": dial tcp 10.133.0.122:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-5rd2m Startup probe error: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-5rd2m Startup probe failed: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-nmbnd Startup probe failed: Get "http://10.133.0.123:15021/healthz/ready": dial tcp 10.133.0.123:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-25ttn Startup probe error: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb Startup probe error: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb Startup probe failed: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-7ntr6 Startup probe error: Get "http://10.133.0.124:15021/healthz/ready": dial tcp 10.133.0.124:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-jqdtb Startup probe failed: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-7ntr6 Startup probe failed: Get "http://10.133.0.124:15021/healthz/ready": dial tcp 10.133.0.124:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-25ttn Startup probe failed: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-nmbnd Startup probe error: Get "http://10.133.0.123:15021/healthz/ready": dial tcp 10.133.0.123:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-285rr Startup probe error: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-jqdtb Startup probe error: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-285rr Startup probe failed: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j Startup probe failed: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp Startup probe failed: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk Startup probe error: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk Startup probe failed: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j Startup probe error: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp Startup probe error: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused... 57m Normal SuccessfulCreate replicaset/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c Created pod: e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc 57m Normal SuccessfulCreate replicaset/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c (combined from similar events): Created pod: e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-m464d 57m Normal SuccessfulCreate replicaset/e2e-worker-w5-ef6b43-openshift-default-679658c459 Created pod: e2e-worker-w5-ef6b43-openshift-default-679658c459-gvmnx 57m Normal ScalingReplicaSet deployment/e2e-worker-w3-9a6cf2-openshift-default Scaled up replica set e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c from 8 to 10 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-9a6cf2-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-ef6b43-openshift-default New size: 7; reason: All metrics below target 57m Normal ScalingReplicaSet deployment/e2e-worker-w5-ef6b43-openshift-default Scaled up replica set e2e-worker-w5-ef6b43-openshift-default-679658c459 from 6 to 7 57m Normal Started pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-m464d Container started 57m Normal ScalingReplicaSet deployment/e2e-worker-w4-4e2b27-openshift-default Scaled up replica set e2e-worker-w4-4e2b27-openshift-default-996dc4fdf from 8 to 9 57m Normal AddedInterface pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq Add eth0 [10.132.0.69/23] from ovn-kubernetes 57m Normal AddedInterface pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-m464d Add eth0 [10.133.0.129/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-m464d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal SuccessfulCreate replicaset/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf Created pod: e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq 57m Normal AddedInterface pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-gvmnx Add eth0 [10.134.0.81/23] from ovn-kubernetes 57m Normal AddedInterface pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc Add eth0 [10.132.0.68/23] from ovn-kubernetes 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-4e2b27-openshift-default New size: 9; reason: All metrics below target 57m Normal Created pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-m464d Container created 57m Normal Pulled pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc Container created 57m Normal Started pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc Container started 57m Normal Pulled pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-gvmnx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Started pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-gvmnx Container started 57m Normal Created pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq Container created 57m Normal Started pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq Container started 57m Normal Pulled pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-gvmnx Container created 57m Warning ProbeError pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-2ssdg Readiness probe error: Get "http://10.133.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 57m Warning Unhealthy pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-2ssdg Readiness probe failed: Get "http://10.133.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 57m Warning ProbeError pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-m464d Startup probe error: Get "http://10.133.0.129:15021/healthz/ready": dial tcp 10.133.0.129:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-m464d Startup probe failed: Get "http://10.133.0.129:15021/healthz/ready": dial tcp 10.133.0.129:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-gvmnx Startup probe failed: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-gvmnx Startup probe error: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc Startup probe failed: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc Startup probe error: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-m464d Startup probe failed: Get "http://10.133.0.129:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 57m Warning ProbeError pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-m464d Startup probe error: Get "http://10.133.0.129:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 57m Warning Unhealthy pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq Startup probe failed: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq Startup probe error: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc Startup probe failed: Get "http://10.132.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 57m Warning ProbeError pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc Startup probe error: Get "http://10.132.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-284b8e-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 57m Normal ScalingReplicaSet deployment/e2e-worker-w6-284b8e-openshift-default Scaled up replica set e2e-worker-w6-284b8e-openshift-default-77c5d9855 from 3 to 5 57m Normal SuccessfulCreate replicaset/e2e-worker-w6-284b8e-openshift-default-77c5d9855 Created pod: e2e-worker-w6-284b8e-openshift-default-77c5d9855-qptp7 57m Normal SuccessfulCreate replicaset/e2e-worker-w6-284b8e-openshift-default-77c5d9855 Created pod: e2e-worker-w6-284b8e-openshift-default-77c5d9855-252cj 57m Normal AddedInterface pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-252cj Add eth0 [10.134.0.82/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-qptp7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Pulled pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-252cj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-252cj Container created 57m Normal Started pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-252cj Container started 57m Normal Started pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-qptp7 Container started 57m Normal Created pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-qptp7 Container created 57m Normal AddedInterface pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-qptp7 Add eth0 [10.133.0.136/23] from ovn-kubernetes 57m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-77bfb58f8c Deleted pod: maas-default-gateway-openshift-default-77bfb58f8c-tcck7 57m Normal Killing pod/maas-default-gateway-openshift-default-77bfb58f8c-tcck7 Stopping container istio-proxy 57m Warning ProbeError pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-qptp7 Startup probe error: Get "http://10.133.0.136:15021/healthz/ready": dial tcp 10.133.0.136:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-252cj Startup probe error: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-252cj Startup probe failed: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-qptp7 Startup probe failed: Get "http://10.133.0.136:15021/healthz/ready": dial tcp 10.133.0.136:15021: connect: connection refused 57m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-77bfb58f8c from 2 to 1 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-c2dd3b-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 57m Normal AddedInterface pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-dxqcj Add eth0 [10.134.0.83/23] from ovn-kubernetes 57m Normal SuccessfulCreate replicaset/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f Created pod: e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-79tql 57m Normal SuccessfulCreate replicaset/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f Created pod: e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-dxqcj 57m Normal ScalingReplicaSet deployment/e2e-worker-w1-c2dd3b-openshift-default Scaled up replica set e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f from 6 to 8 57m Normal Started pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-dxqcj Container started 57m Normal AddedInterface pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-79tql Add eth0 [10.133.0.141/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-79tql Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-dxqcj Container created 57m Normal Created pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-79tql Container created 57m Normal Started pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-79tql Container started 57m Normal Pulled pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-dxqcj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Warning ProbeError pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-dxqcj Startup probe error: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-dxqcj Startup probe failed: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-79tql Startup probe failed: Get "http://10.133.0.141:15021/healthz/ready": dial tcp 10.133.0.141:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-79tql Startup probe error: Get "http://10.133.0.141:15021/healthz/ready": dial tcp 10.133.0.141:15021: connect: connection refused... 57m Normal AddedLabel gateway/e2e-ait-c39c4123 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-c39c4123 57m Normal ScalingReplicaSet deployment/e2e-ait-c39c4123-openshift-default Scaled up replica set e2e-ait-c39c4123-openshift-default-769746bdd5 from 0 to 1 57m Normal SuccessfulCreate replicaset/e2e-ait-c39c4123-openshift-default-769746bdd5 Created pod: e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh 57m Normal AddedLabel gateway/e2e-migrate-2ccb9fb8-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-2ccb9fb8-gw 57m Normal ScalingReplicaSet deployment/e2e-migrate-2ccb9fb8-gw-openshift-default Scaled up replica set e2e-migrate-2ccb9fb8-gw-openshift-default-bd688b445 from 0 to 1 57m Normal SuccessfulCreate replicaset/e2e-migrate-2ccb9fb8-gw-openshift-default-bd688b445 Created pod: e2e-migrate-2ccb9fb8-gw-openshift-default-bd688b445-9vzqb 57m Warning ProbeError pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-qrspz Readiness probe error: Get "http://10.134.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Warning Unhealthy pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-qrspz Readiness probe failed: Get "http://10.134.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning Unhealthy pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5kdtc Readiness probe failed: Get "http://10.133.0.114:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9 Readiness probe error: Get "http://10.132.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Warning Unhealthy pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9 Readiness probe failed: Get "http://10.132.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5kdtc Readiness probe error: Get "http://10.133.0.114:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Normal AddedInterface pod/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh Add eth0 [10.132.0.72/23] from ovn-kubernetes 56m Normal AddedInterface pod/e2e-migrate-2ccb9fb8-gw-openshift-default-bd688b445-9vzqb Add eth0 [10.134.0.84/23] from ovn-kubernetes 56m Normal AddedLabel gateway/e2e-derive-4dff0429 Added label istio.io/rev=openshift-gateway to gateway e2e-derive-4dff0429 56m Normal ScalingReplicaSet deployment/e2e-derive-4dff0429-openshift-default Scaled up replica set e2e-derive-4dff0429-openshift-default-559487f9c7 from 0 to 1 56m Normal SuccessfulCreate replicaset/e2e-derive-4dff0429-openshift-default-559487f9c7 Created pod: e2e-derive-4dff0429-openshift-default-559487f9c7-22gzh 56m Normal AddedInterface pod/e2e-derive-4dff0429-openshift-default-559487f9c7-22gzh Add eth0 [10.133.0.147/23] from ovn-kubernetes 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-2ccb9fb8-856588d575 Created pod: payload-pre-processing-e2e-migrate-2ccb9fb8-856588d575-hbzhv 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-2ccb9fb8-856588d575 Error creating: pods "payload-pre-processing-e2e-migrate-2ccb9fb8-856588d575-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8: serviceaccount "payload-processing-e2e-migrate-2ccb9fb8" not found 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-2ccb9fb8 Scaled up replica set payload-processing-e2e-migrate-2ccb9fb8-6647598fff from 0 to 1 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-2ccb9fb8-6647598fff Created pod: payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-2ccb9fb8 Scaled up replica set payload-pre-processing-e2e-migrate-2ccb9fb8-856588d575 from 0 to 1 56m Warning FailedCreate replicaset/payload-processing-e2e-migrate-2ccb9fb8-6647598fff Error creating: pods "payload-processing-e2e-migrate-2ccb9fb8-6647598fff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-2ccb9fb8: serviceaccount "payload-processing-e2e-migrate-2ccb9fb8" not found 56m Normal Created pod/payload-pre-processing-e2e-migrate-2ccb9fb8-856588d575-hbzhv Container created 56m Normal Started pod/payload-pre-processing-e2e-migrate-2ccb9fb8-856588d575-hbzhv Container started 56m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-2ccb9fb8-856588d575-hbzhv Add eth0 [10.133.0.148/23] from ovn-kubernetes 56m Normal Pulled pod/payload-pre-processing-e2e-migrate-2ccb9fb8-856588d575-hbzhv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg Add eth0 [10.132.0.73/23] from ovn-kubernetes 56m Normal Pulled pod/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Created pod/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg Container created 56m Normal Started pod/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg Container started 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-c39c4123 Scaled up replica set payload-processing-e2e-ait-c39c4123-78b4dd8875 from 0 to 1 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-c39c4123-78b4dd8875 Created pod: payload-processing-e2e-ait-c39c4123-78b4dd8875-hdg55 56m Warning FailedCreate replicaset/payload-processing-e2e-ait-c39c4123-78b4dd8875 Error creating: pods "payload-processing-e2e-ait-c39c4123-78b4dd8875-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-c39c4123: serviceaccount "payload-processing-e2e-ait-c39c4123" not found 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-c39c4123-64c6db69f9 Created pod: payload-pre-processing-e2e-ait-c39c4123-64c6db69f9-77z5f 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-c39c4123-64c6db69f9 Error creating: pods "payload-pre-processing-e2e-ait-c39c4123-64c6db69f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-c39c4123: serviceaccount "payload-processing-e2e-ait-c39c4123" not found 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-c39c4123 Scaled up replica set payload-pre-processing-e2e-ait-c39c4123-64c6db69f9 from 0 to 1 56m Normal Created pod/payload-pre-processing-e2e-ait-c39c4123-64c6db69f9-77z5f Container created 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-284b8e-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 56m Normal Started pod/payload-pre-processing-e2e-ait-c39c4123-64c6db69f9-77z5f Container started 56m Normal Created pod/payload-processing-e2e-ait-c39c4123-78b4dd8875-hdg55 Container created 56m Normal Started pod/payload-processing-e2e-ait-c39c4123-78b4dd8875-hdg55 Container started 56m Normal Pulled pod/payload-pre-processing-e2e-ait-c39c4123-64c6db69f9-77z5f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal SuccessfulCreate replicaset/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf (combined from similar events): Created pod: e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-bmbxc 56m Normal Pulled pod/payload-processing-e2e-ait-c39c4123-78b4dd8875-hdg55 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-4e2b27-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 56m Normal AddedInterface pod/payload-processing-e2e-ait-c39c4123-78b4dd8875-hdg55 Add eth0 [10.133.0.151/23] from ovn-kubernetes 56m Normal AddedInterface pod/payload-pre-processing-e2e-ait-c39c4123-64c6db69f9-77z5f Add eth0 [10.133.0.150/23] from ovn-kubernetes 56m Normal ScalingReplicaSet deployment/e2e-worker-w4-4e2b27-openshift-default Scaled up replica set e2e-worker-w4-4e2b27-openshift-default-996dc4fdf from 9 to 10 56m Normal SuccessfulCreate replicaset/e2e-worker-w6-284b8e-openshift-default-77c5d9855 Created pod: e2e-worker-w6-284b8e-openshift-default-77c5d9855-qtwpp 56m Normal Pulled pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-qtwpp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal SuccessfulCreate replicaset/e2e-worker-w6-284b8e-openshift-default-77c5d9855 Created pod: e2e-worker-w6-284b8e-openshift-default-77c5d9855-djjtz 56m Normal AddedInterface pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-djjtz Add eth0 [10.133.0.152/23] from ovn-kubernetes 56m Normal SuccessfulCreate replicaset/e2e-worker-w6-284b8e-openshift-default-77c5d9855 Created pod: e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz 56m Normal AddedInterface pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-bmbxc Add eth0 [10.134.0.85/23] from ovn-kubernetes 56m Normal SuccessfulCreate replicaset/e2e-worker-w6-284b8e-openshift-default-77c5d9855 (combined from similar events): Created pod: e2e-worker-w6-284b8e-openshift-default-77c5d9855-gh4d7 56m Normal AddedInterface pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-gh4d7 Add eth0 [10.134.0.87/23] from ovn-kubernetes 56m Normal AddedInterface pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-qtwpp Add eth0 [10.134.0.86/23] from ovn-kubernetes 56m Normal Started pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-bmbxc Container started 56m Normal SuccessfulCreate replicaset/e2e-worker-w6-284b8e-openshift-default-77c5d9855 Created pod: e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd 56m Normal Pulled pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-djjtz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Pulled pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-bmbxc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-djjtz Container created 56m Normal AddedInterface pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd Add eth0 [10.132.0.74/23] from ovn-kubernetes 56m Normal Started pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-djjtz Container started 56m Normal Created pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-bmbxc Container created 56m Normal AddedInterface pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz Add eth0 [10.132.0.75/23] from ovn-kubernetes 56m Normal ScalingReplicaSet deployment/e2e-worker-w6-284b8e-openshift-default Scaled up replica set e2e-worker-w6-284b8e-openshift-default-77c5d9855 from 5 to 10 56m Normal Started pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-qtwpp Container started 56m Normal Started pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz Container started 56m Normal Pulled pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-gh4d7 Container started 56m Normal Created pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-gh4d7 Container created 56m Normal Pulled pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-gh4d7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal AddedLabel gateway/e2e-mt-c7cd2d37 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-c7cd2d37 56m Normal Started pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd Container started 56m Normal Created pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz Container created 56m Normal Created pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-qtwpp Container created 56m Normal Pulled pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd Container created 56m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 56m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-bxq4p 56m Normal ScalingReplicaSet deployment/e2e-mt-c7cd2d37-openshift-default Scaled up replica set e2e-mt-c7cd2d37-openshift-default-6458d58469 from 0 to 1 56m Normal SuccessfulCreate replicaset/e2e-mt-c7cd2d37-openshift-default-6458d58469 Created pod: e2e-mt-c7cd2d37-openshift-default-6458d58469-rkntm 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-4dff0429 Scaled up replica set payload-processing-e2e-derive-4dff0429-cc8778797 from 0 to 1 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-4dff0429 Scaled up replica set payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc from 0 to 1 56m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-4dff0429-cc8778797 Created pod: payload-processing-e2e-derive-4dff0429-cc8778797-59x9m 56m Normal AddedInterface pod/e2e-mt-c7cd2d37-openshift-default-6458d58469-rkntm Add eth0 [10.133.0.153/23] from ovn-kubernetes 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc Created pod: payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz 56m Warning FailedCreate replicaset/payload-processing-e2e-derive-4dff0429-cc8778797 Error creating: pods "payload-processing-e2e-derive-4dff0429-cc8778797-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-4dff0429: serviceaccount "payload-processing-e2e-derive-4dff0429" not found 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc Error creating: pods "payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-4dff0429: serviceaccount "payload-processing-e2e-derive-4dff0429" not found 56m Normal Pulled pod/payload-processing-e2e-derive-4dff0429-cc8778797-59x9m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-bxq4p Add eth0 [10.134.0.88/23] from ovn-kubernetes 56m Normal AddedInterface pod/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz Add eth0 [10.132.0.76/23] from ovn-kubernetes 56m Normal AddedInterface pod/payload-processing-e2e-derive-4dff0429-cc8778797-59x9m Add eth0 [10.133.0.154/23] from ovn-kubernetes 56m Normal Created pod/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz Container created 56m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-bxq4p Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 56m Normal Started pod/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz Container started 56m Normal Pulled pod/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Created pod/payload-processing-e2e-derive-4dff0429-cc8778797-59x9m Container created 56m Normal Started pod/payload-processing-e2e-derive-4dff0429-cc8778797-59x9m Container started 56m Warning Unhealthy pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-djjtz Startup probe failed: Get "http://10.133.0.152:15021/healthz/ready": dial tcp 10.133.0.152:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-djjtz Startup probe error: Get "http://10.133.0.152:15021/healthz/ready": dial tcp 10.133.0.152:15021: connect: connection refused... 56m Warning ProbeError pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-bmbxc Startup probe error: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused... 56m Warning ProbeError pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-gh4d7 Startup probe error: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused... 56m Warning ProbeError pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-qtwpp Startup probe error: Get "http://10.134.0.86:15021/healthz/ready": dial tcp 10.134.0.86:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-qtwpp Startup probe failed: Get "http://10.134.0.86:15021/healthz/ready": dial tcp 10.134.0.86:15021: connect: connection refused 56m Warning Unhealthy pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-gh4d7 Startup probe failed: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused 56m Warning Unhealthy pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-bmbxc Startup probe failed: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz Startup probe error: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd Startup probe failed: Get "http://10.132.0.74:15021/healthz/ready": dial tcp 10.132.0.74:15021: connect: connection refused 56m Warning Unhealthy pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz Startup probe failed: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd Startup probe error: Get "http://10.132.0.74:15021/healthz/ready": dial tcp 10.132.0.74:15021: connect: connection refused... 56m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-bxq4p Container started 56m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-bxq4p Container created 56m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-bxq4p Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 5.805s (5.806s including waiting). Image size: 185476477 bytes. 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-c7cd2d37 Scaled up replica set payload-processing-e2e-mt-c7cd2d37-5cb657cf9c from 0 to 1 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-c7cd2d37-5cb657cf9c Created pod: payload-processing-e2e-mt-c7cd2d37-5cb657cf9c-cbzrr 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-c7cd2d37 Scaled up replica set payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f from 0 to 1 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f Created pod: payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8 56m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-bxq4p Readiness probe error: HTTP probe failed with statuscode: 503... 56m Warning FailedCreate replicaset/payload-processing-e2e-mt-c7cd2d37-5cb657cf9c Error creating: pods "payload-processing-e2e-mt-c7cd2d37-5cb657cf9c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-c7cd2d37: serviceaccount "payload-processing-e2e-mt-c7cd2d37" not found 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f Error creating: pods "payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-c7cd2d37: serviceaccount "payload-processing-e2e-mt-c7cd2d37" not found 56m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-bxq4p Readiness probe failed: HTTP probe failed with statuscode: 503 56m Normal Pulled pod/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8 Add eth0 [10.132.0.77/23] from ovn-kubernetes 56m Normal Created pod/payload-processing-e2e-mt-c7cd2d37-5cb657cf9c-cbzrr Container created 56m Normal SuccessfulCreate replicaset/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f (combined from similar events): Created pod: e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f 56m Normal Pulled pod/payload-processing-e2e-mt-c7cd2d37-5cb657cf9c-cbzrr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Created pod/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8 Container created 56m Normal Started pod/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8 Container started 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-c2dd3b-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 56m Normal Started pod/payload-processing-e2e-mt-c7cd2d37-5cb657cf9c-cbzrr Container started 56m Normal SuccessfulCreate replicaset/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f Created pod: e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x 56m Normal ScalingReplicaSet deployment/e2e-worker-w1-c2dd3b-openshift-default Scaled up replica set e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f from 8 to 10 56m Normal AddedInterface pod/payload-processing-e2e-mt-c7cd2d37-5cb657cf9c-cbzrr Add eth0 [10.133.0.156/23] from ovn-kubernetes 56m Normal Started pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x Container started 56m Normal Created pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x Container created 56m Normal AddedInterface pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f Add eth0 [10.132.0.79/23] from ovn-kubernetes 56m Normal Started pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f Container started 56m Normal Pulled pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f Container created 56m Normal Pulled pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x Add eth0 [10.132.0.78/23] from ovn-kubernetes 56m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: cpu resource utilization (percentage of request) above target 56m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 3 56m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-m56v7 56m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-m56v7 Add eth0 [10.133.0.157/23] from ovn-kubernetes 56m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-m56v7 Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 56m Warning Unhealthy pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp Readiness probe failed: Get "http://10.132.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp Readiness probe error: Get "http://10.132.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Warning Unhealthy pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-rhzmq Readiness probe failed: Get "http://10.134.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-rhzmq Readiness probe error: Get "http://10.134.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Warning ProbeError pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq Readiness probe error: Get "http://10.132.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Warning Unhealthy pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq Readiness probe failed: Get "http://10.132.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x Startup probe error: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f Startup probe failed: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused 56m Warning Unhealthy pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x Startup probe failed: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused 56m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-m56v7 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 6.206s (6.206s including waiting). Image size: 185476477 bytes. 56m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-m56v7 Container created 56m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-m56v7 Container started 56m Warning ProbeError pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f Startup probe error: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused... 56m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-m56v7 Readiness probe error: HTTP probe failed with statuscode: 503... 56m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-m56v7 Readiness probe failed: HTTP probe failed with statuscode: 503 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-2ccb9fb8-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-2ccb9fb8-gw-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-77bfb58f8c from 1 to 4 55m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-77bfb58f8c Created pod: maas-default-gateway-openshift-default-77bfb58f8c-5dsgz 55m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-77bfb58f8c Created pod: maas-default-gateway-openshift-default-77bfb58f8c-sz8lg 55m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 55m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-77bfb58f8c Created pod: maas-default-gateway-openshift-default-77bfb58f8c-x8dsw 55m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz Container created 55m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-sz8lg Container started 55m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz Add eth0 [10.132.0.82/23] from ovn-kubernetes 55m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz Container started 55m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-sz8lg Add eth0 [10.134.0.90/23] from ovn-kubernetes 55m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw Add eth0 [10.132.0.83/23] from ovn-kubernetes 55m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw Container created 55m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw Container started 55m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-sz8lg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-sz8lg Container created 55m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw Startup probe failed: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused 55m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-sz8lg Startup probe error: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused... 55m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-sz8lg Startup probe failed: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused 55m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw Startup probe error: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused... 55m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz Startup probe error: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused... 55m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz Startup probe failed: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused 55m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw Startup probe failed: Get "http://10.132.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 55m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw Startup probe error: Get "http://10.132.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 55m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: 55m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-77bfb58f8c Created pod: maas-default-gateway-openshift-default-77bfb58f8c-7g2ql 55m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-77bfb58f8c from 4 to 5 55m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-7g2ql Add eth0 [10.133.0.158/23] from ovn-kubernetes 55m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-7g2ql Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-7g2ql Container created 55m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-7g2ql Container started 55m Normal Created pod/e2e-migrate-2ccb9fb8-gw-openshift-default-bd688b445-9vzqb Container created 55m Normal Started pod/e2e-migrate-2ccb9fb8-gw-openshift-default-bd688b445-9vzqb Container started 55m Normal Pulled pod/e2e-migrate-2ccb9fb8-gw-openshift-default-bd688b445-9vzqb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Pulled pod/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Started pod/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh Container started 55m Normal Created pod/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh Container created 55m Normal Pulled pod/e2e-derive-4dff0429-openshift-default-559487f9c7-22gzh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/e2e-derive-4dff0429-openshift-default-559487f9c7-22gzh Container created 55m Normal Started pod/e2e-derive-4dff0429-openshift-default-559487f9c7-22gzh Container started 55m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-7g2ql Startup probe error: Get "http://10.133.0.158:15021/healthz/ready": dial tcp 10.133.0.158:15021: connect: connection refused... 55m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-7g2ql Startup probe failed: Get "http://10.133.0.158:15021/healthz/ready": dial tcp 10.133.0.158:15021: connect: connection refused 55m Warning Unhealthy pod/e2e-migrate-2ccb9fb8-gw-openshift-default-bd688b445-9vzqb Startup probe failed: Get "http://10.134.0.84:15021/healthz/ready": dial tcp 10.134.0.84:15021: connect: connection refused 55m Warning ProbeError pod/e2e-migrate-2ccb9fb8-gw-openshift-default-bd688b445-9vzqb Startup probe error: Get "http://10.134.0.84:15021/healthz/ready": dial tcp 10.134.0.84:15021: connect: connection refused... 55m Warning ProbeError pod/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused 55m Warning Unhealthy pod/e2e-derive-4dff0429-openshift-default-559487f9c7-22gzh Startup probe failed: Get "http://10.133.0.147:15021/healthz/ready": dial tcp 10.133.0.147:15021: connect: connection refused 55m Warning ProbeError pod/e2e-derive-4dff0429-openshift-default-559487f9c7-22gzh Startup probe error: Get "http://10.133.0.147:15021/healthz/ready": dial tcp 10.133.0.147:15021: connect: connection refused... 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-c39c4123-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-c39c4123-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-4dff0429-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-4dff0429-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-77bfb58f8c from 5 to 8 55m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-77bfb58f8c Created pod: maas-default-gateway-openshift-default-77bfb58f8c-lq8nh 55m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-77bfb58f8c Created pod: maas-default-gateway-openshift-default-77bfb58f8c-2kgm4 55m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-77bfb58f8c Created pod: maas-default-gateway-openshift-default-77bfb58f8c-s27r2 55m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-lq8nh Add eth0 [10.134.0.92/23] from ovn-kubernetes 55m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-2kgm4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-lq8nh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-lq8nh Container created 55m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-lq8nh Container started 55m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-2kgm4 Add eth0 [10.133.0.161/23] from ovn-kubernetes 55m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 Container started 55m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 Container created 55m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-2kgm4 Container started 55m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-2kgm4 Container created 55m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 Add eth0 [10.132.0.84/23] from ovn-kubernetes 55m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-lq8nh Startup probe error: Get "http://10.134.0.92:15021/healthz/ready": dial tcp 10.134.0.92:15021: connect: connection refused... 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-lq8nh Startup probe failed: Get "http://10.134.0.92:15021/healthz/ready": dial tcp 10.134.0.92:15021: connect: connection refused 54m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 Startup probe error: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused... 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 Startup probe failed: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused 54m Normal Killing pod/payload-processing-e2e-ait-c39c4123-78b4dd8875-hdg55 Stopping container payload-processing 54m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-2kgm4 Startup probe error: Get "http://10.133.0.161:15021/healthz/ready": dial tcp 10.133.0.161:15021: connect: connection refused... 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-2kgm4 Startup probe failed: Get "http://10.133.0.161:15021/healthz/ready": dial tcp 10.133.0.161:15021: connect: connection refused 54m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-lq8nh Startup probe error: Get "http://10.134.0.92:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-lq8nh Startup probe failed: Get "http://10.134.0.92:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Normal Killing pod/payload-pre-processing-e2e-ait-c39c4123-64c6db69f9-77z5f Stopping container payload-pre-processing 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-4dff0429-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-4dff0429-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Normal Killing pod/payload-processing-e2e-migrate-2ccb9fb8-6647598fff-shhfg Stopping container payload-processing 54m Normal Killing pod/payload-pre-processing-e2e-migrate-2ccb9fb8-856588d575-hbzhv Stopping container payload-pre-processing 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-2kgm4 Startup probe failed: Get "http://10.133.0.161:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 Startup probe error: Get "http://10.132.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 Startup probe failed: Get "http://10.132.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-2kgm4 Startup probe error: Get "http://10.133.0.161:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-c39c4123-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-c39c4123-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Normal Pulled pod/e2e-mt-c7cd2d37-openshift-default-6458d58469-rkntm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-mt-c7cd2d37-openshift-default-6458d58469-rkntm Container created 54m Normal Started pod/e2e-mt-c7cd2d37-openshift-default-6458d58469-rkntm Container started 54m Normal Killing pod/payload-pre-processing-e2e-derive-4dff0429-86b7c4f8dc-x55zz Stopping container payload-pre-processing 54m Normal Killing pod/payload-processing-e2e-derive-4dff0429-cc8778797-59x9m Stopping container payload-processing 54m Warning Unhealthy pod/e2e-mt-c7cd2d37-openshift-default-6458d58469-rkntm Startup probe failed: Get "http://10.133.0.153:15021/healthz/ready": dial tcp 10.133.0.153:15021: connect: connection refused 54m Warning ProbeError pod/e2e-mt-c7cd2d37-openshift-default-6458d58469-rkntm Startup probe error: Get "http://10.133.0.153:15021/healthz/ready": dial tcp 10.133.0.153:15021: connect: connection refused... 54m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-77bfb58f8c Created pod: maas-default-gateway-openshift-default-77bfb58f8c-zw295 54m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-77bfb58f8c Created pod: maas-default-gateway-openshift-default-77bfb58f8c-n2h8k 54m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-77bfb58f8c from 8 to 10 54m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-n2h8k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-zw295 Container started 54m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-n2h8k Container started 54m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-n2h8k Container created 54m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-zw295 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-zw295 Add eth0 [10.133.0.165/23] from ovn-kubernetes 54m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-zw295 Container created 54m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-n2h8k Add eth0 [10.134.0.93/23] from ovn-kubernetes 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-2ccb9fb8-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-2ccb9fb8-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-zw295 Startup probe error: Get "http://10.133.0.165:15021/healthz/ready": dial tcp 10.133.0.165:15021: connect: connection refused... 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-zw295 Startup probe failed: Get "http://10.133.0.165:15021/healthz/ready": dial tcp 10.133.0.165:15021: connect: connection refused 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-n2h8k Startup probe failed: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused 54m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-n2h8k Startup probe error: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused... 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-c7cd2d37-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-c7cd2d37-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Normal Killing pod/e2e-ait-c39c4123-openshift-default-769746bdd5-l4nbh Stopping container istio-proxy 54m Normal AddedLabel gateway/e2e-ait-5935b530 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-5935b530 54m Normal SuccessfulCreate replicaset/e2e-ait-5935b530-openshift-default-845c9c8749 Created pod: e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp 54m Normal ScalingReplicaSet deployment/e2e-ait-5935b530-openshift-default Scaled up replica set e2e-ait-5935b530-openshift-default-845c9c8749 from 0 to 1 54m Normal Killing pod/e2e-migrate-2ccb9fb8-gw-openshift-default-bd688b445-9vzqb Stopping container istio-proxy 54m Normal AddedLabel gateway/e2e-ait-e942516f Added label istio.io/rev=openshift-gateway to gateway e2e-ait-e942516f 54m Normal AddedInterface pod/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp Add eth0 [10.132.0.85/23] from ovn-kubernetes 54m Normal SuccessfulCreate replicaset/e2e-ait-e942516f-openshift-default-7df544c549 Created pod: e2e-ait-e942516f-openshift-default-7df544c549-lq6lh 54m Normal ScalingReplicaSet deployment/e2e-ait-e942516f-openshift-default Scaled up replica set e2e-ait-e942516f-openshift-default-7df544c549 from 0 to 1 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-c7cd2d37-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-c7cd2d37-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Normal AddedInterface pod/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh Add eth0 [10.132.0.86/23] from ovn-kubernetes 54m Warning ProbeError pod/e2e-migrate-2ccb9fb8-gw-openshift-default-bd688b445-9vzqb Readiness probe error: HTTP probe failed with statuscode: 503... 54m Warning Unhealthy pod/e2e-migrate-2ccb9fb8-gw-openshift-default-bd688b445-9vzqb Readiness probe failed: HTTP probe failed with statuscode: 503 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-5935b530-5674796675 Created pod: payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-5935b530-5f947bc7f Created pod: payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg 54m Warning FailedCreate replicaset/payload-processing-e2e-ait-5935b530-5f947bc7f Error creating: pods "payload-processing-e2e-ait-5935b530-5f947bc7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-5935b530: serviceaccount "payload-processing-e2e-ait-5935b530" not found 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-5935b530 Scaled up replica set payload-processing-e2e-ait-5935b530-5f947bc7f from 0 to 1 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-5935b530-5674796675 Error creating: pods "payload-pre-processing-e2e-ait-5935b530-5674796675-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-5935b530: serviceaccount "payload-processing-e2e-ait-5935b530" not found 54m Normal Killing pod/e2e-derive-4dff0429-openshift-default-559487f9c7-22gzh Stopping container istio-proxy 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-5935b530 Scaled up replica set payload-pre-processing-e2e-ait-5935b530-5674796675 from 0 to 1 54m Normal AddedInterface pod/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg Add eth0 [10.132.0.87/23] from ovn-kubernetes 54m Normal AddedInterface pod/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg Add eth0 [10.132.0.88/23] from ovn-kubernetes 54m Normal Started pod/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg Container started 54m Normal Pulled pod/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Started pod/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg Container started 54m Normal Created pod/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg Container created 54m Normal Pulled pod/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Created pod/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg Container created 53m Warning FailedCreate replicaset/payload-processing-e2e-ait-e942516f-6cb97dbb95 Error creating: pods "payload-processing-e2e-ait-e942516f-6cb97dbb95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-e942516f: serviceaccount "payload-processing-e2e-ait-e942516f" not found 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-e942516f Scaled up replica set payload-processing-e2e-ait-e942516f-6cb97dbb95 from 0 to 1 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-e942516f-6466f98c67 Created pod: payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-e942516f-6466f98c67 Error creating: pods "payload-pre-processing-e2e-ait-e942516f-6466f98c67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-e942516f: serviceaccount "payload-processing-e2e-ait-e942516f" not found 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-e942516f Scaled up replica set payload-pre-processing-e2e-ait-e942516f-6466f98c67 from 0 to 1 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-e942516f-6cb97dbb95 Created pod: payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846 53m Normal Started pod/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846 Container started 53m Normal Pulled pod/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Created pod/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc Container created 53m Normal Started pod/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc Container started 53m Normal AddedInterface pod/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc Add eth0 [10.132.0.89/23] from ovn-kubernetes 53m Normal AddedInterface pod/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846 Add eth0 [10.132.0.90/23] from ovn-kubernetes 53m Normal Pulled pod/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Created pod/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846 Container created 53m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 4 53m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 4; reason: cpu resource utilization (percentage of request) above target 53m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-n2q6l Add eth0 [10.133.0.166/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-n2q6l 53m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-n2q6l Container started 53m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-n2q6l Container created 53m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-n2q6l Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 53m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-n2q6l Readiness probe failed: HTTP probe failed with statuscode: 503 53m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-n2q6l Readiness probe error: HTTP probe failed with statuscode: 503... 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-e942516f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-e942516f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Normal ScalingReplicaSet deployment/e2e-worker-w2-4d9ea0-openshift-default Scaled down replica set e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc from 9 to 6 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-4d9ea0-openshift-default New size: 6; reason: All metrics below target 53m Normal Killing pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j Stopping container istio-proxy 53m Normal Killing pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-wjhwc Stopping container istio-proxy 53m Normal Killing pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-wv79b Stopping container istio-proxy 53m Normal SuccessfulDelete replicaset/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc Deleted pod: e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-kqz2j 53m Normal SuccessfulDelete replicaset/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc Deleted pod: e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-wjhwc 53m Normal SuccessfulDelete replicaset/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc Deleted pod: e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-wv79b 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-4d9ea0-openshift-default New size: 5; reason: All metrics below target 52m Normal SuccessfulDelete replicaset/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc Deleted pod: e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-tglx5 52m Normal Killing pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-tglx5 Stopping container istio-proxy 52m Normal ScalingReplicaSet deployment/e2e-worker-w2-4d9ea0-openshift-default Scaled down replica set e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc from 6 to 5 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-ef6b43-openshift-default New size: 4; reason: All metrics below target 52m Normal Killing pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-gvmnx Stopping container istio-proxy 52m Normal Killing pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-zpmjw Stopping container istio-proxy 52m Normal SuccessfulDelete replicaset/e2e-worker-w5-ef6b43-openshift-default-679658c459 Deleted pod: e2e-worker-w5-ef6b43-openshift-default-679658c459-zpmjw 52m Normal Killing pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-jqdtb Stopping container istio-proxy 52m Normal SuccessfulDelete replicaset/e2e-worker-w5-ef6b43-openshift-default-679658c459 Deleted pod: e2e-worker-w5-ef6b43-openshift-default-679658c459-gvmnx 52m Normal ScalingReplicaSet deployment/e2e-worker-w5-ef6b43-openshift-default Scaled down replica set e2e-worker-w5-ef6b43-openshift-default-679658c459 from 7 to 4 52m Normal SuccessfulDelete replicaset/e2e-worker-w5-ef6b43-openshift-default-679658c459 Deleted pod: e2e-worker-w5-ef6b43-openshift-default-679658c459-jqdtb 52m Warning ProbeError pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-zpmjw Readiness probe error: HTTP probe failed with statuscode: 503... 52m Warning Unhealthy pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-zpmjw Readiness probe failed: HTTP probe failed with statuscode: 503 52m Normal ScalingReplicaSet deployment/e2e-mt-6b983e02-openshift-default Scaled up replica set e2e-mt-6b983e02-openshift-default-f8946f76f from 0 to 1 52m Normal AddedLabel gateway/e2e-mt-6b983e02 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-6b983e02 52m Normal SuccessfulCreate replicaset/e2e-mt-6b983e02-openshift-default-f8946f76f Created pod: e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz 52m Normal AddedInterface pod/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz Add eth0 [10.132.0.92/23] from ovn-kubernetes 52m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669 Created pod: e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v 52m Normal AddedLabel gateway/e2e-shared-a-w2-f4a242 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-f4a242 52m Normal Killing pod/payload-processing-e2e-mt-c7cd2d37-5cb657cf9c-cbzrr Stopping container payload-processing 52m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-f4a242-openshift-default Scaled up replica set e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669 from 0 to 1 52m Normal Killing pod/payload-pre-processing-e2e-mt-c7cd2d37-5cf7fbb97f-f8qg8 Stopping container payload-pre-processing 52m Normal AddedInterface pod/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v Add eth0 [10.132.0.93/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh Container created 52m Warning FailedCreate replicaset/payload-processing-e2e-mt-6b983e02-66b998455c Error creating: pods "payload-processing-e2e-mt-6b983e02-66b998455c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-6b983e02: serviceaccount "payload-processing-e2e-mt-6b983e02" not found 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-6b983e02-58d76c55bc Error creating: pods "payload-pre-processing-e2e-mt-6b983e02-58d76c55bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-6b983e02: serviceaccount "payload-processing-e2e-mt-6b983e02" not found 52m Normal Started pod/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh Container started 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-6b983e02 Scaled up replica set payload-processing-e2e-mt-6b983e02-66b998455c from 0 to 1 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-6b983e02 Scaled up replica set payload-pre-processing-e2e-mt-6b983e02-58d76c55bc from 0 to 1 52m Normal Started pod/payload-pre-processing-e2e-mt-6b983e02-58d76c55bc-jqfd5 Container started 52m Normal Pulled pod/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp Container created 52m Normal Started pod/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp Container started 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-6b983e02-58d76c55bc Created pod: payload-pre-processing-e2e-mt-6b983e02-58d76c55bc-jqfd5 52m Normal Created pod/payload-pre-processing-e2e-mt-6b983e02-58d76c55bc-jqfd5 Container created 52m Normal Pulled pod/payload-pre-processing-e2e-mt-6b983e02-58d76c55bc-jqfd5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-6b983e02-66b998455c Created pod: payload-processing-e2e-mt-6b983e02-66b998455c-swpb9 52m Normal AddedInterface pod/payload-pre-processing-e2e-mt-6b983e02-58d76c55bc-jqfd5 Add eth0 [10.134.0.95/23] from ovn-kubernetes 52m Normal AddedInterface pod/payload-processing-e2e-mt-6b983e02-66b998455c-swpb9 Add eth0 [10.134.0.96/23] from ovn-kubernetes 52m Normal Pulled pod/payload-processing-e2e-mt-6b983e02-66b998455c-swpb9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Created pod/payload-processing-e2e-mt-6b983e02-66b998455c-swpb9 Container created 52m Normal Started pod/payload-processing-e2e-mt-6b983e02-66b998455c-swpb9 Container started 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-f4a242 Scaled up replica set payload-processing-e2e-shared-a-w2-f4a242-66587fdf75 from 0 to 1 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-f4a242-6cd88bd755 Error creating: pods "payload-pre-processing-e2e-shared-a-w2-f4a242-6cd88bd755-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-f4a242: serviceaccount "payload-processing-e2e-shared-a-w2-f4a242" not found 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-f4a242-6cd88bd755 Created pod: payload-pre-processing-e2e-shared-a-w2-f4a242-6cd88bd755-lkvsn 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-f4a242 Scaled up replica set payload-pre-processing-e2e-shared-a-w2-f4a242-6cd88bd755 from 0 to 1 52m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-f4a242-66587fdf75 Error creating: pods "payload-processing-e2e-shared-a-w2-f4a242-66587fdf75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-f4a242: serviceaccount "payload-processing-e2e-shared-a-w2-f4a242" not found 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-f4a242-66587fdf75 Created pod: payload-processing-e2e-shared-a-w2-f4a242-66587fdf75-qghj6 52m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-f4a242-6cd88bd755-lkvsn Container created 52m Normal Started pod/payload-processing-e2e-shared-a-w2-f4a242-66587fdf75-qghj6 Container started 52m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-f4a242-6cd88bd755-lkvsn Add eth0 [10.133.0.168/23] from ovn-kubernetes 52m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-f4a242-6cd88bd755-lkvsn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Created pod/payload-processing-e2e-shared-a-w2-f4a242-66587fdf75-qghj6 Container created 52m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-f4a242-6cd88bd755-lkvsn Container started 52m Normal Pulled pod/payload-processing-e2e-shared-a-w2-f4a242-66587fdf75-qghj6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-f4a242-66587fdf75-qghj6 Add eth0 [10.133.0.169/23] from ovn-kubernetes 52m Warning ProbeError pod/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh Startup probe error: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh Startup probe failed: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused 52m Warning ProbeError pod/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp Startup probe error: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp Startup probe failed: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-5935b530-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-5935b530-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-e942516f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-e942516f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-5935b530-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-5935b530-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Normal Killing pod/payload-processing-e2e-ait-5935b530-5f947bc7f-4gthg Stopping container payload-processing 52m Normal Killing pod/payload-pre-processing-e2e-ait-5935b530-5674796675-g5pxg Stopping container payload-pre-processing 52m Normal AddedLabel gateway/e2e-shared-a-w3-06366c Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-06366c 52m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-06366c-openshift-default Scaled up replica set e2e-shared-a-w3-06366c-openshift-default-6cd8c77f65 from 0 to 1 52m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-06366c-openshift-default-6cd8c77f65 Created pod: e2e-shared-a-w3-06366c-openshift-default-6cd8c77f65-8swpg 52m Normal AddedInterface pod/e2e-shared-a-w3-06366c-openshift-default-6cd8c77f65-8swpg Add eth0 [10.133.0.172/23] from ovn-kubernetes 51m Normal Killing pod/payload-processing-e2e-ait-e942516f-6cb97dbb95-kd846 Stopping container payload-processing 51m Normal Killing pod/payload-pre-processing-e2e-ait-e942516f-6466f98c67-zrgjc Stopping container payload-pre-processing 51m Normal Killing pod/e2e-mt-c7cd2d37-openshift-default-6458d58469-rkntm Stopping container istio-proxy 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w3-06366c Scaled up replica set payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f from 0 to 1 51m Normal AddedLabel gateway/e2e-api-a-b7107c Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-b7107c 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f Created pod: payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh 51m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f Error creating: pods "payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-06366c: serviceaccount "payload-processing-e2e-shared-a-w3-06366c" not found 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c Error creating: pods "payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-06366c: serviceaccount "payload-processing-e2e-shared-a-w3-06366c" not found 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w3-06366c Scaled up replica set payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c from 0 to 1 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c Created pod: payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s 51m Normal AddedInterface pod/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh Add eth0 [10.132.0.95/23] from ovn-kubernetes 51m Normal ScalingReplicaSet deployment/e2e-api-a-b7107c-openshift-default Scaled up replica set e2e-api-a-b7107c-openshift-default-778f76c545 from 0 to 1 51m Normal SuccessfulCreate replicaset/e2e-api-a-b7107c-openshift-default-778f76c545 Created pod: e2e-api-a-b7107c-openshift-default-778f76c545-8hgqg 51m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s Add eth0 [10.132.0.94/23] from ovn-kubernetes 51m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-api-a-b7107c-openshift-default-778f76c545-8hgqg Add eth0 [10.133.0.173/23] from ovn-kubernetes 51m Normal Started pod/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh Container started 51m Normal Pulled pod/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Created pod/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh Container created 51m Normal Created pod/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s Container created 51m Normal Started pod/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s Container started 51m Normal SuccessfulCreate replicaset/e2e-api-a-d3b375-openshift-default-5b854d5c65 Created pod: e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76 51m Normal ScalingReplicaSet deployment/e2e-api-a-d3b375-openshift-default Scaled up replica set e2e-api-a-d3b375-openshift-default-5b854d5c65 from 0 to 1 51m Normal AddedLabel gateway/e2e-api-a-d3b375 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-d3b375 51m Normal AddedInterface pod/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76 Add eth0 [10.132.0.96/23] from ovn-kubernetes 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-b7107c-d874d78b7 Error creating: pods "payload-pre-processing-e2e-api-a-b7107c-d874d78b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-b7107c: serviceaccount "payload-processing-e2e-api-a-b7107c" not found 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-b7107c Scaled up replica set payload-pre-processing-e2e-api-a-b7107c-d874d78b7 from 0 to 1 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-b7107c Scaled up replica set payload-processing-e2e-api-a-b7107c-65f4b7cf5 from 0 to 1 51m Warning FailedCreate replicaset/payload-processing-e2e-api-a-b7107c-65f4b7cf5 Error creating: pods "payload-processing-e2e-api-a-b7107c-65f4b7cf5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-b7107c: serviceaccount "payload-processing-e2e-api-a-b7107c" not found 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-b7107c-65f4b7cf5 Created pod: payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-b7107c-d874d78b7 Created pod: payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv 51m Normal Started pod/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb Container started 51m Normal Started pod/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv Container started 51m Normal Created pod/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv Container created 51m Normal Pulled pod/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv Add eth0 [10.132.0.97/23] from ovn-kubernetes 51m Normal Pulled pod/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Created pod/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb Container created 51m Normal AddedInterface pod/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb Add eth0 [10.132.0.98/23] from ovn-kubernetes 51m Normal Killing pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-qrspz Stopping container istio-proxy 51m Normal Killing pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-lwg4n Stopping container istio-proxy 51m Normal Killing pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5khvm Stopping container istio-proxy 51m Normal SuccessfulDelete replicaset/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc Deleted pod: e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-qrspz 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-dd6574-openshift-default New size: 7; reason: All metrics below target 51m Normal ScalingReplicaSet deployment/e2e-worker-w0-dd6574-openshift-default Scaled down replica set e2e-worker-w0-dd6574-openshift-default-5d877f7ccc from 10 to 7 51m Normal SuccessfulDelete replicaset/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc Deleted pod: e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-lwg4n 51m Normal SuccessfulDelete replicaset/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc Deleted pod: e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5khvm 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-d3b375 Scaled up replica set payload-processing-e2e-api-a-d3b375-bc5458c5 from 0 to 1 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-d3b375-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-d3b375 Scaled up replica set payload-pre-processing-e2e-api-a-d3b375-78cfb749f from 0 to 1 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-d3b375-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-d3b375-bc5458c5 Created pod: payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd 51m Warning FailedCreate replicaset/payload-processing-e2e-api-a-d3b375-bc5458c5 Error creating: pods "payload-processing-e2e-api-a-d3b375-bc5458c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-d3b375: serviceaccount "payload-processing-e2e-api-a-d3b375" not found 51m Warning ProbeError pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5khvm Readiness probe error: HTTP probe failed with statuscode: 503... 51m Warning Unhealthy pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5khvm Readiness probe failed: HTTP probe failed with statuscode: 503 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-d3b375-78cfb749f Created pod: payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-d3b375-78cfb749f Error creating: pods "payload-pre-processing-e2e-api-a-d3b375-78cfb749f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-d3b375: serviceaccount "payload-processing-e2e-api-a-d3b375" not found 51m Normal Pulled pod/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Started pod/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db Container started 51m Normal Created pod/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db Container created 51m Normal Pulled pod/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db Add eth0 [10.132.0.99/23] from ovn-kubernetes 51m Normal AddedInterface pod/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd Add eth0 [10.132.0.100/23] from ovn-kubernetes 51m Warning ProbeError pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-lwg4n Readiness probe error: HTTP probe failed with statuscode: 503... 51m Warning Unhealthy pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-lwg4n Readiness probe failed: HTTP probe failed with statuscode: 503 51m Normal Created pod/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd Container created 51m Normal Started pod/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd Container started 51m Normal ScalingReplicaSet deployment/e2e-worker-w1-c2dd3b-openshift-default Scaled down replica set e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f from 10 to 6 51m Normal SuccessfulDelete replicaset/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f Deleted pod: e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb 51m Normal SuccessfulDelete replicaset/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f Deleted pod: e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf 51m Normal Killing pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-x4jxb Stopping container istio-proxy 51m Normal SuccessfulDelete replicaset/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f Deleted pod: e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f 51m Normal Killing pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-czd4f Stopping container istio-proxy 51m Normal SuccessfulDelete replicaset/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f Deleted pod: e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x 51m Normal Killing pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-mvg8x Stopping container istio-proxy 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-c2dd3b-openshift-default New size: 6; reason: All metrics below target 51m Normal Killing pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf Stopping container istio-proxy 51m Normal ScalingReplicaSet deployment/e2e-worker-w0-dd6574-openshift-default Scaled down replica set e2e-worker-w0-dd6574-openshift-default-5d877f7ccc from 7 to 6 51m Normal SuccessfulDelete replicaset/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc Deleted pod: e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-dd6574-openshift-default New size: 6; reason: All metrics below target 51m Normal Killing pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-595t9 Stopping container istio-proxy 51m Warning FailedCreate replicaset/e2e-ait-5935b530-openshift-default-845c9c8749 Error creating: pods "e2e-ait-5935b530-openshift-default-845c9c8749-" is forbidden: error looking up service account openshift-ingress/e2e-ait-5935b530-openshift-default: serviceaccount "e2e-ait-5935b530-openshift-default" not found 51m Normal Killing pod/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp Stopping container istio-proxy 51m Normal ScalingReplicaSet deployment/e2e-ait-5935b530-openshift-default Scaled up replica set e2e-ait-5935b530-openshift-default-845c9c8749 from 0 to 1 51m Normal AddedLabel gateway/e2e-shared-a-w6-9f77b7 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w6-9f77b7 51m Warning Unhealthy pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf Readiness probe failed: HTTP probe failed with statuscode: 503 51m Normal AddedInterface pod/e2e-shared-a-w6-9f77b7-openshift-default-6c9975d87-xngcd Add eth0 [10.134.0.99/23] from ovn-kubernetes 51m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-9f77b7-openshift-default-6c9975d87 Created pod: e2e-shared-a-w6-9f77b7-openshift-default-6c9975d87-xngcd 51m Warning ProbeError pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-q7htf Readiness probe error: HTTP probe failed with statuscode: 503... 51m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-9f77b7-openshift-default Scaled up replica set e2e-shared-a-w6-9f77b7-openshift-default-6c9975d87 from 0 to 1 51m Normal Killing pod/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh Stopping container istio-proxy 51m Warning Unhealthy pod/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh Readiness probe failed: HTTP probe failed with statuscode: 503 51m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-d4d045-openshift-default-d6546474c Created pod: e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz 51m Warning ProbeError pod/e2e-ait-e942516f-openshift-default-7df544c549-lq6lh Readiness probe error: HTTP probe failed with statuscode: 503... 51m Normal AddedLabel gateway/e2e-shared-a-w1-d4d045 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w1-d4d045 51m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-d4d045-openshift-default Scaled up replica set e2e-shared-a-w1-d4d045-openshift-default-d6546474c from 0 to 1 51m Warning Unhealthy pod/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp Readiness probe failed: HTTP probe failed with statuscode: 503 51m Normal AddedInterface pod/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz Add eth0 [10.132.0.101/23] from ovn-kubernetes 51m Warning ProbeError pod/e2e-ait-5935b530-openshift-default-845c9c8749-rnnpp Readiness probe error: HTTP probe failed with statuscode: 503... 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w6-9f77b7 Scaled up replica set payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5 from 0 to 1 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5 Created pod: payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w6-9f77b7 Scaled up replica set payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c from 0 to 1 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5 Error creating: pods "payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7: serviceaccount "payload-processing-e2e-shared-a-w6-9f77b7" not found 51m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c Error creating: pods "payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-9f77b7: serviceaccount "payload-processing-e2e-shared-a-w6-9f77b7" not found 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c Created pod: payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn 51m Normal Pulled pod/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Started pod/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn Container started 51m Normal Created pod/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn Container created 51m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf Add eth0 [10.132.0.102/23] from ovn-kubernetes 51m Normal AddedInterface pod/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn Add eth0 [10.132.0.103/23] from ovn-kubernetes 51m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Created pod/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf Container created 51m Normal Started pod/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf Container started 51m Normal Killing pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-tvcl2 Stopping container istio-proxy 51m Normal SuccessfulDelete replicaset/e2e-worker-w5-ef6b43-openshift-default-679658c459 Deleted pod: e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht 51m Normal Killing pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht Stopping container istio-proxy 51m Normal Killing pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-m464d Stopping container istio-proxy 51m Normal SuccessfulDelete replicaset/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c Deleted pod: e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-sn5cj 51m Normal Killing pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-sn5cj Stopping container istio-proxy 51m Normal SuccessfulDelete replicaset/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c Deleted pod: e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-tvcl2 51m Normal ScalingReplicaSet deployment/e2e-worker-w5-ef6b43-openshift-default Scaled down replica set e2e-worker-w5-ef6b43-openshift-default-679658c459 from 4 to 3 51m Normal SuccessfulDelete replicaset/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c Deleted pod: e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-m464d 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-ef6b43-openshift-default New size: 3; reason: All metrics below target 51m Normal ScalingReplicaSet deployment/e2e-worker-w3-9a6cf2-openshift-default Scaled down replica set e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c from 10 to 7 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-9a6cf2-openshift-default New size: 7; reason: All metrics below target 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-4e2b27-openshift-default New size: 6; reason: All metrics below target 51m Normal ScalingReplicaSet deployment/e2e-worker-w6-284b8e-openshift-default Scaled down replica set e2e-worker-w6-284b8e-openshift-default-77c5d9855 from 10 to 9 51m Normal SuccessfulDelete replicaset/e2e-worker-w6-284b8e-openshift-default-77c5d9855 Deleted pod: e2e-worker-w6-284b8e-openshift-default-77c5d9855-252cj 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-284b8e-openshift-default New size: 9; reason: All metrics below target 51m Normal SuccessfulDelete replicaset/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf Deleted pod: e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-bmbxc 51m Normal SuccessfulDelete replicaset/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf Deleted pod: e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-285rr 51m Normal SuccessfulDelete replicaset/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf Deleted pod: e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-25ttn 51m Normal Killing pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-bmbxc Stopping container istio-proxy 51m Normal Killing pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-4ttqx Stopping container istio-proxy 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w1-d4d045 Scaled up replica set payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c from 0 to 1 51m Normal Killing pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-252cj Stopping container istio-proxy 51m Normal ScalingReplicaSet deployment/e2e-worker-w4-4e2b27-openshift-default Scaled down replica set e2e-worker-w4-4e2b27-openshift-default-996dc4fdf from 10 to 6 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w1-d4d045 Scaled up replica set payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685 from 0 to 1 51m Normal SuccessfulDelete replicaset/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf Deleted pod: e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-4ttqx 51m Normal Killing pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-285rr Stopping container istio-proxy 51m Normal Killing pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-25ttn Stopping container istio-proxy 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c Created pod: payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c Error creating: pods "payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045: serviceaccount "payload-processing-e2e-shared-a-w1-d4d045" not found 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685 Created pod: payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t 51m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685 Error creating: pods "payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-d4d045: serviceaccount "payload-processing-e2e-shared-a-w1-d4d045" not found 51m Normal AddedInterface pod/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t Add eth0 [10.132.0.105/23] from ovn-kubernetes 51m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Created pod/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r Container created 51m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r Add eth0 [10.132.0.104/23] from ovn-kubernetes 51m Normal Started pod/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t Container started 51m Normal Created pod/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t Container created 51m Normal Pulled pod/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Started pod/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r Container started 51m Warning Unhealthy pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-4ttqx Readiness probe failed: HTTP probe failed with statuscode: 503 51m Warning ProbeError pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-4ttqx Readiness probe error: HTTP probe failed with statuscode: 503... 51m Warning ProbeError pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-tvcl2 Readiness probe error: HTTP probe failed with statuscode: 503... 51m Warning ProbeError pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht Readiness probe error: HTTP probe failed with statuscode: 503... 51m Warning Unhealthy pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-n69ht Readiness probe failed: HTTP probe failed with statuscode: 503 51m Warning ProbeError pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-sn5cj Readiness probe error: HTTP probe failed with statuscode: 503... 51m Warning Unhealthy pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-tvcl2 Readiness probe failed: HTTP probe failed with statuscode: 503 51m Warning Unhealthy pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-sn5cj Readiness probe failed: HTTP probe failed with statuscode: 503 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-d4d045-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-d4d045-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Normal Killing pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc Stopping container istio-proxy 50m Normal SuccessfulDelete replicaset/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c Deleted pod: e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-zqjfc 50m Normal ScalingReplicaSet deployment/e2e-worker-w3-9a6cf2-openshift-default Scaled down replica set e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c from 7 to 6 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-9a6cf2-openshift-default New size: 6; reason: All metrics below target 50m Normal Killing pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd Stopping container istio-proxy 50m Normal ScalingReplicaSet deployment/e2e-worker-w6-284b8e-openshift-default Scaled down replica set e2e-worker-w6-284b8e-openshift-default-77c5d9855 from 9 to 6 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-284b8e-openshift-default New size: 6; reason: All metrics below target 50m Normal Killing pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz Stopping container istio-proxy 50m Normal Killing pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq Stopping container istio-proxy 50m Normal SuccessfulDelete replicaset/e2e-worker-w6-284b8e-openshift-default-77c5d9855 Deleted pod: e2e-worker-w6-284b8e-openshift-default-77c5d9855-j8sqz 50m Normal SuccessfulDelete replicaset/e2e-worker-w6-284b8e-openshift-default-77c5d9855 Deleted pod: e2e-worker-w6-284b8e-openshift-default-77c5d9855-clnpd 50m Normal SuccessfulDelete replicaset/e2e-worker-w6-284b8e-openshift-default-77c5d9855 Deleted pod: e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq 50m Normal Pulled pod/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v Container started 50m Normal Created pod/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v Container created 50m Warning ProbeError pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq Readiness probe error: HTTP probe failed with statuscode: 503... 50m Warning Unhealthy pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-jfkdq Readiness probe failed: HTTP probe failed with statuscode: 503 50m Normal Pulled pod/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz Container started 50m Normal Created pod/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz Container created 50m Warning ProbeError pod/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v Startup probe error: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v Startup probe failed: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz Startup probe failed: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused 50m Warning ProbeError pod/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz Startup probe error: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused... 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-6b983e02-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-6b983e02-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-f4a242-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-f4a242-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Normal AddedLabel gateway/e2e-shared-b-w2-f22129 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-f22129 50m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-f22129-openshift-default Scaled up replica set e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f from 0 to 1 50m Normal AddedInterface pod/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg Add eth0 [10.132.0.108/23] from ovn-kubernetes 50m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f Created pod: e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-f22129 Scaled up replica set payload-processing-e2e-shared-b-w2-f22129-75dfd8d647 from 0 to 1 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf Created pod: payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh 50m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647 Error creating: pods "payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-f22129: serviceaccount "payload-processing-e2e-shared-b-w2-f22129" not found 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf Error creating: pods "payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-f22129: serviceaccount "payload-processing-e2e-shared-b-w2-f22129" not found 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-f22129 Scaled up replica set payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf from 0 to 1 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647 Created pod: payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq 50m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh Container started 50m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh Add eth0 [10.132.0.109/23] from ovn-kubernetes 50m Normal Pulled pod/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-6b983e02-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Normal Created pod/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq Container created 50m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq Add eth0 [10.132.0.110/23] from ovn-kubernetes 50m Normal Started pod/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq Container started 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-6b983e02-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh Container created 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-f4a242-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-f4a242-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning ProbeError pod/e2e-shared-a-w3-06366c-openshift-default-6cd8c77f65-8swpg Readiness probe error: Get "http://10.133.0.172:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-shared-a-w3-06366c-openshift-default-6cd8c77f65-8swpg Readiness probe failed: Get "http://10.133.0.172:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Normal Pulled pod/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76 Container created 50m Normal Started pod/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76 Container started 50m Normal Created pod/e2e-shared-a-w3-06366c-openshift-default-6cd8c77f65-8swpg Container created 50m Normal Pulled pod/e2e-api-a-b7107c-openshift-default-778f76c545-8hgqg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Pulled pod/e2e-shared-a-w3-06366c-openshift-default-6cd8c77f65-8swpg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-api-a-b7107c-openshift-default-778f76c545-8hgqg Container created 50m Normal Started pod/e2e-api-a-b7107c-openshift-default-778f76c545-8hgqg Container started 50m Normal Started pod/e2e-shared-a-w3-06366c-openshift-default-6cd8c77f65-8swpg Container started 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-b7107c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-b7107c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning Unhealthy pod/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76 Startup probe failed: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused 50m Warning ProbeError pod/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76 Startup probe error: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-shared-a-w3-06366c-openshift-default-6cd8c77f65-8swpg Startup probe failed: Get "http://10.133.0.172:15021/healthz/ready": dial tcp 10.133.0.172:15021: connect: connection refused 50m Warning ProbeError pod/e2e-shared-a-w3-06366c-openshift-default-6cd8c77f65-8swpg Startup probe error: Get "http://10.133.0.172:15021/healthz/ready": dial tcp 10.133.0.172:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-api-a-b7107c-openshift-default-778f76c545-8hgqg Startup probe error: Get "http://10.133.0.173:15021/healthz/ready": dial tcp 10.133.0.173:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-api-a-b7107c-openshift-default-778f76c545-8hgqg Startup probe failed: Get "http://10.133.0.173:15021/healthz/ready": dial tcp 10.133.0.173:15021: connect: connection refused 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-06366c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-06366c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-d3b375-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-d3b375-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Normal AddedLabel gateway/e2e-shared-b-w3-f13b18 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-f13b18 49m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf Created pod: e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7 49m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-f13b18-openshift-default Scaled up replica set e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf from 0 to 1 49m Normal AddedInterface pod/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7 Add eth0 [10.132.0.111/23] from ovn-kubernetes 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w3-f13b18 Scaled up replica set payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9 from 0 to 1 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-b7107c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9 Error creating: pods "payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18: serviceaccount "payload-processing-e2e-shared-b-w3-f13b18" not found 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w3-f13b18 Scaled up replica set payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf from 0 to 1 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-b7107c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9 Created pod: payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf Error creating: pods "payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-f13b18: serviceaccount "payload-processing-e2e-shared-b-w3-f13b18" not found 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf Created pod: payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms 49m Normal Started pod/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms Container started 49m Normal Created pod/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms Container created 49m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms Add eth0 [10.132.0.112/23] from ovn-kubernetes 49m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt Add eth0 [10.132.0.113/23] from ovn-kubernetes 49m Normal Created pod/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt Container created 49m Normal Started pod/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt Container started 49m Normal Pulled pod/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-06366c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-06366c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning ProbeError pod/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz Readiness probe error: Get "http://10.132.0.101:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz Readiness probe failed: Get "http://10.132.0.101:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Normal Pulled pod/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz Container started 49m Normal Created pod/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz Container created 49m Normal AddedLabel gateway/e2e-api-b-32de96 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-32de96 49m Normal SuccessfulCreate replicaset/e2e-api-b-32de96-openshift-default-65db64bf97 Created pod: e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf 49m Normal ScalingReplicaSet deployment/e2e-api-b-32de96-openshift-default Scaled up replica set e2e-api-b-32de96-openshift-default-65db64bf97 from 0 to 1 49m Normal AddedInterface pod/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf Add eth0 [10.132.0.114/23] from ovn-kubernetes 49m Warning Unhealthy pod/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz Startup probe failed: Get "http://10.132.0.101:15021/healthz/ready": dial tcp 10.132.0.101:15021: connect: connection refused 49m Warning ProbeError pod/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz Startup probe error: Get "http://10.132.0.101:15021/healthz/ready": dial tcp 10.132.0.101:15021: connect: connection refused... 49m Normal AddedLabel gateway/e2e-api-b-a1bf7d Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-a1bf7d 49m Normal ScalingReplicaSet deployment/e2e-api-b-a1bf7d-openshift-default Scaled up replica set e2e-api-b-a1bf7d-openshift-default-7cd996948f from 0 to 1 49m Normal AddedInterface pod/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq Add eth0 [10.132.0.115/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/e2e-api-b-a1bf7d-openshift-default-7cd996948f Created pod: e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-32de96-5dc88f798c Created pod: payload-pre-processing-e2e-api-b-32de96-5dc88f798c-hxh64 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-32de96 Scaled up replica set payload-pre-processing-e2e-api-b-32de96-5dc88f798c from 0 to 1 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-32de96 Scaled up replica set payload-processing-e2e-api-b-32de96-5ff6bbb5ff from 0 to 1 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-32de96-5ff6bbb5ff Created pod: payload-processing-e2e-api-b-32de96-5ff6bbb5ff-rqwzc 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-32de96-5dc88f798c Error creating: pods "payload-pre-processing-e2e-api-b-32de96-5dc88f798c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-32de96: serviceaccount "payload-processing-e2e-api-b-32de96" not found 49m Warning FailedCreate replicaset/payload-processing-e2e-api-b-32de96-5ff6bbb5ff Error creating: pods "payload-processing-e2e-api-b-32de96-5ff6bbb5ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-32de96: serviceaccount "payload-processing-e2e-api-b-32de96" not found 49m Normal Started pod/payload-pre-processing-e2e-api-b-32de96-5dc88f798c-hxh64 Container started 49m Normal Created pod/payload-pre-processing-e2e-api-b-32de96-5dc88f798c-hxh64 Container created 49m Normal Started pod/payload-processing-e2e-api-b-32de96-5ff6bbb5ff-rqwzc Container started 49m Normal AddedInterface pod/payload-processing-e2e-api-b-32de96-5ff6bbb5ff-rqwzc Add eth0 [10.134.0.105/23] from ovn-kubernetes 49m Normal Pulled pod/payload-pre-processing-e2e-api-b-32de96-5dc88f798c-hxh64 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Pulled pod/payload-processing-e2e-api-b-32de96-5ff6bbb5ff-rqwzc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Created pod/payload-processing-e2e-api-b-32de96-5ff6bbb5ff-rqwzc Container created 49m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-32de96-5dc88f798c-hxh64 Add eth0 [10.134.0.104/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-a1bf7d-6955977c5 Created pod: payload-pre-processing-e2e-api-b-a1bf7d-6955977c5-wvmp9 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c Created pod: payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk 49m Warning FailedCreate replicaset/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c Error creating: pods "payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-a1bf7d: serviceaccount "payload-processing-e2e-api-b-a1bf7d" not found 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-a1bf7d-6955977c5 Error creating: pods "payload-pre-processing-e2e-api-b-a1bf7d-6955977c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-a1bf7d: serviceaccount "payload-processing-e2e-api-b-a1bf7d" not found 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-a1bf7d Scaled up replica set payload-pre-processing-e2e-api-b-a1bf7d-6955977c5 from 0 to 1 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-a1bf7d Scaled up replica set payload-processing-e2e-api-b-a1bf7d-5499dfdd6c from 0 to 1 49m Normal Pulled pod/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-a1bf7d-6955977c5-wvmp9 Add eth0 [10.133.0.174/23] from ovn-kubernetes 49m Normal Pulled pod/payload-pre-processing-e2e-api-b-a1bf7d-6955977c5-wvmp9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk Add eth0 [10.132.0.116/23] from ovn-kubernetes 49m Normal Created pod/payload-pre-processing-e2e-api-b-a1bf7d-6955977c5-wvmp9 Container created 49m Normal Started pod/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk Container started 49m Normal Created pod/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk Container created 49m Normal Started pod/payload-pre-processing-e2e-api-b-a1bf7d-6955977c5-wvmp9 Container started 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-d4d045-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-d4d045-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Normal ScalingReplicaSet deployment/e2e-worker-w2-4d9ea0-openshift-default Scaled down replica set e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc from 5 to 4 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-4d9ea0-openshift-default New size: 4; reason: All metrics below target 48m Normal SuccessfulDelete replicaset/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc Deleted pod: e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-jltps 48m Normal Killing pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-jltps Stopping container istio-proxy 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-32de96-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-32de96-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning ProbeError pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-jltps Readiness probe error: HTTP probe failed with statuscode: 503... 48m Warning Unhealthy pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-jltps Readiness probe failed: HTTP probe failed with statuscode: 503 48m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-77bfb58f8c Deleted pod: maas-default-gateway-openshift-default-77bfb58f8c-7g2ql 48m Normal Killing pod/maas-default-gateway-openshift-default-77bfb58f8c-2kgm4 Stopping container istio-proxy 48m Normal Killing pod/maas-default-gateway-openshift-default-77bfb58f8c-7g2ql Stopping container istio-proxy 48m Normal Started pod/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg Container started 48m Normal Pulled pod/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg Container created 48m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-77bfb58f8c Deleted pod: maas-default-gateway-openshift-default-77bfb58f8c-2kgm4 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-f22129-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-f22129-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Normal AddedLabel gateway/e2e-shared-b-w6-5451a5 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w6-5451a5 48m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-5451a5-openshift-default Scaled up replica set e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47 from 0 to 1 48m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47 Created pod: e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc 48m Normal AddedInterface pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc Add eth0 [10.132.0.117/23] from ovn-kubernetes 48m Warning ProbeError pod/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg Startup probe error: Get "http://10.132.0.108:15021/healthz/ready": dial tcp 10.132.0.108:15021: connect: connection refused... 48m Normal Pulled pod/e2e-shared-a-w6-9f77b7-openshift-default-6c9975d87-xngcd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Warning Unhealthy pod/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg Startup probe failed: Get "http://10.132.0.108:15021/healthz/ready": dial tcp 10.132.0.108:15021: connect: connection refused 48m Normal Started pod/e2e-shared-a-w6-9f77b7-openshift-default-6c9975d87-xngcd Container started 48m Normal Created pod/e2e-shared-a-w6-9f77b7-openshift-default-6c9975d87-xngcd Container created 48m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-n2q6l Stopping container discovery 48m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: All metrics below target 48m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 4 to 3 48m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-n2q6l 48m Normal AddedLabel gateway/e2e-shared-b-w1-66c72e Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w1-66c72e 48m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-66c72e-openshift-default-759cb84997 Created pod: e2e-shared-b-w1-66c72e-openshift-default-759cb84997-zklx9 48m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-66c72e-openshift-default Scaled up replica set e2e-shared-b-w1-66c72e-openshift-default-759cb84997 from 0 to 1 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w6-5451a5 Scaled up replica set payload-processing-e2e-shared-b-w6-5451a5-566f6b7dbf from 0 to 1 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w6-5451a5 Scaled up replica set payload-pre-processing-e2e-shared-b-w6-5451a5-54b9b789c9 from 0 to 1 48m Warning ProbeError pod/e2e-shared-a-w6-9f77b7-openshift-default-6c9975d87-xngcd Startup probe error: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-shared-a-w6-9f77b7-openshift-default-6c9975d87-xngcd Startup probe failed: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused 48m Normal AddedInterface pod/e2e-shared-b-w1-66c72e-openshift-default-759cb84997-zklx9 Add eth0 [10.133.0.175/23] from ovn-kubernetes 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w6-5451a5-54b9b789c9 Error creating: pods "payload-pre-processing-e2e-shared-b-w6-5451a5-54b9b789c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-5451a5: serviceaccount "payload-processing-e2e-shared-b-w6-5451a5" not found 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w6-5451a5-54b9b789c9 Created pod: payload-pre-processing-e2e-shared-b-w6-5451a5-54b9b789c9-xs2rv 48m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w6-5451a5-566f6b7dbf Error creating: pods "payload-processing-e2e-shared-b-w6-5451a5-566f6b7dbf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-5451a5: serviceaccount "payload-processing-e2e-shared-b-w6-5451a5" not found 48m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w6-5451a5-54b9b789c9-xs2rv Add eth0 [10.133.0.176/23] from ovn-kubernetes 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w6-5451a5-566f6b7dbf Created pod: payload-processing-e2e-shared-b-w6-5451a5-566f6b7dbf-gmzc5 48m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w6-5451a5-54b9b789c9-xs2rv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-processing-e2e-shared-b-w6-5451a5-566f6b7dbf-gmzc5 Add eth0 [10.133.0.177/23] from ovn-kubernetes 48m Normal Pulled pod/payload-processing-e2e-shared-b-w6-5451a5-566f6b7dbf-gmzc5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/payload-pre-processing-e2e-shared-b-w6-5451a5-54b9b789c9-xs2rv Container created 48m Normal Started pod/payload-pre-processing-e2e-shared-b-w6-5451a5-54b9b789c9-xs2rv Container started 48m Normal Created pod/payload-processing-e2e-shared-b-w6-5451a5-566f6b7dbf-gmzc5 Container created 48m Normal Started pod/payload-processing-e2e-shared-b-w6-5451a5-566f6b7dbf-gmzc5 Container started 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w1-66c72e-7cd79f45c5 Created pod: payload-pre-processing-e2e-shared-b-w1-66c72e-7cd79f45c5-68fdp 48m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w1-66c72e-66896d7d67 Error creating: pods "payload-processing-e2e-shared-b-w1-66c72e-66896d7d67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-66c72e: serviceaccount "payload-processing-e2e-shared-b-w1-66c72e" not found 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w1-66c72e Scaled up replica set payload-processing-e2e-shared-b-w1-66c72e-66896d7d67 from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w1-66c72e-66896d7d67 Created pod: payload-processing-e2e-shared-b-w1-66c72e-66896d7d67-k7dq8 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w1-66c72e Scaled up replica set payload-pre-processing-e2e-shared-b-w1-66c72e-7cd79f45c5 from 0 to 1 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w1-66c72e-7cd79f45c5 Error creating: pods "payload-pre-processing-e2e-shared-b-w1-66c72e-7cd79f45c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-66c72e: serviceaccount "payload-processing-e2e-shared-b-w1-66c72e" not found 48m Normal Started pod/payload-pre-processing-e2e-shared-b-w1-66c72e-7cd79f45c5-68fdp Container started 48m Normal Created pod/payload-pre-processing-e2e-shared-b-w1-66c72e-7cd79f45c5-68fdp Container created 48m Normal Started pod/payload-processing-e2e-shared-b-w1-66c72e-66896d7d67-k7dq8 Container started 48m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w1-66c72e-7cd79f45c5-68fdp Add eth0 [10.133.0.178/23] from ovn-kubernetes 48m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w1-66c72e-7cd79f45c5-68fdp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-processing-e2e-shared-b-w1-66c72e-66896d7d67-k7dq8 Add eth0 [10.133.0.179/23] from ovn-kubernetes 48m Normal Pulled pod/payload-processing-e2e-shared-b-w1-66c72e-66896d7d67-k7dq8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/payload-processing-e2e-shared-b-w1-66c72e-66896d7d67-k7dq8 Container created 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-9f77b7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-9f77b7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-5451a5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-5451a5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-9f77b7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-9f77b7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-f22129-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-f22129-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Normal Killing pod/payload-pre-processing-e2e-mt-6b983e02-58d76c55bc-jqfd5 Stopping container payload-pre-processing 47m Normal Killing pod/payload-processing-e2e-mt-6b983e02-66b998455c-swpb9 Stopping container payload-processing 47m Warning Unhealthy pod/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf Readiness probe failed: Get "http://10.132.0.114:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/payload-pre-processing-e2e-mt-6b983e02-58d76c55bc-jqfd5 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 47m Warning ProbeError pod/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf Readiness probe error: Get "http://10.132.0.114:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/payload-processing-e2e-mt-6b983e02-66b998455c-swpb9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 47m Normal Created pod/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf Container created 47m Normal Started pod/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf Container started 47m Normal Pulled pod/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Warning Unhealthy pod/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf Startup probe failed: Get "http://10.132.0.114:15021/healthz/ready": dial tcp 10.132.0.114:15021: connect: connection refused 47m Warning ProbeError pod/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf Startup probe error: Get "http://10.132.0.114:15021/healthz/ready": dial tcp 10.132.0.114:15021: connect: connection refused... 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-32de96-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-32de96-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-c2dd3b-openshift-default New size: 5; reason: All metrics below target 46m Normal Killing pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-7twrh Stopping container istio-proxy 46m Warning ProbeError pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-48wgm Readiness probe error: HTTP probe failed with statuscode: 503... 46m Normal ScalingReplicaSet deployment/e2e-worker-w5-ef6b43-openshift-default Scaled down replica set e2e-worker-w5-ef6b43-openshift-default-679658c459 from 3 to 2 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-9a6cf2-openshift-default New size: 5; reason: All metrics below target 46m Normal ScalingReplicaSet deployment/e2e-worker-w2-4d9ea0-openshift-default Scaled down replica set e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc from 4 to 3 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-4d9ea0-openshift-default New size: 3; reason: All metrics below target 46m Normal ScalingReplicaSet deployment/e2e-worker-w3-9a6cf2-openshift-default Scaled down replica set e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c from 6 to 5 46m Warning Unhealthy pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-48wgm Readiness probe failed: HTTP probe failed with statuscode: 503 46m Normal SuccessfulDelete replicaset/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f Deleted pod: e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-48wgm 46m Normal Killing pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-48wgm Stopping container istio-proxy 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-ef6b43-openshift-default New size: 2; reason: All metrics below target 46m Normal SuccessfulDelete replicaset/e2e-worker-w5-ef6b43-openshift-default-679658c459 Deleted pod: e2e-worker-w5-ef6b43-openshift-default-679658c459-nmbnd 46m Normal SuccessfulDelete replicaset/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc Deleted pod: e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d 46m Normal SuccessfulDelete replicaset/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c Deleted pod: e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-7twrh 46m Normal ScalingReplicaSet deployment/e2e-worker-w1-c2dd3b-openshift-default Scaled down replica set e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f from 6 to 5 46m Normal Killing pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-nmbnd Stopping container istio-proxy 46m Normal ScalingReplicaSet deployment/e2e-worker-w4-4e2b27-openshift-default Scaled down replica set e2e-worker-w4-4e2b27-openshift-default-996dc4fdf from 6 to 5 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-4e2b27-openshift-default New size: 5; reason: All metrics below target 46m Normal Killing pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp Stopping container istio-proxy 46m Normal Killing pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d Stopping container istio-proxy 46m Normal Killing pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5kdtc Stopping container istio-proxy 46m Normal ScalingReplicaSet deployment/e2e-worker-w0-dd6574-openshift-default Scaled down replica set e2e-worker-w0-dd6574-openshift-default-5d877f7ccc from 6 to 4 46m Normal SuccessfulDelete replicaset/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc Deleted pod: e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tzgp6 46m Normal SuccessfulDelete replicaset/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf Deleted pod: e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp 46m Normal Killing pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tzgp6 Stopping container istio-proxy 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-dd6574-openshift-default New size: 4; reason: All metrics below target 46m Normal ScalingReplicaSet deployment/e2e-worker-w6-284b8e-openshift-default Scaled down replica set e2e-worker-w6-284b8e-openshift-default-77c5d9855 from 6 to 5 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-284b8e-openshift-default New size: 5; reason: All metrics below target 46m Normal SuccessfulDelete replicaset/e2e-worker-w6-284b8e-openshift-default-77c5d9855 Deleted pod: e2e-worker-w6-284b8e-openshift-default-77c5d9855-gh4d7 46m Normal SuccessfulDelete replicaset/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc Deleted pod: e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-5kdtc 46m Normal Killing pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-gh4d7 Stopping container istio-proxy 46m Warning Unhealthy pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning ProbeError pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-46p2d Readiness probe error: HTTP probe failed with statuscode: 503... 46m Warning ProbeError pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-7twrh Readiness probe error: HTTP probe failed with statuscode: 503... 46m Warning ProbeError pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tzgp6 Readiness probe error: HTTP probe failed with statuscode: 503... 46m Warning Unhealthy pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-7twrh Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning Unhealthy pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tzgp6 Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning Unhealthy pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-nmbnd Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning ProbeError pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-nmbnd Readiness probe error: HTTP probe failed with statuscode: 503... 46m Warning Unhealthy pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-gh4d7 Readiness probe failed: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp Readiness probe failed: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-xxqjp Readiness probe error: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-gh4d7 Readiness probe error: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused... 46m Normal Killing pod/maas-default-gateway-openshift-default-77bfb58f8c-n2h8k Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-77bfb58f8c Deleted pod: maas-default-gateway-openshift-default-77bfb58f8c-n2h8k 46m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-77bfb58f8c Deleted pod: maas-default-gateway-openshift-default-77bfb58f8c-lq8nh 46m Normal Killing pod/maas-default-gateway-openshift-default-77bfb58f8c-lq8nh Stopping container istio-proxy 46m Warning ProbeError pod/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq Readiness probe error: Get "http://10.132.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq Readiness probe failed: Get "http://10.132.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Normal AddedInterface pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-vhbmj Add eth0 [10.133.0.199/23] from ovn-kubernetes 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-5451a5-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 46m Normal AddedInterface pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-cflq9 Add eth0 [10.134.0.112/23] from ovn-kubernetes 46m Normal AddedInterface pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-jq4p8 Add eth0 [10.133.0.198/23] from ovn-kubernetes 46m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47 Created pod: e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-cflq9 46m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47 Created pod: e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-jq4p8 46m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47 Created pod: e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-vhbmj 46m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-5451a5-openshift-default Scaled up replica set e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47 from 1 to 4 46m Normal Pulled pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-cflq9 Container started 46m Normal Killing pod/payload-processing-e2e-shared-a-w3-06366c-5c966f6c9f-6k5nh Stopping container payload-processing 46m Normal Pulled pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-jq4p8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Pulled pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-vhbmj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-vhbmj Container started 46m Normal Pulled pod/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-jq4p8 Container created 46m Normal Created pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-vhbmj Container created 46m Normal Pulled pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-cflq9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Warning Unhealthy pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc Readiness probe failed: Get "http://10.132.0.117:15021/healthz/ready": dial tcp 10.132.0.117:15021: connect: connection refused 46m Warning ProbeError pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc Readiness probe error: Get "http://10.132.0.117:15021/healthz/ready": dial tcp 10.132.0.117:15021: connect: connection refused... 46m Normal Created pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-cflq9 Container created 46m Normal Started pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-jq4p8 Container started 46m Normal Started pod/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq Container started 46m Normal Started pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc Container started 46m Normal Created pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc Container created 46m Normal Killing pod/payload-pre-processing-e2e-shared-a-w3-06366c-6cc657fc4c-4b96s Stopping container payload-pre-processing 46m Normal Created pod/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq Container created 46m Normal Killing pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-7hdld Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f Deleted pod: e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-79tql 46m Normal ScalingReplicaSet deployment/e2e-worker-w1-c2dd3b-openshift-default Scaled down replica set e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f from 5 to 3 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-c2dd3b-openshift-default New size: 3; reason: All metrics below target 46m Normal SuccessfulDelete replicaset/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f Deleted pod: e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-7hdld 46m Normal Killing pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-79tql Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc Deleted pod: e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x 46m Normal Killing pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x Stopping container istio-proxy 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-dd6574-openshift-default New size: 3; reason: All metrics below target 46m Warning Unhealthy pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning ProbeError pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-tqf5x Readiness probe error: HTTP probe failed with statuscode: 503... 46m Warning ProbeError pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-79tql Readiness probe error: HTTP probe failed with statuscode: 503... 46m Warning Unhealthy pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-jq4p8 Startup probe failed: Get "http://10.133.0.198:15021/healthz/ready": dial tcp 10.133.0.198:15021: connect: connection refused 46m Warning ProbeError pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-jq4p8 Startup probe error: Get "http://10.133.0.198:15021/healthz/ready": dial tcp 10.133.0.198:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-79tql Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning Unhealthy pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-cflq9 Startup probe failed: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused 46m Warning ProbeError pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-cflq9 Startup probe error: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-7hdld Readiness probe error: HTTP probe failed with statuscode: 503... 46m Warning Unhealthy pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-7hdld Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning Unhealthy pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-vhbmj Startup probe failed: Get "http://10.133.0.199:15021/healthz/ready": dial tcp 10.133.0.199:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc Startup probe failed: Get "http://10.132.0.117:15021/healthz/ready": dial tcp 10.132.0.117:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq Startup probe failed: Get "http://10.132.0.115:15021/healthz/ready": dial tcp 10.132.0.115:15021: connect: connection refused 46m Warning ProbeError pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-vhbmj Startup probe error: Get "http://10.133.0.199:15021/healthz/ready": dial tcp 10.133.0.199:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc Startup probe error: Get "http://10.132.0.117:15021/healthz/ready": dial tcp 10.132.0.117:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq Startup probe error: Get "http://10.132.0.115:15021/healthz/ready": dial tcp 10.132.0.115:15021: connect: connection refused... 46m Normal Pulled pod/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7 Container started 46m Normal Created pod/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7 Container created 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-a1bf7d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-a1bf7d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-mwdzc Add eth0 [10.134.0.114/23] from ovn-kubernetes 46m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-6swr2 Container started 46m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-cd928 Container started 46m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-6swr2 Add eth0 [10.133.0.202/23] from ovn-kubernetes 46m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-mwdzc Container created 46m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-mwdzc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-cd928 Container created 46m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-pjj8k Container started 46m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-6swr2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-cd928 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-pjj8k Container created 46m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-mwdzc Container started 46m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-cd928 Add eth0 [10.133.0.201/23] from ovn-kubernetes 46m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-6swr2 Container created 46m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-pjj8k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-pjj8k Add eth0 [10.134.0.113/23] from ovn-kubernetes 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-5451a5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-5451a5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning Unhealthy pod/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7 Startup probe failed: Get "http://10.132.0.111:15021/healthz/ready": dial tcp 10.132.0.111:15021: connect: connection refused 46m Warning ProbeError pod/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7 Startup probe error: Get "http://10.132.0.111:15021/healthz/ready": dial tcp 10.132.0.111:15021: connect: connection refused... 46m Normal SuccessfulDelete replicaset/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c Deleted pod: e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp 46m Normal Killing pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp Stopping container istio-proxy 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-9a6cf2-openshift-default New size: 4; reason: All metrics below target 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-4e2b27-openshift-default New size: 4; reason: All metrics below target 46m Normal Killing pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-fsc7d Stopping container istio-proxy 46m Warning ProbeError pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp Readiness probe error: HTTP probe failed with statuscode: 503... 46m Normal SuccessfulDelete replicaset/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf Deleted pod: e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-fsc7d 46m Warning Unhealthy pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-d9kkp Readiness probe failed: HTTP probe failed with statuscode: 503 46m Normal ScalingReplicaSet deployment/e2e-worker-w4-4e2b27-openshift-default Scaled down replica set e2e-worker-w4-4e2b27-openshift-default-996dc4fdf from 5 to 4 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-6swr2 Startup probe failed: Get "http://10.133.0.202:15021/healthz/ready": dial tcp 10.133.0.202:15021: connect: connection refused 46m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-6swr2 Startup probe error: Get "http://10.133.0.202:15021/healthz/ready": dial tcp 10.133.0.202:15021: connect: connection refused... 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-cd928 Startup probe failed: Get "http://10.133.0.201:15021/healthz/ready": dial tcp 10.133.0.201:15021: connect: connection refused 46m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-cd928 Startup probe error: Get "http://10.133.0.201:15021/healthz/ready": dial tcp 10.133.0.201:15021: connect: connection refused... 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-pjj8k Startup probe failed: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused 46m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-pjj8k Startup probe error: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused... 46m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-mwdzc Startup probe error: Get "http://10.134.0.114:15021/healthz/ready": dial tcp 10.134.0.114:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-fsc7d Readiness probe error: HTTP probe failed with statuscode: 503... 46m Warning Unhealthy pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-fsc7d Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-mwdzc Startup probe failed: Get "http://10.134.0.114:15021/healthz/ready": dial tcp 10.134.0.114:15021: connect: connection refused 46m Normal Killing pod/payload-processing-e2e-api-a-b7107c-65f4b7cf5-xblnb Stopping container payload-processing 46m Normal Killing pod/payload-pre-processing-e2e-api-a-b7107c-d874d78b7-pp8nv Stopping container payload-pre-processing 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-f13b18-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-f13b18-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Normal Killing pod/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd Stopping container payload-processing 45m Normal Killing pod/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db Stopping container payload-pre-processing 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-5451a5-openshift-default New size: 8; reason: 45m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-5451a5-openshift-default Scaled up replica set e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47 from 4 to 8 45m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47 Created pod: e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl 45m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47 Created pod: e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k 45m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47 Created pod: e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-c2rhx 45m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47 Created pod: e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-nwvxb 45m Normal AddedInterface pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-c2rhx Add eth0 [10.133.0.204/23] from ovn-kubernetes 45m Normal Pulled pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Pulled pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k Container started 45m Normal Pulled pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-c2rhx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k Container created 45m Normal Started pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-c2rhx Container started 45m Normal Started pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl Container started 45m Normal Created pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl Container created 45m Normal Started pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-nwvxb Container started 45m Normal Created pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-nwvxb Container created 45m Normal Pulled pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-nwvxb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-nwvxb Add eth0 [10.134.0.115/23] from ovn-kubernetes 45m Normal Created pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-c2rhx Container created 45m Normal AddedInterface pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl Add eth0 [10.132.0.125/23] from ovn-kubernetes 45m Normal AddedInterface pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k Add eth0 [10.132.0.124/23] from ovn-kubernetes 45m Normal Killing pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-rhzmq Stopping container istio-proxy 45m Normal SuccessfulDelete replicaset/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c Deleted pod: e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-rhzmq 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-4e2b27-openshift-default New size: 3; reason: All metrics below target 45m Normal ScalingReplicaSet deployment/e2e-worker-w6-284b8e-openshift-default Scaled down replica set e2e-worker-w6-284b8e-openshift-default-77c5d9855 from 5 to 3 45m Normal Killing pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-qptp7 Stopping container istio-proxy 45m Normal Killing pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns Stopping container istio-proxy 45m Warning Unhealthy pod/payload-processing-e2e-api-a-d3b375-bc5458c5-7dcpd Readiness probe failed: timeout: failed to connect service "10.132.0.100:50052" within 1s: context deadline exceeded 45m Warning Unhealthy pod/payload-pre-processing-e2e-api-a-d3b375-78cfb749f-2k2db Readiness probe failed: timeout: failed to connect service "10.132.0.99:50052" within 1s: context deadline exceeded 45m Normal SuccessfulDelete replicaset/e2e-worker-w6-284b8e-openshift-default-77c5d9855 Deleted pod: e2e-worker-w6-284b8e-openshift-default-77c5d9855-qptp7 45m Normal SuccessfulDelete replicaset/e2e-worker-w6-284b8e-openshift-default-77c5d9855 Deleted pod: e2e-worker-w6-284b8e-openshift-default-77c5d9855-djjtz 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-284b8e-openshift-default New size: 3; reason: All metrics below target 45m Normal SuccessfulDelete replicaset/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf Deleted pod: e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns 45m Normal Killing pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-djjtz Stopping container istio-proxy 45m Warning ProbeError pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-rhzmq Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning Unhealthy pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-rhzmq Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-nwvxb Startup probe error: Get "http://10.134.0.115:15021/healthz/ready": dial tcp 10.134.0.115:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning Unhealthy pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-nwvxb Startup probe failed: Get "http://10.134.0.115:15021/healthz/ready": dial tcp 10.134.0.115:15021: connect: connection refused 45m Warning Unhealthy pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-c2rhx Startup probe failed: Get "http://10.133.0.204:15021/healthz/ready": dial tcp 10.133.0.204:15021: connect: connection refused 45m Warning ProbeError pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-8bfns Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning ProbeError pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-c2rhx Startup probe error: Get "http://10.133.0.204:15021/healthz/ready": dial tcp 10.133.0.204:15021: connect: connection refused... 45m Warning ProbeError pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl Startup probe error: Get "http://10.132.0.125:15021/healthz/ready": dial tcp 10.132.0.125:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k Startup probe failed: Get "http://10.132.0.124:15021/healthz/ready": dial tcp 10.132.0.124:15021: connect: connection refused 45m Warning Unhealthy pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl Startup probe failed: Get "http://10.132.0.125:15021/healthz/ready": dial tcp 10.132.0.125:15021: connect: connection refused 45m Warning ProbeError pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k Startup probe error: Get "http://10.132.0.124:15021/healthz/ready": dial tcp 10.132.0.124:15021: connect: connection refused... 45m Warning ProbeError pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-qptp7 Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning Unhealthy pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-djjtz Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning Unhealthy pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-qptp7 Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-djjtz Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-a1bf7d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-a1bf7d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-f13b18-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-f13b18-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47 Created pod: e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-khkfr 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-5451a5-openshift-default New size: 9; reason: All metrics below target 45m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-5451a5-openshift-default Scaled up replica set e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47 from 8 to 9 45m Normal Started pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-khkfr Container started 45m Normal Created pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-khkfr Container created 45m Normal Pulled pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-khkfr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-khkfr Add eth0 [10.134.0.116/23] from ovn-kubernetes 45m Warning Unhealthy pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-khkfr Startup probe failed: Get "http://10.134.0.116:15021/healthz/ready": dial tcp 10.134.0.116:15021: connect: connection refused 45m Warning ProbeError pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-khkfr Startup probe error: Get "http://10.134.0.116:15021/healthz/ready": dial tcp 10.134.0.116:15021: connect: connection refused... 45m Normal Killing pod/payload-pre-processing-e2e-shared-a-w6-9f77b7-75844768f5-lkfwf Stopping container payload-pre-processing 45m Normal Killing pod/payload-processing-e2e-shared-a-w6-9f77b7-56fd964c9c-2jvsn Stopping container payload-processing 45m Normal Killing pod/payload-processing-e2e-shared-a-w1-d4d045-68bdbbc685-g254t Stopping container payload-processing 45m Normal Killing pod/payload-pre-processing-e2e-shared-a-w1-d4d045-8d456f96c-cjl6r Stopping container payload-pre-processing 45m Normal Killing pod/e2e-shared-a-w3-06366c-openshift-default-6cd8c77f65-8swpg Stopping container istio-proxy 45m Warning Unhealthy pod/e2e-shared-a-w3-06366c-openshift-default-6cd8c77f65-8swpg Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/e2e-shared-a-w3-06366c-openshift-default-6cd8c77f65-8swpg Readiness probe error: HTTP probe failed with statuscode: 503... 44m Warning ProbeError pod/e2e-shared-b-w1-66c72e-openshift-default-759cb84997-zklx9 Readiness probe error: Get "http://10.133.0.175:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Warning Unhealthy pod/e2e-shared-b-w1-66c72e-openshift-default-759cb84997-zklx9 Readiness probe failed: Get "http://10.133.0.175:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Normal Started pod/e2e-shared-b-w1-66c72e-openshift-default-759cb84997-zklx9 Container started 44m Normal Created pod/e2e-shared-b-w1-66c72e-openshift-default-759cb84997-zklx9 Container created 44m Normal Pulled pod/e2e-shared-b-w1-66c72e-openshift-default-759cb84997-zklx9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Killing pod/payload-pre-processing-e2e-shared-b-w3-f13b18-6fc8bf95cf-hlbms Stopping container payload-pre-processing 44m Normal Killing pod/payload-processing-e2e-shared-b-w3-f13b18-b56b9dfc9-snjpt Stopping container payload-processing 44m Warning Unhealthy pod/e2e-shared-b-w1-66c72e-openshift-default-759cb84997-zklx9 Startup probe failed: Get "http://10.133.0.175:15021/healthz/ready": dial tcp 10.133.0.175:15021: connect: connection refused 44m Warning ProbeError pod/e2e-shared-b-w1-66c72e-openshift-default-759cb84997-zklx9 Startup probe error: Get "http://10.133.0.175:15021/healthz/ready": dial tcp 10.133.0.175:15021: connect: connection refused... 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-66c72e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-66c72e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-6b983e02 Scaled up replica set payload-processing-e2e-mt-6b983e02-66b998455c from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-6b983e02-58d76c55bc Created pod: payload-pre-processing-e2e-mt-6b983e02-58d76c55bc-hvlc4 44m Warning FailedCreate replicaset/payload-processing-e2e-mt-6b983e02-66b998455c Error creating: pods "payload-processing-e2e-mt-6b983e02-66b998455c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-6b983e02: serviceaccount "payload-processing-e2e-mt-6b983e02" not found 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-6b983e02-58d76c55bc Error creating: pods "payload-pre-processing-e2e-mt-6b983e02-58d76c55bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-6b983e02: serviceaccount "payload-processing-e2e-mt-6b983e02" not found 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-6b983e02 Scaled up replica set payload-pre-processing-e2e-mt-6b983e02-58d76c55bc from 0 to 1 44m Normal Started pod/payload-pre-processing-e2e-mt-6b983e02-58d76c55bc-hvlc4 Container started 44m Normal Pulled pod/payload-pre-processing-e2e-mt-6b983e02-58d76c55bc-hvlc4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-pre-processing-e2e-mt-6b983e02-58d76c55bc-hvlc4 Add eth0 [10.133.0.209/23] from ovn-kubernetes 44m Normal AddedInterface pod/payload-processing-e2e-mt-6b983e02-66b998455c-t7x6x Add eth0 [10.133.0.210/23] from ovn-kubernetes 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-6b983e02-66b998455c Created pod: payload-processing-e2e-mt-6b983e02-66b998455c-t7x6x 44m Normal Created pod/payload-pre-processing-e2e-mt-6b983e02-58d76c55bc-hvlc4 Container created 44m Normal Started pod/payload-processing-e2e-mt-6b983e02-66b998455c-t7x6x Container started 44m Normal Pulled pod/payload-processing-e2e-mt-6b983e02-66b998455c-t7x6x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-processing-e2e-mt-6b983e02-66b998455c-t7x6x Container created 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-66c72e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-66c72e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal Killing pod/e2e-shared-a-w6-9f77b7-openshift-default-6c9975d87-xngcd Stopping container istio-proxy 44m Warning Unhealthy pod/e2e-shared-a-w6-9f77b7-openshift-default-6c9975d87-xngcd Readiness probe failed: HTTP probe failed with statuscode: 503 44m Warning ProbeError pod/e2e-shared-a-w6-9f77b7-openshift-default-6c9975d87-xngcd Readiness probe error: HTTP probe failed with statuscode: 503... 44m Normal Killing pod/e2e-shared-a-w1-d4d045-openshift-default-d6546474c-d6vsz Stopping container istio-proxy 43m Normal AddedLabel gateway/e2e-inf-a-7cc1b1 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-7cc1b1 43m Normal ScalingReplicaSet deployment/e2e-inf-a-7cc1b1-openshift-default Scaled up replica set e2e-inf-a-7cc1b1-openshift-default-84d569b9c from 0 to 1 43m Normal SuccessfulCreate replicaset/e2e-inf-a-7cc1b1-openshift-default-84d569b9c Created pod: e2e-inf-a-7cc1b1-openshift-default-84d569b9c-pkbzx 43m Normal AddedInterface pod/e2e-inf-a-7cc1b1-openshift-default-84d569b9c-pkbzx Add eth0 [10.133.0.213/23] from ovn-kubernetes 43m Normal Killing pod/payload-processing-e2e-shared-b-w6-5451a5-566f6b7dbf-gmzc5 Stopping container payload-processing 43m Normal Killing pod/payload-pre-processing-e2e-shared-b-w6-5451a5-54b9b789c9-xs2rv Stopping container payload-pre-processing 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-c2dd3b-openshift-default New size: 2; reason: All metrics below target 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-4d9ea0-openshift-default New size: 2; reason: All metrics below target 43m Normal SuccessfulDelete replicaset/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc Deleted pod: e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-tkbbb 43m Normal SuccessfulDelete replicaset/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f Deleted pod: e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-5rd2m 43m Normal SuccessfulDelete replicaset/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c Deleted pod: e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6 43m Normal Killing pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6 Stopping container istio-proxy 43m Normal Killing pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-tkbbb Stopping container istio-proxy 43m Normal Killing pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-5rd2m Stopping container istio-proxy 43m Warning ProbeError pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cf9jh Readiness probe error: HTTP probe failed with statuscode: 503... 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-7cc1b1 Scaled up replica set payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799 from 0 to 1 43m Normal SuccessfulDelete replicaset/e2e-worker-w6-284b8e-openshift-default-77c5d9855 Deleted pod: e2e-worker-w6-284b8e-openshift-default-77c5d9855-qtwpp 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-284b8e-openshift-default New size: 2; reason: All metrics below target 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-7cc1b1 Scaled up replica set payload-processing-e2e-inf-a-7cc1b1-7487bc448c from 0 to 1 43m Normal SuccessfulDelete replicaset/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf Deleted pod: e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cf9jh 43m Warning Unhealthy pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cf9jh Readiness probe failed: HTTP probe failed with statuscode: 503 43m Normal Killing pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cf9jh Stopping container istio-proxy 43m Warning Unhealthy pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-tkbbb Readiness probe failed: HTTP probe failed with statuscode: 503 43m Warning ProbeError pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-tkbbb Readiness probe error: HTTP probe failed with statuscode: 503... 43m Normal Killing pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-qtwpp Stopping container istio-proxy 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799 Error creating: pods "payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-7cc1b1: serviceaccount "payload-processing-e2e-inf-a-7cc1b1" not found 43m Warning ProbeError pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6 Readiness probe error: HTTP probe failed with statuscode: 503... 43m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 43m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799 Created pod: payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-7cc1b1-7487bc448c Created pod: payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57 43m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-7cc1b1-7487bc448c Error creating: pods "payload-processing-e2e-inf-a-7cc1b1-7487bc448c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-7cc1b1: serviceaccount "payload-processing-e2e-inf-a-7cc1b1" not found 43m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-bxq4p Stopping container discovery 43m Normal Killing pod/payload-processing-e2e-shared-b-w1-66c72e-66896d7d67-k7dq8 Stopping container payload-processing 43m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-bxq4p 43m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-bxq4p Readiness probe failed: Get "http://10.134.0.88:8080/ready": dial tcp 10.134.0.88:8080: connect: connection refused 43m Warning Unhealthy pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gp5g6 Readiness probe failed: HTTP probe failed with statuscode: 503 43m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-bxq4p Readiness probe error: Get "http://10.134.0.88:8080/ready": dial tcp 10.134.0.88:8080: connect: connection refused... 43m Normal AddedInterface pod/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57 Add eth0 [10.132.0.127/23] from ovn-kubernetes 43m Warning Unhealthy pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-5rd2m Readiness probe failed: HTTP probe failed with statuscode: 503 43m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv Add eth0 [10.132.0.126/23] from ovn-kubernetes 43m Normal Started pod/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57 Container started 43m Normal Started pod/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv Container started 43m Normal Created pod/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv Container created 43m Normal Created pod/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57 Container created 43m Normal Killing pod/payload-pre-processing-e2e-shared-b-w1-66c72e-7cd79f45c5-68fdp Stopping container payload-pre-processing 43m Normal Pulled pod/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Pulled pod/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Warning ProbeError pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-5rd2m Readiness probe error: HTTP probe failed with statuscode: 503... 43m Warning Unhealthy pod/payload-processing-e2e-shared-b-w6-5451a5-566f6b7dbf-gmzc5 Readiness probe failed: timeout: failed to connect service "10.133.0.177:50052" within 1s: context deadline exceeded 43m Warning Unhealthy pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-qtwpp Readiness probe failed: HTTP probe failed with statuscode: 503 43m Warning ProbeError pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-qtwpp Readiness probe error: HTTP probe failed with statuscode: 503... 43m Normal Killing pod/e2e-shared-b-w3-f13b18-openshift-default-76f6ddb5bf-kvjd7 Stopping container istio-proxy 43m Normal Killing pod/e2e-api-a-b7107c-openshift-default-778f76c545-8hgqg Stopping container istio-proxy 43m Normal Killing pod/e2e-api-a-d3b375-openshift-default-5b854d5c65-9th76 Stopping container istio-proxy 43m Normal Killing pod/payload-pre-processing-e2e-api-b-a1bf7d-6955977c5-wvmp9 Stopping container payload-pre-processing 43m Normal Killing pod/payload-processing-e2e-api-b-a1bf7d-5499dfdd6c-6dqjk Stopping container payload-processing 43m Normal Killing pod/payload-processing-e2e-api-b-32de96-5ff6bbb5ff-rqwzc Stopping container payload-processing 43m Normal Killing pod/payload-pre-processing-e2e-api-b-32de96-5dc88f798c-hxh64 Stopping container payload-pre-processing 42m Normal SuccessfulCreate replicaset/e2e-mt-6b983e02-openshift-default-f8946f76f Created pod: e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s 42m Normal SuccessfulCreate replicaset/e2e-mt-6b983e02-openshift-default-f8946f76f Created pod: e2e-mt-6b983e02-openshift-default-f8946f76f-d9dcg 42m Normal ScalingReplicaSet deployment/e2e-mt-6b983e02-openshift-default Scaled up replica set e2e-mt-6b983e02-openshift-default-f8946f76f from 1 to 4 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-6b983e02-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 42m Normal AddedInterface pod/e2e-mt-6b983e02-openshift-default-f8946f76f-d9dcg Add eth0 [10.134.0.118/23] from ovn-kubernetes 42m Normal AddedInterface pod/e2e-mt-6b983e02-openshift-default-f8946f76f-hgtfw Add eth0 [10.133.0.216/23] from ovn-kubernetes 42m Normal SuccessfulCreate replicaset/e2e-mt-6b983e02-openshift-default-f8946f76f Created pod: e2e-mt-6b983e02-openshift-default-f8946f76f-hgtfw 42m Normal Pulled pod/e2e-mt-6b983e02-openshift-default-f8946f76f-hgtfw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Started pod/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s Container started 42m Normal Created pod/e2e-mt-6b983e02-openshift-default-f8946f76f-hgtfw Container created 42m Normal AddedInterface pod/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s Add eth0 [10.132.0.128/23] from ovn-kubernetes 42m Normal Started pod/e2e-mt-6b983e02-openshift-default-f8946f76f-hgtfw Container started 42m Normal Started pod/e2e-mt-6b983e02-openshift-default-f8946f76f-d9dcg Container started 42m Normal Pulled pod/e2e-mt-6b983e02-openshift-default-f8946f76f-d9dcg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-mt-6b983e02-openshift-default-f8946f76f-d9dcg Container created 42m Normal Pulled pod/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s Container created 42m Normal Killing pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-cflq9 Stopping container istio-proxy 42m Normal Killing pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-khkfr Stopping container istio-proxy 42m Normal Killing pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-xltrl Stopping container istio-proxy 42m Normal Killing pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-r9fsc Stopping container istio-proxy 42m Normal Killing pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-nwvxb Stopping container istio-proxy 42m Normal Killing pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-c2rhx Stopping container istio-proxy 42m Normal Killing pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-zgf6k Stopping container istio-proxy 42m Normal Killing pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-jq4p8 Stopping container istio-proxy 42m Warning Unhealthy pod/e2e-mt-6b983e02-openshift-default-f8946f76f-d9dcg Startup probe failed: Get "http://10.134.0.118:15021/healthz/ready": dial tcp 10.134.0.118:15021: connect: connection refused 42m Warning Unhealthy pod/e2e-mt-6b983e02-openshift-default-f8946f76f-hgtfw Startup probe failed: Get "http://10.133.0.216:15021/healthz/ready": dial tcp 10.133.0.216:15021: connect: connection refused 42m Warning ProbeError pod/e2e-mt-6b983e02-openshift-default-f8946f76f-hgtfw Startup probe error: Get "http://10.133.0.216:15021/healthz/ready": dial tcp 10.133.0.216:15021: connect: connection refused... 42m Warning ProbeError pod/e2e-mt-6b983e02-openshift-default-f8946f76f-d9dcg Startup probe error: Get "http://10.134.0.118:15021/healthz/ready": dial tcp 10.134.0.118:15021: connect: connection refused... 42m Normal AddedLabel gateway/e2e-api-a-d0946d Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-d0946d 42m Normal Killing pod/e2e-shared-b-w6-5451a5-openshift-default-9b4d8df47-vhbmj Stopping container istio-proxy 42m Warning ProbeError pod/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s Startup probe error: Get "http://10.132.0.128:15021/healthz/ready": dial tcp 10.132.0.128:15021: connect: connection refused... 42m Normal SuccessfulCreate replicaset/e2e-api-a-d0946d-openshift-default-6dc6f68d55 Created pod: e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk 42m Warning Unhealthy pod/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s Startup probe failed: Get "http://10.132.0.128:15021/healthz/ready": dial tcp 10.132.0.128:15021: connect: connection refused 42m Normal ScalingReplicaSet deployment/e2e-api-a-d0946d-openshift-default Scaled up replica set e2e-api-a-d0946d-openshift-default-6dc6f68d55 from 0 to 1 42m Normal AddedInterface pod/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk Add eth0 [10.132.0.129/23] from ovn-kubernetes 42m Normal Killing pod/e2e-shared-b-w1-66c72e-openshift-default-759cb84997-zklx9 Stopping container istio-proxy 42m Normal AddedLabel gateway/e2e-inf-a-686bbf Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-686bbf 42m Normal SuccessfulCreate replicaset/e2e-inf-a-686bbf-openshift-default-65b6c69fbc Created pod: e2e-inf-a-686bbf-openshift-default-65b6c69fbc-vwvdd 42m Normal ScalingReplicaSet deployment/e2e-inf-a-686bbf-openshift-default Scaled up replica set e2e-inf-a-686bbf-openshift-default-65b6c69fbc from 0 to 1 42m Normal AddedInterface pod/e2e-inf-a-686bbf-openshift-default-65b6c69fbc-vwvdd Add eth0 [10.134.0.120/23] from ovn-kubernetes 42m Normal SuccessfulCreate replicaset/e2e-mt-6b983e02-openshift-default-f8946f76f Created pod: e2e-mt-6b983e02-openshift-default-f8946f76f-76pr7 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-d0946d Scaled up replica set payload-pre-processing-e2e-api-a-d0946d-8465488fff from 0 to 1 42m Normal ScalingReplicaSet deployment/e2e-mt-6b983e02-openshift-default Scaled up replica set e2e-mt-6b983e02-openshift-default-f8946f76f from 4 to 8 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-6b983e02-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 42m Normal SuccessfulCreate replicaset/e2e-mt-6b983e02-openshift-default-f8946f76f Created pod: e2e-mt-6b983e02-openshift-default-f8946f76f-gdqtv 42m Normal SuccessfulCreate replicaset/e2e-mt-6b983e02-openshift-default-f8946f76f Created pod: e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q 42m Normal SuccessfulCreate replicaset/e2e-mt-6b983e02-openshift-default-f8946f76f Created pod: e2e-mt-6b983e02-openshift-default-f8946f76f-p7dbl 42m Warning FailedCreate replicaset/payload-processing-e2e-api-a-d0946d-59f4d669d9 Error creating: pods "payload-processing-e2e-api-a-d0946d-59f4d669d9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-d0946d: serviceaccount "payload-processing-e2e-api-a-d0946d" not found 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-d0946d-59f4d669d9 Created pod: payload-processing-e2e-api-a-d0946d-59f4d669d9-k4wgp 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-d0946d Scaled up replica set payload-processing-e2e-api-a-d0946d-59f4d669d9 from 0 to 1 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-d0946d-8465488fff Error creating: pods "payload-pre-processing-e2e-api-a-d0946d-8465488fff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-d0946d: serviceaccount "payload-processing-e2e-api-a-d0946d" not found 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-d0946d-8465488fff Created pod: payload-pre-processing-e2e-api-a-d0946d-8465488fff-wnnmg 42m Normal Started pod/e2e-mt-6b983e02-openshift-default-f8946f76f-p7dbl Container started 42m Normal Pulled pod/payload-processing-e2e-api-a-d0946d-59f4d669d9-k4wgp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-mt-6b983e02-openshift-default-f8946f76f-76pr7 Container created 42m Normal Created pod/payload-processing-e2e-api-a-d0946d-59f4d669d9-k4wgp Container created 42m Normal Created pod/e2e-mt-6b983e02-openshift-default-f8946f76f-p7dbl Container created 42m Normal Started pod/payload-pre-processing-e2e-api-a-d0946d-8465488fff-wnnmg Container started 42m Normal Created pod/payload-pre-processing-e2e-api-a-d0946d-8465488fff-wnnmg Container created 42m Normal AddedInterface pod/e2e-mt-6b983e02-openshift-default-f8946f76f-gdqtv Add eth0 [10.133.0.218/23] from ovn-kubernetes 42m Normal Pulled pod/e2e-mt-6b983e02-openshift-default-f8946f76f-gdqtv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q Add eth0 [10.132.0.130/23] from ovn-kubernetes 42m Normal Started pod/payload-processing-e2e-api-a-d0946d-59f4d669d9-k4wgp Container started 42m Normal Created pod/e2e-mt-6b983e02-openshift-default-f8946f76f-gdqtv Container created 42m Normal Pulled pod/payload-pre-processing-e2e-api-a-d0946d-8465488fff-wnnmg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-d0946d-8465488fff-wnnmg Add eth0 [10.134.0.122/23] from ovn-kubernetes 42m Normal Pulled pod/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q Container created 42m Normal Started pod/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q Container started 42m Normal Pulled pod/e2e-mt-6b983e02-openshift-default-f8946f76f-p7dbl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/e2e-mt-6b983e02-openshift-default-f8946f76f-p7dbl Add eth0 [10.133.0.217/23] from ovn-kubernetes 42m Normal Pulled pod/e2e-mt-6b983e02-openshift-default-f8946f76f-76pr7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-processing-e2e-api-a-d0946d-59f4d669d9-k4wgp Add eth0 [10.134.0.123/23] from ovn-kubernetes 42m Normal AddedInterface pod/e2e-mt-6b983e02-openshift-default-f8946f76f-76pr7 Add eth0 [10.134.0.121/23] from ovn-kubernetes 42m Normal Started pod/e2e-mt-6b983e02-openshift-default-f8946f76f-76pr7 Container started 42m Normal Started pod/e2e-mt-6b983e02-openshift-default-f8946f76f-gdqtv Container started 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-d0946d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-d0946d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning Unhealthy pod/e2e-mt-6b983e02-openshift-default-f8946f76f-76pr7 Startup probe failed: Get "http://10.134.0.121:15021/healthz/ready": dial tcp 10.134.0.121:15021: connect: connection refused 42m Warning ProbeError pod/e2e-mt-6b983e02-openshift-default-f8946f76f-76pr7 Startup probe error: Get "http://10.134.0.121:15021/healthz/ready": dial tcp 10.134.0.121:15021: connect: connection refused... 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-686bbf Scaled up replica set payload-processing-e2e-inf-a-686bbf-7f776f59c7 from 0 to 1 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-686bbf Scaled up replica set payload-pre-processing-e2e-inf-a-686bbf-56f69586dc from 0 to 1 42m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-686bbf-7f776f59c7 Error creating: pods "payload-processing-e2e-inf-a-686bbf-7f776f59c7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-686bbf: serviceaccount "payload-processing-e2e-inf-a-686bbf" not found 42m Warning ProbeError pod/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q Startup probe error: Get "http://10.132.0.130:15021/healthz/ready": dial tcp 10.132.0.130:15021: connect: connection refused... 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-686bbf-56f69586dc Created pod: payload-pre-processing-e2e-inf-a-686bbf-56f69586dc-fczlv 42m Warning ProbeError pod/e2e-mt-6b983e02-openshift-default-f8946f76f-p7dbl Startup probe error: Get "http://10.133.0.217:15021/healthz/ready": dial tcp 10.133.0.217:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q Startup probe failed: Get "http://10.132.0.130:15021/healthz/ready": dial tcp 10.132.0.130:15021: connect: connection refused 42m Normal Started pod/payload-processing-e2e-inf-a-686bbf-7f776f59c7-ssgkm Container started 42m Normal Created pod/payload-processing-e2e-inf-a-686bbf-7f776f59c7-ssgkm Container created 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-686bbf-56f69586dc Error creating: pods "payload-pre-processing-e2e-inf-a-686bbf-56f69586dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-686bbf: serviceaccount "payload-processing-e2e-inf-a-686bbf" not found 42m Normal Pulled pod/payload-processing-e2e-inf-a-686bbf-7f776f59c7-ssgkm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-686bbf-56f69586dc-fczlv Add eth0 [10.134.0.125/23] from ovn-kubernetes 42m Warning Unhealthy pod/e2e-mt-6b983e02-openshift-default-f8946f76f-gdqtv Startup probe failed: Get "http://10.133.0.218:15021/healthz/ready": dial tcp 10.133.0.218:15021: connect: connection refused 42m Warning ProbeError pod/e2e-mt-6b983e02-openshift-default-f8946f76f-gdqtv Startup probe error: Get "http://10.133.0.218:15021/healthz/ready": dial tcp 10.133.0.218:15021: connect: connection refused... 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-686bbf-7f776f59c7 Created pod: payload-processing-e2e-inf-a-686bbf-7f776f59c7-ssgkm 42m Normal AddedInterface pod/payload-processing-e2e-inf-a-686bbf-7f776f59c7-ssgkm Add eth0 [10.134.0.124/23] from ovn-kubernetes 42m Warning Unhealthy pod/e2e-mt-6b983e02-openshift-default-f8946f76f-p7dbl Startup probe failed: Get "http://10.133.0.217:15021/healthz/ready": dial tcp 10.133.0.217:15021: connect: connection refused 42m Normal Pulled pod/payload-pre-processing-e2e-inf-a-686bbf-56f69586dc-fczlv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Started pod/payload-pre-processing-e2e-inf-a-686bbf-56f69586dc-fczlv Container started 42m Normal Created pod/payload-pre-processing-e2e-inf-a-686bbf-56f69586dc-fczlv Container created 42m Normal SuccessfulCreate replicaset/e2e-mt-6b983e02-openshift-default-f8946f76f Created pod: e2e-mt-6b983e02-openshift-default-f8946f76f-28fpg 42m Normal ScalingReplicaSet deployment/e2e-mt-6b983e02-openshift-default Scaled up replica set e2e-mt-6b983e02-openshift-default-f8946f76f from 8 to 9 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-6b983e02-openshift-default New size: 9; reason: 42m Normal Pulled pod/e2e-mt-6b983e02-openshift-default-f8946f76f-28fpg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Started pod/e2e-mt-6b983e02-openshift-default-f8946f76f-28fpg Container started 42m Normal AddedInterface pod/e2e-mt-6b983e02-openshift-default-f8946f76f-28fpg Add eth0 [10.134.0.127/23] from ovn-kubernetes 42m Normal Created pod/e2e-mt-6b983e02-openshift-default-f8946f76f-28fpg Container created 42m Warning ProbeError pod/e2e-mt-6b983e02-openshift-default-f8946f76f-28fpg Startup probe error: Get "http://10.134.0.127:15021/healthz/ready": dial tcp 10.134.0.127:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-mt-6b983e02-openshift-default-f8946f76f-28fpg Startup probe failed: Get "http://10.134.0.127:15021/healthz/ready": dial tcp 10.134.0.127:15021: connect: connection refused 42m Normal Killing pod/e2e-api-b-32de96-openshift-default-65db64bf97-8m9pf Stopping container istio-proxy 42m Normal SuccessfulCreate replicaset/e2e-inf-a-c1243c-openshift-default-55ff45c9dc Created pod: e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9 42m Normal AddedLabel gateway/e2e-inf-a-c1243c Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-c1243c 42m Normal ScalingReplicaSet deployment/e2e-inf-a-c1243c-openshift-default Scaled up replica set e2e-inf-a-c1243c-openshift-default-55ff45c9dc from 0 to 1 42m Normal AddedInterface pod/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9 Add eth0 [10.132.0.131/23] from ovn-kubernetes 42m Normal Killing pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc Deleted pod: e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd 42m Normal ScalingReplicaSet deployment/e2e-worker-w0-dd6574-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w0-dd6574-openshift-default-5d877f7ccc from 3 to 2 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-dd6574-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 42m Normal Killing pod/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq Stopping container istio-proxy 42m Warning ProbeError pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd Readiness probe error: HTTP probe failed with statuscode: 503... 42m Warning Unhealthy pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-hsqsd Readiness probe failed: HTTP probe failed with statuscode: 503 42m Normal Killing pod/payload-processing-e2e-mt-6b983e02-66b998455c-t7x6x Stopping container payload-processing 42m Warning Unhealthy pod/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq Readiness probe failed: HTTP probe failed with statuscode: 503 42m Normal Killing pod/payload-pre-processing-e2e-mt-6b983e02-58d76c55bc-hvlc4 Stopping container payload-pre-processing 42m Warning ProbeError pod/e2e-api-b-a1bf7d-openshift-default-7cd996948f-jp4rq Readiness probe error: HTTP probe failed with statuscode: 503... 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-c1243c Scaled up replica set payload-pre-processing-e2e-inf-a-c1243c-55d765c54c from 0 to 1 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-c1243c Scaled up replica set payload-processing-e2e-inf-a-c1243c-6b7b549f89 from 0 to 1 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c Error creating: pods "payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-c1243c: serviceaccount "payload-processing-e2e-inf-a-c1243c" not found 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c Created pod: payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r 41m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r Add eth0 [10.132.0.132/23] from ovn-kubernetes 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-c1243c-6b7b549f89 Created pod: payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv 41m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-c1243c-6b7b549f89 Error creating: pods "payload-processing-e2e-inf-a-c1243c-6b7b549f89-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-c1243c: serviceaccount "payload-processing-e2e-inf-a-c1243c" not found 41m Normal Pulled pod/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv Add eth0 [10.132.0.133/23] from ovn-kubernetes 41m Normal Started pod/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv Container started 41m Normal Started pod/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r Container started 41m Normal Created pod/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r Container created 41m Normal Pulled pod/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv Container created 41m Normal Killing pod/payload-pre-processing-e2e-worker-w0-dd6574-5fd9c947b7-x7c6s Stopping container payload-pre-processing 41m Normal Killing pod/payload-processing-e2e-worker-w0-dd6574-79864967d7-f8wcm Stopping container payload-processing 41m Normal AddedLabel gateway/e2e-inf-b-d4f799 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-d4f799 41m Normal SuccessfulCreate replicaset/e2e-inf-b-d4f799-openshift-default-76569fd959 Created pod: e2e-inf-b-d4f799-openshift-default-76569fd959-phrdn 41m Normal ScalingReplicaSet deployment/e2e-inf-b-d4f799-openshift-default Scaled up replica set e2e-inf-b-d4f799-openshift-default-76569fd959 from 0 to 1 41m Normal AddedInterface pod/e2e-inf-b-d4f799-openshift-default-76569fd959-phrdn Add eth0 [10.133.0.223/23] from ovn-kubernetes 41m Normal Pulled pod/e2e-inf-a-7cc1b1-openshift-default-84d569b9c-pkbzx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-inf-a-7cc1b1-openshift-default-84d569b9c-pkbzx Container created 41m Normal Started pod/e2e-inf-a-7cc1b1-openshift-default-84d569b9c-pkbzx Container started 41m Normal ScalingReplicaSet deployment/e2e-inf-a-3566f9-openshift-default Scaled up replica set e2e-inf-a-3566f9-openshift-default-5c89fbd849 from 0 to 1 41m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-dd6574-5fd9c947b7-x7c6s Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 41m Normal AddedLabel gateway/e2e-inf-a-3566f9 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-3566f9 41m Normal SuccessfulCreate replicaset/e2e-inf-a-3566f9-openshift-default-5c89fbd849 Created pod: e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-7cc1b1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning Unhealthy pod/payload-processing-e2e-worker-w0-dd6574-79864967d7-f8wcm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 41m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-d4f799-56b5f49845 Error creating: pods "payload-processing-e2e-inf-b-d4f799-56b5f49845-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-d4f799: serviceaccount "payload-processing-e2e-inf-b-d4f799" not found 41m Normal AddedInterface pod/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db Add eth0 [10.132.0.134/23] from ovn-kubernetes 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-d4f799 Scaled up replica set payload-processing-e2e-inf-b-d4f799-56b5f49845 from 0 to 1 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-d4f799 Scaled up replica set payload-pre-processing-e2e-inf-b-d4f799-54b658bd4c from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-d4f799-54b658bd4c Created pod: payload-pre-processing-e2e-inf-b-d4f799-54b658bd4c-hkmkb 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-d4f799-54b658bd4c Error creating: pods "payload-pre-processing-e2e-inf-b-d4f799-54b658bd4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-d4f799: serviceaccount "payload-processing-e2e-inf-b-d4f799" not found 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-7cc1b1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-d4f799-56b5f49845 Created pod: payload-processing-e2e-inf-b-d4f799-56b5f49845-8vdrg 41m Normal Created pod/payload-pre-processing-e2e-inf-b-d4f799-54b658bd4c-hkmkb Container created 41m Normal Started pod/payload-processing-e2e-inf-b-d4f799-56b5f49845-8vdrg Container started 41m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-d4f799-54b658bd4c-hkmkb Add eth0 [10.133.0.224/23] from ovn-kubernetes 41m Normal Pulled pod/payload-pre-processing-e2e-inf-b-d4f799-54b658bd4c-hkmkb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Pulled pod/payload-processing-e2e-inf-b-d4f799-56b5f49845-8vdrg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Started pod/payload-pre-processing-e2e-inf-b-d4f799-54b658bd4c-hkmkb Container started 41m Normal Created pod/payload-processing-e2e-inf-b-d4f799-56b5f49845-8vdrg Container created 41m Normal AddedInterface pod/payload-processing-e2e-inf-b-d4f799-56b5f49845-8vdrg Add eth0 [10.133.0.225/23] from ovn-kubernetes 41m Warning ProbeError pod/e2e-inf-a-7cc1b1-openshift-default-84d569b9c-pkbzx Startup probe error: Get "http://10.133.0.213:15021/healthz/ready": dial tcp 10.133.0.213:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-inf-a-7cc1b1-openshift-default-84d569b9c-pkbzx Startup probe failed: Get "http://10.133.0.213:15021/healthz/ready": dial tcp 10.133.0.213:15021: connect: connection refused 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-3566f9 Scaled up replica set payload-pre-processing-e2e-inf-a-3566f9-5654cbf979 from 0 to 1 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-3566f9 Scaled up replica set payload-processing-e2e-inf-a-3566f9-6d8fc84c79 from 0 to 1 41m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-3566f9-6d8fc84c79 Error creating: pods "payload-processing-e2e-inf-a-3566f9-6d8fc84c79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-3566f9: serviceaccount "payload-processing-e2e-inf-a-3566f9" not found 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-3566f9-5654cbf979 Error creating: pods "payload-pre-processing-e2e-inf-a-3566f9-5654cbf979-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-3566f9: serviceaccount "payload-processing-e2e-inf-a-3566f9" not found 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-3566f9-6d8fc84c79 Created pod: payload-processing-e2e-inf-a-3566f9-6d8fc84c79-l9lpf 41m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-3566f9-5654cbf979-7v5tp Add eth0 [10.133.0.226/23] from ovn-kubernetes 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-3566f9-5654cbf979 Created pod: payload-pre-processing-e2e-inf-a-3566f9-5654cbf979-7v5tp 41m Normal Started pod/payload-pre-processing-e2e-inf-a-3566f9-5654cbf979-7v5tp Container started 41m Normal Pulled pod/payload-pre-processing-e2e-inf-a-3566f9-5654cbf979-7v5tp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Started pod/payload-processing-e2e-inf-a-3566f9-6d8fc84c79-l9lpf Container started 41m Normal Created pod/payload-processing-e2e-inf-a-3566f9-6d8fc84c79-l9lpf Container created 41m Normal Pulled pod/payload-processing-e2e-inf-a-3566f9-6d8fc84c79-l9lpf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/payload-processing-e2e-inf-a-3566f9-6d8fc84c79-l9lpf Add eth0 [10.133.0.227/23] from ovn-kubernetes 41m Normal Created pod/payload-pre-processing-e2e-inf-a-3566f9-5654cbf979-7v5tp Container created 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-7cc1b1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-7cc1b1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-3566f9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-3566f9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal Created pod/e2e-inf-a-686bbf-openshift-default-65b6c69fbc-vwvdd Container created 40m Normal Started pod/e2e-inf-a-686bbf-openshift-default-65b6c69fbc-vwvdd Container started 40m Normal Pulled pod/e2e-inf-a-686bbf-openshift-default-65b6c69fbc-vwvdd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Pulled pod/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk Container created 40m Normal Started pod/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk Container started 40m Warning ProbeError pod/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk Startup probe error: Get "http://10.132.0.129:15021/healthz/ready": dial tcp 10.132.0.129:15021: connect: connection refused... 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-686bbf-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning ProbeError pod/e2e-inf-a-686bbf-openshift-default-65b6c69fbc-vwvdd Startup probe error: Get "http://10.134.0.120:15021/healthz/ready": dial tcp 10.134.0.120:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-inf-a-686bbf-openshift-default-65b6c69fbc-vwvdd Startup probe failed: Get "http://10.134.0.120:15021/healthz/ready": dial tcp 10.134.0.120:15021: connect: connection refused 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-686bbf-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning Unhealthy pod/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk Startup probe failed: Get "http://10.132.0.129:15021/healthz/ready": dial tcp 10.132.0.129:15021: connect: connection refused 40m Normal Killing pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-9bjf7 Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w0-dd6574-openshift-default-5d877f7ccc-255tp Stopping container istio-proxy 40m Normal AddedLabel gateway/e2e-api-b-3d2b29 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-3d2b29 40m Normal SuccessfulCreate replicaset/e2e-api-b-3d2b29-openshift-default-6dd77d6d55 Created pod: e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4 40m Normal AddedInterface pod/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4 Add eth0 [10.132.0.135/23] from ovn-kubernetes 40m Normal ScalingReplicaSet deployment/e2e-api-b-3d2b29-openshift-default Scaled up replica set e2e-api-b-3d2b29-openshift-default-6dd77d6d55 from 0 to 1 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-d0946d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-d0946d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-686bbf-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-3d2b29-7d6d7767 Created pod: payload-pre-processing-e2e-api-b-3d2b29-7d6d7767-724mf 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-3d2b29 Scaled up replica set payload-pre-processing-e2e-api-b-3d2b29-7d6d7767 from 0 to 1 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-686bbf-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-3d2b29 Scaled up replica set payload-processing-e2e-api-b-3d2b29-568d868777 from 0 to 1 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-3d2b29-7d6d7767 Error creating: pods "payload-pre-processing-e2e-api-b-3d2b29-7d6d7767-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-3d2b29: serviceaccount "payload-processing-e2e-api-b-3d2b29" not found 40m Warning FailedCreate replicaset/payload-processing-e2e-api-b-3d2b29-568d868777 Error creating: pods "payload-processing-e2e-api-b-3d2b29-568d868777-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-3d2b29: serviceaccount "payload-processing-e2e-api-b-3d2b29" not found 40m Normal Started pod/payload-pre-processing-e2e-api-b-3d2b29-7d6d7767-724mf Container started 40m Normal Pulled pod/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-3d2b29-568d868777 Created pod: payload-processing-e2e-api-b-3d2b29-568d868777-xbm67 40m Normal AddedInterface pod/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67 Add eth0 [10.132.0.136/23] from ovn-kubernetes 40m Normal Created pod/payload-pre-processing-e2e-api-b-3d2b29-7d6d7767-724mf Container created 40m Normal Pulled pod/payload-pre-processing-e2e-api-b-3d2b29-7d6d7767-724mf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-3d2b29-7d6d7767-724mf Add eth0 [10.133.0.228/23] from ovn-kubernetes 40m Normal Created pod/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67 Container created 40m Normal Started pod/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67 Container started 40m Normal Pulled pod/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Started pod/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9 Container started 40m Normal Created pod/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9 Container created 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-c1243c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-c1243c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal Killing pod/e2e-mt-6b983e02-openshift-default-f8946f76f-cn65s Stopping container istio-proxy 40m Normal Killing pod/e2e-mt-6b983e02-openshift-default-f8946f76f-28fpg Stopping container istio-proxy 40m Normal Killing pod/e2e-mt-6b983e02-openshift-default-f8946f76f-hgtfw Stopping container istio-proxy 40m Normal Killing pod/e2e-mt-6b983e02-openshift-default-f8946f76f-p7dbl Stopping container istio-proxy 40m Normal Killing pod/e2e-mt-6b983e02-openshift-default-f8946f76f-gdqtv Stopping container istio-proxy 40m Normal Killing pod/e2e-mt-6b983e02-openshift-default-f8946f76f-77r6q Stopping container istio-proxy 40m Normal Killing pod/e2e-mt-6b983e02-openshift-default-f8946f76f-d9dcg Stopping container istio-proxy 40m Normal Killing pod/e2e-mt-6b983e02-openshift-default-f8946f76f-76pr7 Stopping container istio-proxy 40m Normal Killing pod/e2e-mt-6b983e02-openshift-default-f8946f76f-7ffnz Stopping container istio-proxy 40m Warning Unhealthy pod/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9 Startup probe failed: Get "http://10.132.0.131:15021/healthz/ready": dial tcp 10.132.0.131:15021: connect: connection refused 40m Warning ProbeError pod/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9 Startup probe error: Get "http://10.132.0.131:15021/healthz/ready": dial tcp 10.132.0.131:15021: connect: connection refused... 40m Normal SuccessfulCreate replicaset/e2e-inf-b-11537e-openshift-default-7fc77994d5 Created pod: e2e-inf-b-11537e-openshift-default-7fc77994d5-9hpxw 40m Normal ScalingReplicaSet deployment/e2e-inf-b-11537e-openshift-default Scaled up replica set e2e-inf-b-11537e-openshift-default-7fc77994d5 from 0 to 1 40m Normal AddedLabel gateway/e2e-inf-b-11537e Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-11537e 40m Normal AddedInterface pod/e2e-inf-b-11537e-openshift-default-7fc77994d5-9hpxw Add eth0 [10.133.0.229/23] from ovn-kubernetes 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9 Error creating: pods "payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-11537e: serviceaccount "payload-processing-e2e-inf-b-11537e" not found 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-11537e Scaled up replica set payload-processing-e2e-inf-b-11537e-59489cb799 from 0 to 1 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-11537e Scaled up replica set payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9 from 0 to 1 39m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-11537e-59489cb799 Error creating: pods "payload-processing-e2e-inf-b-11537e-59489cb799-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-11537e: serviceaccount "payload-processing-e2e-inf-b-11537e" not found 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9 Created pod: payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-11537e-59489cb799 Created pod: payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd 39m Normal Started pod/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q Container started 39m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q Add eth0 [10.132.0.137/23] from ovn-kubernetes 39m Normal Pulled pod/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Pulled pod/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Created pod/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd Container created 39m Normal Created pod/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q Container created 39m Normal AddedInterface pod/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd Add eth0 [10.132.0.138/23] from ovn-kubernetes 39m Normal Started pod/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd Container started 39m Normal Killing pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-dxqcj Stopping container istio-proxy 39m Normal SuccessfulDelete replicaset/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f Deleted pod: e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-dxqcj 39m Normal ScalingReplicaSet deployment/e2e-worker-w1-c2dd3b-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f from 2 to 1 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-c2dd3b-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-284b8e-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 39m Warning Unhealthy pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-dxqcj Readiness probe failed: HTTP probe failed with statuscode: 503 39m Normal Killing pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-64wq6 Stopping container istio-proxy 39m Normal Killing pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-2ssdg Stopping container istio-proxy 39m Normal ScalingReplicaSet deployment/e2e-worker-w6-284b8e-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w6-284b8e-openshift-default-77c5d9855 from 2 to 1 39m Normal SuccessfulDelete replicaset/e2e-worker-w6-284b8e-openshift-default-77c5d9855 Deleted pod: e2e-worker-w6-284b8e-openshift-default-77c5d9855-64wq6 39m Warning ProbeError pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-dxqcj Readiness probe error: HTTP probe failed with statuscode: 503... 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-9a6cf2-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 39m Normal ScalingReplicaSet deployment/e2e-worker-w3-9a6cf2-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c from 2 to 1 39m Normal SuccessfulDelete replicaset/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c Deleted pod: e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-2ssdg 39m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-rp6l2 Readiness probe failed: Get "http://10.133.0.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 39m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-77bfb58f8c Deleted pod: maas-default-gateway-openshift-default-77bfb58f8c-cd928 39m Normal Killing pod/maas-default-gateway-openshift-default-77bfb58f8c-zw295 Stopping container istio-proxy 39m Normal Killing pod/maas-default-gateway-openshift-default-77bfb58f8c-6swr2 Stopping container istio-proxy 39m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-rp6l2 Readiness probe error: Get "http://10.133.0.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 39m Normal Killing pod/maas-default-gateway-openshift-default-77bfb58f8c-cd928 Stopping container istio-proxy 39m Normal ScalingReplicaSet deployment/e2e-inf-b-70d237-openshift-default Scaled up replica set e2e-inf-b-70d237-openshift-default-86895f855 from 0 to 1 39m Normal SuccessfulCreate replicaset/e2e-inf-b-70d237-openshift-default-86895f855 Created pod: e2e-inf-b-70d237-openshift-default-86895f855-4l98n 39m Normal AddedLabel gateway/e2e-inf-b-70d237 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-70d237 39m Normal AddedInterface pod/e2e-inf-b-70d237-openshift-default-86895f855-4l98n Add eth0 [10.133.0.230/23] from ovn-kubernetes 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-c1243c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-zw295 Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-c1243c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-zw295 Readiness probe failed: HTTP probe failed with statuscode: 503 39m Normal Created pod/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db Container created 39m Normal Created pod/e2e-inf-b-d4f799-openshift-default-76569fd959-phrdn Container created 39m Normal Pulled pod/e2e-inf-b-d4f799-openshift-default-76569fd959-phrdn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Pulled pod/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Started pod/e2e-inf-b-d4f799-openshift-default-76569fd959-phrdn Container started 39m Normal Started pod/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db Container started 39m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-cd928 Readiness probe error: Get "http://10.133.0.201:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 39m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-cd928 Readiness probe failed: Get "http://10.133.0.201:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-70d237 Scaled up replica set payload-processing-e2e-inf-b-70d237-c848cd89 from 0 to 1 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-70d237-7ddc758bff Error creating: pods "payload-pre-processing-e2e-inf-b-70d237-7ddc758bff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-70d237: serviceaccount "payload-processing-e2e-inf-b-70d237" not found 39m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-70d237-c848cd89 Error creating: pods "payload-processing-e2e-inf-b-70d237-c848cd89-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-70d237: serviceaccount "payload-processing-e2e-inf-b-70d237" not found 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-70d237-7ddc758bff Created pod: payload-pre-processing-e2e-inf-b-70d237-7ddc758bff-6kzcg 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-70d237 Scaled up replica set payload-pre-processing-e2e-inf-b-70d237-7ddc758bff from 0 to 1 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-70d237-c848cd89 Created pod: payload-processing-e2e-inf-b-70d237-c848cd89-hph72 39m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-70d237-7ddc758bff-6kzcg Add eth0 [10.133.0.231/23] from ovn-kubernetes 39m Normal Started pod/payload-pre-processing-e2e-inf-b-70d237-7ddc758bff-6kzcg Container started 39m Normal Created pod/payload-pre-processing-e2e-inf-b-70d237-7ddc758bff-6kzcg Container created 39m Normal AddedInterface pod/payload-processing-e2e-inf-b-70d237-c848cd89-hph72 Add eth0 [10.133.0.232/23] from ovn-kubernetes 39m Normal Pulled pod/payload-processing-e2e-inf-b-70d237-c848cd89-hph72 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Created pod/payload-processing-e2e-inf-b-70d237-c848cd89-hph72 Container created 39m Normal Pulled pod/payload-pre-processing-e2e-inf-b-70d237-7ddc758bff-6kzcg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Started pod/payload-processing-e2e-inf-b-70d237-c848cd89-hph72 Container started 39m Warning Unhealthy pod/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db Startup probe failed: Get "http://10.132.0.134:15021/healthz/ready": dial tcp 10.132.0.134:15021: connect: connection refused 39m Warning ProbeError pod/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db Startup probe error: Get "http://10.132.0.134:15021/healthz/ready": dial tcp 10.132.0.134:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-inf-b-d4f799-openshift-default-76569fd959-phrdn Startup probe failed: Get "http://10.133.0.223:15021/healthz/ready": dial tcp 10.133.0.223:15021: connect: connection refused 39m Warning ProbeError pod/e2e-inf-b-d4f799-openshift-default-76569fd959-phrdn Startup probe error: Get "http://10.133.0.223:15021/healthz/ready": dial tcp 10.133.0.223:15021: connect: connection refused... 39m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-77bfb58f8c (combined from similar events): Created pod: maas-default-gateway-openshift-default-77bfb58f8c-s5kwr 39m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-s5kwr Container created 39m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-kch8g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-s5kwr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-662qj Container started 39m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-662qj Container created 39m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-662qj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-662qj Add eth0 [10.133.0.233/23] from ovn-kubernetes 39m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-s5kwr Container started 39m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-kch8g Add eth0 [10.133.0.234/23] from ovn-kubernetes 39m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-s5kwr Add eth0 [10.133.0.235/23] from ovn-kubernetes 39m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-kch8g Container created 39m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-kch8g Container started 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-d4f799-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-d4f799-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-kch8g Startup probe error: Get "http://10.133.0.234:15021/healthz/ready": dial tcp 10.133.0.234:15021: connect: connection refused... 39m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-kch8g Startup probe failed: Get "http://10.133.0.234:15021/healthz/ready": dial tcp 10.133.0.234:15021: connect: connection refused 39m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-s5kwr Startup probe failed: Get "http://10.133.0.235:15021/healthz/ready": dial tcp 10.133.0.235:15021: connect: connection refused 39m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-s5kwr Startup probe error: Get "http://10.133.0.235:15021/healthz/ready": dial tcp 10.133.0.235:15021: connect: connection refused... 39m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-662qj Startup probe failed: Get "http://10.133.0.233:15021/healthz/ready": dial tcp 10.133.0.233:15021: connect: connection refused 39m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-662qj Startup probe error: Get "http://10.133.0.233:15021/healthz/ready": dial tcp 10.133.0.233:15021: connect: connection refused... 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-3566f9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-3566f9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-d4f799-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-d4f799-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Normal AddedLabel gateway/e2e-inf-b-bcb857 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-bcb857 38m Normal SuccessfulCreate replicaset/e2e-inf-b-bcb857-openshift-default-f5cd4c5f Created pod: e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2 38m Normal AddedInterface pod/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2 Add eth0 [10.132.0.139/23] from ovn-kubernetes 38m Normal ScalingReplicaSet deployment/e2e-inf-b-bcb857-openshift-default Scaled up replica set e2e-inf-b-bcb857-openshift-default-f5cd4c5f from 0 to 1 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-bcb857-55c67bc8f9 Error creating: pods "payload-pre-processing-e2e-inf-b-bcb857-55c67bc8f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-bcb857: serviceaccount "payload-processing-e2e-inf-b-bcb857" not found 38m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-bcb857-6bb8b97657 Error creating: pods "payload-processing-e2e-inf-b-bcb857-6bb8b97657-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-bcb857: serviceaccount "payload-processing-e2e-inf-b-bcb857" not found 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-bcb857 Scaled up replica set payload-pre-processing-e2e-inf-b-bcb857-55c67bc8f9 from 0 to 1 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-bcb857-55c67bc8f9 Created pod: payload-pre-processing-e2e-inf-b-bcb857-55c67bc8f9-44fjz 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-bcb857 Scaled up replica set payload-processing-e2e-inf-b-bcb857-6bb8b97657 from 0 to 1 38m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-bcb857-55c67bc8f9-44fjz Add eth0 [10.133.0.237/23] from ovn-kubernetes 38m Normal Started pod/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n Container started 38m Normal Created pod/payload-pre-processing-e2e-inf-b-bcb857-55c67bc8f9-44fjz Container created 38m Normal AddedInterface pod/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n Add eth0 [10.132.0.140/23] from ovn-kubernetes 38m Normal Created pod/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n Container created 38m Normal Started pod/payload-pre-processing-e2e-inf-b-bcb857-55c67bc8f9-44fjz Container started 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-bcb857-6bb8b97657 Created pod: payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n 38m Normal Pulled pod/payload-pre-processing-e2e-inf-b-bcb857-55c67bc8f9-44fjz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Pulled pod/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Killing pod/payload-processing-e2e-worker-w5-ef6b43-74fb5cfdbf-n4kdw Stopping container payload-processing 38m Normal Killing pod/payload-pre-processing-e2e-worker-w5-ef6b43-fb6db9dc5-8kf8j Stopping container payload-pre-processing 38m Warning ProbeError pod/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4 Readiness probe error: Get "http://10.132.0.135:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning Unhealthy pod/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4 Readiness probe failed: Get "http://10.132.0.135:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Normal Started pod/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4 Container started 38m Normal Created pod/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4 Container created 38m Normal Pulled pod/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal SuccessfulDelete replicaset/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf Deleted pod: e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-7ntr6 38m Normal Killing pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-7ntr6 Stopping container istio-proxy 38m Normal ScalingReplicaSet deployment/e2e-worker-w4-4e2b27-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w4-4e2b27-openshift-default-996dc4fdf from 2 to 1 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-4e2b27-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 38m Warning Unhealthy pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-7ntr6 Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning ProbeError pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-7ntr6 Readiness probe error: HTTP probe failed with statuscode: 503... 38m Warning ProbeError pod/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4 Startup probe error: Get "http://10.132.0.135:15021/healthz/ready": dial tcp 10.132.0.135:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4 Startup probe failed: Get "http://10.132.0.135:15021/healthz/ready": dial tcp 10.132.0.135:15021: connect: connection refused 38m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w5-ef6b43-fb6db9dc5-8kf8j Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 38m Warning Unhealthy pod/payload-processing-e2e-worker-w5-ef6b43-74fb5cfdbf-n4kdw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-3d2b29-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-3d2b29-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-sz8lg Readiness probe error: Get "http://10.134.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-sz8lg Readiness probe failed: Get "http://10.134.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning ProbeError pod/e2e-inf-b-11537e-openshift-default-7fc77994d5-9hpxw Readiness probe error: Get "http://10.133.0.229:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning Unhealthy pod/e2e-inf-b-11537e-openshift-default-7fc77994d5-9hpxw Readiness probe failed: Get "http://10.133.0.229:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Normal Started pod/e2e-inf-b-70d237-openshift-default-86895f855-4l98n Container started 37m Normal Pulled pod/e2e-inf-b-70d237-openshift-default-86895f855-4l98n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Created pod/e2e-inf-b-70d237-openshift-default-86895f855-4l98n Container created 37m Normal Started pod/e2e-inf-b-11537e-openshift-default-7fc77994d5-9hpxw Container started 37m Normal Created pod/e2e-inf-b-11537e-openshift-default-7fc77994d5-9hpxw Container created 37m Normal Pulled pod/e2e-inf-b-11537e-openshift-default-7fc77994d5-9hpxw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-70d237-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-70d237-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning Unhealthy pod/e2e-inf-b-70d237-openshift-default-86895f855-4l98n Startup probe failed: Get "http://10.133.0.230:15021/healthz/ready": dial tcp 10.133.0.230:15021: connect: connection refused 37m Warning Unhealthy pod/e2e-inf-b-11537e-openshift-default-7fc77994d5-9hpxw Startup probe failed: Get "http://10.133.0.229:15021/healthz/ready": dial tcp 10.133.0.229:15021: connect: connection refused 37m Warning ProbeError pod/e2e-inf-b-11537e-openshift-default-7fc77994d5-9hpxw Startup probe error: Get "http://10.133.0.229:15021/healthz/ready": dial tcp 10.133.0.229:15021: connect: connection refused... 37m Warning ProbeError pod/e2e-inf-b-70d237-openshift-default-86895f855-4l98n Startup probe error: Get "http://10.133.0.230:15021/healthz/ready": dial tcp 10.133.0.230:15021: connect: connection refused... 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-11537e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-11537e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Normal Killing pod/payload-pre-processing-e2e-api-a-d0946d-8465488fff-wnnmg Stopping container payload-pre-processing 37m Normal Killing pod/payload-processing-e2e-api-a-d0946d-59f4d669d9-k4wgp Stopping container payload-processing 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-70d237-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-70d237-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-11537e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-11537e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal Killing pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-vhzlk Stopping container istio-proxy 36m Normal Killing pod/e2e-worker-w5-ef6b43-openshift-default-679658c459-cl9rj Stopping container istio-proxy 35m Normal Started pod/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2 Container started 35m Normal Created pod/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2 Container created 35m Normal Pulled pod/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Warning ProbeError pod/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2 Startup probe error: Get "http://10.132.0.139:15021/healthz/ready": dial tcp 10.132.0.139:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2 Startup probe failed: Get "http://10.132.0.139:15021/healthz/ready": dial tcp 10.132.0.139:15021: connect: connection refused 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-bcb857-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-bcb857-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-bcb857-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-bcb857-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Normal Killing pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-bbrxc Stopping container istio-proxy 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-4d9ea0-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 35m Normal SuccessfulDelete replicaset/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc Deleted pod: e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-bbrxc 35m Normal ScalingReplicaSet deployment/e2e-worker-w2-4d9ea0-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc from 2 to 1 35m Warning Unhealthy pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-bbrxc Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-bbrxc Readiness probe error: HTTP probe failed with statuscode: 503... 35m Normal Killing pod/e2e-api-a-d0946d-openshift-default-6dc6f68d55-h8lfk Stopping container istio-proxy 34m Normal Killing pod/payload-processing-e2e-api-b-3d2b29-568d868777-xbm67 Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-api-b-3d2b29-7d6d7767-724mf Stopping container payload-pre-processing 34m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-m56v7 Readiness probe error: Get "http://10.133.0.157:8080/ready": dial tcp 10.133.0.157:8080: connect: connection refused... 34m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 34m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 34m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-m56v7 34m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-m56v7 Readiness probe failed: Get "http://10.133.0.157:8080/ready": dial tcp 10.133.0.157:8080: connect: connection refused 34m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-m56v7 Stopping container discovery 32m Normal Killing pod/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv Stopping container payload-pre-processing 32m Normal Killing pod/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57 Stopping container payload-processing 32m Warning Unhealthy pod/payload-processing-e2e-inf-a-7cc1b1-7487bc448c-jrv57 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-7cc1b1-5cdbb97799-mqwtv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Normal Killing pod/maas-default-gateway-openshift-default-77bfb58f8c-rp6l2 Stopping container istio-proxy 32m Normal Killing pod/maas-default-gateway-openshift-default-77bfb58f8c-pjj8k Stopping container istio-proxy 32m Normal Killing pod/maas-default-gateway-openshift-default-77bfb58f8c-662qj Stopping container istio-proxy 32m Normal Killing pod/maas-default-gateway-openshift-default-77bfb58f8c-kch8g Stopping container istio-proxy 32m Normal Killing pod/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 Stopping container istio-proxy 32m Normal Killing pod/maas-default-gateway-openshift-default-77bfb58f8c-mwdzc Stopping container istio-proxy 32m Normal Killing pod/maas-default-gateway-openshift-default-77bfb58f8c-s5kwr Stopping container istio-proxy 32m Normal Killing pod/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4 Stopping container istio-proxy 32m Normal ScalingReplicaSet deployment/e2e-inf-a-c4a6b9-openshift-default Scaled up replica set e2e-inf-a-c4a6b9-openshift-default-65db8fb597 from 0 to 1 32m Normal AddedLabel gateway/e2e-inf-a-c4a6b9 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-c4a6b9 32m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-pjj8k Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-mwdzc Readiness probe failed: HTTP probe failed with statuscode: 503 32m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-mwdzc Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-pjj8k Readiness probe failed: HTTP probe failed with statuscode: 503 32m Normal SuccessfulCreate replicaset/e2e-inf-a-c4a6b9-openshift-default-65db8fb597 Created pod: e2e-inf-a-c4a6b9-openshift-default-65db8fb597-2lkb7 32m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 Readiness probe failed: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused 32m Normal AddedInterface pod/e2e-inf-a-c4a6b9-openshift-default-65db8fb597-2lkb7 Add eth0 [10.133.0.248/23] from ovn-kubernetes 32m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-s27r2 Readiness probe error: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4 Readiness probe failed: HTTP probe failed with statuscode: 503 32m Warning ProbeError pod/e2e-api-b-3d2b29-openshift-default-6dd77d6d55-sxrk4 Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-c4a6b9-68dd84bb55 Error creating: pods "payload-processing-e2e-inf-a-c4a6b9-68dd84bb55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-c4a6b9: serviceaccount "payload-processing-e2e-inf-a-c4a6b9" not found 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-c4a6b9-fcb9b94f9 Created pod: payload-pre-processing-e2e-inf-a-c4a6b9-fcb9b94f9-s642g 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-c4a6b9 Scaled up replica set payload-pre-processing-e2e-inf-a-c4a6b9-fcb9b94f9 from 0 to 1 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-c4a6b9 Scaled up replica set payload-processing-e2e-inf-a-c4a6b9-68dd84bb55 from 0 to 1 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-c4a6b9-fcb9b94f9 Error creating: pods "payload-pre-processing-e2e-inf-a-c4a6b9-fcb9b94f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-c4a6b9: serviceaccount "payload-processing-e2e-inf-a-c4a6b9" not found 32m Normal Started pod/payload-processing-e2e-inf-a-c4a6b9-68dd84bb55-8gv8t Container started 32m Normal Pulled pod/payload-processing-e2e-inf-a-c4a6b9-68dd84bb55-8gv8t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Created pod/payload-processing-e2e-inf-a-c4a6b9-68dd84bb55-8gv8t Container created 32m Normal Pulled pod/payload-pre-processing-e2e-inf-a-c4a6b9-fcb9b94f9-s642g Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-c4a6b9-68dd84bb55 Created pod: payload-processing-e2e-inf-a-c4a6b9-68dd84bb55-8gv8t 32m Normal Created pod/payload-pre-processing-e2e-inf-a-c4a6b9-fcb9b94f9-s642g Container created 32m Normal AddedInterface pod/payload-processing-e2e-inf-a-c4a6b9-68dd84bb55-8gv8t Add eth0 [10.133.1.4/23] from ovn-kubernetes 32m Normal Started pod/payload-pre-processing-e2e-inf-a-c4a6b9-fcb9b94f9-s642g Container started 32m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-c4a6b9-fcb9b94f9-s642g Add eth0 [10.133.1.3/23] from ovn-kubernetes 31m Normal Killing pod/e2e-inf-a-7cc1b1-openshift-default-84d569b9c-pkbzx Stopping container istio-proxy 31m Normal Killing pod/payload-processing-e2e-inf-b-d4f799-56b5f49845-8vdrg Stopping container payload-processing 31m Normal Killing pod/payload-pre-processing-e2e-inf-b-d4f799-54b658bd4c-hkmkb Stopping container payload-pre-processing 30m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-d4f799-54b658bd4c-hkmkb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Warning Unhealthy pod/payload-processing-e2e-inf-b-d4f799-56b5f49845-8vdrg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Normal Killing pod/payload-pre-processing-e2e-inf-a-686bbf-56f69586dc-fczlv Stopping container payload-pre-processing 30m Normal Killing pod/payload-processing-e2e-inf-a-686bbf-7f776f59c7-ssgkm Stopping container payload-processing 30m Normal Started pod/e2e-inf-a-c4a6b9-openshift-default-65db8fb597-2lkb7 Container started 30m Normal Pulled pod/e2e-inf-a-c4a6b9-openshift-default-65db8fb597-2lkb7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-686bbf-56f69586dc-fczlv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Normal Created pod/e2e-inf-a-c4a6b9-openshift-default-65db8fb597-2lkb7 Container created 30m Warning Unhealthy pod/e2e-inf-a-c4a6b9-openshift-default-65db8fb597-2lkb7 Startup probe failed: Get "http://10.133.0.248:15021/healthz/ready": dial tcp 10.133.0.248:15021: connect: connection refused 30m Warning ProbeError pod/e2e-inf-a-c4a6b9-openshift-default-65db8fb597-2lkb7 Startup probe error: Get "http://10.133.0.248:15021/healthz/ready": dial tcp 10.133.0.248:15021: connect: connection refused... 30m Normal Killing pod/e2e-inf-b-d4f799-openshift-default-76569fd959-phrdn Stopping container istio-proxy 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-c4a6b9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-c4a6b9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Normal Killing pod/e2e-inf-a-686bbf-openshift-default-65b6c69fbc-vwvdd Stopping container istio-proxy 29m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-f4a242-6cd88bd755-lkvsn Stopping container payload-pre-processing 29m Normal Killing pod/payload-processing-e2e-shared-a-w2-f4a242-66587fdf75-qghj6 Stopping container payload-processing 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-c4a6b9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-c4a6b9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Normal Killing pod/payload-processing-e2e-inf-b-11537e-59489cb799-gbjhd Stopping container payload-processing 29m Normal Killing pod/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q Stopping container payload-pre-processing 29m Normal SuccessfulCreate replicaset/e2e-inf-b-49c06d-openshift-default-795bb8bd77 Created pod: e2e-inf-b-49c06d-openshift-default-795bb8bd77-pgndv 29m Normal AddedLabel gateway/e2e-inf-b-49c06d Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-49c06d 29m Normal ScalingReplicaSet deployment/e2e-inf-b-49c06d-openshift-default Scaled up replica set e2e-inf-b-49c06d-openshift-default-795bb8bd77 from 0 to 1 29m Normal AddedInterface pod/e2e-inf-b-49c06d-openshift-default-795bb8bd77-pgndv Add eth0 [10.133.1.14/23] from ovn-kubernetes 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-49c06d Scaled up replica set payload-processing-e2e-inf-b-49c06d-85d8b67ff5 from 0 to 1 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-49c06d Scaled up replica set payload-pre-processing-e2e-inf-b-49c06d-67bcf7b5c9 from 0 to 1 29m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-49c06d-67bcf7b5c9-l2spf Add eth0 [10.133.1.16/23] from ovn-kubernetes 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-49c06d-85d8b67ff5 Created pod: payload-processing-e2e-inf-b-49c06d-85d8b67ff5-lks6v 29m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-49c06d-85d8b67ff5 Error creating: pods "payload-processing-e2e-inf-b-49c06d-85d8b67ff5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-49c06d: serviceaccount "payload-processing-e2e-inf-b-49c06d" not found 29m Normal Killing pod/e2e-inf-b-11537e-openshift-default-7fc77994d5-9hpxw Stopping container istio-proxy 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-49c06d-67bcf7b5c9 Created pod: payload-pre-processing-e2e-inf-b-49c06d-67bcf7b5c9-l2spf 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-49c06d-67bcf7b5c9 Error creating: pods "payload-pre-processing-e2e-inf-b-49c06d-67bcf7b5c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-49c06d: serviceaccount "payload-processing-e2e-inf-b-49c06d" not found 29m Normal Pulled pod/payload-pre-processing-e2e-inf-b-49c06d-67bcf7b5c9-l2spf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/payload-processing-e2e-inf-b-49c06d-85d8b67ff5-lks6v Add eth0 [10.133.1.17/23] from ovn-kubernetes 29m Normal Pulled pod/payload-processing-e2e-inf-b-49c06d-85d8b67ff5-lks6v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal Created pod/payload-processing-e2e-inf-b-49c06d-85d8b67ff5-lks6v Container created 29m Normal Started pod/payload-processing-e2e-inf-b-49c06d-85d8b67ff5-lks6v Container started 29m Normal Started pod/payload-pre-processing-e2e-inf-b-49c06d-67bcf7b5c9-l2spf Container started 29m Normal Created pod/payload-pre-processing-e2e-inf-b-49c06d-67bcf7b5c9-l2spf Container created 29m Normal Killing pod/payload-processing-e2e-inf-a-3566f9-6d8fc84c79-l9lpf Stopping container payload-processing 29m Normal Killing pod/payload-pre-processing-e2e-inf-a-3566f9-5654cbf979-7v5tp Stopping container payload-pre-processing 29m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-11537e-7b49bb46f9-cjh4q Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Normal Killing pod/payload-processing-e2e-worker-w1-c2dd3b-65f5c79bd5-z22x2 Stopping container payload-processing 29m Normal Killing pod/payload-pre-processing-e2e-worker-w1-c2dd3b-576654d767-9lckv Stopping container payload-pre-processing 29m Normal Killing pod/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r Stopping container payload-pre-processing 29m Normal Killing pod/payload-processing-e2e-inf-a-c1243c-6b7b549f89-cwzrv Stopping container payload-processing 29m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-3566f9-5654cbf979-7v5tp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w1-c2dd3b-576654d767-9lckv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Warning Unhealthy pod/payload-processing-e2e-worker-w1-c2dd3b-65f5c79bd5-z22x2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Normal Killing pod/e2e-inf-a-3566f9-openshift-default-5c89fbd849-nt9db Stopping container istio-proxy 28m Normal Killing pod/e2e-shared-a-w2-f4a242-openshift-default-5dc7b77669-d6k8v Stopping container istio-proxy 28m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-c1243c-55d765c54c-tqk4r Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Normal Killing pod/payload-processing-e2e-inf-b-bcb857-6bb8b97657-jsm9n Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-inf-b-bcb857-55c67bc8f9-44fjz Stopping container payload-pre-processing 28m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-f22129-7d7ddf8fcf-zdhqh Stopping container payload-pre-processing 28m Normal Killing pod/payload-processing-e2e-shared-b-w2-f22129-75dfd8d647-kjdsq Stopping container payload-processing 28m Normal Killing pod/e2e-inf-a-c1243c-openshift-default-55ff45c9dc-5kdf9 Stopping container istio-proxy 28m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-bcb857-55c67bc8f9-44fjz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Normal Killing pod/payload-pre-processing-e2e-inf-b-70d237-7ddc758bff-6kzcg Stopping container payload-pre-processing 28m Normal Killing pod/payload-processing-e2e-inf-b-70d237-c848cd89-hph72 Stopping container payload-processing 28m Normal Created pod/e2e-inf-b-49c06d-openshift-default-795bb8bd77-pgndv Container created 28m Normal Started pod/e2e-inf-b-49c06d-openshift-default-795bb8bd77-pgndv Container started 28m Normal Pulled pod/e2e-inf-b-49c06d-openshift-default-795bb8bd77-pgndv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Warning Unhealthy pod/e2e-inf-b-49c06d-openshift-default-795bb8bd77-pgndv Startup probe failed: Get "http://10.133.1.14:15021/healthz/ready": dial tcp 10.133.1.14:15021: connect: connection refused 28m Warning ProbeError pod/e2e-inf-b-49c06d-openshift-default-795bb8bd77-pgndv Startup probe error: Get "http://10.133.1.14:15021/healthz/ready": dial tcp 10.133.1.14:15021: connect: connection refused... 28m Normal Killing pod/e2e-inf-b-bcb857-openshift-default-f5cd4c5f-wzkg2 Stopping container istio-proxy 27m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-70d237-7ddc758bff-6kzcg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-49c06d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-49c06d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Normal Killing pod/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg Stopping container istio-proxy 27m Normal Killing pod/payload-pre-processing-e2e-worker-w3-9a6cf2-7d98fcf679-zzqnq Stopping container payload-pre-processing 27m Warning Unhealthy pod/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg Readiness probe failed: HTTP probe failed with statuscode: 503 27m Warning ProbeError pod/e2e-shared-b-w2-f22129-openshift-default-6f55f9b56f-gtbbg Readiness probe error: HTTP probe failed with statuscode: 503... 27m Normal Killing pod/payload-processing-e2e-worker-w3-9a6cf2-5f5bb8bd95-48467 Stopping container payload-processing 27m Warning FailedCreate replicaset/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f Error creating: pods "e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-" is forbidden: error looking up service account openshift-ingress/e2e-worker-w1-c2dd3b-openshift-default: serviceaccount "e2e-worker-w1-c2dd3b-openshift-default" not found 27m Normal ScalingReplicaSet deployment/e2e-worker-w1-c2dd3b-openshift-default Scaled up replica set e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f from 0 to 1 27m Normal Killing pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-lmksp Stopping container istio-proxy 27m Warning Unhealthy pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-lmksp Readiness probe failed: HTTP probe failed with statuscode: 503 27m Warning ProbeError pod/e2e-worker-w1-c2dd3b-openshift-default-55c8f4465f-lmksp Readiness probe error: HTTP probe failed with statuscode: 503... 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-49c06d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-49c06d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 27m Normal Killing pod/e2e-inf-b-70d237-openshift-default-86895f855-4l98n Stopping container istio-proxy 27m Normal Killing pod/payload-pre-processing-e2e-worker-w2-4d9ea0-7c6896dc8c-d9bwg Stopping container payload-pre-processing 27m Normal Killing pod/payload-processing-e2e-worker-w2-4d9ea0-75b8748dc7-bqkgq Stopping container payload-processing 27m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-77bfb58f8c from 3 to 2 27m Normal Killing pod/maas-default-gateway-openshift-default-77bfb58f8c-5dsgz Stopping container istio-proxy 27m Warning Unhealthy pod/payload-processing-e2e-worker-w3-9a6cf2-5f5bb8bd95-48467 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w3-9a6cf2-7d98fcf679-zzqnq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal Killing pod/payload-pre-processing-e2e-worker-w4-4e2b27-987d59d59-4j6sw Stopping container payload-pre-processing 27m Normal Killing pod/payload-processing-e2e-worker-w4-4e2b27-5569b97fc9-q286k Stopping container payload-processing 27m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-4d9ea0-7c6896dc8c-d9bwg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal Killing pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gnxj5 Stopping container istio-proxy 27m Warning Unhealthy pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gnxj5 Readiness probe failed: HTTP probe failed with statuscode: 503 27m Warning ProbeError pod/e2e-worker-w3-9a6cf2-openshift-default-5d598fdc7c-gnxj5 Readiness probe error: HTTP probe failed with statuscode: 503... 27m Normal Killing pod/e2e-worker-w2-4d9ea0-openshift-default-5568bb4fbc-k78zf Stopping container istio-proxy 26m Normal Killing pod/e2e-worker-w4-4e2b27-openshift-default-996dc4fdf-cjfnq Stopping container istio-proxy 23m Normal Killing pod/payload-pre-processing-e2e-inf-a-c4a6b9-fcb9b94f9-s642g Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-inf-a-c4a6b9-68dd84bb55-8gv8t Stopping container payload-processing 22m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-c4a6b9-fcb9b94f9-s642g Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning Unhealthy pod/payload-processing-e2e-inf-a-c4a6b9-68dd84bb55-8gv8t Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Normal Killing pod/e2e-inf-a-c4a6b9-openshift-default-65db8fb597-2lkb7 Stopping container istio-proxy 22m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-c4a6b9-fcb9b94f9-s642g Readiness probe failed: timeout: failed to connect service "10.133.1.3:50052" within 1s: context deadline exceeded 22m Normal Killing pod/payload-processing-e2e-inf-b-49c06d-85d8b67ff5-lks6v Stopping container payload-processing 22m Normal Killing pod/payload-pre-processing-e2e-inf-b-49c06d-67bcf7b5c9-l2spf Stopping container payload-pre-processing 22m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 1; reason: All metrics below target 22m Normal Killing pod/maas-default-gateway-openshift-default-77bfb58f8c-sz8lg Stopping container istio-proxy 22m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-77bfb58f8c (combined from similar events): Deleted pod: maas-default-gateway-openshift-default-77bfb58f8c-sz8lg 22m Warning FailedCreate replicaset/e2e-inf-b-49c06d-openshift-default-795bb8bd77 Error creating: pods "e2e-inf-b-49c06d-openshift-default-795bb8bd77-" is forbidden: error looking up service account openshift-ingress/e2e-inf-b-49c06d-openshift-default: serviceaccount "e2e-inf-b-49c06d-openshift-default" not found 22m Normal ScalingReplicaSet deployment/e2e-inf-b-49c06d-openshift-default Scaled up replica set e2e-inf-b-49c06d-openshift-default-795bb8bd77 from 0 to 1 22m Normal Killing pod/e2e-inf-b-49c06d-openshift-default-795bb8bd77-pgndv Stopping container istio-proxy 22m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-49c06d-67bcf7b5c9-l2spf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning Unhealthy pod/payload-processing-e2e-inf-b-49c06d-85d8b67ff5-lks6v Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Normal Killing pod/payload-pre-processing-e2e-worker-w6-284b8e-548df84bc-wznlk Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t Stopping container payload-processing 21m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w6-284b8e-548df84bc-wznlk Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Normal Killing pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-dz5hw Stopping container istio-proxy 21m Warning Unhealthy pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-dz5hw Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning ProbeError pod/e2e-worker-w6-284b8e-openshift-default-77c5d9855-dz5hw Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning Unhealthy pod/payload-processing-e2e-worker-w6-284b8e-69d467cf85-tp68t Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-zt7w9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-zt7w9 Add eth0 [10.133.1.42/23] from ovn-kubernetes 21m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-zt7w9 Container started 21m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-zt7w9 Container created 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-zt7w9 Startup probe failed: Get "http://10.133.1.42:15021/healthz/ready": dial tcp 10.133.1.42:15021: connect: connection refused 21m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-zt7w9 Startup probe error: Get "http://10.133.1.42:15021/healthz/ready": dial tcp 10.133.1.42:15021: connect: connection refused... 21m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-zt7w9 Startup probe error: Get "http://10.133.1.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-zt7w9 Startup probe failed: Get "http://10.133.1.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 20m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-27kg9 Container started 20m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-kqfwb Add eth0 [10.134.0.145/23] from ovn-kubernetes 20m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-kqfwb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-kqfwb Container created 20m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-27kg9 Add eth0 [10.133.1.44/23] from ovn-kubernetes 20m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-kqfwb Container started 20m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-27kg9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-27kg9 Container created 19m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-kqfwb Startup probe error: Get "http://10.134.0.145:15021/healthz/ready": dial tcp 10.134.0.145:15021: connect: connection refused... 19m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-kqfwb Startup probe failed: Get "http://10.134.0.145:15021/healthz/ready": dial tcp 10.134.0.145:15021: connect: connection refused 19m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-27kg9 Startup probe failed: Get "http://10.133.1.44:15021/healthz/ready": dial tcp 10.133.1.44:15021: connect: connection refused 19m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-27kg9 Startup probe error: Get "http://10.133.1.44:15021/healthz/ready": dial tcp 10.133.1.44:15021: connect: connection refused... 19m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-kqfwb Startup probe failed: Get "http://10.134.0.145:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 19m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-27kg9 Startup probe error: Get "http://10.133.1.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 19m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-27kg9 Startup probe failed: Get "http://10.133.1.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 19m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-kqfwb Startup probe error: Get "http://10.134.0.145:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 19m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-zndhx Add eth0 [10.134.0.146/23] from ovn-kubernetes 19m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k Add eth0 [10.132.0.152/23] from ovn-kubernetes 19m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k Container created 19m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k Container started 19m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-zndhx Container started 19m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-zndhx Container created 19m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-zndhx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default (combined from similar events): New size: 6; reason: 19m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k Startup probe error: Get "http://10.132.0.152:15021/healthz/ready": dial tcp 10.132.0.152:15021: connect: connection refused... 19m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-zndhx Startup probe error: Get "http://10.134.0.146:15021/healthz/ready": dial tcp 10.134.0.146:15021: connect: connection refused... 19m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k Startup probe failed: Get "http://10.132.0.152:15021/healthz/ready": dial tcp 10.132.0.152:15021: connect: connection refused 19m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-zndhx Startup probe failed: Get "http://10.134.0.146:15021/healthz/ready": dial tcp 10.134.0.146:15021: connect: connection refused 19m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 19m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-77bfb58f8c Created pod: maas-default-gateway-openshift-default-77bfb58f8c-hft2d 19m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt Add eth0 [10.132.0.153/23] from ovn-kubernetes 19m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt Container created 19m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt Container started 19m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-hft2d Add eth0 [10.133.1.47/23] from ovn-kubernetes 19m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-hft2d Container started 19m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-hft2d Container created 19m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-hft2d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-hft2d Startup probe failed: Get "http://10.133.1.47:15021/healthz/ready": dial tcp 10.133.1.47:15021: connect: connection refused 18m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-hft2d Startup probe error: Get "http://10.133.1.47:15021/healthz/ready": dial tcp 10.133.1.47:15021: connect: connection refused... 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt Startup probe failed: Get "http://10.132.0.153:15021/healthz/ready": dial tcp 10.132.0.153:15021: connect: connection refused 18m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt Startup probe error: Get "http://10.132.0.153:15021/healthz/ready": dial tcp 10.132.0.153:15021: connect: connection refused... 18m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 18m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default (combined from similar events): Scaled up replica set maas-default-gateway-openshift-default-77bfb58f8c from 8 to 10 18m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-js4dr Add eth0 [10.133.1.48/23] from ovn-kubernetes 18m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-ndztn Container started 18m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-ndztn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Normal Pulled pod/maas-default-gateway-openshift-default-77bfb58f8c-js4dr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Normal AddedInterface pod/maas-default-gateway-openshift-default-77bfb58f8c-ndztn Add eth0 [10.134.0.148/23] from ovn-kubernetes 18m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-js4dr Container created 18m Normal Created pod/maas-default-gateway-openshift-default-77bfb58f8c-ndztn Container created 18m Normal Started pod/maas-default-gateway-openshift-default-77bfb58f8c-js4dr Container started 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-js4dr Startup probe failed: Get "http://10.133.1.48:15021/healthz/ready": dial tcp 10.133.1.48:15021: connect: connection refused 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-ndztn Startup probe failed: Get "http://10.134.0.148:15021/healthz/ready": dial tcp 10.134.0.148:15021: connect: connection refused 18m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-js4dr Startup probe error: Get "http://10.133.1.48:15021/healthz/ready": dial tcp 10.133.1.48:15021: connect: connection refused... 18m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-ndztn Startup probe error: Get "http://10.134.0.148:15021/healthz/ready": dial tcp 10.134.0.148:15021: connect: connection refused... 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-ndztn Startup probe failed: Get "http://10.134.0.148:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 18m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-ndztn Startup probe error: Get "http://10.134.0.148:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-js4dr Startup probe failed: Get "http://10.133.1.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 18m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-js4dr Startup probe error: Get "http://10.133.1.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 17m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k Readiness probe error: Get "http://10.132.0.152:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 17m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-6wt7k Readiness probe failed: Get "http://10.132.0.152:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 17m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-zndhx Readiness probe failed: Get "http://10.134.0.146:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 17m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-zndhx Readiness probe error: Get "http://10.134.0.146:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw Readiness probe failed: Get "http://10.132.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 16m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-x8dsw Readiness probe error: Get "http://10.132.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-zt7w9 Readiness probe failed: Get "http://10.133.1.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 15m Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-zt7w9 Readiness probe error: Get "http://10.133.1.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 14m Normal AddedInterface pod/e2e-worker-single-main-d16d3a-openshift-default-784968dbdft7lb9 Add eth0 [10.133.1.59/23] from ovn-kubernetes 14m Normal ScalingReplicaSet deployment/e2e-worker-single-main-d16d3a-openshift-default Scaled up replica set e2e-worker-single-main-d16d3a-openshift-default-784968dbdf from 0 to 1 14m Normal AddedLabel gateway/e2e-worker-single-main-d16d3a Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-d16d3a 14m Normal SuccessfulCreate replicaset/e2e-worker-single-main-d16d3a-openshift-default-784968dbdf Created pod: e2e-worker-single-main-d16d3a-openshift-default-784968dbdft7lb9 14m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-d16d3a-7b8449b599 Created pod: payload-pre-processing-e2e-worker-single-main-d16d3a-7b844t5d5k 14m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-d16d3a Scaled up replica set payload-processing-e2e-worker-single-main-d16d3a-69cd7f47b5 from 0 to 1 14m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-d16d3a-69cd7f47b5 Created pod: payload-processing-e2e-worker-single-main-d16d3a-69cd7f47bb94qp 14m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-d16d3a-69cd7f47b5 Error creating: pods "payload-processing-e2e-worker-single-main-d16d3a-69cd7f47b5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-d16d3a: serviceaccount "payload-processing-e2e-worker-single-main-d16d3a" not found 14m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-d16d3a Scaled up replica set payload-pre-processing-e2e-worker-single-main-d16d3a-7b8449b599 from 0 to 1 14m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-d16d3a-7b8449b599 Error creating: pods "payload-pre-processing-e2e-worker-single-main-d16d3a-7b8449b599-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-d16d3a: serviceaccount "payload-processing-e2e-worker-single-main-d16d3a" not found 14m Normal Created pod/payload-pre-processing-e2e-worker-single-main-d16d3a-7b844t5d5k Container created 14m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-d16d3a-7b844t5d5k Add eth0 [10.133.1.60/23] from ovn-kubernetes 14m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-d16d3a-7b844t5d5k Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 14m Normal Started pod/payload-processing-e2e-worker-single-main-d16d3a-69cd7f47bb94qp Container started 14m Normal Created pod/payload-processing-e2e-worker-single-main-d16d3a-69cd7f47bb94qp Container created 14m Normal Started pod/payload-pre-processing-e2e-worker-single-main-d16d3a-7b844t5d5k Container started 14m Normal Pulled pod/payload-processing-e2e-worker-single-main-d16d3a-69cd7f47bb94qp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 14m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-d16d3a-69cd7f47bb94qp Add eth0 [10.133.1.61/23] from ovn-kubernetes 14m Normal Pulled pod/e2e-worker-single-main-d16d3a-openshift-default-784968dbdft7lb9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 14m Normal Created pod/e2e-worker-single-main-d16d3a-openshift-default-784968dbdft7lb9 Container created 14m Normal Started pod/e2e-worker-single-main-d16d3a-openshift-default-784968dbdft7lb9 Container started 14m Warning ProbeError pod/e2e-worker-single-main-d16d3a-openshift-default-784968dbdft7lb9 Startup probe error: Get "http://10.133.1.59:15021/healthz/ready": dial tcp 10.133.1.59:15021: connect: connection refused... 14m Warning Unhealthy pod/e2e-worker-single-main-d16d3a-openshift-default-784968dbdft7lb9 Startup probe failed: Get "http://10.133.1.59:15021/healthz/ready": dial tcp 10.133.1.59:15021: connect: connection refused 14m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-d16d3a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 14m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-d16d3a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 13m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-d16d3a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 13m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-d16d3a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 13m Normal Killing pod/payload-processing-e2e-worker-single-main-d16d3a-69cd7f47bb94qp Stopping container payload-processing 13m Normal Killing pod/payload-pre-processing-e2e-worker-single-main-d16d3a-7b844t5d5k Stopping container payload-pre-processing 13m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-d16d3a-7b844t5d5k Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 13m Normal Killing pod/e2e-worker-single-main-d16d3a-openshift-default-784968dbdft7lb9 Stopping container istio-proxy 13m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-d16d3a-69cd7f47bb94qp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 10m Normal ScalingReplicaSet deployment/e2e-worker-single-main-c19d19-openshift-default Scaled up replica set e2e-worker-single-main-c19d19-openshift-default-56589d87bf from 0 to 1 10m Normal AddedLabel gateway/e2e-worker-single-main-c19d19 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-c19d19 10m Normal SuccessfulCreate replicaset/e2e-worker-single-main-c19d19-openshift-default-56589d87bf Created pod: e2e-worker-single-main-c19d19-openshift-default-56589d87bf2d6q9 10m Normal AddedInterface pod/e2e-worker-single-main-c19d19-openshift-default-56589d87bf2d6q9 Add eth0 [10.133.1.68/23] from ovn-kubernetes 10m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-c19d19 Scaled up replica set payload-processing-e2e-worker-single-main-c19d19-5bb7bc6d77 from 0 to 1 10m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-c19d19-5bb7bc6d77 Created pod: payload-processing-e2e-worker-single-main-c19d19-5bb7bc6d7ptddr 10m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-c19d19-6df95749d7 Created pod: payload-pre-processing-e2e-worker-single-main-c19d19-6df95ch8kv 10m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-c19d19-6df95749d7 Error creating: pods "payload-pre-processing-e2e-worker-single-main-c19d19-6df95749d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-c19d19: serviceaccount "payload-processing-e2e-worker-single-main-c19d19" not found 10m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-c19d19-5bb7bc6d77 Error creating: pods "payload-processing-e2e-worker-single-main-c19d19-5bb7bc6d77-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-c19d19: serviceaccount "payload-processing-e2e-worker-single-main-c19d19" not found 10m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-c19d19 Scaled up replica set payload-pre-processing-e2e-worker-single-main-c19d19-6df95749d7 from 0 to 1 10m Normal Created pod/payload-pre-processing-e2e-worker-single-main-c19d19-6df95ch8kv Container created 10m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-c19d19-5bb7bc6d7ptddr Add eth0 [10.133.1.70/23] from ovn-kubernetes 10m Normal Started pod/payload-pre-processing-e2e-worker-single-main-c19d19-6df95ch8kv Container started 10m Normal Started pod/payload-processing-e2e-worker-single-main-c19d19-5bb7bc6d7ptddr Container started 10m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-c19d19-6df95ch8kv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 10m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-c19d19-6df95ch8kv Add eth0 [10.133.1.69/23] from ovn-kubernetes 10m Normal Pulled pod/payload-processing-e2e-worker-single-main-c19d19-5bb7bc6d7ptddr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 10m Normal Created pod/payload-processing-e2e-worker-single-main-c19d19-5bb7bc6d7ptddr Container created 10m Normal Created pod/e2e-worker-single-main-c19d19-openshift-default-56589d87bf2d6q9 Container created 10m Normal Started pod/e2e-worker-single-main-c19d19-openshift-default-56589d87bf2d6q9 Container started 10m Normal Pulled pod/e2e-worker-single-main-c19d19-openshift-default-56589d87bf2d6q9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 10m Warning ProbeError pod/e2e-worker-single-main-c19d19-openshift-default-56589d87bf2d6q9 Startup probe error: Get "http://10.133.1.68:15021/healthz/ready": dial tcp 10.133.1.68:15021: connect: connection refused... 10m Warning Unhealthy pod/e2e-worker-single-main-c19d19-openshift-default-56589d87bf2d6q9 Startup probe failed: Get "http://10.133.1.68:15021/healthz/ready": dial tcp 10.133.1.68:15021: connect: connection refused 9m49s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-c19d19-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 9m49s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-c19d19-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 9m34s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-c19d19-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 9m34s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-c19d19-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 8m55s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-c19d19-6df95ch8kv Stopping container payload-pre-processing 8m55s Normal Killing pod/payload-processing-e2e-worker-single-main-c19d19-5bb7bc6d7ptddr Stopping container payload-processing 8m44s Normal Killing pod/e2e-worker-single-main-c19d19-openshift-default-56589d87bf2d6q9 Stopping container istio-proxy 8m33s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-c19d19-5bb7bc6d7ptddr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 8m33s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-c19d19-6df95ch8kv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 7m48s Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-kqfwb Readiness probe error: Get "http://10.134.0.145:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 7m48s Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-27kg9 Readiness probe failed: Get "http://10.133.1.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 7m48s Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-kqfwb Readiness probe failed: Get "http://10.134.0.145:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 7m48s Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-27kg9 Readiness probe error: Get "http://10.133.1.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 7m7s Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt Readiness probe error: Get "http://10.132.0.153:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 7m7s Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-6l5rt Readiness probe failed: Get "http://10.132.0.153:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 7m7s Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-hft2d Readiness probe error: Get "http://10.133.1.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 7m7s Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-hft2d Readiness probe failed: Get "http://10.133.1.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 6m17s Normal AddedLabel gateway/e2e-ait-0abfb3fc Added label istio.io/rev=openshift-gateway to gateway e2e-ait-0abfb3fc 6m16s Normal ScalingReplicaSet deployment/e2e-ait-0abfb3fc-openshift-default Scaled up replica set e2e-ait-0abfb3fc-openshift-default-6b6598d5c9 from 0 to 1 6m16s Normal SuccessfulCreate replicaset/e2e-ait-0abfb3fc-openshift-default-6b6598d5c9 Created pod: e2e-ait-0abfb3fc-openshift-default-6b6598d5c9-wjdtm 6m16s Normal AddedInterface pod/e2e-ait-0abfb3fc-openshift-default-6b6598d5c9-wjdtm Add eth0 [10.133.1.77/23] from ovn-kubernetes 6m10s Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-0abfb3fc Scaled up replica set payload-processing-e2e-ait-0abfb3fc-d894d79c from 0 to 1 6m10s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-0abfb3fc Scaled up replica set payload-pre-processing-e2e-ait-0abfb3fc-56984c987f from 0 to 1 6m10s Warning FailedCreate replicaset/payload-processing-e2e-ait-0abfb3fc-d894d79c Error creating: pods "payload-processing-e2e-ait-0abfb3fc-d894d79c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-0abfb3fc: serviceaccount "payload-processing-e2e-ait-0abfb3fc" not found 6m10s Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-0abfb3fc-56984c987f Error creating: pods "payload-pre-processing-e2e-ait-0abfb3fc-56984c987f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-0abfb3fc: serviceaccount "payload-processing-e2e-ait-0abfb3fc" not found 6m9s Normal AddedInterface pod/payload-processing-e2e-ait-0abfb3fc-d894d79c-v5882 Add eth0 [10.133.1.78/23] from ovn-kubernetes 6m9s Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-0abfb3fc-d894d79c Created pod: payload-processing-e2e-ait-0abfb3fc-d894d79c-v5882 6m9s Normal AddedInterface pod/payload-pre-processing-e2e-ait-0abfb3fc-56984c987f-wd88r Add eth0 [10.133.1.79/23] from ovn-kubernetes 6m9s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-0abfb3fc-56984c987f Created pod: payload-pre-processing-e2e-ait-0abfb3fc-56984c987f-wd88r 6m9s Normal Pulled pod/payload-processing-e2e-ait-0abfb3fc-d894d79c-v5882 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 6m9s Normal Created pod/payload-processing-e2e-ait-0abfb3fc-d894d79c-v5882 Container created 6m9s Normal Started pod/payload-processing-e2e-ait-0abfb3fc-d894d79c-v5882 Container started 6m8s Normal Pulled pod/payload-pre-processing-e2e-ait-0abfb3fc-56984c987f-wd88r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 6m8s Normal Started pod/payload-pre-processing-e2e-ait-0abfb3fc-56984c987f-wd88r Container started 6m8s Normal Created pod/payload-pre-processing-e2e-ait-0abfb3fc-56984c987f-wd88r Container created 5m51s Normal Started pod/e2e-ait-0abfb3fc-openshift-default-6b6598d5c9-wjdtm Container started 5m51s Normal Pulled pod/e2e-ait-0abfb3fc-openshift-default-6b6598d5c9-wjdtm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 5m51s Normal Created pod/e2e-ait-0abfb3fc-openshift-default-6b6598d5c9-wjdtm Container created 5m45s Warning Unhealthy pod/e2e-ait-0abfb3fc-openshift-default-6b6598d5c9-wjdtm Startup probe failed: Get "http://10.133.1.77:15021/healthz/ready": dial tcp 10.133.1.77:15021: connect: connection refused 5m45s Warning ProbeError pod/e2e-ait-0abfb3fc-openshift-default-6b6598d5c9-wjdtm Startup probe error: Get "http://10.133.1.77:15021/healthz/ready": dial tcp 10.133.1.77:15021: connect: connection refused... 5m31s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-0abfb3fc-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 5m31s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-0abfb3fc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 5m30s Normal Killing pod/payload-pre-processing-e2e-ait-0abfb3fc-56984c987f-wd88r Stopping container payload-pre-processing 5m30s Normal Killing pod/payload-processing-e2e-ait-0abfb3fc-d894d79c-v5882 Stopping container payload-processing 5m16s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-0abfb3fc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 5m16s Normal Killing pod/e2e-ait-0abfb3fc-openshift-default-6b6598d5c9-wjdtm Stopping container istio-proxy 5m16s Warning FailedCreate replicaset/e2e-ait-0abfb3fc-openshift-default-6b6598d5c9 Error creating: pods "e2e-ait-0abfb3fc-openshift-default-6b6598d5c9-" is forbidden: error looking up service account openshift-ingress/e2e-ait-0abfb3fc-openshift-default: serviceaccount "e2e-ait-0abfb3fc-openshift-default" not found 5m16s Normal ScalingReplicaSet deployment/e2e-ait-0abfb3fc-openshift-default Scaled up replica set e2e-ait-0abfb3fc-openshift-default-6b6598d5c9 from 0 to 1 5m16s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-0abfb3fc-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 5m14s Warning ProbeError pod/e2e-ait-0abfb3fc-openshift-default-6b6598d5c9-wjdtm Readiness probe error: HTTP probe failed with statuscode: 503... 5m14s Warning Unhealthy pod/e2e-ait-0abfb3fc-openshift-default-6b6598d5c9-wjdtm Readiness probe failed: HTTP probe failed with statuscode: 503 5m14s Normal SuccessfulCreate replicaset/e2e-authcfg-http-d9f155-openshift-default-7c8fbc7f6f Created pod: e2e-authcfg-http-d9f155-openshift-default-7c8fbc7f6f-tbrw9 5m14s Normal ScalingReplicaSet deployment/e2e-authcfg-http-d9f155-openshift-default Scaled up replica set e2e-authcfg-http-d9f155-openshift-default-7c8fbc7f6f from 0 to 1 5m14s Normal AddedLabel gateway/e2e-authcfg-http-d9f155 Added label istio.io/rev=openshift-gateway to gateway e2e-authcfg-http-d9f155 5m13s Normal AddedInterface pod/e2e-authcfg-http-d9f155-openshift-default-7c8fbc7f6f-tbrw9 Add eth0 [10.133.1.81/23] from ovn-kubernetes 5m7s Warning FailedCreate replicaset/payload-pre-processing-e2e-authcfg-http-d9f155-595bf59c4c Error creating: pods "payload-pre-processing-e2e-authcfg-http-d9f155-595bf59c4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-d9f155: serviceaccount "payload-processing-e2e-authcfg-http-d9f155" not found 5m7s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-authcfg-http-d9f155 Scaled up replica set payload-pre-processing-e2e-authcfg-http-d9f155-595bf59c4c from 0 to 1 5m7s Warning FailedCreate replicaset/payload-processing-e2e-authcfg-http-d9f155-57d66cf5bc Error creating: pods "payload-processing-e2e-authcfg-http-d9f155-57d66cf5bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-d9f155: serviceaccount "payload-processing-e2e-authcfg-http-d9f155" not found 5m7s Normal ScalingReplicaSet deployment/payload-processing-e2e-authcfg-http-d9f155 Scaled up replica set payload-processing-e2e-authcfg-http-d9f155-57d66cf5bc from 0 to 1 5m7s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-authcfg-http-d9f155-595bf59c4c Created pod: payload-pre-processing-e2e-authcfg-http-d9f155-595bf59c4c-7vgc7 5m6s Normal AddedInterface pod/payload-processing-e2e-authcfg-http-d9f155-57d66cf5bc-bh2tj Add eth0 [10.133.1.83/23] from ovn-kubernetes 5m6s Normal Started pod/payload-processing-e2e-authcfg-http-d9f155-57d66cf5bc-bh2tj Container started 5m6s Normal SuccessfulCreate replicaset/payload-processing-e2e-authcfg-http-d9f155-57d66cf5bc Created pod: payload-processing-e2e-authcfg-http-d9f155-57d66cf5bc-bh2tj 5m6s Normal Pulled pod/payload-processing-e2e-authcfg-http-d9f155-57d66cf5bc-bh2tj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m6s Normal Created pod/payload-processing-e2e-authcfg-http-d9f155-57d66cf5bc-bh2tj Container created 5m6s Normal AddedInterface pod/payload-pre-processing-e2e-authcfg-http-d9f155-595bf59c4c-7vgc7 Add eth0 [10.133.1.82/23] from ovn-kubernetes 5m5s Normal Started pod/payload-pre-processing-e2e-authcfg-http-d9f155-595bf59c4c-7vgc7 Container started 5m5s Normal Created pod/payload-pre-processing-e2e-authcfg-http-d9f155-595bf59c4c-7vgc7 Container created 5m5s Normal Pulled pod/payload-pre-processing-e2e-authcfg-http-d9f155-595bf59c4c-7vgc7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 4m46s Normal Started pod/e2e-authcfg-http-d9f155-openshift-default-7c8fbc7f6f-tbrw9 Container started 4m46s Normal Pulled pod/e2e-authcfg-http-d9f155-openshift-default-7c8fbc7f6f-tbrw9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 4m46s Normal Created pod/e2e-authcfg-http-d9f155-openshift-default-7c8fbc7f6f-tbrw9 Container created 4m40s Warning Unhealthy pod/e2e-authcfg-http-d9f155-openshift-default-7c8fbc7f6f-tbrw9 Startup probe failed: Get "http://10.133.1.81:15021/healthz/ready": dial tcp 10.133.1.81:15021: connect: connection refused 4m40s Warning ProbeError pod/e2e-authcfg-http-d9f155-openshift-default-7c8fbc7f6f-tbrw9 Startup probe error: Get "http://10.133.1.81:15021/healthz/ready": dial tcp 10.133.1.81:15021: connect: connection refused... 3m43s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-d9f155-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 3m43s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-d9f155-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 3m11s Normal Killing pod/payload-processing-e2e-authcfg-http-d9f155-57d66cf5bc-bh2tj Stopping container payload-processing 3m11s Normal Killing pod/payload-pre-processing-e2e-authcfg-http-d9f155-595bf59c4c-7vgc7 Stopping container payload-pre-processing 2m53s Normal Killing pod/e2e-authcfg-http-d9f155-openshift-default-7c8fbc7f6f-tbrw9 Stopping container istio-proxy 2m35s Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-ndztn Readiness probe failed: Get "http://10.134.0.148:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m35s Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-ndztn Readiness probe error: Get "http://10.134.0.148:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 2m34s Warning ProbeError pod/maas-default-gateway-openshift-default-77bfb58f8c-js4dr Readiness probe error: Get "http://10.133.1.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 2m34s Warning Unhealthy pod/maas-default-gateway-openshift-default-77bfb58f8c-js4dr Readiness probe failed: Get "http://10.133.1.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)