=== events in openshift-ingress (2026-11-03T19:50:49Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 39m Normal Scheduled pod/payload-processing-e2e-api-b-efd81c-768b7754c9-hrk6f Successfully assigned openshift-ingress/payload-processing-e2e-api-b-efd81c-768b7754c9-hrk6f to ip-10-0-128-95.us-west-2.compute.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-4320e5-57dcbbd995-2cfzs Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-4320e5-57dcbbd995-2cfzs to ip-10-0-128-95.us-west-2.compute.internal 60m Normal Scheduled pod/payload-pre-processing-7b98bb678c-zrpdh Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-zrpdh to ip-10-0-128-95.us-west-2.compute.internal 61m Normal Scheduled pod/maas-default-gateway-openshift-default-7854949c9-bw6rx Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7854949c9-bw6rx to ip-10-0-143-45.us-west-2.compute.internal 23m Normal Scheduled pod/maas-default-gateway-openshift-default-7854949c9-8pc6b Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7854949c9-8pc6b to ip-10-0-128-95.us-west-2.compute.internal 33m Normal Scheduled pod/maas-default-gateway-openshift-default-7854949c9-8n78f Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7854949c9-8n78f to ip-10-0-128-95.us-west-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-7fnvf Successfully assigned openshift-ingress/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-7fnvf to ip-10-0-143-45.us-west-2.compute.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-7854949c9-c62ln Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c62ln to ip-10-0-128-95.us-west-2.compute.internal 61m Normal Scheduled pod/maas-default-gateway-openshift-default-7854949c9-dffvr Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7854949c9-dffvr to ip-10-0-143-45.us-west-2.compute.internal 6m35s Normal Scheduled pod/e2e-ait-50babf0d-openshift-default-6bc768cbf7-7j7jf Successfully assigned openshift-ingress/e2e-ait-50babf0d-openshift-default-6bc768cbf7-7j7jf to ip-10-0-128-95.us-west-2.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-api-a-e1f8f8-67d6f99995-blb62 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-e1f8f8-67d6f99995-blb62 to ip-10-0-128-95.us-west-2.compute.internal 33m Normal Scheduled pod/e2e-inf-b-942af8-openshift-default-878544c45-bpzgg Successfully assigned openshift-ingress/e2e-inf-b-942af8-openshift-default-878544c45-bpzgg to ip-10-0-139-161.us-west-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-7854949c9-8vt58 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7854949c9-8vt58 to ip-10-0-143-45.us-west-2.compute.internal 61m Normal Scheduled pod/maas-default-gateway-openshift-default-7854949c9-5rsd6 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7854949c9-5rsd6 to ip-10-0-128-95.us-west-2.compute.internal 6m27s Normal Scheduled pod/payload-pre-processing-e2e-ait-50babf0d-7c8c48b5f5-hg85d Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-50babf0d-7c8c48b5f5-hg85d to ip-10-0-128-95.us-west-2.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-ait-6765e083-6df79b4b87-jx2qz Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-6765e083-6df79b4b87-jx2qz to ip-10-0-128-95.us-west-2.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-ait-6f198bfb-587965948c-z9n4x Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-6f198bfb-587965948c-z9n4x to ip-10-0-128-95.us-west-2.compute.internal 36m Normal Scheduled pod/e2e-inf-a-9dd9f2-openshift-default-db6f4568c-k2wkl Successfully assigned openshift-ingress/e2e-inf-a-9dd9f2-openshift-default-db6f4568c-k2wkl to ip-10-0-139-161.us-west-2.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-ait-90774e61-5d6679746c-zph6g Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-90774e61-5d6679746c-zph6g to ip-10-0-128-95.us-west-2.compute.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-api-a-042f66-954b7d67f-8fk7f Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-042f66-954b7d67f-8fk7f to ip-10-0-139-161.us-west-2.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-api-a-ba5674-64fb9d85b7-kq6fs Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-ba5674-64fb9d85b7-kq6fs to ip-10-0-128-95.us-west-2.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-api-a-e1f8f8-9d858999-6gpzs Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-e1f8f8-9d858999-6gpzs to ip-10-0-128-95.us-west-2.compute.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-api-b-69e2b9-66b5bf4547-4xrkq Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-69e2b9-66b5bf4547-4xrkq to ip-10-0-143-45.us-west-2.compute.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-api-b-efd81c-76f4d9c555-hgzvs Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-efd81c-76f4d9c555-hgzvs to ip-10-0-139-161.us-west-2.compute.internal 45m Normal Scheduled pod/e2e-ait-6765e083-openshift-default-8486d6c5fc-5df52 Successfully assigned openshift-ingress/e2e-ait-6765e083-openshift-default-8486d6c5fc-5df52 to ip-10-0-139-161.us-west-2.compute.internal 73m Normal Scheduled pod/router-default-7577979dbf-qm2k5 Successfully assigned openshift-ingress/router-default-7577979dbf-qm2k5 to ip-10-0-128-95.us-west-2.compute.internal 73m Warning FailedScheduling pod/router-default-7577979dbf-qm2k5 no nodes available to schedule pods 79m Warning FailedScheduling pod/router-default-7577979dbf-qm2k5 no nodes available to schedule pods 15m Normal Scheduled pod/e2e-worker-single-main-020814-openshift-default-75d7fd8777hwxkd Successfully assigned openshift-ingress/e2e-worker-single-main-020814-openshift-default-75d7fd8777hwxkd to ip-10-0-128-95.us-west-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-s4jqx Successfully assigned openshift-ingress/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-s4jqx to ip-10-0-143-45.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-rf5ck Successfully assigned openshift-ingress/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-rf5ck to ip-10-0-128-95.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-5ktvr Successfully assigned openshift-ingress/e2e-worker-w6-fba266-openshift-default-5ff997597f-5ktvr to ip-10-0-139-161.us-west-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-cxpf5 Successfully assigned openshift-ingress/e2e-worker-w6-fba266-openshift-default-5ff997597f-cxpf5 to ip-10-0-128-95.us-west-2.compute.internal 44m Normal Scheduled pod/e2e-shared-b-w4-58ac09-openshift-default-74d79cb8cc-g995s Successfully assigned openshift-ingress/e2e-shared-b-w4-58ac09-openshift-default-74d79cb8cc-g995s to ip-10-0-128-95.us-west-2.compute.internal 11m Normal Scheduled pod/e2e-worker-single-main-439a13-openshift-default-75d865948fmgxp8 Successfully assigned openshift-ingress/e2e-worker-single-main-439a13-openshift-default-75d865948fmgxp8 to ip-10-0-128-95.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-cttst Successfully assigned openshift-ingress/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-cttst to ip-10-0-128-95.us-west-2.compute.internal 59m Normal Scheduled pod/payload-processing-e2e-worker-w6-fba266-56687d67d5-5d54f Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-fba266-56687d67d5-5d54f to ip-10-0-128-95.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-6bjb5 Successfully assigned openshift-ingress/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-6bjb5 to ip-10-0-143-45.us-west-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-hsnb4 Successfully assigned openshift-ingress/e2e-worker-w2-32a59b-openshift-default-86974868b5-hsnb4 to ip-10-0-128-95.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-n6kr9 Successfully assigned openshift-ingress/e2e-worker-w2-32a59b-openshift-default-86974868b5-n6kr9 to ip-10-0-139-161.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-ntspd Successfully assigned openshift-ingress/e2e-worker-w2-32a59b-openshift-default-86974868b5-ntspd to ip-10-0-139-161.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-hrs6j Successfully assigned openshift-ingress/e2e-worker-w2-32a59b-openshift-default-86974868b5-hrs6j to ip-10-0-128-95.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-t4rd9 Successfully assigned openshift-ingress/e2e-worker-w2-32a59b-openshift-default-86974868b5-t4rd9 to ip-10-0-139-161.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-ait-6f198bfb-openshift-default-b76b5d689-dgcgz Successfully assigned openshift-ingress/e2e-ait-6f198bfb-openshift-default-b76b5d689-dgcgz to ip-10-0-128-95.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-w296j Successfully assigned openshift-ingress/e2e-worker-w2-32a59b-openshift-default-86974868b5-w296j to ip-10-0-143-45.us-west-2.compute.internal 59m Normal Scheduled pod/payload-processing-e2e-worker-w5-3a7056-7bc4d594f7-wklrp Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-3a7056-7bc4d594f7-wklrp to ip-10-0-128-95.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-cj9vk Successfully assigned openshift-ingress/e2e-worker-w0-9686c3-openshift-default-85685b9d55-cj9vk to ip-10-0-143-45.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2vv6p Successfully assigned openshift-ingress/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2vv6p to ip-10-0-139-161.us-west-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-d7xjn Successfully assigned openshift-ingress/e2e-worker-w6-fba266-openshift-default-5ff997597f-d7xjn to ip-10-0-139-161.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2mq99 Successfully assigned openshift-ingress/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2mq99 to ip-10-0-139-161.us-west-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-wjpvd Successfully assigned openshift-ingress/e2e-worker-w0-9686c3-openshift-default-85685b9d55-wjpvd to ip-10-0-128-95.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-xk4f4 Successfully assigned openshift-ingress/e2e-worker-w0-9686c3-openshift-default-85685b9d55-xk4f4 to ip-10-0-139-161.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-dn2p9 Successfully assigned openshift-ingress/e2e-worker-w6-fba266-openshift-default-5ff997597f-dn2p9 to ip-10-0-143-45.us-west-2.compute.internal 59m Normal Scheduled pod/payload-processing-e2e-worker-w4-2e052a-c6598ff57-9w5t5 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-2e052a-c6598ff57-9w5t5 to ip-10-0-128-95.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-shared-b-w2-b485d8-openshift-default-788458b655-bff6t Successfully assigned openshift-ingress/e2e-shared-b-w2-b485d8-openshift-default-788458b655-bff6t to ip-10-0-128-95.us-west-2.compute.internal 47m Normal Scheduled pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-jdpnw Successfully assigned openshift-ingress/e2e-worker-w6-fba266-openshift-default-5ff997597f-jdpnw to ip-10-0-143-45.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-lh6wf Successfully assigned openshift-ingress/e2e-worker-w6-fba266-openshift-default-5ff997597f-lh6wf to ip-10-0-143-45.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-z4np8 Successfully assigned openshift-ingress/e2e-worker-w0-9686c3-openshift-default-85685b9d55-z4np8 to ip-10-0-128-95.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-zfzp9 Successfully assigned openshift-ingress/e2e-worker-w0-9686c3-openshift-default-85685b9d55-zfzp9 to ip-10-0-143-45.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-xp5mr Successfully assigned openshift-ingress/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-xp5mr to ip-10-0-139-161.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-ait-90774e61-openshift-default-5f66579499-5f2mm Successfully assigned openshift-ingress/e2e-ait-90774e61-openshift-default-5f66579499-5f2mm to ip-10-0-143-45.us-west-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-vdgg5 Successfully assigned openshift-ingress/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-vdgg5 to ip-10-0-128-95.us-west-2.compute.internal 59m Normal Scheduled pod/payload-processing-e2e-worker-w3-a4cacd-656dcc8dd5-2wst7 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-a4cacd-656dcc8dd5-2wst7 to ip-10-0-128-95.us-west-2.compute.internal 47m Normal Scheduled pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-qkdmt Successfully assigned openshift-ingress/e2e-worker-w6-fba266-openshift-default-5ff997597f-qkdmt to ip-10-0-128-95.us-west-2.compute.internal 47m Normal Scheduled pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-ql4lz Successfully assigned openshift-ingress/e2e-worker-w6-fba266-openshift-default-5ff997597f-ql4lz to ip-10-0-128-95.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-tbwhw Successfully assigned openshift-ingress/e2e-worker-w6-fba266-openshift-default-5ff997597f-tbwhw to ip-10-0-128-95.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-wfcwg Successfully assigned openshift-ingress/e2e-worker-w6-fba266-openshift-default-5ff997597f-wfcwg to ip-10-0-139-161.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t7qrn Successfully assigned openshift-ingress/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t7qrn to ip-10-0-143-45.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t4xjk Successfully assigned openshift-ingress/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t4xjk to ip-10-0-143-45.us-west-2.compute.internal 38m Normal Scheduled pod/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-z2f5g Successfully assigned openshift-ingress/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-z2f5g to ip-10-0-143-45.us-west-2.compute.internal 59m Normal Scheduled pod/payload-processing-e2e-worker-w2-32a59b-6bd5df9875-tlgmv Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-32a59b-6bd5df9875-tlgmv to ip-10-0-139-161.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-szxrv Successfully assigned openshift-ingress/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-szxrv to ip-10-0-128-95.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-p87bt Successfully assigned openshift-ingress/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-p87bt to ip-10-0-139-161.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-n5xtx Successfully assigned openshift-ingress/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-n5xtx to ip-10-0-128-95.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-d5ppq Successfully assigned openshift-ingress/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-d5ppq to ip-10-0-139-161.us-west-2.compute.internal 39m Normal Scheduled pod/e2e-api-a-042f66-openshift-default-7859c48d9c-lqvl5 Successfully assigned openshift-ingress/e2e-api-a-042f66-openshift-default-7859c48d9c-lqvl5 to ip-10-0-128-95.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-64gzp Successfully assigned openshift-ingress/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-64gzp to ip-10-0-139-161.us-west-2.compute.internal 40m Normal Scheduled pod/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-rfb2d Successfully assigned openshift-ingress/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-rfb2d to ip-10-0-128-95.us-west-2.compute.internal 67m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-p2sxw Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-p2sxw to ip-10-0-143-45.us-west-2.compute.internal 5m18s Normal Scheduled pod/payload-pre-processing-e2e-authcfg-http-ad43e7-7d7956b4ff-xwtpt Successfully assigned openshift-ingress/payload-pre-processing-e2e-authcfg-http-ad43e7-7d7956b4ff-xwtpt to ip-10-0-128-95.us-west-2.compute.internal 59m Normal Scheduled pod/payload-processing-e2e-worker-w1-355e1a-57bbcfbbb9-ggfc4 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-355e1a-57bbcfbbb9-ggfc4 to ip-10-0-139-161.us-west-2.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-derive-30b92815-c97b7d6c5-gpg5c Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-30b92815-c97b7d6c5-gpg5c to ip-10-0-128-95.us-west-2.compute.internal 38m Normal Scheduled pod/payload-processing-e2e-api-b-69e2b9-558bf74655-wtjfw Successfully assigned openshift-ingress/payload-processing-e2e-api-b-69e2b9-558bf74655-wtjfw to ip-10-0-143-45.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-9kbd6 Successfully assigned openshift-ingress/e2e-worker-w2-32a59b-openshift-default-86974868b5-9kbd6 to ip-10-0-128-95.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-8vrc9 Successfully assigned openshift-ingress/e2e-worker-w2-32a59b-openshift-default-86974868b5-8vrc9 to ip-10-0-143-45.us-west-2.compute.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-65e151-74c97f7547-dzk27 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-65e151-74c97f7547-dzk27 to ip-10-0-143-45.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-4qtm8 Successfully assigned openshift-ingress/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-4qtm8 to ip-10-0-139-161.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-5kgqh Successfully assigned openshift-ingress/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-5kgqh to ip-10-0-139-161.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-api-a-ba5674-openshift-default-767964dd8c-9x84g Successfully assigned openshift-ingress/e2e-api-a-ba5674-openshift-default-767964dd8c-9x84g to ip-10-0-128-95.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-67lkx Successfully assigned openshift-ingress/e2e-worker-w2-32a59b-openshift-default-86974868b5-67lkx to ip-10-0-143-45.us-west-2.compute.internal 59m Normal Scheduled pod/payload-processing-e2e-worker-w0-9686c3-779d4f5849-mnqkq Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-9686c3-779d4f5849-mnqkq to ip-10-0-143-45.us-west-2.compute.internal 10m Normal Scheduled pod/payload-processing-e2e-worker-single-main-439a13-7d55dcff4925rx Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-439a13-7d55dcff4925rx to ip-10-0-128-95.us-west-2.compute.internal 15m Normal Scheduled pod/payload-processing-e2e-worker-single-main-020814-6f56b5c9fdkhk4 Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-020814-6f56b5c9fdkhk4 to ip-10-0-128-95.us-west-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-shared-b-w4-58ac09-c585d4959-gx5f9 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-58ac09-c585d4959-gx5f9 to ip-10-0-143-45.us-west-2.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-b485d8-657bb4cc85-6p2ht Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-b485d8-657bb4cc85-6p2ht to ip-10-0-128-95.us-west-2.compute.internal 40m Normal Scheduled pod/payload-processing-e2e-shared-b-w1-97e020-b5c87c667-vjfmc Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w1-97e020-b5c87c667-vjfmc to ip-10-0-128-95.us-west-2.compute.internal 40m Normal Scheduled pod/payload-processing-e2e-shared-b-w0-a93912-6dc7f84989-48g7r Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-a93912-6dc7f84989-48g7r to ip-10-0-128-95.us-west-2.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-shared-a-w4-9e277a-5dc7dd98bc-dt7fq Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-9e277a-5dc7dd98bc-dt7fq to ip-10-0-139-161.us-west-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-770e67-dbf74bfbf-vhp4v Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-770e67-dbf74bfbf-vhp4v to ip-10-0-128-95.us-west-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-shared-a-w1-bf3e1c-75d744986c-jnxtg Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w1-bf3e1c-75d744986c-jnxtg to ip-10-0-128-95.us-west-2.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-shared-a-w0-ecf40a-5b5dd7dcff-nr722 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-ecf40a-5b5dd7dcff-nr722 to ip-10-0-128-95.us-west-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-mt-5ac14ff6-cf94fcf6c-75dv9 Successfully assigned openshift-ingress/payload-processing-e2e-mt-5ac14ff6-cf94fcf6c-75dv9 to ip-10-0-128-95.us-west-2.compute.internal 27m Normal Scheduled pod/payload-processing-e2e-mt-3d74f801-d4cf58969-zjtsc Successfully assigned openshift-ingress/payload-processing-e2e-mt-3d74f801-d4cf58969-zjtsc to ip-10-0-128-95.us-west-2.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-mt-3d74f801-d4cf58969-wlgw2 Successfully assigned openshift-ingress/payload-processing-e2e-mt-3d74f801-d4cf58969-wlgw2 to ip-10-0-128-95.us-west-2.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-migrate-82cd41ab-f78868cf5-thnf4 Successfully assigned openshift-ingress/payload-processing-e2e-migrate-82cd41ab-f78868cf5-thnf4 to ip-10-0-128-95.us-west-2.compute.internal 41m Normal Scheduled pod/e2e-api-a-e1f8f8-openshift-default-676bdb65df-bsq58 Successfully assigned openshift-ingress/e2e-api-a-e1f8f8-openshift-default-676bdb65df-bsq58 to ip-10-0-128-95.us-west-2.compute.internal 32m Normal Scheduled pod/payload-processing-e2e-inf-b-942af8-78446b5f5f-45fb4 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-942af8-78446b5f5f-45fb4 to ip-10-0-139-161.us-west-2.compute.internal 35m Normal Scheduled pod/payload-processing-e2e-inf-a-9dd9f2-674c89d7cc-s977s Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-9dd9f2-674c89d7cc-s977s to ip-10-0-139-161.us-west-2.compute.internal 37m Normal Scheduled pod/payload-processing-e2e-inf-a-65e151-7cfcd654b7-rg6rd Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-65e151-7cfcd654b7-rg6rd to ip-10-0-143-45.us-west-2.compute.internal 39m Normal Scheduled pod/payload-processing-e2e-inf-a-4320e5-5854dbd4c-bkzr4 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-4320e5-5854dbd4c-bkzr4 to ip-10-0-143-45.us-west-2.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-derive-30b92815-7984849f75-2fvrh Successfully assigned openshift-ingress/payload-processing-e2e-derive-30b92815-7984849f75-2fvrh to ip-10-0-128-95.us-west-2.compute.internal 5m18s Normal Scheduled pod/payload-processing-e2e-authcfg-http-ad43e7-66b7bc6d87-gn4qd Successfully assigned openshift-ingress/payload-processing-e2e-authcfg-http-ad43e7-66b7bc6d87-gn4qd to ip-10-0-128-95.us-west-2.compute.internal 43m Normal Scheduled pod/e2e-shared-a-w1-bf3e1c-openshift-default-b444fc74f-fpcv7 Successfully assigned openshift-ingress/e2e-shared-a-w1-bf3e1c-openshift-default-b444fc74f-fpcv7 to ip-10-0-143-45.us-west-2.compute.internal 65m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-7d45d5c7898tnnp Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-7d45d5c7898tnnp to ip-10-0-128-95.us-west-2.compute.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-9dd9f2-58b64d8db7-7x68t Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-9dd9f2-58b64d8db7-7x68t to ip-10-0-139-161.us-west-2.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-api-a-ba5674-7b8cb7b449-mt424 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-ba5674-7b8cb7b449-mt424 to ip-10-0-128-95.us-west-2.compute.internal 39m Normal Scheduled pod/payload-processing-e2e-api-a-042f66-6f8988d659-hhk2t Successfully assigned openshift-ingress/payload-processing-e2e-api-a-042f66-6f8988d659-hhk2t to ip-10-0-139-161.us-west-2.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-ait-90774e61-5758cd8cfc-mstxt Successfully assigned openshift-ingress/payload-processing-e2e-ait-90774e61-5758cd8cfc-mstxt to ip-10-0-128-95.us-west-2.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-ait-6f198bfb-7dccb8fbcf-r24t4 Successfully assigned openshift-ingress/payload-processing-e2e-ait-6f198bfb-7dccb8fbcf-r24t4 to ip-10-0-139-161.us-west-2.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-ait-6765e083-55c596cf9f-nrbv8 Successfully assigned openshift-ingress/payload-processing-e2e-ait-6765e083-55c596cf9f-nrbv8 to ip-10-0-128-95.us-west-2.compute.internal 6m27s Normal Scheduled pod/payload-processing-e2e-ait-50babf0d-6db96d59f5-8826z Successfully assigned openshift-ingress/payload-processing-e2e-ait-50babf0d-6db96d59f5-8826z to ip-10-0-128-95.us-west-2.compute.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-942af8-6d9ff55467-fwgpd Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-942af8-6d9ff55467-fwgpd to ip-10-0-128-95.us-west-2.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-migrate-82cd41ab-799f447d67-ps6pq Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-82cd41ab-799f447d67-ps6pq to ip-10-0-139-161.us-west-2.compute.internal 38m Normal Scheduled pod/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-f7z62 Successfully assigned openshift-ingress/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-f7z62 to ip-10-0-139-161.us-west-2.compute.internal 38m Normal Scheduled pod/e2e-api-b-69e2b9-openshift-default-545d9ffd69-nhgvx Successfully assigned openshift-ingress/e2e-api-b-69e2b9-openshift-default-545d9ffd69-nhgvx to ip-10-0-139-161.us-west-2.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-mt-3d74f801-67d6d795dc-65gv9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-3d74f801-67d6d795dc-65gv9 to ip-10-0-139-161.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-shared-a-w0-ecf40a-openshift-default-5fc9fb679c-7fhf5 Successfully assigned openshift-ingress/e2e-shared-a-w0-ecf40a-openshift-default-5fc9fb679c-7fhf5 to ip-10-0-128-95.us-west-2.compute.internal 37m Normal Scheduled pod/e2e-inf-a-65e151-openshift-default-5478585457-cxqss Successfully assigned openshift-ingress/e2e-inf-a-65e151-openshift-default-5478585457-cxqss to ip-10-0-143-45.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-592dz Successfully assigned openshift-ingress/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-592dz to ip-10-0-128-95.us-west-2.compute.internal 60m Normal Scheduled pod/payload-processing-55fbbbf9dc-kt9k6 Successfully assigned openshift-ingress/payload-processing-55fbbbf9dc-kt9k6 to ip-10-0-128-95.us-west-2.compute.internal 48m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-79xhw Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-79xhw to ip-10-0-143-45.us-west-2.compute.internal 64m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-qjtdk Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qjtdk to ip-10-0-128-95.us-west-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-7854949c9-fmq4c Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7854949c9-fmq4c to ip-10-0-128-95.us-west-2.compute.internal 33m Normal Scheduled pod/maas-default-gateway-openshift-default-7854949c9-hm546 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7854949c9-hm546 to ip-10-0-139-161.us-west-2.compute.internal 23m Normal Scheduled pod/maas-default-gateway-openshift-default-7854949c9-hvlvx Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7854949c9-hvlvx to ip-10-0-139-161.us-west-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-7854949c9-jr2ss Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7854949c9-jr2ss to ip-10-0-139-161.us-west-2.compute.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-7854949c9-m4ph9 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7854949c9-m4ph9 to ip-10-0-143-45.us-west-2.compute.internal 47m Normal Scheduled pod/maas-default-gateway-openshift-default-7854949c9-m4w4c Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7854949c9-m4w4c to ip-10-0-128-95.us-west-2.compute.internal 59m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-fba266-5df47f6575-qfhx2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-fba266-5df47f6575-qfhx2 to ip-10-0-128-95.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-6tlg2 Successfully assigned openshift-ingress/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-6tlg2 to ip-10-0-128-95.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-c9kw5 Successfully assigned openshift-ingress/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-c9kw5 to ip-10-0-143-45.us-west-2.compute.internal 39m Normal Scheduled pod/e2e-api-b-efd81c-openshift-default-7956bf8cfc-cfjrn Successfully assigned openshift-ingress/e2e-api-b-efd81c-openshift-default-7956bf8cfc-cfjrn to ip-10-0-128-95.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-2t6vs Successfully assigned openshift-ingress/e2e-worker-w2-32a59b-openshift-default-86974868b5-2t6vs to ip-10-0-128-95.us-west-2.compute.internal 33m Normal Scheduled pod/maas-default-gateway-openshift-default-7854949c9-npq2p Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7854949c9-npq2p to ip-10-0-128-95.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-chgfh Successfully assigned openshift-ingress/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-chgfh to ip-10-0-139-161.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-fwfsb Successfully assigned openshift-ingress/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-fwfsb to ip-10-0-128-95.us-west-2.compute.internal 45m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-xl65b Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xl65b to ip-10-0-139-161.us-west-2.compute.internal 59m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-3a7056-68d95fb54c-8kq4m Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-3a7056-68d95fb54c-8kq4m to ip-10-0-143-45.us-west-2.compute.internal 23m Normal Scheduled pod/maas-default-gateway-openshift-default-7854949c9-q7v6d Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7854949c9-q7v6d to ip-10-0-139-161.us-west-2.compute.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-mt-3d74f801-67d6d795dc-lfpcx Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-3d74f801-67d6d795dc-lfpcx to ip-10-0-128-95.us-west-2.compute.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-7854949c9-r6cjm Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7854949c9-r6cjm to ip-10-0-128-95.us-west-2.compute.internal 22m Normal Scheduled pod/maas-default-gateway-openshift-default-7854949c9-tg88d Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7854949c9-tg88d to ip-10-0-128-95.us-west-2.compute.internal 40m Normal Scheduled pod/e2e-shared-b-w0-a93912-openshift-default-65f75877d7-6pjbg Successfully assigned openshift-ingress/e2e-shared-b-w0-a93912-openshift-default-65f75877d7-6pjbg to ip-10-0-128-95.us-west-2.compute.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-7854949c9-v7mn2 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7854949c9-v7mn2 to ip-10-0-143-45.us-west-2.compute.internal 33m Normal Scheduled pod/maas-default-gateway-openshift-default-7854949c9-vdlrm Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7854949c9-vdlrm to ip-10-0-128-95.us-west-2.compute.internal 59m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-2e052a-564f4b8d67-mz5p5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-2e052a-564f4b8d67-mz5p5 to ip-10-0-128-95.us-west-2.compute.internal 5m27s Normal Scheduled pod/e2e-authcfg-http-ad43e7-openshift-default-5d97569dcc-p2ckt Successfully assigned openshift-ingress/e2e-authcfg-http-ad43e7-openshift-default-5d97569dcc-p2ckt to ip-10-0-128-95.us-west-2.compute.internal 44m Normal Scheduled pod/e2e-mt-5ac14ff6-openshift-default-5859bb68c7-d4zd6 Successfully assigned openshift-ingress/e2e-mt-5ac14ff6-openshift-default-5859bb68c7-d4zd6 to ip-10-0-128-95.us-west-2.compute.internal 67m Normal Scheduled pod/maas-default-gateway-openshift-default-7854949c9-vtbjc Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7854949c9-vtbjc to ip-10-0-128-95.us-west-2.compute.internal 65m Normal Scheduled pod/kube-auth-proxy-b8f876b79-4bl78 Successfully assigned openshift-ingress/kube-auth-proxy-b8f876b79-4bl78 to ip-10-0-143-45.us-west-2.compute.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-7854949c9-w6fls Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7854949c9-w6fls to ip-10-0-139-161.us-west-2.compute.internal 46m Normal Scheduled pod/maas-default-gateway-openshift-default-7854949c9-x4nc2 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7854949c9-x4nc2 to ip-10-0-139-161.us-west-2.compute.internal 47m Normal Scheduled pod/maas-default-gateway-openshift-default-7854949c9-zlt64 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zlt64 to ip-10-0-143-45.us-west-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-mt-5ac14ff6-7679584699-qqfnb Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-5ac14ff6-7679584699-qqfnb to ip-10-0-128-95.us-west-2.compute.internal 59m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-a4cacd-57c5b47ff9-kfm57 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-a4cacd-57c5b47ff9-kfm57 to ip-10-0-128-95.us-west-2.compute.internal 34m Normal Scheduled pod/e2e-mt-3d74f801-openshift-default-78ddf467c-kv428 Successfully assigned openshift-ingress/e2e-mt-3d74f801-openshift-default-78ddf467c-kv428 to ip-10-0-128-95.us-west-2.compute.internal 65m Normal Scheduled pod/kube-auth-proxy-b8f876b79-hw55c Successfully assigned openshift-ingress/kube-auth-proxy-b8f876b79-hw55c to ip-10-0-139-161.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-b6d57 Successfully assigned openshift-ingress/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-b6d57 to ip-10-0-139-161.us-west-2.compute.internal 61m Normal Scheduled pod/maas-default-gateway-openshift-default-7854949c9-6t8sd Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7854949c9-6t8sd to ip-10-0-139-161.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-j6l6d Successfully assigned openshift-ingress/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-j6l6d to ip-10-0-128-95.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-8m7jw Successfully assigned openshift-ingress/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-8m7jw to ip-10-0-143-45.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-derive-30b92815-openshift-default-746cd4b59c-9tpf6 Successfully assigned openshift-ingress/e2e-derive-30b92815-openshift-default-746cd4b59c-9tpf6 to ip-10-0-143-45.us-west-2.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w0-ecf40a-857b7d867f-zgpw9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ecf40a-857b7d867f-zgpw9 to ip-10-0-128-95.us-west-2.compute.internal 59m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-32a59b-5b54fcd66c-2nl8z Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-32a59b-5b54fcd66c-2nl8z to ip-10-0-128-95.us-west-2.compute.internal 46m Normal Scheduled pod/e2e-migrate-82cd41ab-gw-openshift-default-7c64cf7b5-x82rn Successfully assigned openshift-ingress/e2e-migrate-82cd41ab-gw-openshift-default-7c64cf7b5-x82rn to ip-10-0-128-95.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-jk6qn Successfully assigned openshift-ingress/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-jk6qn to ip-10-0-139-161.us-west-2.compute.internal 46m Normal Scheduled pod/e2e-shared-a-w4-9e277a-openshift-default-59d89f6f57-gz72j Successfully assigned openshift-ingress/e2e-shared-a-w4-9e277a-openshift-default-59d89f6f57-gz72j to ip-10-0-128-95.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-nkkk8 Successfully assigned openshift-ingress/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-nkkk8 to ip-10-0-143-45.us-west-2.compute.internal 44m Normal Scheduled pod/e2e-shared-a-w2-770e67-openshift-default-974c475f5-dlf59 Successfully assigned openshift-ingress/e2e-shared-a-w2-770e67-openshift-default-974c475f5-dlf59 to ip-10-0-128-95.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-phbjz Successfully assigned openshift-ingress/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-phbjz to ip-10-0-128-95.us-west-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w1-bf3e1c-6bc6cfb875-rhwwt Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w1-bf3e1c-6bc6cfb875-rhwwt to ip-10-0-128-95.us-west-2.compute.internal 59m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-355e1a-5c86d6b775-2kbjt Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-355e1a-5c86d6b775-2kbjt to ip-10-0-128-95.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-jwhh6 Successfully assigned openshift-ingress/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-jwhh6 to ip-10-0-128-95.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-tpqgh Successfully assigned openshift-ingress/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-tpqgh to ip-10-0-128-95.us-west-2.compute.internal 67m Normal Scheduled pod/openshift-ai-inference-openshift-ai-inference-6f8685f4cf-qbp4r Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-6f8685f4cf-qbp4r to ip-10-0-128-95.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-vbkvm Successfully assigned openshift-ingress/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-vbkvm to ip-10-0-143-45.us-west-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-xmnlp Successfully assigned openshift-ingress/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-xmnlp to ip-10-0-143-45.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-z9f5d Successfully assigned openshift-ingress/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-z9f5d to ip-10-0-128-95.us-west-2.compute.internal 39m Normal Scheduled pod/e2e-inf-a-4320e5-openshift-default-656696d5c-2ltcj Successfully assigned openshift-ingress/e2e-inf-a-4320e5-openshift-default-656696d5c-2ltcj to ip-10-0-139-161.us-west-2.compute.internal 47m Normal Scheduled pod/maas-default-gateway-openshift-default-7854949c9-2fxrb Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7854949c9-2fxrb to ip-10-0-139-161.us-west-2.compute.internal 59m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-9686c3-6f6bdb5d59-twjx8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-9686c3-6f6bdb5d59-twjx8 to ip-10-0-128-95.us-west-2.compute.internal 10m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-439a13-95c6dnrtkf Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-439a13-95c6dnrtkf to ip-10-0-128-95.us-west-2.compute.internal 15m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-020814-67dfbqkbgf Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-020814-67dfbqkbgf to ip-10-0-128-95.us-west-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w4-58ac09-7846d7dccf-bch78 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-58ac09-7846d7dccf-bch78 to ip-10-0-143-45.us-west-2.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-b485d8-d5fd6c795-n8wxl Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b485d8-d5fd6c795-n8wxl to ip-10-0-128-95.us-west-2.compute.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w1-97e020-76946549b7-mf7h8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w1-97e020-76946549b7-mf7h8 to ip-10-0-128-95.us-west-2.compute.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w0-a93912-574cd97df5-w5gbw Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a93912-574cd97df5-w5gbw to ip-10-0-128-95.us-west-2.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w4-9e277a-6f7b5d4955-p29sm Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-9e277a-6f7b5d4955-p29sm to ip-10-0-139-161.us-west-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-770e67-85547c9dc-9mtrq Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-770e67-85547c9dc-9mtrq to ip-10-0-128-95.us-west-2.compute.internal 79m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 79m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 79m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-7577979dbf from 0 to 1 79m Normal SuccessfulCreate replicaset/router-default-7577979dbf Created pod: router-default-7577979dbf-qm2k5 79m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 73m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 73m Warning FailedMount pod/router-default-7577979dbf-qm2k5 MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 73m Warning FailedMount pod/router-default-7577979dbf-qm2k5 MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 72m Warning FailedMount pod/router-default-7577979dbf-qm2k5 MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 72m Warning FailedMount pod/router-default-7577979dbf-qm2k5 MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 72m Normal AddedInterface pod/router-default-7577979dbf-qm2k5 Add eth0 [10.132.0.22/23] from ovn-kubernetes 72m Normal Pulled pod/router-default-7577979dbf-qm2k5 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 72m Normal Started pod/router-default-7577979dbf-qm2k5 Container started 72m Normal Created pod/router-default-7577979dbf-qm2k5 Container created 67m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 67m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 67m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-p2sxw 67m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-p2sxw Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 67m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-p2sxw Add eth0 [10.133.0.18/23] from ovn-kubernetes 67m Normal Started pod/istiod-openshift-gateway-7876bfcb74-p2sxw Container started 67m Normal Created pod/istiod-openshift-gateway-7876bfcb74-p2sxw Container created 67m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-p2sxw Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.897s (3.897s including waiting). Image size: 185476477 bytes. 67m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-p2sxw Readiness probe failed: HTTP probe failed with statuscode: 503 67m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-p2sxw Readiness probe error: HTTP probe failed with statuscode: 503... 67m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7854949c9 Created pod: maas-default-gateway-openshift-default-7854949c9-vtbjc 67m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7854949c9 from 0 to 1 67m Normal ScalingReplicaSet deployment/openshift-ai-inference-openshift-ai-inference Scaled up replica set openshift-ai-inference-openshift-ai-inference-6f8685f4cf from 0 to 1 67m Normal AddedInterface pod/openshift-ai-inference-openshift-ai-inference-6f8685f4cf-qbp4r Add eth0 [10.132.0.34/23] from ovn-kubernetes 67m Normal SuccessfulCreate replicaset/openshift-ai-inference-openshift-ai-inference-6f8685f4cf Created pod: openshift-ai-inference-openshift-ai-inference-6f8685f4cf-qbp4r 67m Normal Pulling pod/maas-default-gateway-openshift-default-7854949c9-vtbjc Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 67m Normal AddedInterface pod/maas-default-gateway-openshift-default-7854949c9-vtbjc Add eth0 [10.132.0.33/23] from ovn-kubernetes 67m Normal EnsuringLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensuring load balancer 67m Normal Pulling pod/openshift-ai-inference-openshift-ai-inference-6f8685f4cf-qbp4r Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 67m Normal EnsuredLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensured load balancer 66m Normal Pulled pod/openshift-ai-inference-openshift-ai-inference-6f8685f4cf-qbp4r Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 13.723s (13.723s including waiting). Image size: 183555333 bytes. 66m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-vtbjc Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 13.748s (13.748s including waiting). Image size: 183555333 bytes. 66m Normal Created pod/openshift-ai-inference-openshift-ai-inference-6f8685f4cf-qbp4r Container created 66m Normal Started pod/openshift-ai-inference-openshift-ai-inference-6f8685f4cf-qbp4r Container started 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 65m Normal AddedInterface pod/kube-auth-proxy-b8f876b79-4bl78 Add eth0 [10.133.0.20/23] from ovn-kubernetes 65m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-7d45d5c789 from 0 to 1 65m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-b8f876b79 from 0 to 2 65m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-7d45d5c789 Created pod: data-science-gateway-data-science-gateway-class-7d45d5c7898tnnp 65m Normal SuccessfulCreate replicaset/kube-auth-proxy-b8f876b79 Created pod: kube-auth-proxy-b8f876b79-4bl78 65m Normal SuccessfulCreate replicaset/kube-auth-proxy-b8f876b79 Created pod: kube-auth-proxy-b8f876b79-hw55c 65m Normal Started pod/data-science-gateway-data-science-gateway-class-7d45d5c7898tnnp Container started 65m Normal Created pod/data-science-gateway-data-science-gateway-class-7d45d5c7898tnnp Container created 65m Normal Pulled pod/data-science-gateway-data-science-gateway-class-7d45d5c7898tnnp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-7d45d5c7898tnnp Add eth0 [10.132.0.45/23] from ovn-kubernetes 65m Normal Pulling pod/kube-auth-proxy-b8f876b79-hw55c Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 65m Normal Pulling pod/kube-auth-proxy-b8f876b79-4bl78 Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 65m Normal AddedInterface pod/kube-auth-proxy-b8f876b79-hw55c Add eth0 [10.134.0.24/23] from ovn-kubernetes 65m Normal Pulled pod/kube-auth-proxy-b8f876b79-4bl78 Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.841s (2.841s including waiting). Image size: 188904677 bytes. 65m Normal Created pod/kube-auth-proxy-b8f876b79-4bl78 Container created 65m Normal Started pod/kube-auth-proxy-b8f876b79-4bl78 Container started 65m Normal Started pod/kube-auth-proxy-b8f876b79-hw55c Container started 65m Normal Created pod/kube-auth-proxy-b8f876b79-hw55c Container created 65m Normal Pulled pod/kube-auth-proxy-b8f876b79-hw55c Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 3.736s (3.736s including waiting). Image size: 188904677 bytes. 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 65m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 65m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 64m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-qjtdk 64m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 64m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-qjtdk Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 64m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-qjtdk Add eth0 [10.132.0.51/23] from ovn-kubernetes 64m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-qjtdk Container started 64m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-qjtdk Container created 64m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-qjtdk Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.831s (3.831s including waiting). Image size: 185476477 bytes. 64m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-qjtdk Readiness probe failed: HTTP probe failed with statuscode: 503 64m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-qjtdk Readiness probe error: HTTP probe failed with statuscode: 503... 64m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-p2sxw 64m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 64m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-p2sxw Stopping container discovery 63m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-vtbjc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Started pod/maas-default-gateway-openshift-default-7854949c9-vtbjc Container started 63m Normal Created pod/maas-default-gateway-openshift-default-7854949c9-vtbjc Container created 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 63m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 63m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-vtbjc Startup probe error: Get "http://10.132.0.33:15021/healthz/ready": dial tcp 10.132.0.33:15021: connect: connection refused... 63m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-vtbjc Startup probe failed: Get "http://10.132.0.33:15021/healthz/ready": dial tcp 10.132.0.33:15021: connect: connection refused 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 63m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 63m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7854949c9 Created pod: maas-default-gateway-openshift-default-7854949c9-dffvr 61m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7854949c9 from 1 to 2 61m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 61m Normal Pulling pod/maas-default-gateway-openshift-default-7854949c9-dffvr Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 61m Normal AddedInterface pod/maas-default-gateway-openshift-default-7854949c9-dffvr Add eth0 [10.133.0.31/23] from ovn-kubernetes 61m Normal Started pod/maas-default-gateway-openshift-default-7854949c9-dffvr Container started 61m Normal Created pod/maas-default-gateway-openshift-default-7854949c9-dffvr Container created 61m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-dffvr Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.168s (4.168s including waiting). Image size: 183555333 bytes. 61m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-dffvr Startup probe failed: Get "http://10.133.0.31:15021/healthz/ready": dial tcp 10.133.0.31:15021: connect: connection refused 61m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-dffvr Startup probe error: Get "http://10.133.0.31:15021/healthz/ready": dial tcp 10.133.0.31:15021: connect: connection refused... 61m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-dffvr Startup probe failed: Get "http://10.133.0.31:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-dffvr Startup probe error: Get "http://10.133.0.31:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7854949c9 Created pod: maas-default-gateway-openshift-default-7854949c9-5rsd6 61m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7854949c9 from 2 to 4 61m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7854949c9 Created pod: maas-default-gateway-openshift-default-7854949c9-6t8sd 61m Normal AddedInterface pod/maas-default-gateway-openshift-default-7854949c9-6t8sd Add eth0 [10.134.0.35/23] from ovn-kubernetes 61m Normal Started pod/maas-default-gateway-openshift-default-7854949c9-5rsd6 Container started 61m Normal Pulling pod/maas-default-gateway-openshift-default-7854949c9-6t8sd Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 61m Normal AddedInterface pod/maas-default-gateway-openshift-default-7854949c9-5rsd6 Add eth0 [10.132.0.54/23] from ovn-kubernetes 61m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-5rsd6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Created pod/maas-default-gateway-openshift-default-7854949c9-5rsd6 Container created 61m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-6t8sd Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.029s (4.029s including waiting). Image size: 183555333 bytes. 61m Normal Created pod/maas-default-gateway-openshift-default-7854949c9-6t8sd Container created 61m Normal Started pod/maas-default-gateway-openshift-default-7854949c9-6t8sd Container started 61m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-5rsd6 Startup probe failed: Get "http://10.132.0.54:15021/healthz/ready": dial tcp 10.132.0.54:15021: connect: connection refused 61m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-5rsd6 Startup probe error: Get "http://10.132.0.54:15021/healthz/ready": dial tcp 10.132.0.54:15021: connect: connection refused... 61m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-5rsd6 Startup probe failed: Get "http://10.132.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-5rsd6 Startup probe error: Get "http://10.132.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-6t8sd Startup probe failed: Get "http://10.134.0.35:15021/healthz/ready": dial tcp 10.134.0.35:15021: connect: connection refused 61m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-6t8sd Startup probe error: Get "http://10.134.0.35:15021/healthz/ready": dial tcp 10.134.0.35:15021: connect: connection refused... 61m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-6t8sd Startup probe failed: Get "http://10.134.0.35:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-6t8sd Startup probe error: Get "http://10.134.0.35:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-bw6rx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Created pod/maas-default-gateway-openshift-default-7854949c9-bw6rx Container created 61m Normal Started pod/maas-default-gateway-openshift-default-7854949c9-bw6rx Container started 61m Normal AddedInterface pod/maas-default-gateway-openshift-default-7854949c9-bw6rx Add eth0 [10.133.0.35/23] from ovn-kubernetes 61m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7854949c9 Created pod: maas-default-gateway-openshift-default-7854949c9-bw6rx 61m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-bw6rx Startup probe error: Get "http://10.133.0.35:15021/healthz/ready": dial tcp 10.133.0.35:15021: connect: connection refused... 61m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-bw6rx Startup probe failed: Get "http://10.133.0.35:15021/healthz/ready": dial tcp 10.133.0.35:15021: connect: connection refused 60m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-55fbbbf9dc from 0 to 1 60m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 60m Warning FailedCreate replicaset/payload-processing-55fbbbf9dc Error creating: pods "payload-processing-55fbbbf9dc-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 60m Normal SuccessfulCreate replicaset/payload-processing-55fbbbf9dc Created pod: payload-processing-55fbbbf9dc-kt9k6 60m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 60m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-zrpdh 60m Normal AddedInterface pod/payload-processing-55fbbbf9dc-kt9k6 Add eth0 [10.132.0.57/23] from ovn-kubernetes 60m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-zrpdh Add eth0 [10.132.0.56/23] from ovn-kubernetes 60m Normal Pulling pod/payload-pre-processing-7b98bb678c-zrpdh Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 60m Normal Pulling pod/payload-processing-55fbbbf9dc-kt9k6 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 60m Normal Pulled pod/payload-processing-55fbbbf9dc-kt9k6 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.964s (2.964s including waiting). Image size: 125353963 bytes. 60m Normal Created pod/payload-processing-55fbbbf9dc-kt9k6 Container created 60m Normal Created pod/payload-pre-processing-7b98bb678c-zrpdh Container created 60m Normal Pulled pod/payload-pre-processing-7b98bb678c-zrpdh Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.124s (3.124s including waiting). Image size: 125353963 bytes. 60m Normal Started pod/payload-pre-processing-7b98bb678c-zrpdh Container started 60m Normal Started pod/payload-processing-55fbbbf9dc-kt9k6 Container started 59m Normal AddedLabel gateway/e2e-worker-w1-355e1a Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-355e1a 59m Normal ScalingReplicaSet deployment/e2e-worker-w6-fba266-openshift-default Scaled up replica set e2e-worker-w6-fba266-openshift-default-5ff997597f from 0 to 1 59m Normal AddedLabel gateway/e2e-worker-w3-a4cacd Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-a4cacd 59m Normal SuccessfulCreate replicaset/e2e-worker-w6-fba266-openshift-default-5ff997597f Created pod: e2e-worker-w6-fba266-openshift-default-5ff997597f-cxpf5 59m Normal AddedLabel gateway/e2e-worker-w5-3a7056 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-3a7056 59m Normal ScalingReplicaSet deployment/e2e-worker-w4-2e052a-openshift-default Scaled up replica set e2e-worker-w4-2e052a-openshift-default-6886cf66dc from 0 to 1 59m Normal AddedLabel gateway/e2e-worker-w4-2e052a Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-2e052a 59m Normal AddedLabel gateway/e2e-worker-w0-9686c3 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-9686c3 59m Normal SuccessfulCreate replicaset/e2e-worker-w4-2e052a-openshift-default-6886cf66dc Created pod: e2e-worker-w4-2e052a-openshift-default-6886cf66dc-vdgg5 59m Normal AddedLabel gateway/e2e-worker-w6-fba266 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-fba266 59m Normal AddedLabel gateway/e2e-worker-w2-32a59b Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-32a59b 59m Normal AddedInterface pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-xmnlp Add eth0 [10.133.0.37/23] from ovn-kubernetes 59m Normal AddedInterface pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-cxpf5 Add eth0 [10.132.0.59/23] from ovn-kubernetes 59m Normal ScalingReplicaSet deployment/e2e-worker-w0-9686c3-openshift-default Scaled up replica set e2e-worker-w0-9686c3-openshift-default-85685b9d55 from 0 to 1 59m Normal SuccessfulCreate replicaset/e2e-worker-w0-9686c3-openshift-default-85685b9d55 Created pod: e2e-worker-w0-9686c3-openshift-default-85685b9d55-wjpvd 59m Normal SuccessfulCreate replicaset/e2e-worker-w1-355e1a-openshift-default-d7d9459f9 Created pod: e2e-worker-w1-355e1a-openshift-default-d7d9459f9-7fnvf 59m Normal ScalingReplicaSet deployment/e2e-worker-w1-355e1a-openshift-default Scaled up replica set e2e-worker-w1-355e1a-openshift-default-d7d9459f9 from 0 to 1 59m Normal SuccessfulCreate replicaset/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c Created pod: e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-xmnlp 59m Normal ScalingReplicaSet deployment/e2e-worker-w3-a4cacd-openshift-default Scaled up replica set e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c from 0 to 1 59m Normal AddedInterface pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-vdgg5 Add eth0 [10.132.0.60/23] from ovn-kubernetes 59m Normal ScalingReplicaSet deployment/e2e-worker-w2-32a59b-openshift-default Scaled up replica set e2e-worker-w2-32a59b-openshift-default-86974868b5 from 0 to 1 59m Normal AddedInterface pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-hsnb4 Add eth0 [10.132.0.62/23] from ovn-kubernetes 59m Normal AddedInterface pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-7fnvf Add eth0 [10.133.0.38/23] from ovn-kubernetes 59m Normal ScalingReplicaSet deployment/e2e-worker-w5-3a7056-openshift-default Scaled up replica set e2e-worker-w5-3a7056-openshift-default-85ccfddcf from 0 to 1 59m Normal SuccessfulCreate replicaset/e2e-worker-w5-3a7056-openshift-default-85ccfddcf Created pod: e2e-worker-w5-3a7056-openshift-default-85ccfddcf-s4jqx 59m Normal AddedInterface pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-wjpvd Add eth0 [10.132.0.61/23] from ovn-kubernetes 59m Normal SuccessfulCreate replicaset/e2e-worker-w2-32a59b-openshift-default-86974868b5 Created pod: e2e-worker-w2-32a59b-openshift-default-86974868b5-hsnb4 59m Normal AddedInterface pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-s4jqx Add eth0 [10.133.0.39/23] from ovn-kubernetes 59m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-fba266 Scaled up replica set payload-processing-e2e-worker-w6-fba266-56687d67d5 from 0 to 1 59m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-fba266-5df47f6575 Created pod: payload-pre-processing-e2e-worker-w6-fba266-5df47f6575-qfhx2 59m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-fba266-56687d67d5 Error creating: pods "payload-processing-e2e-worker-w6-fba266-56687d67d5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-fba266: serviceaccount "payload-processing-e2e-worker-w6-fba266" not found 59m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-fba266-56687d67d5 Created pod: payload-processing-e2e-worker-w6-fba266-56687d67d5-5d54f 59m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-fba266-5df47f6575 Error creating: pods "payload-pre-processing-e2e-worker-w6-fba266-5df47f6575-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-fba266: serviceaccount "payload-processing-e2e-worker-w6-fba266" not found 59m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-fba266 Scaled up replica set payload-pre-processing-e2e-worker-w6-fba266-5df47f6575 from 0 to 1 59m Normal Created pod/payload-processing-e2e-worker-w6-fba266-56687d67d5-5d54f Container created 59m Normal AddedInterface pod/payload-processing-e2e-worker-w6-fba266-56687d67d5-5d54f Add eth0 [10.132.0.64/23] from ovn-kubernetes 59m Normal Created pod/payload-pre-processing-e2e-worker-w6-fba266-5df47f6575-qfhx2 Container created 59m Normal Started pod/payload-pre-processing-e2e-worker-w6-fba266-5df47f6575-qfhx2 Container started 59m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-fba266-5df47f6575-qfhx2 Add eth0 [10.132.0.63/23] from ovn-kubernetes 59m Normal Started pod/payload-processing-e2e-worker-w6-fba266-56687d67d5-5d54f Container started 59m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-fba266-5df47f6575-qfhx2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal Pulled pod/payload-processing-e2e-worker-w6-fba266-56687d67d5-5d54f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-2e052a-c6598ff57 Error creating: pods "payload-processing-e2e-worker-w4-2e052a-c6598ff57-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-2e052a: serviceaccount "payload-processing-e2e-worker-w4-2e052a" not found 59m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-2e052a-564f4b8d67 Created pod: payload-pre-processing-e2e-worker-w4-2e052a-564f4b8d67-mz5p5 59m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-2e052a Scaled up replica set payload-processing-e2e-worker-w4-2e052a-c6598ff57 from 0 to 1 59m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-2e052a-564f4b8d67 Error creating: pods "payload-pre-processing-e2e-worker-w4-2e052a-564f4b8d67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-2e052a: serviceaccount "payload-processing-e2e-worker-w4-2e052a" not found 59m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-2e052a Scaled up replica set payload-pre-processing-e2e-worker-w4-2e052a-564f4b8d67 from 0 to 1 59m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-2e052a-c6598ff57 Created pod: payload-processing-e2e-worker-w4-2e052a-c6598ff57-9w5t5 59m Normal Started pod/payload-processing-e2e-worker-w4-2e052a-c6598ff57-9w5t5 Container started 59m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-2e052a-564f4b8d67-mz5p5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal Started pod/payload-pre-processing-e2e-worker-w4-2e052a-564f4b8d67-mz5p5 Container started 59m Normal Created pod/payload-pre-processing-e2e-worker-w4-2e052a-564f4b8d67-mz5p5 Container created 59m Normal AddedInterface pod/payload-processing-e2e-worker-w4-2e052a-c6598ff57-9w5t5 Add eth0 [10.132.0.66/23] from ovn-kubernetes 59m Normal Created pod/payload-processing-e2e-worker-w4-2e052a-c6598ff57-9w5t5 Container created 59m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-2e052a-564f4b8d67-mz5p5 Add eth0 [10.132.0.65/23] from ovn-kubernetes 59m Normal Pulled pod/payload-processing-e2e-worker-w4-2e052a-c6598ff57-9w5t5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-a4cacd Scaled up replica set payload-processing-e2e-worker-w3-a4cacd-656dcc8dd5 from 0 to 1 59m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-a4cacd Scaled up replica set payload-pre-processing-e2e-worker-w3-a4cacd-57c5b47ff9 from 0 to 1 59m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-a4cacd-656dcc8dd5 Created pod: payload-processing-e2e-worker-w3-a4cacd-656dcc8dd5-2wst7 59m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-a4cacd-57c5b47ff9 Error creating: pods "payload-pre-processing-e2e-worker-w3-a4cacd-57c5b47ff9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-a4cacd: serviceaccount "payload-processing-e2e-worker-w3-a4cacd" not found 59m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-a4cacd-57c5b47ff9-kfm57 Add eth0 [10.132.0.67/23] from ovn-kubernetes 59m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-a4cacd-656dcc8dd5 Error creating: pods "payload-processing-e2e-worker-w3-a4cacd-656dcc8dd5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-a4cacd: serviceaccount "payload-processing-e2e-worker-w3-a4cacd" not found 59m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-a4cacd-57c5b47ff9 Created pod: payload-pre-processing-e2e-worker-w3-a4cacd-57c5b47ff9-kfm57 59m Normal Created pod/payload-processing-e2e-worker-w3-a4cacd-656dcc8dd5-2wst7 Container created 59m Normal Started pod/payload-pre-processing-e2e-worker-w3-a4cacd-57c5b47ff9-kfm57 Container started 59m Normal Started pod/payload-processing-e2e-worker-w3-a4cacd-656dcc8dd5-2wst7 Container started 59m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-a4cacd-57c5b47ff9-kfm57 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/payload-processing-e2e-worker-w3-a4cacd-656dcc8dd5-2wst7 Add eth0 [10.132.0.68/23] from ovn-kubernetes 59m Normal Created pod/payload-pre-processing-e2e-worker-w3-a4cacd-57c5b47ff9-kfm57 Container created 59m Normal Pulled pod/payload-processing-e2e-worker-w3-a4cacd-656dcc8dd5-2wst7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-355e1a-5c86d6b775 Created pod: payload-pre-processing-e2e-worker-w1-355e1a-5c86d6b775-2kbjt 59m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-355e1a Scaled up replica set payload-processing-e2e-worker-w1-355e1a-57bbcfbbb9 from 0 to 1 59m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-355e1a-5c86d6b775 Error creating: pods "payload-pre-processing-e2e-worker-w1-355e1a-5c86d6b775-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-355e1a: serviceaccount "payload-processing-e2e-worker-w1-355e1a" not found 59m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-355e1a Scaled up replica set payload-pre-processing-e2e-worker-w1-355e1a-5c86d6b775 from 0 to 1 59m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-355e1a-57bbcfbbb9 Error creating: pods "payload-processing-e2e-worker-w1-355e1a-57bbcfbbb9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-355e1a: serviceaccount "payload-processing-e2e-worker-w1-355e1a" not found 59m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-355e1a-57bbcfbbb9 Created pod: payload-processing-e2e-worker-w1-355e1a-57bbcfbbb9-ggfc4 59m Normal AddedInterface pod/payload-processing-e2e-worker-w1-355e1a-57bbcfbbb9-ggfc4 Add eth0 [10.134.0.37/23] from ovn-kubernetes 59m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-355e1a-5c86d6b775-2kbjt Add eth0 [10.132.0.69/23] from ovn-kubernetes 59m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-355e1a-5c86d6b775-2kbjt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal Created pod/payload-pre-processing-e2e-worker-w1-355e1a-5c86d6b775-2kbjt Container created 59m Normal Started pod/payload-pre-processing-e2e-worker-w1-355e1a-5c86d6b775-2kbjt Container started 59m Normal Pulling pod/payload-processing-e2e-worker-w1-355e1a-57bbcfbbb9-ggfc4 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 59m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-32a59b Scaled up replica set payload-processing-e2e-worker-w2-32a59b-6bd5df9875 from 0 to 1 59m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-32a59b-5b54fcd66c Error creating: pods "payload-pre-processing-e2e-worker-w2-32a59b-5b54fcd66c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-32a59b: serviceaccount "payload-processing-e2e-worker-w2-32a59b" not found 59m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-32a59b-6bd5df9875 Created pod: payload-processing-e2e-worker-w2-32a59b-6bd5df9875-tlgmv 59m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-32a59b-6bd5df9875 Error creating: pods "payload-processing-e2e-worker-w2-32a59b-6bd5df9875-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-32a59b: serviceaccount "payload-processing-e2e-worker-w2-32a59b" not found 59m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-32a59b Scaled up replica set payload-pre-processing-e2e-worker-w2-32a59b-5b54fcd66c from 0 to 1 59m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-32a59b-5b54fcd66c Created pod: payload-pre-processing-e2e-worker-w2-32a59b-5b54fcd66c-2nl8z 59m Normal Created pod/payload-processing-e2e-worker-w1-355e1a-57bbcfbbb9-ggfc4 Container created 59m Normal Started pod/payload-pre-processing-e2e-worker-w2-32a59b-5b54fcd66c-2nl8z Container started 59m Normal Created pod/payload-pre-processing-e2e-worker-w2-32a59b-5b54fcd66c-2nl8z Container created 59m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-32a59b-5b54fcd66c-2nl8z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-32a59b-5b54fcd66c-2nl8z Add eth0 [10.132.0.70/23] from ovn-kubernetes 59m Normal Pulled pod/payload-processing-e2e-worker-w1-355e1a-57bbcfbbb9-ggfc4 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.709s (2.709s including waiting). Image size: 125353963 bytes. 59m Normal AddedInterface pod/payload-processing-e2e-worker-w2-32a59b-6bd5df9875-tlgmv Add eth0 [10.134.0.38/23] from ovn-kubernetes 59m Normal Started pod/payload-processing-e2e-worker-w1-355e1a-57bbcfbbb9-ggfc4 Container started 59m Normal Started pod/payload-processing-e2e-worker-w2-32a59b-6bd5df9875-tlgmv Container started 59m Normal Created pod/payload-processing-e2e-worker-w2-32a59b-6bd5df9875-tlgmv Container created 59m Normal Pulled pod/payload-processing-e2e-worker-w2-32a59b-6bd5df9875-tlgmv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-9686c3 Scaled up replica set payload-pre-processing-e2e-worker-w0-9686c3-6f6bdb5d59 from 0 to 1 59m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-9686c3-779d4f5849 Error creating: pods "payload-processing-e2e-worker-w0-9686c3-779d4f5849-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-9686c3: serviceaccount "payload-processing-e2e-worker-w0-9686c3" not found 59m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-9686c3-6f6bdb5d59 Error creating: pods "payload-pre-processing-e2e-worker-w0-9686c3-6f6bdb5d59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-9686c3: serviceaccount "payload-processing-e2e-worker-w0-9686c3" not found 59m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-9686c3 Scaled up replica set payload-processing-e2e-worker-w0-9686c3-779d4f5849 from 0 to 1 59m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-9686c3-6f6bdb5d59 Created pod: payload-pre-processing-e2e-worker-w0-9686c3-6f6bdb5d59-twjx8 59m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-9686c3-779d4f5849 Created pod: payload-processing-e2e-worker-w0-9686c3-779d4f5849-mnqkq 59m Normal Pulled pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-vdgg5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/payload-processing-e2e-worker-w0-9686c3-779d4f5849-mnqkq Add eth0 [10.133.0.42/23] from ovn-kubernetes 59m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-9686c3-6f6bdb5d59-twjx8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-9686c3-6f6bdb5d59-twjx8 Add eth0 [10.132.0.71/23] from ovn-kubernetes 59m Normal Created pod/payload-pre-processing-e2e-worker-w0-9686c3-6f6bdb5d59-twjx8 Container created 59m Normal Created pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-vdgg5 Container created 59m Normal Started pod/payload-pre-processing-e2e-worker-w0-9686c3-6f6bdb5d59-twjx8 Container started 59m Normal Started pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-vdgg5 Container started 59m Normal Pulling pod/payload-processing-e2e-worker-w0-9686c3-779d4f5849-mnqkq Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 59m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-a4cacd-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-a4cacd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 59m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-9686c3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-9686c3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Normal Pulled pod/payload-processing-e2e-worker-w0-9686c3-779d4f5849-mnqkq Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.658s (2.658s including waiting). Image size: 125353963 bytes. 59m Normal Created pod/payload-processing-e2e-worker-w0-9686c3-779d4f5849-mnqkq Container created 59m Normal Started pod/payload-processing-e2e-worker-w0-9686c3-779d4f5849-mnqkq Container started 59m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-3a7056 Scaled up replica set payload-processing-e2e-worker-w5-3a7056-7bc4d594f7 from 0 to 1 59m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-3a7056-7bc4d594f7 Created pod: payload-processing-e2e-worker-w5-3a7056-7bc4d594f7-wklrp 59m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-3a7056-7bc4d594f7 Error creating: pods "payload-processing-e2e-worker-w5-3a7056-7bc4d594f7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-3a7056: serviceaccount "payload-processing-e2e-worker-w5-3a7056" not found 59m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-3a7056-68d95fb54c Error creating: pods "payload-pre-processing-e2e-worker-w5-3a7056-68d95fb54c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-3a7056: serviceaccount "payload-processing-e2e-worker-w5-3a7056" not found 59m Warning ProbeError pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-xmnlp Readiness probe error: Get "http://10.133.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-3a7056-68d95fb54c Created pod: payload-pre-processing-e2e-worker-w5-3a7056-68d95fb54c-8kq4m 59m Warning Unhealthy pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-xmnlp Readiness probe failed: Get "http://10.133.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-3a7056 Scaled up replica set payload-pre-processing-e2e-worker-w5-3a7056-68d95fb54c from 0 to 1 59m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-3a7056-68d95fb54c-8kq4m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal Created pod/payload-processing-e2e-worker-w5-3a7056-7bc4d594f7-wklrp Container created 59m Normal Created pod/payload-pre-processing-e2e-worker-w5-3a7056-68d95fb54c-8kq4m Container created 59m Normal AddedInterface pod/payload-processing-e2e-worker-w5-3a7056-7bc4d594f7-wklrp Add eth0 [10.132.0.72/23] from ovn-kubernetes 59m Normal Pulled pod/payload-processing-e2e-worker-w5-3a7056-7bc4d594f7-wklrp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal Started pod/payload-pre-processing-e2e-worker-w5-3a7056-68d95fb54c-8kq4m Container started 59m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-3a7056-68d95fb54c-8kq4m Add eth0 [10.133.0.43/23] from ovn-kubernetes 59m Normal Started pod/payload-processing-e2e-worker-w5-3a7056-7bc4d594f7-wklrp Container started 59m Warning ProbeError pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-vdgg5 Startup probe error: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-vdgg5 Startup probe failed: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused 59m Normal Started pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-xmnlp Container started 59m Normal Created pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-xmnlp Container created 59m Normal Pulled pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-xmnlp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Warning Unhealthy pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-xmnlp Startup probe failed: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-xmnlp Startup probe error: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused... 59m Normal Created pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-hsnb4 Container created 59m Normal Started pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-hsnb4 Container started 59m Normal Pulled pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-hsnb4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-7fnvf Container created 59m Normal Started pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-7fnvf Container started 59m Normal Pulled pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-7fnvf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-2e052a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-2e052a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-355e1a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 59m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-355e1a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 59m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-32a59b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-32a59b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 59m Warning ProbeError pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-7fnvf Startup probe error: Get "http://10.133.0.38:15021/healthz/ready": dial tcp 10.133.0.38:15021: connect: connection refused... 59m Warning ProbeError pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-hsnb4 Startup probe error: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-7fnvf Startup probe failed: Get "http://10.133.0.38:15021/healthz/ready": dial tcp 10.133.0.38:15021: connect: connection refused 59m Warning Unhealthy pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-hsnb4 Startup probe failed: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused 58m Normal Started pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-wjpvd Container started 58m Normal Pulled pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-wjpvd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-wjpvd Container created 58m Warning Unhealthy pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-wjpvd Startup probe failed: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-2e052a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-2e052a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning ProbeError pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-wjpvd Startup probe error: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused... 58m Normal Started pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-cxpf5 Container started 58m Normal Pulled pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-cxpf5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-cxpf5 Container created 58m Warning ProbeError pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-cxpf5 Startup probe error: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-cxpf5 Startup probe failed: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused 58m Normal Pulled pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-s4jqx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-s4jqx Container created 58m Normal Started pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-s4jqx Container started 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-fba266-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-a4cacd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-a4cacd-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-fba266-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-32a59b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-355e1a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-355e1a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-9686c3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-9686c3-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-32a59b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning ProbeError pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-s4jqx Startup probe error: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-s4jqx Startup probe failed: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-fba266-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-fba266-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-3a7056-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-3a7056-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-3a7056-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-3a7056-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7854949c9 Deleted pod: maas-default-gateway-openshift-default-7854949c9-5rsd6 55m Normal Killing pod/maas-default-gateway-openshift-default-7854949c9-5rsd6 Stopping container istio-proxy 55m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7854949c9 from 5 to 4 55m Normal ScalingReplicaSet deployment/e2e-worker-w6-fba266-openshift-default Scaled up replica set e2e-worker-w6-fba266-openshift-default-5ff997597f from 1 to 2 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-fba266-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 55m Normal SuccessfulCreate replicaset/e2e-worker-w6-fba266-openshift-default-5ff997597f Created pod: e2e-worker-w6-fba266-openshift-default-5ff997597f-d7xjn 55m Normal AddedInterface pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-d7xjn Add eth0 [10.134.0.47/23] from ovn-kubernetes 55m Normal Created pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-d7xjn Container created 55m Normal Pulled pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-d7xjn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Started pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-d7xjn Container started 55m Warning ProbeError pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-d7xjn Startup probe error: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-d7xjn Startup probe failed: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused 54m Normal SuccessfulCreate replicaset/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c Created pod: e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-tpqgh 54m Normal ScalingReplicaSet deployment/e2e-worker-w3-a4cacd-openshift-default Scaled up replica set e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c from 1 to 2 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-a4cacd-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal AddedInterface pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-tpqgh Add eth0 [10.132.0.82/23] from ovn-kubernetes 54m Normal ScalingReplicaSet deployment/e2e-worker-w5-3a7056-openshift-default Scaled up replica set e2e-worker-w5-3a7056-openshift-default-85ccfddcf from 1 to 2 54m Normal Started pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-tpqgh Container started 54m Normal Created pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-tpqgh Container created 54m Normal Pulled pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-tpqgh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal SuccessfulCreate replicaset/e2e-worker-w5-3a7056-openshift-default-85ccfddcf Created pod: e2e-worker-w5-3a7056-openshift-default-85ccfddcf-cttst 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-3a7056-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal AddedInterface pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-cttst Add eth0 [10.132.0.83/23] from ovn-kubernetes 54m Normal Created pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-cttst Container created 54m Normal Started pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-cttst Container started 54m Normal Pulled pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-cttst Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Warning Unhealthy pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-cttst Startup probe failed: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-cttst Startup probe error: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused... 54m Warning ProbeError pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-tpqgh Startup probe error: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-tpqgh Startup probe failed: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused 54m Normal ScalingReplicaSet deployment/e2e-worker-w4-2e052a-openshift-default Scaled up replica set e2e-worker-w4-2e052a-openshift-default-6886cf66dc from 1 to 2 54m Normal SuccessfulCreate replicaset/e2e-worker-w4-2e052a-openshift-default-6886cf66dc Created pod: e2e-worker-w4-2e052a-openshift-default-6886cf66dc-8m7jw 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-2e052a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal Created pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-8m7jw Container created 54m Normal Pulled pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-8m7jw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-8m7jw Add eth0 [10.133.0.51/23] from ovn-kubernetes 54m Normal Started pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-8m7jw Container started 54m Warning ProbeError pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-8m7jw Startup probe error: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-8m7jw Startup probe failed: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused 54m Normal Killing pod/maas-default-gateway-openshift-default-7854949c9-bw6rx Stopping container istio-proxy 54m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7854949c9 Deleted pod: maas-default-gateway-openshift-default-7854949c9-bw6rx 53m Normal Pulled pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-8vrc9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-8vrc9 Add eth0 [10.133.0.52/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/e2e-worker-w2-32a59b-openshift-default-86974868b5 Created pod: e2e-worker-w2-32a59b-openshift-default-86974868b5-8vrc9 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-32a59b-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w2-32a59b-openshift-default Scaled up replica set e2e-worker-w2-32a59b-openshift-default-86974868b5 from 1 to 2 53m Normal Started pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-8vrc9 Container started 53m Normal Created pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-8vrc9 Container created 53m Warning Unhealthy pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-8vrc9 Startup probe failed: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-8vrc9 Startup probe error: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused... 53m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 2; reason: All metrics below target 53m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7854949c9 from 3 to 2 53m Normal Killing pod/maas-default-gateway-openshift-default-7854949c9-6t8sd Stopping container istio-proxy 53m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7854949c9 Deleted pod: maas-default-gateway-openshift-default-7854949c9-6t8sd 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-2e052a-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w4-2e052a-openshift-default Scaled up replica set e2e-worker-w4-2e052a-openshift-default-6886cf66dc from 2 to 3 53m Normal SuccessfulCreate replicaset/e2e-worker-w4-2e052a-openshift-default-6886cf66dc Created pod: e2e-worker-w4-2e052a-openshift-default-6886cf66dc-p87bt 53m Normal ScalingReplicaSet deployment/e2e-worker-w1-355e1a-openshift-default Scaled up replica set e2e-worker-w1-355e1a-openshift-default-d7d9459f9 from 1 to 2 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-355e1a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 53m Normal AddedInterface pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-p87bt Add eth0 [10.134.0.48/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/e2e-worker-w1-355e1a-openshift-default-d7d9459f9 Created pod: e2e-worker-w1-355e1a-openshift-default-d7d9459f9-phbjz 53m Normal Pulled pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-phbjz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-phbjz Add eth0 [10.132.0.102/23] from ovn-kubernetes 53m Normal Created pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-phbjz Container created 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-3a7056-openshift-default-85ccfddcf Created pod: e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2vv6p 53m Normal Created pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-p87bt Container created 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-3a7056-openshift-default-85ccfddcf Created pod: e2e-worker-w5-3a7056-openshift-default-85ccfddcf-rf5ck 53m Normal ScalingReplicaSet deployment/e2e-worker-w5-3a7056-openshift-default Scaled up replica set e2e-worker-w5-3a7056-openshift-default-85ccfddcf from 2 to 4 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-3a7056-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 53m Normal Started pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-phbjz Container started 53m Normal Started pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-p87bt Container started 53m Normal Pulled pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-p87bt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-rf5ck Add eth0 [10.132.0.103/23] from ovn-kubernetes 53m Normal Started pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-rf5ck Container started 53m Normal Pulled pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-rf5ck Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2vv6p Container started 53m Normal AddedInterface pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2vv6p Add eth0 [10.134.0.49/23] from ovn-kubernetes 53m Normal Created pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-rf5ck Container created 53m Normal Created pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2vv6p Container created 53m Normal Pulled pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2vv6p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Warning Unhealthy pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-p87bt Startup probe failed: Get "http://10.134.0.48:15021/healthz/ready": dial tcp 10.134.0.48:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-p87bt Startup probe error: Get "http://10.134.0.48:15021/healthz/ready": dial tcp 10.134.0.48:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-phbjz Startup probe error: Get "http://10.132.0.102:15021/healthz/ready": dial tcp 10.132.0.102:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-phbjz Startup probe failed: Get "http://10.132.0.102:15021/healthz/ready": dial tcp 10.132.0.102:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2vv6p Startup probe error: Get "http://10.134.0.49:15021/healthz/ready": dial tcp 10.134.0.49:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2vv6p Startup probe failed: Get "http://10.134.0.49:15021/healthz/ready": dial tcp 10.134.0.49:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-rf5ck Startup probe error: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-rf5ck Startup probe failed: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused 52m Normal SuccessfulCreate replicaset/e2e-worker-w0-9686c3-openshift-default-85685b9d55 Created pod: e2e-worker-w0-9686c3-openshift-default-85685b9d55-cj9vk 52m Normal AddedInterface pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-cj9vk Add eth0 [10.133.0.55/23] from ovn-kubernetes 52m Normal ScalingReplicaSet deployment/e2e-worker-w0-9686c3-openshift-default Scaled up replica set e2e-worker-w0-9686c3-openshift-default-85685b9d55 from 1 to 2 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-9686c3-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 52m Normal Pulled pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-cj9vk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-cj9vk Container created 52m Normal Started pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-cj9vk Container started 52m Warning Unhealthy pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-cj9vk Startup probe failed: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-cj9vk Startup probe error: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused... 52m Normal SuccessfulCreate replicaset/e2e-worker-w6-fba266-openshift-default-5ff997597f Created pod: e2e-worker-w6-fba266-openshift-default-5ff997597f-lh6wf 52m Normal ScalingReplicaSet deployment/e2e-worker-w6-fba266-openshift-default Scaled up replica set e2e-worker-w6-fba266-openshift-default-5ff997597f from 2 to 3 52m Normal AddedInterface pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-lh6wf Add eth0 [10.133.0.56/23] from ovn-kubernetes 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-fba266-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 52m Normal ScalingReplicaSet deployment/e2e-worker-w1-355e1a-openshift-default Scaled up replica set e2e-worker-w1-355e1a-openshift-default-d7d9459f9 from 2 to 3 52m Normal AddedInterface pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-b6d57 Add eth0 [10.134.0.53/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-lh6wf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal SuccessfulCreate replicaset/e2e-worker-w1-355e1a-openshift-default-d7d9459f9 Created pod: e2e-worker-w1-355e1a-openshift-default-d7d9459f9-b6d57 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-355e1a-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 52m Normal Started pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-lh6wf Container started 52m Normal Created pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-lh6wf Container created 52m Normal Started pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-b6d57 Container started 52m Normal Created pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-b6d57 Container created 52m Normal Pulled pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-b6d57 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Warning ProbeError pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-lh6wf Startup probe error: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-lh6wf Startup probe failed: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-b6d57 Startup probe error: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-b6d57 Startup probe failed: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused 51m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7854949c9 Deleted pod: maas-default-gateway-openshift-default-7854949c9-dffvr 51m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 1; reason: All metrics below target 51m Normal Killing pod/maas-default-gateway-openshift-default-7854949c9-dffvr Stopping container istio-proxy 51m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7854949c9 from 2 to 1 51m Normal Pulled pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-ntspd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-ntspd Container started 51m Normal SuccessfulCreate replicaset/e2e-worker-w2-32a59b-openshift-default-86974868b5 Created pod: e2e-worker-w2-32a59b-openshift-default-86974868b5-ntspd 51m Normal AddedInterface pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-ntspd Add eth0 [10.134.0.54/23] from ovn-kubernetes 51m Normal ScalingReplicaSet deployment/e2e-worker-w2-32a59b-openshift-default Scaled up replica set e2e-worker-w2-32a59b-openshift-default-86974868b5 from 2 to 3 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-32a59b-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 51m Normal Created pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-ntspd Container created 51m Warning Unhealthy pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-ntspd Startup probe failed: Get "http://10.134.0.54:15021/healthz/ready": dial tcp 10.134.0.54:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-ntspd Startup probe error: Get "http://10.134.0.54:15021/healthz/ready": dial tcp 10.134.0.54:15021: connect: connection refused... 51m Normal SuccessfulCreate replicaset/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c Created pod: e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-6tlg2 51m Normal SuccessfulCreate replicaset/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c Created pod: e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-chgfh 51m Normal AddedInterface pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-chgfh Add eth0 [10.134.0.55/23] from ovn-kubernetes 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-a4cacd-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 51m Normal ScalingReplicaSet deployment/e2e-worker-w3-a4cacd-openshift-default Scaled up replica set e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c from 2 to 4 51m Normal AddedInterface pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-6tlg2 Add eth0 [10.132.0.107/23] from ovn-kubernetes 51m Normal Started pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-chgfh Container started 51m Normal Created pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-chgfh Container created 51m Normal Pulled pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-chgfh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-6tlg2 Container started 51m Normal Created pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-6tlg2 Container created 51m Normal Pulled pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-6tlg2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Warning Unhealthy pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-chgfh Startup probe failed: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-chgfh Startup probe error: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-6tlg2 Startup probe failed: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-6tlg2 Startup probe error: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused... 50m Normal SuccessfulCreate replicaset/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c Created pod: e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-4qtm8 50m Normal SuccessfulCreate replicaset/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c Created pod: e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-vbkvm 50m Normal ScalingReplicaSet deployment/e2e-worker-w4-2e052a-openshift-default Scaled up replica set e2e-worker-w4-2e052a-openshift-default-6886cf66dc from 3 to 6 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-a4cacd-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-2e052a-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 50m Normal SuccessfulCreate replicaset/e2e-worker-w4-2e052a-openshift-default-6886cf66dc Created pod: e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t4xjk 50m Normal SuccessfulCreate replicaset/e2e-worker-w4-2e052a-openshift-default-6886cf66dc Created pod: e2e-worker-w4-2e052a-openshift-default-6886cf66dc-xp5mr 50m Normal SuccessfulCreate replicaset/e2e-worker-w4-2e052a-openshift-default-6886cf66dc Created pod: e2e-worker-w4-2e052a-openshift-default-6886cf66dc-szxrv 50m Normal ScalingReplicaSet deployment/e2e-worker-w3-a4cacd-openshift-default Scaled up replica set e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c from 4 to 8 50m Normal SuccessfulCreate replicaset/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c Created pod: e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-c9kw5 50m Normal SuccessfulCreate replicaset/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c Created pod: e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-fwfsb 50m Normal Pulled pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t4xjk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-z9f5d Add eth0 [10.132.0.121/23] from ovn-kubernetes 50m Normal Created pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-c9kw5 Container created 50m Normal SuccessfulCreate replicaset/e2e-worker-w1-355e1a-openshift-default-d7d9459f9 Created pod: e2e-worker-w1-355e1a-openshift-default-d7d9459f9-jk6qn 50m Normal AddedInterface pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-fwfsb Add eth0 [10.132.0.120/23] from ovn-kubernetes 50m Normal ScalingReplicaSet deployment/e2e-worker-w1-355e1a-openshift-default Scaled up replica set e2e-worker-w1-355e1a-openshift-default-d7d9459f9 from 3 to 5 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-355e1a-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 50m Normal AddedInterface pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-szxrv Add eth0 [10.132.0.119/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-szxrv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-szxrv Container created 50m Normal Started pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-szxrv Container started 50m Normal AddedInterface pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t4xjk Add eth0 [10.133.0.62/23] from ovn-kubernetes 50m Normal AddedInterface pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-c9kw5 Add eth0 [10.133.0.63/23] from ovn-kubernetes 50m Normal Created pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t4xjk Container created 50m Normal Started pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t4xjk Container started 50m Normal Pulled pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-c9kw5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Pulled pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-jk6qn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-c9kw5 Container started 50m Normal Pulled pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-z9f5d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Pulled pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-fwfsb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-xp5mr Add eth0 [10.134.0.58/23] from ovn-kubernetes 50m Normal Created pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-fwfsb Container created 50m Normal Started pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-fwfsb Container started 50m Normal Pulled pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-xp5mr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-vbkvm Container started 50m Normal Created pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-vbkvm Container created 50m Normal Pulled pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-vbkvm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-vbkvm Add eth0 [10.133.0.64/23] from ovn-kubernetes 50m Normal Created pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-xp5mr Container created 50m Normal Started pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-xp5mr Container started 50m Normal SuccessfulCreate replicaset/e2e-worker-w1-355e1a-openshift-default-d7d9459f9 Created pod: e2e-worker-w1-355e1a-openshift-default-d7d9459f9-z9f5d 50m Normal AddedInterface pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-4qtm8 Add eth0 [10.134.0.59/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-4qtm8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-4qtm8 Container created 50m Normal Started pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-4qtm8 Container started 50m Normal AddedInterface pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-jk6qn Add eth0 [10.134.0.60/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-jk6qn Container started 49m Normal Created pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-jk6qn Container created 49m Normal Started pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-z9f5d Container started 49m Normal Created pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-z9f5d Container created 49m Warning ProbeError pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-xp5mr Startup probe error: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-xp5mr Startup probe failed: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t4xjk Startup probe error: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-4qtm8 Startup probe failed: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-jk6qn Startup probe failed: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-jk6qn Startup probe error: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-vbkvm Startup probe failed: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-vbkvm Startup probe error: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t4xjk Startup probe failed: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-c9kw5 Startup probe error: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-c9kw5 Startup probe failed: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-szxrv Startup probe error: Get "http://10.132.0.119:15021/healthz/ready": dial tcp 10.132.0.119:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-szxrv Startup probe failed: Get "http://10.132.0.119:15021/healthz/ready": dial tcp 10.132.0.119:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-4qtm8 Startup probe error: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-z9f5d Startup probe error: Get "http://10.132.0.121:15021/healthz/ready": dial tcp 10.132.0.121:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-z9f5d Startup probe failed: Get "http://10.132.0.121:15021/healthz/ready": dial tcp 10.132.0.121:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-fwfsb Startup probe error: Get "http://10.132.0.120:15021/healthz/ready": dial tcp 10.132.0.120:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-fwfsb Startup probe failed: Get "http://10.132.0.120:15021/healthz/ready": dial tcp 10.132.0.120:15021: connect: connection refused 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-a4cacd-openshift-default New size: 9; reason: 49m Normal ScalingReplicaSet deployment/e2e-worker-w3-a4cacd-openshift-default Scaled up replica set e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c from 8 to 9 49m Normal SuccessfulCreate replicaset/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c Created pod: e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-5kgqh 49m Normal AddedInterface pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-5kgqh Add eth0 [10.134.0.61/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-5kgqh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-5kgqh Container created 49m Normal Started pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-5kgqh Container started 49m Warning Unhealthy pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-5kgqh Startup probe failed: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-5kgqh Startup probe error: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused... 49m Normal ScalingReplicaSet deployment/e2e-worker-w2-32a59b-openshift-default Scaled up replica set e2e-worker-w2-32a59b-openshift-default-86974868b5 from 3 to 6 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-32a59b-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 49m Normal Started pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-67lkx Container started 49m Normal Created pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-67lkx Container created 49m Normal SuccessfulCreate replicaset/e2e-worker-w5-3a7056-openshift-default-85ccfddcf Created pod: e2e-worker-w5-3a7056-openshift-default-85ccfddcf-6bjb5 49m Normal SuccessfulCreate replicaset/e2e-worker-w5-3a7056-openshift-default-85ccfddcf Created pod: e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2mq99 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-3a7056-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 49m Normal ScalingReplicaSet deployment/e2e-worker-w5-3a7056-openshift-default Scaled up replica set e2e-worker-w5-3a7056-openshift-default-85ccfddcf from 4 to 6 49m Normal AddedInterface pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-n6kr9 Add eth0 [10.134.0.62/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-n6kr9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-n6kr9 Container created 49m Normal AddedInterface pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-67lkx Add eth0 [10.133.0.65/23] from ovn-kubernetes 49m Normal AddedInterface pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2mq99 Add eth0 [10.134.0.63/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-n6kr9 Container started 49m Normal Started pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-9kbd6 Container started 49m Normal Created pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-9kbd6 Container created 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-32a59b-openshift-default-86974868b5 Created pod: e2e-worker-w2-32a59b-openshift-default-86974868b5-9kbd6 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-32a59b-openshift-default-86974868b5 Created pod: e2e-worker-w2-32a59b-openshift-default-86974868b5-67lkx 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-32a59b-openshift-default-86974868b5 Created pod: e2e-worker-w2-32a59b-openshift-default-86974868b5-n6kr9 49m Normal AddedInterface pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-9kbd6 Add eth0 [10.132.0.127/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-9kbd6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-6bjb5 Add eth0 [10.133.0.66/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-67lkx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Pulled pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-6bjb5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2mq99 Container started 49m Normal Started pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-6bjb5 Container started 49m Normal Pulled pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2mq99 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-6bjb5 Container created 49m Normal Created pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2mq99 Container created 49m Warning Unhealthy pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-n6kr9 Startup probe failed: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-6bjb5 Startup probe error: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-67lkx Startup probe failed: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2mq99 Startup probe failed: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-6bjb5 Startup probe failed: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-67lkx Startup probe error: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2mq99 Startup probe error: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-n6kr9 Startup probe error: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-9kbd6 Startup probe failed: Get "http://10.132.0.127:15021/healthz/ready": dial tcp 10.132.0.127:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-9kbd6 Startup probe error: Get "http://10.132.0.127:15021/healthz/ready": dial tcp 10.132.0.127:15021: connect: connection refused... 49m Normal AddedInterface pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-2t6vs Add eth0 [10.132.0.134/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-2t6vs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-32a59b-openshift-default-86974868b5 Created pod: e2e-worker-w2-32a59b-openshift-default-86974868b5-2t6vs 49m Normal ScalingReplicaSet deployment/e2e-worker-w2-32a59b-openshift-default Scaled up replica set e2e-worker-w2-32a59b-openshift-default-86974868b5 from 6 to 7 49m Normal Created pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-2t6vs Container created 49m Normal Started pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-2t6vs Container started 49m Warning ProbeError pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-2t6vs Startup probe error: Get "http://10.132.0.134:15021/healthz/ready": dial tcp 10.132.0.134:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-2t6vs Startup probe failed: Get "http://10.132.0.134:15021/healthz/ready": dial tcp 10.132.0.134:15021: connect: connection refused 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-9686c3-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 49m Normal ScalingReplicaSet deployment/e2e-worker-w0-9686c3-openshift-default Scaled up replica set e2e-worker-w0-9686c3-openshift-default-85685b9d55 from 2 to 4 49m Normal SuccessfulCreate replicaset/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c (combined from similar events): Created pod: e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-jwhh6 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-9686c3-openshift-default-85685b9d55 Created pod: e2e-worker-w0-9686c3-openshift-default-85685b9d55-xk4f4 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-a4cacd-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 49m Normal ScalingReplicaSet deployment/e2e-worker-w3-a4cacd-openshift-default Scaled up replica set e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c from 9 to 10 48m Normal Created pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-z4np8 Container created 48m Normal Started pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-xk4f4 Container started 48m Normal AddedInterface pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-xk4f4 Add eth0 [10.134.0.64/23] from ovn-kubernetes 48m Normal AddedInterface pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-z4np8 Add eth0 [10.132.0.142/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-z4np8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-jwhh6 Add eth0 [10.132.0.140/23] from ovn-kubernetes 48m Normal SuccessfulCreate replicaset/e2e-worker-w0-9686c3-openshift-default-85685b9d55 Created pod: e2e-worker-w0-9686c3-openshift-default-85685b9d55-z4np8 48m Normal Started pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-z4np8 Container started 48m Normal Pulled pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-xk4f4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-xk4f4 Container created 48m Normal Pulled pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-jwhh6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-jwhh6 Container created 48m Normal Started pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-jwhh6 Container started 48m Warning ProbeError pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-jwhh6 Startup probe error: Get "http://10.132.0.140:15021/healthz/ready": dial tcp 10.132.0.140:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-jwhh6 Startup probe failed: Get "http://10.132.0.140:15021/healthz/ready": dial tcp 10.132.0.140:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-xk4f4 Startup probe error: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-xk4f4 Startup probe failed: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-z4np8 Startup probe error: Get "http://10.132.0.142:15021/healthz/ready": dial tcp 10.132.0.142:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-z4np8 Startup probe failed: Get "http://10.132.0.142:15021/healthz/ready": dial tcp 10.132.0.142:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-xk4f4 Startup probe error: Get "http://10.134.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-xk4f4 Startup probe failed: Get "http://10.134.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-jwhh6 Startup probe failed: Get "http://10.132.0.140:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-jwhh6 Startup probe error: Get "http://10.132.0.140:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Normal AddedLabel gateway/e2e-ait-90774e61 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-90774e61 48m Normal ScalingReplicaSet deployment/e2e-ait-90774e61-openshift-default Scaled up replica set e2e-ait-90774e61-openshift-default-5f66579499 from 0 to 1 48m Normal AddedInterface pod/e2e-ait-90774e61-openshift-default-5f66579499-5f2mm Add eth0 [10.133.0.69/23] from ovn-kubernetes 48m Normal SuccessfulCreate replicaset/e2e-ait-90774e61-openshift-default-5f66579499 Created pod: e2e-ait-90774e61-openshift-default-5f66579499-5f2mm 48m Normal Pulled pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-zfzp9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Warning ProbeError pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-jwhh6 Readiness probe error: Get "http://10.132.0.140:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-jwhh6 Readiness probe failed: Get "http://10.132.0.140:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-z4np8 Startup probe failed: Get "http://10.132.0.142:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Normal AddedInterface pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-zfzp9 Add eth0 [10.133.0.70/23] from ovn-kubernetes 48m Normal ScalingReplicaSet deployment/e2e-worker-w0-9686c3-openshift-default Scaled up replica set e2e-worker-w0-9686c3-openshift-default-85685b9d55 from 4 to 5 48m Normal Created pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-zfzp9 Container created 48m Normal Started pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-zfzp9 Container started 48m Normal SuccessfulCreate replicaset/e2e-worker-w0-9686c3-openshift-default-85685b9d55 Created pod: e2e-worker-w0-9686c3-openshift-default-85685b9d55-zfzp9 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-9686c3-openshift-default New size: 5; reason: All metrics below target 48m Warning ProbeError pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-z4np8 Startup probe error: Get "http://10.132.0.142:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-90774e61 Scaled up replica set payload-processing-e2e-ait-90774e61-5758cd8cfc from 0 to 1 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-90774e61 Scaled up replica set payload-pre-processing-e2e-ait-90774e61-5d6679746c from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-90774e61-5758cd8cfc Created pod: payload-processing-e2e-ait-90774e61-5758cd8cfc-mstxt 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-90774e61-5d6679746c Created pod: payload-pre-processing-e2e-ait-90774e61-5d6679746c-zph6g 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-90774e61-5d6679746c Error creating: pods "payload-pre-processing-e2e-ait-90774e61-5d6679746c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-90774e61: serviceaccount "payload-processing-e2e-ait-90774e61" not found 48m Warning FailedCreate replicaset/payload-processing-e2e-ait-90774e61-5758cd8cfc Error creating: pods "payload-processing-e2e-ait-90774e61-5758cd8cfc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-90774e61: serviceaccount "payload-processing-e2e-ait-90774e61" not found 48m Normal Created pod/payload-processing-e2e-ait-90774e61-5758cd8cfc-mstxt Container created 48m Normal Pulled pod/payload-pre-processing-e2e-ait-90774e61-5d6679746c-zph6g Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedLabel gateway/e2e-ait-6f198bfb Added label istio.io/rev=openshift-gateway to gateway e2e-ait-6f198bfb 48m Normal Created pod/payload-pre-processing-e2e-ait-90774e61-5d6679746c-zph6g Container created 48m Normal AddedInterface pod/payload-processing-e2e-ait-90774e61-5758cd8cfc-mstxt Add eth0 [10.132.0.147/23] from ovn-kubernetes 48m Normal Started pod/payload-pre-processing-e2e-ait-90774e61-5d6679746c-zph6g Container started 48m Normal Pulled pod/payload-processing-e2e-ait-90774e61-5758cd8cfc-mstxt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-pre-processing-e2e-ait-90774e61-5d6679746c-zph6g Add eth0 [10.132.0.148/23] from ovn-kubernetes 48m Normal Started pod/payload-processing-e2e-ait-90774e61-5758cd8cfc-mstxt Container started 48m Warning Unhealthy pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-zfzp9 Startup probe failed: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused 48m Normal ScalingReplicaSet deployment/e2e-ait-6f198bfb-openshift-default Scaled up replica set e2e-ait-6f198bfb-openshift-default-b76b5d689 from 0 to 1 48m Normal SuccessfulCreate replicaset/e2e-ait-6f198bfb-openshift-default-b76b5d689 Created pod: e2e-ait-6f198bfb-openshift-default-b76b5d689-dgcgz 48m Warning ProbeError pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-zfzp9 Startup probe error: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused... 48m Normal AddedInterface pod/e2e-ait-6f198bfb-openshift-default-b76b5d689-dgcgz Add eth0 [10.132.0.149/23] from ovn-kubernetes 48m Normal ScalingReplicaSet deployment/e2e-derive-30b92815-openshift-default Scaled up replica set e2e-derive-30b92815-openshift-default-746cd4b59c from 0 to 1 48m Normal SuccessfulCreate replicaset/e2e-derive-30b92815-openshift-default-746cd4b59c Created pod: e2e-derive-30b92815-openshift-default-746cd4b59c-9tpf6 48m Normal AddedLabel gateway/e2e-derive-30b92815 Added label istio.io/rev=openshift-gateway to gateway e2e-derive-30b92815 48m Normal AddedInterface pod/e2e-derive-30b92815-openshift-default-746cd4b59c-9tpf6 Add eth0 [10.133.0.71/23] from ovn-kubernetes 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-6f198bfb Scaled up replica set payload-processing-e2e-ait-6f198bfb-7dccb8fbcf from 0 to 1 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-6f198bfb-587965948c Error creating: pods "payload-pre-processing-e2e-ait-6f198bfb-587965948c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-6f198bfb: serviceaccount "payload-processing-e2e-ait-6f198bfb" not found 48m Warning FailedCreate replicaset/payload-processing-e2e-ait-6f198bfb-7dccb8fbcf Error creating: pods "payload-processing-e2e-ait-6f198bfb-7dccb8fbcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-6f198bfb: serviceaccount "payload-processing-e2e-ait-6f198bfb" not found 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-6f198bfb-7dccb8fbcf Created pod: payload-processing-e2e-ait-6f198bfb-7dccb8fbcf-r24t4 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-6f198bfb Scaled up replica set payload-pre-processing-e2e-ait-6f198bfb-587965948c from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-6f198bfb-587965948c Created pod: payload-pre-processing-e2e-ait-6f198bfb-587965948c-z9n4x 48m Normal AddedInterface pod/payload-pre-processing-e2e-ait-6f198bfb-587965948c-z9n4x Add eth0 [10.132.0.152/23] from ovn-kubernetes 48m Normal Created pod/payload-processing-e2e-ait-6f198bfb-7dccb8fbcf-r24t4 Container created 48m Normal Created pod/payload-pre-processing-e2e-ait-6f198bfb-587965948c-z9n4x Container created 48m Normal Pulled pod/payload-pre-processing-e2e-ait-6f198bfb-587965948c-z9n4x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Pulled pod/payload-processing-e2e-ait-6f198bfb-7dccb8fbcf-r24t4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Started pod/payload-pre-processing-e2e-ait-6f198bfb-587965948c-z9n4x Container started 48m Normal Started pod/payload-processing-e2e-ait-6f198bfb-7dccb8fbcf-r24t4 Container started 48m Normal AddedInterface pod/payload-processing-e2e-ait-6f198bfb-7dccb8fbcf-r24t4 Add eth0 [10.134.0.65/23] from ovn-kubernetes 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-30b92815 Scaled up replica set payload-processing-e2e-derive-30b92815-7984849f75 from 0 to 1 48m Warning FailedCreate replicaset/payload-processing-e2e-derive-30b92815-7984849f75 Error creating: pods "payload-processing-e2e-derive-30b92815-7984849f75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-30b92815: serviceaccount "payload-processing-e2e-derive-30b92815" not found 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-30b92815 Scaled up replica set payload-pre-processing-e2e-derive-30b92815-c97b7d6c5 from 0 to 1 48m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 48m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-30b92815-c97b7d6c5 Created pod: payload-pre-processing-e2e-derive-30b92815-c97b7d6c5-gpg5c 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-30b92815-7984849f75 Created pod: payload-processing-e2e-derive-30b92815-7984849f75-2fvrh 48m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-79xhw 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-30b92815-c97b7d6c5 Error creating: pods "payload-pre-processing-e2e-derive-30b92815-c97b7d6c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-30b92815: serviceaccount "payload-processing-e2e-derive-30b92815" not found 48m Normal Created pod/payload-processing-e2e-derive-30b92815-7984849f75-2fvrh Container created 48m Normal AddedInterface pod/payload-pre-processing-e2e-derive-30b92815-c97b7d6c5-gpg5c Add eth0 [10.132.0.154/23] from ovn-kubernetes 48m Normal Created pod/payload-pre-processing-e2e-derive-30b92815-c97b7d6c5-gpg5c Container created 48m Normal Started pod/payload-pre-processing-e2e-derive-30b92815-c97b7d6c5-gpg5c Container started 48m Normal AddedInterface pod/payload-processing-e2e-derive-30b92815-7984849f75-2fvrh Add eth0 [10.132.0.155/23] from ovn-kubernetes 48m Normal Pulled pod/payload-processing-e2e-derive-30b92815-7984849f75-2fvrh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Pulled pod/payload-pre-processing-e2e-derive-30b92815-c97b7d6c5-gpg5c Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Started pod/payload-processing-e2e-derive-30b92815-7984849f75-2fvrh Container started 48m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-79xhw Add eth0 [10.133.0.72/23] from ovn-kubernetes 48m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-79xhw Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 48m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-79xhw Container started 48m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-79xhw Container created 48m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-79xhw Readiness probe error: HTTP probe failed with statuscode: 503... 48m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-79xhw Readiness probe failed: HTTP probe failed with statuscode: 503 48m Normal Started pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-tbwhw Container started 48m Normal AddedInterface pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-tbwhw Add eth0 [10.132.0.157/23] from ovn-kubernetes 48m Normal AddedInterface pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-5ktvr Add eth0 [10.134.0.66/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t7qrn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t7qrn Add eth0 [10.133.0.74/23] from ovn-kubernetes 48m Normal SuccessfulCreate replicaset/e2e-worker-w4-2e052a-openshift-default-6886cf66dc Created pod: e2e-worker-w4-2e052a-openshift-default-6886cf66dc-592dz 48m Normal SuccessfulCreate replicaset/e2e-worker-w4-2e052a-openshift-default-6886cf66dc Created pod: e2e-worker-w4-2e052a-openshift-default-6886cf66dc-d5ppq 48m Normal SuccessfulCreate replicaset/e2e-worker-w4-2e052a-openshift-default-6886cf66dc Created pod: e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t7qrn 48m Normal AddedInterface pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-dn2p9 Add eth0 [10.133.0.73/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-dn2p9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal SuccessfulCreate replicaset/e2e-worker-w4-2e052a-openshift-default-6886cf66dc (combined from similar events): Created pod: e2e-worker-w4-2e052a-openshift-default-6886cf66dc-n5xtx 48m Normal ScalingReplicaSet deployment/e2e-worker-w6-fba266-openshift-default Scaled up replica set e2e-worker-w6-fba266-openshift-default-5ff997597f from 3 to 6 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-fba266-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 48m Normal SuccessfulCreate replicaset/e2e-worker-w6-fba266-openshift-default-5ff997597f Created pod: e2e-worker-w6-fba266-openshift-default-5ff997597f-dn2p9 48m Normal Created pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-dn2p9 Container created 48m Normal Started pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-dn2p9 Container started 48m Normal SuccessfulCreate replicaset/e2e-worker-w6-fba266-openshift-default-5ff997597f Created pod: e2e-worker-w6-fba266-openshift-default-5ff997597f-5ktvr 48m Normal SuccessfulCreate replicaset/e2e-worker-w6-fba266-openshift-default-5ff997597f Created pod: e2e-worker-w6-fba266-openshift-default-5ff997597f-tbwhw 48m Normal AddedInterface pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-n5xtx Add eth0 [10.132.0.159/23] from ovn-kubernetes 48m Normal AddedInterface pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-d5ppq Add eth0 [10.134.0.67/23] from ovn-kubernetes 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-2e052a-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 48m Normal ScalingReplicaSet deployment/e2e-worker-w4-2e052a-openshift-default Scaled up replica set e2e-worker-w4-2e052a-openshift-default-6886cf66dc from 6 to 10 48m Normal AddedInterface pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-592dz Add eth0 [10.132.0.158/23] from ovn-kubernetes 48m Normal Created pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-tbwhw Container created 48m Normal Pulled pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-tbwhw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-5ktvr Container started 48m Normal SuccessfulCreate replicaset/e2e-worker-w1-355e1a-openshift-default-d7d9459f9 Created pod: e2e-worker-w1-355e1a-openshift-default-d7d9459f9-64gzp 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-32a59b-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 48m Normal AddedInterface pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-j6l6d Add eth0 [10.132.0.160/23] from ovn-kubernetes 48m Normal SuccessfulCreate replicaset/e2e-worker-w1-355e1a-openshift-default-d7d9459f9 Created pod: e2e-worker-w1-355e1a-openshift-default-d7d9459f9-j6l6d 48m Normal ScalingReplicaSet deployment/e2e-worker-w2-32a59b-openshift-default Scaled up replica set e2e-worker-w2-32a59b-openshift-default-86974868b5 from 7 to 10 48m Normal SuccessfulCreate replicaset/e2e-worker-w2-32a59b-openshift-default-86974868b5 (combined from similar events): Created pod: e2e-worker-w2-32a59b-openshift-default-86974868b5-hrs6j 48m Normal Started pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-n5xtx Container started 48m Normal Created pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-n5xtx Container created 48m Normal Pulled pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-n5xtx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal SuccessfulCreate replicaset/e2e-worker-w1-355e1a-openshift-default-d7d9459f9 Created pod: e2e-worker-w1-355e1a-openshift-default-d7d9459f9-nkkk8 48m Normal AddedInterface pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-nkkk8 Add eth0 [10.133.0.75/23] from ovn-kubernetes 48m Normal Started pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-d5ppq Container started 48m Normal Created pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-d5ppq Container created 48m Normal Pulled pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-d5ppq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t7qrn Container created 48m Normal Started pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t7qrn Container started 48m Normal AddedInterface pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-64gzp Add eth0 [10.134.0.68/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-64gzp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Pulled pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-592dz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-592dz Container created 48m Normal Started pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-592dz Container started 48m Normal SuccessfulCreate replicaset/e2e-worker-w2-32a59b-openshift-default-86974868b5 Created pod: e2e-worker-w2-32a59b-openshift-default-86974868b5-t4rd9 48m Normal SuccessfulCreate replicaset/e2e-worker-w2-32a59b-openshift-default-86974868b5 Created pod: e2e-worker-w2-32a59b-openshift-default-86974868b5-w296j 48m Normal Pulled pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-nkkk8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Pulled pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-5ktvr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal ScalingReplicaSet deployment/e2e-worker-w1-355e1a-openshift-default Scaled up replica set e2e-worker-w1-355e1a-openshift-default-d7d9459f9 from 5 to 8 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-355e1a-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 48m Normal Created pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-5ktvr Container created 48m Normal Started pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-64gzp Container started 48m Normal Pulled pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-t4rd9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Pulled pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-j6l6d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-t4rd9 Container started 48m Normal Created pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-j6l6d Container created 48m Normal Created pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-t4rd9 Container created 48m Normal Created pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-64gzp Container created 48m Normal AddedInterface pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-t4rd9 Add eth0 [10.134.0.69/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-w296j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-j6l6d Container started 48m Normal AddedInterface pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-w296j Add eth0 [10.133.0.76/23] from ovn-kubernetes 48m Normal Started pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-nkkk8 Container started 48m Normal Created pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-nkkk8 Container created 48m Normal Pulled pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-hrs6j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-hrs6j Add eth0 [10.132.0.161/23] from ovn-kubernetes 48m Normal Started pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-w296j Container started 48m Normal Created pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-w296j Container created 48m Normal Started pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-hrs6j Container started 48m Normal Created pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-hrs6j Container created 48m Warning Unhealthy pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-lh6wf Readiness probe failed: Get "http://10.133.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-lh6wf Readiness probe error: Get "http://10.133.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-8vrc9 Readiness probe error: Get "http://10.133.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-8vrc9 Readiness probe failed: Get "http://10.133.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-cxpf5 Readiness probe failed: Get "http://10.132.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-cxpf5 Readiness probe error: Get "http://10.132.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-xp5mr Readiness probe failed: Get "http://10.134.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-xp5mr Readiness probe error: Get "http://10.134.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/payload-pre-processing-e2e-ait-90774e61-5d6679746c-zph6g Readiness probe failed: timeout: failed to connect service "10.132.0.148:50052" within 1s: context deadline exceeded 48m Warning Unhealthy pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-szxrv Readiness probe failed: Get "http://10.132.0.119:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-szxrv Readiness probe error: Get "http://10.132.0.119:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t7qrn Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-9kbd6 Readiness probe failed: Get "http://10.132.0.127:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-dn2p9 Startup probe failed: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-dn2p9 Startup probe error: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t7qrn Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-d5ppq Startup probe error: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-d5ppq Startup probe failed: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-9kbd6 Readiness probe error: Get "http://10.132.0.127:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-nkkk8 Startup probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-5ktvr Startup probe error: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-5ktvr Startup probe failed: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-nkkk8 Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t7qrn Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-w296j Startup probe failed: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-tbwhw Startup probe failed: Get "http://10.132.0.157:15021/healthz/ready": dial tcp 10.132.0.157:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-t4rd9 Startup probe error: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-w296j Startup probe error: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-t4rd9 Startup probe failed: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-n5xtx Startup probe failed: Get "http://10.132.0.159:15021/healthz/ready": dial tcp 10.132.0.159:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-592dz Startup probe failed: Get "http://10.132.0.158:15021/healthz/ready": dial tcp 10.132.0.158:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-64gzp Startup probe failed: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-64gzp Startup probe error: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t7qrn Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-dn2p9 Startup probe failed: Get "http://10.133.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-tbwhw Startup probe error: Get "http://10.132.0.157:15021/healthz/ready": dial tcp 10.132.0.157:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-592dz Startup probe error: Get "http://10.132.0.158:15021/healthz/ready": dial tcp 10.132.0.158:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-dn2p9 Startup probe error: Get "http://10.133.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-n5xtx Startup probe error: Get "http://10.132.0.159:15021/healthz/ready": dial tcp 10.132.0.159:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-5ktvr Startup probe error: Get "http://10.134.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-5ktvr Startup probe failed: Get "http://10.134.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-j6l6d Startup probe failed: Get "http://10.132.0.160:15021/healthz/ready": dial tcp 10.132.0.160:15021: connect: connection refused 48m Normal SuccessfulCreate replicaset/e2e-worker-w6-fba266-openshift-default-5ff997597f Created pod: e2e-worker-w6-fba266-openshift-default-5ff997597f-wfcwg 48m Warning Unhealthy pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-w296j Startup probe failed: Get "http://10.133.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Normal AddedInterface pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-wfcwg Add eth0 [10.134.0.70/23] from ovn-kubernetes 48m Warning Unhealthy pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-t4rd9 Startup probe failed: Get "http://10.134.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Normal ScalingReplicaSet deployment/e2e-worker-w6-fba266-openshift-default Scaled up replica set e2e-worker-w6-fba266-openshift-default-5ff997597f from 6 to 7 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-fba266-openshift-default New size: 7; reason: All metrics below target 48m Warning ProbeError pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-w296j Startup probe error: Get "http://10.133.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Normal Created pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-wfcwg Container created 47m Warning ProbeError pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-t4rd9 Startup probe error: Get "http://10.134.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-j6l6d Startup probe error: Get "http://10.132.0.160:15021/healthz/ready": dial tcp 10.132.0.160:15021: connect: connection refused... 47m Normal Pulled pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-wfcwg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Started pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-wfcwg Container started 47m Warning Unhealthy pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-hrs6j Startup probe failed: Get "http://10.132.0.161:15021/healthz/ready": dial tcp 10.132.0.161:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-hrs6j Startup probe error: Get "http://10.132.0.161:15021/healthz/ready": dial tcp 10.132.0.161:15021: connect: connection refused... 47m Warning ProbeError pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-wfcwg Startup probe error: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-wfcwg Startup probe failed: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-hsnb4 Readiness probe error: Get "http://10.132.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-hsnb4 Readiness probe failed: Get "http://10.132.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-vdgg5 Readiness probe error: Get "http://10.132.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-vdgg5 Readiness probe failed: Get "http://10.132.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Normal Pulled pod/e2e-ait-90774e61-openshift-default-5f66579499-5f2mm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-ait-90774e61-openshift-default-5f66579499-5f2mm Container created 47m Normal Started pod/e2e-ait-90774e61-openshift-default-5f66579499-5f2mm Container started 47m Warning ProbeError pod/e2e-ait-90774e61-openshift-default-5f66579499-5f2mm Startup probe error: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-ait-90774e61-openshift-default-5f66579499-5f2mm Startup probe failed: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused 47m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7854949c9 from 1 to 4 47m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7854949c9 Created pod: maas-default-gateway-openshift-default-7854949c9-zlt64 47m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7854949c9 Created pod: maas-default-gateway-openshift-default-7854949c9-2fxrb 47m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7854949c9 Created pod: maas-default-gateway-openshift-default-7854949c9-m4w4c 47m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 47m Normal AddedInterface pod/maas-default-gateway-openshift-default-7854949c9-m4w4c Add eth0 [10.132.0.163/23] from ovn-kubernetes 47m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-m4w4c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-2fxrb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/maas-default-gateway-openshift-default-7854949c9-zlt64 Add eth0 [10.133.0.77/23] from ovn-kubernetes 47m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-zlt64 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/maas-default-gateway-openshift-default-7854949c9-zlt64 Container created 47m Normal Created pod/maas-default-gateway-openshift-default-7854949c9-2fxrb Container created 47m Normal Started pod/maas-default-gateway-openshift-default-7854949c9-2fxrb Container started 47m Normal Started pod/maas-default-gateway-openshift-default-7854949c9-m4w4c Container started 47m Normal AddedInterface pod/maas-default-gateway-openshift-default-7854949c9-2fxrb Add eth0 [10.134.0.74/23] from ovn-kubernetes 47m Normal Created pod/maas-default-gateway-openshift-default-7854949c9-m4w4c Container created 47m Normal Started pod/maas-default-gateway-openshift-default-7854949c9-zlt64 Container started 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-90774e61-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-90774e61-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Normal ScalingReplicaSet deployment/e2e-worker-w6-fba266-openshift-default Scaled up replica set e2e-worker-w6-fba266-openshift-default-5ff997597f from 7 to 10 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-fba266-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 47m Normal SuccessfulCreate replicaset/e2e-worker-w6-fba266-openshift-default-5ff997597f Created pod: e2e-worker-w6-fba266-openshift-default-5ff997597f-qkdmt 47m Normal SuccessfulCreate replicaset/e2e-worker-w6-fba266-openshift-default-5ff997597f Created pod: e2e-worker-w6-fba266-openshift-default-5ff997597f-jdpnw 47m Normal SuccessfulCreate replicaset/e2e-worker-w6-fba266-openshift-default-5ff997597f (combined from similar events): Created pod: e2e-worker-w6-fba266-openshift-default-5ff997597f-ql4lz 46m Normal Pulled pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-ql4lz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-qkdmt Container started 46m Normal AddedInterface pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-jdpnw Add eth0 [10.133.0.78/23] from ovn-kubernetes 46m Normal Created pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-jdpnw Container created 46m Normal Started pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-jdpnw Container started 46m Normal AddedInterface pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-qkdmt Add eth0 [10.132.0.164/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-qkdmt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-qkdmt Container created 46m Normal Pulled pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-jdpnw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-ql4lz Add eth0 [10.132.0.165/23] from ovn-kubernetes 46m Normal Started pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-ql4lz Container started 46m Normal Created pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-ql4lz Container created 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-2fxrb Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused 46m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-zlt64 Startup probe error: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused... 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-zlt64 Startup probe failed: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused 46m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-2fxrb Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused... 46m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-m4w4c Startup probe error: Get "http://10.132.0.163:15021/healthz/ready": dial tcp 10.132.0.163:15021: connect: connection refused... 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-m4w4c Startup probe failed: Get "http://10.132.0.163:15021/healthz/ready": dial tcp 10.132.0.163:15021: connect: connection refused 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-zlt64 Startup probe failed: Get "http://10.133.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-zlt64 Startup probe error: Get "http://10.133.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning ProbeError pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-jdpnw Startup probe error: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused... 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-m4w4c Startup probe failed: Get "http://10.132.0.163:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-m4w4c Startup probe error: Get "http://10.132.0.163:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-jdpnw Startup probe failed: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-ql4lz Startup probe error: Get "http://10.132.0.165:15021/healthz/ready": dial tcp 10.132.0.165:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-qkdmt Startup probe failed: Get "http://10.132.0.164:15021/healthz/ready": dial tcp 10.132.0.164:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-qkdmt Startup probe error: Get "http://10.132.0.164:15021/healthz/ready": dial tcp 10.132.0.164:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-ql4lz Startup probe failed: Get "http://10.132.0.165:15021/healthz/ready": dial tcp 10.132.0.165:15021: connect: connection refused 46m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: 46m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7854949c9 from 4 to 5 46m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7854949c9 Created pod: maas-default-gateway-openshift-default-7854949c9-x4nc2 46m Normal Created pod/maas-default-gateway-openshift-default-7854949c9-x4nc2 Container created 46m Normal AddedInterface pod/maas-default-gateway-openshift-default-7854949c9-x4nc2 Add eth0 [10.134.0.75/23] from ovn-kubernetes 46m Normal Started pod/maas-default-gateway-openshift-default-7854949c9-x4nc2 Container started 46m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-x4nc2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-90774e61-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-90774e61-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal Killing pod/payload-pre-processing-e2e-ait-90774e61-5d6679746c-zph6g Stopping container payload-pre-processing 46m Normal Killing pod/payload-processing-e2e-ait-90774e61-5758cd8cfc-mstxt Stopping container payload-processing 46m Normal Started pod/e2e-derive-30b92815-openshift-default-746cd4b59c-9tpf6 Container started 46m Normal Created pod/e2e-derive-30b92815-openshift-default-746cd4b59c-9tpf6 Container created 46m Normal Pulled pod/e2e-derive-30b92815-openshift-default-746cd4b59c-9tpf6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-ait-6f198bfb-openshift-default-b76b5d689-dgcgz Container started 46m Normal Created pod/e2e-ait-6f198bfb-openshift-default-b76b5d689-dgcgz Container created 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-x4nc2 Startup probe failed: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused 46m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-x4nc2 Startup probe error: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused... 46m Normal Pulled pod/e2e-ait-6f198bfb-openshift-default-b76b5d689-dgcgz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-6f198bfb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-6f198bfb-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning ProbeError pod/e2e-derive-30b92815-openshift-default-746cd4b59c-9tpf6 Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-derive-30b92815-openshift-default-746cd4b59c-9tpf6 Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-30b92815-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-30b92815-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning ProbeError pod/e2e-ait-6f198bfb-openshift-default-b76b5d689-dgcgz Startup probe error: Get "http://10.132.0.149:15021/healthz/ready": dial tcp 10.132.0.149:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-ait-6f198bfb-openshift-default-b76b5d689-dgcgz Startup probe failed: Get "http://10.132.0.149:15021/healthz/ready": dial tcp 10.132.0.149:15021: connect: connection refused 46m Normal Killing pod/payload-pre-processing-e2e-ait-6f198bfb-587965948c-z9n4x Stopping container payload-pre-processing 46m Normal Killing pod/payload-processing-e2e-ait-6f198bfb-7dccb8fbcf-r24t4 Stopping container payload-processing 46m Normal Killing pod/payload-pre-processing-e2e-derive-30b92815-c97b7d6c5-gpg5c Stopping container payload-pre-processing 46m Normal Killing pod/payload-processing-e2e-derive-30b92815-7984849f75-2fvrh Stopping container payload-processing 46m Normal Killing pod/e2e-ait-90774e61-openshift-default-5f66579499-5f2mm Stopping container istio-proxy 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-6f198bfb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-6f198bfb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal AddedLabel gateway/e2e-migrate-82cd41ab-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-82cd41ab-gw 46m Normal ScalingReplicaSet deployment/e2e-migrate-82cd41ab-gw-openshift-default Scaled up replica set e2e-migrate-82cd41ab-gw-openshift-default-7c64cf7b5 from 0 to 1 46m Normal SuccessfulCreate replicaset/e2e-migrate-82cd41ab-gw-openshift-default-7c64cf7b5 Created pod: e2e-migrate-82cd41ab-gw-openshift-default-7c64cf7b5-x82rn 46m Normal AddedLabel gateway/e2e-shared-a-w4-9e277a Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-9e277a 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-30b92815-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-30b92815-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-9e277a-openshift-default Scaled up replica set e2e-shared-a-w4-9e277a-openshift-default-59d89f6f57 from 0 to 1 46m Normal AddedInterface pod/e2e-migrate-82cd41ab-gw-openshift-default-7c64cf7b5-x82rn Add eth0 [10.132.0.170/23] from ovn-kubernetes 46m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-9e277a-openshift-default-59d89f6f57 Created pod: e2e-shared-a-w4-9e277a-openshift-default-59d89f6f57-gz72j 46m Normal AddedInterface pod/e2e-shared-a-w4-9e277a-openshift-default-59d89f6f57-gz72j Add eth0 [10.132.0.171/23] from ovn-kubernetes 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-82cd41ab Scaled up replica set payload-processing-e2e-migrate-82cd41ab-f78868cf5 from 0 to 1 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-82cd41ab-799f447d67 Error creating: pods "payload-pre-processing-e2e-migrate-82cd41ab-799f447d67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-82cd41ab: serviceaccount "payload-processing-e2e-migrate-82cd41ab" not found 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-82cd41ab Scaled up replica set payload-pre-processing-e2e-migrate-82cd41ab-799f447d67 from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-82cd41ab-f78868cf5 Created pod: payload-processing-e2e-migrate-82cd41ab-f78868cf5-thnf4 45m Warning FailedCreate replicaset/payload-processing-e2e-migrate-82cd41ab-f78868cf5 Error creating: pods "payload-processing-e2e-migrate-82cd41ab-f78868cf5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-82cd41ab: serviceaccount "payload-processing-e2e-migrate-82cd41ab" not found 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-82cd41ab-799f447d67 Created pod: payload-pre-processing-e2e-migrate-82cd41ab-799f447d67-ps6pq 45m Normal Pulled pod/payload-processing-e2e-migrate-82cd41ab-f78868cf5-thnf4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Started pod/payload-pre-processing-e2e-migrate-82cd41ab-799f447d67-ps6pq Container started 45m Normal Created pod/payload-pre-processing-e2e-migrate-82cd41ab-799f447d67-ps6pq Container created 45m Normal Pulled pod/payload-pre-processing-e2e-migrate-82cd41ab-799f447d67-ps6pq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-82cd41ab-799f447d67-ps6pq Add eth0 [10.134.0.76/23] from ovn-kubernetes 45m Normal AddedInterface pod/payload-processing-e2e-migrate-82cd41ab-f78868cf5-thnf4 Add eth0 [10.132.0.172/23] from ovn-kubernetes 45m Normal Started pod/payload-processing-e2e-migrate-82cd41ab-f78868cf5-thnf4 Container started 45m Normal Created pod/payload-processing-e2e-migrate-82cd41ab-f78868cf5-thnf4 Container created 45m Normal Killing pod/e2e-derive-30b92815-openshift-default-746cd4b59c-9tpf6 Stopping container istio-proxy 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w4-9e277a Scaled up replica set payload-processing-e2e-shared-a-w4-9e277a-5dc7dd98bc from 0 to 1 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w4-9e277a-6f7b5d4955 Error creating: pods "payload-pre-processing-e2e-shared-a-w4-9e277a-6f7b5d4955-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-9e277a: serviceaccount "payload-processing-e2e-shared-a-w4-9e277a" not found 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w4-9e277a Scaled up replica set payload-pre-processing-e2e-shared-a-w4-9e277a-6f7b5d4955 from 0 to 1 45m Normal Killing pod/e2e-ait-6f198bfb-openshift-default-b76b5d689-dgcgz Stopping container istio-proxy 45m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w4-9e277a-5dc7dd98bc Error creating: pods "payload-processing-e2e-shared-a-w4-9e277a-5dc7dd98bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-9e277a: serviceaccount "payload-processing-e2e-shared-a-w4-9e277a" not found 45m Normal AddedLabel gateway/e2e-ait-6765e083 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-6765e083 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w4-9e277a-6f7b5d4955 Created pod: payload-pre-processing-e2e-shared-a-w4-9e277a-6f7b5d4955-p29sm 45m Normal SuccessfulCreate replicaset/e2e-ait-6765e083-openshift-default-8486d6c5fc Created pod: e2e-ait-6765e083-openshift-default-8486d6c5fc-5df52 45m Normal Created pod/payload-pre-processing-e2e-shared-a-w4-9e277a-6f7b5d4955-p29sm Container created 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w4-9e277a-5dc7dd98bc Created pod: payload-processing-e2e-shared-a-w4-9e277a-5dc7dd98bc-dt7fq 45m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w4-9e277a-6f7b5d4955-p29sm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w4-9e277a-6f7b5d4955-p29sm Add eth0 [10.134.0.77/23] from ovn-kubernetes 45m Normal Started pod/payload-pre-processing-e2e-shared-a-w4-9e277a-6f7b5d4955-p29sm Container started 45m Normal ScalingReplicaSet deployment/e2e-ait-6765e083-openshift-default Scaled up replica set e2e-ait-6765e083-openshift-default-8486d6c5fc from 0 to 1 45m Warning ProbeError pod/e2e-derive-30b92815-openshift-default-746cd4b59c-9tpf6 Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning Unhealthy pod/e2e-derive-30b92815-openshift-default-746cd4b59c-9tpf6 Readiness probe failed: HTTP probe failed with statuscode: 503 45m Normal AddedInterface pod/payload-processing-e2e-shared-a-w4-9e277a-5dc7dd98bc-dt7fq Add eth0 [10.134.0.78/23] from ovn-kubernetes 45m Normal Started pod/payload-processing-e2e-shared-a-w4-9e277a-5dc7dd98bc-dt7fq Container started 45m Normal AddedInterface pod/e2e-ait-6765e083-openshift-default-8486d6c5fc-5df52 Add eth0 [10.134.0.79/23] from ovn-kubernetes 45m Normal Pulled pod/payload-processing-e2e-shared-a-w4-9e277a-5dc7dd98bc-dt7fq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-processing-e2e-shared-a-w4-9e277a-5dc7dd98bc-dt7fq Container created 45m Warning Unhealthy pod/e2e-ait-6f198bfb-openshift-default-b76b5d689-dgcgz Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/e2e-ait-6f198bfb-openshift-default-b76b5d689-dgcgz Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-6765e083-6df79b4b87 Error creating: pods "payload-pre-processing-e2e-ait-6765e083-6df79b4b87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-6765e083: serviceaccount "payload-processing-e2e-ait-6765e083" not found 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-6765e083 Scaled up replica set payload-processing-e2e-ait-6765e083-55c596cf9f from 0 to 1 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-6765e083 Scaled up replica set payload-pre-processing-e2e-ait-6765e083-6df79b4b87 from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-6765e083-6df79b4b87 Created pod: payload-pre-processing-e2e-ait-6765e083-6df79b4b87-jx2qz 45m Warning FailedCreate replicaset/payload-processing-e2e-ait-6765e083-55c596cf9f Error creating: pods "payload-processing-e2e-ait-6765e083-55c596cf9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-6765e083: serviceaccount "payload-processing-e2e-ait-6765e083" not found 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-6765e083-55c596cf9f Created pod: payload-processing-e2e-ait-6765e083-55c596cf9f-nrbv8 45m Normal Created pod/payload-processing-e2e-ait-6765e083-55c596cf9f-nrbv8 Container created 45m Normal AddedInterface pod/payload-processing-e2e-ait-6765e083-55c596cf9f-nrbv8 Add eth0 [10.132.0.173/23] from ovn-kubernetes 45m Normal Pulled pod/payload-processing-e2e-ait-6765e083-55c596cf9f-nrbv8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/payload-pre-processing-e2e-ait-6765e083-6df79b4b87-jx2qz Add eth0 [10.132.0.174/23] from ovn-kubernetes 45m Normal Started pod/payload-processing-e2e-ait-6765e083-55c596cf9f-nrbv8 Container started 45m Normal Started pod/payload-pre-processing-e2e-ait-6765e083-6df79b4b87-jx2qz Container started 45m Normal Pulled pod/payload-pre-processing-e2e-ait-6765e083-6df79b4b87-jx2qz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-pre-processing-e2e-ait-6765e083-6df79b4b87-jx2qz Container created 45m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-xl65b 45m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: cpu resource utilization (percentage of request) above target 45m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 3 45m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-xl65b Add eth0 [10.134.0.80/23] from ovn-kubernetes 45m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-xl65b Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 45m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-xl65b Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.977s (3.977s including waiting). Image size: 185476477 bytes. 45m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-xl65b Container created 45m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-xl65b Container started 45m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-xl65b Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-xl65b Readiness probe failed: HTTP probe failed with statuscode: 503 45m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 45m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7854949c9 Created pod: maas-default-gateway-openshift-default-7854949c9-r6cjm 45m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7854949c9 Created pod: maas-default-gateway-openshift-default-7854949c9-v7mn2 45m Normal Created pod/maas-default-gateway-openshift-default-7854949c9-r6cjm Container created 45m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-r6cjm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/maas-default-gateway-openshift-default-7854949c9-v7mn2 Container created 45m Normal Started pod/maas-default-gateway-openshift-default-7854949c9-v7mn2 Container started 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-7854949c9-v7mn2 Add eth0 [10.133.0.80/23] from ovn-kubernetes 45m Normal Started pod/maas-default-gateway-openshift-default-7854949c9-r6cjm Container started 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-7854949c9-r6cjm Add eth0 [10.132.0.175/23] from ovn-kubernetes 45m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-v7mn2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-v7mn2 Startup probe error: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused... 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-r6cjm Startup probe failed: Get "http://10.132.0.175:15021/healthz/ready": dial tcp 10.132.0.175:15021: connect: connection refused 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-v7mn2 Startup probe failed: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused 45m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-r6cjm Startup probe error: Get "http://10.132.0.175:15021/healthz/ready": dial tcp 10.132.0.175:15021: connect: connection refused... 45m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default (combined from similar events): Scaled up replica set maas-default-gateway-openshift-default-7854949c9 from 7 to 10 45m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7854949c9 Created pod: maas-default-gateway-openshift-default-7854949c9-c62ln 45m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7854949c9 Created pod: maas-default-gateway-openshift-default-7854949c9-w6fls 45m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7854949c9 Created pod: maas-default-gateway-openshift-default-7854949c9-m4ph9 45m Normal Created pod/maas-default-gateway-openshift-default-7854949c9-m4ph9 Container created 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-7854949c9-w6fls Add eth0 [10.134.0.81/23] from ovn-kubernetes 45m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-c62ln Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/maas-default-gateway-openshift-default-7854949c9-w6fls Container created 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-7854949c9-m4ph9 Add eth0 [10.133.0.82/23] from ovn-kubernetes 45m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-m4ph9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/maas-default-gateway-openshift-default-7854949c9-m4ph9 Container started 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-7854949c9-c62ln Add eth0 [10.132.0.176/23] from ovn-kubernetes 45m Normal Created pod/maas-default-gateway-openshift-default-7854949c9-c62ln Container created 45m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-w6fls Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/maas-default-gateway-openshift-default-7854949c9-w6fls Container started 45m Normal Started pod/maas-default-gateway-openshift-default-7854949c9-c62ln Container started 44m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-770e67-openshift-default Scaled up replica set e2e-shared-a-w2-770e67-openshift-default-974c475f5 from 0 to 1 44m Normal AddedLabel gateway/e2e-shared-a-w2-770e67 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-770e67 44m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-770e67-openshift-default-974c475f5 Created pod: e2e-shared-a-w2-770e67-openshift-default-974c475f5-dlf59 44m Normal AddedInterface pod/e2e-shared-a-w2-770e67-openshift-default-974c475f5-dlf59 Add eth0 [10.132.0.177/23] from ovn-kubernetes 44m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-c62ln Startup probe error: Get "http://10.132.0.176:15021/healthz/ready": dial tcp 10.132.0.176:15021: connect: connection refused... 44m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-c62ln Startup probe failed: Get "http://10.132.0.176:15021/healthz/ready": dial tcp 10.132.0.176:15021: connect: connection refused 44m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-w6fls Startup probe failed: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused 44m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-w6fls Startup probe error: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused... 44m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-m4ph9 Startup probe error: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused... 44m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-m4ph9 Startup probe failed: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-770e67 Scaled up replica set payload-processing-e2e-shared-a-w2-770e67-dbf74bfbf from 0 to 1 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-770e67 Scaled up replica set payload-pre-processing-e2e-shared-a-w2-770e67-85547c9dc from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-770e67-dbf74bfbf Created pod: payload-processing-e2e-shared-a-w2-770e67-dbf74bfbf-vhp4v 44m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-770e67-dbf74bfbf Error creating: pods "payload-processing-e2e-shared-a-w2-770e67-dbf74bfbf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-770e67: serviceaccount "payload-processing-e2e-shared-a-w2-770e67" not found 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-770e67-85547c9dc Created pod: payload-pre-processing-e2e-shared-a-w2-770e67-85547c9dc-9mtrq 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-770e67-85547c9dc Error creating: pods "payload-pre-processing-e2e-shared-a-w2-770e67-85547c9dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-770e67: serviceaccount "payload-processing-e2e-shared-a-w2-770e67" not found 44m Normal Created pod/payload-processing-e2e-shared-a-w2-770e67-dbf74bfbf-vhp4v Container created 44m Normal Pulled pod/payload-processing-e2e-shared-a-w2-770e67-dbf74bfbf-vhp4v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-770e67-dbf74bfbf-vhp4v Add eth0 [10.132.0.179/23] from ovn-kubernetes 44m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-770e67-85547c9dc-9mtrq Container created 44m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-770e67-85547c9dc-9mtrq Container started 44m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-770e67-85547c9dc-9mtrq Add eth0 [10.132.0.178/23] from ovn-kubernetes 44m Normal Started pod/payload-processing-e2e-shared-a-w2-770e67-dbf74bfbf-vhp4v Container started 44m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-770e67-85547c9dc-9mtrq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal ScalingReplicaSet deployment/e2e-mt-5ac14ff6-openshift-default Scaled up replica set e2e-mt-5ac14ff6-openshift-default-5859bb68c7 from 0 to 1 44m Normal SuccessfulCreate replicaset/e2e-mt-5ac14ff6-openshift-default-5859bb68c7 Created pod: e2e-mt-5ac14ff6-openshift-default-5859bb68c7-d4zd6 44m Normal AddedLabel gateway/e2e-mt-5ac14ff6 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-5ac14ff6 44m Normal AddedInterface pod/e2e-mt-5ac14ff6-openshift-default-5859bb68c7-d4zd6 Add eth0 [10.132.0.180/23] from ovn-kubernetes 44m Warning ProbeError pod/e2e-ait-6765e083-openshift-default-8486d6c5fc-5df52 Readiness probe error: Get "http://10.134.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Warning Unhealthy pod/e2e-ait-6765e083-openshift-default-8486d6c5fc-5df52 Readiness probe failed: Get "http://10.134.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-5ac14ff6 Scaled up replica set payload-pre-processing-e2e-mt-5ac14ff6-7679584699 from 0 to 1 44m Warning Unhealthy pod/e2e-migrate-82cd41ab-gw-openshift-default-7c64cf7b5-x82rn Readiness probe failed: Get "http://10.132.0.170:15021/healthz/ready": read tcp 10.132.0.2:51874->10.132.0.170:15021: read: connection reset by peer 44m Normal Created pod/e2e-ait-6765e083-openshift-default-8486d6c5fc-5df52 Container created 44m Warning ProbeError pod/e2e-migrate-82cd41ab-gw-openshift-default-7c64cf7b5-x82rn Readiness probe error: Get "http://10.132.0.170:15021/healthz/ready": read tcp 10.132.0.2:51874->10.132.0.170:15021: read: connection reset by peer... 44m Normal Started pod/e2e-ait-6765e083-openshift-default-8486d6c5fc-5df52 Container started 44m Normal Pulled pod/e2e-ait-6765e083-openshift-default-8486d6c5fc-5df52 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-5ac14ff6 Scaled up replica set payload-processing-e2e-mt-5ac14ff6-cf94fcf6c from 0 to 1 44m Warning FailedCreate replicaset/payload-processing-e2e-mt-5ac14ff6-cf94fcf6c Error creating: pods "payload-processing-e2e-mt-5ac14ff6-cf94fcf6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-5ac14ff6: serviceaccount "payload-processing-e2e-mt-5ac14ff6" not found 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-5ac14ff6-7679584699 Created pod: payload-pre-processing-e2e-mt-5ac14ff6-7679584699-qqfnb 44m Normal Pulled pod/e2e-migrate-82cd41ab-gw-openshift-default-7c64cf7b5-x82rn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-5ac14ff6-cf94fcf6c Created pod: payload-processing-e2e-mt-5ac14ff6-cf94fcf6c-75dv9 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-5ac14ff6-7679584699 Error creating: pods "payload-pre-processing-e2e-mt-5ac14ff6-7679584699-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-5ac14ff6: serviceaccount "payload-processing-e2e-mt-5ac14ff6" not found 44m Normal AddedInterface pod/payload-pre-processing-e2e-mt-5ac14ff6-7679584699-qqfnb Add eth0 [10.132.0.183/23] from ovn-kubernetes 44m Normal Started pod/e2e-migrate-82cd41ab-gw-openshift-default-7c64cf7b5-x82rn Container started 44m Normal Created pod/e2e-migrate-82cd41ab-gw-openshift-default-7c64cf7b5-x82rn Container created 44m Normal Pulled pod/payload-processing-e2e-mt-5ac14ff6-cf94fcf6c-75dv9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-processing-e2e-mt-5ac14ff6-cf94fcf6c-75dv9 Add eth0 [10.132.0.182/23] from ovn-kubernetes 44m Normal Started pod/payload-pre-processing-e2e-mt-5ac14ff6-7679584699-qqfnb Container started 44m Normal Pulled pod/payload-pre-processing-e2e-mt-5ac14ff6-7679584699-qqfnb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-pre-processing-e2e-mt-5ac14ff6-7679584699-qqfnb Container created 44m Normal Killing pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-6bjb5 Stopping container istio-proxy 44m Normal Created pod/payload-processing-e2e-mt-5ac14ff6-cf94fcf6c-75dv9 Container created 44m Normal Started pod/payload-processing-e2e-mt-5ac14ff6-cf94fcf6c-75dv9 Container started 44m Normal SuccessfulDelete replicaset/e2e-worker-w5-3a7056-openshift-default-85ccfddcf Deleted pod: e2e-worker-w5-3a7056-openshift-default-85ccfddcf-6bjb5 44m Normal ScalingReplicaSet deployment/e2e-worker-w5-3a7056-openshift-default Scaled down replica set e2e-worker-w5-3a7056-openshift-default-85ccfddcf from 6 to 4 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-3a7056-openshift-default New size: 4; reason: All metrics below target 44m Normal SuccessfulDelete replicaset/e2e-worker-w5-3a7056-openshift-default-85ccfddcf Deleted pod: e2e-worker-w5-3a7056-openshift-default-85ccfddcf-rf5ck 44m Normal Killing pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-rf5ck Stopping container istio-proxy 44m Warning Unhealthy pod/e2e-ait-6765e083-openshift-default-8486d6c5fc-5df52 Startup probe failed: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused 44m Warning ProbeError pod/e2e-ait-6765e083-openshift-default-8486d6c5fc-5df52 Startup probe error: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused... 44m Warning ProbeError pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-rf5ck Readiness probe error: HTTP probe failed with statuscode: 503... 44m Warning ProbeError pod/e2e-migrate-82cd41ab-gw-openshift-default-7c64cf7b5-x82rn Startup probe error: Get "http://10.132.0.170:15021/healthz/ready": dial tcp 10.132.0.170:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-migrate-82cd41ab-gw-openshift-default-7c64cf7b5-x82rn Startup probe failed: Get "http://10.132.0.170:15021/healthz/ready": dial tcp 10.132.0.170:15021: connect: connection refused 44m Warning Unhealthy pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-rf5ck Readiness probe failed: HTTP probe failed with statuscode: 503 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-6765e083-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-6765e083-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-58ac09-openshift-default Scaled up replica set e2e-shared-b-w4-58ac09-openshift-default-74d79cb8cc from 0 to 1 44m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-58ac09-openshift-default-74d79cb8cc Created pod: e2e-shared-b-w4-58ac09-openshift-default-74d79cb8cc-g995s 44m Normal AddedLabel gateway/e2e-shared-b-w4-58ac09 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-58ac09 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-6765e083-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal AddedInterface pod/e2e-shared-b-w4-58ac09-openshift-default-74d79cb8cc-g995s Add eth0 [10.132.0.185/23] from ovn-kubernetes 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-6765e083-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal Killing pod/payload-processing-e2e-migrate-82cd41ab-f78868cf5-thnf4 Stopping container payload-processing 44m Normal Killing pod/payload-pre-processing-e2e-migrate-82cd41ab-799f447d67-ps6pq Stopping container payload-pre-processing 43m Warning Unhealthy pod/e2e-shared-a-w4-9e277a-openshift-default-59d89f6f57-gz72j Readiness probe failed: Get "http://10.132.0.171:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Warning ProbeError pod/e2e-shared-a-w4-9e277a-openshift-default-59d89f6f57-gz72j Readiness probe error: Get "http://10.132.0.171:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w4-58ac09 Scaled up replica set payload-processing-e2e-shared-b-w4-58ac09-c585d4959 from 0 to 1 43m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w4-58ac09-c585d4959 Error creating: pods "payload-processing-e2e-shared-b-w4-58ac09-c585d4959-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-58ac09: serviceaccount "payload-processing-e2e-shared-b-w4-58ac09" not found 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w4-58ac09-c585d4959 Created pod: payload-processing-e2e-shared-b-w4-58ac09-c585d4959-gx5f9 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w4-58ac09-7846d7dccf Error creating: pods "payload-pre-processing-e2e-shared-b-w4-58ac09-7846d7dccf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-58ac09: serviceaccount "payload-processing-e2e-shared-b-w4-58ac09" not found 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w4-58ac09 Scaled up replica set payload-pre-processing-e2e-shared-b-w4-58ac09-7846d7dccf from 0 to 1 43m Normal AddedInterface pod/payload-processing-e2e-shared-b-w4-58ac09-c585d4959-gx5f9 Add eth0 [10.133.0.84/23] from ovn-kubernetes 43m Normal Created pod/payload-processing-e2e-shared-b-w4-58ac09-c585d4959-gx5f9 Container created 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w4-58ac09-7846d7dccf Created pod: payload-pre-processing-e2e-shared-b-w4-58ac09-7846d7dccf-bch78 43m Normal Pulled pod/payload-processing-e2e-shared-b-w4-58ac09-c585d4959-gx5f9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-pre-processing-e2e-shared-b-w4-58ac09-7846d7dccf-bch78 Container started 43m Normal Created pod/payload-pre-processing-e2e-shared-b-w4-58ac09-7846d7dccf-bch78 Container created 43m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w4-58ac09-7846d7dccf-bch78 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w4-58ac09-7846d7dccf-bch78 Add eth0 [10.133.0.85/23] from ovn-kubernetes 43m Normal Started pod/payload-processing-e2e-shared-b-w4-58ac09-c585d4959-gx5f9 Container started 43m Normal Created pod/e2e-shared-a-w4-9e277a-openshift-default-59d89f6f57-gz72j Container created 43m Normal Started pod/e2e-shared-a-w4-9e277a-openshift-default-59d89f6f57-gz72j Container started 43m Normal Pulled pod/e2e-shared-a-w4-9e277a-openshift-default-59d89f6f57-gz72j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-82cd41ab-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-82cd41ab-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning Unhealthy pod/e2e-shared-a-w4-9e277a-openshift-default-59d89f6f57-gz72j Startup probe failed: Get "http://10.132.0.171:15021/healthz/ready": dial tcp 10.132.0.171:15021: connect: connection refused 43m Warning ProbeError pod/e2e-shared-a-w4-9e277a-openshift-default-59d89f6f57-gz72j Startup probe error: Get "http://10.132.0.171:15021/healthz/ready": dial tcp 10.132.0.171:15021: connect: connection refused... 43m Normal Killing pod/payload-processing-e2e-ait-6765e083-55c596cf9f-nrbv8 Stopping container payload-processing 43m Normal Killing pod/payload-pre-processing-e2e-ait-6765e083-6df79b4b87-jx2qz Stopping container payload-pre-processing 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-9e277a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-9e277a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal ScalingReplicaSet deployment/e2e-worker-w0-9686c3-openshift-default Scaled down replica set e2e-worker-w0-9686c3-openshift-default-85685b9d55 from 5 to 4 43m Normal SuccessfulDelete replicaset/e2e-worker-w0-9686c3-openshift-default-85685b9d55 Deleted pod: e2e-worker-w0-9686c3-openshift-default-85685b9d55-cj9vk 43m Normal Killing pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-cj9vk Stopping container istio-proxy 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-9686c3-openshift-default New size: 4; reason: All metrics below target 43m Warning Unhealthy pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-cj9vk Readiness probe failed: HTTP probe failed with statuscode: 503 43m Warning ProbeError pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-cj9vk Readiness probe error: HTTP probe failed with statuscode: 503... 43m Warning ProbeError pod/e2e-shared-a-w2-770e67-openshift-default-974c475f5-dlf59 Readiness probe error: Get "http://10.132.0.177:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Warning Unhealthy pod/e2e-shared-a-w2-770e67-openshift-default-974c475f5-dlf59 Readiness probe failed: Get "http://10.132.0.177:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Normal Started pod/e2e-shared-a-w2-770e67-openshift-default-974c475f5-dlf59 Container started 43m Normal Created pod/e2e-shared-a-w2-770e67-openshift-default-974c475f5-dlf59 Container created 43m Normal Pulled pod/e2e-shared-a-w2-770e67-openshift-default-974c475f5-dlf59 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-9e277a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-9e277a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning Unhealthy pod/e2e-shared-a-w2-770e67-openshift-default-974c475f5-dlf59 Startup probe failed: Get "http://10.132.0.177:15021/healthz/ready": dial tcp 10.132.0.177:15021: connect: connection refused 43m Warning ProbeError pod/e2e-shared-a-w2-770e67-openshift-default-974c475f5-dlf59 Startup probe error: Get "http://10.132.0.177:15021/healthz/ready": dial tcp 10.132.0.177:15021: connect: connection refused... 43m Normal Killing pod/e2e-migrate-82cd41ab-gw-openshift-default-7c64cf7b5-x82rn Stopping container istio-proxy 43m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-bf3e1c-openshift-default Scaled up replica set e2e-shared-a-w1-bf3e1c-openshift-default-b444fc74f from 0 to 1 43m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-bf3e1c-openshift-default-b444fc74f Created pod: e2e-shared-a-w1-bf3e1c-openshift-default-b444fc74f-fpcv7 43m Normal AddedLabel gateway/e2e-shared-a-w1-bf3e1c Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w1-bf3e1c 43m Normal SuccessfulDelete replicaset/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c Deleted pod: e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-jwhh6 43m Normal Killing pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-tpqgh Stopping container istio-proxy 43m Normal ScalingReplicaSet deployment/e2e-worker-w3-a4cacd-openshift-default Scaled down replica set e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c from 10 to 5 43m Normal Killing pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-fwfsb Stopping container istio-proxy 43m Normal Killing pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-z4np8 Stopping container istio-proxy 43m Normal Killing pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-jwhh6 Stopping container istio-proxy 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-9686c3-openshift-default New size: 3; reason: All metrics below target 43m Normal Killing pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-chgfh Stopping container istio-proxy 43m Normal Killing pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-6tlg2 Stopping container istio-proxy 43m Normal AddedInterface pod/e2e-shared-a-w1-bf3e1c-openshift-default-b444fc74f-fpcv7 Add eth0 [10.133.0.87/23] from ovn-kubernetes 43m Normal SuccessfulDelete replicaset/e2e-worker-w0-9686c3-openshift-default-85685b9d55 Deleted pod: e2e-worker-w0-9686c3-openshift-default-85685b9d55-z4np8 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-a4cacd-openshift-default New size: 5; reason: All metrics below target 43m Normal SuccessfulDelete replicaset/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c Deleted pod: e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-chgfh 43m Normal SuccessfulDelete replicaset/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c Deleted pod: e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-6tlg2 43m Normal SuccessfulDelete replicaset/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c Deleted pod: e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-fwfsb 43m Normal ScalingReplicaSet deployment/e2e-worker-w0-9686c3-openshift-default Scaled down replica set e2e-worker-w0-9686c3-openshift-default-85685b9d55 from 4 to 3 43m Normal SuccessfulDelete replicaset/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c Deleted pod: e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-tpqgh 43m Warning ProbeError pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-tpqgh Readiness probe error: HTTP probe failed with statuscode: 503... 43m Warning Unhealthy pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-tpqgh Readiness probe failed: HTTP probe failed with statuscode: 503 43m Warning ProbeError pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-6tlg2 Readiness probe error: HTTP probe failed with statuscode: 503... 43m Warning Unhealthy pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-6tlg2 Readiness probe failed: HTTP probe failed with statuscode: 503 43m Warning Unhealthy pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-chgfh Readiness probe failed: HTTP probe failed with statuscode: 503 43m Warning ProbeError pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-chgfh Readiness probe error: HTTP probe failed with statuscode: 503... 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w1-bf3e1c Scaled up replica set payload-processing-e2e-shared-a-w1-bf3e1c-75d744986c from 0 to 1 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w1-bf3e1c Scaled up replica set payload-pre-processing-e2e-shared-a-w1-bf3e1c-6bc6cfb875 from 0 to 1 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w1-bf3e1c-6bc6cfb875 Error creating: pods "payload-pre-processing-e2e-shared-a-w1-bf3e1c-6bc6cfb875-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-bf3e1c: serviceaccount "payload-processing-e2e-shared-a-w1-bf3e1c" not found 43m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w1-bf3e1c-75d744986c Error creating: pods "payload-processing-e2e-shared-a-w1-bf3e1c-75d744986c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-bf3e1c: serviceaccount "payload-processing-e2e-shared-a-w1-bf3e1c" not found 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w1-bf3e1c-75d744986c Created pod: payload-processing-e2e-shared-a-w1-bf3e1c-75d744986c-jnxtg 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w1-bf3e1c-6bc6cfb875 Created pod: payload-pre-processing-e2e-shared-a-w1-bf3e1c-6bc6cfb875-rhwwt 43m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w1-bf3e1c-6bc6cfb875-rhwwt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-processing-e2e-shared-a-w1-bf3e1c-75d744986c-jnxtg Container created 43m Normal Created pod/payload-pre-processing-e2e-shared-a-w1-bf3e1c-6bc6cfb875-rhwwt Container created 43m Normal Started pod/payload-processing-e2e-shared-a-w1-bf3e1c-75d744986c-jnxtg Container started 43m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w1-bf3e1c-6bc6cfb875-rhwwt Add eth0 [10.132.0.188/23] from ovn-kubernetes 43m Normal Pulled pod/payload-processing-e2e-shared-a-w1-bf3e1c-75d744986c-jnxtg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-processing-e2e-shared-a-w1-bf3e1c-75d744986c-jnxtg Add eth0 [10.132.0.187/23] from ovn-kubernetes 43m Normal Started pod/payload-pre-processing-e2e-shared-a-w1-bf3e1c-6bc6cfb875-rhwwt Container started 42m Normal Killing pod/e2e-ait-6765e083-openshift-default-8486d6c5fc-5df52 Stopping container istio-proxy 42m Normal AddedLabel gateway/e2e-shared-a-w0-ecf40a Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-ecf40a 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-770e67-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-770e67-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-ecf40a-openshift-default Scaled up replica set e2e-shared-a-w0-ecf40a-openshift-default-5fc9fb679c from 0 to 1 42m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-ecf40a-openshift-default-5fc9fb679c Created pod: e2e-shared-a-w0-ecf40a-openshift-default-5fc9fb679c-7fhf5 42m Normal AddedInterface pod/e2e-shared-a-w0-ecf40a-openshift-default-5fc9fb679c-7fhf5 Add eth0 [10.132.0.189/23] from ovn-kubernetes 42m Warning Unhealthy pod/e2e-ait-6765e083-openshift-default-8486d6c5fc-5df52 Readiness probe failed: HTTP probe failed with statuscode: 503 42m Warning ProbeError pod/e2e-ait-6765e083-openshift-default-8486d6c5fc-5df52 Readiness probe error: HTTP probe failed with statuscode: 503... 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w0-ecf40a Scaled up replica set payload-processing-e2e-shared-a-w0-ecf40a-5b5dd7dcff from 0 to 1 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w0-ecf40a Scaled up replica set payload-pre-processing-e2e-shared-a-w0-ecf40a-857b7d867f from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w0-ecf40a-5b5dd7dcff Created pod: payload-processing-e2e-shared-a-w0-ecf40a-5b5dd7dcff-nr722 42m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w0-ecf40a-5b5dd7dcff Error creating: pods "payload-processing-e2e-shared-a-w0-ecf40a-5b5dd7dcff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-ecf40a: serviceaccount "payload-processing-e2e-shared-a-w0-ecf40a" not found 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w0-ecf40a-857b7d867f Error creating: pods "payload-pre-processing-e2e-shared-a-w0-ecf40a-857b7d867f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-ecf40a: serviceaccount "payload-processing-e2e-shared-a-w0-ecf40a" not found 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w0-ecf40a-857b7d867f Created pod: payload-pre-processing-e2e-shared-a-w0-ecf40a-857b7d867f-zgpw9 42m Normal AddedLabel gateway/e2e-api-a-ba5674 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-ba5674 42m Normal SuccessfulCreate replicaset/e2e-api-a-ba5674-openshift-default-767964dd8c Created pod: e2e-api-a-ba5674-openshift-default-767964dd8c-9x84g 42m Normal ScalingReplicaSet deployment/e2e-api-a-ba5674-openshift-default Scaled up replica set e2e-api-a-ba5674-openshift-default-767964dd8c from 0 to 1 42m Normal Started pod/payload-pre-processing-e2e-shared-a-w0-ecf40a-857b7d867f-zgpw9 Container started 42m Normal AddedInterface pod/payload-processing-e2e-shared-a-w0-ecf40a-5b5dd7dcff-nr722 Add eth0 [10.132.0.190/23] from ovn-kubernetes 42m Normal Pulled pod/payload-processing-e2e-shared-a-w0-ecf40a-5b5dd7dcff-nr722 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Created pod/payload-processing-e2e-shared-a-w0-ecf40a-5b5dd7dcff-nr722 Container created 42m Normal Started pod/payload-processing-e2e-shared-a-w0-ecf40a-5b5dd7dcff-nr722 Container started 42m Normal AddedInterface pod/e2e-api-a-ba5674-openshift-default-767964dd8c-9x84g Add eth0 [10.132.0.193/23] from ovn-kubernetes 42m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w0-ecf40a-857b7d867f-zgpw9 Add eth0 [10.132.0.192/23] from ovn-kubernetes 42m Normal Created pod/payload-pre-processing-e2e-shared-a-w0-ecf40a-857b7d867f-zgpw9 Container created 42m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w0-ecf40a-857b7d867f-zgpw9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal ScalingReplicaSet deployment/e2e-worker-w1-355e1a-openshift-default Scaled down replica set e2e-worker-w1-355e1a-openshift-default-d7d9459f9 from 8 to 6 42m Normal Killing pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-jk6qn Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w1-355e1a-openshift-default-d7d9459f9 Deleted pod: e2e-worker-w1-355e1a-openshift-default-d7d9459f9-64gzp 42m Normal SuccessfulDelete replicaset/e2e-worker-w1-355e1a-openshift-default-d7d9459f9 Deleted pod: e2e-worker-w1-355e1a-openshift-default-d7d9459f9-jk6qn 42m Normal Killing pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-64gzp Stopping container istio-proxy 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-355e1a-openshift-default New size: 6; reason: All metrics below target 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-5ac14ff6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-5ac14ff6-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-b485d8-openshift-default Scaled up replica set e2e-shared-b-w2-b485d8-openshift-default-788458b655 from 0 to 1 42m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-b485d8-openshift-default-788458b655 Created pod: e2e-shared-b-w2-b485d8-openshift-default-788458b655-bff6t 42m Normal Pulled pod/e2e-mt-5ac14ff6-openshift-default-5859bb68c7-d4zd6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal AddedLabel gateway/e2e-shared-b-w2-b485d8 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-b485d8 42m Normal Started pod/e2e-mt-5ac14ff6-openshift-default-5859bb68c7-d4zd6 Container started 42m Normal Created pod/e2e-mt-5ac14ff6-openshift-default-5859bb68c7-d4zd6 Container created 42m Warning Unhealthy pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-jk6qn Readiness probe failed: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused 42m Warning ProbeError pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-jk6qn Readiness probe error: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused... 42m Normal AddedInterface pod/e2e-shared-b-w2-b485d8-openshift-default-788458b655-bff6t Add eth0 [10.132.0.194/23] from ovn-kubernetes 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-ba5674 Scaled up replica set payload-processing-e2e-api-a-ba5674-7b8cb7b449 from 0 to 1 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-ba5674 Scaled up replica set payload-pre-processing-e2e-api-a-ba5674-64fb9d85b7 from 0 to 1 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-ba5674-64fb9d85b7 Error creating: pods "payload-pre-processing-e2e-api-a-ba5674-64fb9d85b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-ba5674: serviceaccount "payload-processing-e2e-api-a-ba5674" not found 42m Warning FailedCreate replicaset/payload-processing-e2e-api-a-ba5674-7b8cb7b449 Error creating: pods "payload-processing-e2e-api-a-ba5674-7b8cb7b449-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-ba5674: serviceaccount "payload-processing-e2e-api-a-ba5674" not found 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-ba5674-64fb9d85b7 Created pod: payload-pre-processing-e2e-api-a-ba5674-64fb9d85b7-kq6fs 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-ba5674-7b8cb7b449 Created pod: payload-processing-e2e-api-a-ba5674-7b8cb7b449-mt424 42m Normal Created pod/payload-pre-processing-e2e-api-a-ba5674-64fb9d85b7-kq6fs Container created 42m Normal Started pod/payload-pre-processing-e2e-api-a-ba5674-64fb9d85b7-kq6fs Container started 42m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-ba5674-64fb9d85b7-kq6fs Add eth0 [10.132.0.195/23] from ovn-kubernetes 42m Normal Pulled pod/payload-pre-processing-e2e-api-a-ba5674-64fb9d85b7-kq6fs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Started pod/payload-processing-e2e-api-a-ba5674-7b8cb7b449-mt424 Container started 42m Normal Created pod/payload-processing-e2e-api-a-ba5674-7b8cb7b449-mt424 Container created 42m Normal Pulled pod/payload-processing-e2e-api-a-ba5674-7b8cb7b449-mt424 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-processing-e2e-api-a-ba5674-7b8cb7b449-mt424 Add eth0 [10.132.0.196/23] from ovn-kubernetes 42m Warning Unhealthy pod/e2e-mt-5ac14ff6-openshift-default-5859bb68c7-d4zd6 Startup probe failed: Get "http://10.132.0.180:15021/healthz/ready": dial tcp 10.132.0.180:15021: connect: connection refused 42m Warning ProbeError pod/e2e-mt-5ac14ff6-openshift-default-5859bb68c7-d4zd6 Startup probe error: Get "http://10.132.0.180:15021/healthz/ready": dial tcp 10.132.0.180:15021: connect: connection refused... 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-b485d8-657bb4cc85 Created pod: payload-processing-e2e-shared-b-w2-b485d8-657bb4cc85-6p2ht 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-b485d8-d5fd6c795 Error creating: pods "payload-pre-processing-e2e-shared-b-w2-b485d8-d5fd6c795-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-b485d8: serviceaccount "payload-processing-e2e-shared-b-w2-b485d8" not found 42m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-b485d8-657bb4cc85 Error creating: pods "payload-processing-e2e-shared-b-w2-b485d8-657bb4cc85-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-b485d8: serviceaccount "payload-processing-e2e-shared-b-w2-b485d8" not found 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-b485d8 Scaled up replica set payload-processing-e2e-shared-b-w2-b485d8-657bb4cc85 from 0 to 1 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-b485d8 Scaled up replica set payload-pre-processing-e2e-shared-b-w2-b485d8-d5fd6c795 from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-b485d8-d5fd6c795 Created pod: payload-pre-processing-e2e-shared-b-w2-b485d8-d5fd6c795-n8wxl 42m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-b485d8-d5fd6c795-n8wxl Add eth0 [10.132.0.197/23] from ovn-kubernetes 42m Normal Created pod/payload-processing-e2e-shared-b-w2-b485d8-657bb4cc85-6p2ht Container created 42m Normal Pulled pod/payload-processing-e2e-shared-b-w2-b485d8-657bb4cc85-6p2ht Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-355e1a-openshift-default New size: 5; reason: All metrics below target 42m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-b485d8-d5fd6c795-n8wxl Container started 42m Normal Killing pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-z9f5d Stopping container istio-proxy 42m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-b485d8-d5fd6c795-n8wxl Container created 42m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-b485d8-d5fd6c795-n8wxl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-b485d8-657bb4cc85-6p2ht Add eth0 [10.132.0.198/23] from ovn-kubernetes 42m Normal SuccessfulDelete replicaset/e2e-worker-w1-355e1a-openshift-default-d7d9459f9 Deleted pod: e2e-worker-w1-355e1a-openshift-default-d7d9459f9-z9f5d 42m Normal ScalingReplicaSet deployment/e2e-worker-w1-355e1a-openshift-default Scaled down replica set e2e-worker-w1-355e1a-openshift-default-d7d9459f9 from 6 to 5 42m Normal Started pod/payload-processing-e2e-shared-b-w2-b485d8-657bb4cc85-6p2ht Container started 42m Warning ProbeError pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-z9f5d Readiness probe error: Get "http://10.132.0.121:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Warning Unhealthy pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-z9f5d Readiness probe failed: Get "http://10.132.0.121:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Warning Unhealthy pod/e2e-shared-b-w4-58ac09-openshift-default-74d79cb8cc-g995s Readiness probe failed: Get "http://10.132.0.185:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Warning ProbeError pod/e2e-shared-b-w4-58ac09-openshift-default-74d79cb8cc-g995s Readiness probe error: Get "http://10.132.0.185:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Normal Created pod/e2e-shared-b-w4-58ac09-openshift-default-74d79cb8cc-g995s Container created 42m Normal Started pod/e2e-shared-b-w4-58ac09-openshift-default-74d79cb8cc-g995s Container started 42m Normal Pulled pod/e2e-shared-b-w4-58ac09-openshift-default-74d79cb8cc-g995s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal SuccessfulDelete replicaset/e2e-worker-w2-32a59b-openshift-default-86974868b5 Deleted pod: e2e-worker-w2-32a59b-openshift-default-86974868b5-2t6vs 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-32a59b-openshift-default New size: 7; reason: All metrics below target 42m Normal SuccessfulDelete replicaset/e2e-worker-w2-32a59b-openshift-default-86974868b5 Deleted pod: e2e-worker-w2-32a59b-openshift-default-86974868b5-9kbd6 42m Normal ScalingReplicaSet deployment/e2e-worker-w2-32a59b-openshift-default Scaled down replica set e2e-worker-w2-32a59b-openshift-default-86974868b5 from 10 to 7 42m Normal SuccessfulDelete replicaset/e2e-worker-w2-32a59b-openshift-default-86974868b5 Deleted pod: e2e-worker-w2-32a59b-openshift-default-86974868b5-hrs6j 42m Normal Killing pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-9kbd6 Stopping container istio-proxy 42m Normal Killing pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-2t6vs Stopping container istio-proxy 42m Normal Killing pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-hrs6j Stopping container istio-proxy 42m Warning ProbeError pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-hrs6j Readiness probe error: HTTP probe failed with statuscode: 503... 42m Warning Unhealthy pod/e2e-shared-b-w4-58ac09-openshift-default-74d79cb8cc-g995s Startup probe failed: Get "http://10.132.0.185:15021/healthz/ready": dial tcp 10.132.0.185:15021: connect: connection refused 42m Warning ProbeError pod/e2e-shared-b-w4-58ac09-openshift-default-74d79cb8cc-g995s Startup probe error: Get "http://10.132.0.185:15021/healthz/ready": dial tcp 10.132.0.185:15021: connect: connection refused... 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-5ac14ff6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-5ac14ff6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning Unhealthy pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-2t6vs Readiness probe failed: Get "http://10.132.0.134:15021/healthz/ready": dial tcp 10.132.0.134:15021: connect: connection refused 42m Warning ProbeError pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-2t6vs Readiness probe error: Get "http://10.132.0.134:15021/healthz/ready": dial tcp 10.132.0.134:15021: connect: connection refused... 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-58ac09-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-58ac09-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal SuccessfulDelete replicaset/e2e-worker-w6-fba266-openshift-default-5ff997597f Deleted pod: e2e-worker-w6-fba266-openshift-default-5ff997597f-ql4lz 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-fba266-openshift-default New size: 6; reason: All metrics below target 41m Normal SuccessfulDelete replicaset/e2e-worker-w6-fba266-openshift-default-5ff997597f Deleted pod: e2e-worker-w6-fba266-openshift-default-5ff997597f-cxpf5 41m Normal SuccessfulDelete replicaset/e2e-worker-w6-fba266-openshift-default-5ff997597f Deleted pod: e2e-worker-w6-fba266-openshift-default-5ff997597f-qkdmt 41m Normal SuccessfulDelete replicaset/e2e-worker-w6-fba266-openshift-default-5ff997597f Deleted pod: e2e-worker-w6-fba266-openshift-default-5ff997597f-tbwhw 41m Normal ScalingReplicaSet deployment/e2e-worker-w6-fba266-openshift-default Scaled down replica set e2e-worker-w6-fba266-openshift-default-5ff997597f from 10 to 6 41m Normal SuccessfulDelete replicaset/e2e-worker-w4-2e052a-openshift-default-6886cf66dc Deleted pod: e2e-worker-w4-2e052a-openshift-default-6886cf66dc-vdgg5 41m Normal ScalingReplicaSet deployment/e2e-worker-w4-2e052a-openshift-default Scaled down replica set e2e-worker-w4-2e052a-openshift-default-6886cf66dc from 10 to 6 41m Normal Killing pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-szxrv Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-cxpf5 Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-tbwhw Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-ql4lz Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-vdgg5 Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w4-2e052a-openshift-default-6886cf66dc Deleted pod: e2e-worker-w4-2e052a-openshift-default-6886cf66dc-592dz 41m Warning ProbeError pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-592dz Readiness probe error: HTTP probe failed with statuscode: 503... 41m Normal Killing pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-592dz Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-qkdmt Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w4-2e052a-openshift-default-6886cf66dc Deleted pod: e2e-worker-w4-2e052a-openshift-default-6886cf66dc-szxrv 41m Normal Killing pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-n5xtx Stopping container istio-proxy 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-2e052a-openshift-default New size: 6; reason: All metrics below target 41m Normal SuccessfulDelete replicaset/e2e-worker-w4-2e052a-openshift-default-6886cf66dc Deleted pod: e2e-worker-w4-2e052a-openshift-default-6886cf66dc-n5xtx 41m Warning ProbeError pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-n5xtx Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning ProbeError pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-tbwhw Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-cxpf5 Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning ProbeError pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-cxpf5 Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-58ac09-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-58ac09-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal AddedLabel gateway/e2e-api-a-e1f8f8 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-e1f8f8 41m Normal SuccessfulCreate replicaset/e2e-api-a-e1f8f8-openshift-default-676bdb65df Created pod: e2e-api-a-e1f8f8-openshift-default-676bdb65df-bsq58 41m Normal ScalingReplicaSet deployment/e2e-api-a-e1f8f8-openshift-default Scaled up replica set e2e-api-a-e1f8f8-openshift-default-676bdb65df from 0 to 1 41m Normal AddedInterface pod/e2e-api-a-e1f8f8-openshift-default-676bdb65df-bsq58 Add eth0 [10.132.0.200/23] from ovn-kubernetes 41m Warning FailedCreate replicaset/payload-processing-e2e-api-a-e1f8f8-67d6f99995 Error creating: pods "payload-processing-e2e-api-a-e1f8f8-67d6f99995-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-e1f8f8: serviceaccount "payload-processing-e2e-api-a-e1f8f8" not found 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-e1f8f8-9d858999 Error creating: pods "payload-pre-processing-e2e-api-a-e1f8f8-9d858999-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-e1f8f8: serviceaccount "payload-processing-e2e-api-a-e1f8f8" not found 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-e1f8f8-67d6f99995 Created pod: payload-processing-e2e-api-a-e1f8f8-67d6f99995-blb62 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-e1f8f8-9d858999 Created pod: payload-pre-processing-e2e-api-a-e1f8f8-9d858999-6gpzs 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-e1f8f8 Scaled up replica set payload-pre-processing-e2e-api-a-e1f8f8-9d858999 from 0 to 1 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-e1f8f8 Scaled up replica set payload-processing-e2e-api-a-e1f8f8-67d6f99995 from 0 to 1 41m Normal Started pod/payload-pre-processing-e2e-api-a-e1f8f8-9d858999-6gpzs Container started 41m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-e1f8f8-9d858999-6gpzs Add eth0 [10.132.0.201/23] from ovn-kubernetes 41m Normal Pulled pod/payload-pre-processing-e2e-api-a-e1f8f8-9d858999-6gpzs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-pre-processing-e2e-api-a-e1f8f8-9d858999-6gpzs Container created 41m Normal Started pod/payload-processing-e2e-api-a-e1f8f8-67d6f99995-blb62 Container started 41m Normal AddedInterface pod/payload-processing-e2e-api-a-e1f8f8-67d6f99995-blb62 Add eth0 [10.132.0.202/23] from ovn-kubernetes 41m Normal Pulled pod/payload-processing-e2e-api-a-e1f8f8-67d6f99995-blb62 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-processing-e2e-api-a-e1f8f8-67d6f99995-blb62 Container created 41m Normal Started pod/e2e-shared-a-w1-bf3e1c-openshift-default-b444fc74f-fpcv7 Container started 41m Normal Created pod/e2e-shared-a-w1-bf3e1c-openshift-default-b444fc74f-fpcv7 Container created 41m Normal Pulled pod/e2e-shared-a-w1-bf3e1c-openshift-default-b444fc74f-fpcv7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-bf3e1c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-bf3e1c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning Unhealthy pod/e2e-shared-a-w1-bf3e1c-openshift-default-b444fc74f-fpcv7 Startup probe failed: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused 41m Warning ProbeError pod/e2e-shared-a-w1-bf3e1c-openshift-default-b444fc74f-fpcv7 Startup probe error: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused... 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-bf3e1c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-bf3e1c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning Unhealthy pod/e2e-api-a-ba5674-openshift-default-767964dd8c-9x84g Readiness probe failed: Get "http://10.132.0.193:15021/healthz/ready": read tcp 10.132.0.2:58376->10.132.0.193:15021: read: connection reset by peer 40m Warning ProbeError pod/e2e-api-a-ba5674-openshift-default-767964dd8c-9x84g Readiness probe error: Get "http://10.132.0.193:15021/healthz/ready": read tcp 10.132.0.2:58376->10.132.0.193:15021: read: connection reset by peer... 40m Normal Pulled pod/e2e-api-a-ba5674-openshift-default-767964dd8c-9x84g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-api-a-ba5674-openshift-default-767964dd8c-9x84g Container created 40m Normal Started pod/e2e-api-a-ba5674-openshift-default-767964dd8c-9x84g Container started 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-9686c3-openshift-default New size: 2; reason: All metrics below target 40m Normal ScalingReplicaSet deployment/e2e-worker-w0-9686c3-openshift-default Scaled down replica set e2e-worker-w0-9686c3-openshift-default-85685b9d55 from 3 to 2 40m Normal SuccessfulDelete replicaset/e2e-worker-w0-9686c3-openshift-default-85685b9d55 Deleted pod: e2e-worker-w0-9686c3-openshift-default-85685b9d55-zfzp9 40m Normal ScalingReplicaSet deployment/e2e-worker-w2-32a59b-openshift-default Scaled down replica set e2e-worker-w2-32a59b-openshift-default-86974868b5 from 7 to 5 40m Normal SuccessfulDelete replicaset/e2e-worker-w2-32a59b-openshift-default-86974868b5 Deleted pod: e2e-worker-w2-32a59b-openshift-default-86974868b5-w296j 40m Normal Killing pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-zfzp9 Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w2-32a59b-openshift-default-86974868b5 Deleted pod: e2e-worker-w2-32a59b-openshift-default-86974868b5-t4rd9 40m Normal AddedLabel gateway/e2e-shared-b-w1-97e020 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w1-97e020 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-32a59b-openshift-default New size: 5; reason: All metrics below target 40m Normal Killing pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-t4rd9 Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-w296j Stopping container istio-proxy 40m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-97e020-openshift-default-777b4d5657 Created pod: e2e-shared-b-w1-97e020-openshift-default-777b4d5657-rfb2d 40m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-97e020-openshift-default Scaled up replica set e2e-shared-b-w1-97e020-openshift-default-777b4d5657 from 0 to 1 40m Normal AddedInterface pod/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-rfb2d Add eth0 [10.132.0.207/23] from ovn-kubernetes 40m Warning ProbeError pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-zfzp9 Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning ProbeError pod/e2e-api-a-ba5674-openshift-default-767964dd8c-9x84g Startup probe error: Get "http://10.132.0.193:15021/healthz/ready": dial tcp 10.132.0.193:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-api-a-ba5674-openshift-default-767964dd8c-9x84g Startup probe failed: Get "http://10.132.0.193:15021/healthz/ready": dial tcp 10.132.0.193:15021: connect: connection refused 40m Warning Unhealthy pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-zfzp9 Readiness probe failed: HTTP probe failed with statuscode: 503 40m Normal Killing pod/payload-processing-e2e-mt-5ac14ff6-cf94fcf6c-75dv9 Stopping container payload-processing 40m Normal Killing pod/payload-pre-processing-e2e-mt-5ac14ff6-7679584699-qqfnb Stopping container payload-pre-processing 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w1-97e020 Scaled up replica set payload-processing-e2e-shared-b-w1-97e020-b5c87c667 from 0 to 1 40m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w1-97e020-b5c87c667 Error creating: pods "payload-processing-e2e-shared-b-w1-97e020-b5c87c667-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-97e020: serviceaccount "payload-processing-e2e-shared-b-w1-97e020" not found 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w1-97e020-b5c87c667 Created pod: payload-processing-e2e-shared-b-w1-97e020-b5c87c667-vjfmc 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w1-97e020-76946549b7 Error creating: pods "payload-pre-processing-e2e-shared-b-w1-97e020-76946549b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-97e020: serviceaccount "payload-processing-e2e-shared-b-w1-97e020" not found 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w1-97e020-76946549b7 Created pod: payload-pre-processing-e2e-shared-b-w1-97e020-76946549b7-mf7h8 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w1-97e020 Scaled up replica set payload-pre-processing-e2e-shared-b-w1-97e020-76946549b7 from 0 to 1 40m Normal Started pod/payload-pre-processing-e2e-shared-b-w1-97e020-76946549b7-mf7h8 Container started 40m Normal Created pod/payload-pre-processing-e2e-shared-b-w1-97e020-76946549b7-mf7h8 Container created 40m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w1-97e020-76946549b7-mf7h8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w1-97e020-76946549b7-mf7h8 Add eth0 [10.132.0.209/23] from ovn-kubernetes 40m Normal Started pod/payload-processing-e2e-shared-b-w1-97e020-b5c87c667-vjfmc Container started 40m Normal Created pod/payload-processing-e2e-shared-b-w1-97e020-b5c87c667-vjfmc Container created 40m Normal Pulled pod/payload-processing-e2e-shared-b-w1-97e020-b5c87c667-vjfmc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/payload-processing-e2e-shared-b-w1-97e020-b5c87c667-vjfmc Add eth0 [10.132.0.208/23] from ovn-kubernetes 40m Normal SuccessfulDelete replicaset/e2e-worker-w5-3a7056-openshift-default-85ccfddcf Deleted pod: e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2mq99 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-3a7056-openshift-default New size: 3; reason: All metrics below target 40m Normal ScalingReplicaSet deployment/e2e-worker-w5-3a7056-openshift-default Scaled down replica set e2e-worker-w5-3a7056-openshift-default-85ccfddcf from 4 to 3 40m Normal Killing pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2mq99 Stopping container istio-proxy 40m Warning ProbeError pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2mq99 Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2mq99 Readiness probe failed: HTTP probe failed with statuscode: 503 40m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-a93912-openshift-default-65f75877d7 Created pod: e2e-shared-b-w0-a93912-openshift-default-65f75877d7-6pjbg 40m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-a93912-openshift-default Scaled up replica set e2e-shared-b-w0-a93912-openshift-default-65f75877d7 from 0 to 1 40m Normal AddedLabel gateway/e2e-shared-b-w0-a93912 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-a93912 40m Normal AddedInterface pod/e2e-shared-b-w0-a93912-openshift-default-65f75877d7-6pjbg Add eth0 [10.132.0.211/23] from ovn-kubernetes 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-ba5674-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-ba5674-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal Created pod/e2e-shared-a-w0-ecf40a-openshift-default-5fc9fb679c-7fhf5 Container created 40m Normal Pulled pod/e2e-shared-a-w0-ecf40a-openshift-default-5fc9fb679c-7fhf5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Started pod/e2e-shared-a-w0-ecf40a-openshift-default-5fc9fb679c-7fhf5 Container started 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-ecf40a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-ecf40a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w0-a93912 Scaled up replica set payload-processing-e2e-shared-b-w0-a93912-6dc7f84989 from 0 to 1 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w0-a93912 Scaled up replica set payload-pre-processing-e2e-shared-b-w0-a93912-574cd97df5 from 0 to 1 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w0-a93912-574cd97df5 Created pod: payload-pre-processing-e2e-shared-b-w0-a93912-574cd97df5-w5gbw 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w0-a93912-6dc7f84989 Created pod: payload-processing-e2e-shared-b-w0-a93912-6dc7f84989-48g7r 40m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w0-a93912-6dc7f84989 Error creating: pods "payload-processing-e2e-shared-b-w0-a93912-6dc7f84989-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-a93912: serviceaccount "payload-processing-e2e-shared-b-w0-a93912" not found 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w0-a93912-574cd97df5 Error creating: pods "payload-pre-processing-e2e-shared-b-w0-a93912-574cd97df5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-a93912: serviceaccount "payload-processing-e2e-shared-b-w0-a93912" not found 40m Normal Pulled pod/payload-processing-e2e-shared-b-w0-a93912-6dc7f84989-48g7r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w0-a93912-574cd97df5-w5gbw Add eth0 [10.132.0.212/23] from ovn-kubernetes 40m Normal AddedInterface pod/payload-processing-e2e-shared-b-w0-a93912-6dc7f84989-48g7r Add eth0 [10.132.0.213/23] from ovn-kubernetes 40m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w0-a93912-574cd97df5-w5gbw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Created pod/payload-pre-processing-e2e-shared-b-w0-a93912-574cd97df5-w5gbw Container created 40m Normal Started pod/payload-pre-processing-e2e-shared-b-w0-a93912-574cd97df5-w5gbw Container started 40m Normal Created pod/payload-processing-e2e-shared-b-w0-a93912-6dc7f84989-48g7r Container created 40m Normal Started pod/payload-processing-e2e-shared-b-w0-a93912-6dc7f84989-48g7r Container started 40m Warning Unhealthy pod/e2e-shared-a-w0-ecf40a-openshift-default-5fc9fb679c-7fhf5 Startup probe failed: Get "http://10.132.0.189:15021/healthz/ready": dial tcp 10.132.0.189:15021: connect: connection refused 40m Warning ProbeError pod/e2e-shared-a-w0-ecf40a-openshift-default-5fc9fb679c-7fhf5 Startup probe error: Get "http://10.132.0.189:15021/healthz/ready": dial tcp 10.132.0.189:15021: connect: connection refused... 40m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-m4w4c Readiness probe failed: Get "http://10.132.0.163:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-m4w4c Readiness probe error: Get "http://10.132.0.163:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-ba5674-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-ba5674-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Normal ScalingReplicaSet deployment/e2e-api-b-efd81c-openshift-default Scaled up replica set e2e-api-b-efd81c-openshift-default-7956bf8cfc from 0 to 1 39m Normal AddedLabel gateway/e2e-api-b-efd81c Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-efd81c 39m Normal SuccessfulCreate replicaset/e2e-api-b-efd81c-openshift-default-7956bf8cfc Created pod: e2e-api-b-efd81c-openshift-default-7956bf8cfc-cfjrn 39m Normal AddedInterface pod/e2e-api-b-efd81c-openshift-default-7956bf8cfc-cfjrn Add eth0 [10.132.0.214/23] from ovn-kubernetes 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-efd81c Scaled up replica set payload-processing-e2e-api-b-efd81c-768b7754c9 from 0 to 1 39m Warning FailedCreate replicaset/payload-processing-e2e-api-b-efd81c-768b7754c9 Error creating: pods "payload-processing-e2e-api-b-efd81c-768b7754c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-efd81c: serviceaccount "payload-processing-e2e-api-b-efd81c" not found 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-efd81c-768b7754c9 Created pod: payload-processing-e2e-api-b-efd81c-768b7754c9-hrk6f 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-efd81c-76f4d9c555 Error creating: pods "payload-pre-processing-e2e-api-b-efd81c-76f4d9c555-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-efd81c: serviceaccount "payload-processing-e2e-api-b-efd81c" not found 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-efd81c-76f4d9c555 Created pod: payload-pre-processing-e2e-api-b-efd81c-76f4d9c555-hgzvs 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-efd81c Scaled up replica set payload-pre-processing-e2e-api-b-efd81c-76f4d9c555 from 0 to 1 39m Normal Started pod/payload-pre-processing-e2e-api-b-efd81c-76f4d9c555-hgzvs Container started 39m Normal Created pod/payload-pre-processing-e2e-api-b-efd81c-76f4d9c555-hgzvs Container created 39m Normal Pulled pod/payload-pre-processing-e2e-api-b-efd81c-76f4d9c555-hgzvs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/payload-processing-e2e-api-b-efd81c-768b7754c9-hrk6f Add eth0 [10.132.0.215/23] from ovn-kubernetes 39m Normal Pulled pod/payload-processing-e2e-api-b-efd81c-768b7754c9-hrk6f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-efd81c-76f4d9c555-hgzvs Add eth0 [10.134.0.84/23] from ovn-kubernetes 39m Normal Created pod/payload-processing-e2e-api-b-efd81c-768b7754c9-hrk6f Container created 39m Normal Started pod/payload-processing-e2e-api-b-efd81c-768b7754c9-hrk6f Container started 39m Normal SuccessfulCreate replicaset/e2e-api-a-042f66-openshift-default-7859c48d9c Created pod: e2e-api-a-042f66-openshift-default-7859c48d9c-lqvl5 39m Normal AddedLabel gateway/e2e-api-a-042f66 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-042f66 39m Normal ScalingReplicaSet deployment/e2e-api-a-042f66-openshift-default Scaled up replica set e2e-api-a-042f66-openshift-default-7859c48d9c from 0 to 1 39m Normal AddedInterface pod/e2e-api-a-042f66-openshift-default-7859c48d9c-lqvl5 Add eth0 [10.132.0.216/23] from ovn-kubernetes 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-ecf40a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-ecf40a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal Pulled pod/e2e-shared-b-w2-b485d8-openshift-default-788458b655-bff6t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-shared-b-w2-b485d8-openshift-default-788458b655-bff6t Container created 39m Normal Started pod/e2e-shared-b-w2-b485d8-openshift-default-788458b655-bff6t Container started 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-042f66 Scaled up replica set payload-pre-processing-e2e-api-a-042f66-954b7d67f from 0 to 1 39m Warning FailedCreate replicaset/payload-processing-e2e-api-a-042f66-6f8988d659 Error creating: pods "payload-processing-e2e-api-a-042f66-6f8988d659-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-042f66: serviceaccount "payload-processing-e2e-api-a-042f66" not found 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-042f66 Scaled up replica set payload-processing-e2e-api-a-042f66-6f8988d659 from 0 to 1 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-042f66-954b7d67f Error creating: pods "payload-pre-processing-e2e-api-a-042f66-954b7d67f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-042f66: serviceaccount "payload-processing-e2e-api-a-042f66" not found 39m Normal AddedInterface pod/payload-processing-e2e-api-a-042f66-6f8988d659-hhk2t Add eth0 [10.134.0.85/23] from ovn-kubernetes 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-042f66-954b7d67f Created pod: payload-pre-processing-e2e-api-a-042f66-954b7d67f-8fk7f 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-042f66-6f8988d659 Created pod: payload-processing-e2e-api-a-042f66-6f8988d659-hhk2t 39m Normal Pulled pod/payload-processing-e2e-api-a-042f66-6f8988d659-hhk2t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-042f66-954b7d67f-8fk7f Add eth0 [10.134.0.86/23] from ovn-kubernetes 39m Normal Pulled pod/payload-pre-processing-e2e-api-a-042f66-954b7d67f-8fk7f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Created pod/payload-pre-processing-e2e-api-a-042f66-954b7d67f-8fk7f Container created 39m Normal Started pod/payload-processing-e2e-api-a-042f66-6f8988d659-hhk2t Container started 39m Normal Started pod/payload-pre-processing-e2e-api-a-042f66-954b7d67f-8fk7f Container started 39m Normal Created pod/payload-processing-e2e-api-a-042f66-6f8988d659-hhk2t Container created 39m Normal Killing pod/e2e-mt-5ac14ff6-openshift-default-5859bb68c7-d4zd6 Stopping container istio-proxy 39m Normal AddedLabel gateway/e2e-inf-a-4320e5 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-4320e5 39m Normal ScalingReplicaSet deployment/e2e-inf-a-4320e5-openshift-default Scaled up replica set e2e-inf-a-4320e5-openshift-default-656696d5c from 0 to 1 39m Warning Unhealthy pod/e2e-mt-5ac14ff6-openshift-default-5859bb68c7-d4zd6 Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning ProbeError pod/e2e-mt-5ac14ff6-openshift-default-5859bb68c7-d4zd6 Readiness probe error: HTTP probe failed with statuscode: 503... 39m Normal AddedInterface pod/e2e-inf-a-4320e5-openshift-default-656696d5c-2ltcj Add eth0 [10.134.0.87/23] from ovn-kubernetes 39m Normal SuccessfulCreate replicaset/e2e-inf-a-4320e5-openshift-default-656696d5c Created pod: e2e-inf-a-4320e5-openshift-default-656696d5c-2ltcj 39m Warning ProbeError pod/e2e-shared-b-w2-b485d8-openshift-default-788458b655-bff6t Startup probe error: Get "http://10.132.0.194:15021/healthz/ready": dial tcp 10.132.0.194:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-shared-b-w2-b485d8-openshift-default-788458b655-bff6t Startup probe failed: Get "http://10.132.0.194:15021/healthz/ready": dial tcp 10.132.0.194:15021: connect: connection refused 39m Normal ScalingReplicaSet deployment/e2e-worker-w5-3a7056-openshift-default Scaled down replica set e2e-worker-w5-3a7056-openshift-default-85ccfddcf from 3 to 2 39m Normal SuccessfulDelete replicaset/e2e-worker-w5-3a7056-openshift-default-85ccfddcf Deleted pod: e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2vv6p 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-3a7056-openshift-default New size: 2; reason: All metrics below target 39m Normal Killing pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2vv6p Stopping container istio-proxy 39m Warning Unhealthy pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2vv6p Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning ProbeError pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-2vv6p Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning ProbeError pod/e2e-api-a-e1f8f8-openshift-default-676bdb65df-bsq58 Readiness probe error: Get "http://10.132.0.200:15021/healthz/ready": read tcp 10.132.0.2:43926->10.132.0.200:15021: read: connection reset by peer... 39m Warning Unhealthy pod/e2e-api-a-e1f8f8-openshift-default-676bdb65df-bsq58 Readiness probe failed: Get "http://10.132.0.200:15021/healthz/ready": read tcp 10.132.0.2:43926->10.132.0.200:15021: read: connection reset by peer 39m Normal Created pod/e2e-api-a-e1f8f8-openshift-default-676bdb65df-bsq58 Container created 39m Normal Started pod/e2e-api-a-e1f8f8-openshift-default-676bdb65df-bsq58 Container started 39m Normal Pulled pod/e2e-api-a-e1f8f8-openshift-default-676bdb65df-bsq58 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-4320e5-57dcbbd995 Created pod: payload-pre-processing-e2e-inf-a-4320e5-57dcbbd995-2cfzs 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-4320e5 Scaled up replica set payload-pre-processing-e2e-inf-a-4320e5-57dcbbd995 from 0 to 1 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-4320e5 Scaled up replica set payload-processing-e2e-inf-a-4320e5-5854dbd4c from 0 to 1 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-4320e5-5854dbd4c Created pod: payload-processing-e2e-inf-a-4320e5-5854dbd4c-bkzr4 39m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-4320e5-5854dbd4c Error creating: pods "payload-processing-e2e-inf-a-4320e5-5854dbd4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-4320e5: serviceaccount "payload-processing-e2e-inf-a-4320e5" not found 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-b485d8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-4320e5-57dcbbd995 Error creating: pods "payload-pre-processing-e2e-inf-a-4320e5-57dcbbd995-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-4320e5: serviceaccount "payload-processing-e2e-inf-a-4320e5" not found 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-b485d8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal Pulled pod/payload-processing-e2e-inf-a-4320e5-5854dbd4c-bkzr4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/payload-processing-e2e-inf-a-4320e5-5854dbd4c-bkzr4 Add eth0 [10.133.0.89/23] from ovn-kubernetes 39m Normal Created pod/payload-processing-e2e-inf-a-4320e5-5854dbd4c-bkzr4 Container created 39m Normal Started pod/payload-processing-e2e-inf-a-4320e5-5854dbd4c-bkzr4 Container started 39m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-4320e5-57dcbbd995-2cfzs Add eth0 [10.132.0.217/23] from ovn-kubernetes 39m Normal Started pod/payload-pre-processing-e2e-inf-a-4320e5-57dcbbd995-2cfzs Container started 39m Normal Created pod/payload-pre-processing-e2e-inf-a-4320e5-57dcbbd995-2cfzs Container created 39m Normal Pulled pod/payload-pre-processing-e2e-inf-a-4320e5-57dcbbd995-2cfzs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Warning ProbeError pod/e2e-api-a-e1f8f8-openshift-default-676bdb65df-bsq58 Startup probe error: Get "http://10.132.0.200:15021/healthz/ready": dial tcp 10.132.0.200:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-api-a-e1f8f8-openshift-default-676bdb65df-bsq58 Startup probe failed: Get "http://10.132.0.200:15021/healthz/ready": dial tcp 10.132.0.200:15021: connect: connection refused 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-e1f8f8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-e1f8f8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-b485d8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-b485d8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-4320e5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-4320e5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-e1f8f8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-e1f8f8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Normal AddedLabel gateway/e2e-api-b-69e2b9 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-69e2b9 38m Normal ScalingReplicaSet deployment/e2e-worker-w3-a4cacd-openshift-default Scaled down replica set e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c from 5 to 3 38m Normal Killing pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-vbkvm Stopping container istio-proxy 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-a4cacd-openshift-default New size: 3; reason: All metrics below target 38m Normal SuccessfulDelete replicaset/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c Deleted pod: e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-c9kw5 38m Normal SuccessfulDelete replicaset/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c Deleted pod: e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-vbkvm 38m Normal Killing pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-c9kw5 Stopping container istio-proxy 38m Normal ScalingReplicaSet deployment/e2e-api-b-69e2b9-openshift-default Scaled up replica set e2e-api-b-69e2b9-openshift-default-545d9ffd69 from 0 to 1 38m Normal AddedInterface pod/e2e-api-b-69e2b9-openshift-default-545d9ffd69-nhgvx Add eth0 [10.134.0.89/23] from ovn-kubernetes 38m Normal SuccessfulCreate replicaset/e2e-api-b-69e2b9-openshift-default-545d9ffd69 Created pod: e2e-api-b-69e2b9-openshift-default-545d9ffd69-nhgvx 38m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-97e020-openshift-default Scaled up replica set e2e-shared-b-w1-97e020-openshift-default-777b4d5657 from 1 to 3 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-97e020-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 38m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-97e020-openshift-default-777b4d5657 Created pod: e2e-shared-b-w1-97e020-openshift-default-777b4d5657-f7z62 38m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-97e020-openshift-default-777b4d5657 Created pod: e2e-shared-b-w1-97e020-openshift-default-777b4d5657-z2f5g 38m Normal AddedInterface pod/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-f7z62 Add eth0 [10.134.0.90/23] from ovn-kubernetes 38m Warning ProbeError pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-c9kw5 Readiness probe error: HTTP probe failed with statuscode: 503... 38m Normal Started pod/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-z2f5g Container started 38m Normal Created pod/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-z2f5g Container created 38m Warning ProbeError pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-vbkvm Readiness probe error: HTTP probe failed with statuscode: 503... 38m Warning Unhealthy pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-vbkvm Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning Unhealthy pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-c9kw5 Readiness probe failed: HTTP probe failed with statuscode: 503 38m Normal Pulled pod/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-z2f5g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-z2f5g Add eth0 [10.133.0.90/23] from ovn-kubernetes 38m Normal Created pod/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-rfb2d Container created 38m Normal Started pod/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-f7z62 Container started 38m Normal Pulled pod/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-rfb2d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Pulled pod/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-f7z62 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Started pod/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-rfb2d Container started 38m Normal Created pod/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-f7z62 Container created 38m Warning Unhealthy pod/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-z2f5g Startup probe failed: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused 38m Warning ProbeError pod/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-f7z62 Startup probe error: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused... 38m Warning ProbeError pod/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-z2f5g Startup probe error: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-f7z62 Startup probe failed: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused 38m Warning ProbeError pod/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-rfb2d Startup probe error: Get "http://10.132.0.207:15021/healthz/ready": dial tcp 10.132.0.207:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-rfb2d Startup probe failed: Get "http://10.132.0.207:15021/healthz/ready": dial tcp 10.132.0.207:15021: connect: connection refused 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-69e2b9 Scaled up replica set payload-pre-processing-e2e-api-b-69e2b9-66b5bf4547 from 0 to 1 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-69e2b9 Scaled up replica set payload-processing-e2e-api-b-69e2b9-558bf74655 from 0 to 1 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-69e2b9-558bf74655 Created pod: payload-processing-e2e-api-b-69e2b9-558bf74655-wtjfw 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-69e2b9-66b5bf4547 Created pod: payload-pre-processing-e2e-api-b-69e2b9-66b5bf4547-4xrkq 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-69e2b9-66b5bf4547 Error creating: pods "payload-pre-processing-e2e-api-b-69e2b9-66b5bf4547-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-69e2b9: serviceaccount "payload-processing-e2e-api-b-69e2b9" not found 38m Warning FailedCreate replicaset/payload-processing-e2e-api-b-69e2b9-558bf74655 Error creating: pods "payload-processing-e2e-api-b-69e2b9-558bf74655-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-69e2b9: serviceaccount "payload-processing-e2e-api-b-69e2b9" not found 38m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-69e2b9-66b5bf4547-4xrkq Add eth0 [10.133.0.91/23] from ovn-kubernetes 38m Normal Created pod/payload-processing-e2e-api-b-69e2b9-558bf74655-wtjfw Container created 38m Normal Started pod/payload-processing-e2e-api-b-69e2b9-558bf74655-wtjfw Container started 38m Normal Started pod/payload-pre-processing-e2e-api-b-69e2b9-66b5bf4547-4xrkq Container started 38m Normal Created pod/payload-pre-processing-e2e-api-b-69e2b9-66b5bf4547-4xrkq Container created 38m Normal Pulled pod/payload-pre-processing-e2e-api-b-69e2b9-66b5bf4547-4xrkq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Pulled pod/payload-processing-e2e-api-b-69e2b9-558bf74655-wtjfw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/payload-processing-e2e-api-b-69e2b9-558bf74655-wtjfw Add eth0 [10.133.0.92/23] from ovn-kubernetes 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-97e020-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-97e020-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-97e020-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-97e020-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Normal Pulled pod/e2e-shared-b-w0-a93912-openshift-default-65f75877d7-6pjbg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Started pod/e2e-shared-b-w0-a93912-openshift-default-65f75877d7-6pjbg Container started 37m Normal Created pod/e2e-shared-b-w0-a93912-openshift-default-65f75877d7-6pjbg Container created 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-fba266-openshift-default New size: 5; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/e2e-worker-w6-fba266-openshift-default Scaled down replica set e2e-worker-w6-fba266-openshift-default-5ff997597f from 6 to 5 37m Normal SuccessfulDelete replicaset/e2e-worker-w6-fba266-openshift-default-5ff997597f Deleted pod: e2e-worker-w6-fba266-openshift-default-5ff997597f-lh6wf 37m Normal ScalingReplicaSet deployment/e2e-worker-w1-355e1a-openshift-default Scaled down replica set e2e-worker-w1-355e1a-openshift-default-d7d9459f9 from 5 to 3 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-2e052a-openshift-default New size: 5; reason: All metrics below target 37m Warning Unhealthy pod/e2e-shared-b-w0-a93912-openshift-default-65f75877d7-6pjbg Startup probe failed: Get "http://10.132.0.211:15021/healthz/ready": dial tcp 10.132.0.211:15021: connect: connection refused 37m Normal SuccessfulDelete replicaset/e2e-worker-w1-355e1a-openshift-default-d7d9459f9 Deleted pod: e2e-worker-w1-355e1a-openshift-default-d7d9459f9-phbjz 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-355e1a-openshift-default New size: 3; reason: All metrics below target 37m Warning ProbeError pod/e2e-shared-b-w0-a93912-openshift-default-65f75877d7-6pjbg Startup probe error: Get "http://10.132.0.211:15021/healthz/ready": dial tcp 10.132.0.211:15021: connect: connection refused... 37m Normal SuccessfulDelete replicaset/e2e-worker-w4-2e052a-openshift-default-6886cf66dc Deleted pod: e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t4xjk 37m Normal SuccessfulDelete replicaset/e2e-worker-w1-355e1a-openshift-default-d7d9459f9 Deleted pod: e2e-worker-w1-355e1a-openshift-default-d7d9459f9-nkkk8 37m Normal Killing pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-lh6wf Stopping container istio-proxy 37m Normal Killing pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t4xjk Stopping container istio-proxy 37m Normal Killing pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-phbjz Stopping container istio-proxy 37m Normal Killing pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-nkkk8 Stopping container istio-proxy 37m Normal ScalingReplicaSet deployment/e2e-worker-w4-2e052a-openshift-default Scaled down replica set e2e-worker-w4-2e052a-openshift-default-6886cf66dc from 6 to 5 37m Warning Unhealthy pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-lh6wf Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-lh6wf Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning ProbeError pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-phbjz Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-phbjz Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning Unhealthy pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t4xjk Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t4xjk Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-a93912-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-a93912-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Normal ScalingReplicaSet deployment/e2e-inf-a-65e151-openshift-default Scaled up replica set e2e-inf-a-65e151-openshift-default-5478585457 from 0 to 1 37m Normal Killing pod/payload-processing-e2e-shared-a-w1-bf3e1c-75d744986c-jnxtg Stopping container payload-processing 37m Normal SuccessfulCreate replicaset/e2e-inf-a-65e151-openshift-default-5478585457 Created pod: e2e-inf-a-65e151-openshift-default-5478585457-cxqss 37m Normal Killing pod/payload-pre-processing-e2e-shared-a-w1-bf3e1c-6bc6cfb875-rhwwt Stopping container payload-pre-processing 37m Normal AddedLabel gateway/e2e-inf-a-65e151 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-65e151 37m Normal AddedInterface pod/e2e-inf-a-65e151-openshift-default-5478585457-cxqss Add eth0 [10.133.0.94/23] from ovn-kubernetes 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-a93912-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-a93912-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-65e151 Scaled up replica set payload-pre-processing-e2e-inf-a-65e151-74c97f7547 from 0 to 1 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-65e151 Scaled up replica set payload-processing-e2e-inf-a-65e151-7cfcd654b7 from 0 to 1 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-65e151-74c97f7547 Error creating: pods "payload-pre-processing-e2e-inf-a-65e151-74c97f7547-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-65e151: serviceaccount "payload-processing-e2e-inf-a-65e151" not found 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-65e151-74c97f7547 Created pod: payload-pre-processing-e2e-inf-a-65e151-74c97f7547-dzk27 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-65e151-7cfcd654b7 Created pod: payload-processing-e2e-inf-a-65e151-7cfcd654b7-rg6rd 37m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-65e151-7cfcd654b7 Error creating: pods "payload-processing-e2e-inf-a-65e151-7cfcd654b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-65e151: serviceaccount "payload-processing-e2e-inf-a-65e151" not found 37m Normal Started pod/payload-processing-e2e-inf-a-65e151-7cfcd654b7-rg6rd Container started 37m Normal Created pod/payload-processing-e2e-inf-a-65e151-7cfcd654b7-rg6rd Container created 37m Normal Pulled pod/payload-processing-e2e-inf-a-65e151-7cfcd654b7-rg6rd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/payload-processing-e2e-inf-a-65e151-7cfcd654b7-rg6rd Add eth0 [10.133.0.96/23] from ovn-kubernetes 37m Normal Started pod/payload-pre-processing-e2e-inf-a-65e151-74c97f7547-dzk27 Container started 37m Normal Created pod/payload-pre-processing-e2e-inf-a-65e151-74c97f7547-dzk27 Container created 37m Normal Pulled pod/payload-pre-processing-e2e-inf-a-65e151-74c97f7547-dzk27 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-65e151-74c97f7547-dzk27 Add eth0 [10.133.0.95/23] from ovn-kubernetes 37m Normal SuccessfulDelete replicaset/e2e-worker-w6-fba266-openshift-default-5ff997597f Deleted pod: e2e-worker-w6-fba266-openshift-default-5ff997597f-d7xjn 37m Normal ScalingReplicaSet deployment/e2e-worker-w4-2e052a-openshift-default Scaled down replica set e2e-worker-w4-2e052a-openshift-default-6886cf66dc from 5 to 4 37m Normal Killing pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-5ktvr Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/e2e-worker-w6-fba266-openshift-default-5ff997597f Deleted pod: e2e-worker-w6-fba266-openshift-default-5ff997597f-5ktvr 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-32a59b-openshift-default New size: 3; reason: All metrics below target 37m Normal Killing pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-ntspd Stopping container istio-proxy 37m Normal Killing pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-d5ppq Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/e2e-worker-w2-32a59b-openshift-default-86974868b5 Deleted pod: e2e-worker-w2-32a59b-openshift-default-86974868b5-8vrc9 37m Normal Killing pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-8vrc9 Stopping container istio-proxy 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-2e052a-openshift-default New size: 4; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/e2e-worker-w2-32a59b-openshift-default Scaled down replica set e2e-worker-w2-32a59b-openshift-default-86974868b5 from 5 to 3 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-fba266-openshift-default New size: 3; reason: All metrics below target 37m Normal Killing pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-d7xjn Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/e2e-worker-w4-2e052a-openshift-default-6886cf66dc Deleted pod: e2e-worker-w4-2e052a-openshift-default-6886cf66dc-d5ppq 37m Normal ScalingReplicaSet deployment/e2e-worker-w6-fba266-openshift-default Scaled down replica set e2e-worker-w6-fba266-openshift-default-5ff997597f from 5 to 3 37m Normal SuccessfulDelete replicaset/e2e-worker-w2-32a59b-openshift-default-86974868b5 Deleted pod: e2e-worker-w2-32a59b-openshift-default-86974868b5-ntspd 37m Warning ProbeError pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-8vrc9 Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-8vrc9 Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning Unhealthy pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-ntspd Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-ntspd Readiness probe error: HTTP probe failed with statuscode: 503... 37m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-xl65b 37m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 37m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-xl65b Stopping container discovery 37m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 37m Normal Killing pod/payload-pre-processing-e2e-shared-a-w0-ecf40a-857b7d867f-zgpw9 Stopping container payload-pre-processing 37m Normal Killing pod/payload-processing-e2e-shared-a-w0-ecf40a-5b5dd7dcff-nr722 Stopping container payload-processing 36m Normal Pulled pod/e2e-inf-a-4320e5-openshift-default-656696d5c-2ltcj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/e2e-inf-a-4320e5-openshift-default-656696d5c-2ltcj Container started 36m Normal Created pod/e2e-inf-a-4320e5-openshift-default-656696d5c-2ltcj Container created 36m Normal Started pod/e2e-api-a-042f66-openshift-default-7859c48d9c-lqvl5 Container started 36m Normal Pulled pod/e2e-api-a-042f66-openshift-default-7859c48d9c-lqvl5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Created pod/e2e-api-a-042f66-openshift-default-7859c48d9c-lqvl5 Container created 36m Warning Unhealthy pod/e2e-inf-a-4320e5-openshift-default-656696d5c-2ltcj Startup probe failed: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused 36m Warning ProbeError pod/e2e-inf-a-4320e5-openshift-default-656696d5c-2ltcj Startup probe error: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused... 36m Warning ProbeError pod/e2e-api-a-042f66-openshift-default-7859c48d9c-lqvl5 Startup probe error: Get "http://10.132.0.216:15021/healthz/ready": dial tcp 10.132.0.216:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-api-a-042f66-openshift-default-7859c48d9c-lqvl5 Startup probe failed: Get "http://10.132.0.216:15021/healthz/ready": dial tcp 10.132.0.216:15021: connect: connection refused 36m Normal Killing pod/payload-processing-e2e-api-a-ba5674-7b8cb7b449-mt424 Stopping container payload-processing 36m Normal Killing pod/payload-pre-processing-e2e-api-a-ba5674-64fb9d85b7-kq6fs Stopping container payload-pre-processing 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-4320e5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-4320e5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal Killing pod/payload-pre-processing-e2e-api-a-042f66-954b7d67f-8fk7f Stopping container payload-pre-processing 36m Normal Killing pod/payload-processing-e2e-api-a-042f66-6f8988d659-hhk2t Stopping container payload-processing 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-042f66-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-042f66-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning Unhealthy pod/e2e-api-b-efd81c-openshift-default-7956bf8cfc-cfjrn Readiness probe failed: Get "http://10.132.0.214:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 36m Warning ProbeError pod/e2e-api-b-efd81c-openshift-default-7956bf8cfc-cfjrn Readiness probe error: Get "http://10.132.0.214:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 36m Normal Pulled pod/e2e-api-b-efd81c-openshift-default-7956bf8cfc-cfjrn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/e2e-api-b-efd81c-openshift-default-7956bf8cfc-cfjrn Container started 36m Normal Created pod/e2e-api-b-efd81c-openshift-default-7956bf8cfc-cfjrn Container created 36m Normal Killing pod/payload-processing-e2e-inf-a-4320e5-5854dbd4c-bkzr4 Stopping container payload-processing 36m Normal Killing pod/payload-pre-processing-e2e-inf-a-4320e5-57dcbbd995-2cfzs Stopping container payload-pre-processing 36m Warning Unhealthy pod/e2e-api-b-efd81c-openshift-default-7956bf8cfc-cfjrn Startup probe failed: Get "http://10.132.0.214:15021/healthz/ready": dial tcp 10.132.0.214:15021: connect: connection refused 36m Warning ProbeError pod/e2e-api-b-efd81c-openshift-default-7956bf8cfc-cfjrn Startup probe error: Get "http://10.132.0.214:15021/healthz/ready": dial tcp 10.132.0.214:15021: connect: connection refused... 36m Warning Unhealthy pod/payload-processing-e2e-inf-a-4320e5-5854dbd4c-bkzr4 Readiness probe failed: timeout: failed to connect service "10.133.0.89:50052" within 1s: context deadline exceeded 36m Normal Killing pod/e2e-api-a-042f66-openshift-default-7859c48d9c-lqvl5 Stopping container istio-proxy 36m Normal AddedLabel gateway/e2e-inf-a-9dd9f2 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-9dd9f2 36m Normal ScalingReplicaSet deployment/e2e-inf-a-9dd9f2-openshift-default Scaled up replica set e2e-inf-a-9dd9f2-openshift-default-db6f4568c from 0 to 1 36m Normal AddedInterface pod/e2e-inf-a-9dd9f2-openshift-default-db6f4568c-k2wkl Add eth0 [10.134.0.97/23] from ovn-kubernetes 36m Normal SuccessfulCreate replicaset/e2e-inf-a-9dd9f2-openshift-default-db6f4568c Created pod: e2e-inf-a-9dd9f2-openshift-default-db6f4568c-k2wkl 35m Normal Killing pod/e2e-inf-a-4320e5-openshift-default-656696d5c-2ltcj Stopping container istio-proxy 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-9dd9f2 Scaled up replica set payload-pre-processing-e2e-inf-a-9dd9f2-58b64d8db7 from 0 to 1 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-9dd9f2 Scaled up replica set payload-processing-e2e-inf-a-9dd9f2-674c89d7cc from 0 to 1 35m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-9dd9f2-674c89d7cc Error creating: pods "payload-processing-e2e-inf-a-9dd9f2-674c89d7cc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-9dd9f2: serviceaccount "payload-processing-e2e-inf-a-9dd9f2" not found 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-9dd9f2-674c89d7cc Created pod: payload-processing-e2e-inf-a-9dd9f2-674c89d7cc-s977s 35m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-9dd9f2-58b64d8db7-7x68t Add eth0 [10.134.0.98/23] from ovn-kubernetes 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-9dd9f2-58b64d8db7 Error creating: pods "payload-pre-processing-e2e-inf-a-9dd9f2-58b64d8db7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-9dd9f2: serviceaccount "payload-processing-e2e-inf-a-9dd9f2" not found 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-9dd9f2-58b64d8db7 Created pod: payload-pre-processing-e2e-inf-a-9dd9f2-58b64d8db7-7x68t 35m Normal Created pod/payload-pre-processing-e2e-inf-a-9dd9f2-58b64d8db7-7x68t Container created 35m Warning ProbeError pod/e2e-shared-a-w1-bf3e1c-openshift-default-b444fc74f-fpcv7 Readiness probe error: HTTP probe failed with statuscode: 503... 35m Normal Started pod/payload-processing-e2e-inf-a-9dd9f2-674c89d7cc-s977s Container started 35m Normal Created pod/payload-processing-e2e-inf-a-9dd9f2-674c89d7cc-s977s Container created 35m Normal Pulled pod/payload-processing-e2e-inf-a-9dd9f2-674c89d7cc-s977s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/payload-processing-e2e-inf-a-9dd9f2-674c89d7cc-s977s Add eth0 [10.134.0.99/23] from ovn-kubernetes 35m Normal Started pod/payload-pre-processing-e2e-inf-a-9dd9f2-58b64d8db7-7x68t Container started 35m Normal Pulled pod/payload-pre-processing-e2e-inf-a-9dd9f2-58b64d8db7-7x68t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Killing pod/e2e-shared-a-w1-bf3e1c-openshift-default-b444fc74f-fpcv7 Stopping container istio-proxy 35m Warning Unhealthy pod/e2e-shared-a-w1-bf3e1c-openshift-default-b444fc74f-fpcv7 Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-efd81c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-efd81c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Normal Killing pod/payload-processing-e2e-shared-b-w1-97e020-b5c87c667-vjfmc Stopping container payload-processing 35m Normal Killing pod/payload-pre-processing-e2e-shared-b-w1-97e020-76946549b7-mf7h8 Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-worker-w6-fba266-56687d67d5-5d54f Stopping container payload-processing 35m Normal Killing pod/payload-pre-processing-e2e-worker-w6-fba266-5df47f6575-qfhx2 Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-api-a-e1f8f8-67d6f99995-blb62 Stopping container payload-processing 35m Normal Killing pod/payload-pre-processing-e2e-api-a-e1f8f8-9d858999-6gpzs Stopping container payload-pre-processing 35m Normal Killing pod/e2e-shared-a-w0-ecf40a-openshift-default-5fc9fb679c-7fhf5 Stopping container istio-proxy 35m Warning Unhealthy pod/e2e-shared-a-w0-ecf40a-openshift-default-5fc9fb679c-7fhf5 Readiness probe failed: Get "http://10.132.0.189:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 35m Warning ProbeError pod/e2e-shared-a-w0-ecf40a-openshift-default-5fc9fb679c-7fhf5 Readiness probe error: Get "http://10.132.0.189:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 35m Warning Unhealthy pod/payload-processing-e2e-api-a-e1f8f8-67d6f99995-blb62 Readiness probe failed: timeout: failed to connect service "10.132.0.202:50052" within 1s: context deadline exceeded 35m Normal Killing pod/payload-processing-e2e-shared-b-w0-a93912-6dc7f84989-48g7r Stopping container payload-processing 35m Normal Killing pod/payload-pre-processing-e2e-shared-b-w0-a93912-574cd97df5-w5gbw Stopping container payload-pre-processing 35m Warning Unhealthy pod/e2e-api-b-69e2b9-openshift-default-545d9ffd69-nhgvx Readiness probe failed: Get "http://10.134.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 35m Warning ProbeError pod/e2e-api-b-69e2b9-openshift-default-545d9ffd69-nhgvx Readiness probe error: Get "http://10.134.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 35m Normal Created pod/e2e-api-b-69e2b9-openshift-default-545d9ffd69-nhgvx Container created 35m Normal Started pod/e2e-api-b-69e2b9-openshift-default-545d9ffd69-nhgvx Container started 35m Normal Pulled pod/e2e-api-b-69e2b9-openshift-default-545d9ffd69-nhgvx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Warning ProbeError pod/e2e-api-b-69e2b9-openshift-default-545d9ffd69-nhgvx Startup probe error: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-api-b-69e2b9-openshift-default-545d9ffd69-nhgvx Startup probe failed: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-69e2b9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-69e2b9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-69e2b9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-69e2b9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7854949c9 Deleted pod: maas-default-gateway-openshift-default-7854949c9-vtbjc 34m Normal Killing pod/maas-default-gateway-openshift-default-7854949c9-c62ln Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7854949c9 Deleted pod: maas-default-gateway-openshift-default-7854949c9-m4w4c 34m Normal Killing pod/maas-default-gateway-openshift-default-7854949c9-m4w4c Stopping container istio-proxy 34m Normal Killing pod/maas-default-gateway-openshift-default-7854949c9-vtbjc Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7854949c9 Deleted pod: maas-default-gateway-openshift-default-7854949c9-r6cjm 34m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7854949c9 Deleted pod: maas-default-gateway-openshift-default-7854949c9-c62ln 34m Normal Killing pod/maas-default-gateway-openshift-default-7854949c9-r6cjm Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/e2e-worker-w4-2e052a-openshift-default-6886cf66dc Deleted pod: e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t7qrn 34m Normal ScalingReplicaSet deployment/e2e-worker-w4-2e052a-openshift-default Scaled down replica set e2e-worker-w4-2e052a-openshift-default-6886cf66dc from 4 to 3 34m Normal Killing pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-t7qrn Stopping container istio-proxy 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-2e052a-openshift-default New size: 3; reason: All metrics below target 34m Normal Killing pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-xk4f4 Stopping container istio-proxy 34m Normal ScalingReplicaSet deployment/e2e-worker-w0-9686c3-openshift-default Scaled down replica set e2e-worker-w0-9686c3-openshift-default-85685b9d55 from 2 to 1 34m Normal SuccessfulDelete replicaset/e2e-worker-w0-9686c3-openshift-default-85685b9d55 Deleted pod: e2e-worker-w0-9686c3-openshift-default-85685b9d55-xk4f4 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-9686c3-openshift-default New size: 1; reason: All metrics below target 34m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-c62ln Readiness probe error: Get "http://10.132.0.176:15021/healthz/ready": dial tcp 10.132.0.176:15021: connect: connection refused... 34m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-c62ln Readiness probe failed: Get "http://10.132.0.176:15021/healthz/ready": dial tcp 10.132.0.176:15021: connect: connection refused 34m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-r6cjm Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-r6cjm Readiness probe error: HTTP probe failed with statuscode: 503... 34m Warning ProbeError pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-xk4f4 Readiness probe error: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-xk4f4 Readiness probe failed: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused 34m Normal Killing pod/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-f7z62 Stopping container istio-proxy 34m Normal Killing pod/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-z2f5g Stopping container istio-proxy 34m Normal Killing pod/e2e-shared-b-w1-97e020-openshift-default-777b4d5657-rfb2d Stopping container istio-proxy 34m Normal AddedLabel gateway/e2e-mt-3d74f801 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-3d74f801 34m Normal ScalingReplicaSet deployment/e2e-mt-3d74f801-openshift-default Scaled up replica set e2e-mt-3d74f801-openshift-default-78ddf467c from 0 to 1 34m Normal SuccessfulCreate replicaset/e2e-mt-3d74f801-openshift-default-78ddf467c Created pod: e2e-mt-3d74f801-openshift-default-78ddf467c-kv428 34m Normal ScalingReplicaSet deployment/e2e-worker-w3-a4cacd-openshift-default Scaled down replica set e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c from 3 to 2 34m Normal SuccessfulDelete replicaset/e2e-worker-w2-32a59b-openshift-default-86974868b5 Deleted pod: e2e-worker-w2-32a59b-openshift-default-86974868b5-67lkx 34m Normal Killing pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-67lkx Stopping container istio-proxy 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-a4cacd-openshift-default New size: 2; reason: All metrics below target 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-3a7056-openshift-default New size: 1; reason: All metrics below target 34m Normal Killing pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-s4jqx Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c Deleted pod: e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-5kgqh 34m Normal Killing pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-xp5mr Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/e2e-worker-w1-355e1a-openshift-default-d7d9459f9 Deleted pod: e2e-worker-w1-355e1a-openshift-default-d7d9459f9-b6d57 34m Normal SuccessfulDelete replicaset/e2e-worker-w4-2e052a-openshift-default-6886cf66dc Deleted pod: e2e-worker-w4-2e052a-openshift-default-6886cf66dc-xp5mr 34m Normal ScalingReplicaSet deployment/e2e-worker-w5-3a7056-openshift-default Scaled down replica set e2e-worker-w5-3a7056-openshift-default-85ccfddcf from 2 to 1 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-2e052a-openshift-default New size: 2; reason: All metrics below target 34m Normal Killing pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-b6d57 Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/e2e-worker-w5-3a7056-openshift-default-85ccfddcf Deleted pod: e2e-worker-w5-3a7056-openshift-default-85ccfddcf-s4jqx 34m Normal ScalingReplicaSet deployment/e2e-worker-w1-355e1a-openshift-default Scaled down replica set e2e-worker-w1-355e1a-openshift-default-d7d9459f9 from 3 to 2 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-32a59b-openshift-default New size: 2; reason: All metrics below target 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-355e1a-openshift-default New size: 2; reason: All metrics below target 34m Normal Killing pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-5kgqh Stopping container istio-proxy 34m Normal Killing pod/payload-pre-processing-e2e-inf-a-65e151-74c97f7547-dzk27 Stopping container payload-pre-processing 34m Normal Killing pod/payload-processing-e2e-inf-a-65e151-7cfcd654b7-rg6rd Stopping container payload-processing 34m Normal AddedInterface pod/e2e-mt-3d74f801-openshift-default-78ddf467c-kv428 Add eth0 [10.132.0.234/23] from ovn-kubernetes 34m Warning ProbeError pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-b6d57 Readiness probe error: HTTP probe failed with statuscode: 503... 34m Warning Unhealthy pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-b6d57 Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning Unhealthy pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-xp5mr Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning ProbeError pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-5kgqh Readiness probe error: HTTP probe failed with statuscode: 503... 34m Warning Unhealthy pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-5kgqh Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning ProbeError pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-xp5mr Readiness probe error: HTTP probe failed with statuscode: 503... 34m Warning Unhealthy pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-67lkx Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning ProbeError pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-67lkx Readiness probe error: HTTP probe failed with statuscode: 503... 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-3d74f801 Scaled up replica set payload-pre-processing-e2e-mt-3d74f801-67d6d795dc from 0 to 1 34m Warning ProbeError pod/e2e-inf-a-65e151-openshift-default-5478585457-cxqss Readiness probe error: Get "http://10.133.0.94:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 34m Warning Unhealthy pod/e2e-inf-a-65e151-openshift-default-5478585457-cxqss Readiness probe failed: Get "http://10.133.0.94:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-3d74f801 Scaled up replica set payload-processing-e2e-mt-3d74f801-d4cf58969 from 0 to 1 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-3d74f801-d4cf58969 Created pod: payload-processing-e2e-mt-3d74f801-d4cf58969-wlgw2 34m Warning FailedCreate replicaset/payload-processing-e2e-mt-3d74f801-d4cf58969 Error creating: pods "payload-processing-e2e-mt-3d74f801-d4cf58969-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-3d74f801: serviceaccount "payload-processing-e2e-mt-3d74f801" not found 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-3d74f801-67d6d795dc Error creating: pods "payload-pre-processing-e2e-mt-3d74f801-67d6d795dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-3d74f801: serviceaccount "payload-processing-e2e-mt-3d74f801" not found 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-3d74f801-67d6d795dc Created pod: payload-pre-processing-e2e-mt-3d74f801-67d6d795dc-65gv9 34m Normal AddedInterface pod/payload-processing-e2e-mt-3d74f801-d4cf58969-wlgw2 Add eth0 [10.132.0.235/23] from ovn-kubernetes 34m Normal Created pod/payload-pre-processing-e2e-mt-3d74f801-67d6d795dc-65gv9 Container created 34m Normal Pulled pod/payload-processing-e2e-mt-3d74f801-d4cf58969-wlgw2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Created pod/e2e-inf-a-65e151-openshift-default-5478585457-cxqss Container created 34m Normal Pulled pod/e2e-inf-a-65e151-openshift-default-5478585457-cxqss Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Started pod/e2e-inf-a-65e151-openshift-default-5478585457-cxqss Container started 34m Normal AddedInterface pod/payload-pre-processing-e2e-mt-3d74f801-67d6d795dc-65gv9 Add eth0 [10.134.0.110/23] from ovn-kubernetes 34m Normal Killing pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-wfcwg Stopping container istio-proxy 34m Normal Killing pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-dn2p9 Stopping container istio-proxy 34m Normal Killing pod/e2e-worker-w6-fba266-openshift-default-5ff997597f-jdpnw Stopping container istio-proxy 34m Normal Created pod/payload-processing-e2e-mt-3d74f801-d4cf58969-wlgw2 Container created 34m Normal Started pod/payload-processing-e2e-mt-3d74f801-d4cf58969-wlgw2 Container started 34m Normal Started pod/payload-pre-processing-e2e-mt-3d74f801-67d6d795dc-65gv9 Container started 34m Normal Pulled pod/payload-pre-processing-e2e-mt-3d74f801-67d6d795dc-65gv9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Warning FailedCreate replicaset/e2e-shared-b-w0-a93912-openshift-default-65f75877d7 Error creating: pods "e2e-shared-b-w0-a93912-openshift-default-65f75877d7-" is forbidden: error looking up service account openshift-ingress/e2e-shared-b-w0-a93912-openshift-default: serviceaccount "e2e-shared-b-w0-a93912-openshift-default" not found 33m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-a93912-openshift-default Scaled up replica set e2e-shared-b-w0-a93912-openshift-default-65f75877d7 from 0 to 1 33m Normal Killing pod/e2e-shared-b-w0-a93912-openshift-default-65f75877d7-6pjbg Stopping container istio-proxy 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-3d74f801-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-3d74f801-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning Unhealthy pod/e2e-inf-a-65e151-openshift-default-5478585457-cxqss Startup probe failed: Get "http://10.133.0.94:15021/healthz/ready": dial tcp 10.133.0.94:15021: connect: connection refused 33m Warning ProbeError pod/e2e-shared-b-w0-a93912-openshift-default-65f75877d7-6pjbg Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning ProbeError pod/e2e-inf-a-65e151-openshift-default-5478585457-cxqss Startup probe error: Get "http://10.133.0.94:15021/healthz/ready": dial tcp 10.133.0.94:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-shared-b-w0-a93912-openshift-default-65f75877d7-6pjbg Readiness probe failed: HTTP probe failed with statuscode: 503 33m Normal Created pod/e2e-inf-a-9dd9f2-openshift-default-db6f4568c-k2wkl Container created 33m Normal Pulled pod/e2e-inf-a-9dd9f2-openshift-default-db6f4568c-k2wkl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Started pod/e2e-inf-a-9dd9f2-openshift-default-db6f4568c-k2wkl Container started 33m Normal Killing pod/e2e-api-a-ba5674-openshift-default-767964dd8c-9x84g Stopping container istio-proxy 33m Warning ProbeError pod/e2e-inf-a-9dd9f2-openshift-default-db6f4568c-k2wkl Startup probe error: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-inf-a-9dd9f2-openshift-default-db6f4568c-k2wkl Startup probe failed: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused 33m Normal Killing pod/payload-processing-e2e-api-b-efd81c-768b7754c9-hrk6f Stopping container payload-processing 33m Normal Killing pod/payload-pre-processing-e2e-api-b-efd81c-76f4d9c555-hgzvs Stopping container payload-pre-processing 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-9dd9f2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-9dd9f2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-2fxrb Readiness probe failed: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 33m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-2fxrb Readiness probe error: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 33m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-m4ph9 Readiness probe error: Get "http://10.133.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 33m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-v7mn2 Readiness probe error: Get "http://10.133.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 33m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-v7mn2 Readiness probe failed: Get "http://10.133.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 33m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-m4ph9 Readiness probe failed: Get "http://10.133.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 33m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-w6fls Readiness probe error: Get "http://10.134.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 33m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-w6fls Readiness probe failed: Get "http://10.134.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-65e151-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-65e151-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-9dd9f2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-9dd9f2-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Normal Created pod/maas-default-gateway-openshift-default-7854949c9-hm546 Container created 33m Normal AddedInterface pod/maas-default-gateway-openshift-default-7854949c9-8n78f Add eth0 [10.132.0.238/23] from ovn-kubernetes 33m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7854949c9 Created pod: maas-default-gateway-openshift-default-7854949c9-vdlrm 33m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7854949c9 Created pod: maas-default-gateway-openshift-default-7854949c9-npq2p 33m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7854949c9 Created pod: maas-default-gateway-openshift-default-7854949c9-8n78f 33m Normal AddedInterface pod/maas-default-gateway-openshift-default-7854949c9-vdlrm Add eth0 [10.132.0.237/23] from ovn-kubernetes 33m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7854949c9 Created pod: maas-default-gateway-openshift-default-7854949c9-hm546 33m Normal Started pod/maas-default-gateway-openshift-default-7854949c9-hm546 Container started 33m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-hm546 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Started pod/maas-default-gateway-openshift-default-7854949c9-8n78f Container started 33m Normal Created pod/maas-default-gateway-openshift-default-7854949c9-8n78f Container created 33m Normal AddedInterface pod/maas-default-gateway-openshift-default-7854949c9-hm546 Add eth0 [10.134.0.111/23] from ovn-kubernetes 33m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-npq2p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/maas-default-gateway-openshift-default-7854949c9-npq2p Add eth0 [10.132.0.239/23] from ovn-kubernetes 33m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7854949c9 from 6 to 10 33m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-8n78f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Started pod/maas-default-gateway-openshift-default-7854949c9-npq2p Container started 33m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-vdlrm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/maas-default-gateway-openshift-default-7854949c9-npq2p Container created 33m Normal Started pod/maas-default-gateway-openshift-default-7854949c9-vdlrm Container started 33m Normal Created pod/maas-default-gateway-openshift-default-7854949c9-vdlrm Container created 33m Normal ScalingReplicaSet deployment/e2e-inf-b-942af8-openshift-default Scaled up replica set e2e-inf-b-942af8-openshift-default-878544c45 from 0 to 1 33m Normal AddedLabel gateway/e2e-inf-b-942af8 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-942af8 33m Normal SuccessfulCreate replicaset/e2e-inf-b-942af8-openshift-default-878544c45 Created pod: e2e-inf-b-942af8-openshift-default-878544c45-bpzgg 33m Normal AddedInterface pod/e2e-inf-b-942af8-openshift-default-878544c45-bpzgg Add eth0 [10.134.0.112/23] from ovn-kubernetes 33m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-hm546 Startup probe failed: Get "http://10.134.0.111:15021/healthz/ready": dial tcp 10.134.0.111:15021: connect: connection refused 33m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-hm546 Startup probe error: Get "http://10.134.0.111:15021/healthz/ready": dial tcp 10.134.0.111:15021: connect: connection refused... 33m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-vdlrm Startup probe error: Get "http://10.132.0.237:15021/healthz/ready": dial tcp 10.132.0.237:15021: connect: connection refused... 33m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-vdlrm Startup probe failed: Get "http://10.132.0.237:15021/healthz/ready": dial tcp 10.132.0.237:15021: connect: connection refused 33m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-8n78f Startup probe failed: Get "http://10.132.0.238:15021/healthz/ready": dial tcp 10.132.0.238:15021: connect: connection refused 33m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-npq2p Startup probe error: Get "http://10.132.0.239:15021/healthz/ready": dial tcp 10.132.0.239:15021: connect: connection refused... 33m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-npq2p Startup probe failed: Get "http://10.132.0.239:15021/healthz/ready": dial tcp 10.132.0.239:15021: connect: connection refused 33m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-8n78f Startup probe error: Get "http://10.132.0.238:15021/healthz/ready": dial tcp 10.132.0.238:15021: connect: connection refused... 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-942af8 Scaled up replica set payload-processing-e2e-inf-b-942af8-78446b5f5f from 0 to 1 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-942af8 Scaled up replica set payload-pre-processing-e2e-inf-b-942af8-6d9ff55467 from 0 to 1 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-942af8-78446b5f5f Created pod: payload-processing-e2e-inf-b-942af8-78446b5f5f-45fb4 32m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-942af8-78446b5f5f Error creating: pods "payload-processing-e2e-inf-b-942af8-78446b5f5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-942af8: serviceaccount "payload-processing-e2e-inf-b-942af8" not found 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-942af8-6d9ff55467 Created pod: payload-pre-processing-e2e-inf-b-942af8-6d9ff55467-fwgpd 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-942af8-6d9ff55467 Error creating: pods "payload-pre-processing-e2e-inf-b-942af8-6d9ff55467-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-942af8: serviceaccount "payload-processing-e2e-inf-b-942af8" not found 32m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-942af8-6d9ff55467-fwgpd Add eth0 [10.132.0.240/23] from ovn-kubernetes 32m Normal Created pod/payload-pre-processing-e2e-inf-b-942af8-6d9ff55467-fwgpd Container created 32m Normal Created pod/payload-processing-e2e-inf-b-942af8-78446b5f5f-45fb4 Container created 32m Normal Pulled pod/payload-pre-processing-e2e-inf-b-942af8-6d9ff55467-fwgpd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Started pod/payload-pre-processing-e2e-inf-b-942af8-6d9ff55467-fwgpd Container started 32m Normal Pulled pod/payload-processing-e2e-inf-b-942af8-78446b5f5f-45fb4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Started pod/payload-processing-e2e-inf-b-942af8-78446b5f5f-45fb4 Container started 32m Normal AddedInterface pod/payload-processing-e2e-inf-b-942af8-78446b5f5f-45fb4 Add eth0 [10.134.0.113/23] from ovn-kubernetes 32m Normal Killing pod/e2e-inf-a-65e151-openshift-default-5478585457-cxqss Stopping container istio-proxy 32m Normal Killing pod/e2e-api-a-e1f8f8-openshift-default-676bdb65df-bsq58 Stopping container istio-proxy 32m Warning ProbeError pod/e2e-api-a-e1f8f8-openshift-default-676bdb65df-bsq58 Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning Unhealthy pod/e2e-api-a-e1f8f8-openshift-default-676bdb65df-bsq58 Readiness probe failed: HTTP probe failed with statuscode: 503 32m Normal Killing pod/payload-processing-e2e-worker-w5-3a7056-7bc4d594f7-wklrp Stopping container payload-processing 32m Normal Killing pod/payload-pre-processing-e2e-worker-w5-3a7056-68d95fb54c-8kq4m Stopping container payload-pre-processing 32m Normal Killing pod/payload-processing-e2e-api-b-69e2b9-558bf74655-wtjfw Stopping container payload-processing 32m Normal Killing pod/payload-pre-processing-e2e-api-b-69e2b9-66b5bf4547-4xrkq Stopping container payload-pre-processing 32m Normal Killing pod/e2e-api-b-efd81c-openshift-default-7956bf8cfc-cfjrn Stopping container istio-proxy 32m Normal Killing pod/payload-pre-processing-e2e-worker-w0-9686c3-6f6bdb5d59-twjx8 Stopping container payload-pre-processing 32m Normal Killing pod/payload-processing-e2e-worker-w0-9686c3-779d4f5849-mnqkq Stopping container payload-processing 32m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w5-3a7056-68d95fb54c-8kq4m Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Warning Unhealthy pod/payload-processing-e2e-worker-w5-3a7056-7bc4d594f7-wklrp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Normal Killing pod/payload-processing-e2e-worker-w3-a4cacd-656dcc8dd5-2wst7 Stopping container payload-processing 32m Normal Killing pod/payload-pre-processing-e2e-worker-w3-a4cacd-57c5b47ff9-kfm57 Stopping container payload-pre-processing 31m Normal Created pod/e2e-mt-3d74f801-openshift-default-78ddf467c-kv428 Container created 31m Normal Started pod/e2e-mt-3d74f801-openshift-default-78ddf467c-kv428 Container started 31m Normal Pulled pod/e2e-mt-3d74f801-openshift-default-78ddf467c-kv428 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-9686c3-6f6bdb5d59-twjx8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Warning Unhealthy pod/payload-processing-e2e-worker-w0-9686c3-779d4f5849-mnqkq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Warning ProbeError pod/e2e-mt-3d74f801-openshift-default-78ddf467c-kv428 Startup probe error: Get "http://10.132.0.234:15021/healthz/ready": dial tcp 10.132.0.234:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-mt-3d74f801-openshift-default-78ddf467c-kv428 Startup probe failed: Get "http://10.132.0.234:15021/healthz/ready": dial tcp 10.132.0.234:15021: connect: connection refused 31m Warning Unhealthy pod/payload-processing-e2e-worker-w3-a4cacd-656dcc8dd5-2wst7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w3-a4cacd-57c5b47ff9-kfm57 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Normal Killing pod/e2e-api-b-69e2b9-openshift-default-545d9ffd69-nhgvx Stopping container istio-proxy 31m Normal Killing pod/payload-pre-processing-e2e-shared-a-w4-9e277a-6f7b5d4955-p29sm Stopping container payload-pre-processing 31m Normal Killing pod/payload-processing-e2e-shared-a-w4-9e277a-5dc7dd98bc-dt7fq Stopping container payload-processing 31m Warning ProbeError pod/e2e-inf-b-942af8-openshift-default-878544c45-bpzgg Readiness probe error: Get "http://10.134.0.112:15021/healthz/ready": read tcp 10.134.0.2:59216->10.134.0.112:15021: read: connection reset by peer... 31m Warning Unhealthy pod/e2e-inf-b-942af8-openshift-default-878544c45-bpzgg Readiness probe failed: Get "http://10.134.0.112:15021/healthz/ready": read tcp 10.134.0.2:59216->10.134.0.112:15021: read: connection reset by peer 31m Normal Created pod/e2e-inf-b-942af8-openshift-default-878544c45-bpzgg Container created 31m Normal Started pod/e2e-inf-b-942af8-openshift-default-878544c45-bpzgg Container started 31m Normal Pulled pod/e2e-inf-b-942af8-openshift-default-878544c45-bpzgg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Killing pod/e2e-worker-w5-3a7056-openshift-default-85ccfddcf-cttst Stopping container istio-proxy 31m Warning ProbeError pod/e2e-inf-b-942af8-openshift-default-878544c45-bpzgg Startup probe error: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-inf-b-942af8-openshift-default-878544c45-bpzgg Startup probe failed: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-3d74f801-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-3d74f801-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-942af8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-942af8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Normal Killing pod/e2e-shared-a-w4-9e277a-openshift-default-59d89f6f57-gz72j Stopping container istio-proxy 31m Warning Unhealthy pod/e2e-shared-a-w4-9e277a-openshift-default-59d89f6f57-gz72j Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning ProbeError pod/e2e-shared-a-w4-9e277a-openshift-default-59d89f6f57-gz72j Readiness probe error: HTTP probe failed with statuscode: 503... 30m Normal Killing pod/e2e-worker-w0-9686c3-openshift-default-85685b9d55-wjpvd Stopping container istio-proxy 30m Normal Killing pod/payload-processing-e2e-shared-b-w4-58ac09-c585d4959-gx5f9 Stopping container payload-processing 30m Normal Killing pod/payload-pre-processing-e2e-shared-b-w4-58ac09-7846d7dccf-bch78 Stopping container payload-pre-processing 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-942af8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-942af8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Normal Killing pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-xmnlp Stopping container istio-proxy 30m Normal Killing pod/e2e-worker-w3-a4cacd-openshift-default-559dcbbf8c-4qtm8 Stopping container istio-proxy 30m Normal Killing pod/e2e-shared-b-w4-58ac09-openshift-default-74d79cb8cc-g995s Stopping container istio-proxy 30m Warning ProbeError pod/e2e-shared-b-w4-58ac09-openshift-default-74d79cb8cc-g995s Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning Unhealthy pod/e2e-shared-b-w4-58ac09-openshift-default-74d79cb8cc-g995s Readiness probe failed: HTTP probe failed with statuscode: 503 29m Normal Killing pod/payload-processing-e2e-worker-w4-2e052a-c6598ff57-9w5t5 Stopping container payload-processing 29m Normal Killing pod/payload-pre-processing-e2e-worker-w4-2e052a-564f4b8d67-mz5p5 Stopping container payload-pre-processing 29m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w4-2e052a-564f4b8d67-mz5p5 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Normal Killing pod/payload-pre-processing-e2e-mt-3d74f801-67d6d795dc-65gv9 Stopping container payload-pre-processing 29m Normal Killing pod/payload-processing-e2e-mt-3d74f801-d4cf58969-wlgw2 Stopping container payload-processing 28m Warning Unhealthy pod/payload-processing-e2e-mt-3d74f801-d4cf58969-wlgw2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Warning Unhealthy pod/payload-pre-processing-e2e-mt-3d74f801-67d6d795dc-65gv9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Normal SuccessfulDelete replicaset/e2e-worker-w4-2e052a-openshift-default-6886cf66dc Deleted pod: e2e-worker-w4-2e052a-openshift-default-6886cf66dc-8m7jw 28m Normal ScalingReplicaSet deployment/e2e-worker-w4-2e052a-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w4-2e052a-openshift-default-6886cf66dc from 2 to 1 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-2e052a-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 28m Normal Killing pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-8m7jw Stopping container istio-proxy 28m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 28m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 28m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-qjtdk 28m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-qjtdk Readiness probe failed: Get "http://10.132.0.51:8080/ready": dial tcp 10.132.0.51:8080: connect: connection refused 28m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-qjtdk Readiness probe error: Get "http://10.132.0.51:8080/ready": dial tcp 10.132.0.51:8080: connect: connection refused... 28m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-qjtdk Stopping container discovery 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-355e1a-openshift-default New size: 1; reason: All metrics below target 28m Normal ScalingReplicaSet deployment/e2e-worker-w1-355e1a-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w1-355e1a-openshift-default-d7d9459f9 from 2 to 1 28m Normal Killing pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-7fnvf Stopping container istio-proxy 28m Normal SuccessfulDelete replicaset/e2e-worker-w1-355e1a-openshift-default-d7d9459f9 Deleted pod: e2e-worker-w1-355e1a-openshift-default-d7d9459f9-7fnvf 28m Normal Killing pod/e2e-worker-w4-2e052a-openshift-default-6886cf66dc-p87bt Stopping container istio-proxy 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-3d74f801 Scaled up replica set payload-processing-e2e-mt-3d74f801-d4cf58969 from 0 to 1 27m Warning FailedCreate replicaset/payload-processing-e2e-mt-3d74f801-d4cf58969 Error creating: pods "payload-processing-e2e-mt-3d74f801-d4cf58969-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-3d74f801: serviceaccount "payload-processing-e2e-mt-3d74f801" not found 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-3d74f801 Scaled up replica set payload-pre-processing-e2e-mt-3d74f801-67d6d795dc from 0 to 1 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-3d74f801-67d6d795dc Created pod: payload-pre-processing-e2e-mt-3d74f801-67d6d795dc-lfpcx 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-3d74f801-67d6d795dc Error creating: pods "payload-pre-processing-e2e-mt-3d74f801-67d6d795dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-3d74f801: serviceaccount "payload-processing-e2e-mt-3d74f801" not found 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-3d74f801-d4cf58969 Created pod: payload-processing-e2e-mt-3d74f801-d4cf58969-zjtsc 27m Normal Started pod/payload-processing-e2e-mt-3d74f801-d4cf58969-zjtsc Container started 27m Normal AddedInterface pod/payload-processing-e2e-mt-3d74f801-d4cf58969-zjtsc Add eth0 [10.132.0.248/23] from ovn-kubernetes 27m Normal Started pod/payload-pre-processing-e2e-mt-3d74f801-67d6d795dc-lfpcx Container started 27m Normal Created pod/payload-pre-processing-e2e-mt-3d74f801-67d6d795dc-lfpcx Container created 27m Normal Pulled pod/payload-pre-processing-e2e-mt-3d74f801-67d6d795dc-lfpcx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/payload-pre-processing-e2e-mt-3d74f801-67d6d795dc-lfpcx Add eth0 [10.132.0.249/23] from ovn-kubernetes 27m Normal Pulled pod/payload-processing-e2e-mt-3d74f801-d4cf58969-zjtsc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal Created pod/payload-processing-e2e-mt-3d74f801-d4cf58969-zjtsc Container created 26m Normal Killing pod/maas-default-gateway-openshift-default-7854949c9-w6fls Stopping container istio-proxy 26m Normal Killing pod/maas-default-gateway-openshift-default-7854949c9-2fxrb Stopping container istio-proxy 26m Normal Killing pod/maas-default-gateway-openshift-default-7854949c9-x4nc2 Stopping container istio-proxy 26m Normal Killing pod/maas-default-gateway-openshift-default-7854949c9-hm546 Stopping container istio-proxy 26m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7854949c9 Deleted pod: maas-default-gateway-openshift-default-7854949c9-2fxrb 26m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7854949c9 Deleted pod: maas-default-gateway-openshift-default-7854949c9-hm546 26m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7854949c9 Deleted pod: maas-default-gateway-openshift-default-7854949c9-w6fls 26m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7854949c9 from 10 to 6 26m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: All metrics below target 26m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7854949c9 Deleted pod: maas-default-gateway-openshift-default-7854949c9-x4nc2 26m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-2fxrb Readiness probe error: HTTP probe failed with statuscode: 503... 26m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-2fxrb Readiness probe failed: HTTP probe failed with statuscode: 503 26m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-w6fls Readiness probe error: HTTP probe failed with statuscode: 503... 26m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-w6fls Readiness probe failed: HTTP probe failed with statuscode: 503 26m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-x4nc2 Readiness probe error: HTTP probe failed with statuscode: 503... 26m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-x4nc2 Readiness probe failed: HTTP probe failed with statuscode: 503 26m Normal Killing pod/payload-processing-e2e-inf-a-9dd9f2-674c89d7cc-s977s Stopping container payload-processing 26m Normal Killing pod/payload-pre-processing-e2e-inf-a-9dd9f2-58b64d8db7-7x68t Stopping container payload-pre-processing 26m Normal Killing pod/payload-processing-e2e-mt-3d74f801-d4cf58969-zjtsc Stopping container payload-processing 26m Normal Killing pod/payload-pre-processing-e2e-mt-3d74f801-67d6d795dc-lfpcx Stopping container payload-pre-processing 26m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-9dd9f2-58b64d8db7-7x68t Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Normal Killing pod/e2e-inf-a-9dd9f2-openshift-default-db6f4568c-k2wkl Stopping container istio-proxy 25m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7854949c9 from 6 to 4 25m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7854949c9 Deleted pod: maas-default-gateway-openshift-default-7854949c9-8n78f 25m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: All metrics below target 25m Normal Killing pod/maas-default-gateway-openshift-default-7854949c9-8n78f Stopping container istio-proxy 25m Normal Killing pod/maas-default-gateway-openshift-default-7854949c9-npq2p Stopping container istio-proxy 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-32a59b-openshift-default New size: 1; reason: All metrics below target 25m Normal SuccessfulDelete replicaset/e2e-worker-w2-32a59b-openshift-default-86974868b5 Deleted pod: e2e-worker-w2-32a59b-openshift-default-86974868b5-hsnb4 25m Normal Killing pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-hsnb4 Stopping container istio-proxy 25m Normal ScalingReplicaSet deployment/e2e-worker-w2-32a59b-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w2-32a59b-openshift-default-86974868b5 from 2 to 1 25m Normal Killing pod/payload-processing-e2e-inf-b-942af8-78446b5f5f-45fb4 Stopping container payload-processing 25m Normal Killing pod/payload-pre-processing-e2e-inf-b-942af8-6d9ff55467-fwgpd Stopping container payload-pre-processing 25m Normal Killing pod/maas-default-gateway-openshift-default-7854949c9-m4ph9 Stopping container istio-proxy 25m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7854949c9 from 4 to 3 25m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7854949c9 (combined from similar events): Deleted pod: maas-default-gateway-openshift-default-7854949c9-m4ph9 25m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: All metrics below target 25m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-m4ph9 Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-m4ph9 Readiness probe failed: HTTP probe failed with statuscode: 503 25m Normal Killing pod/e2e-mt-3d74f801-openshift-default-78ddf467c-kv428 Stopping container istio-proxy 25m Normal Killing pod/e2e-inf-b-942af8-openshift-default-878544c45-bpzgg Stopping container istio-proxy 25m Warning ProbeError pod/e2e-inf-b-942af8-openshift-default-878544c45-bpzgg Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-942af8-6d9ff55467-fwgpd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Warning Unhealthy pod/e2e-inf-b-942af8-openshift-default-878544c45-bpzgg Readiness probe failed: HTTP probe failed with statuscode: 503 25m Normal Killing pod/payload-pre-processing-e2e-worker-w1-355e1a-5c86d6b775-2kbjt Stopping container payload-pre-processing 25m Normal Killing pod/payload-processing-e2e-worker-w1-355e1a-57bbcfbbb9-ggfc4 Stopping container payload-processing 25m Normal Killing pod/payload-processing-e2e-shared-a-w2-770e67-dbf74bfbf-vhp4v Stopping container payload-processing 25m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-770e67-85547c9dc-9mtrq Stopping container payload-pre-processing 24m Normal Killing pod/e2e-shared-a-w2-770e67-openshift-default-974c475f5-dlf59 Stopping container istio-proxy 24m Normal Killing pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-j6l6d Stopping container istio-proxy 24m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-b485d8-d5fd6c795-n8wxl Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-shared-b-w2-b485d8-657bb4cc85-6p2ht Stopping container payload-processing 24m Warning Unhealthy pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-j6l6d Readiness probe failed: HTTP probe failed with statuscode: 503 24m Warning ProbeError pod/e2e-worker-w1-355e1a-openshift-default-d7d9459f9-j6l6d Readiness probe error: HTTP probe failed with statuscode: 503... 24m Normal Killing pod/e2e-shared-b-w2-b485d8-openshift-default-788458b655-bff6t Stopping container istio-proxy 24m Normal Killing pod/payload-pre-processing-e2e-worker-w2-32a59b-5b54fcd66c-2nl8z Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-worker-w2-32a59b-6bd5df9875-tlgmv Stopping container payload-processing 23m Normal Killing pod/e2e-worker-w2-32a59b-openshift-default-86974868b5-n6kr9 Stopping container istio-proxy 23m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7854949c9 from 3 to 6 23m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 23m Normal Started pod/maas-default-gateway-openshift-default-7854949c9-hvlvx Container started 23m Normal Created pod/maas-default-gateway-openshift-default-7854949c9-8pc6b Container created 23m Normal AddedInterface pod/maas-default-gateway-openshift-default-7854949c9-q7v6d Add eth0 [10.134.0.135/23] from ovn-kubernetes 23m Normal Created pod/maas-default-gateway-openshift-default-7854949c9-hvlvx Container created 23m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-hvlvx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-q7v6d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Created pod/maas-default-gateway-openshift-default-7854949c9-q7v6d Container created 23m Normal Started pod/maas-default-gateway-openshift-default-7854949c9-q7v6d Container started 23m Normal Started pod/maas-default-gateway-openshift-default-7854949c9-8pc6b Container started 23m Normal AddedInterface pod/maas-default-gateway-openshift-default-7854949c9-8pc6b Add eth0 [10.132.1.3/23] from ovn-kubernetes 23m Normal AddedInterface pod/maas-default-gateway-openshift-default-7854949c9-hvlvx Add eth0 [10.134.0.134/23] from ovn-kubernetes 23m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-8pc6b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-q7v6d Startup probe failed: Get "http://10.134.0.135:15021/healthz/ready": dial tcp 10.134.0.135:15021: connect: connection refused 23m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-hvlvx Startup probe failed: Get "http://10.134.0.134:15021/healthz/ready": dial tcp 10.134.0.134:15021: connect: connection refused 23m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-hvlvx Startup probe error: Get "http://10.134.0.134:15021/healthz/ready": dial tcp 10.134.0.134:15021: connect: connection refused... 23m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-q7v6d Startup probe error: Get "http://10.134.0.135:15021/healthz/ready": dial tcp 10.134.0.135:15021: connect: connection refused... 23m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-8pc6b Startup probe error: Get "http://10.132.1.3:15021/healthz/ready": dial tcp 10.132.1.3:15021: connect: connection refused... 23m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-8pc6b Startup probe failed: Get "http://10.132.1.3:15021/healthz/ready": dial tcp 10.132.1.3:15021: connect: connection refused 22m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7854949c9 from 6 to 7 22m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: All metrics below target 22m Normal AddedInterface pod/maas-default-gateway-openshift-default-7854949c9-tg88d Add eth0 [10.132.1.5/23] from ovn-kubernetes 22m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-tg88d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Created pod/maas-default-gateway-openshift-default-7854949c9-tg88d Container created 22m Normal Started pod/maas-default-gateway-openshift-default-7854949c9-tg88d Container started 22m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-tg88d Startup probe failed: Get "http://10.132.1.5:15021/healthz/ready": dial tcp 10.132.1.5:15021: connect: connection refused 22m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-tg88d Startup probe error: Get "http://10.132.1.5:15021/healthz/ready": dial tcp 10.132.1.5:15021: connect: connection refused... 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-7854949c9-jr2ss Add eth0 [10.134.0.136/23] from ovn-kubernetes 21m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 21m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7854949c9 from 7 to 9 21m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-8vt58 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-7854949c9-8vt58 Add eth0 [10.133.0.106/23] from ovn-kubernetes 21m Normal Created pod/maas-default-gateway-openshift-default-7854949c9-8vt58 Container created 21m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-jr2ss Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Created pod/maas-default-gateway-openshift-default-7854949c9-jr2ss Container created 21m Normal Started pod/maas-default-gateway-openshift-default-7854949c9-8vt58 Container started 21m Normal Started pod/maas-default-gateway-openshift-default-7854949c9-jr2ss Container started 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-8vt58 Startup probe failed: Get "http://10.133.0.106:15021/healthz/ready": dial tcp 10.133.0.106:15021: connect: connection refused 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-jr2ss Startup probe failed: Get "http://10.134.0.136:15021/healthz/ready": dial tcp 10.134.0.136:15021: connect: connection refused 21m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-8vt58 Startup probe error: Get "http://10.133.0.106:15021/healthz/ready": dial tcp 10.133.0.106:15021: connect: connection refused... 21m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-jr2ss Startup probe error: Get "http://10.134.0.136:15021/healthz/ready": dial tcp 10.134.0.136:15021: connect: connection refused... 21m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 21m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7854949c9 from 9 to 10 21m Normal Started pod/maas-default-gateway-openshift-default-7854949c9-fmq4c Container started 21m Normal Created pod/maas-default-gateway-openshift-default-7854949c9-fmq4c Container created 21m Normal Pulled pod/maas-default-gateway-openshift-default-7854949c9-fmq4c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-7854949c9-fmq4c Add eth0 [10.132.1.8/23] from ovn-kubernetes 21m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-zlt64 Readiness probe error: Get "http://10.133.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-zlt64 Readiness probe failed: Get "http://10.133.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 21m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-fmq4c Startup probe error: Get "http://10.132.1.8:15021/healthz/ready": dial tcp 10.132.1.8:15021: connect: connection refused... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-fmq4c Startup probe failed: Get "http://10.132.1.8:15021/healthz/ready": dial tcp 10.132.1.8:15021: connect: connection refused 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-fmq4c Startup probe failed: Get "http://10.132.1.8:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 21m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-fmq4c Startup probe error: Get "http://10.132.1.8:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 19m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-fmq4c Readiness probe error: Get "http://10.132.1.8:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 19m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-fmq4c Readiness probe failed: Get "http://10.132.1.8:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-8pc6b Readiness probe failed: Get "http://10.132.1.3:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 18m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-8pc6b Readiness probe error: Get "http://10.132.1.3:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 18m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-q7v6d Readiness probe error: Get "http://10.134.0.135:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 18m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-hvlvx Readiness probe error: Get "http://10.134.0.134:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-hvlvx Readiness probe failed: Get "http://10.134.0.134:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-q7v6d Readiness probe failed: Get "http://10.134.0.135:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 18m Warning Unhealthy pod/maas-default-gateway-openshift-default-7854949c9-vdlrm Readiness probe failed: Get "http://10.132.0.237:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 18m Warning ProbeError pod/maas-default-gateway-openshift-default-7854949c9-vdlrm Readiness probe error: Get "http://10.132.0.237:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 15m Normal SuccessfulCreate replicaset/e2e-worker-single-main-020814-openshift-default-75d7fd8777 Created pod: e2e-worker-single-main-020814-openshift-default-75d7fd8777hwxkd 15m Normal AddedLabel gateway/e2e-worker-single-main-020814 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-020814 15m Normal ScalingReplicaSet deployment/e2e-worker-single-main-020814-openshift-default Scaled up replica set e2e-worker-single-main-020814-openshift-default-75d7fd8777 from 0 to 1 15m Normal AddedInterface pod/e2e-worker-single-main-020814-openshift-default-75d7fd8777hwxkd Add eth0 [10.132.1.22/23] from ovn-kubernetes 15m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-020814-67dfbfc555 Error creating: pods "payload-pre-processing-e2e-worker-single-main-020814-67dfbfc555-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-020814: serviceaccount "payload-processing-e2e-worker-single-main-020814" not found 15m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-020814-6f56b5c9fc Error creating: pods "payload-processing-e2e-worker-single-main-020814-6f56b5c9fc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-020814: serviceaccount "payload-processing-e2e-worker-single-main-020814" not found 15m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-020814-67dfbfc555 Created pod: payload-pre-processing-e2e-worker-single-main-020814-67dfbqkbgf 15m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-020814 Scaled up replica set payload-pre-processing-e2e-worker-single-main-020814-67dfbfc555 from 0 to 1 15m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-020814 Scaled up replica set payload-processing-e2e-worker-single-main-020814-6f56b5c9fc from 0 to 1 15m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-020814-67dfbqkbgf Add eth0 [10.132.1.23/23] from ovn-kubernetes 15m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-020814-6f56b5c9fdkhk4 Add eth0 [10.132.1.24/23] from ovn-kubernetes 15m Normal Created pod/payload-pre-processing-e2e-worker-single-main-020814-67dfbqkbgf Container created 15m Normal Started pod/payload-pre-processing-e2e-worker-single-main-020814-67dfbqkbgf Container started 15m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-020814-67dfbqkbgf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 15m Normal Started pod/payload-processing-e2e-worker-single-main-020814-6f56b5c9fdkhk4 Container started 15m Normal Created pod/payload-processing-e2e-worker-single-main-020814-6f56b5c9fdkhk4 Container created 15m Normal Pulled pod/payload-processing-e2e-worker-single-main-020814-6f56b5c9fdkhk4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 15m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-020814-6f56b5c9fc Created pod: payload-processing-e2e-worker-single-main-020814-6f56b5c9fdkhk4 15m Normal Created pod/e2e-worker-single-main-020814-openshift-default-75d7fd8777hwxkd Container created 15m Normal Pulled pod/e2e-worker-single-main-020814-openshift-default-75d7fd8777hwxkd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 15m Normal Started pod/e2e-worker-single-main-020814-openshift-default-75d7fd8777hwxkd Container started 15m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-020814-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 15m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-020814-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 15m Warning Unhealthy pod/e2e-worker-single-main-020814-openshift-default-75d7fd8777hwxkd Startup probe failed: Get "http://10.132.1.22:15021/healthz/ready": dial tcp 10.132.1.22:15021: connect: connection refused 15m Warning ProbeError pod/e2e-worker-single-main-020814-openshift-default-75d7fd8777hwxkd Startup probe error: Get "http://10.132.1.22:15021/healthz/ready": dial tcp 10.132.1.22:15021: connect: connection refused... 14m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-020814-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 14m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-020814-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 14m Normal Killing pod/payload-pre-processing-e2e-worker-single-main-020814-67dfbqkbgf Stopping container payload-pre-processing 14m Normal Killing pod/payload-processing-e2e-worker-single-main-020814-6f56b5c9fdkhk4 Stopping container payload-processing 13m Normal Killing pod/e2e-worker-single-main-020814-openshift-default-75d7fd8777hwxkd Stopping container istio-proxy 13m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-020814-67dfbqkbgf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 13m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-020814-6f56b5c9fdkhk4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 11m Normal AddedLabel gateway/e2e-worker-single-main-439a13 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-439a13 11m Normal ScalingReplicaSet deployment/e2e-worker-single-main-439a13-openshift-default Scaled up replica set e2e-worker-single-main-439a13-openshift-default-75d865948f from 0 to 1 11m Normal SuccessfulCreate replicaset/e2e-worker-single-main-439a13-openshift-default-75d865948f Created pod: e2e-worker-single-main-439a13-openshift-default-75d865948fmgxp8 11m Normal AddedInterface pod/e2e-worker-single-main-439a13-openshift-default-75d865948fmgxp8 Add eth0 [10.132.1.31/23] from ovn-kubernetes 10m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-439a13 Scaled up replica set payload-processing-e2e-worker-single-main-439a13-7d55dcff49 from 0 to 1 10m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-439a13-7d55dcff49 Created pod: payload-processing-e2e-worker-single-main-439a13-7d55dcff4925rx 10m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-439a13-95c6d87d7 Created pod: payload-pre-processing-e2e-worker-single-main-439a13-95c6dnrtkf 10m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-439a13-7d55dcff49 Error creating: pods "payload-processing-e2e-worker-single-main-439a13-7d55dcff49-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-439a13: serviceaccount "payload-processing-e2e-worker-single-main-439a13" not found 10m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-439a13-95c6d87d7 Error creating: pods "payload-pre-processing-e2e-worker-single-main-439a13-95c6d87d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-439a13: serviceaccount "payload-processing-e2e-worker-single-main-439a13" not found 10m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-439a13 Scaled up replica set payload-pre-processing-e2e-worker-single-main-439a13-95c6d87d7 from 0 to 1 10m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-439a13-7d55dcff4925rx Add eth0 [10.132.1.33/23] from ovn-kubernetes 10m Normal Started pod/payload-pre-processing-e2e-worker-single-main-439a13-95c6dnrtkf Container started 10m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-439a13-95c6dnrtkf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 10m Normal Created pod/payload-processing-e2e-worker-single-main-439a13-7d55dcff4925rx Container created 10m Normal Started pod/payload-processing-e2e-worker-single-main-439a13-7d55dcff4925rx Container started 10m Normal Pulled pod/payload-processing-e2e-worker-single-main-439a13-7d55dcff4925rx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 10m Normal Created pod/payload-pre-processing-e2e-worker-single-main-439a13-95c6dnrtkf Container created 10m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-439a13-95c6dnrtkf Add eth0 [10.132.1.32/23] from ovn-kubernetes 10m Normal Pulled pod/e2e-worker-single-main-439a13-openshift-default-75d865948fmgxp8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 10m Normal Created pod/e2e-worker-single-main-439a13-openshift-default-75d865948fmgxp8 Container created 10m Normal Started pod/e2e-worker-single-main-439a13-openshift-default-75d865948fmgxp8 Container started 10m Warning Unhealthy pod/e2e-worker-single-main-439a13-openshift-default-75d865948fmgxp8 Startup probe failed: Get "http://10.132.1.31:15021/healthz/ready": dial tcp 10.132.1.31:15021: connect: connection refused 10m Warning ProbeError pod/e2e-worker-single-main-439a13-openshift-default-75d865948fmgxp8 Startup probe error: Get "http://10.132.1.31:15021/healthz/ready": dial tcp 10.132.1.31:15021: connect: connection refused... 10m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-439a13-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 10m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-439a13-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 9m21s Normal Killing pod/payload-processing-e2e-worker-single-main-439a13-7d55dcff4925rx Stopping container payload-processing 9m21s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-439a13-95c6dnrtkf Stopping container payload-pre-processing 9m9s Normal Killing pod/e2e-worker-single-main-439a13-openshift-default-75d865948fmgxp8 Stopping container istio-proxy 9m3s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-439a13-95c6dnrtkf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 9m3s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-439a13-7d55dcff4925rx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 6m35s Normal ScalingReplicaSet deployment/e2e-ait-50babf0d-openshift-default Scaled up replica set e2e-ait-50babf0d-openshift-default-6bc768cbf7 from 0 to 1 6m35s Normal SuccessfulCreate replicaset/e2e-ait-50babf0d-openshift-default-6bc768cbf7 Created pod: e2e-ait-50babf0d-openshift-default-6bc768cbf7-7j7jf 6m35s Normal AddedLabel gateway/e2e-ait-50babf0d Added label istio.io/rev=openshift-gateway to gateway e2e-ait-50babf0d 6m34s Normal AddedInterface pod/e2e-ait-50babf0d-openshift-default-6bc768cbf7-7j7jf Add eth0 [10.132.1.43/23] from ovn-kubernetes 6m28s Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-50babf0d-7c8c48b5f5 Error creating: pods "payload-pre-processing-e2e-ait-50babf0d-7c8c48b5f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-50babf0d: serviceaccount "payload-processing-e2e-ait-50babf0d" not found 6m28s Warning FailedCreate replicaset/payload-processing-e2e-ait-50babf0d-6db96d59f5 Error creating: pods "payload-processing-e2e-ait-50babf0d-6db96d59f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-50babf0d: serviceaccount "payload-processing-e2e-ait-50babf0d" not found 6m28s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-50babf0d Scaled up replica set payload-pre-processing-e2e-ait-50babf0d-7c8c48b5f5 from 0 to 1 6m28s Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-50babf0d Scaled up replica set payload-processing-e2e-ait-50babf0d-6db96d59f5 from 0 to 1 6m27s Normal Started pod/payload-processing-e2e-ait-50babf0d-6db96d59f5-8826z Container started 6m27s Normal Created pod/payload-processing-e2e-ait-50babf0d-6db96d59f5-8826z Container created 6m27s Normal AddedInterface pod/payload-processing-e2e-ait-50babf0d-6db96d59f5-8826z Add eth0 [10.132.1.44/23] from ovn-kubernetes 6m27s Normal AddedInterface pod/payload-pre-processing-e2e-ait-50babf0d-7c8c48b5f5-hg85d Add eth0 [10.132.1.45/23] from ovn-kubernetes 6m27s Normal Pulled pod/payload-processing-e2e-ait-50babf0d-6db96d59f5-8826z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 6m27s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-50babf0d-7c8c48b5f5 Created pod: payload-pre-processing-e2e-ait-50babf0d-7c8c48b5f5-hg85d 6m27s Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-50babf0d-6db96d59f5 Created pod: payload-processing-e2e-ait-50babf0d-6db96d59f5-8826z 6m26s Normal Started pod/payload-pre-processing-e2e-ait-50babf0d-7c8c48b5f5-hg85d Container started 6m26s Normal Pulled pod/payload-pre-processing-e2e-ait-50babf0d-7c8c48b5f5-hg85d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 6m26s Normal Created pod/payload-pre-processing-e2e-ait-50babf0d-7c8c48b5f5-hg85d Container created 6m7s Normal Started pod/e2e-ait-50babf0d-openshift-default-6bc768cbf7-7j7jf Container started 6m7s Normal Created pod/e2e-ait-50babf0d-openshift-default-6bc768cbf7-7j7jf Container created 6m7s Normal Pulled pod/e2e-ait-50babf0d-openshift-default-6bc768cbf7-7j7jf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 6m1s Warning ProbeError pod/e2e-ait-50babf0d-openshift-default-6bc768cbf7-7j7jf Startup probe error: Get "http://10.132.1.43:15021/healthz/ready": dial tcp 10.132.1.43:15021: connect: connection refused... 6m1s Warning Unhealthy pod/e2e-ait-50babf0d-openshift-default-6bc768cbf7-7j7jf Startup probe failed: Get "http://10.132.1.43:15021/healthz/ready": dial tcp 10.132.1.43:15021: connect: connection refused 5m49s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-50babf0d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 5m49s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-50babf0d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 5m43s Normal Killing pod/payload-processing-e2e-ait-50babf0d-6db96d59f5-8826z Stopping container payload-processing 5m43s Normal Killing pod/payload-pre-processing-e2e-ait-50babf0d-7c8c48b5f5-hg85d Stopping container payload-pre-processing 5m34s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-50babf0d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 5m34s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-50babf0d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 5m28s Normal Killing pod/e2e-ait-50babf0d-openshift-default-6bc768cbf7-7j7jf Stopping container istio-proxy 5m27s Normal AddedLabel gateway/e2e-authcfg-http-ad43e7 Added label istio.io/rev=openshift-gateway to gateway e2e-authcfg-http-ad43e7 5m27s Normal ScalingReplicaSet deployment/e2e-authcfg-http-ad43e7-openshift-default Scaled up replica set e2e-authcfg-http-ad43e7-openshift-default-5d97569dcc from 0 to 1 5m27s Normal SuccessfulCreate replicaset/e2e-authcfg-http-ad43e7-openshift-default-5d97569dcc Created pod: e2e-authcfg-http-ad43e7-openshift-default-5d97569dcc-p2ckt 5m26s Normal AddedInterface pod/e2e-authcfg-http-ad43e7-openshift-default-5d97569dcc-p2ckt Add eth0 [10.132.1.47/23] from ovn-kubernetes 5m19s Warning FailedCreate replicaset/payload-processing-e2e-authcfg-http-ad43e7-66b7bc6d87 Error creating: pods "payload-processing-e2e-authcfg-http-ad43e7-66b7bc6d87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-ad43e7: serviceaccount "payload-processing-e2e-authcfg-http-ad43e7" not found 5m19s Warning FailedCreate replicaset/payload-pre-processing-e2e-authcfg-http-ad43e7-7d7956b4ff Error creating: pods "payload-pre-processing-e2e-authcfg-http-ad43e7-7d7956b4ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-ad43e7: serviceaccount "payload-processing-e2e-authcfg-http-ad43e7" not found 5m19s Normal ScalingReplicaSet deployment/payload-processing-e2e-authcfg-http-ad43e7 Scaled up replica set payload-processing-e2e-authcfg-http-ad43e7-66b7bc6d87 from 0 to 1 5m19s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-authcfg-http-ad43e7 Scaled up replica set payload-pre-processing-e2e-authcfg-http-ad43e7-7d7956b4ff from 0 to 1 5m19s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-authcfg-http-ad43e7-7d7956b4ff Created pod: payload-pre-processing-e2e-authcfg-http-ad43e7-7d7956b4ff-xwtpt 5m18s Normal Created pod/payload-processing-e2e-authcfg-http-ad43e7-66b7bc6d87-gn4qd Container created 5m18s Normal AddedInterface pod/payload-pre-processing-e2e-authcfg-http-ad43e7-7d7956b4ff-xwtpt Add eth0 [10.132.1.48/23] from ovn-kubernetes 5m18s Normal Started pod/payload-pre-processing-e2e-authcfg-http-ad43e7-7d7956b4ff-xwtpt Container started 5m18s Normal AddedInterface pod/payload-processing-e2e-authcfg-http-ad43e7-66b7bc6d87-gn4qd Add eth0 [10.132.1.49/23] from ovn-kubernetes 5m18s Normal Pulled pod/payload-processing-e2e-authcfg-http-ad43e7-66b7bc6d87-gn4qd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m18s Normal Pulled pod/payload-pre-processing-e2e-authcfg-http-ad43e7-7d7956b4ff-xwtpt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m18s Normal Started pod/payload-processing-e2e-authcfg-http-ad43e7-66b7bc6d87-gn4qd Container started 5m18s Normal Created pod/payload-pre-processing-e2e-authcfg-http-ad43e7-7d7956b4ff-xwtpt Container created 5m18s Normal SuccessfulCreate replicaset/payload-processing-e2e-authcfg-http-ad43e7-66b7bc6d87 Created pod: payload-processing-e2e-authcfg-http-ad43e7-66b7bc6d87-gn4qd 4m58s Normal Pulled pod/e2e-authcfg-http-ad43e7-openshift-default-5d97569dcc-p2ckt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 4m58s Normal Created pod/e2e-authcfg-http-ad43e7-openshift-default-5d97569dcc-p2ckt Container created 4m58s Normal Started pod/e2e-authcfg-http-ad43e7-openshift-default-5d97569dcc-p2ckt Container started 4m56s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-ad43e7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m56s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-ad43e7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m52s Warning ProbeError pod/e2e-authcfg-http-ad43e7-openshift-default-5d97569dcc-p2ckt Startup probe error: Get "http://10.132.1.47:15021/healthz/ready": dial tcp 10.132.1.47:15021: connect: connection refused... 4m52s Warning Unhealthy pod/e2e-authcfg-http-ad43e7-openshift-default-5d97569dcc-p2ckt Startup probe failed: Get "http://10.132.1.47:15021/healthz/ready": dial tcp 10.132.1.47:15021: connect: connection refused 4m26s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-ad43e7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 4m26s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-ad43e7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 3m25s Normal Killing pod/payload-processing-e2e-authcfg-http-ad43e7-66b7bc6d87-gn4qd Stopping container payload-processing 3m25s Normal Killing pod/payload-pre-processing-e2e-authcfg-http-ad43e7-7d7956b4ff-xwtpt Stopping container payload-pre-processing 3m6s Normal Killing pod/e2e-authcfg-http-ad43e7-openshift-default-5d97569dcc-p2ckt Stopping container istio-proxy