=== events in openshift-ingress (2026-10-31T00:56:15Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 11m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-e98014-6989c9wz5h Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-e98014-6989c9wz5h to ip-10-0-131-123.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-6gwbm Successfully assigned openshift-ingress/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-6gwbm to ip-10-0-141-180.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-4strh Successfully assigned openshift-ingress/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-4strh to ip-10-0-139-66.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-hrh5g Successfully assigned openshift-ingress/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-hrh5g to ip-10-0-131-123.us-west-2.compute.internal 65m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-mkvqm Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-mkvqm to ip-10-0-139-66.us-west-2.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-api-b-1e5c33-78944dcc56-xzctn Successfully assigned openshift-ingress/payload-processing-e2e-api-b-1e5c33-78944dcc56-xzctn to ip-10-0-139-66.us-west-2.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-api-b-39e3a8-77d67d47f8-msm6p Successfully assigned openshift-ingress/payload-processing-e2e-api-b-39e3a8-77d67d47f8-msm6p to ip-10-0-141-180.us-west-2.compute.internal 64m Normal Scheduled pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-jqc5p Successfully assigned openshift-ingress/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-jqc5p to ip-10-0-141-180.us-west-2.compute.internal 39m Normal Scheduled pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-q6d2w Successfully assigned openshift-ingress/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-q6d2w to ip-10-0-139-66.us-west-2.compute.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-ait-7c8802f8-84c649965f-8cltq Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-7c8802f8-84c649965f-8cltq to ip-10-0-141-180.us-west-2.compute.internal 51m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-8fws9 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-8fws9 to ip-10-0-141-180.us-west-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-03c703-68ccf588f9-957fw Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-03c703-68ccf588f9-957fw to ip-10-0-141-180.us-west-2.compute.internal 58m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-944gh Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-944gh to ip-10-0-141-180.us-west-2.compute.internal 23m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-875hj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-875hj to ip-10-0-131-123.us-west-2.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-348a33-6987f947f-cqp2m Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-348a33-6987f947f-cqp2m to ip-10-0-141-180.us-west-2.compute.internal 65m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-bd4hv Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-bd4hv to ip-10-0-131-123.us-west-2.compute.internal 23m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-x28pp Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-x28pp to ip-10-0-131-123.us-west-2.compute.internal 39m Normal Scheduled pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-q6whh Successfully assigned openshift-ingress/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-q6whh to ip-10-0-141-180.us-west-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-mgtkw Successfully assigned openshift-ingress/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-mgtkw to ip-10-0-139-66.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-mlkdz Successfully assigned openshift-ingress/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-mlkdz to ip-10-0-141-180.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-pg48c Successfully assigned openshift-ingress/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-pg48c to ip-10-0-131-123.us-west-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-56c5v Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-56c5v to ip-10-0-141-180.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-ait-7c8802f8-openshift-default-74499c95cc-v5qrl Successfully assigned openshift-ingress/e2e-ait-7c8802f8-openshift-default-74499c95cc-v5qrl to ip-10-0-141-180.us-west-2.compute.internal 51m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-4trk8 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-4trk8 to ip-10-0-139-66.us-west-2.compute.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-76b073-68467cb4b7-pf7qq Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-76b073-68467cb4b7-pf7qq to ip-10-0-141-180.us-west-2.compute.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-ddf61a-67ff4567bc-mrc2x Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-ddf61a-67ff4567bc-mrc2x to ip-10-0-139-66.us-west-2.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-37abc0-69469c56c-6bzrh Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-37abc0-69469c56c-6bzrh to ip-10-0-139-66.us-west-2.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-460d4e-c97d659-rcrqj Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-460d4e-c97d659-rcrqj to ip-10-0-131-123.us-west-2.compute.internal 35m Normal Scheduled pod/payload-processing-e2e-api-b-d687ff-7666858cfd-wh6tj Successfully assigned openshift-ingress/payload-processing-e2e-api-b-d687ff-7666858cfd-wh6tj to ip-10-0-139-66.us-west-2.compute.internal 58m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-cxzjj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-cxzjj to ip-10-0-139-66.us-west-2.compute.internal 23m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-4sgzb Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-4sgzb to ip-10-0-141-180.us-west-2.compute.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-49837c-5c557bd847-pn6w5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-49837c-5c557bd847-pn6w5 to ip-10-0-139-66.us-west-2.compute.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-migrate-d32a7cc0-6b7768cb67-ntrqr Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-d32a7cc0-6b7768cb67-ntrqr to ip-10-0-141-180.us-west-2.compute.internal 35m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-49lgw Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-49lgw to ip-10-0-131-123.us-west-2.compute.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-mt-a251fdfb-6f8487684c-6qzl9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-a251fdfb-6f8487684c-6qzl9 to ip-10-0-141-180.us-west-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-2xfw2 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-2xfw2 to ip-10-0-131-123.us-west-2.compute.internal 39m Normal Scheduled pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-p8ljt Successfully assigned openshift-ingress/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-p8ljt to ip-10-0-131-123.us-west-2.compute.internal 5m24s Normal Scheduled pod/payload-processing-e2e-authcfg-http-a3c14c-69bc5c5c66-pg9zj Successfully assigned openshift-ingress/payload-processing-e2e-authcfg-http-a3c14c-69bc5c5c66-pg9zj to ip-10-0-141-180.us-west-2.compute.internal 49m Normal Scheduled pod/payload-processing-e2e-derive-09efcb6d-699987c7fd-j2v69 Successfully assigned openshift-ingress/payload-processing-e2e-derive-09efcb6d-699987c7fd-j2v69 to ip-10-0-141-180.us-west-2.compute.internal 6m41s Normal Scheduled pod/e2e-ait-8214dff9-openshift-default-7b77bdd57c-zm5nl Successfully assigned openshift-ingress/e2e-ait-8214dff9-openshift-default-7b77bdd57c-zm5nl to ip-10-0-141-180.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-px9v2 Successfully assigned openshift-ingress/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-px9v2 to ip-10-0-131-123.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-q7dtb Successfully assigned openshift-ingress/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-q7dtb to ip-10-0-141-180.us-west-2.compute.internal 27m Normal Scheduled pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-sz5n2 Successfully assigned openshift-ingress/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-sz5n2 to ip-10-0-141-180.us-west-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w3-d9e782-588c96f65f-jbm6v Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d9e782-588c96f65f-jbm6v to ip-10-0-131-123.us-west-2.compute.internal 58m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-m59d5 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-m59d5 to ip-10-0-139-66.us-west-2.compute.internal 39m Normal Scheduled pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-kl5fq Successfully assigned openshift-ingress/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-kl5fq to ip-10-0-131-123.us-west-2.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-api-a-c9c21e-747fd5b8cf-9s6gf Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-c9c21e-747fd5b8cf-9s6gf to ip-10-0-131-123.us-west-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-inf-a-03c703-587b89bbdd-vzfkm Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-03c703-587b89bbdd-vzfkm to ip-10-0-141-180.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-t28zs Successfully assigned openshift-ingress/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-t28zs to ip-10-0-139-66.us-west-2.compute.internal 39m Normal Scheduled pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-j9skc Successfully assigned openshift-ingress/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-j9skc to ip-10-0-139-66.us-west-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w0-957019-76d548dcf-bbqz5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-957019-76d548dcf-bbqz5 to ip-10-0-141-180.us-west-2.compute.internal 65m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-x5f97 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-x5f97 to ip-10-0-139-66.us-west-2.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-inf-a-348a33-7f4b555888-dk2l2 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-348a33-7f4b555888-dk2l2 to ip-10-0-131-123.us-west-2.compute.internal 39m Normal Scheduled pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-fxvfs Successfully assigned openshift-ingress/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-fxvfs to ip-10-0-141-180.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-ait-8e7d0098-openshift-default-7c65b994b7-ntss6 Successfully assigned openshift-ingress/e2e-ait-8e7d0098-openshift-default-7c65b994b7-ntss6 to ip-10-0-139-66.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-t4ms2 Successfully assigned openshift-ingress/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-t4ms2 to ip-10-0-141-180.us-west-2.compute.internal 58m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-xsgv7 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-xsgv7 to ip-10-0-141-180.us-west-2.compute.internal 38m Normal Scheduled pod/payload-processing-e2e-inf-a-76b073-78787bc78-d6l68 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-76b073-78787bc78-d6l68 to ip-10-0-141-180.us-west-2.compute.internal 36m Normal Scheduled pod/payload-processing-e2e-inf-a-ddf61a-5f445c7548-fwpmj Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-ddf61a-5f445c7548-fwpmj to ip-10-0-139-66.us-west-2.compute.internal 39m Normal Scheduled pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-c55xx Successfully assigned openshift-ingress/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-c55xx to ip-10-0-141-180.us-west-2.compute.internal 65m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-lwhxc Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-lwhxc to ip-10-0-141-180.us-west-2.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-api-b-1e5c33-56fbff6867-ckkk9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-1e5c33-56fbff6867-ckkk9 to ip-10-0-139-66.us-west-2.compute.internal 35m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-jpjkl Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-jpjkl to ip-10-0-141-180.us-west-2.compute.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w6-f7b8d8-576dfbf9f9-d9d8l Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w6-f7b8d8-576dfbf9f9-d9d8l to ip-10-0-141-180.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-ait-8e7d0098-openshift-default-7c65b994b7-tc65h Successfully assigned openshift-ingress/e2e-ait-8e7d0098-openshift-default-7c65b994b7-tc65h to ip-10-0-141-180.us-west-2.compute.internal 69m Normal Scheduled pod/kube-auth-proxy-546bf7cb95-r7gvf Successfully assigned openshift-ingress/kube-auth-proxy-546bf7cb95-r7gvf to ip-10-0-139-66.us-west-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-shared-b-w4-4a3dba-57c6cf5fd4-n7zpn Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-4a3dba-57c6cf5fd4-n7zpn to ip-10-0-131-123.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-7w9dq Successfully assigned openshift-ingress/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-7w9dq to ip-10-0-141-180.us-west-2.compute.internal 64m Normal Scheduled pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-ggkpj Successfully assigned openshift-ingress/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-ggkpj to ip-10-0-131-123.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-migrate-d32a7cc0-gw-openshift-default-74cb955697-d8gkc Successfully assigned openshift-ingress/e2e-migrate-d32a7cc0-gw-openshift-default-74cb955697-d8gkc to ip-10-0-139-66.us-west-2.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-inf-b-37abc0-54fc77c78-vvrhm Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-37abc0-54fc77c78-vvrhm to ip-10-0-131-123.us-west-2.compute.internal 69m Normal Scheduled pod/kube-auth-proxy-546bf7cb95-4gjhl Successfully assigned openshift-ingress/kube-auth-proxy-546bf7cb95-4gjhl to ip-10-0-131-123.us-west-2.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-inf-b-460d4e-6457b77684-dttcx Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-460d4e-6457b77684-dttcx to ip-10-0-141-180.us-west-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w4-4a3dba-576dcfd7cf-6x2bs Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-4a3dba-576dcfd7cf-6x2bs to ip-10-0-131-123.us-west-2.compute.internal 63m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-46b586-669f464d79-fsqx4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-46b586-669f464d79-fsqx4 to ip-10-0-141-180.us-west-2.compute.internal 37m Normal Scheduled pod/payload-processing-e2e-api-a-ae3cd0-9bb5478db-92qxb Successfully assigned openshift-ingress/payload-processing-e2e-api-a-ae3cd0-9bb5478db-92qxb to ip-10-0-141-180.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-hnqpt Successfully assigned openshift-ingress/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-hnqpt to ip-10-0-139-66.us-west-2.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-api-b-39e3a8-545496b975-zvh7k Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-39e3a8-545496b975-zvh7k to ip-10-0-141-180.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-hs4pf Successfully assigned openshift-ingress/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-hs4pf to ip-10-0-141-180.us-west-2.compute.internal 35m Normal Scheduled pod/payload-processing-e2e-inf-b-49837c-79c55dc878-vd2k9 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-49837c-79c55dc878-vd2k9 to ip-10-0-139-66.us-west-2.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-mt-a251fdfb-6f8487684c-s2hz7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-a251fdfb-6f8487684c-s2hz7 to ip-10-0-139-66.us-west-2.compute.internal 52m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-wqjzk Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wqjzk to ip-10-0-131-123.us-west-2.compute.internal 47m Normal Scheduled pod/e2e-api-a-71b76a-openshift-default-77db967c97-lh4ld Successfully assigned openshift-ingress/e2e-api-a-71b76a-openshift-default-77db967c97-lh4ld to ip-10-0-141-180.us-west-2.compute.internal 46m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-kzzph Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-kzzph to ip-10-0-139-66.us-west-2.compute.internal 52m Normal Scheduled pod/payload-processing-e2e-migrate-d32a7cc0-76fd97d5d6-62rcv Successfully assigned openshift-ingress/payload-processing-e2e-migrate-d32a7cc0-76fd97d5d6-62rcv to ip-10-0-141-180.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-m8pb2 Successfully assigned openshift-ingress/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-m8pb2 to ip-10-0-139-66.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-pc4b9 Successfully assigned openshift-ingress/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-pc4b9 to ip-10-0-141-180.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-r8xrl Successfully assigned openshift-ingress/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-r8xrl to ip-10-0-139-66.us-west-2.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w4-54ad8c-7b7f75c84c-nntg5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-54ad8c-7b7f75c84c-nntg5 to ip-10-0-141-180.us-west-2.compute.internal 74m Normal Scheduled pod/router-default-66d97c9c97-qqxj5 Successfully assigned openshift-ingress/router-default-66d97c9c97-qqxj5 to ip-10-0-141-180.us-west-2.compute.internal 75m Warning FailedScheduling pod/router-default-66d97c9c97-qqxj5 no nodes available to schedule pods 81m Warning FailedScheduling pod/router-default-66d97c9c97-qqxj5 no nodes available to schedule pods 53m Normal Scheduled pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-smvsh Successfully assigned openshift-ingress/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-smvsh to ip-10-0-131-123.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-vx2nt Successfully assigned openshift-ingress/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-vx2nt to ip-10-0-131-123.us-west-2.compute.internal 68m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-8t7ww Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8t7ww to ip-10-0-141-180.us-west-2.compute.internal 35m Normal Scheduled pod/e2e-inf-b-49837c-openshift-default-566c644bb9-wb52d Successfully assigned openshift-ingress/e2e-inf-b-49837c-openshift-default-566c644bb9-wb52d to ip-10-0-131-123.us-west-2.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-mt-a251fdfb-6fb6bf7d6b-w2z4v Successfully assigned openshift-ingress/payload-processing-e2e-mt-a251fdfb-6fb6bf7d6b-w2z4v to ip-10-0-131-123.us-west-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-zgt8d Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-zgt8d to ip-10-0-131-123.us-west-2.compute.internal 38m Normal Scheduled pod/payload-processing-e2e-api-a-92e375-75c9fbbc84-nt42r Successfully assigned openshift-ingress/payload-processing-e2e-api-a-92e375-75c9fbbc84-nt42r to ip-10-0-141-180.us-west-2.compute.internal 38m Normal Scheduled pod/e2e-api-a-92e375-openshift-default-6987df58bc-jzxns Successfully assigned openshift-ingress/e2e-api-a-92e375-openshift-default-6987df58bc-jzxns to ip-10-0-141-180.us-west-2.compute.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-ait-0d96587a-584d7b5dff-ccjj6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-0d96587a-584d7b5dff-ccjj6 to ip-10-0-141-180.us-west-2.compute.internal 58m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-q6mtd Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-q6mtd to ip-10-0-139-66.us-west-2.compute.internal 6m33s Normal Scheduled pod/payload-pre-processing-e2e-ait-8214dff9-854cb688f9-z4k5h Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-8214dff9-854cb688f9-z4k5h to ip-10-0-131-123.us-west-2.compute.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-ait-8e7d0098-6794f6ff67-429f5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-8e7d0098-6794f6ff67-429f5 to ip-10-0-141-180.us-west-2.compute.internal 47m Normal Scheduled pod/e2e-shared-b-w6-0d75be-openshift-default-57f99d9cbc-smttn Successfully assigned openshift-ingress/e2e-shared-b-w6-0d75be-openshift-default-57f99d9cbc-smttn to ip-10-0-141-180.us-west-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-pqml2 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-pqml2 to ip-10-0-139-66.us-west-2.compute.internal 44m Normal Scheduled pod/e2e-shared-b-w4-4a3dba-openshift-default-dc9664475-zxb5s Successfully assigned openshift-ingress/e2e-shared-b-w4-4a3dba-openshift-default-dc9664475-zxb5s to ip-10-0-141-180.us-west-2.compute.internal 51m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-sgc28 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-sgc28 to ip-10-0-131-123.us-west-2.compute.internal 64m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-35a187-7887c98dff-5h96p Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-35a187-7887c98dff-5h96p to ip-10-0-141-180.us-west-2.compute.internal 64m Normal Scheduled pod/payload-processing-e2e-worker-w6-35a187-59b76ccb5b-25xkv Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-35a187-59b76ccb5b-25xkv to ip-10-0-139-66.us-west-2.compute.internal 15m Normal Scheduled pod/e2e-worker-single-main-0e6572-openshift-default-5c475f9cbf97qjl Successfully assigned openshift-ingress/e2e-worker-single-main-0e6572-openshift-default-5c475f9cbf97qjl to ip-10-0-141-180.us-west-2.compute.internal 49m Normal Scheduled pod/payload-processing-e2e-mt-a251fdfb-6fb6bf7d6b-xmmgr Successfully assigned openshift-ingress/payload-processing-e2e-mt-a251fdfb-6fb6bf7d6b-xmmgr to ip-10-0-141-180.us-west-2.compute.internal 65m Normal Scheduled pod/payload-processing-7bb9489676-2kpl5 Successfully assigned openshift-ingress/payload-processing-7bb9489676-2kpl5 to ip-10-0-141-180.us-west-2.compute.internal 43m Normal Scheduled pod/e2e-shared-b-w3-d9e782-openshift-default-7ccffd49b5-7k8qc Successfully assigned openshift-ingress/e2e-shared-b-w3-d9e782-openshift-default-7ccffd49b5-7k8qc to ip-10-0-131-123.us-west-2.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w3-ab0ac9-7978cbf74c-qxdvb Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ab0ac9-7978cbf74c-qxdvb to ip-10-0-141-180.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-inf-b-460d4e-openshift-default-575d5fbfb7-ftjgz Successfully assigned openshift-ingress/e2e-inf-b-460d4e-openshift-default-575d5fbfb7-ftjgz to ip-10-0-141-180.us-west-2.compute.internal 37m Normal Scheduled pod/e2e-api-a-ae3cd0-openshift-default-5dbcd797d5-hvcv4 Successfully assigned openshift-ingress/e2e-api-a-ae3cd0-openshift-default-5dbcd797d5-hvcv4 to ip-10-0-141-180.us-west-2.compute.internal 64m Normal Scheduled pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-2mn85 Successfully assigned openshift-ingress/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-2mn85 to ip-10-0-131-123.us-west-2.compute.internal 63m Normal Scheduled pod/payload-processing-e2e-worker-w5-fe01fa-698f78d94b-4wmlb Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-fe01fa-698f78d94b-4wmlb to ip-10-0-141-180.us-west-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-znmqk Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-znmqk to ip-10-0-139-66.us-west-2.compute.internal 39m Normal Scheduled pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-qmsfx Successfully assigned openshift-ingress/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-qmsfx to ip-10-0-141-180.us-west-2.compute.internal 63m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-fe01fa-576c754f8c-8dm7l Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-fe01fa-576c754f8c-8dm7l to ip-10-0-139-66.us-west-2.compute.internal 51m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-thj8z Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-thj8z to ip-10-0-139-66.us-west-2.compute.internal 44m Normal Scheduled pod/e2e-shared-b-w0-957019-openshift-default-59bc4d44f9-8xtcd Successfully assigned openshift-ingress/e2e-shared-b-w0-957019-openshift-default-59bc4d44f9-8xtcd to ip-10-0-141-180.us-west-2.compute.internal 63m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-34f1ce-9b77f84c9-jfnmg Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-34f1ce-9b77f84c9-jfnmg to ip-10-0-139-66.us-west-2.compute.internal 11m Normal Scheduled pod/e2e-worker-single-main-e98014-openshift-default-596b484cdf5xsgc Successfully assigned openshift-ingress/e2e-worker-single-main-e98014-openshift-default-596b484cdf5xsgc to ip-10-0-141-180.us-west-2.compute.internal 63m Normal Scheduled pod/payload-processing-e2e-worker-w4-34f1ce-56d77c46fd-9wsff Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-34f1ce-56d77c46fd-9wsff to ip-10-0-139-66.us-west-2.compute.internal 52m Normal Scheduled pod/payload-processing-e2e-mt-e0751b65-769966b6c4-9pw2x Successfully assigned openshift-ingress/payload-processing-e2e-mt-e0751b65-769966b6c4-9pw2x to ip-10-0-139-66.us-west-2.compute.internal 64m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-986d37-5d7d64658c-h2tgp Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-986d37-5d7d64658c-h2tgp to ip-10-0-141-180.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-shared-a-w6-f7b8d8-openshift-default-6b8ffbcc97-cccbw Successfully assigned openshift-ingress/e2e-shared-a-w6-f7b8d8-openshift-default-6b8ffbcc97-cccbw to ip-10-0-139-66.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-4jhn8 Successfully assigned openshift-ingress/e2e-worker-w0-46b586-openshift-default-86c598c96c-4jhn8 to ip-10-0-131-123.us-west-2.compute.internal 41m Normal Scheduled pod/e2e-api-a-c9c21e-openshift-default-749bcfc96c-wjzsh Successfully assigned openshift-ingress/e2e-api-a-c9c21e-openshift-default-749bcfc96c-wjzsh to ip-10-0-141-180.us-west-2.compute.internal 35m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-mz295 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-mz295 to ip-10-0-141-180.us-west-2.compute.internal 63m Normal Scheduled pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-6hww6 Successfully assigned openshift-ingress/e2e-worker-w0-46b586-openshift-default-86c598c96c-6hww6 to ip-10-0-139-66.us-west-2.compute.internal 58m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-vt4mg Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-vt4mg to ip-10-0-131-123.us-west-2.compute.internal 64m Normal Scheduled pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-8ckgt Successfully assigned openshift-ingress/e2e-worker-w0-46b586-openshift-default-86c598c96c-8ckgt to ip-10-0-131-123.us-west-2.compute.internal 64m Normal Scheduled pod/payload-processing-e2e-worker-w3-986d37-695476d5cd-99x6j Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-986d37-695476d5cd-99x6j to ip-10-0-141-180.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-wvq8z Successfully assigned openshift-ingress/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-wvq8z to ip-10-0-131-123.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-ccdzj Successfully assigned openshift-ingress/e2e-worker-w0-46b586-openshift-default-86c598c96c-ccdzj to ip-10-0-141-180.us-west-2.compute.internal 34m Normal Scheduled pod/e2e-inf-b-37abc0-openshift-default-65b888884f-pqjfl Successfully assigned openshift-ingress/e2e-inf-b-37abc0-openshift-default-65b888884f-pqjfl to ip-10-0-141-180.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8bt76 Successfully assigned openshift-ingress/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8bt76 to ip-10-0-141-180.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-f6k98 Successfully assigned openshift-ingress/e2e-worker-w0-46b586-openshift-default-86c598c96c-f6k98 to ip-10-0-139-66.us-west-2.compute.internal 64m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-27e82b-5968f9f875-t9qz2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-27e82b-5968f9f875-t9qz2 to ip-10-0-141-180.us-west-2.compute.internal 47m Normal Scheduled pod/e2e-shared-a-w4-54ad8c-openshift-default-575c545fdf-gvfbr Successfully assigned openshift-ingress/e2e-shared-a-w4-54ad8c-openshift-default-575c545fdf-gvfbr to ip-10-0-141-180.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-jdcvx Successfully assigned openshift-ingress/e2e-worker-w0-46b586-openshift-default-86c598c96c-jdcvx to ip-10-0-141-180.us-west-2.compute.internal 47m Normal Scheduled pod/payload-processing-e2e-shared-a-w0-d5360e-854fc9bc5d-mc76v Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-d5360e-854fc9bc5d-mc76v to ip-10-0-141-180.us-west-2.compute.internal 34m Normal Scheduled pod/e2e-api-b-1e5c33-openshift-default-6f5f6b8dc9-lmv7w Successfully assigned openshift-ingress/e2e-api-b-1e5c33-openshift-default-6f5f6b8dc9-lmv7w to ip-10-0-131-123.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-kzqr4 Successfully assigned openshift-ingress/e2e-worker-w0-46b586-openshift-default-86c598c96c-kzqr4 to ip-10-0-141-180.us-west-2.compute.internal 64m Normal Scheduled pod/payload-processing-e2e-worker-w2-27e82b-64f94b9586-wb68f Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-27e82b-64f94b9586-wb68f to ip-10-0-141-180.us-west-2.compute.internal 58m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-wftsq Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-wftsq to ip-10-0-141-180.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-q6bx8 Successfully assigned openshift-ingress/e2e-worker-w0-46b586-openshift-default-86c598c96c-q6bx8 to ip-10-0-131-123.us-west-2.compute.internal 52m Normal Scheduled pod/payload-processing-e2e-ait-0d96587a-58d9d8846-j62hj Successfully assigned openshift-ingress/payload-processing-e2e-ait-0d96587a-58d9d8846-j62hj to ip-10-0-141-180.us-west-2.compute.internal 63m Normal Scheduled pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-tgc8q Successfully assigned openshift-ingress/e2e-worker-w0-46b586-openshift-default-86c598c96c-tgc8q to ip-10-0-141-180.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-thfjt Successfully assigned openshift-ingress/e2e-worker-w0-46b586-openshift-default-86c598c96c-thfjt to ip-10-0-139-66.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8hzf8 Successfully assigned openshift-ingress/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8hzf8 to ip-10-0-139-66.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8qtdz Successfully assigned openshift-ingress/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8qtdz to ip-10-0-139-66.us-west-2.compute.internal 63m Normal Scheduled pod/payload-processing-e2e-worker-w1-5c3374-7969595c9d-gzhtd Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-5c3374-7969595c9d-gzhtd to ip-10-0-141-180.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-d44pq Successfully assigned openshift-ingress/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-d44pq to ip-10-0-141-180.us-west-2.compute.internal 46m Normal Scheduled pod/e2e-shared-a-w3-ab0ac9-openshift-default-54c6546bdf-lxwwq Successfully assigned openshift-ingress/e2e-shared-a-w3-ab0ac9-openshift-default-54c6546bdf-lxwwq to ip-10-0-141-180.us-west-2.compute.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w0-d5360e-b6fb568dc-htsps Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d5360e-b6fb568dc-htsps to ip-10-0-141-180.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-qxg67 Successfully assigned openshift-ingress/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-qxg67 to ip-10-0-139-66.us-west-2.compute.internal 63m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-5c3374-f56fc6f79-7b9pp Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-5c3374-f56fc6f79-7b9pp to ip-10-0-141-180.us-west-2.compute.internal 51m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-wv5pf Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-wv5pf to ip-10-0-139-66.us-west-2.compute.internal 45m Normal Scheduled pod/e2e-api-b-39e3a8-openshift-default-6b6658797-ghqtx Successfully assigned openshift-ingress/e2e-api-b-39e3a8-openshift-default-6b6658797-ghqtx to ip-10-0-141-180.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t5hkt Successfully assigned openshift-ingress/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t5hkt to ip-10-0-131-123.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t762r Successfully assigned openshift-ingress/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t762r to ip-10-0-141-180.us-west-2.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-shared-a-w3-ab0ac9-847986794b-bbf7n Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-ab0ac9-847986794b-bbf7n to ip-10-0-141-180.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-w6bmh Successfully assigned openshift-ingress/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-w6bmh to ip-10-0-131-123.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-xkbnw Successfully assigned openshift-ingress/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-xkbnw to ip-10-0-141-180.us-west-2.compute.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-api-a-ae3cd0-7ffbd9dc87-l54tr Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-ae3cd0-7ffbd9dc87-l54tr to ip-10-0-141-180.us-west-2.compute.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-mt-e0751b65-7d7cd79459-76xvm Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-e0751b65-7d7cd79459-76xvm to ip-10-0-141-180.us-west-2.compute.internal 36m Normal Scheduled pod/e2e-inf-a-ddf61a-openshift-default-f57f54bc-4gvfk Successfully assigned openshift-ingress/e2e-inf-a-ddf61a-openshift-default-f57f54bc-4gvfk to ip-10-0-141-180.us-west-2.compute.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-api-a-92e375-6d87d5b947-4249b Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-92e375-6d87d5b947-4249b to ip-10-0-141-180.us-west-2.compute.internal 69m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-58bf96fd7cbr2hg Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-58bf96fd7cbr2hg to ip-10-0-141-180.us-west-2.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-shared-a-w4-54ad8c-7c679ffcbb-kmx4b Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-54ad8c-7c679ffcbb-kmx4b to ip-10-0-141-180.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-4jrqx Successfully assigned openshift-ingress/e2e-worker-w6-35a187-openshift-default-5b54c65f75-4jrqx to ip-10-0-139-66.us-west-2.compute.internal 71m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-x544h Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-x544h to ip-10-0-131-123.us-west-2.compute.internal 65m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-hkgx7 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-hkgx7 to ip-10-0-131-123.us-west-2.compute.internal 71m Normal Scheduled pod/openshift-ai-inference-openshift-ai-inference-7464bdcdc9-dmjmb Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-7464bdcdc9-dmjmb to ip-10-0-141-180.us-west-2.compute.internal 63m Normal Scheduled pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-657lr Successfully assigned openshift-ingress/e2e-worker-w6-35a187-openshift-default-5b54c65f75-657lr to ip-10-0-131-123.us-west-2.compute.internal 35m Normal Scheduled pod/e2e-api-b-d687ff-openshift-default-5c75fcb6d5-25thf Successfully assigned openshift-ingress/e2e-api-b-d687ff-openshift-default-5c75fcb6d5-25thf to ip-10-0-141-180.us-west-2.compute.internal 63m Normal Scheduled pod/payload-processing-e2e-worker-w0-46b586-556df6d97b-dcfpj Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-46b586-556df6d97b-dcfpj to ip-10-0-139-66.us-west-2.compute.internal 11m Normal Scheduled pod/payload-processing-e2e-worker-single-main-e98014-54d9bccbbjcft2 Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-e98014-54d9bccbbjcft2 to ip-10-0-131-123.us-west-2.compute.internal 15m Normal Scheduled pod/payload-processing-e2e-worker-single-main-0e6572-64778475dfmdr6 Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-0e6572-64778475dfmdr6 to ip-10-0-131-123.us-west-2.compute.internal 49m Normal Scheduled pod/payload-processing-e2e-shared-a-w6-f7b8d8-6b866d558b-hq859 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w6-f7b8d8-6b866d558b-hq859 to ip-10-0-139-66.us-west-2.compute.internal 51m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-dpcqs Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-dpcqs to ip-10-0-141-180.us-west-2.compute.internal 64m Normal Scheduled pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-678tp Successfully assigned openshift-ingress/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-678tp to ip-10-0-141-180.us-west-2.compute.internal 47m Normal Scheduled pod/e2e-shared-a-w0-d5360e-openshift-default-fbccfc549-d828w Successfully assigned openshift-ingress/e2e-shared-a-w0-d5360e-openshift-default-fbccfc549-d828w to ip-10-0-141-180.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-84bch Successfully assigned openshift-ingress/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-84bch to ip-10-0-131-123.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-bn695 Successfully assigned openshift-ingress/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-bn695 to ip-10-0-131-123.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-j2j46 Successfully assigned openshift-ingress/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-j2j46 to ip-10-0-141-180.us-west-2.compute.internal 47m Normal Scheduled pod/payload-processing-e2e-shared-b-w6-0d75be-7b66b84856-j28w4 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w6-0d75be-7b66b84856-j28w4 to ip-10-0-141-180.us-west-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-z6jt4 Successfully assigned openshift-ingress/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-z6jt4 to ip-10-0-139-66.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-x59pr Successfully assigned openshift-ingress/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-x59pr to ip-10-0-139-66.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-vb9r5 Successfully assigned openshift-ingress/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-vb9r5 to ip-10-0-141-180.us-west-2.compute.internal 5m32s Normal Scheduled pod/e2e-authcfg-http-a3c14c-openshift-default-5b7bd5fc65-bztx4 Successfully assigned openshift-ingress/e2e-authcfg-http-a3c14c-openshift-default-5b7bd5fc65-bztx4 to ip-10-0-131-123.us-west-2.compute.internal 47m Normal Scheduled pod/payload-processing-e2e-api-a-71b76a-7b898f6878-hmdwh Successfully assigned openshift-ingress/payload-processing-e2e-api-a-71b76a-7b898f6878-hmdwh to ip-10-0-139-66.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-n4hsl Successfully assigned openshift-ingress/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-n4hsl to ip-10-0-139-66.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-ppwlm Successfully assigned openshift-ingress/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-ppwlm to ip-10-0-139-66.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-mt-e0751b65-openshift-default-5846888d89-2pzt4 Successfully assigned openshift-ingress/e2e-mt-e0751b65-openshift-default-5846888d89-2pzt4 to ip-10-0-139-66.us-west-2.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-api-a-c9c21e-588d55cd7b-pk2ss Successfully assigned openshift-ingress/payload-processing-e2e-api-a-c9c21e-588d55cd7b-pk2ss to ip-10-0-131-123.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-ait-0d96587a-openshift-default-bcfbd49cc-xrrp4 Successfully assigned openshift-ingress/e2e-ait-0d96587a-openshift-default-bcfbd49cc-xrrp4 to ip-10-0-141-180.us-west-2.compute.internal 38m Normal Scheduled pod/e2e-inf-a-76b073-openshift-default-697d759dd5-ql7sd Successfully assigned openshift-ingress/e2e-inf-a-76b073-openshift-default-697d759dd5-ql7sd to ip-10-0-141-180.us-west-2.compute.internal 23m Normal Scheduled pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-s4kvf Successfully assigned openshift-ingress/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-s4kvf to ip-10-0-131-123.us-west-2.compute.internal 15m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-0e6572-8468772hmv Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-0e6572-8468772hmv to ip-10-0-131-123.us-west-2.compute.internal 49m Normal Scheduled pod/payload-processing-e2e-ait-7c8802f8-5db6686694-5hbns Successfully assigned openshift-ingress/payload-processing-e2e-ait-7c8802f8-5db6686694-5hbns to ip-10-0-141-180.us-west-2.compute.internal 65m Normal Scheduled pod/payload-pre-processing-7b98bb678c-njw5h Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-njw5h to ip-10-0-141-180.us-west-2.compute.internal 63m Normal Scheduled pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-xgpsz Successfully assigned openshift-ingress/e2e-worker-w6-35a187-openshift-default-5b54c65f75-xgpsz to ip-10-0-141-180.us-west-2.compute.internal 39m Normal Scheduled pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-z9l6k Successfully assigned openshift-ingress/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-z9l6k to ip-10-0-139-66.us-west-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-shared-b-w3-d9e782-67987686-hwzmf Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-d9e782-67987686-hwzmf to ip-10-0-141-180.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-derive-09efcb6d-openshift-default-7c46fd677c-6wtzg Successfully assigned openshift-ingress/e2e-derive-09efcb6d-openshift-default-7c46fd677c-6wtzg to ip-10-0-141-180.us-west-2.compute.internal 35m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-gtzrp Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-gtzrp to ip-10-0-139-66.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-txzbm Successfully assigned openshift-ingress/e2e-worker-w6-35a187-openshift-default-5b54c65f75-txzbm to ip-10-0-131-123.us-west-2.compute.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w6-0d75be-75f9855f5-kftq4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0d75be-75f9855f5-kftq4 to ip-10-0-141-180.us-west-2.compute.internal 64m Normal Scheduled pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-qc65q Successfully assigned openshift-ingress/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-qc65q to ip-10-0-131-123.us-west-2.compute.internal 71m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-pnm97 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-pnm97 to ip-10-0-141-180.us-west-2.compute.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-api-b-d687ff-7c6c95ffc9-4xkwz Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-d687ff-7c6c95ffc9-4xkwz to ip-10-0-139-66.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-2qnns Successfully assigned openshift-ingress/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-2qnns to ip-10-0-141-180.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-4dqrc Successfully assigned openshift-ingress/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-4dqrc to ip-10-0-131-123.us-west-2.compute.internal 63m Normal Scheduled pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-rb7qp Successfully assigned openshift-ingress/e2e-worker-w6-35a187-openshift-default-5b54c65f75-rb7qp to ip-10-0-139-66.us-west-2.compute.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-derive-09efcb6d-c7bfbb5cf-fsvrl Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-09efcb6d-c7bfbb5cf-fsvrl to ip-10-0-141-180.us-west-2.compute.internal 5m24s Normal Scheduled pod/payload-pre-processing-e2e-authcfg-http-a3c14c-775858b75-gthbc Successfully assigned openshift-ingress/payload-pre-processing-e2e-authcfg-http-a3c14c-775858b75-gthbc to ip-10-0-131-123.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-qsdr8 Successfully assigned openshift-ingress/e2e-worker-w6-35a187-openshift-default-5b54c65f75-qsdr8 to ip-10-0-131-123.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-nr4qz Successfully assigned openshift-ingress/e2e-worker-w6-35a187-openshift-default-5b54c65f75-nr4qz to ip-10-0-141-180.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-klh6t Successfully assigned openshift-ingress/e2e-worker-w6-35a187-openshift-default-5b54c65f75-klh6t to ip-10-0-141-180.us-west-2.compute.internal 63m Normal Scheduled pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-bsqhk Successfully assigned openshift-ingress/e2e-worker-w6-35a187-openshift-default-5b54c65f75-bsqhk to ip-10-0-139-66.us-west-2.compute.internal 60m Normal Scheduled pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-7gm84 Successfully assigned openshift-ingress/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-7gm84 to ip-10-0-141-180.us-west-2.compute.internal 45m Normal Scheduled pod/e2e-inf-a-03c703-openshift-default-78bd587bff-5m2n8 Successfully assigned openshift-ingress/e2e-inf-a-03c703-openshift-default-78bd587bff-5m2n8 to ip-10-0-141-180.us-west-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-shared-b-w0-957019-788c577f9b-2shs7 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-957019-788c577f9b-2shs7 to ip-10-0-141-180.us-west-2.compute.internal 51m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-zvnnz Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-zvnnz to ip-10-0-141-180.us-west-2.compute.internal 64m Normal Scheduled pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-94tkp Successfully assigned openshift-ingress/e2e-worker-w6-35a187-openshift-default-5b54c65f75-94tkp to ip-10-0-141-180.us-west-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-n2wcf Successfully assigned openshift-ingress/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-n2wcf to ip-10-0-139-66.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-inf-a-348a33-openshift-default-5775c46d7-w9cm4 Successfully assigned openshift-ingress/e2e-inf-a-348a33-openshift-default-5775c46d7-w9cm4 to ip-10-0-131-123.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-khcqd Successfully assigned openshift-ingress/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-khcqd to ip-10-0-141-180.us-west-2.compute.internal 35m Normal Scheduled pod/maas-default-gateway-openshift-default-8f6f87f59-nn2nk Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-8f6f87f59-nn2nk to ip-10-0-139-66.us-west-2.compute.internal 52m Normal Scheduled pod/payload-processing-e2e-ait-8e7d0098-8f775757b-4mt6m Successfully assigned openshift-ingress/payload-processing-e2e-ait-8e7d0098-8f775757b-4mt6m to ip-10-0-141-180.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-dxpzb Successfully assigned openshift-ingress/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-dxpzb to ip-10-0-131-123.us-west-2.compute.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-api-a-71b76a-64f77d669f-hm2nn Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-71b76a-64f77d669f-hm2nn to ip-10-0-141-180.us-west-2.compute.internal 6m33s Normal Scheduled pod/payload-processing-e2e-ait-8214dff9-85b686f698-krx74 Successfully assigned openshift-ingress/payload-processing-e2e-ait-8214dff9-85b686f698-krx74 to ip-10-0-131-123.us-west-2.compute.internal 81m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-66d97c9c97 from 0 to 1 81m Normal SuccessfulCreate replicaset/router-default-66d97c9c97 Created pod: router-default-66d97c9c97-qqxj5 81m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 81m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 81m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 74m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 74m Warning FailedMount pod/router-default-66d97c9c97-qqxj5 MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 74m Warning FailedMount pod/router-default-66d97c9c97-qqxj5 MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 74m Warning FailedMount pod/router-default-66d97c9c97-qqxj5 MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 74m Warning FailedMount pod/router-default-66d97c9c97-qqxj5 MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 74m Normal Created pod/router-default-66d97c9c97-qqxj5 Container created 74m Normal AddedInterface pod/router-default-66d97c9c97-qqxj5 Add eth0 [10.132.0.14/23] from ovn-kubernetes 74m Normal Pulled pod/router-default-66d97c9c97-qqxj5 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 74m Normal Started pod/router-default-66d97c9c97-qqxj5 Container started 71m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 71m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-x544h 71m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-x544h Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 71m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-x544h Add eth0 [10.133.0.18/23] from ovn-kubernetes 71m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 71m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-x544h Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.47s (4.47s including waiting). Image size: 185476477 bytes. 71m Normal Created pod/istiod-openshift-gateway-7876bfcb74-x544h Container created 71m Normal Started pod/istiod-openshift-gateway-7876bfcb74-x544h Container started 71m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-x544h Readiness probe failed: HTTP probe failed with statuscode: 503 71m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-x544h Readiness probe error: HTTP probe failed with statuscode: 503... 71m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-8f6f87f59 Created pod: maas-default-gateway-openshift-default-8f6f87f59-pnm97 71m Normal EnsuringLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensuring load balancer 71m Normal SuccessfulCreate replicaset/openshift-ai-inference-openshift-ai-inference-7464bdcdc9 Created pod: openshift-ai-inference-openshift-ai-inference-7464bdcdc9-dmjmb 71m Normal ScalingReplicaSet deployment/openshift-ai-inference-openshift-ai-inference Scaled up replica set openshift-ai-inference-openshift-ai-inference-7464bdcdc9 from 0 to 1 71m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-pnm97 Add eth0 [10.132.0.36/23] from ovn-kubernetes 71m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-8f6f87f59 from 0 to 1 71m Normal AddedInterface pod/openshift-ai-inference-openshift-ai-inference-7464bdcdc9-dmjmb Add eth0 [10.132.0.37/23] from ovn-kubernetes 71m Normal Pulling pod/openshift-ai-inference-openshift-ai-inference-7464bdcdc9-dmjmb Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 71m Normal Pulling pod/maas-default-gateway-openshift-default-8f6f87f59-pnm97 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 70m Normal EnsuredLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensured load balancer 70m Normal Started pod/openshift-ai-inference-openshift-ai-inference-7464bdcdc9-dmjmb Container started 70m Normal Pulled pod/openshift-ai-inference-openshift-ai-inference-7464bdcdc9-dmjmb Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 9.124s (9.124s including waiting). Image size: 183555333 bytes. 70m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-pnm97 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 9.593s (9.593s including waiting). Image size: 183555333 bytes. 70m Normal Created pod/openshift-ai-inference-openshift-ai-inference-7464bdcdc9-dmjmb Container created 70m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 70m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 69m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-546bf7cb95 from 0 to 2 69m Normal SuccessfulCreate replicaset/kube-auth-proxy-546bf7cb95 Created pod: kube-auth-proxy-546bf7cb95-r7gvf 69m Normal SuccessfulCreate replicaset/kube-auth-proxy-546bf7cb95 Created pod: kube-auth-proxy-546bf7cb95-4gjhl 69m Normal AddedInterface pod/kube-auth-proxy-546bf7cb95-4gjhl Add eth0 [10.133.0.20/23] from ovn-kubernetes 69m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-58bf96fd7c from 0 to 1 69m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-58bf96fd7c Created pod: data-science-gateway-data-science-gateway-class-58bf96fd7cbr2hg 69m Normal Pulling pod/kube-auth-proxy-546bf7cb95-r7gvf Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 69m Normal AddedInterface pod/kube-auth-proxy-546bf7cb95-r7gvf Add eth0 [10.134.0.24/23] from ovn-kubernetes 69m Normal Pulling pod/kube-auth-proxy-546bf7cb95-4gjhl Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 69m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-58bf96fd7cbr2hg Add eth0 [10.132.0.45/23] from ovn-kubernetes 69m Normal Started pod/data-science-gateway-data-science-gateway-class-58bf96fd7cbr2hg Container started 69m Normal Pulled pod/data-science-gateway-data-science-gateway-class-58bf96fd7cbr2hg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 69m Normal Created pod/data-science-gateway-data-science-gateway-class-58bf96fd7cbr2hg Container created 69m Normal Pulled pod/kube-auth-proxy-546bf7cb95-4gjhl Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 3.521s (3.521s including waiting). Image size: 188904677 bytes. 69m Normal Pulled pod/kube-auth-proxy-546bf7cb95-r7gvf Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 3.565s (3.565s including waiting). Image size: 188904677 bytes. 69m Normal Created pod/kube-auth-proxy-546bf7cb95-4gjhl Container created 69m Normal Started pod/kube-auth-proxy-546bf7cb95-r7gvf Container started 69m Normal Created pod/kube-auth-proxy-546bf7cb95-r7gvf Container created 69m Normal Started pod/kube-auth-proxy-546bf7cb95-4gjhl Container started 69m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 69m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 69m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 69m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 69m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 69m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 68m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-8t7ww 68m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 68m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-8t7ww Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 68m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-8t7ww Add eth0 [10.132.0.49/23] from ovn-kubernetes 68m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-8t7ww Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.915s (4.915s including waiting). Image size: 185476477 bytes. 68m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-8t7ww Container created 68m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-8t7ww Container started 68m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-8t7ww Readiness probe failed: HTTP probe failed with statuscode: 503 68m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-8t7ww Readiness probe error: HTTP probe failed with statuscode: 503... 68m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 68m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-x544h 68m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-x544h Stopping container discovery 68m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 68m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 68m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-pnm97 Readiness probe error: Get "http://10.132.0.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 68m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-pnm97 Readiness probe failed: Get "http://10.132.0.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 68m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-pnm97 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 68m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-pnm97 Container started 68m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-pnm97 Container created 68m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 68m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 67m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-pnm97 Startup probe error: Get "http://10.132.0.36:15021/healthz/ready": dial tcp 10.132.0.36:15021: connect: connection refused... 67m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-pnm97 Startup probe failed: Get "http://10.132.0.36:15021/healthz/ready": dial tcp 10.132.0.36:15021: connect: connection refused 67m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 67m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 67m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 67m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 65m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-8f6f87f59 Created pod: maas-default-gateway-openshift-default-8f6f87f59-hkgx7 65m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-8f6f87f59 from 1 to 2 65m Normal Pulling pod/maas-default-gateway-openshift-default-8f6f87f59-hkgx7 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 65m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-hkgx7 Add eth0 [10.133.0.30/23] from ovn-kubernetes 65m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-hkgx7 Container created 65m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-hkgx7 Container started 65m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-hkgx7 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.485s (4.485s including waiting). Image size: 183555333 bytes. 65m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-hkgx7 Startup probe error: Get "http://10.133.0.30:15021/healthz/ready": dial tcp 10.133.0.30:15021: connect: connection refused... 65m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-hkgx7 Startup probe failed: Get "http://10.133.0.30:15021/healthz/ready": dial tcp 10.133.0.30:15021: connect: connection refused 65m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-hkgx7 Startup probe error: Get "http://10.133.0.30:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 65m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-hkgx7 Startup probe failed: Get "http://10.133.0.30:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 65m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-8f6f87f59 Created pod: maas-default-gateway-openshift-default-8f6f87f59-x5f97 65m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-8f6f87f59 Created pod: maas-default-gateway-openshift-default-8f6f87f59-bd4hv 65m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-bd4hv Add eth0 [10.133.0.33/23] from ovn-kubernetes 65m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-x5f97 Add eth0 [10.134.0.39/23] from ovn-kubernetes 65m Normal Pulling pod/maas-default-gateway-openshift-default-8f6f87f59-x5f97 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 65m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-7bb9489676 from 0 to 1 65m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-bd4hv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-bd4hv Container created 65m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-bd4hv Container started 65m Normal SuccessfulCreate replicaset/payload-processing-7bb9489676 Created pod: payload-processing-7bb9489676-2kpl5 65m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-njw5h 65m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 65m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 65m Warning FailedCreate replicaset/payload-processing-7bb9489676 Error creating: pods "payload-processing-7bb9489676-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 65m Normal AddedInterface pod/payload-processing-7bb9489676-2kpl5 Add eth0 [10.132.0.53/23] from ovn-kubernetes 65m Normal Pulling pod/payload-pre-processing-7b98bb678c-njw5h Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 65m Normal Pulling pod/payload-processing-7bb9489676-2kpl5 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 65m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-njw5h Add eth0 [10.132.0.52/23] from ovn-kubernetes 65m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-x5f97 Container started 65m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-x5f97 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.136s (4.136s including waiting). Image size: 183555333 bytes. 65m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-x5f97 Container created 65m Normal Pulled pod/payload-processing-7bb9489676-2kpl5 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.926s (2.926s including waiting). Image size: 125353965 bytes. 65m Normal Pulled pod/payload-pre-processing-7b98bb678c-njw5h Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.06s (3.06s including waiting). Image size: 125353965 bytes. 65m Normal Created pod/payload-pre-processing-7b98bb678c-njw5h Container created 65m Normal Started pod/payload-pre-processing-7b98bb678c-njw5h Container started 65m Normal Created pod/payload-processing-7bb9489676-2kpl5 Container created 65m Normal Started pod/payload-processing-7bb9489676-2kpl5 Container started 65m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-bd4hv Startup probe error: Get "http://10.133.0.33:15021/healthz/ready": dial tcp 10.133.0.33:15021: connect: connection refused... 65m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-bd4hv Startup probe failed: Get "http://10.133.0.33:15021/healthz/ready": dial tcp 10.133.0.33:15021: connect: connection refused 65m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-x5f97 Startup probe error: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused... 65m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-x5f97 Startup probe failed: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused 65m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: All metrics below target 65m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-8f6f87f59 from 4 to 6 65m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-8f6f87f59 Created pod: maas-default-gateway-openshift-default-8f6f87f59-lwhxc 65m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-8f6f87f59 Created pod: maas-default-gateway-openshift-default-8f6f87f59-mkvqm 65m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-lwhxc Container started 65m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-mkvqm Container created 65m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-lwhxc Container created 65m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-lwhxc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-mkvqm Container started 65m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-mkvqm Add eth0 [10.134.0.40/23] from ovn-kubernetes 65m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-lwhxc Add eth0 [10.132.0.54/23] from ovn-kubernetes 65m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-mkvqm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-lwhxc Startup probe failed: Get "http://10.132.0.54:15021/healthz/ready": dial tcp 10.132.0.54:15021: connect: connection refused 64m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-lwhxc Startup probe error: Get "http://10.132.0.54:15021/healthz/ready": dial tcp 10.132.0.54:15021: connect: connection refused... 64m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-mkvqm Startup probe failed: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused 64m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-mkvqm Startup probe error: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused... 64m Normal AddedLabel gateway/e2e-worker-w4-34f1ce Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-34f1ce 64m Normal AddedLabel gateway/e2e-worker-w5-fe01fa Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-fe01fa 64m Normal SuccessfulCreate replicaset/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc Created pod: e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-jqc5p 64m Normal AddedLabel gateway/e2e-worker-w0-46b586 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-46b586 64m Normal ScalingReplicaSet deployment/e2e-worker-w3-986d37-openshift-default Scaled up replica set e2e-worker-w3-986d37-openshift-default-5bc8bd86fc from 0 to 1 64m Normal AddedLabel gateway/e2e-worker-w3-986d37 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-986d37 64m Normal AddedLabel gateway/e2e-worker-w6-35a187 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-35a187 64m Normal ScalingReplicaSet deployment/e2e-worker-w4-34f1ce-openshift-default Scaled up replica set e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf from 0 to 1 64m Normal SuccessfulCreate replicaset/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 Created pod: e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-qc65q 64m Normal AddedLabel gateway/e2e-worker-w1-5c3374 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-5c3374 64m Normal AddedLabel gateway/e2e-worker-w2-27e82b Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-27e82b 64m Normal AddedInterface pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-ggkpj Add eth0 [10.133.0.35/23] from ovn-kubernetes 64m Normal ScalingReplicaSet deployment/e2e-worker-w2-27e82b-openshift-default Scaled up replica set e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 from 0 to 1 64m Normal SuccessfulCreate replicaset/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf Created pod: e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-ggkpj 64m Normal AddedInterface pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-qc65q Add eth0 [10.133.0.36/23] from ovn-kubernetes 64m Normal AddedInterface pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-94tkp Add eth0 [10.132.0.56/23] from ovn-kubernetes 64m Normal SuccessfulCreate replicaset/e2e-worker-w0-46b586-openshift-default-86c598c96c Created pod: e2e-worker-w0-46b586-openshift-default-86c598c96c-8ckgt 64m Normal ScalingReplicaSet deployment/e2e-worker-w6-35a187-openshift-default Scaled up replica set e2e-worker-w6-35a187-openshift-default-5b54c65f75 from 0 to 1 64m Normal AddedInterface pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-jqc5p Add eth0 [10.132.0.55/23] from ovn-kubernetes 64m Normal ScalingReplicaSet deployment/e2e-worker-w0-46b586-openshift-default Scaled up replica set e2e-worker-w0-46b586-openshift-default-86c598c96c from 0 to 1 64m Normal ScalingReplicaSet deployment/e2e-worker-w5-fe01fa-openshift-default Scaled up replica set e2e-worker-w5-fe01fa-openshift-default-564f5c6485 from 0 to 1 64m Normal SuccessfulCreate replicaset/e2e-worker-w5-fe01fa-openshift-default-564f5c6485 Created pod: e2e-worker-w5-fe01fa-openshift-default-564f5c6485-2mn85 64m Normal SuccessfulCreate replicaset/e2e-worker-w6-35a187-openshift-default-5b54c65f75 Created pod: e2e-worker-w6-35a187-openshift-default-5b54c65f75-94tkp 64m Normal AddedInterface pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-2mn85 Add eth0 [10.133.0.38/23] from ovn-kubernetes 64m Normal AddedInterface pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-678tp Add eth0 [10.132.0.57/23] from ovn-kubernetes 64m Normal SuccessfulCreate replicaset/e2e-worker-w1-5c3374-openshift-default-d6ff699c7 Created pod: e2e-worker-w1-5c3374-openshift-default-d6ff699c7-678tp 64m Normal AddedInterface pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-8ckgt Add eth0 [10.133.0.37/23] from ovn-kubernetes 64m Normal ScalingReplicaSet deployment/e2e-worker-w1-5c3374-openshift-default Scaled up replica set e2e-worker-w1-5c3374-openshift-default-d6ff699c7 from 0 to 1 64m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-986d37 Scaled up replica set payload-processing-e2e-worker-w3-986d37-695476d5cd from 0 to 1 64m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-986d37-5d7d64658c Created pod: payload-pre-processing-e2e-worker-w3-986d37-5d7d64658c-h2tgp 64m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-986d37-5d7d64658c-h2tgp Add eth0 [10.132.0.58/23] from ovn-kubernetes 64m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-986d37 Scaled up replica set payload-pre-processing-e2e-worker-w3-986d37-5d7d64658c from 0 to 1 64m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-986d37-695476d5cd Error creating: pods "payload-processing-e2e-worker-w3-986d37-695476d5cd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-986d37: serviceaccount "payload-processing-e2e-worker-w3-986d37" not found 64m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-986d37-695476d5cd Created pod: payload-processing-e2e-worker-w3-986d37-695476d5cd-99x6j 64m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-986d37-5d7d64658c Error creating: pods "payload-pre-processing-e2e-worker-w3-986d37-5d7d64658c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-986d37: serviceaccount "payload-processing-e2e-worker-w3-986d37" not found 64m Normal Started pod/payload-pre-processing-e2e-worker-w3-986d37-5d7d64658c-h2tgp Container started 64m Normal Created pod/payload-processing-e2e-worker-w3-986d37-695476d5cd-99x6j Container created 64m Normal Started pod/payload-processing-e2e-worker-w3-986d37-695476d5cd-99x6j Container started 64m Normal Pulled pod/payload-processing-e2e-worker-w3-986d37-695476d5cd-99x6j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 64m Normal Created pod/payload-pre-processing-e2e-worker-w3-986d37-5d7d64658c-h2tgp Container created 64m Normal AddedInterface pod/payload-processing-e2e-worker-w3-986d37-695476d5cd-99x6j Add eth0 [10.132.0.59/23] from ovn-kubernetes 64m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-986d37-5d7d64658c-h2tgp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 64m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-27e82b-5968f9f875 Error creating: pods "payload-pre-processing-e2e-worker-w2-27e82b-5968f9f875-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-27e82b: serviceaccount "payload-processing-e2e-worker-w2-27e82b" not found 64m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-27e82b-64f94b9586 Error creating: pods "payload-processing-e2e-worker-w2-27e82b-64f94b9586-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-27e82b: serviceaccount "payload-processing-e2e-worker-w2-27e82b" not found 64m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-27e82b Scaled up replica set payload-processing-e2e-worker-w2-27e82b-64f94b9586 from 0 to 1 64m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-27e82b-5968f9f875 Created pod: payload-pre-processing-e2e-worker-w2-27e82b-5968f9f875-t9qz2 64m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-27e82b Scaled up replica set payload-pre-processing-e2e-worker-w2-27e82b-5968f9f875 from 0 to 1 64m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-27e82b-5968f9f875-t9qz2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 64m Normal Started pod/payload-pre-processing-e2e-worker-w2-27e82b-5968f9f875-t9qz2 Container started 64m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-27e82b-64f94b9586 Created pod: payload-processing-e2e-worker-w2-27e82b-64f94b9586-wb68f 64m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-27e82b-5968f9f875-t9qz2 Add eth0 [10.132.0.60/23] from ovn-kubernetes 64m Normal Pulled pod/payload-processing-e2e-worker-w2-27e82b-64f94b9586-wb68f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 64m Normal Created pod/payload-pre-processing-e2e-worker-w2-27e82b-5968f9f875-t9qz2 Container created 64m Normal Started pod/payload-processing-e2e-worker-w2-27e82b-64f94b9586-wb68f Container started 64m Normal Created pod/payload-processing-e2e-worker-w2-27e82b-64f94b9586-wb68f Container created 64m Normal AddedInterface pod/payload-processing-e2e-worker-w2-27e82b-64f94b9586-wb68f Add eth0 [10.132.0.61/23] from ovn-kubernetes 64m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-46b586-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-46b586-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-35a187 Scaled up replica set payload-processing-e2e-worker-w6-35a187-59b76ccb5b from 0 to 1 64m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-35a187-7887c98dff Error creating: pods "payload-pre-processing-e2e-worker-w6-35a187-7887c98dff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-35a187: serviceaccount "payload-processing-e2e-worker-w6-35a187" not found 64m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-35a187 Scaled up replica set payload-pre-processing-e2e-worker-w6-35a187-7887c98dff from 0 to 1 64m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-35a187-59b76ccb5b Created pod: payload-processing-e2e-worker-w6-35a187-59b76ccb5b-25xkv 64m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-35a187-7887c98dff Created pod: payload-pre-processing-e2e-worker-w6-35a187-7887c98dff-5h96p 64m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-35a187-59b76ccb5b Error creating: pods "payload-processing-e2e-worker-w6-35a187-59b76ccb5b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-35a187: serviceaccount "payload-processing-e2e-worker-w6-35a187" not found 64m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-35a187-7887c98dff-5h96p Add eth0 [10.132.0.62/23] from ovn-kubernetes 64m Normal Pulling pod/payload-processing-e2e-worker-w6-35a187-59b76ccb5b-25xkv Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 64m Normal Started pod/payload-pre-processing-e2e-worker-w6-35a187-7887c98dff-5h96p Container started 64m Normal Created pod/payload-pre-processing-e2e-worker-w6-35a187-7887c98dff-5h96p Container created 64m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-35a187-7887c98dff-5h96p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 64m Normal AddedInterface pod/payload-processing-e2e-worker-w6-35a187-59b76ccb5b-25xkv Add eth0 [10.134.0.41/23] from ovn-kubernetes 64m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-fe01fa Scaled up replica set payload-processing-e2e-worker-w5-fe01fa-698f78d94b from 0 to 1 64m Normal Pulled pod/payload-processing-e2e-worker-w6-35a187-59b76ccb5b-25xkv Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.036s (3.036s including waiting). Image size: 125353965 bytes. 64m Normal Created pod/payload-processing-e2e-worker-w6-35a187-59b76ccb5b-25xkv Container created 64m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-fe01fa-576c754f8c Created pod: payload-pre-processing-e2e-worker-w5-fe01fa-576c754f8c-8dm7l 64m Normal Started pod/payload-processing-e2e-worker-w6-35a187-59b76ccb5b-25xkv Container started 64m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-fe01fa-576c754f8c Error creating: pods "payload-pre-processing-e2e-worker-w5-fe01fa-576c754f8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-fe01fa: serviceaccount "payload-processing-e2e-worker-w5-fe01fa" not found 64m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-fe01fa Scaled up replica set payload-pre-processing-e2e-worker-w5-fe01fa-576c754f8c from 0 to 1 64m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-fe01fa-698f78d94b Error creating: pods "payload-processing-e2e-worker-w5-fe01fa-698f78d94b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-fe01fa: serviceaccount "payload-processing-e2e-worker-w5-fe01fa" not found 63m Normal Created pod/payload-pre-processing-e2e-worker-w5-fe01fa-576c754f8c-8dm7l Container created 63m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-fe01fa-698f78d94b Created pod: payload-processing-e2e-worker-w5-fe01fa-698f78d94b-4wmlb 63m Normal Started pod/payload-pre-processing-e2e-worker-w5-fe01fa-576c754f8c-8dm7l Container started 63m Normal Started pod/payload-processing-e2e-worker-w5-fe01fa-698f78d94b-4wmlb Container started 63m Normal Created pod/payload-processing-e2e-worker-w5-fe01fa-698f78d94b-4wmlb Container created 63m Normal Pulled pod/payload-processing-e2e-worker-w5-fe01fa-698f78d94b-4wmlb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/payload-processing-e2e-worker-w5-fe01fa-698f78d94b-4wmlb Add eth0 [10.132.0.63/23] from ovn-kubernetes 63m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-fe01fa-576c754f8c-8dm7l Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-fe01fa-576c754f8c-8dm7l Add eth0 [10.134.0.42/23] from ovn-kubernetes 63m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-34f1ce Scaled up replica set payload-pre-processing-e2e-worker-w4-34f1ce-9b77f84c9 from 0 to 1 63m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-34f1ce Scaled up replica set payload-processing-e2e-worker-w4-34f1ce-56d77c46fd from 0 to 1 63m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-34f1ce-9b77f84c9 Created pod: payload-pre-processing-e2e-worker-w4-34f1ce-9b77f84c9-jfnmg 63m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-34f1ce-56d77c46fd Error creating: pods "payload-processing-e2e-worker-w4-34f1ce-56d77c46fd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-34f1ce: serviceaccount "payload-processing-e2e-worker-w4-34f1ce" not found 63m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-34f1ce-9b77f84c9 Error creating: pods "payload-pre-processing-e2e-worker-w4-34f1ce-9b77f84c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-34f1ce: serviceaccount "payload-processing-e2e-worker-w4-34f1ce" not found 63m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-34f1ce-56d77c46fd Created pod: payload-processing-e2e-worker-w4-34f1ce-56d77c46fd-9wsff 63m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-34f1ce-9b77f84c9-jfnmg Add eth0 [10.134.0.43/23] from ovn-kubernetes 63m Normal Started pod/payload-pre-processing-e2e-worker-w4-34f1ce-9b77f84c9-jfnmg Container started 63m Normal AddedInterface pod/payload-processing-e2e-worker-w4-34f1ce-56d77c46fd-9wsff Add eth0 [10.134.0.44/23] from ovn-kubernetes 63m Normal Pulled pod/payload-processing-e2e-worker-w4-34f1ce-56d77c46fd-9wsff Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal Started pod/payload-processing-e2e-worker-w4-34f1ce-56d77c46fd-9wsff Container started 63m Normal Created pod/payload-processing-e2e-worker-w4-34f1ce-56d77c46fd-9wsff Container created 63m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-34f1ce-9b77f84c9-jfnmg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal Created pod/payload-pre-processing-e2e-worker-w4-34f1ce-9b77f84c9-jfnmg Container created 63m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-46b586-669f464d79 Created pod: payload-pre-processing-e2e-worker-w0-46b586-669f464d79-fsqx4 63m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-46b586 Scaled up replica set payload-pre-processing-e2e-worker-w0-46b586-669f464d79 from 0 to 1 63m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-46b586-556df6d97b Error creating: pods "payload-processing-e2e-worker-w0-46b586-556df6d97b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-46b586: serviceaccount "payload-processing-e2e-worker-w0-46b586" not found 63m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-46b586-669f464d79 Error creating: pods "payload-pre-processing-e2e-worker-w0-46b586-669f464d79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-46b586: serviceaccount "payload-processing-e2e-worker-w0-46b586" not found 63m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-46b586 Scaled up replica set payload-processing-e2e-worker-w0-46b586-556df6d97b from 0 to 1 63m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-46b586-669f464d79-fsqx4 Add eth0 [10.132.0.64/23] from ovn-kubernetes 63m Normal Created pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-qc65q Container created 63m Normal Created pod/payload-pre-processing-e2e-worker-w0-46b586-669f464d79-fsqx4 Container created 63m Normal Pulled pod/payload-processing-e2e-worker-w0-46b586-556df6d97b-dcfpj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal Started pod/payload-pre-processing-e2e-worker-w0-46b586-669f464d79-fsqx4 Container started 63m Normal AddedInterface pod/payload-processing-e2e-worker-w0-46b586-556df6d97b-dcfpj Add eth0 [10.134.0.45/23] from ovn-kubernetes 63m Normal Pulled pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-qc65q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-46b586-556df6d97b Created pod: payload-processing-e2e-worker-w0-46b586-556df6d97b-dcfpj 63m Normal Started pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-qc65q Container started 63m Normal Started pod/payload-processing-e2e-worker-w0-46b586-556df6d97b-dcfpj Container started 63m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-46b586-669f464d79-fsqx4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal Created pod/payload-processing-e2e-worker-w0-46b586-556df6d97b-dcfpj Container created 63m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-5c3374 Scaled up replica set payload-processing-e2e-worker-w1-5c3374-7969595c9d from 0 to 1 63m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-5c3374 Scaled up replica set payload-pre-processing-e2e-worker-w1-5c3374-f56fc6f79 from 0 to 1 63m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-5c3374-f56fc6f79-7b9pp Add eth0 [10.132.0.65/23] from ovn-kubernetes 63m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-5c3374-7969595c9d Created pod: payload-processing-e2e-worker-w1-5c3374-7969595c9d-gzhtd 63m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-5c3374-f56fc6f79 Created pod: payload-pre-processing-e2e-worker-w1-5c3374-f56fc6f79-7b9pp 63m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-5c3374-f56fc6f79 Error creating: pods "payload-pre-processing-e2e-worker-w1-5c3374-f56fc6f79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-5c3374: serviceaccount "payload-processing-e2e-worker-w1-5c3374" not found 63m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-5c3374-7969595c9d Error creating: pods "payload-processing-e2e-worker-w1-5c3374-7969595c9d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-5c3374: serviceaccount "payload-processing-e2e-worker-w1-5c3374" not found 63m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-35a187-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-35a187-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-5c3374-f56fc6f79-7b9pp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/payload-processing-e2e-worker-w1-5c3374-7969595c9d-gzhtd Add eth0 [10.132.0.66/23] from ovn-kubernetes 63m Normal Created pod/payload-pre-processing-e2e-worker-w1-5c3374-f56fc6f79-7b9pp Container created 63m Normal Started pod/payload-pre-processing-e2e-worker-w1-5c3374-f56fc6f79-7b9pp Container started 63m Normal Started pod/payload-processing-e2e-worker-w1-5c3374-7969595c9d-gzhtd Container started 63m Normal Pulled pod/payload-processing-e2e-worker-w1-5c3374-7969595c9d-gzhtd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal Created pod/payload-processing-e2e-worker-w1-5c3374-7969595c9d-gzhtd Container created 63m Warning Unhealthy pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-qc65q Startup probe failed: Get "http://10.133.0.36:15021/healthz/ready": dial tcp 10.133.0.36:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-qc65q Startup probe error: Get "http://10.133.0.36:15021/healthz/ready": dial tcp 10.133.0.36:15021: connect: connection refused... 63m Normal Started pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-ggkpj Container started 63m Normal Pulled pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-ggkpj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Created pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-ggkpj Container created 63m Normal Started pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-678tp Container started 63m Normal Created pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-678tp Container created 63m Normal Pulled pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-678tp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Warning Unhealthy pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-ggkpj Startup probe failed: Get "http://10.133.0.35:15021/healthz/ready": dial tcp 10.133.0.35:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-ggkpj Startup probe error: Get "http://10.133.0.35:15021/healthz/ready": dial tcp 10.133.0.35:15021: connect: connection refused... 63m Warning ProbeError pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-678tp Startup probe error: Get "http://10.132.0.57:15021/healthz/ready": dial tcp 10.132.0.57:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-678tp Startup probe failed: Get "http://10.132.0.57:15021/healthz/ready": dial tcp 10.132.0.57:15021: connect: connection refused 63m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-34f1ce-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-34f1ce-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 63m Normal SuccessfulCreate replicaset/e2e-worker-w6-35a187-openshift-default-5b54c65f75 Created pod: e2e-worker-w6-35a187-openshift-default-5b54c65f75-rb7qp 63m Normal SuccessfulCreate replicaset/e2e-worker-w6-35a187-openshift-default-5b54c65f75 Created pod: e2e-worker-w6-35a187-openshift-default-5b54c65f75-657lr 63m Normal SuccessfulCreate replicaset/e2e-worker-w6-35a187-openshift-default-5b54c65f75 Created pod: e2e-worker-w6-35a187-openshift-default-5b54c65f75-xgpsz 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-35a187-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 63m Normal ScalingReplicaSet deployment/e2e-worker-w6-35a187-openshift-default Scaled up replica set e2e-worker-w6-35a187-openshift-default-5b54c65f75 from 1 to 4 63m Normal Started pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-rb7qp Container started 63m Normal AddedInterface pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-xgpsz Add eth0 [10.132.0.67/23] from ovn-kubernetes 63m Normal Started pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-xgpsz Container started 63m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-5c3374-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Normal Created pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-rb7qp Container created 63m Normal Pulled pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-rb7qp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-rb7qp Add eth0 [10.134.0.47/23] from ovn-kubernetes 63m Normal Created pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-xgpsz Container created 63m Normal Pulled pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-xgpsz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-657lr Add eth0 [10.133.0.45/23] from ovn-kubernetes 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-5c3374-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Normal Pulled pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-657lr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Created pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-657lr Container created 63m Normal Started pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-657lr Container started 63m Warning ProbeError pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-94tkp Readiness probe error: Get "http://10.132.0.56:15021/healthz/ready": dial tcp 10.132.0.56:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-94tkp Readiness probe failed: Get "http://10.132.0.56:15021/healthz/ready": dial tcp 10.132.0.56:15021: connect: connection refused 63m Normal Created pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-94tkp Container created 63m Normal Started pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-94tkp Container started 63m Normal Pulled pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-94tkp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Warning ProbeError pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-2mn85 Readiness probe error: Get "http://10.133.0.38:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 63m Warning Unhealthy pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-2mn85 Readiness probe failed: Get "http://10.133.0.38:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 63m Normal Pulled pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-2mn85 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Warning Unhealthy pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-rb7qp Startup probe failed: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused 63m Warning Unhealthy pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-xgpsz Startup probe failed: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-xgpsz Startup probe error: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused... 63m Warning ProbeError pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-657lr Startup probe error: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused... 63m Normal Started pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-2mn85 Container started 63m Normal Created pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-2mn85 Container created 63m Warning ProbeError pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-rb7qp Startup probe error: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-657lr Startup probe failed: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused 63m Warning Unhealthy pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-94tkp Startup probe failed: Get "http://10.132.0.56:15021/healthz/ready": dial tcp 10.132.0.56:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-94tkp Startup probe error: Get "http://10.132.0.56:15021/healthz/ready": dial tcp 10.132.0.56:15021: connect: connection refused... 63m Warning ProbeError pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-2mn85 Startup probe error: Get "http://10.133.0.38:15021/healthz/ready": dial tcp 10.133.0.38:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-2mn85 Startup probe failed: Get "http://10.133.0.38:15021/healthz/ready": dial tcp 10.133.0.38:15021: connect: connection refused 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-27e82b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-27e82b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-35a187-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-35a187-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 63m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-5c3374-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 63m Normal AddedInterface pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-tgc8q Add eth0 [10.132.0.68/23] from ovn-kubernetes 63m Normal Started pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-jqc5p Container started 63m Normal Created pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-jqc5p Container created 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-46b586-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 63m Normal ScalingReplicaSet deployment/e2e-worker-w0-46b586-openshift-default Scaled up replica set e2e-worker-w0-46b586-openshift-default-86c598c96c from 1 to 3 63m Normal Pulled pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-jqc5p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal SuccessfulCreate replicaset/e2e-worker-w0-46b586-openshift-default-86c598c96c Created pod: e2e-worker-w0-46b586-openshift-default-86c598c96c-6hww6 63m Normal SuccessfulCreate replicaset/e2e-worker-w0-46b586-openshift-default-86c598c96c Created pod: e2e-worker-w0-46b586-openshift-default-86c598c96c-tgc8q 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-5c3374-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 63m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-fe01fa-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-fe01fa-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Normal Pulled pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-8ckgt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-6hww6 Add eth0 [10.134.0.49/23] from ovn-kubernetes 63m Normal Started pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-8ckgt Container started 63m Normal Created pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-8ckgt Container created 63m Normal Created pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-tgc8q Container created 63m Normal Pulled pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-tgc8q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Started pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-tgc8q Container started 63m Normal Pulled pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-6hww6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Created pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-6hww6 Container created 63m Normal Started pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-6hww6 Container started 63m Warning Unhealthy pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-8ckgt Startup probe failed: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-jqc5p Startup probe error: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-jqc5p Startup probe failed: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-8ckgt Startup probe error: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-tgc8q Startup probe failed: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-6hww6 Startup probe error: Get "http://10.134.0.49:15021/healthz/ready": dial tcp 10.134.0.49:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-6hww6 Startup probe failed: Get "http://10.134.0.49:15021/healthz/ready": dial tcp 10.134.0.49:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-tgc8q Startup probe error: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused... 63m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-34f1ce-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-34f1ce-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Normal ScalingReplicaSet deployment/e2e-worker-w6-35a187-openshift-default Scaled up replica set e2e-worker-w6-35a187-openshift-default-5b54c65f75 from 4 to 5 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-35a187-openshift-default New size: 5; reason: 63m Normal SuccessfulCreate replicaset/e2e-worker-w6-35a187-openshift-default-5b54c65f75 Created pod: e2e-worker-w6-35a187-openshift-default-5b54c65f75-bsqhk 63m Normal AddedInterface pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-bsqhk Add eth0 [10.134.0.50/23] from ovn-kubernetes 63m Normal Started pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-bsqhk Container started 63m Normal Created pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-bsqhk Container created 63m Normal Pulled pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-bsqhk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-fe01fa-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 63m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-fe01fa-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 62m Warning ProbeError pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-bsqhk Startup probe error: Get "http://10.134.0.50:15021/healthz/ready": dial tcp 10.134.0.50:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-bsqhk Startup probe failed: Get "http://10.134.0.50:15021/healthz/ready": dial tcp 10.134.0.50:15021: connect: connection refused 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-986d37-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-986d37-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-46b586-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 62m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-46b586-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Normal SuccessfulCreate replicaset/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 Created pod: e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-7gm84 60m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-27e82b-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 60m Normal ScalingReplicaSet deployment/e2e-worker-w2-27e82b-openshift-default Scaled up replica set e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 from 1 to 2 60m Normal AddedInterface pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-7gm84 Add eth0 [10.132.0.72/23] from ovn-kubernetes 60m Normal Pulled pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-7gm84 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Started pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-7gm84 Container started 60m Normal Created pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-7gm84 Container created 60m Warning Unhealthy pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-7gm84 Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-7gm84 Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused... 60m Normal Killing pod/maas-default-gateway-openshift-default-8f6f87f59-pnm97 Stopping container istio-proxy 60m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-8f6f87f59 from 6 to 3 60m Normal Killing pod/maas-default-gateway-openshift-default-8f6f87f59-mkvqm Stopping container istio-proxy 60m Normal Killing pod/maas-default-gateway-openshift-default-8f6f87f59-x5f97 Stopping container istio-proxy 60m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-8f6f87f59 Deleted pod: maas-default-gateway-openshift-default-8f6f87f59-pnm97 60m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-8f6f87f59 Deleted pod: maas-default-gateway-openshift-default-8f6f87f59-x5f97 60m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-8f6f87f59 Deleted pod: maas-default-gateway-openshift-default-8f6f87f59-mkvqm 60m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-pnm97 Readiness probe failed: HTTP probe failed with statuscode: 503 60m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-pnm97 Readiness probe error: HTTP probe failed with statuscode: 503... 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-27e82b-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 59m Normal ScalingReplicaSet deployment/e2e-worker-w2-27e82b-openshift-default Scaled up replica set e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 from 2 to 3 59m Normal SuccessfulCreate replicaset/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 Created pod: e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-z6jt4 59m Normal AddedInterface pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-z6jt4 Add eth0 [10.134.0.56/23] from ovn-kubernetes 59m Normal Started pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-z6jt4 Container started 59m Normal Created pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-z6jt4 Container created 59m Normal Pulled pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-z6jt4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Warning Unhealthy pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-z6jt4 Startup probe failed: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-z6jt4 Startup probe error: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused... 59m Normal Created pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-mgtkw Container created 59m Normal Pulled pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-mgtkw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal ScalingReplicaSet deployment/e2e-worker-w3-986d37-openshift-default Scaled up replica set e2e-worker-w3-986d37-openshift-default-5bc8bd86fc from 1 to 2 59m Normal AddedInterface pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-mgtkw Add eth0 [10.134.0.57/23] from ovn-kubernetes 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-986d37-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 59m Normal SuccessfulCreate replicaset/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc Created pod: e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-mgtkw 59m Normal Started pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-mgtkw Container started 59m Warning ProbeError pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-mgtkw Startup probe error: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-mgtkw Startup probe failed: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-34f1ce-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 58m Normal SuccessfulCreate replicaset/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf Created pod: e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-6gwbm 58m Normal ScalingReplicaSet deployment/e2e-worker-w4-34f1ce-openshift-default Scaled up replica set e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf from 1 to 2 58m Normal AddedInterface pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-6gwbm Add eth0 [10.132.0.88/23] from ovn-kubernetes 58m Normal Created pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-6gwbm Container created 58m Normal Pulled pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-6gwbm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal SuccessfulCreate replicaset/e2e-worker-w5-fe01fa-openshift-default-564f5c6485 Created pod: e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t762r 58m Normal ScalingReplicaSet deployment/e2e-worker-w5-fe01fa-openshift-default Scaled up replica set e2e-worker-w5-fe01fa-openshift-default-564f5c6485 from 1 to 2 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-fe01fa-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 58m Normal Started pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-6gwbm Container started 58m Normal AddedInterface pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t762r Add eth0 [10.132.0.89/23] from ovn-kubernetes 58m Normal Created pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t762r Container created 58m Normal Started pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t762r Container started 58m Normal Pulled pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t762r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-q6mtd Container started 58m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 58m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-8f6f87f59 Created pod: maas-default-gateway-openshift-default-8f6f87f59-cxzjj 58m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-8f6f87f59 Created pod: maas-default-gateway-openshift-default-8f6f87f59-944gh 58m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-8f6f87f59 Created pod: maas-default-gateway-openshift-default-8f6f87f59-q6mtd 58m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-q6mtd Add eth0 [10.134.0.60/23] from ovn-kubernetes 58m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-q6mtd Container created 58m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-q6mtd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-8f6f87f59 from 3 to 6 58m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-cxzjj Add eth0 [10.134.0.61/23] from ovn-kubernetes 58m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-944gh Add eth0 [10.132.0.91/23] from ovn-kubernetes 58m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-cxzjj Container created 58m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-944gh Container started 58m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-944gh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-cxzjj Container started 58m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-944gh Container created 58m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-cxzjj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Warning Unhealthy pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-6gwbm Startup probe failed: Get "http://10.132.0.88:15021/healthz/ready": dial tcp 10.132.0.88:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-6gwbm Startup probe error: Get "http://10.132.0.88:15021/healthz/ready": dial tcp 10.132.0.88:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t762r Startup probe failed: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t762r Startup probe error: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused... 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-q6mtd Startup probe failed: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused 58m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-cxzjj Startup probe error: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused... 58m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-q6mtd Startup probe error: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused... 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-cxzjj Startup probe failed: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused 58m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-944gh Startup probe error: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused... 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-944gh Startup probe failed: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused 58m Normal Started pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-nr4qz Container started 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-35a187-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 58m Normal Pulled pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-nr4qz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal SuccessfulCreate replicaset/e2e-worker-w6-35a187-openshift-default-5b54c65f75 Created pod: e2e-worker-w6-35a187-openshift-default-5b54c65f75-txzbm 58m Normal ScalingReplicaSet deployment/e2e-worker-w6-35a187-openshift-default Scaled up replica set e2e-worker-w6-35a187-openshift-default-5b54c65f75 from 5 to 7 58m Normal AddedInterface pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-nr4qz Add eth0 [10.132.0.93/23] from ovn-kubernetes 58m Normal SuccessfulCreate replicaset/e2e-worker-w6-35a187-openshift-default-5b54c65f75 Created pod: e2e-worker-w6-35a187-openshift-default-5b54c65f75-nr4qz 58m Normal Created pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-nr4qz Container created 58m Normal Pulled pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-txzbm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-txzbm Container created 58m Normal Started pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-txzbm Container started 58m Normal AddedInterface pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-txzbm Add eth0 [10.133.0.52/23] from ovn-kubernetes 58m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-xsgv7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-m59d5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-m59d5 Container created 58m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-m59d5 Add eth0 [10.134.0.62/23] from ovn-kubernetes 58m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-m59d5 Container started 58m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-wftsq Add eth0 [10.132.0.95/23] from ovn-kubernetes 58m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-8f6f87f59 from 6 to 10 58m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-xsgv7 Add eth0 [10.132.0.94/23] from ovn-kubernetes 58m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: 58m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-wftsq Container started 58m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-wftsq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-vt4mg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-xsgv7 Container created 58m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-xsgv7 Container started 58m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-vt4mg Container started 58m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-wftsq Container created 58m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-vt4mg Container created 58m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-vt4mg Add eth0 [10.133.0.53/23] from ovn-kubernetes 58m Warning ProbeError pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-txzbm Startup probe error: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-txzbm Startup probe failed: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused 58m Warning Unhealthy pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-nr4qz Startup probe failed: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-nr4qz Startup probe error: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused... 58m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-m59d5 Startup probe error: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused... 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-m59d5 Startup probe failed: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-vt4mg Startup probe failed: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused 58m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-vt4mg Startup probe error: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused... 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-wftsq Startup probe failed: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-xsgv7 Startup probe failed: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused 58m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-xsgv7 Startup probe error: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused... 58m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-wftsq Startup probe error: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused... 58m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-wftsq Startup probe error: Get "http://10.132.0.95:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-xsgv7 Startup probe failed: Get "http://10.132.0.94:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-wftsq Startup probe failed: Get "http://10.132.0.95:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-xsgv7 Startup probe error: Get "http://10.132.0.94:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Normal SuccessfulCreate replicaset/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 Created pod: e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-2qnns 58m Normal ScalingReplicaSet deployment/e2e-worker-w2-27e82b-openshift-default Scaled up replica set e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 from 3 to 4 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-27e82b-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 58m Normal SuccessfulCreate replicaset/e2e-worker-w0-46b586-openshift-default-86c598c96c Created pod: e2e-worker-w0-46b586-openshift-default-86c598c96c-ccdzj 58m Normal SuccessfulCreate replicaset/e2e-worker-w0-46b586-openshift-default-86c598c96c Created pod: e2e-worker-w0-46b586-openshift-default-86c598c96c-f6k98 58m Normal AddedInterface pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-2qnns Add eth0 [10.132.0.98/23] from ovn-kubernetes 58m Normal SuccessfulCreate replicaset/e2e-worker-w0-46b586-openshift-default-86c598c96c Created pod: e2e-worker-w0-46b586-openshift-default-86c598c96c-q6bx8 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-46b586-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 58m Normal ScalingReplicaSet deployment/e2e-worker-w0-46b586-openshift-default Scaled up replica set e2e-worker-w0-46b586-openshift-default-86c598c96c from 3 to 6 58m Normal AddedInterface pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-q6bx8 Add eth0 [10.133.0.54/23] from ovn-kubernetes 58m Normal Pulled pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-ccdzj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Started pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-f6k98 Container started 58m Normal Created pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-q6bx8 Container created 58m Normal Pulled pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-f6k98 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-ccdzj Add eth0 [10.132.0.99/23] from ovn-kubernetes 58m Normal Created pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-f6k98 Container created 58m Normal Started pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-2qnns Container started 58m Normal Pulled pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-q6bx8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Started pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-ccdzj Container started 58m Normal Created pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-ccdzj Container created 58m Normal Created pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-2qnns Container created 58m Normal Pulled pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-2qnns Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Started pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-q6bx8 Container started 58m Normal AddedInterface pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-f6k98 Add eth0 [10.134.0.64/23] from ovn-kubernetes 57m Warning Unhealthy pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-2qnns Startup probe failed: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-q6bx8 Startup probe error: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-q6bx8 Startup probe failed: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-2qnns Startup probe error: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-ccdzj Startup probe error: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-f6k98 Startup probe error: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-ccdzj Startup probe failed: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-f6k98 Startup probe failed: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-34f1ce-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 57m Normal SuccessfulCreate replicaset/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf Created pod: e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-hnqpt 57m Normal ScalingReplicaSet deployment/e2e-worker-w4-34f1ce-openshift-default Scaled up replica set e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf from 2 to 3 57m Normal Started pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-hnqpt Container started 57m Normal Pulled pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-hnqpt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-hnqpt Container created 57m Normal AddedInterface pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-hnqpt Add eth0 [10.134.0.66/23] from ovn-kubernetes 57m Warning ProbeError pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-hnqpt Startup probe error: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-hnqpt Startup probe failed: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-986d37-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 57m Normal ScalingReplicaSet deployment/e2e-worker-w3-986d37-openshift-default Scaled up replica set e2e-worker-w3-986d37-openshift-default-5bc8bd86fc from 2 to 3 57m Normal SuccessfulCreate replicaset/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc Created pod: e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-hrh5g 57m Normal Started pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-hrh5g Container started 57m Normal Created pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-hrh5g Container created 57m Normal Pulled pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-hrh5g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-hrh5g Add eth0 [10.133.0.58/23] from ovn-kubernetes 57m Normal SuccessfulCreate replicaset/e2e-worker-w1-5c3374-openshift-default-d6ff699c7 Created pod: e2e-worker-w1-5c3374-openshift-default-d6ff699c7-84bch 57m Normal AddedInterface pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8hzf8 Add eth0 [10.134.0.67/23] from ovn-kubernetes 57m Normal ScalingReplicaSet deployment/e2e-worker-w5-fe01fa-openshift-default Scaled up replica set e2e-worker-w5-fe01fa-openshift-default-564f5c6485 from 2 to 4 57m Normal SuccessfulCreate replicaset/e2e-worker-w5-fe01fa-openshift-default-564f5c6485 Created pod: e2e-worker-w5-fe01fa-openshift-default-564f5c6485-d44pq 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-fe01fa-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 57m Normal SuccessfulCreate replicaset/e2e-worker-w5-fe01fa-openshift-default-564f5c6485 Created pod: e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8hzf8 57m Normal Created pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-84bch Container created 57m Normal Started pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-84bch Container started 57m Normal AddedInterface pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-84bch Add eth0 [10.133.0.59/23] from ovn-kubernetes 57m Normal Created pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-d44pq Container created 57m Normal Started pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-d44pq Container started 57m Normal Pulled pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-84bch Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8hzf8 Container created 57m Normal Pulled pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-d44pq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Pulled pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8hzf8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Started pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8hzf8 Container started 57m Normal AddedInterface pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-d44pq Add eth0 [10.132.0.100/23] from ovn-kubernetes 57m Warning ProbeError pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-hrh5g Startup probe error: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-hrh5g Startup probe failed: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-d44pq Startup probe error: Get "http://10.132.0.100:15021/healthz/ready": dial tcp 10.132.0.100:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8hzf8 Startup probe failed: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8hzf8 Startup probe error: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-d44pq Startup probe failed: Get "http://10.132.0.100:15021/healthz/ready": dial tcp 10.132.0.100:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-84bch Startup probe failed: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-84bch Startup probe error: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused... 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-35a187-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 56m Normal SuccessfulCreate replicaset/e2e-worker-w6-35a187-openshift-default-5b54c65f75 Created pod: e2e-worker-w6-35a187-openshift-default-5b54c65f75-4jrqx 56m Normal ScalingReplicaSet deployment/e2e-worker-w6-35a187-openshift-default Scaled up replica set e2e-worker-w6-35a187-openshift-default-5b54c65f75 from 7 to 10 56m Normal SuccessfulCreate replicaset/e2e-worker-w6-35a187-openshift-default-5b54c65f75 (combined from similar events): Created pod: e2e-worker-w6-35a187-openshift-default-5b54c65f75-klh6t 56m Normal SuccessfulCreate replicaset/e2e-worker-w6-35a187-openshift-default-5b54c65f75 Created pod: e2e-worker-w6-35a187-openshift-default-5b54c65f75-qsdr8 56m Normal Started pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-klh6t Container started 56m Normal AddedInterface pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-4jrqx Add eth0 [10.134.0.68/23] from ovn-kubernetes 56m Normal Created pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-klh6t Container created 56m Normal Pulled pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-klh6t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Pulled pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-qsdr8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-klh6t Add eth0 [10.132.0.102/23] from ovn-kubernetes 56m Normal AddedInterface pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-qsdr8 Add eth0 [10.133.0.60/23] from ovn-kubernetes 56m Normal Created pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-qsdr8 Container created 56m Normal Started pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-4jrqx Container started 56m Normal Pulled pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-4jrqx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-4jrqx Container created 56m Normal Started pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-qsdr8 Container started 56m Warning ProbeError pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-qsdr8 Startup probe error: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-qsdr8 Startup probe failed: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused 56m Warning Unhealthy pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-4jrqx Startup probe failed: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-4jrqx Startup probe error: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused... 56m Warning ProbeError pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-klh6t Startup probe error: Get "http://10.132.0.102:15021/healthz/ready": dial tcp 10.132.0.102:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-klh6t Startup probe failed: Get "http://10.132.0.102:15021/healthz/ready": dial tcp 10.132.0.102:15021: connect: connection refused 56m Normal SuccessfulCreate replicaset/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 Created pod: e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-4dqrc 56m Normal ScalingReplicaSet deployment/e2e-worker-w2-27e82b-openshift-default Scaled up replica set e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 from 4 to 6 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-27e82b-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 56m Normal SuccessfulCreate replicaset/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 Created pod: e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-n2wcf 56m Normal AddedInterface pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-4dqrc Add eth0 [10.133.0.61/23] from ovn-kubernetes 56m Normal AddedInterface pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-n2wcf Add eth0 [10.134.0.70/23] from ovn-kubernetes 56m Normal Pulled pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-4dqrc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-n2wcf Container started 56m Normal Created pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-n2wcf Container created 56m Normal Started pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-4dqrc Container started 56m Normal Created pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-4dqrc Container created 56m Normal Pulled pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-n2wcf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Warning ProbeError pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-n2wcf Startup probe error: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-4dqrc Startup probe failed: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-4dqrc Startup probe error: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-n2wcf Startup probe failed: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused 56m Normal ScalingReplicaSet deployment/e2e-worker-w0-46b586-openshift-default Scaled up replica set e2e-worker-w0-46b586-openshift-default-86c598c96c from 6 to 10 56m Normal SuccessfulCreate replicaset/e2e-worker-w0-46b586-openshift-default-86c598c96c (combined from similar events): Created pod: e2e-worker-w0-46b586-openshift-default-86c598c96c-jdcvx 56m Normal SuccessfulCreate replicaset/e2e-worker-w0-46b586-openshift-default-86c598c96c Created pod: e2e-worker-w0-46b586-openshift-default-86c598c96c-4jhn8 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-46b586-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 56m Normal SuccessfulCreate replicaset/e2e-worker-w0-46b586-openshift-default-86c598c96c Created pod: e2e-worker-w0-46b586-openshift-default-86c598c96c-kzqr4 56m Normal SuccessfulCreate replicaset/e2e-worker-w0-46b586-openshift-default-86c598c96c Created pod: e2e-worker-w0-46b586-openshift-default-86c598c96c-thfjt 56m Normal Created pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-kzqr4 Container created 56m Normal Pulled pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-kzqr4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Pulled pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-jdcvx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-thfjt Container started 56m Normal Pulled pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-thfjt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Pulled pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-4jhn8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-jdcvx Add eth0 [10.132.0.104/23] from ovn-kubernetes 56m Normal Created pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-thfjt Container created 56m Normal Created pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-4jhn8 Container created 56m Normal Started pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-4jhn8 Container started 56m Normal AddedInterface pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-kzqr4 Add eth0 [10.132.0.103/23] from ovn-kubernetes 56m Normal AddedInterface pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-thfjt Add eth0 [10.134.0.71/23] from ovn-kubernetes 56m Normal Started pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-kzqr4 Container started 56m Normal AddedInterface pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-4jhn8 Add eth0 [10.133.0.62/23] from ovn-kubernetes 56m Normal Started pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-jdcvx Container started 56m Normal Created pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-jdcvx Container created 56m Warning Unhealthy pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-thfjt Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-4jhn8 Startup probe error: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-4jhn8 Startup probe failed: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-thfjt Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused... 56m Warning ProbeError pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-kzqr4 Startup probe error: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-kzqr4 Startup probe failed: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-jdcvx Startup probe error: Get "http://10.132.0.104:15021/healthz/ready": dial tcp 10.132.0.104:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-jdcvx Startup probe failed: Get "http://10.132.0.104:15021/healthz/ready": dial tcp 10.132.0.104:15021: connect: connection refused 54m Normal ScalingReplicaSet deployment/e2e-worker-w3-986d37-openshift-default Scaled up replica set e2e-worker-w3-986d37-openshift-default-5bc8bd86fc from 3 to 5 54m Normal Pulled pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-px9v2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-q7dtb Add eth0 [10.132.0.115/23] from ovn-kubernetes 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-986d37-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 54m Normal AddedInterface pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-px9v2 Add eth0 [10.133.0.65/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-q7dtb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal SuccessfulCreate replicaset/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc Created pod: e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-px9v2 54m Normal SuccessfulCreate replicaset/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc Created pod: e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-q7dtb 54m Normal Started pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-px9v2 Container started 54m Normal Created pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-px9v2 Container created 54m Normal Started pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-q7dtb Container started 54m Normal Created pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-q7dtb Container created 54m Warning Unhealthy pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-px9v2 Startup probe failed: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-px9v2 Startup probe error: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-q7dtb Startup probe failed: Get "http://10.132.0.115:15021/healthz/ready": dial tcp 10.132.0.115:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-q7dtb Startup probe error: Get "http://10.132.0.115:15021/healthz/ready": dial tcp 10.132.0.115:15021: connect: connection refused... 54m Normal SuccessfulCreate replicaset/e2e-worker-w5-fe01fa-openshift-default-564f5c6485 Created pod: e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8qtdz 54m Normal ScalingReplicaSet deployment/e2e-worker-w5-fe01fa-openshift-default Scaled up replica set e2e-worker-w5-fe01fa-openshift-default-564f5c6485 from 4 to 6 54m Normal SuccessfulCreate replicaset/e2e-worker-w1-5c3374-openshift-default-d6ff699c7 Created pod: e2e-worker-w1-5c3374-openshift-default-d6ff699c7-j2j46 54m Normal SuccessfulCreate replicaset/e2e-worker-w1-5c3374-openshift-default-d6ff699c7 Created pod: e2e-worker-w1-5c3374-openshift-default-d6ff699c7-n4hsl 54m Normal ScalingReplicaSet deployment/e2e-worker-w1-5c3374-openshift-default Scaled up replica set e2e-worker-w1-5c3374-openshift-default-d6ff699c7 from 2 to 4 54m Normal SuccessfulCreate replicaset/e2e-worker-w5-fe01fa-openshift-default-564f5c6485 Created pod: e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t5hkt 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-5c3374-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-fe01fa-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 54m Normal Created pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-j2j46 Container created 54m Normal AddedInterface pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-j2j46 Add eth0 [10.132.0.119/23] from ovn-kubernetes 54m Normal AddedInterface pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t5hkt Add eth0 [10.133.0.66/23] from ovn-kubernetes 54m Normal Started pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-n4hsl Container started 54m Normal Created pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-n4hsl Container created 54m Normal AddedInterface pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8qtdz Add eth0 [10.134.0.78/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-n4hsl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-n4hsl Add eth0 [10.134.0.77/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t5hkt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Started pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8qtdz Container started 54m Normal Started pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-j2j46 Container started 54m Normal Started pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t5hkt Container started 54m Normal Pulled pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-j2j46 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t5hkt Container created 54m Normal Created pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8qtdz Container created 54m Normal Pulled pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8qtdz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Warning Unhealthy pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t5hkt Startup probe failed: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t5hkt Startup probe error: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused... 54m Warning ProbeError pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-n4hsl Startup probe error: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8qtdz Startup probe failed: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused 54m Warning Unhealthy pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-n4hsl Startup probe failed: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused 54m Warning Unhealthy pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-j2j46 Startup probe failed: Get "http://10.132.0.119:15021/healthz/ready": dial tcp 10.132.0.119:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-j2j46 Startup probe error: Get "http://10.132.0.119:15021/healthz/ready": dial tcp 10.132.0.119:15021: connect: connection refused... 54m Warning ProbeError pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8qtdz Startup probe error: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused... 54m Normal SuccessfulCreate replicaset/e2e-worker-w1-5c3374-openshift-default-d6ff699c7 Created pod: e2e-worker-w1-5c3374-openshift-default-d6ff699c7-ppwlm 54m Normal SuccessfulCreate replicaset/e2e-worker-w1-5c3374-openshift-default-d6ff699c7 Created pod: e2e-worker-w1-5c3374-openshift-default-d6ff699c7-bn695 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-5c3374-openshift-default New size: 6; reason: All metrics below target 54m Normal ScalingReplicaSet deployment/e2e-worker-w1-5c3374-openshift-default Scaled up replica set e2e-worker-w1-5c3374-openshift-default-d6ff699c7 from 4 to 6 54m Normal Started pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-bn695 Container started 54m Normal Created pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-bn695 Container created 54m Normal Pulled pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-bn695 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-ppwlm Add eth0 [10.134.0.79/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-ppwlm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-ppwlm Container created 54m Normal Started pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-ppwlm Container started 54m Normal AddedInterface pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-bn695 Add eth0 [10.133.0.67/23] from ovn-kubernetes 54m Warning Unhealthy pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-bn695 Startup probe failed: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-ppwlm Startup probe error: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-ppwlm Startup probe failed: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-bn695 Startup probe error: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused... 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-27e82b-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 53m Normal SuccessfulCreate replicaset/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 Created pod: e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-khcqd 53m Normal SuccessfulCreate replicaset/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 Created pod: e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-x59pr 53m Normal SuccessfulCreate replicaset/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 Created pod: e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-dxpzb 53m Normal ScalingReplicaSet deployment/e2e-worker-w2-27e82b-openshift-default Scaled up replica set e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 from 6 to 9 53m Normal Created pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-dxpzb Container created 53m Normal AddedInterface pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-dxpzb Add eth0 [10.133.0.68/23] from ovn-kubernetes 53m Normal Killing pod/maas-default-gateway-openshift-default-8f6f87f59-wftsq Stopping container istio-proxy 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-fe01fa-openshift-default-564f5c6485 Created pod: e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8bt76 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-fe01fa-openshift-default-564f5c6485 Created pod: e2e-worker-w5-fe01fa-openshift-default-564f5c6485-w6bmh 53m Normal Created pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-x59pr Container created 53m Normal Pulled pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-x59pr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-x59pr Add eth0 [10.134.0.81/23] from ovn-kubernetes 53m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-8f6f87f59 Deleted pod: maas-default-gateway-openshift-default-8f6f87f59-944gh 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-fe01fa-openshift-default-564f5c6485 Created pod: e2e-worker-w5-fe01fa-openshift-default-564f5c6485-qxg67 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-fe01fa-openshift-default-564f5c6485 (combined from similar events): Created pod: e2e-worker-w5-fe01fa-openshift-default-564f5c6485-xkbnw 53m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-8f6f87f59 Deleted pod: maas-default-gateway-openshift-default-8f6f87f59-wftsq 53m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-8f6f87f59 Deleted pod: maas-default-gateway-openshift-default-8f6f87f59-xsgv7 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-fe01fa-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w5-fe01fa-openshift-default Scaled up replica set e2e-worker-w5-fe01fa-openshift-default-564f5c6485 from 6 to 10 53m Normal Started pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-khcqd Container started 53m Normal Killing pod/maas-default-gateway-openshift-default-8f6f87f59-944gh Stopping container istio-proxy 53m Normal Created pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-khcqd Container created 53m Normal Pulled pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-dxpzb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-x59pr Container started 53m Normal Started pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-dxpzb Container started 53m Normal Pulled pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-khcqd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-khcqd Add eth0 [10.132.0.127/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-w6bmh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Pulled pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8bt76 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Killing pod/maas-default-gateway-openshift-default-8f6f87f59-xsgv7 Stopping container istio-proxy 53m Normal AddedInterface pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-xkbnw Add eth0 [10.132.0.129/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-xkbnw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-w6bmh Container created 53m Normal Started pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-w6bmh Container started 53m Normal AddedInterface pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-w6bmh Add eth0 [10.133.0.69/23] from ovn-kubernetes 53m Normal Started pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8bt76 Container started 53m Normal AddedInterface pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-qxg67 Add eth0 [10.134.0.82/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-qxg67 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8bt76 Container created 53m Normal AddedInterface pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8bt76 Add eth0 [10.132.0.128/23] from ovn-kubernetes 53m Normal Started pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-qxg67 Container started 53m Normal Created pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-qxg67 Container created 53m Normal Started pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-xkbnw Container started 53m Normal Created pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-xkbnw Container created 53m Warning ProbeError pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-x59pr Startup probe error: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-dxpzb Startup probe failed: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-dxpzb Startup probe error: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-x59pr Startup probe failed: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-w6bmh Startup probe failed: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-w6bmh Startup probe error: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-khcqd Startup probe error: Get "http://10.132.0.127:15021/healthz/ready": dial tcp 10.132.0.127:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-khcqd Startup probe failed: Get "http://10.132.0.127:15021/healthz/ready": dial tcp 10.132.0.127:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-qxg67 Startup probe failed: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-qxg67 Startup probe error: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-xkbnw Startup probe error: Get "http://10.132.0.129:15021/healthz/ready": dial tcp 10.132.0.129:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8bt76 Startup probe error: Get "http://10.132.0.128:15021/healthz/ready": dial tcp 10.132.0.128:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-xkbnw Startup probe failed: Get "http://10.132.0.129:15021/healthz/ready": dial tcp 10.132.0.129:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8bt76 Startup probe failed: Get "http://10.132.0.128:15021/healthz/ready": dial tcp 10.132.0.128:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-w6bmh Startup probe failed: Get "http://10.133.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 53m Warning Unhealthy pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-khcqd Startup probe failed: Get "http://10.132.0.127:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 53m Warning ProbeError pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-qxg67 Startup probe error: Get "http://10.134.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 53m Warning Unhealthy pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-qxg67 Startup probe failed: Get "http://10.134.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 53m Warning ProbeError pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-w6bmh Startup probe error: Get "http://10.133.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 53m Warning Unhealthy pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8bt76 Startup probe failed: Get "http://10.132.0.128:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 53m Warning ProbeError pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-khcqd Startup probe error: Get "http://10.132.0.127:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 53m Warning ProbeError pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8bt76 Startup probe error: Get "http://10.132.0.128:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 53m Warning Unhealthy pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-xkbnw Startup probe failed: Get "http://10.132.0.129:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 53m Warning ProbeError pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8bt76 Startup probe error: HTTP probe failed with statuscode: 503... 53m Warning ProbeError pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-xkbnw Startup probe error: Get "http://10.132.0.129:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 53m Normal ScalingReplicaSet deployment/e2e-worker-w4-34f1ce-openshift-default Scaled up replica set e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf from 3 to 5 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-34f1ce-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 53m Normal SuccessfulCreate replicaset/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc Created pod: e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-4strh 53m Normal SuccessfulCreate replicaset/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf Created pod: e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-7w9dq 53m Normal SuccessfulCreate replicaset/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf Created pod: e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-smvsh 53m Normal ScalingReplicaSet deployment/e2e-worker-w3-986d37-openshift-default Scaled up replica set e2e-worker-w3-986d37-openshift-default-5bc8bd86fc from 5 to 6 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-986d37-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 53m Normal Created pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-4strh Container created 53m Normal Pulled pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-4strh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal SuccessfulCreate replicaset/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 (combined from similar events): Created pod: e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-vb9r5 53m Normal Created pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-smvsh Container created 53m Normal Pulled pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-smvsh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-vb9r5 Add eth0 [10.132.0.137/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-4strh Add eth0 [10.134.0.84/23] from ovn-kubernetes 53m Normal Started pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-7w9dq Container started 53m Normal Pulled pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-vb9r5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-smvsh Add eth0 [10.133.0.70/23] from ovn-kubernetes 53m Normal ScalingReplicaSet deployment/e2e-worker-w2-27e82b-openshift-default Scaled up replica set e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 from 9 to 10 53m Normal Started pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-4strh Container started 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-27e82b-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 53m Normal Created pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-7w9dq Container created 53m Normal Pulled pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-7w9dq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-7w9dq Add eth0 [10.132.0.136/23] from ovn-kubernetes 53m Normal Started pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-smvsh Container started 53m Normal Created pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-vb9r5 Container created 53m Normal Started pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-vb9r5 Container started 53m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-8f6f87f59 Deleted pod: maas-default-gateway-openshift-default-8f6f87f59-vt4mg 53m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-8f6f87f59 Deleted pod: maas-default-gateway-openshift-default-8f6f87f59-q6mtd 53m Normal Killing pod/maas-default-gateway-openshift-default-8f6f87f59-q6mtd Stopping container istio-proxy 53m Normal Killing pod/maas-default-gateway-openshift-default-8f6f87f59-m59d5 Stopping container istio-proxy 53m Normal Killing pod/maas-default-gateway-openshift-default-8f6f87f59-vt4mg Stopping container istio-proxy 53m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-8f6f87f59 Deleted pod: maas-default-gateway-openshift-default-8f6f87f59-cxzjj 53m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-8f6f87f59 from 7 to 3 53m Normal Killing pod/maas-default-gateway-openshift-default-8f6f87f59-cxzjj Stopping container istio-proxy 53m Warning ProbeError pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-7w9dq Startup probe error: Get "http://10.132.0.136:15021/healthz/ready": dial tcp 10.132.0.136:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-smvsh Startup probe failed: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-7w9dq Startup probe failed: Get "http://10.132.0.136:15021/healthz/ready": dial tcp 10.132.0.136:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-smvsh Startup probe error: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-vb9r5 Startup probe error: Get "http://10.132.0.137:15021/healthz/ready": dial tcp 10.132.0.137:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-vb9r5 Startup probe failed: Get "http://10.132.0.137:15021/healthz/ready": dial tcp 10.132.0.137:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-4strh Startup probe failed: Get "http://10.134.0.84:15021/healthz/ready": dial tcp 10.134.0.84:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-4strh Startup probe error: Get "http://10.134.0.84:15021/healthz/ready": dial tcp 10.134.0.84:15021: connect: connection refused... 52m Normal SuccessfulCreate replicaset/e2e-ait-0d96587a-openshift-default-bcfbd49cc Created pod: e2e-ait-0d96587a-openshift-default-bcfbd49cc-xrrp4 52m Normal ScalingReplicaSet deployment/e2e-ait-0d96587a-openshift-default Scaled up replica set e2e-ait-0d96587a-openshift-default-bcfbd49cc from 0 to 1 52m Normal AddedLabel gateway/e2e-ait-0d96587a Added label istio.io/rev=openshift-gateway to gateway e2e-ait-0d96587a 52m Warning Unhealthy pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-hrh5g Readiness probe failed: Get "http://10.133.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-hrh5g Readiness probe error: Get "http://10.133.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-4jrqx Readiness probe error: Get "http://10.134.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8hzf8 Readiness probe failed: Get "http://10.134.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8hzf8 Readiness probe error: Get "http://10.134.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-klh6t Readiness probe failed: Get "http://10.132.0.102:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-klh6t Readiness probe error: Get "http://10.132.0.102:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-q7dtb Readiness probe failed: Get "http://10.132.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning Unhealthy pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-4jrqx Readiness probe failed: Get "http://10.134.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-q7dtb Readiness probe error: Get "http://10.132.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Normal AddedInterface pod/e2e-ait-0d96587a-openshift-default-bcfbd49cc-xrrp4 Add eth0 [10.132.0.144/23] from ovn-kubernetes 52m Warning ProbeError pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-7w9dq Readiness probe error: Get "http://10.132.0.136:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-7w9dq Readiness probe failed: Get "http://10.132.0.136:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Normal ScalingReplicaSet deployment/e2e-ait-8e7d0098-openshift-default Scaled up replica set e2e-ait-8e7d0098-openshift-default-7c65b994b7 from 0 to 1 52m Normal AddedLabel gateway/e2e-ait-8e7d0098 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-8e7d0098 52m Normal SuccessfulCreate replicaset/e2e-ait-8e7d0098-openshift-default-7c65b994b7 Created pod: e2e-ait-8e7d0098-openshift-default-7c65b994b7-ntss6 52m Normal AddedInterface pod/e2e-ait-8e7d0098-openshift-default-7c65b994b7-ntss6 Add eth0 [10.134.0.89/23] from ovn-kubernetes 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-8e7d0098 Scaled up replica set payload-pre-processing-e2e-ait-8e7d0098-6794f6ff67 from 0 to 1 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-8e7d0098 Scaled up replica set payload-processing-e2e-ait-8e7d0098-8f775757b from 0 to 1 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-8e7d0098-6794f6ff67 Created pod: payload-pre-processing-e2e-ait-8e7d0098-6794f6ff67-429f5 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-8e7d0098-8f775757b Created pod: payload-processing-e2e-ait-8e7d0098-8f775757b-4mt6m 52m Warning FailedCreate replicaset/payload-processing-e2e-ait-8e7d0098-8f775757b Error creating: pods "payload-processing-e2e-ait-8e7d0098-8f775757b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-8e7d0098: serviceaccount "payload-processing-e2e-ait-8e7d0098" not found 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-8e7d0098-6794f6ff67 Error creating: pods "payload-pre-processing-e2e-ait-8e7d0098-6794f6ff67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-8e7d0098: serviceaccount "payload-processing-e2e-ait-8e7d0098" not found 52m Normal Pulled pod/payload-pre-processing-e2e-ait-8e7d0098-6794f6ff67-429f5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/payload-pre-processing-e2e-ait-8e7d0098-6794f6ff67-429f5 Add eth0 [10.132.0.147/23] from ovn-kubernetes 52m Normal Started pod/payload-pre-processing-e2e-ait-8e7d0098-6794f6ff67-429f5 Container started 52m Normal Created pod/payload-pre-processing-e2e-ait-8e7d0098-6794f6ff67-429f5 Container created 52m Normal AddedInterface pod/payload-processing-e2e-ait-8e7d0098-8f775757b-4mt6m Add eth0 [10.132.0.148/23] from ovn-kubernetes 52m Normal Pulled pod/payload-processing-e2e-ait-8e7d0098-8f775757b-4mt6m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Created pod/payload-processing-e2e-ait-8e7d0098-8f775757b-4mt6m Container created 52m Normal Started pod/payload-processing-e2e-ait-8e7d0098-8f775757b-4mt6m Container started 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-0d96587a Scaled up replica set payload-processing-e2e-ait-0d96587a-58d9d8846 from 0 to 1 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-0d96587a-584d7b5dff Created pod: payload-pre-processing-e2e-ait-0d96587a-584d7b5dff-ccjj6 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-0d96587a-584d7b5dff Error creating: pods "payload-pre-processing-e2e-ait-0d96587a-584d7b5dff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-0d96587a: serviceaccount "payload-processing-e2e-ait-0d96587a" not found 52m Warning FailedCreate replicaset/payload-processing-e2e-ait-0d96587a-58d9d8846 Error creating: pods "payload-processing-e2e-ait-0d96587a-58d9d8846-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-0d96587a: serviceaccount "payload-processing-e2e-ait-0d96587a" not found 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-0d96587a Scaled up replica set payload-pre-processing-e2e-ait-0d96587a-584d7b5dff from 0 to 1 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-0d96587a-58d9d8846 Created pod: payload-processing-e2e-ait-0d96587a-58d9d8846-j62hj 52m Normal AddedInterface pod/payload-pre-processing-e2e-ait-0d96587a-584d7b5dff-ccjj6 Add eth0 [10.132.0.149/23] from ovn-kubernetes 52m Normal SuccessfulCreate replicaset/e2e-mt-e0751b65-openshift-default-5846888d89 Created pod: e2e-mt-e0751b65-openshift-default-5846888d89-2pzt4 52m Normal Created pod/payload-processing-e2e-ait-0d96587a-58d9d8846-j62hj Container created 52m Normal Started pod/payload-pre-processing-e2e-ait-0d96587a-584d7b5dff-ccjj6 Container started 52m Normal Pulled pod/payload-pre-processing-e2e-ait-0d96587a-584d7b5dff-ccjj6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal AddedLabel gateway/e2e-mt-e0751b65 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-e0751b65 52m Normal AddedInterface pod/payload-processing-e2e-ait-0d96587a-58d9d8846-j62hj Add eth0 [10.132.0.150/23] from ovn-kubernetes 52m Normal Pulled pod/payload-processing-e2e-ait-0d96587a-58d9d8846-j62hj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal ScalingReplicaSet deployment/e2e-mt-e0751b65-openshift-default Scaled up replica set e2e-mt-e0751b65-openshift-default-5846888d89 from 0 to 1 52m Normal Created pod/payload-pre-processing-e2e-ait-0d96587a-584d7b5dff-ccjj6 Container created 52m Normal Started pod/payload-processing-e2e-ait-0d96587a-58d9d8846-j62hj Container started 52m Normal AddedInterface pod/e2e-mt-e0751b65-openshift-default-5846888d89-2pzt4 Add eth0 [10.134.0.90/23] from ovn-kubernetes 52m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-wqjzk 52m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 52m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 52m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-wqjzk Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 52m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-wqjzk Container started 52m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-wqjzk Add eth0 [10.133.0.71/23] from ovn-kubernetes 52m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-wqjzk Container created 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-34f1ce-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 52m Normal ScalingReplicaSet deployment/e2e-worker-w4-34f1ce-openshift-default Scaled up replica set e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf from 5 to 10 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf Created pod: e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-m8pb2 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf Created pod: e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-pc4b9 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf Created pod: e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-r8xrl 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf (combined from similar events): Created pod: e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-hs4pf 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf Created pod: e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-vx2nt 52m Normal Pulled pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-vx2nt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal SuccessfulCreate replicaset/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc (combined from similar events): Created pod: e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-t4ms2 52m Normal AddedInterface pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-t28zs Add eth0 [10.134.0.93/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-pc4b9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-wqjzk Readiness probe failed: HTTP probe failed with statuscode: 503 52m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-wqjzk Readiness probe error: HTTP probe failed with statuscode: 503... 52m Normal ScalingReplicaSet deployment/e2e-worker-w3-986d37-openshift-default Scaled up replica set e2e-worker-w3-986d37-openshift-default-5bc8bd86fc from 6 to 10 52m Normal AddedInterface pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-r8xrl Add eth0 [10.134.0.91/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-r8xrl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-r8xrl Container created 52m Normal Started pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-m8pb2 Container started 52m Normal Created pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-m8pb2 Container created 52m Normal Started pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-vx2nt Container started 52m Normal Pulled pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-m8pb2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-m8pb2 Add eth0 [10.134.0.92/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-r8xrl Container started 52m Normal AddedInterface pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-pc4b9 Add eth0 [10.132.0.152/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-hs4pf Container started 52m Normal Created pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-hs4pf Container created 52m Normal Pulled pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-hs4pf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-hs4pf Add eth0 [10.132.0.153/23] from ovn-kubernetes 52m Normal Created pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-vx2nt Container created 52m Normal SuccessfulCreate replicaset/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc Created pod: e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-mlkdz 52m Normal AddedInterface pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-vx2nt Add eth0 [10.133.0.72/23] from ovn-kubernetes 52m Normal SuccessfulCreate replicaset/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc Created pod: e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-pg48c 52m Normal SuccessfulCreate replicaset/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc Created pod: e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-t28zs 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-986d37-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 52m Normal AddedInterface pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-mlkdz Add eth0 [10.132.0.154/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-t4ms2 Container started 52m Normal Created pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-pc4b9 Container created 52m Normal Created pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-mlkdz Container created 52m Normal Pulled pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-mlkdz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Started pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-t28zs Container started 52m Normal Created pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-t28zs Container created 52m Normal AddedInterface pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-pg48c Add eth0 [10.133.0.73/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-mlkdz Container started 52m Normal Pulled pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-t4ms2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Pulled pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-pg48c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Started pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-pc4b9 Container started 52m Normal AddedInterface pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-t4ms2 Add eth0 [10.132.0.155/23] from ovn-kubernetes 52m Normal Created pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-t4ms2 Container created 52m Normal Created pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-pg48c Container created 52m Normal Started pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-pg48c Container started 52m Normal Pulled pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-t28zs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-e0751b65 Scaled up replica set payload-processing-e2e-mt-e0751b65-769966b6c4 from 0 to 1 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-e0751b65 Scaled up replica set payload-pre-processing-e2e-mt-e0751b65-7d7cd79459 from 0 to 1 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-e0751b65-769966b6c4 Created pod: payload-processing-e2e-mt-e0751b65-769966b6c4-9pw2x 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-e0751b65-7d7cd79459 Created pod: payload-pre-processing-e2e-mt-e0751b65-7d7cd79459-76xvm 52m Warning FailedCreate replicaset/payload-processing-e2e-mt-e0751b65-769966b6c4 Error creating: pods "payload-processing-e2e-mt-e0751b65-769966b6c4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-e0751b65: serviceaccount "payload-processing-e2e-mt-e0751b65" not found 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-e0751b65-7d7cd79459 Error creating: pods "payload-pre-processing-e2e-mt-e0751b65-7d7cd79459-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-e0751b65: serviceaccount "payload-processing-e2e-mt-e0751b65" not found 52m Normal AddedInterface pod/payload-processing-e2e-mt-e0751b65-769966b6c4-9pw2x Add eth0 [10.134.0.94/23] from ovn-kubernetes 52m Normal Pulled pod/payload-processing-e2e-mt-e0751b65-769966b6c4-9pw2x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Created pod/payload-processing-e2e-mt-e0751b65-769966b6c4-9pw2x Container created 52m Normal Started pod/payload-processing-e2e-mt-e0751b65-769966b6c4-9pw2x Container started 52m Warning ProbeError pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-vx2nt Startup probe error: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-vx2nt Startup probe failed: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused 52m Normal AddedInterface pod/payload-pre-processing-e2e-mt-e0751b65-7d7cd79459-76xvm Add eth0 [10.132.0.156/23] from ovn-kubernetes 52m Warning Unhealthy pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-pg48c Startup probe failed: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-pg48c Startup probe error: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-r8xrl Startup probe failed: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-t28zs Startup probe failed: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused 52m Normal Pulled pod/payload-pre-processing-e2e-mt-e0751b65-7d7cd79459-76xvm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Warning ProbeError pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-t28zs Startup probe error: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-r8xrl Startup probe error: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-m8pb2 Startup probe error: Get "http://10.134.0.92:15021/healthz/ready": dial tcp 10.134.0.92:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-m8pb2 Startup probe failed: Get "http://10.134.0.92:15021/healthz/ready": dial tcp 10.134.0.92:15021: connect: connection refused 52m Normal AddedLabel gateway/e2e-migrate-d32a7cc0-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-d32a7cc0-gw 52m Normal Created pod/payload-pre-processing-e2e-mt-e0751b65-7d7cd79459-76xvm Container created 52m Normal Started pod/payload-pre-processing-e2e-mt-e0751b65-7d7cd79459-76xvm Container started 52m Normal SuccessfulCreate replicaset/e2e-migrate-d32a7cc0-gw-openshift-default-74cb955697 Created pod: e2e-migrate-d32a7cc0-gw-openshift-default-74cb955697-d8gkc 52m Normal ScalingReplicaSet deployment/e2e-migrate-d32a7cc0-gw-openshift-default Scaled up replica set e2e-migrate-d32a7cc0-gw-openshift-default-74cb955697 from 0 to 1 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-8e7d0098-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Normal AddedInterface pod/e2e-migrate-d32a7cc0-gw-openshift-default-74cb955697-d8gkc Add eth0 [10.134.0.95/23] from ovn-kubernetes 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-8e7d0098-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning ProbeError pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-hs4pf Startup probe error: Get "http://10.132.0.153:15021/healthz/ready": dial tcp 10.132.0.153:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-hs4pf Startup probe failed: Get "http://10.132.0.153:15021/healthz/ready": dial tcp 10.132.0.153:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-t4ms2 Startup probe error: Get "http://10.132.0.155:15021/healthz/ready": dial tcp 10.132.0.155:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-mlkdz Startup probe failed: Get "http://10.132.0.154:15021/healthz/ready": dial tcp 10.132.0.154:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-mlkdz Startup probe error: Get "http://10.132.0.154:15021/healthz/ready": dial tcp 10.132.0.154:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-t4ms2 Startup probe failed: Get "http://10.132.0.155:15021/healthz/ready": dial tcp 10.132.0.155:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-pc4b9 Startup probe error: Get "http://10.132.0.152:15021/healthz/ready": dial tcp 10.132.0.152:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-pc4b9 Startup probe failed: Get "http://10.132.0.152:15021/healthz/ready": dial tcp 10.132.0.152:15021: connect: connection refused 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-d32a7cc0 Scaled up replica set payload-pre-processing-e2e-migrate-d32a7cc0-6b7768cb67 from 0 to 1 52m Warning FailedCreate replicaset/payload-processing-e2e-migrate-d32a7cc0-76fd97d5d6 Error creating: pods "payload-processing-e2e-migrate-d32a7cc0-76fd97d5d6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-d32a7cc0: serviceaccount "payload-processing-e2e-migrate-d32a7cc0" not found 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-d32a7cc0-6b7768cb67 Error creating: pods "payload-pre-processing-e2e-migrate-d32a7cc0-6b7768cb67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-d32a7cc0: serviceaccount "payload-processing-e2e-migrate-d32a7cc0" not found 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-d32a7cc0 Scaled up replica set payload-processing-e2e-migrate-d32a7cc0-76fd97d5d6 from 0 to 1 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-d32a7cc0-76fd97d5d6 Created pod: payload-processing-e2e-migrate-d32a7cc0-76fd97d5d6-62rcv 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-d32a7cc0-6b7768cb67 Created pod: payload-pre-processing-e2e-migrate-d32a7cc0-6b7768cb67-ntrqr 52m Normal Created pod/payload-processing-e2e-migrate-d32a7cc0-76fd97d5d6-62rcv Container created 52m Normal Started pod/payload-processing-e2e-migrate-d32a7cc0-76fd97d5d6-62rcv Container started 52m Normal Pulled pod/payload-pre-processing-e2e-migrate-d32a7cc0-6b7768cb67-ntrqr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/payload-processing-e2e-migrate-d32a7cc0-76fd97d5d6-62rcv Add eth0 [10.132.0.157/23] from ovn-kubernetes 52m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-d32a7cc0-6b7768cb67-ntrqr Add eth0 [10.132.0.158/23] from ovn-kubernetes 52m Normal Pulled pod/payload-processing-e2e-migrate-d32a7cc0-76fd97d5d6-62rcv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Started pod/payload-pre-processing-e2e-migrate-d32a7cc0-6b7768cb67-ntrqr Container started 52m Normal Created pod/payload-pre-processing-e2e-migrate-d32a7cc0-6b7768cb67-ntrqr Container created 51m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 51m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-zvnnz Container created 51m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-thj8z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-thj8z Add eth0 [10.134.0.98/23] from ovn-kubernetes 51m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-zvnnz Add eth0 [10.132.0.159/23] from ovn-kubernetes 51m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-zvnnz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-zvnnz Container started 51m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-thj8z Container created 51m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-thj8z Container started 51m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-thj8z Startup probe error: Get "http://10.134.0.98:15021/healthz/ready": dial tcp 10.134.0.98:15021: connect: connection refused... 51m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-thj8z Startup probe failed: Get "http://10.134.0.98:15021/healthz/ready": dial tcp 10.134.0.98:15021: connect: connection refused 51m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-zvnnz Startup probe failed: Get "http://10.132.0.159:15021/healthz/ready": dial tcp 10.132.0.159:15021: connect: connection refused 51m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-zvnnz Startup probe error: Get "http://10.132.0.159:15021/healthz/ready": dial tcp 10.132.0.159:15021: connect: connection refused... 51m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default (combined from similar events): Scaled up replica set maas-default-gateway-openshift-default-8f6f87f59 from 5 to 10 51m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-8f6f87f59 (combined from similar events): Created pod: maas-default-gateway-openshift-default-8f6f87f59-sgc28 51m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-wv5pf Add eth0 [10.134.0.99/23] from ovn-kubernetes 51m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-dpcqs Add eth0 [10.132.0.161/23] from ovn-kubernetes 51m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-4trk8 Add eth0 [10.134.0.100/23] from ovn-kubernetes 51m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-4trk8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-8fws9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-8fws9 Add eth0 [10.132.0.160/23] from ovn-kubernetes 51m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-4trk8 Container created 51m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-sgc28 Add eth0 [10.133.0.74/23] from ovn-kubernetes 51m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-sgc28 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-wv5pf Container started 51m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-wv5pf Container created 51m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-wv5pf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-4trk8 Container started 51m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-8fws9 Container created 51m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-dpcqs Container started 51m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-dpcqs Container created 51m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-dpcqs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-8fws9 Container started 51m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-sgc28 Container created 51m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-sgc28 Container started 51m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-4trk8 Startup probe error: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused... 51m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-4trk8 Startup probe failed: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused 51m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-wv5pf Startup probe failed: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused 51m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-wv5pf Startup probe error: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused... 51m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-sgc28 Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused... 51m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-sgc28 Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused 51m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-8fws9 Startup probe failed: Get "http://10.132.0.160:15021/healthz/ready": dial tcp 10.132.0.160:15021: connect: connection refused 51m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-8fws9 Startup probe error: Get "http://10.132.0.160:15021/healthz/ready": dial tcp 10.132.0.160:15021: connect: connection refused... 51m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-dpcqs Startup probe failed: Get "http://10.132.0.161:15021/healthz/ready": dial tcp 10.132.0.161:15021: connect: connection refused 51m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-dpcqs Startup probe error: Get "http://10.132.0.161:15021/healthz/ready": dial tcp 10.132.0.161:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-d44pq Readiness probe failed: Get "http://10.132.0.100:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-d44pq Readiness probe error: Get "http://10.132.0.100:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Normal SuccessfulCreate replicaset/e2e-ait-8e7d0098-openshift-default-7c65b994b7 Created pod: e2e-ait-8e7d0098-openshift-default-7c65b994b7-tc65h 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ait-8e7d0098-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 51m Normal ScalingReplicaSet deployment/e2e-ait-8e7d0098-openshift-default Scaled up replica set e2e-ait-8e7d0098-openshift-default-7c65b994b7 from 1 to 2 51m Normal Started pod/e2e-ait-8e7d0098-openshift-default-7c65b994b7-tc65h Container started 51m Normal AddedInterface pod/e2e-ait-8e7d0098-openshift-default-7c65b994b7-tc65h Add eth0 [10.132.0.162/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-ait-8e7d0098-openshift-default-7c65b994b7-tc65h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-ait-8e7d0098-openshift-default-7c65b994b7-tc65h Container created 51m Normal Pulled pod/e2e-mt-e0751b65-openshift-default-5846888d89-2pzt4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-ait-8e7d0098-openshift-default-7c65b994b7-ntss6 Container started 51m Normal Pulled pod/e2e-ait-8e7d0098-openshift-default-7c65b994b7-ntss6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Warning Unhealthy pod/e2e-ait-8e7d0098-openshift-default-7c65b994b7-tc65h Startup probe failed: Get "http://10.132.0.162:15021/healthz/ready": dial tcp 10.132.0.162:15021: connect: connection refused 51m Normal Created pod/e2e-mt-e0751b65-openshift-default-5846888d89-2pzt4 Container created 51m Normal Created pod/e2e-ait-8e7d0098-openshift-default-7c65b994b7-ntss6 Container created 51m Normal Started pod/e2e-mt-e0751b65-openshift-default-5846888d89-2pzt4 Container started 51m Warning ProbeError pod/e2e-ait-8e7d0098-openshift-default-7c65b994b7-tc65h Startup probe error: Get "http://10.132.0.162:15021/healthz/ready": dial tcp 10.132.0.162:15021: connect: connection refused... 51m Normal Created pod/e2e-ait-0d96587a-openshift-default-bcfbd49cc-xrrp4 Container created 51m Normal Pulled pod/e2e-ait-0d96587a-openshift-default-bcfbd49cc-xrrp4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-ait-0d96587a-openshift-default-bcfbd49cc-xrrp4 Container started 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-e0751b65-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-e0751b65-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning ProbeError pod/e2e-mt-e0751b65-openshift-default-5846888d89-2pzt4 Startup probe error: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-mt-e0751b65-openshift-default-5846888d89-2pzt4 Startup probe failed: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-ait-8e7d0098-openshift-default-7c65b994b7-ntss6 Startup probe failed: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused 51m Warning ProbeError pod/e2e-ait-8e7d0098-openshift-default-7c65b994b7-ntss6 Startup probe error: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-ait-0d96587a-openshift-default-bcfbd49cc-xrrp4 Startup probe failed: Get "http://10.132.0.144:15021/healthz/ready": dial tcp 10.132.0.144:15021: connect: connection refused 51m Warning ProbeError pod/e2e-ait-0d96587a-openshift-default-bcfbd49cc-xrrp4 Startup probe error: Get "http://10.132.0.144:15021/healthz/ready": dial tcp 10.132.0.144:15021: connect: connection refused... 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-e0751b65-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-e0751b65-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-8e7d0098-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-8e7d0098-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Normal Killing pod/payload-processing-e2e-ait-8e7d0098-8f775757b-4mt6m Stopping container payload-processing 50m Normal Killing pod/payload-pre-processing-e2e-ait-8e7d0098-6794f6ff67-429f5 Stopping container payload-pre-processing 50m Warning Unhealthy pod/payload-processing-e2e-ait-8e7d0098-8f775757b-4mt6m Readiness probe failed: timeout: failed to connect service "10.132.0.148:50052" within 1s: context deadline exceeded 50m Normal Killing pod/payload-pre-processing-e2e-ait-0d96587a-584d7b5dff-ccjj6 Stopping container payload-pre-processing 50m Warning Unhealthy pod/payload-pre-processing-e2e-ait-8e7d0098-6794f6ff67-429f5 Readiness probe failed: timeout: failed to connect service "10.132.0.147:50052" within 1s: context deadline exceeded 50m Normal Killing pod/payload-processing-e2e-ait-0d96587a-58d9d8846-j62hj Stopping container payload-processing 50m Warning Unhealthy pod/payload-pre-processing-e2e-ait-0d96587a-584d7b5dff-ccjj6 Readiness probe failed: timeout: failed to connect service "10.132.0.149:50052" within 1s: context deadline exceeded 50m Normal Created pod/e2e-migrate-d32a7cc0-gw-openshift-default-74cb955697-d8gkc Container created 50m Normal Started pod/e2e-migrate-d32a7cc0-gw-openshift-default-74cb955697-d8gkc Container started 50m Normal Pulled pod/e2e-migrate-d32a7cc0-gw-openshift-default-74cb955697-d8gkc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Warning Unhealthy pod/payload-processing-e2e-ait-0d96587a-58d9d8846-j62hj Readiness probe failed: timeout: failed to connect service "10.132.0.150:50052" within 1s: context deadline exceeded 50m Warning Unhealthy pod/e2e-migrate-d32a7cc0-gw-openshift-default-74cb955697-d8gkc Startup probe failed: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused 50m Warning ProbeError pod/e2e-migrate-d32a7cc0-gw-openshift-default-74cb955697-d8gkc Startup probe error: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused... 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-0d96587a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-0d96587a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Normal Killing pod/e2e-ait-8e7d0098-openshift-default-7c65b994b7-tc65h Stopping container istio-proxy 50m Normal Killing pod/e2e-ait-8e7d0098-openshift-default-7c65b994b7-ntss6 Stopping container istio-proxy 50m Normal AddedLabel gateway/e2e-derive-09efcb6d Added label istio.io/rev=openshift-gateway to gateway e2e-derive-09efcb6d 50m Normal SuccessfulCreate replicaset/e2e-derive-09efcb6d-openshift-default-7c46fd677c Created pod: e2e-derive-09efcb6d-openshift-default-7c46fd677c-6wtzg 50m Normal ScalingReplicaSet deployment/e2e-derive-09efcb6d-openshift-default Scaled up replica set e2e-derive-09efcb6d-openshift-default-7c46fd677c from 0 to 1 50m Normal Killing pod/e2e-ait-0d96587a-openshift-default-bcfbd49cc-xrrp4 Stopping container istio-proxy 50m Warning Unhealthy pod/e2e-ait-8e7d0098-openshift-default-7c65b994b7-ntss6 Readiness probe failed: HTTP probe failed with statuscode: 503 50m Normal AddedInterface pod/e2e-derive-09efcb6d-openshift-default-7c46fd677c-6wtzg Add eth0 [10.132.0.167/23] from ovn-kubernetes 50m Warning ProbeError pod/e2e-ait-8e7d0098-openshift-default-7c65b994b7-ntss6 Readiness probe error: HTTP probe failed with statuscode: 503... 50m Normal AddedLabel gateway/e2e-ait-7c8802f8 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-7c8802f8 50m Normal SuccessfulCreate replicaset/e2e-ait-7c8802f8-openshift-default-74499c95cc Created pod: e2e-ait-7c8802f8-openshift-default-74499c95cc-v5qrl 50m Normal ScalingReplicaSet deployment/e2e-ait-7c8802f8-openshift-default Scaled up replica set e2e-ait-7c8802f8-openshift-default-74499c95cc from 0 to 1 50m Warning Unhealthy pod/e2e-ait-0d96587a-openshift-default-bcfbd49cc-xrrp4 Readiness probe failed: HTTP probe failed with statuscode: 503 50m Normal AddedInterface pod/e2e-ait-7c8802f8-openshift-default-74499c95cc-v5qrl Add eth0 [10.132.0.168/23] from ovn-kubernetes 50m Warning ProbeError pod/e2e-ait-0d96587a-openshift-default-bcfbd49cc-xrrp4 Readiness probe error: HTTP probe failed with statuscode: 503... 50m Normal Killing pod/payload-processing-e2e-migrate-d32a7cc0-76fd97d5d6-62rcv Stopping container payload-processing 50m Normal Killing pod/payload-pre-processing-e2e-migrate-d32a7cc0-6b7768cb67-ntrqr Stopping container payload-pre-processing 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-09efcb6d-c7bfbb5cf Error creating: pods "payload-pre-processing-e2e-derive-09efcb6d-c7bfbb5cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-09efcb6d: serviceaccount "payload-processing-e2e-derive-09efcb6d" not found 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-09efcb6d-c7bfbb5cf Created pod: payload-pre-processing-e2e-derive-09efcb6d-c7bfbb5cf-fsvrl 49m Warning FailedCreate replicaset/payload-processing-e2e-derive-09efcb6d-699987c7fd Error creating: pods "payload-processing-e2e-derive-09efcb6d-699987c7fd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-09efcb6d: serviceaccount "payload-processing-e2e-derive-09efcb6d" not found 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-09efcb6d Scaled up replica set payload-pre-processing-e2e-derive-09efcb6d-c7bfbb5cf from 0 to 1 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-09efcb6d-699987c7fd Created pod: payload-processing-e2e-derive-09efcb6d-699987c7fd-j2v69 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-09efcb6d Scaled up replica set payload-processing-e2e-derive-09efcb6d-699987c7fd from 0 to 1 49m Normal AddedInterface pod/payload-pre-processing-e2e-derive-09efcb6d-c7bfbb5cf-fsvrl Add eth0 [10.132.0.169/23] from ovn-kubernetes 49m Normal Created pod/payload-pre-processing-e2e-derive-09efcb6d-c7bfbb5cf-fsvrl Container created 49m Normal AddedInterface pod/payload-processing-e2e-derive-09efcb6d-699987c7fd-j2v69 Add eth0 [10.132.0.170/23] from ovn-kubernetes 49m Normal Created pod/payload-processing-e2e-derive-09efcb6d-699987c7fd-j2v69 Container created 49m Warning Unhealthy pod/payload-pre-processing-e2e-migrate-d32a7cc0-6b7768cb67-ntrqr Readiness probe failed: timeout: failed to connect service "10.132.0.158:50052" within 1s: context deadline exceeded 49m Warning Unhealthy pod/payload-processing-e2e-migrate-d32a7cc0-76fd97d5d6-62rcv Readiness probe failed: timeout: failed to connect service "10.132.0.157:50052" within 1s: context deadline exceeded 49m Normal Pulled pod/payload-processing-e2e-derive-09efcb6d-699987c7fd-j2v69 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Pulled pod/payload-pre-processing-e2e-derive-09efcb6d-c7bfbb5cf-fsvrl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Started pod/payload-processing-e2e-derive-09efcb6d-699987c7fd-j2v69 Container started 49m Normal Started pod/payload-pre-processing-e2e-derive-09efcb6d-c7bfbb5cf-fsvrl Container started 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-7c8802f8 Scaled up replica set payload-processing-e2e-ait-7c8802f8-5db6686694 from 0 to 1 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-7c8802f8-84c649965f Created pod: payload-pre-processing-e2e-ait-7c8802f8-84c649965f-8cltq 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-7c8802f8-5db6686694 Created pod: payload-processing-e2e-ait-7c8802f8-5db6686694-5hbns 49m Warning FailedCreate replicaset/payload-processing-e2e-ait-7c8802f8-5db6686694 Error creating: pods "payload-processing-e2e-ait-7c8802f8-5db6686694-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-7c8802f8: serviceaccount "payload-processing-e2e-ait-7c8802f8" not found 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-7c8802f8-84c649965f Error creating: pods "payload-pre-processing-e2e-ait-7c8802f8-84c649965f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-7c8802f8: serviceaccount "payload-processing-e2e-ait-7c8802f8" not found 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-7c8802f8 Scaled up replica set payload-pre-processing-e2e-ait-7c8802f8-84c649965f from 0 to 1 49m Normal AddedInterface pod/payload-processing-e2e-ait-7c8802f8-5db6686694-5hbns Add eth0 [10.132.0.172/23] from ovn-kubernetes 49m Normal Pulled pod/payload-pre-processing-e2e-ait-7c8802f8-84c649965f-8cltq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/payload-pre-processing-e2e-ait-7c8802f8-84c649965f-8cltq Add eth0 [10.132.0.171/23] from ovn-kubernetes 49m Normal Pulled pod/payload-processing-e2e-ait-7c8802f8-5db6686694-5hbns Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Created pod/payload-pre-processing-e2e-ait-7c8802f8-84c649965f-8cltq Container created 49m Normal Started pod/payload-pre-processing-e2e-ait-7c8802f8-84c649965f-8cltq Container started 49m Normal Started pod/payload-processing-e2e-ait-7c8802f8-5db6686694-5hbns Container started 49m Normal Created pod/payload-processing-e2e-ait-7c8802f8-5db6686694-5hbns Container created 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-d32a7cc0-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-d32a7cc0-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-7c8802f8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-7c8802f8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Normal AddedLabel gateway/e2e-shared-a-w6-f7b8d8 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w6-f7b8d8 49m Normal Killing pod/e2e-migrate-d32a7cc0-gw-openshift-default-74cb955697-d8gkc Stopping container istio-proxy 49m Normal AddedInterface pod/e2e-shared-a-w6-f7b8d8-openshift-default-6b8ffbcc97-cccbw Add eth0 [10.134.0.101/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-f7b8d8-openshift-default-6b8ffbcc97 Created pod: e2e-shared-a-w6-f7b8d8-openshift-default-6b8ffbcc97-cccbw 49m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-f7b8d8-openshift-default Scaled up replica set e2e-shared-a-w6-f7b8d8-openshift-default-6b8ffbcc97 from 0 to 1 49m Normal ScalingReplicaSet deployment/e2e-worker-w1-5c3374-openshift-default Scaled down replica set e2e-worker-w1-5c3374-openshift-default-d6ff699c7 from 6 to 4 49m Normal SuccessfulDelete replicaset/e2e-worker-w1-5c3374-openshift-default-d6ff699c7 Deleted pod: e2e-worker-w1-5c3374-openshift-default-d6ff699c7-84bch 49m Normal SuccessfulDelete replicaset/e2e-worker-w1-5c3374-openshift-default-d6ff699c7 Deleted pod: e2e-worker-w1-5c3374-openshift-default-d6ff699c7-bn695 49m Normal Killing pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-84bch Stopping container istio-proxy 49m Normal Killing pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-bn695 Stopping container istio-proxy 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-5c3374-openshift-default New size: 4; reason: All metrics below target 49m Warning ProbeError pod/e2e-migrate-d32a7cc0-gw-openshift-default-74cb955697-d8gkc Readiness probe error: HTTP probe failed with statuscode: 503... 49m Warning Unhealthy pod/e2e-migrate-d32a7cc0-gw-openshift-default-74cb955697-d8gkc Readiness probe failed: HTTP probe failed with statuscode: 503 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w6-f7b8d8-576dfbf9f9 Error creating: pods "payload-pre-processing-e2e-shared-a-w6-f7b8d8-576dfbf9f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-f7b8d8: serviceaccount "payload-processing-e2e-shared-a-w6-f7b8d8" not found 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w6-f7b8d8 Scaled up replica set payload-processing-e2e-shared-a-w6-f7b8d8-6b866d558b from 0 to 1 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w6-f7b8d8 Scaled up replica set payload-pre-processing-e2e-shared-a-w6-f7b8d8-576dfbf9f9 from 0 to 1 49m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w6-f7b8d8-6b866d558b Error creating: pods "payload-processing-e2e-shared-a-w6-f7b8d8-6b866d558b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-f7b8d8: serviceaccount "payload-processing-e2e-shared-a-w6-f7b8d8" not found 49m Normal Pulled pod/payload-processing-e2e-shared-a-w6-f7b8d8-6b866d558b-hq859 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w6-f7b8d8-576dfbf9f9 Created pod: payload-pre-processing-e2e-shared-a-w6-f7b8d8-576dfbf9f9-d9d8l 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w6-f7b8d8-6b866d558b Created pod: payload-processing-e2e-shared-a-w6-f7b8d8-6b866d558b-hq859 49m Normal AddedInterface pod/payload-processing-e2e-shared-a-w6-f7b8d8-6b866d558b-hq859 Add eth0 [10.134.0.102/23] from ovn-kubernetes 49m Normal Created pod/payload-processing-e2e-shared-a-w6-f7b8d8-6b866d558b-hq859 Container created 49m Normal Started pod/payload-processing-e2e-shared-a-w6-f7b8d8-6b866d558b-hq859 Container started 49m Normal Started pod/payload-pre-processing-e2e-shared-a-w6-f7b8d8-576dfbf9f9-d9d8l Container started 49m Normal Created pod/payload-pre-processing-e2e-shared-a-w6-f7b8d8-576dfbf9f9-d9d8l Container created 49m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w6-f7b8d8-576dfbf9f9-d9d8l Add eth0 [10.132.0.174/23] from ovn-kubernetes 49m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w6-f7b8d8-576dfbf9f9-d9d8l Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal SuccessfulCreate replicaset/e2e-mt-a251fdfb-openshift-default-7f7f49cc57 Created pod: e2e-mt-a251fdfb-openshift-default-7f7f49cc57-wvq8z 49m Normal ScalingReplicaSet deployment/e2e-mt-a251fdfb-openshift-default Scaled up replica set e2e-mt-a251fdfb-openshift-default-7f7f49cc57 from 0 to 1 49m Normal AddedLabel gateway/e2e-mt-a251fdfb Added label istio.io/rev=openshift-gateway to gateway e2e-mt-a251fdfb 49m Normal AddedInterface pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-wvq8z Add eth0 [10.133.0.77/23] from ovn-kubernetes 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-a251fdfb Scaled up replica set payload-pre-processing-e2e-mt-a251fdfb-6f8487684c from 0 to 1 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-a251fdfb-6f8487684c Created pod: payload-pre-processing-e2e-mt-a251fdfb-6f8487684c-6qzl9 49m Warning FailedCreate replicaset/payload-processing-e2e-mt-a251fdfb-6fb6bf7d6b Error creating: pods "payload-processing-e2e-mt-a251fdfb-6fb6bf7d6b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-a251fdfb: serviceaccount "payload-processing-e2e-mt-a251fdfb" not found 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-a251fdfb Scaled up replica set payload-processing-e2e-mt-a251fdfb-6fb6bf7d6b from 0 to 1 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-a251fdfb-6f8487684c Error creating: pods "payload-pre-processing-e2e-mt-a251fdfb-6f8487684c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-a251fdfb: serviceaccount "payload-processing-e2e-mt-a251fdfb" not found 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-a251fdfb-6fb6bf7d6b Created pod: payload-processing-e2e-mt-a251fdfb-6fb6bf7d6b-xmmgr 49m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-wv5pf Readiness probe failed: Get "http://10.134.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-zvnnz Readiness probe error: Get "http://10.132.0.159:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-wv5pf Readiness probe error: Get "http://10.134.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-zvnnz Readiness probe failed: Get "http://10.132.0.159:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Normal AddedInterface pod/payload-processing-e2e-mt-a251fdfb-6fb6bf7d6b-xmmgr Add eth0 [10.132.0.176/23] from ovn-kubernetes 49m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-dpcqs Readiness probe error: Get "http://10.132.0.161:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Normal AddedInterface pod/payload-pre-processing-e2e-mt-a251fdfb-6f8487684c-6qzl9 Add eth0 [10.132.0.175/23] from ovn-kubernetes 49m Normal Pulled pod/payload-pre-processing-e2e-mt-a251fdfb-6f8487684c-6qzl9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-dpcqs Readiness probe failed: Get "http://10.132.0.161:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Normal Started pod/payload-pre-processing-e2e-mt-a251fdfb-6f8487684c-6qzl9 Container started 49m Normal Created pod/payload-processing-e2e-mt-a251fdfb-6fb6bf7d6b-xmmgr Container created 49m Normal Pulled pod/payload-processing-e2e-mt-a251fdfb-6fb6bf7d6b-xmmgr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Created pod/payload-pre-processing-e2e-mt-a251fdfb-6f8487684c-6qzl9 Container created 49m Normal Started pod/payload-processing-e2e-mt-a251fdfb-6fb6bf7d6b-xmmgr Container started 48m Normal ScalingReplicaSet deployment/e2e-worker-w1-5c3374-openshift-default Scaled down replica set e2e-worker-w1-5c3374-openshift-default-d6ff699c7 from 4 to 3 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-5c3374-openshift-default New size: 3; reason: All metrics below target 48m Normal SuccessfulDelete replicaset/e2e-worker-w1-5c3374-openshift-default-d6ff699c7 Deleted pod: e2e-worker-w1-5c3374-openshift-default-d6ff699c7-n4hsl 48m Normal Killing pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-n4hsl Stopping container istio-proxy 48m Normal Killing pod/payload-pre-processing-e2e-mt-e0751b65-7d7cd79459-76xvm Stopping container payload-pre-processing 48m Normal Killing pod/payload-processing-e2e-mt-e0751b65-769966b6c4-9pw2x Stopping container payload-processing 48m Warning ProbeError pod/e2e-derive-09efcb6d-openshift-default-7c46fd677c-6wtzg Readiness probe error: Get "http://10.132.0.167:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-derive-09efcb6d-openshift-default-7c46fd677c-6wtzg Readiness probe failed: Get "http://10.132.0.167:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Normal Pulled pod/e2e-derive-09efcb6d-openshift-default-7c46fd677c-6wtzg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-derive-09efcb6d-openshift-default-7c46fd677c-6wtzg Container created 48m Normal Started pod/e2e-derive-09efcb6d-openshift-default-7c46fd677c-6wtzg Container started 48m Warning ProbeError pod/e2e-derive-09efcb6d-openshift-default-7c46fd677c-6wtzg Startup probe error: Get "http://10.132.0.167:15021/healthz/ready": dial tcp 10.132.0.167:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-derive-09efcb6d-openshift-default-7c46fd677c-6wtzg Startup probe failed: Get "http://10.132.0.167:15021/healthz/ready": dial tcp 10.132.0.167:15021: connect: connection refused 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-09efcb6d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-09efcb6d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-09efcb6d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-09efcb6d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Normal Killing pod/payload-processing-e2e-derive-09efcb6d-699987c7fd-j2v69 Stopping container payload-processing 47m Normal Killing pod/payload-pre-processing-e2e-derive-09efcb6d-c7bfbb5cf-fsvrl Stopping container payload-pre-processing 47m Normal Pulled pod/e2e-ait-7c8802f8-openshift-default-74499c95cc-v5qrl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-ait-7c8802f8-openshift-default-74499c95cc-v5qrl Container created 47m Normal Started pod/e2e-ait-7c8802f8-openshift-default-74499c95cc-v5qrl Container started 47m Normal Killing pod/payload-pre-processing-e2e-ait-7c8802f8-84c649965f-8cltq Stopping container payload-pre-processing 47m Normal Killing pod/payload-processing-e2e-ait-7c8802f8-5db6686694-5hbns Stopping container payload-processing 47m Normal Started pod/e2e-shared-a-w6-f7b8d8-openshift-default-6b8ffbcc97-cccbw Container started 47m Normal Pulled pod/e2e-shared-a-w6-f7b8d8-openshift-default-6b8ffbcc97-cccbw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-shared-a-w6-f7b8d8-openshift-default-6b8ffbcc97-cccbw Container created 47m Normal Killing pod/e2e-mt-e0751b65-openshift-default-5846888d89-2pzt4 Stopping container istio-proxy 47m Warning Unhealthy pod/e2e-ait-7c8802f8-openshift-default-74499c95cc-v5qrl Startup probe failed: Get "http://10.132.0.168:15021/healthz/ready": dial tcp 10.132.0.168:15021: connect: connection refused 47m Warning ProbeError pod/e2e-ait-7c8802f8-openshift-default-74499c95cc-v5qrl Startup probe error: Get "http://10.132.0.168:15021/healthz/ready": dial tcp 10.132.0.168:15021: connect: connection refused... 47m Warning ProbeError pod/e2e-shared-a-w6-f7b8d8-openshift-default-6b8ffbcc97-cccbw Startup probe error: Get "http://10.134.0.101:15021/healthz/ready": dial tcp 10.134.0.101:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-shared-a-w6-f7b8d8-openshift-default-6b8ffbcc97-cccbw Startup probe failed: Get "http://10.134.0.101:15021/healthz/ready": dial tcp 10.134.0.101:15021: connect: connection refused 47m Normal SuccessfulCreate replicaset/e2e-api-a-71b76a-openshift-default-77db967c97 Created pod: e2e-api-a-71b76a-openshift-default-77db967c97-lh4ld 47m Normal ScalingReplicaSet deployment/e2e-api-a-71b76a-openshift-default Scaled up replica set e2e-api-a-71b76a-openshift-default-77db967c97 from 0 to 1 47m Normal AddedLabel gateway/e2e-api-a-71b76a Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-71b76a 47m Normal AddedInterface pod/e2e-api-a-71b76a-openshift-default-77db967c97-lh4ld Add eth0 [10.132.0.182/23] from ovn-kubernetes 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-7c8802f8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-7c8802f8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Normal SuccessfulDelete replicaset/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 Deleted pod: e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-7gm84 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-27e82b-openshift-default New size: 6; reason: All metrics below target 47m Normal Killing pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-nr4qz Stopping container istio-proxy 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-71b76a Scaled up replica set payload-pre-processing-e2e-api-a-71b76a-64f77d669f from 0 to 1 47m Normal Killing pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-xgpsz Stopping container istio-proxy 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-71b76a Scaled up replica set payload-processing-e2e-api-a-71b76a-7b898f6878 from 0 to 1 47m Normal Killing pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-94tkp Stopping container istio-proxy 47m Normal ScalingReplicaSet deployment/e2e-worker-w2-27e82b-openshift-default Scaled down replica set e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 from 10 to 6 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-71b76a-64f77d669f Error creating: pods "payload-pre-processing-e2e-api-a-71b76a-64f77d669f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-71b76a: serviceaccount "payload-processing-e2e-api-a-71b76a" not found 47m Normal SuccessfulDelete replicaset/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 Deleted pod: e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-khcqd 47m Normal SuccessfulDelete replicaset/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 Deleted pod: e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-2qnns 47m Warning FailedCreate replicaset/payload-processing-e2e-api-a-71b76a-7b898f6878 Error creating: pods "payload-processing-e2e-api-a-71b76a-7b898f6878-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-71b76a: serviceaccount "payload-processing-e2e-api-a-71b76a" not found 47m Normal SuccessfulDelete replicaset/e2e-worker-w6-35a187-openshift-default-5b54c65f75 Deleted pod: e2e-worker-w6-35a187-openshift-default-5b54c65f75-xgpsz 47m Normal SuccessfulDelete replicaset/e2e-worker-w6-35a187-openshift-default-5b54c65f75 Deleted pod: e2e-worker-w6-35a187-openshift-default-5b54c65f75-nr4qz 47m Normal SuccessfulDelete replicaset/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 Deleted pod: e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-vb9r5 47m Normal ScalingReplicaSet deployment/e2e-worker-w6-35a187-openshift-default Scaled down replica set e2e-worker-w6-35a187-openshift-default-5b54c65f75 from 10 to 6 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-35a187-openshift-default New size: 6; reason: All metrics below target 47m Normal SuccessfulDelete replicaset/e2e-worker-w6-35a187-openshift-default-5b54c65f75 Deleted pod: e2e-worker-w6-35a187-openshift-default-5b54c65f75-klh6t 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-f7b8d8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-f7b8d8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Normal Killing pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-klh6t Stopping container istio-proxy 47m Normal SuccessfulDelete replicaset/e2e-worker-w6-35a187-openshift-default-5b54c65f75 Deleted pod: e2e-worker-w6-35a187-openshift-default-5b54c65f75-94tkp 47m Normal Killing pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-kzqr4 Stopping container istio-proxy 47m Normal Killing pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-7gm84 Stopping container istio-proxy 47m Normal Killing pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-jdcvx Stopping container istio-proxy 47m Normal ScalingReplicaSet deployment/e2e-worker-w0-46b586-openshift-default Scaled down replica set e2e-worker-w0-46b586-openshift-default-86c598c96c from 10 to 5 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-46b586-openshift-default New size: 5; reason: All metrics below target 47m Normal Killing pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-2qnns Stopping container istio-proxy 47m Normal SuccessfulDelete replicaset/e2e-worker-w0-46b586-openshift-default-86c598c96c Deleted pod: e2e-worker-w0-46b586-openshift-default-86c598c96c-4jhn8 47m Normal Killing pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-ccdzj Stopping container istio-proxy 47m Normal Killing pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-khcqd Stopping container istio-proxy 47m Normal SuccessfulDelete replicaset/e2e-worker-w0-46b586-openshift-default-86c598c96c Deleted pod: e2e-worker-w0-46b586-openshift-default-86c598c96c-ccdzj 47m Warning Unhealthy pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-xgpsz Readiness probe failed: HTTP probe failed with statuscode: 503 47m Normal Killing pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-tgc8q Stopping container istio-proxy 47m Normal SuccessfulDelete replicaset/e2e-worker-w0-46b586-openshift-default-86c598c96c Deleted pod: e2e-worker-w0-46b586-openshift-default-86c598c96c-kzqr4 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-71b76a-7b898f6878 Created pod: payload-processing-e2e-api-a-71b76a-7b898f6878-hmdwh 47m Normal SuccessfulDelete replicaset/e2e-worker-w0-46b586-openshift-default-86c598c96c Deleted pod: e2e-worker-w0-46b586-openshift-default-86c598c96c-jdcvx 47m Normal SuccessfulDelete replicaset/e2e-worker-w0-46b586-openshift-default-86c598c96c Deleted pod: e2e-worker-w0-46b586-openshift-default-86c598c96c-tgc8q 47m Normal Killing pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-4jhn8 Stopping container istio-proxy 47m Normal Killing pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-vb9r5 Stopping container istio-proxy 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-71b76a-64f77d669f Created pod: payload-pre-processing-e2e-api-a-71b76a-64f77d669f-hm2nn 47m Warning ProbeError pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-xgpsz Readiness probe error: HTTP probe failed with statuscode: 503... 47m Normal Pulled pod/payload-processing-e2e-api-a-71b76a-7b898f6878-hmdwh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-71b76a-64f77d669f-hm2nn Add eth0 [10.132.0.183/23] from ovn-kubernetes 47m Warning ProbeError pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-7gm84 Readiness probe error: HTTP probe failed with statuscode: 503... 47m Normal Created pod/payload-processing-e2e-api-a-71b76a-7b898f6878-hmdwh Container created 47m Normal AddedInterface pod/payload-processing-e2e-api-a-71b76a-7b898f6878-hmdwh Add eth0 [10.134.0.106/23] from ovn-kubernetes 47m Warning Unhealthy pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-94tkp Readiness probe failed: HTTP probe failed with statuscode: 503 47m Warning ProbeError pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-94tkp Readiness probe error: HTTP probe failed with statuscode: 503... 47m Normal Started pod/payload-processing-e2e-api-a-71b76a-7b898f6878-hmdwh Container started 47m Normal Pulled pod/payload-pre-processing-e2e-api-a-71b76a-64f77d669f-hm2nn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Warning Unhealthy pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-7gm84 Readiness probe failed: HTTP probe failed with statuscode: 503 47m Normal Created pod/payload-pre-processing-e2e-api-a-71b76a-64f77d669f-hm2nn Container created 47m Normal Started pod/payload-pre-processing-e2e-api-a-71b76a-64f77d669f-hm2nn Container started 47m Warning ProbeError pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-2qnns Readiness probe error: HTTP probe failed with statuscode: 503... 47m Warning ProbeError pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-nr4qz Readiness probe error: HTTP probe failed with statuscode: 503... 47m Warning Unhealthy pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-nr4qz Readiness probe failed: HTTP probe failed with statuscode: 503 47m Warning Unhealthy pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-2qnns Readiness probe failed: HTTP probe failed with statuscode: 503 47m Warning Unhealthy pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-ccdzj Readiness probe failed: HTTP probe failed with statuscode: 503 47m Warning Unhealthy pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-4jhn8 Readiness probe failed: HTTP probe failed with statuscode: 503 47m Warning ProbeError pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-klh6t Readiness probe error: HTTP probe failed with statuscode: 503... 47m Warning ProbeError pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-4jhn8 Readiness probe error: HTTP probe failed with statuscode: 503... 47m Warning ProbeError pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-ccdzj Readiness probe error: HTTP probe failed with statuscode: 503... 47m Warning Unhealthy pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-klh6t Readiness probe failed: HTTP probe failed with statuscode: 503 47m Warning ProbeError pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-kzqr4 Readiness probe error: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-kzqr4 Readiness probe failed: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused 47m Normal AddedLabel gateway/e2e-shared-b-w6-0d75be Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w6-0d75be 47m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-0d75be-openshift-default Scaled up replica set e2e-shared-b-w6-0d75be-openshift-default-57f99d9cbc from 0 to 1 47m Warning Unhealthy pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-vb9r5 Readiness probe failed: Get "http://10.132.0.137:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-0d75be-openshift-default-57f99d9cbc Created pod: e2e-shared-b-w6-0d75be-openshift-default-57f99d9cbc-smttn 47m Warning ProbeError pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-vb9r5 Readiness probe error: Get "http://10.132.0.137:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Normal AddedInterface pod/e2e-shared-b-w6-0d75be-openshift-default-57f99d9cbc-smttn Add eth0 [10.132.0.184/23] from ovn-kubernetes 47m Normal Started pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-wvq8z Container started 47m Normal Created pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-wvq8z Container created 47m Normal Pulled pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-wvq8z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Killing pod/e2e-derive-09efcb6d-openshift-default-7c46fd677c-6wtzg Stopping container istio-proxy 47m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-d5360e-openshift-default-fbccfc549 Created pod: e2e-shared-a-w0-d5360e-openshift-default-fbccfc549-d828w 47m Normal AddedLabel gateway/e2e-shared-a-w0-d5360e Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-d5360e 47m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-d5360e-openshift-default Scaled up replica set e2e-shared-a-w0-d5360e-openshift-default-fbccfc549 from 0 to 1 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-a251fdfb-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-a251fdfb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-f7b8d8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Normal AddedInterface pod/e2e-shared-a-w0-d5360e-openshift-default-fbccfc549-d828w Add eth0 [10.132.0.185/23] from ovn-kubernetes 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-f7b8d8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w6-0d75be-75f9855f5 Created pod: payload-pre-processing-e2e-shared-b-w6-0d75be-75f9855f5-kftq4 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w6-0d75be-75f9855f5 Error creating: pods "payload-pre-processing-e2e-shared-b-w6-0d75be-75f9855f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-0d75be: serviceaccount "payload-processing-e2e-shared-b-w6-0d75be" not found 47m Warning Unhealthy pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-wvq8z Startup probe failed: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused 47m Warning ProbeError pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-wvq8z Startup probe error: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused... 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w6-0d75be Scaled up replica set payload-pre-processing-e2e-shared-b-w6-0d75be-75f9855f5 from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w6-0d75be-7b66b84856 Created pod: payload-processing-e2e-shared-b-w6-0d75be-7b66b84856-j28w4 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w6-0d75be Scaled up replica set payload-processing-e2e-shared-b-w6-0d75be-7b66b84856 from 0 to 1 47m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w6-0d75be-7b66b84856 Error creating: pods "payload-processing-e2e-shared-b-w6-0d75be-7b66b84856-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-0d75be: serviceaccount "payload-processing-e2e-shared-b-w6-0d75be" not found 47m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w6-0d75be-75f9855f5-kftq4 Add eth0 [10.132.0.186/23] from ovn-kubernetes 47m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w6-0d75be-75f9855f5-kftq4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-processing-e2e-shared-b-w6-0d75be-7b66b84856-j28w4 Container created 47m Normal Started pod/payload-pre-processing-e2e-shared-b-w6-0d75be-75f9855f5-kftq4 Container started 47m Normal Created pod/payload-pre-processing-e2e-shared-b-w6-0d75be-75f9855f5-kftq4 Container created 47m Normal Killing pod/e2e-ait-7c8802f8-openshift-default-74499c95cc-v5qrl Stopping container istio-proxy 47m Normal Started pod/payload-processing-e2e-shared-b-w6-0d75be-7b66b84856-j28w4 Container started 47m Normal AddedInterface pod/payload-processing-e2e-shared-b-w6-0d75be-7b66b84856-j28w4 Add eth0 [10.132.0.187/23] from ovn-kubernetes 47m Normal Pulled pod/payload-processing-e2e-shared-b-w6-0d75be-7b66b84856-j28w4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-54ad8c-openshift-default Scaled up replica set e2e-shared-a-w4-54ad8c-openshift-default-575c545fdf from 0 to 1 47m Normal AddedLabel gateway/e2e-shared-a-w4-54ad8c Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-54ad8c 47m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-54ad8c-openshift-default-575c545fdf Created pod: e2e-shared-a-w4-54ad8c-openshift-default-575c545fdf-gvfbr 47m Normal AddedInterface pod/e2e-shared-a-w4-54ad8c-openshift-default-575c545fdf-gvfbr Add eth0 [10.132.0.188/23] from ovn-kubernetes 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-71b76a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-71b76a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w0-d5360e-b6fb568dc Error creating: pods "payload-pre-processing-e2e-shared-a-w0-d5360e-b6fb568dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-d5360e: serviceaccount "payload-processing-e2e-shared-a-w0-d5360e" not found 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w0-d5360e-854fc9bc5d Created pod: payload-processing-e2e-shared-a-w0-d5360e-854fc9bc5d-mc76v 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w0-d5360e Scaled up replica set payload-pre-processing-e2e-shared-a-w0-d5360e-b6fb568dc from 0 to 1 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w0-d5360e Scaled up replica set payload-processing-e2e-shared-a-w0-d5360e-854fc9bc5d from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w0-d5360e-b6fb568dc Created pod: payload-pre-processing-e2e-shared-a-w0-d5360e-b6fb568dc-htsps 47m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w0-d5360e-854fc9bc5d Error creating: pods "payload-processing-e2e-shared-a-w0-d5360e-854fc9bc5d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-d5360e: serviceaccount "payload-processing-e2e-shared-a-w0-d5360e" not found 47m Normal Started pod/payload-processing-e2e-shared-a-w0-d5360e-854fc9bc5d-mc76v Container started 47m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w0-d5360e-b6fb568dc-htsps Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Started pod/payload-pre-processing-e2e-shared-a-w0-d5360e-b6fb568dc-htsps Container started 47m Normal Pulled pod/payload-processing-e2e-shared-a-w0-d5360e-854fc9bc5d-mc76v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/payload-processing-e2e-shared-a-w0-d5360e-854fc9bc5d-mc76v Add eth0 [10.132.0.190/23] from ovn-kubernetes 47m Normal Created pod/payload-processing-e2e-shared-a-w0-d5360e-854fc9bc5d-mc76v Container created 47m Normal Created pod/payload-pre-processing-e2e-shared-a-w0-d5360e-b6fb568dc-htsps Container created 47m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w0-d5360e-b6fb568dc-htsps Add eth0 [10.132.0.189/23] from ovn-kubernetes 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-986d37-openshift-default New size: 8; reason: All metrics below target 46m Normal Killing pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-q7dtb Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc Deleted pod: e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-mlkdz 46m Normal ScalingReplicaSet deployment/e2e-worker-w3-986d37-openshift-default Scaled down replica set e2e-worker-w3-986d37-openshift-default-5bc8bd86fc from 10 to 8 46m Normal Killing pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-mlkdz Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc Deleted pod: e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-q7dtb 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w4-54ad8c Scaled up replica set payload-processing-e2e-shared-a-w4-54ad8c-7c679ffcbb from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w4-54ad8c-7b7f75c84c Created pod: payload-pre-processing-e2e-shared-a-w4-54ad8c-7b7f75c84c-nntg5 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w4-54ad8c-7b7f75c84c Error creating: pods "payload-pre-processing-e2e-shared-a-w4-54ad8c-7b7f75c84c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-54ad8c: serviceaccount "payload-processing-e2e-shared-a-w4-54ad8c" not found 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w4-54ad8c Scaled up replica set payload-pre-processing-e2e-shared-a-w4-54ad8c-7b7f75c84c from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w4-54ad8c-7c679ffcbb Created pod: payload-processing-e2e-shared-a-w4-54ad8c-7c679ffcbb-kmx4b 46m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w4-54ad8c-7c679ffcbb Error creating: pods "payload-processing-e2e-shared-a-w4-54ad8c-7c679ffcbb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-54ad8c: serviceaccount "payload-processing-e2e-shared-a-w4-54ad8c" not found 46m Normal AddedInterface pod/payload-processing-e2e-shared-a-w4-54ad8c-7c679ffcbb-kmx4b Add eth0 [10.132.0.193/23] from ovn-kubernetes 46m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w4-54ad8c-7b7f75c84c-nntg5 Add eth0 [10.132.0.192/23] from ovn-kubernetes 46m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w4-54ad8c-7b7f75c84c-nntg5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/payload-pre-processing-e2e-shared-a-w4-54ad8c-7b7f75c84c-nntg5 Container created 46m Normal Started pod/payload-pre-processing-e2e-shared-a-w4-54ad8c-7b7f75c84c-nntg5 Container started 46m Warning ProbeError pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-q7dtb Readiness probe error: HTTP probe failed with statuscode: 503... 46m Normal Pulled pod/payload-processing-e2e-shared-a-w4-54ad8c-7c679ffcbb-kmx4b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/payload-processing-e2e-shared-a-w4-54ad8c-7c679ffcbb-kmx4b Container created 46m Normal Started pod/payload-processing-e2e-shared-a-w4-54ad8c-7c679ffcbb-kmx4b Container started 46m Warning Unhealthy pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-q7dtb Readiness probe failed: HTTP probe failed with statuscode: 503 46m Normal SuccessfulDelete replicaset/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf Deleted pod: e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-7w9dq 46m Normal SuccessfulDelete replicaset/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf Deleted pod: e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-pc4b9 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-34f1ce-openshift-default New size: 6; reason: All metrics below target 46m Normal ScalingReplicaSet deployment/e2e-worker-w4-34f1ce-openshift-default Scaled down replica set e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf from 10 to 6 46m Normal SuccessfulDelete replicaset/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf Deleted pod: e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-hs4pf 46m Normal Killing pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-7w9dq Stopping container istio-proxy 46m Normal Killing pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-6gwbm Stopping container istio-proxy 46m Normal Killing pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-hs4pf Stopping container istio-proxy 46m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 3 46m Normal SuccessfulDelete replicaset/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf Deleted pod: e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-6gwbm 46m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-kzzph 46m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: cpu resource utilization (percentage of request) above target 46m Normal Killing pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-pc4b9 Stopping container istio-proxy 46m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-kzzph Add eth0 [10.134.0.107/23] from ovn-kubernetes 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-a251fdfb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-kzzph Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-a251fdfb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal SuccessfulDelete replicaset/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc Deleted pod: e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-4strh 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-986d37-openshift-default New size: 6; reason: All metrics below target 46m Normal ScalingReplicaSet deployment/e2e-worker-w3-986d37-openshift-default Scaled down replica set e2e-worker-w3-986d37-openshift-default-5bc8bd86fc from 8 to 6 46m Normal Killing pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-px9v2 Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc Deleted pod: e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-px9v2 46m Normal Killing pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-4strh Stopping container istio-proxy 46m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-kzzph Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.133s (4.133s including waiting). Image size: 185476477 bytes. 46m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-kzzph Container created 46m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-kzzph Container started 46m Warning ProbeError pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-6gwbm Readiness probe error: Get "http://10.132.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-6gwbm Readiness probe failed: Get "http://10.132.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning Unhealthy pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-px9v2 Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning ProbeError pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-px9v2 Readiness probe error: HTTP probe failed with statuscode: 503... 46m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-kzzph Readiness probe error: HTTP probe failed with statuscode: 503... 46m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-kzzph Readiness probe failed: HTTP probe failed with statuscode: 503 46m Normal AddedLabel gateway/e2e-shared-a-w3-ab0ac9 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-ab0ac9 46m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-ab0ac9-openshift-default-54c6546bdf Created pod: e2e-shared-a-w3-ab0ac9-openshift-default-54c6546bdf-lxwwq 46m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-ab0ac9-openshift-default Scaled up replica set e2e-shared-a-w3-ab0ac9-openshift-default-54c6546bdf from 0 to 1 46m Normal AddedInterface pod/e2e-shared-a-w3-ab0ac9-openshift-default-54c6546bdf-lxwwq Add eth0 [10.132.0.197/23] from ovn-kubernetes 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w3-ab0ac9-7978cbf74c Error creating: pods "payload-pre-processing-e2e-shared-a-w3-ab0ac9-7978cbf74c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-ab0ac9: serviceaccount "payload-processing-e2e-shared-a-w3-ab0ac9" not found 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w3-ab0ac9 Scaled up replica set payload-pre-processing-e2e-shared-a-w3-ab0ac9-7978cbf74c from 0 to 1 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w3-ab0ac9 Scaled up replica set payload-processing-e2e-shared-a-w3-ab0ac9-847986794b from 0 to 1 46m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w3-ab0ac9-847986794b Error creating: pods "payload-processing-e2e-shared-a-w3-ab0ac9-847986794b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-ab0ac9: serviceaccount "payload-processing-e2e-shared-a-w3-ab0ac9" not found 46m Normal Started pod/e2e-api-a-71b76a-openshift-default-77db967c97-lh4ld Container started 46m Normal Pulled pod/e2e-api-a-71b76a-openshift-default-77db967c97-lh4ld Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w3-ab0ac9-847986794b Created pod: payload-processing-e2e-shared-a-w3-ab0ac9-847986794b-bbf7n 46m Normal Created pod/e2e-api-a-71b76a-openshift-default-77db967c97-lh4ld Container created 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w3-ab0ac9-7978cbf74c Created pod: payload-pre-processing-e2e-shared-a-w3-ab0ac9-7978cbf74c-qxdvb 46m Normal Created pod/payload-processing-e2e-shared-a-w3-ab0ac9-847986794b-bbf7n Container created 46m Normal AddedInterface pod/payload-processing-e2e-shared-a-w3-ab0ac9-847986794b-bbf7n Add eth0 [10.132.0.200/23] from ovn-kubernetes 46m Normal Pulled pod/payload-processing-e2e-shared-a-w3-ab0ac9-847986794b-bbf7n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/payload-pre-processing-e2e-shared-a-w3-ab0ac9-7978cbf74c-qxdvb Container created 46m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w3-ab0ac9-7978cbf74c-qxdvb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Started pod/payload-pre-processing-e2e-shared-a-w3-ab0ac9-7978cbf74c-qxdvb Container started 46m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w3-ab0ac9-7978cbf74c-qxdvb Add eth0 [10.132.0.199/23] from ovn-kubernetes 46m Normal Started pod/payload-processing-e2e-shared-a-w3-ab0ac9-847986794b-bbf7n Container started 45m Warning Unhealthy pod/e2e-api-a-71b76a-openshift-default-77db967c97-lh4ld Startup probe failed: Get "http://10.132.0.182:15021/healthz/ready": dial tcp 10.132.0.182:15021: connect: connection refused 45m Warning ProbeError pod/e2e-api-a-71b76a-openshift-default-77db967c97-lh4ld Startup probe error: Get "http://10.132.0.182:15021/healthz/ready": dial tcp 10.132.0.182:15021: connect: connection refused... 45m Normal Killing pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8bt76 Stopping container istio-proxy 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-fe01fa-openshift-default New size: 7; reason: All metrics below target 45m Normal ScalingReplicaSet deployment/e2e-worker-w5-fe01fa-openshift-default Scaled down replica set e2e-worker-w5-fe01fa-openshift-default-564f5c6485 from 10 to 7 45m Normal SuccessfulDelete replicaset/e2e-worker-w5-fe01fa-openshift-default-564f5c6485 Deleted pod: e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8bt76 45m Normal SuccessfulDelete replicaset/e2e-worker-w5-fe01fa-openshift-default-564f5c6485 Deleted pod: e2e-worker-w5-fe01fa-openshift-default-564f5c6485-xkbnw 45m Normal SuccessfulDelete replicaset/e2e-worker-w5-fe01fa-openshift-default-564f5c6485 Deleted pod: e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t762r 45m Normal Killing pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-xkbnw Stopping container istio-proxy 45m Normal Killing pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t762r Stopping container istio-proxy 45m Warning Unhealthy pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t762r Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t762r Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning ProbeError pod/e2e-shared-a-w0-d5360e-openshift-default-fbccfc549-d828w Readiness probe error: Get "http://10.132.0.185:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning Unhealthy pod/e2e-shared-a-w0-d5360e-openshift-default-fbccfc549-d828w Readiness probe failed: Get "http://10.132.0.185:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-71b76a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-71b76a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Normal Created pod/e2e-shared-a-w0-d5360e-openshift-default-fbccfc549-d828w Container created 45m Normal Started pod/e2e-shared-a-w0-d5360e-openshift-default-fbccfc549-d828w Container started 45m Normal Pulled pod/e2e-shared-a-w0-d5360e-openshift-default-fbccfc549-d828w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/e2e-shared-b-w6-0d75be-openshift-default-57f99d9cbc-smttn Container started 45m Normal Created pod/e2e-shared-b-w6-0d75be-openshift-default-57f99d9cbc-smttn Container created 45m Normal Pulled pod/e2e-shared-b-w6-0d75be-openshift-default-57f99d9cbc-smttn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-d5360e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-d5360e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning Unhealthy pod/e2e-shared-b-w6-0d75be-openshift-default-57f99d9cbc-smttn Startup probe failed: Get "http://10.132.0.184:15021/healthz/ready": dial tcp 10.132.0.184:15021: connect: connection refused 45m Warning ProbeError pod/e2e-shared-b-w6-0d75be-openshift-default-57f99d9cbc-smttn Startup probe error: Get "http://10.132.0.184:15021/healthz/ready": dial tcp 10.132.0.184:15021: connect: connection refused... 45m Warning ProbeError pod/e2e-shared-a-w0-d5360e-openshift-default-fbccfc549-d828w Startup probe error: Get "http://10.132.0.185:15021/healthz/ready": dial tcp 10.132.0.185:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-shared-a-w0-d5360e-openshift-default-fbccfc549-d828w Startup probe failed: Get "http://10.132.0.185:15021/healthz/ready": dial tcp 10.132.0.185:15021: connect: connection refused 45m Normal AddedLabel gateway/e2e-api-b-39e3a8 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-39e3a8 45m Normal SuccessfulCreate replicaset/e2e-api-b-39e3a8-openshift-default-6b6658797 Created pod: e2e-api-b-39e3a8-openshift-default-6b6658797-ghqtx 45m Normal ScalingReplicaSet deployment/e2e-api-b-39e3a8-openshift-default Scaled up replica set e2e-api-b-39e3a8-openshift-default-6b6658797 from 0 to 1 45m Normal AddedInterface pod/e2e-api-b-39e3a8-openshift-default-6b6658797-ghqtx Add eth0 [10.132.0.202/23] from ovn-kubernetes 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-0d75be-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-0d75be-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-39e3a8-545496b975 Error creating: pods "payload-pre-processing-e2e-api-b-39e3a8-545496b975-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-39e3a8: serviceaccount "payload-processing-e2e-api-b-39e3a8" not found 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-39e3a8 Scaled up replica set payload-pre-processing-e2e-api-b-39e3a8-545496b975 from 0 to 1 45m Warning FailedCreate replicaset/payload-processing-e2e-api-b-39e3a8-77d67d47f8 Error creating: pods "payload-processing-e2e-api-b-39e3a8-77d67d47f8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-39e3a8: serviceaccount "payload-processing-e2e-api-b-39e3a8" not found 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-39e3a8 Scaled up replica set payload-processing-e2e-api-b-39e3a8-77d67d47f8 from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-39e3a8-545496b975 Created pod: payload-pre-processing-e2e-api-b-39e3a8-545496b975-zvh7k 45m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-39e3a8-545496b975-zvh7k Add eth0 [10.132.0.203/23] from ovn-kubernetes 45m Normal Pulled pod/payload-pre-processing-e2e-api-b-39e3a8-545496b975-zvh7k Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-39e3a8-77d67d47f8 Created pod: payload-processing-e2e-api-b-39e3a8-77d67d47f8-msm6p 45m Normal AddedInterface pod/payload-processing-e2e-api-b-39e3a8-77d67d47f8-msm6p Add eth0 [10.132.0.204/23] from ovn-kubernetes 45m Normal Started pod/payload-pre-processing-e2e-api-b-39e3a8-545496b975-zvh7k Container started 45m Normal Pulled pod/payload-processing-e2e-api-b-39e3a8-77d67d47f8-msm6p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-pre-processing-e2e-api-b-39e3a8-545496b975-zvh7k Container created 45m Normal Started pod/payload-processing-e2e-api-b-39e3a8-77d67d47f8-msm6p Container started 45m Normal Created pod/payload-processing-e2e-api-b-39e3a8-77d67d47f8-msm6p Container created 45m Normal AddedLabel gateway/e2e-inf-a-03c703 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-03c703 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-0d75be-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal ScalingReplicaSet deployment/e2e-inf-a-03c703-openshift-default Scaled up replica set e2e-inf-a-03c703-openshift-default-78bd587bff from 0 to 1 45m Normal SuccessfulCreate replicaset/e2e-inf-a-03c703-openshift-default-78bd587bff Created pod: e2e-inf-a-03c703-openshift-default-78bd587bff-5m2n8 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-0d75be-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal AddedInterface pod/e2e-inf-a-03c703-openshift-default-78bd587bff-5m2n8 Add eth0 [10.132.0.205/23] from ovn-kubernetes 44m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-957019-openshift-default-59bc4d44f9 Created pod: e2e-shared-b-w0-957019-openshift-default-59bc4d44f9-8xtcd 44m Normal AddedLabel gateway/e2e-shared-b-w0-957019 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-957019 44m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-957019-openshift-default Scaled up replica set e2e-shared-b-w0-957019-openshift-default-59bc4d44f9 from 0 to 1 44m Normal AddedInterface pod/e2e-shared-b-w0-957019-openshift-default-59bc4d44f9-8xtcd Add eth0 [10.132.0.207/23] from ovn-kubernetes 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-03c703-68ccf588f9 Error creating: pods "payload-pre-processing-e2e-inf-a-03c703-68ccf588f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-03c703: serviceaccount "payload-processing-e2e-inf-a-03c703" not found 44m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-03c703-587b89bbdd Error creating: pods "payload-processing-e2e-inf-a-03c703-587b89bbdd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-03c703: serviceaccount "payload-processing-e2e-inf-a-03c703" not found 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-03c703 Scaled up replica set payload-pre-processing-e2e-inf-a-03c703-68ccf588f9 from 0 to 1 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-03c703 Scaled up replica set payload-processing-e2e-inf-a-03c703-587b89bbdd from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-03c703-68ccf588f9 Created pod: payload-pre-processing-e2e-inf-a-03c703-68ccf588f9-957fw 44m Normal Created pod/e2e-shared-a-w4-54ad8c-openshift-default-575c545fdf-gvfbr Container created 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-03c703-587b89bbdd Created pod: payload-processing-e2e-inf-a-03c703-587b89bbdd-vzfkm 44m Normal Started pod/e2e-shared-a-w4-54ad8c-openshift-default-575c545fdf-gvfbr Container started 44m Normal Pulled pod/e2e-shared-a-w4-54ad8c-openshift-default-575c545fdf-gvfbr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-pre-processing-e2e-inf-a-03c703-68ccf588f9-957fw Container created 44m Normal Created pod/payload-processing-e2e-inf-a-03c703-587b89bbdd-vzfkm Container created 44m Normal Started pod/payload-processing-e2e-inf-a-03c703-587b89bbdd-vzfkm Container started 44m Normal Started pod/payload-pre-processing-e2e-inf-a-03c703-68ccf588f9-957fw Container started 44m Normal Pulled pod/payload-processing-e2e-inf-a-03c703-587b89bbdd-vzfkm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-processing-e2e-inf-a-03c703-587b89bbdd-vzfkm Add eth0 [10.132.0.209/23] from ovn-kubernetes 44m Normal Pulled pod/payload-pre-processing-e2e-inf-a-03c703-68ccf588f9-957fw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-03c703-68ccf588f9-957fw Add eth0 [10.132.0.208/23] from ovn-kubernetes 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-54ad8c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-54ad8c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-03c703-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-03c703-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w0-957019-788c577f9b Error creating: pods "payload-processing-e2e-shared-b-w0-957019-788c577f9b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-957019: serviceaccount "payload-processing-e2e-shared-b-w0-957019" not found 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w0-957019-76d548dcf Created pod: payload-pre-processing-e2e-shared-b-w0-957019-76d548dcf-bbqz5 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w0-957019-76d548dcf Error creating: pods "payload-pre-processing-e2e-shared-b-w0-957019-76d548dcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-957019: serviceaccount "payload-processing-e2e-shared-b-w0-957019" not found 44m Warning Unhealthy pod/e2e-shared-a-w4-54ad8c-openshift-default-575c545fdf-gvfbr Startup probe failed: Get "http://10.132.0.188:15021/healthz/ready": dial tcp 10.132.0.188:15021: connect: connection refused 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w0-957019-788c577f9b Created pod: payload-processing-e2e-shared-b-w0-957019-788c577f9b-2shs7 44m Warning ProbeError pod/e2e-shared-a-w4-54ad8c-openshift-default-575c545fdf-gvfbr Startup probe error: Get "http://10.132.0.188:15021/healthz/ready": dial tcp 10.132.0.188:15021: connect: connection refused... 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w0-957019 Scaled up replica set payload-processing-e2e-shared-b-w0-957019-788c577f9b from 0 to 1 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w0-957019 Scaled up replica set payload-pre-processing-e2e-shared-b-w0-957019-76d548dcf from 0 to 1 44m Normal Started pod/payload-pre-processing-e2e-shared-b-w0-957019-76d548dcf-bbqz5 Container started 44m Normal Created pod/payload-pre-processing-e2e-shared-b-w0-957019-76d548dcf-bbqz5 Container created 44m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w0-957019-76d548dcf-bbqz5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w0-957019-76d548dcf-bbqz5 Add eth0 [10.132.0.210/23] from ovn-kubernetes 44m Normal AddedInterface pod/payload-processing-e2e-shared-b-w0-957019-788c577f9b-2shs7 Add eth0 [10.132.0.211/23] from ovn-kubernetes 44m Normal Started pod/payload-processing-e2e-shared-b-w0-957019-788c577f9b-2shs7 Container started 44m Normal Created pod/payload-processing-e2e-shared-b-w0-957019-788c577f9b-2shs7 Container created 44m Normal Pulled pod/payload-processing-e2e-shared-b-w0-957019-788c577f9b-2shs7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-4a3dba-openshift-default Scaled up replica set e2e-shared-b-w4-4a3dba-openshift-default-dc9664475 from 0 to 1 44m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-4a3dba-openshift-default-dc9664475 Created pod: e2e-shared-b-w4-4a3dba-openshift-default-dc9664475-zxb5s 44m Normal AddedLabel gateway/e2e-shared-b-w4-4a3dba Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-4a3dba 44m Normal AddedInterface pod/e2e-shared-b-w4-4a3dba-openshift-default-dc9664475-zxb5s Add eth0 [10.132.0.212/23] from ovn-kubernetes 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-d5360e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-d5360e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal Killing pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-x59pr Stopping container istio-proxy 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-27e82b-openshift-default New size: 4; reason: All metrics below target 44m Normal Killing pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-qsdr8 Stopping container istio-proxy 44m Normal SuccessfulDelete replicaset/e2e-worker-w6-35a187-openshift-default-5b54c65f75 Deleted pod: e2e-worker-w6-35a187-openshift-default-5b54c65f75-4jrqx 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-35a187-openshift-default New size: 4; reason: All metrics below target 44m Normal Killing pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-4jrqx Stopping container istio-proxy 44m Normal ScalingReplicaSet deployment/e2e-worker-w6-35a187-openshift-default Scaled down replica set e2e-worker-w6-35a187-openshift-default-5b54c65f75 from 6 to 4 44m Normal Killing pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-dxpzb Stopping container istio-proxy 44m Normal SuccessfulDelete replicaset/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 Deleted pod: e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-x59pr 44m Normal ScalingReplicaSet deployment/e2e-worker-w2-27e82b-openshift-default Scaled down replica set e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 from 6 to 4 44m Normal SuccessfulDelete replicaset/e2e-worker-w6-35a187-openshift-default-5b54c65f75 Deleted pod: e2e-worker-w6-35a187-openshift-default-5b54c65f75-qsdr8 44m Normal SuccessfulDelete replicaset/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 Deleted pod: e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-dxpzb 44m Normal ScalingReplicaSet deployment/e2e-worker-w0-46b586-openshift-default Scaled down replica set e2e-worker-w0-46b586-openshift-default-86c598c96c from 5 to 4 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-46b586-openshift-default New size: 4; reason: All metrics below target 44m Normal SuccessfulDelete replicaset/e2e-worker-w0-46b586-openshift-default-86c598c96c Deleted pod: e2e-worker-w0-46b586-openshift-default-86c598c96c-thfjt 44m Normal Killing pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-thfjt Stopping container istio-proxy 44m Warning Unhealthy pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-qsdr8 Readiness probe failed: HTTP probe failed with statuscode: 503 44m Warning ProbeError pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-qsdr8 Readiness probe error: HTTP probe failed with statuscode: 503... 44m Warning ProbeError pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-thfjt Readiness probe error: HTTP probe failed with statuscode: 503... 44m Warning Unhealthy pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-4jrqx Readiness probe failed: HTTP probe failed with statuscode: 503 44m Warning Unhealthy pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-thfjt Readiness probe failed: HTTP probe failed with statuscode: 503 44m Warning ProbeError pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-4jrqx Readiness probe error: HTTP probe failed with statuscode: 503... 44m Warning Unhealthy pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-x59pr Readiness probe failed: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused 44m Warning ProbeError pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-x59pr Readiness probe error: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-dxpzb Readiness probe failed: HTTP probe failed with statuscode: 503 44m Warning ProbeError pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-dxpzb Readiness probe error: HTTP probe failed with statuscode: 503... 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w4-4a3dba Scaled up replica set payload-processing-e2e-shared-b-w4-4a3dba-57c6cf5fd4 from 0 to 1 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w4-4a3dba Scaled up replica set payload-pre-processing-e2e-shared-b-w4-4a3dba-576dcfd7cf from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w4-4a3dba-57c6cf5fd4 Created pod: payload-processing-e2e-shared-b-w4-4a3dba-57c6cf5fd4-n7zpn 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w4-4a3dba-576dcfd7cf Error creating: pods "payload-pre-processing-e2e-shared-b-w4-4a3dba-576dcfd7cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-4a3dba: serviceaccount "payload-processing-e2e-shared-b-w4-4a3dba" not found 44m Normal AddedInterface pod/payload-processing-e2e-shared-b-w4-4a3dba-57c6cf5fd4-n7zpn Add eth0 [10.133.0.80/23] from ovn-kubernetes 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w4-4a3dba-576dcfd7cf Created pod: payload-pre-processing-e2e-shared-b-w4-4a3dba-576dcfd7cf-6x2bs 44m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w4-4a3dba-57c6cf5fd4 Error creating: pods "payload-processing-e2e-shared-b-w4-4a3dba-57c6cf5fd4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-4a3dba: serviceaccount "payload-processing-e2e-shared-b-w4-4a3dba" not found 44m Normal Pulling pod/payload-pre-processing-e2e-shared-b-w4-4a3dba-576dcfd7cf-6x2bs Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 44m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w4-4a3dba-576dcfd7cf-6x2bs Add eth0 [10.133.0.79/23] from ovn-kubernetes 44m Normal Pulling pod/payload-processing-e2e-shared-b-w4-4a3dba-57c6cf5fd4-n7zpn Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 44m Normal Started pod/payload-processing-e2e-shared-b-w4-4a3dba-57c6cf5fd4-n7zpn Container started 44m Normal Created pod/payload-processing-e2e-shared-b-w4-4a3dba-57c6cf5fd4-n7zpn Container created 44m Normal Pulled pod/payload-processing-e2e-shared-b-w4-4a3dba-57c6cf5fd4-n7zpn Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.434s (3.434s including waiting). Image size: 125353965 bytes. 44m Normal Started pod/payload-pre-processing-e2e-shared-b-w4-4a3dba-576dcfd7cf-6x2bs Container started 44m Normal Created pod/payload-pre-processing-e2e-shared-b-w4-4a3dba-576dcfd7cf-6x2bs Container created 44m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w4-4a3dba-576dcfd7cf-6x2bs Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.678s (3.678s including waiting). Image size: 125353965 bytes. 44m Normal Killing pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-j2j46 Stopping container istio-proxy 44m Normal SuccessfulDelete replicaset/e2e-worker-w1-5c3374-openshift-default-d6ff699c7 Deleted pod: e2e-worker-w1-5c3374-openshift-default-d6ff699c7-j2j46 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-5c3374-openshift-default New size: 2; reason: All metrics below target 44m Normal ScalingReplicaSet deployment/e2e-worker-w1-5c3374-openshift-default Scaled down replica set e2e-worker-w1-5c3374-openshift-default-d6ff699c7 from 3 to 2 44m Warning ProbeError pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-j2j46 Readiness probe error: HTTP probe failed with statuscode: 503... 44m Warning Unhealthy pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-j2j46 Readiness probe failed: HTTP probe failed with statuscode: 503 44m Normal Created pod/e2e-shared-a-w3-ab0ac9-openshift-default-54c6546bdf-lxwwq Container created 44m Normal Pulled pod/e2e-shared-a-w3-ab0ac9-openshift-default-54c6546bdf-lxwwq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Started pod/e2e-shared-a-w3-ab0ac9-openshift-default-54c6546bdf-lxwwq Container started 44m Warning ProbeError pod/e2e-shared-a-w3-ab0ac9-openshift-default-54c6546bdf-lxwwq Startup probe error: Get "http://10.132.0.197:15021/healthz/ready": dial tcp 10.132.0.197:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-shared-a-w3-ab0ac9-openshift-default-54c6546bdf-lxwwq Startup probe failed: Get "http://10.132.0.197:15021/healthz/ready": dial tcp 10.132.0.197:15021: connect: connection refused 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-4a3dba-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-4a3dba-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-ab0ac9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-ab0ac9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-54ad8c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-54ad8c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal Killing pod/payload-pre-processing-e2e-mt-a251fdfb-6f8487684c-6qzl9 Stopping container payload-pre-processing 44m Normal Killing pod/payload-processing-e2e-mt-a251fdfb-6fb6bf7d6b-xmmgr Stopping container payload-processing 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-ab0ac9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-ab0ac9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning Unhealthy pod/payload-processing-e2e-mt-a251fdfb-6fb6bf7d6b-xmmgr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 43m Warning Unhealthy pod/payload-pre-processing-e2e-mt-a251fdfb-6f8487684c-6qzl9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 43m Warning Unhealthy pod/e2e-api-b-39e3a8-openshift-default-6b6658797-ghqtx Readiness probe failed: Get "http://10.132.0.202:15021/healthz/ready": read tcp 10.132.0.2:49362->10.132.0.202:15021: read: connection reset by peer 43m Warning ProbeError pod/e2e-api-b-39e3a8-openshift-default-6b6658797-ghqtx Readiness probe error: Get "http://10.132.0.202:15021/healthz/ready": read tcp 10.132.0.2:49362->10.132.0.202:15021: read: connection reset by peer... 43m Normal Started pod/e2e-api-b-39e3a8-openshift-default-6b6658797-ghqtx Container started 43m Normal Created pod/e2e-api-b-39e3a8-openshift-default-6b6658797-ghqtx Container created 43m Normal Pulled pod/e2e-api-b-39e3a8-openshift-default-6b6658797-ghqtx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-d9e782-openshift-default Scaled up replica set e2e-shared-b-w3-d9e782-openshift-default-7ccffd49b5 from 0 to 1 43m Normal AddedLabel gateway/e2e-shared-b-w3-d9e782 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-d9e782 43m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-d9e782-openshift-default-7ccffd49b5 Created pod: e2e-shared-b-w3-d9e782-openshift-default-7ccffd49b5-7k8qc 43m Normal AddedInterface pod/e2e-shared-b-w3-d9e782-openshift-default-7ccffd49b5-7k8qc Add eth0 [10.133.0.83/23] from ovn-kubernetes 43m Warning Unhealthy pod/e2e-api-b-39e3a8-openshift-default-6b6658797-ghqtx Startup probe failed: Get "http://10.132.0.202:15021/healthz/ready": dial tcp 10.132.0.202:15021: connect: connection refused 43m Warning ProbeError pod/e2e-api-b-39e3a8-openshift-default-6b6658797-ghqtx Startup probe error: Get "http://10.132.0.202:15021/healthz/ready": dial tcp 10.132.0.202:15021: connect: connection refused... 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w3-d9e782-67987686 Created pod: payload-processing-e2e-shared-b-w3-d9e782-67987686-hwzmf 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w3-d9e782 Scaled up replica set payload-processing-e2e-shared-b-w3-d9e782-67987686 from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w3-d9e782-588c96f65f Created pod: payload-pre-processing-e2e-shared-b-w3-d9e782-588c96f65f-jbm6v 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w3-d9e782-588c96f65f Error creating: pods "payload-pre-processing-e2e-shared-b-w3-d9e782-588c96f65f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-d9e782: serviceaccount "payload-processing-e2e-shared-b-w3-d9e782" not found 43m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w3-d9e782-67987686 Error creating: pods "payload-processing-e2e-shared-b-w3-d9e782-67987686-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-d9e782: serviceaccount "payload-processing-e2e-shared-b-w3-d9e782" not found 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w3-d9e782 Scaled up replica set payload-pre-processing-e2e-shared-b-w3-d9e782-588c96f65f from 0 to 1 43m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w3-d9e782-588c96f65f-jbm6v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-processing-e2e-shared-b-w3-d9e782-67987686-hwzmf Add eth0 [10.132.0.215/23] from ovn-kubernetes 43m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w3-d9e782-588c96f65f-jbm6v Add eth0 [10.133.0.84/23] from ovn-kubernetes 43m Normal Created pod/payload-pre-processing-e2e-shared-b-w3-d9e782-588c96f65f-jbm6v Container created 43m Normal Started pod/payload-pre-processing-e2e-shared-b-w3-d9e782-588c96f65f-jbm6v Container started 43m Normal Pulled pod/payload-processing-e2e-shared-b-w3-d9e782-67987686-hwzmf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-processing-e2e-shared-b-w3-d9e782-67987686-hwzmf Container created 43m Normal Started pod/payload-processing-e2e-shared-b-w3-d9e782-67987686-hwzmf Container started 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-39e3a8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-39e3a8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-39e3a8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-39e3a8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal Created pod/e2e-inf-a-03c703-openshift-default-78bd587bff-5m2n8 Container created 42m Normal Pulled pod/e2e-inf-a-03c703-openshift-default-78bd587bff-5m2n8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Started pod/e2e-inf-a-03c703-openshift-default-78bd587bff-5m2n8 Container started 42m Warning Unhealthy pod/e2e-inf-a-03c703-openshift-default-78bd587bff-5m2n8 Startup probe failed: Get "http://10.132.0.205:15021/healthz/ready": dial tcp 10.132.0.205:15021: connect: connection refused 42m Warning ProbeError pod/e2e-inf-a-03c703-openshift-default-78bd587bff-5m2n8 Startup probe error: Get "http://10.132.0.205:15021/healthz/ready": dial tcp 10.132.0.205:15021: connect: connection refused... 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-03c703-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-03c703-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Normal AddedLabel gateway/e2e-inf-a-348a33 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-348a33 42m Normal ScalingReplicaSet deployment/e2e-inf-a-348a33-openshift-default Scaled up replica set e2e-inf-a-348a33-openshift-default-5775c46d7 from 0 to 1 42m Normal SuccessfulCreate replicaset/e2e-inf-a-348a33-openshift-default-5775c46d7 Created pod: e2e-inf-a-348a33-openshift-default-5775c46d7-w9cm4 42m Normal AddedInterface pod/e2e-inf-a-348a33-openshift-default-5775c46d7-w9cm4 Add eth0 [10.133.0.85/23] from ovn-kubernetes 42m Normal Pulled pod/e2e-shared-b-w4-4a3dba-openshift-default-dc9664475-zxb5s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Started pod/e2e-shared-b-w4-4a3dba-openshift-default-dc9664475-zxb5s Container started 42m Normal Created pod/e2e-shared-b-w4-4a3dba-openshift-default-dc9664475-zxb5s Container created 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-348a33 Scaled up replica set payload-pre-processing-e2e-inf-a-348a33-6987f947f from 0 to 1 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-348a33 Scaled up replica set payload-processing-e2e-inf-a-348a33-7f4b555888 from 0 to 1 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-348a33-6987f947f Error creating: pods "payload-pre-processing-e2e-inf-a-348a33-6987f947f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-348a33: serviceaccount "payload-processing-e2e-inf-a-348a33" not found 42m Warning ProbeError pod/e2e-shared-b-w4-4a3dba-openshift-default-dc9664475-zxb5s Startup probe error: Get "http://10.132.0.212:15021/healthz/ready": dial tcp 10.132.0.212:15021: connect: connection refused... 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-348a33-6987f947f Created pod: payload-pre-processing-e2e-inf-a-348a33-6987f947f-cqp2m 42m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-348a33-7f4b555888 Error creating: pods "payload-processing-e2e-inf-a-348a33-7f4b555888-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-348a33: serviceaccount "payload-processing-e2e-inf-a-348a33" not found 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-348a33-7f4b555888 Created pod: payload-processing-e2e-inf-a-348a33-7f4b555888-dk2l2 42m Warning Unhealthy pod/e2e-shared-b-w4-4a3dba-openshift-default-dc9664475-zxb5s Startup probe failed: Get "http://10.132.0.212:15021/healthz/ready": dial tcp 10.132.0.212:15021: connect: connection refused 42m Normal Killing pod/payload-processing-e2e-api-a-71b76a-7b898f6878-hmdwh Stopping container payload-processing 42m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-348a33-6987f947f-cqp2m Add eth0 [10.132.0.217/23] from ovn-kubernetes 42m Normal Killing pod/payload-pre-processing-e2e-api-a-71b76a-64f77d669f-hm2nn Stopping container payload-pre-processing 42m Normal Pulled pod/payload-pre-processing-e2e-inf-a-348a33-6987f947f-cqp2m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Started pod/payload-processing-e2e-inf-a-348a33-7f4b555888-dk2l2 Container started 42m Normal AddedInterface pod/payload-processing-e2e-inf-a-348a33-7f4b555888-dk2l2 Add eth0 [10.133.0.86/23] from ovn-kubernetes 42m Normal Pulled pod/payload-processing-e2e-inf-a-348a33-7f4b555888-dk2l2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Created pod/payload-processing-e2e-inf-a-348a33-7f4b555888-dk2l2 Container created 42m Normal Started pod/payload-pre-processing-e2e-inf-a-348a33-6987f947f-cqp2m Container started 42m Normal Created pod/payload-pre-processing-e2e-inf-a-348a33-6987f947f-cqp2m Container created 42m Normal SuccessfulCreate replicaset/e2e-inf-b-460d4e-openshift-default-575d5fbfb7 Created pod: e2e-inf-b-460d4e-openshift-default-575d5fbfb7-ftjgz 42m Normal AddedLabel gateway/e2e-inf-b-460d4e Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-460d4e 42m Normal ScalingReplicaSet deployment/e2e-inf-b-460d4e-openshift-default Scaled up replica set e2e-inf-b-460d4e-openshift-default-575d5fbfb7 from 0 to 1 42m Normal AddedInterface pod/e2e-inf-b-460d4e-openshift-default-575d5fbfb7-ftjgz Add eth0 [10.132.0.218/23] from ovn-kubernetes 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-a251fdfb Scaled up replica set payload-processing-e2e-mt-a251fdfb-6fb6bf7d6b from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-a251fdfb-6fb6bf7d6b Created pod: payload-processing-e2e-mt-a251fdfb-6fb6bf7d6b-w2z4v 42m Warning FailedCreate replicaset/payload-processing-e2e-mt-a251fdfb-6fb6bf7d6b Error creating: pods "payload-processing-e2e-mt-a251fdfb-6fb6bf7d6b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-a251fdfb: serviceaccount "payload-processing-e2e-mt-a251fdfb" not found 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-a251fdfb-6f8487684c Created pod: payload-pre-processing-e2e-mt-a251fdfb-6f8487684c-s2hz7 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-a251fdfb Scaled up replica set payload-pre-processing-e2e-mt-a251fdfb-6f8487684c from 0 to 1 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-a251fdfb-6f8487684c Error creating: pods "payload-pre-processing-e2e-mt-a251fdfb-6f8487684c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-a251fdfb: serviceaccount "payload-processing-e2e-mt-a251fdfb" not found 42m Normal Pulled pod/payload-pre-processing-e2e-mt-a251fdfb-6f8487684c-s2hz7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Started pod/payload-processing-e2e-mt-a251fdfb-6fb6bf7d6b-w2z4v Container started 42m Normal AddedInterface pod/payload-processing-e2e-mt-a251fdfb-6fb6bf7d6b-w2z4v Add eth0 [10.133.0.87/23] from ovn-kubernetes 42m Normal AddedInterface pod/payload-pre-processing-e2e-mt-a251fdfb-6f8487684c-s2hz7 Add eth0 [10.134.0.111/23] from ovn-kubernetes 42m Normal Started pod/payload-pre-processing-e2e-mt-a251fdfb-6f8487684c-s2hz7 Container started 42m Normal Pulled pod/payload-processing-e2e-mt-a251fdfb-6fb6bf7d6b-w2z4v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Created pod/payload-processing-e2e-mt-a251fdfb-6fb6bf7d6b-w2z4v Container created 42m Normal Created pod/payload-pre-processing-e2e-mt-a251fdfb-6f8487684c-s2hz7 Container created 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-460d4e Scaled up replica set payload-processing-e2e-inf-b-460d4e-6457b77684 from 0 to 1 42m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-460d4e-6457b77684 Error creating: pods "payload-processing-e2e-inf-b-460d4e-6457b77684-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-460d4e: serviceaccount "payload-processing-e2e-inf-b-460d4e" not found 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-460d4e-6457b77684 Created pod: payload-processing-e2e-inf-b-460d4e-6457b77684-dttcx 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-460d4e Scaled up replica set payload-pre-processing-e2e-inf-b-460d4e-c97d659 from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-460d4e-c97d659 Created pod: payload-pre-processing-e2e-inf-b-460d4e-c97d659-rcrqj 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-460d4e-c97d659 Error creating: pods "payload-pre-processing-e2e-inf-b-460d4e-c97d659-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-460d4e: serviceaccount "payload-processing-e2e-inf-b-460d4e" not found 42m Normal Started pod/payload-pre-processing-e2e-inf-b-460d4e-c97d659-rcrqj Container started 42m Normal Created pod/payload-pre-processing-e2e-inf-b-460d4e-c97d659-rcrqj Container created 42m Normal Created pod/payload-processing-e2e-inf-b-460d4e-6457b77684-dttcx Container created 42m Normal Pulled pod/payload-pre-processing-e2e-inf-b-460d4e-c97d659-rcrqj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-460d4e-c97d659-rcrqj Add eth0 [10.133.0.88/23] from ovn-kubernetes 42m Normal AddedInterface pod/payload-processing-e2e-inf-b-460d4e-6457b77684-dttcx Add eth0 [10.132.0.219/23] from ovn-kubernetes 42m Normal Pulled pod/payload-processing-e2e-inf-b-460d4e-6457b77684-dttcx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Started pod/payload-processing-e2e-inf-b-460d4e-6457b77684-dttcx Container started 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-4a3dba-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-4a3dba-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-35a187-openshift-default New size: 3; reason: All metrics below target 41m Normal ScalingReplicaSet deployment/e2e-worker-w6-35a187-openshift-default Scaled down replica set e2e-worker-w6-35a187-openshift-default-5b54c65f75 from 4 to 3 41m Normal SuccessfulDelete replicaset/e2e-worker-w6-35a187-openshift-default-5b54c65f75 Deleted pod: e2e-worker-w6-35a187-openshift-default-5b54c65f75-txzbm 41m Normal Killing pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-txzbm Stopping container istio-proxy 41m Normal AddedLabel gateway/e2e-api-a-c9c21e Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-c9c21e 41m Warning ProbeError pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-txzbm Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-txzbm Readiness probe failed: HTTP probe failed with statuscode: 503 41m Normal ScalingReplicaSet deployment/e2e-api-a-c9c21e-openshift-default Scaled up replica set e2e-api-a-c9c21e-openshift-default-749bcfc96c from 0 to 1 41m Normal SuccessfulCreate replicaset/e2e-api-a-c9c21e-openshift-default-749bcfc96c Created pod: e2e-api-a-c9c21e-openshift-default-749bcfc96c-wjzsh 41m Normal AddedInterface pod/e2e-api-a-c9c21e-openshift-default-749bcfc96c-wjzsh Add eth0 [10.132.0.220/23] from ovn-kubernetes 41m Warning Unhealthy pod/e2e-shared-b-w0-957019-openshift-default-59bc4d44f9-8xtcd Readiness probe failed: Get "http://10.132.0.207:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Warning ProbeError pod/e2e-shared-b-w0-957019-openshift-default-59bc4d44f9-8xtcd Readiness probe error: Get "http://10.132.0.207:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Normal Started pod/e2e-shared-b-w0-957019-openshift-default-59bc4d44f9-8xtcd Container started 41m Normal Pulled pod/e2e-shared-b-w0-957019-openshift-default-59bc4d44f9-8xtcd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-shared-b-w0-957019-openshift-default-59bc4d44f9-8xtcd Container created 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-c9c21e Scaled up replica set payload-processing-e2e-api-a-c9c21e-588d55cd7b from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-c9c21e-588d55cd7b Created pod: payload-processing-e2e-api-a-c9c21e-588d55cd7b-pk2ss 41m Warning FailedCreate replicaset/payload-processing-e2e-api-a-c9c21e-588d55cd7b Error creating: pods "payload-processing-e2e-api-a-c9c21e-588d55cd7b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-c9c21e: serviceaccount "payload-processing-e2e-api-a-c9c21e" not found 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-c9c21e Scaled up replica set payload-pre-processing-e2e-api-a-c9c21e-747fd5b8cf from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-c9c21e-747fd5b8cf Created pod: payload-pre-processing-e2e-api-a-c9c21e-747fd5b8cf-9s6gf 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-c9c21e-747fd5b8cf Error creating: pods "payload-pre-processing-e2e-api-a-c9c21e-747fd5b8cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-c9c21e: serviceaccount "payload-processing-e2e-api-a-c9c21e" not found 41m Normal AddedInterface pod/payload-processing-e2e-api-a-c9c21e-588d55cd7b-pk2ss Add eth0 [10.133.0.90/23] from ovn-kubernetes 41m Normal Started pod/payload-pre-processing-e2e-api-a-c9c21e-747fd5b8cf-9s6gf Container started 41m Normal Pulled pod/payload-processing-e2e-api-a-c9c21e-588d55cd7b-pk2ss Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-processing-e2e-api-a-c9c21e-588d55cd7b-pk2ss Container created 41m Normal SuccessfulDelete replicaset/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf Deleted pod: e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-smvsh 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-34f1ce-openshift-default New size: 4; reason: All metrics below target 41m Normal Killing pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-hnqpt Stopping container istio-proxy 41m Normal Started pod/payload-processing-e2e-api-a-c9c21e-588d55cd7b-pk2ss Container started 41m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-c9c21e-747fd5b8cf-9s6gf Add eth0 [10.133.0.89/23] from ovn-kubernetes 41m Normal ScalingReplicaSet deployment/e2e-worker-w4-34f1ce-openshift-default Scaled down replica set e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf from 6 to 4 41m Normal SuccessfulDelete replicaset/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf Deleted pod: e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-hnqpt 41m Normal Killing pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-smvsh Stopping container istio-proxy 41m Normal Created pod/payload-pre-processing-e2e-api-a-c9c21e-747fd5b8cf-9s6gf Container created 41m Normal Pulled pod/payload-pre-processing-e2e-api-a-c9c21e-747fd5b8cf-9s6gf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Warning Unhealthy pod/e2e-shared-b-w0-957019-openshift-default-59bc4d44f9-8xtcd Startup probe failed: Get "http://10.132.0.207:15021/healthz/ready": dial tcp 10.132.0.207:15021: connect: connection refused 41m Warning ProbeError pod/e2e-shared-b-w0-957019-openshift-default-59bc4d44f9-8xtcd Startup probe error: Get "http://10.132.0.207:15021/healthz/ready": dial tcp 10.132.0.207:15021: connect: connection refused... 41m Normal SuccessfulDelete replicaset/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc Deleted pod: e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-mgtkw 41m Normal SuccessfulDelete replicaset/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc Deleted pod: e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-t4ms2 41m Normal Killing pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-t4ms2 Stopping container istio-proxy 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-986d37-openshift-default New size: 4; reason: All metrics below target 41m Normal Killing pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-mgtkw Stopping container istio-proxy 41m Normal ScalingReplicaSet deployment/e2e-worker-w3-986d37-openshift-default Scaled down replica set e2e-worker-w3-986d37-openshift-default-5bc8bd86fc from 6 to 4 41m Warning Unhealthy pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-z6jt4 Readiness probe failed: HTTP probe failed with statuscode: 503 41m Normal SuccessfulDelete replicaset/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 Deleted pod: e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-z6jt4 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-27e82b-openshift-default New size: 3; reason: All metrics below target 41m Normal Killing pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-z6jt4 Stopping container istio-proxy 41m Warning ProbeError pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-z6jt4 Readiness probe error: HTTP probe failed with statuscode: 503... 41m Normal ScalingReplicaSet deployment/e2e-worker-w5-fe01fa-openshift-default Scaled down replica set e2e-worker-w5-fe01fa-openshift-default-564f5c6485 from 7 to 5 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-46b586-openshift-default New size: 3; reason: All metrics below target 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-fe01fa-openshift-default New size: 5; reason: All metrics below target 41m Normal SuccessfulDelete replicaset/e2e-worker-w5-fe01fa-openshift-default-564f5c6485 Deleted pod: e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8qtdz 41m Normal SuccessfulDelete replicaset/e2e-worker-w5-fe01fa-openshift-default-564f5c6485 Deleted pod: e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t5hkt 41m Normal SuccessfulDelete replicaset/e2e-worker-w0-46b586-openshift-default-86c598c96c Deleted pod: e2e-worker-w0-46b586-openshift-default-86c598c96c-q6bx8 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-957019-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal Killing pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-q6bx8 Stopping container istio-proxy 41m Normal ScalingReplicaSet deployment/e2e-worker-w0-46b586-openshift-default Scaled down replica set e2e-worker-w0-46b586-openshift-default-86c598c96c from 4 to 3 41m Warning ProbeError pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-q6bx8 Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-957019-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning Unhealthy pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-q6bx8 Readiness probe failed: HTTP probe failed with statuscode: 503 41m Normal Killing pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t5hkt Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8qtdz Stopping container istio-proxy 41m Warning ProbeError pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t5hkt Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-t5hkt Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning Unhealthy pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8qtdz Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning ProbeError pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8qtdz Readiness probe error: HTTP probe failed with statuscode: 503... 41m Normal Killing pod/payload-processing-e2e-shared-a-w4-54ad8c-7c679ffcbb-kmx4b Stopping container payload-processing 41m Normal Killing pod/payload-pre-processing-e2e-shared-a-w4-54ad8c-7b7f75c84c-nntg5 Stopping container payload-pre-processing 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-957019-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-957019-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal Pulled pod/e2e-shared-b-w3-d9e782-openshift-default-7ccffd49b5-7k8qc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-d9e782-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-d9e782-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal Created pod/e2e-shared-b-w3-d9e782-openshift-default-7ccffd49b5-7k8qc Container created 41m Normal Started pod/e2e-shared-b-w3-d9e782-openshift-default-7ccffd49b5-7k8qc Container started 40m Warning Unhealthy pod/e2e-shared-b-w3-d9e782-openshift-default-7ccffd49b5-7k8qc Startup probe failed: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused 40m Warning ProbeError pod/e2e-shared-b-w3-d9e782-openshift-default-7ccffd49b5-7k8qc Startup probe error: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused... 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-d9e782-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-d9e782-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal Killing pod/payload-pre-processing-e2e-shared-a-w3-ab0ac9-7978cbf74c-qxdvb Stopping container payload-pre-processing 40m Normal Killing pod/payload-processing-e2e-shared-a-w3-ab0ac9-847986794b-bbf7n Stopping container payload-processing 40m Warning Unhealthy pod/e2e-inf-a-348a33-openshift-default-5775c46d7-w9cm4 Readiness probe failed: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused 40m Warning ProbeError pod/e2e-inf-a-348a33-openshift-default-5775c46d7-w9cm4 Readiness probe error: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused... 40m Normal Started pod/e2e-inf-a-348a33-openshift-default-5775c46d7-w9cm4 Container started 40m Normal Created pod/e2e-inf-a-348a33-openshift-default-5775c46d7-w9cm4 Container created 40m Normal Pulled pod/e2e-inf-a-348a33-openshift-default-5775c46d7-w9cm4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-8f6f87f59 Deleted pod: maas-default-gateway-openshift-default-8f6f87f59-zvnnz 40m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-8f6f87f59 from 10 to 7 40m Normal Killing pod/maas-default-gateway-openshift-default-8f6f87f59-dpcqs Stopping container istio-proxy 40m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: All metrics below target 40m Normal Killing pod/maas-default-gateway-openshift-default-8f6f87f59-8fws9 Stopping container istio-proxy 40m Normal Killing pod/maas-default-gateway-openshift-default-8f6f87f59-zvnnz Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-8f6f87f59 Deleted pod: maas-default-gateway-openshift-default-8f6f87f59-dpcqs 40m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-8f6f87f59 Deleted pod: maas-default-gateway-openshift-default-8f6f87f59-8fws9 40m Normal Killing pod/e2e-shared-a-w4-54ad8c-openshift-default-575c545fdf-gvfbr Stopping container istio-proxy 40m Warning Unhealthy pod/e2e-inf-a-348a33-openshift-default-5775c46d7-w9cm4 Startup probe failed: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused 40m Warning ProbeError pod/e2e-inf-a-348a33-openshift-default-5775c46d7-w9cm4 Startup probe error: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused... 40m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-zvnnz Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-zvnnz Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-dpcqs Readiness probe failed: Get "http://10.132.0.161:15021/healthz/ready": dial tcp 10.132.0.161:15021: connect: connection refused 40m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-dpcqs Readiness probe error: Get "http://10.132.0.161:15021/healthz/ready": dial tcp 10.132.0.161:15021: connect: connection refused... 40m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-8fws9 Readiness probe error: Get "http://10.132.0.160:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-8fws9 Readiness probe failed: Get "http://10.132.0.160:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 39m Normal ScalingReplicaSet deployment/e2e-mt-a251fdfb-openshift-default Scaled up replica set e2e-mt-a251fdfb-openshift-default-7f7f49cc57 from 1 to 4 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-a251fdfb-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 39m Normal SuccessfulCreate replicaset/e2e-mt-a251fdfb-openshift-default-7f7f49cc57 Created pod: e2e-mt-a251fdfb-openshift-default-7f7f49cc57-c55xx 39m Normal SuccessfulCreate replicaset/e2e-mt-a251fdfb-openshift-default-7f7f49cc57 Created pod: e2e-mt-a251fdfb-openshift-default-7f7f49cc57-j9skc 39m Normal SuccessfulCreate replicaset/e2e-mt-a251fdfb-openshift-default-7f7f49cc57 Created pod: e2e-mt-a251fdfb-openshift-default-7f7f49cc57-q6whh 39m Normal AddedInterface pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-q6whh Add eth0 [10.132.0.226/23] from ovn-kubernetes 39m Normal Pulled pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-c55xx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Started pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-c55xx Container started 39m Normal Created pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-c55xx Container created 39m Normal Killing pod/payload-processing-e2e-shared-b-w4-4a3dba-57c6cf5fd4-n7zpn Stopping container payload-processing 39m Normal AddedInterface pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-j9skc Add eth0 [10.134.0.128/23] from ovn-kubernetes 39m Normal Pulled pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-j9skc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Started pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-j9skc Container started 39m Normal Created pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-j9skc Container created 39m Normal Started pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-q6whh Container started 39m Normal AddedInterface pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-c55xx Add eth0 [10.132.0.227/23] from ovn-kubernetes 39m Normal Pulled pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-q6whh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-q6whh Container created 39m Normal Killing pod/payload-pre-processing-e2e-shared-b-w4-4a3dba-576dcfd7cf-6x2bs Stopping container payload-pre-processing 39m Warning Unhealthy pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-q6whh Startup probe failed: Get "http://10.132.0.226:15021/healthz/ready": dial tcp 10.132.0.226:15021: connect: connection refused 39m Warning ProbeError pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-q6whh Startup probe error: Get "http://10.132.0.226:15021/healthz/ready": dial tcp 10.132.0.226:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-c55xx Startup probe failed: Get "http://10.132.0.227:15021/healthz/ready": dial tcp 10.132.0.227:15021: connect: connection refused 39m Warning ProbeError pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-j9skc Startup probe error: Get "http://10.134.0.128:15021/healthz/ready": dial tcp 10.134.0.128:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-j9skc Startup probe failed: Get "http://10.134.0.128:15021/healthz/ready": dial tcp 10.134.0.128:15021: connect: connection refused 39m Warning ProbeError pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-c55xx Startup probe error: Get "http://10.132.0.227:15021/healthz/ready": dial tcp 10.132.0.227:15021: connect: connection refused... 39m Warning Unhealthy pod/payload-processing-e2e-shared-b-w4-4a3dba-57c6cf5fd4-n7zpn Readiness probe failed: timeout: failed to connect service "10.133.0.80:50052" within 1s: context deadline exceeded 39m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w4-4a3dba-576dcfd7cf-6x2bs Readiness probe failed: timeout: failed to connect service "10.133.0.79:50052" within 1s: context deadline exceeded 39m Normal Pulled pod/e2e-inf-b-460d4e-openshift-default-575d5fbfb7-ftjgz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-inf-b-460d4e-openshift-default-575d5fbfb7-ftjgz Container created 39m Normal Started pod/e2e-inf-b-460d4e-openshift-default-575d5fbfb7-ftjgz Container started 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-460d4e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-348a33-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-348a33-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-460d4e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Normal ScalingReplicaSet deployment/e2e-mt-a251fdfb-openshift-default Scaled up replica set e2e-mt-a251fdfb-openshift-default-7f7f49cc57 from 4 to 8 39m Normal SuccessfulCreate replicaset/e2e-mt-a251fdfb-openshift-default-7f7f49cc57 Created pod: e2e-mt-a251fdfb-openshift-default-7f7f49cc57-kl5fq 39m Normal SuccessfulCreate replicaset/e2e-mt-a251fdfb-openshift-default-7f7f49cc57 Created pod: e2e-mt-a251fdfb-openshift-default-7f7f49cc57-fxvfs 39m Normal SuccessfulCreate replicaset/e2e-mt-a251fdfb-openshift-default-7f7f49cc57 Created pod: e2e-mt-a251fdfb-openshift-default-7f7f49cc57-q6d2w 39m Normal SuccessfulCreate replicaset/e2e-mt-a251fdfb-openshift-default-7f7f49cc57 Created pod: e2e-mt-a251fdfb-openshift-default-7f7f49cc57-p8ljt 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-a251fdfb-openshift-default New size: 8; reason: 39m Normal Started pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-kl5fq Container started 39m Normal AddedInterface pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-kl5fq Add eth0 [10.133.0.98/23] from ovn-kubernetes 39m Normal Started pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-p8ljt Container started 39m Normal Created pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-q6d2w Container created 39m Normal Started pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-q6d2w Container started 39m Normal Pulled pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-q6d2w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-fxvfs Add eth0 [10.132.0.228/23] from ovn-kubernetes 39m Normal AddedInterface pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-p8ljt Add eth0 [10.133.0.97/23] from ovn-kubernetes 39m Normal Pulled pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-fxvfs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-fxvfs Container created 39m Normal Started pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-fxvfs Container started 39m Normal Pulled pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-p8ljt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-p8ljt Container created 39m Normal AddedInterface pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-q6d2w Add eth0 [10.134.0.129/23] from ovn-kubernetes 39m Normal Created pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-kl5fq Container created 39m Normal Pulled pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-kl5fq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Warning Unhealthy pod/e2e-inf-b-460d4e-openshift-default-575d5fbfb7-ftjgz Startup probe failed: Get "http://10.132.0.218:15021/healthz/ready": dial tcp 10.132.0.218:15021: connect: connection refused 39m Warning ProbeError pod/e2e-inf-b-460d4e-openshift-default-575d5fbfb7-ftjgz Startup probe error: Get "http://10.132.0.218:15021/healthz/ready": dial tcp 10.132.0.218:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-kl5fq Startup probe failed: Get "http://10.133.0.98:15021/healthz/ready": dial tcp 10.133.0.98:15021: connect: connection refused 39m Warning ProbeError pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-kl5fq Startup probe error: Get "http://10.133.0.98:15021/healthz/ready": dial tcp 10.133.0.98:15021: connect: connection refused... 39m Warning ProbeError pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-p8ljt Startup probe error: Get "http://10.133.0.97:15021/healthz/ready": dial tcp 10.133.0.97:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-p8ljt Startup probe failed: Get "http://10.133.0.97:15021/healthz/ready": dial tcp 10.133.0.97:15021: connect: connection refused 39m Warning Unhealthy pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-q6d2w Startup probe failed: Get "http://10.134.0.129:15021/healthz/ready": dial tcp 10.134.0.129:15021: connect: connection refused 39m Warning ProbeError pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-q6d2w Startup probe error: Get "http://10.134.0.129:15021/healthz/ready": dial tcp 10.134.0.129:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-fxvfs Startup probe failed: Get "http://10.132.0.228:15021/healthz/ready": dial tcp 10.132.0.228:15021: connect: connection refused 39m Warning ProbeError pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-fxvfs Startup probe error: Get "http://10.132.0.228:15021/healthz/ready": dial tcp 10.132.0.228:15021: connect: connection refused... 39m Normal Killing pod/e2e-api-a-71b76a-openshift-default-77db967c97-lh4ld Stopping container istio-proxy 39m Warning ProbeError pod/e2e-api-a-71b76a-openshift-default-77db967c97-lh4ld Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-api-a-71b76a-openshift-default-77db967c97-lh4ld Readiness probe failed: HTTP probe failed with statuscode: 503 39m Normal ScalingReplicaSet deployment/e2e-mt-a251fdfb-openshift-default Scaled up replica set e2e-mt-a251fdfb-openshift-default-7f7f49cc57 from 8 to 10 39m Normal SuccessfulCreate replicaset/e2e-mt-a251fdfb-openshift-default-7f7f49cc57 Created pod: e2e-mt-a251fdfb-openshift-default-7f7f49cc57-z9l6k 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-a251fdfb-openshift-default New size: 10; reason: 39m Normal SuccessfulCreate replicaset/e2e-mt-a251fdfb-openshift-default-7f7f49cc57 (combined from similar events): Created pod: e2e-mt-a251fdfb-openshift-default-7f7f49cc57-qmsfx 39m Normal Pulled pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-z9l6k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Started pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-z9l6k Container started 39m Normal Started pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-qmsfx Container started 39m Normal AddedInterface pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-z9l6k Add eth0 [10.134.0.132/23] from ovn-kubernetes 39m Normal AddedInterface pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-qmsfx Add eth0 [10.132.0.229/23] from ovn-kubernetes 39m Normal Pulled pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-qmsfx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-qmsfx Container created 39m Normal Created pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-z9l6k Container created 39m Warning ProbeError pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-z9l6k Startup probe error: Get "http://10.134.0.132:15021/healthz/ready": dial tcp 10.134.0.132:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-z9l6k Startup probe failed: Get "http://10.134.0.132:15021/healthz/ready": dial tcp 10.134.0.132:15021: connect: connection refused 39m Warning Unhealthy pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-qmsfx Startup probe failed: Get "http://10.132.0.229:15021/healthz/ready": dial tcp 10.132.0.229:15021: connect: connection refused 39m Warning ProbeError pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-qmsfx Startup probe error: Get "http://10.132.0.229:15021/healthz/ready": dial tcp 10.132.0.229:15021: connect: connection refused... 39m Normal Killing pod/payload-pre-processing-e2e-inf-a-348a33-6987f947f-cqp2m Stopping container payload-pre-processing 39m Normal Killing pod/payload-processing-e2e-inf-a-348a33-7f4b555888-dk2l2 Stopping container payload-processing 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-460d4e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-460d4e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal Killing pod/payload-pre-processing-e2e-api-b-39e3a8-545496b975-zvh7k Stopping container payload-pre-processing 39m Normal Killing pod/payload-processing-e2e-api-b-39e3a8-77d67d47f8-msm6p Stopping container payload-processing 39m Normal Killing pod/payload-pre-processing-e2e-mt-a251fdfb-6f8487684c-s2hz7 Stopping container payload-pre-processing 39m Normal Killing pod/payload-processing-e2e-mt-a251fdfb-6fb6bf7d6b-w2z4v Stopping container payload-processing 39m Normal Killing pod/e2e-inf-a-348a33-openshift-default-5775c46d7-w9cm4 Stopping container istio-proxy 38m Normal Killing pod/payload-processing-e2e-inf-a-03c703-587b89bbdd-vzfkm Stopping container payload-processing 38m Normal Killing pod/payload-pre-processing-e2e-inf-a-03c703-68ccf588f9-957fw Stopping container payload-pre-processing 38m Normal Pulled pod/e2e-api-a-c9c21e-openshift-default-749bcfc96c-wjzsh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/e2e-api-a-c9c21e-openshift-default-749bcfc96c-wjzsh Container created 38m Normal Started pod/e2e-api-a-c9c21e-openshift-default-749bcfc96c-wjzsh Container started 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-c9c21e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-c9c21e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Normal Killing pod/e2e-shared-a-w3-ab0ac9-openshift-default-54c6546bdf-lxwwq Stopping container istio-proxy 38m Warning ProbeError pod/e2e-api-a-c9c21e-openshift-default-749bcfc96c-wjzsh Startup probe error: Get "http://10.132.0.220:15021/healthz/ready": dial tcp 10.132.0.220:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-api-a-c9c21e-openshift-default-749bcfc96c-wjzsh Startup probe failed: Get "http://10.132.0.220:15021/healthz/ready": dial tcp 10.132.0.220:15021: connect: connection refused 38m Normal Killing pod/payload-pre-processing-e2e-worker-w2-27e82b-5968f9f875-t9qz2 Stopping container payload-pre-processing 38m Normal Killing pod/payload-processing-e2e-worker-w2-27e82b-64f94b9586-wb68f Stopping container payload-processing 38m Warning Unhealthy pod/e2e-shared-a-w3-ab0ac9-openshift-default-54c6546bdf-lxwwq Readiness probe failed: Get "http://10.132.0.197:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/e2e-shared-a-w3-ab0ac9-openshift-default-54c6546bdf-lxwwq Readiness probe error: Get "http://10.132.0.197:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Normal Killing pod/payload-processing-e2e-api-a-c9c21e-588d55cd7b-pk2ss Stopping container payload-processing 38m Normal Killing pod/payload-pre-processing-e2e-api-a-c9c21e-747fd5b8cf-9s6gf Stopping container payload-pre-processing 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-c9c21e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-c9c21e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Normal Killing pod/payload-processing-e2e-shared-b-w3-d9e782-67987686-hwzmf Stopping container payload-processing 38m Normal Killing pod/payload-pre-processing-e2e-shared-b-w3-d9e782-588c96f65f-jbm6v Stopping container payload-pre-processing 38m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-27e82b-5968f9f875-t9qz2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 38m Warning Unhealthy pod/payload-processing-e2e-worker-w2-27e82b-64f94b9586-wb68f Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 38m Normal Killing pod/e2e-api-a-c9c21e-openshift-default-749bcfc96c-wjzsh Stopping container istio-proxy 38m Normal ScalingReplicaSet deployment/e2e-inf-a-76b073-openshift-default Scaled up replica set e2e-inf-a-76b073-openshift-default-697d759dd5 from 0 to 1 38m Normal AddedLabel gateway/e2e-inf-a-76b073 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-76b073 38m Normal SuccessfulCreate replicaset/e2e-inf-a-76b073-openshift-default-697d759dd5 Created pod: e2e-inf-a-76b073-openshift-default-697d759dd5-ql7sd 38m Normal AddedInterface pod/e2e-inf-a-76b073-openshift-default-697d759dd5-ql7sd Add eth0 [10.132.0.230/23] from ovn-kubernetes 38m Normal Killing pod/e2e-shared-b-w4-4a3dba-openshift-default-dc9664475-zxb5s Stopping container istio-proxy 38m Normal AddedLabel gateway/e2e-api-a-92e375 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-92e375 38m Normal SuccessfulCreate replicaset/e2e-api-a-92e375-openshift-default-6987df58bc Created pod: e2e-api-a-92e375-openshift-default-6987df58bc-jzxns 38m Normal ScalingReplicaSet deployment/e2e-api-a-92e375-openshift-default Scaled up replica set e2e-api-a-92e375-openshift-default-6987df58bc from 0 to 1 38m Normal AddedInterface pod/e2e-api-a-92e375-openshift-default-6987df58bc-jzxns Add eth0 [10.132.0.231/23] from ovn-kubernetes 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-76b073 Scaled up replica set payload-processing-e2e-inf-a-76b073-78787bc78 from 0 to 1 38m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-76b073-78787bc78 Error creating: pods "payload-processing-e2e-inf-a-76b073-78787bc78-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-76b073: serviceaccount "payload-processing-e2e-inf-a-76b073" not found 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-76b073 Scaled up replica set payload-pre-processing-e2e-inf-a-76b073-68467cb4b7 from 0 to 1 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-76b073-78787bc78 Created pod: payload-processing-e2e-inf-a-76b073-78787bc78-d6l68 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-76b073-68467cb4b7 Error creating: pods "payload-pre-processing-e2e-inf-a-76b073-68467cb4b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-76b073: serviceaccount "payload-processing-e2e-inf-a-76b073" not found 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-76b073-68467cb4b7 Created pod: payload-pre-processing-e2e-inf-a-76b073-68467cb4b7-pf7qq 38m Normal Started pod/payload-processing-e2e-inf-a-76b073-78787bc78-d6l68 Container started 38m Normal Created pod/payload-pre-processing-e2e-inf-a-76b073-68467cb4b7-pf7qq Container created 38m Normal Pulled pod/payload-pre-processing-e2e-inf-a-76b073-68467cb4b7-pf7qq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-76b073-68467cb4b7-pf7qq Add eth0 [10.132.0.232/23] from ovn-kubernetes 38m Normal Started pod/payload-pre-processing-e2e-inf-a-76b073-68467cb4b7-pf7qq Container started 38m Normal AddedInterface pod/payload-processing-e2e-inf-a-76b073-78787bc78-d6l68 Add eth0 [10.132.0.233/23] from ovn-kubernetes 38m Normal Created pod/payload-processing-e2e-inf-a-76b073-78787bc78-d6l68 Container created 38m Normal Pulled pod/payload-processing-e2e-inf-a-76b073-78787bc78-d6l68 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-92e375 Scaled up replica set payload-pre-processing-e2e-api-a-92e375-6d87d5b947 from 0 to 1 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-92e375 Scaled up replica set payload-processing-e2e-api-a-92e375-75c9fbbc84 from 0 to 1 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-92e375-6d87d5b947 Error creating: pods "payload-pre-processing-e2e-api-a-92e375-6d87d5b947-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-92e375: serviceaccount "payload-processing-e2e-api-a-92e375" not found 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-92e375-6d87d5b947 Created pod: payload-pre-processing-e2e-api-a-92e375-6d87d5b947-4249b 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-92e375-75c9fbbc84 Created pod: payload-processing-e2e-api-a-92e375-75c9fbbc84-nt42r 38m Warning FailedCreate replicaset/payload-processing-e2e-api-a-92e375-75c9fbbc84 Error creating: pods "payload-processing-e2e-api-a-92e375-75c9fbbc84-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-92e375: serviceaccount "payload-processing-e2e-api-a-92e375" not found 38m Normal AddedInterface pod/payload-processing-e2e-api-a-92e375-75c9fbbc84-nt42r Add eth0 [10.132.0.235/23] from ovn-kubernetes 38m Normal Created pod/payload-pre-processing-e2e-api-a-92e375-6d87d5b947-4249b Container created 38m Normal Pulled pod/payload-processing-e2e-api-a-92e375-75c9fbbc84-nt42r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Created pod/payload-processing-e2e-api-a-92e375-75c9fbbc84-nt42r Container created 38m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-92e375-6d87d5b947-4249b Add eth0 [10.132.0.234/23] from ovn-kubernetes 38m Normal Pulled pod/payload-pre-processing-e2e-api-a-92e375-6d87d5b947-4249b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Started pod/payload-processing-e2e-api-a-92e375-75c9fbbc84-nt42r Container started 38m Normal Started pod/payload-pre-processing-e2e-api-a-92e375-6d87d5b947-4249b Container started 37m Normal Killing pod/e2e-inf-a-03c703-openshift-default-78bd587bff-5m2n8 Stopping container istio-proxy 37m Normal Killing pod/payload-processing-e2e-inf-b-460d4e-6457b77684-dttcx Stopping container payload-processing 37m Normal Killing pod/payload-pre-processing-e2e-inf-b-460d4e-c97d659-rcrqj Stopping container payload-pre-processing 37m Normal Killing pod/e2e-shared-b-w3-d9e782-openshift-default-7ccffd49b5-7k8qc Stopping container istio-proxy 37m Normal AddedLabel gateway/e2e-api-a-ae3cd0 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-ae3cd0 37m Normal SuccessfulCreate replicaset/e2e-api-a-ae3cd0-openshift-default-5dbcd797d5 Created pod: e2e-api-a-ae3cd0-openshift-default-5dbcd797d5-hvcv4 37m Normal ScalingReplicaSet deployment/e2e-api-a-ae3cd0-openshift-default Scaled up replica set e2e-api-a-ae3cd0-openshift-default-5dbcd797d5 from 0 to 1 37m Normal AddedInterface pod/e2e-api-a-ae3cd0-openshift-default-5dbcd797d5-hvcv4 Add eth0 [10.132.0.237/23] from ovn-kubernetes 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-ae3cd0 Scaled up replica set payload-processing-e2e-api-a-ae3cd0-9bb5478db from 0 to 1 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-ae3cd0-7ffbd9dc87 Error creating: pods "payload-pre-processing-e2e-api-a-ae3cd0-7ffbd9dc87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-ae3cd0: serviceaccount "payload-processing-e2e-api-a-ae3cd0" not found 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-ae3cd0-7ffbd9dc87 Created pod: payload-pre-processing-e2e-api-a-ae3cd0-7ffbd9dc87-l54tr 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-ae3cd0 Scaled up replica set payload-pre-processing-e2e-api-a-ae3cd0-7ffbd9dc87 from 0 to 1 37m Warning FailedCreate replicaset/payload-processing-e2e-api-a-ae3cd0-9bb5478db Error creating: pods "payload-processing-e2e-api-a-ae3cd0-9bb5478db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-ae3cd0: serviceaccount "payload-processing-e2e-api-a-ae3cd0" not found 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-ae3cd0-9bb5478db Created pod: payload-processing-e2e-api-a-ae3cd0-9bb5478db-92qxb 37m Normal Started pod/payload-processing-e2e-api-a-ae3cd0-9bb5478db-92qxb Container started 37m Normal AddedInterface pod/payload-processing-e2e-api-a-ae3cd0-9bb5478db-92qxb Add eth0 [10.132.0.239/23] from ovn-kubernetes 37m Normal Created pod/payload-processing-e2e-api-a-ae3cd0-9bb5478db-92qxb Container created 37m Normal Pulled pod/payload-processing-e2e-api-a-ae3cd0-9bb5478db-92qxb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Created pod/payload-pre-processing-e2e-api-a-ae3cd0-7ffbd9dc87-l54tr Container created 37m Normal Started pod/payload-pre-processing-e2e-api-a-ae3cd0-7ffbd9dc87-l54tr Container started 37m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-ae3cd0-7ffbd9dc87-l54tr Add eth0 [10.132.0.238/23] from ovn-kubernetes 37m Normal Pulled pod/payload-pre-processing-e2e-api-a-ae3cd0-7ffbd9dc87-l54tr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Killing pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-q6d2w Stopping container istio-proxy 36m Normal Killing pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-wvq8z Stopping container istio-proxy 36m Normal Killing pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-q6whh Stopping container istio-proxy 36m Normal Killing pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-j9skc Stopping container istio-proxy 36m Normal Killing pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-z9l6k Stopping container istio-proxy 36m Normal Killing pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-fxvfs Stopping container istio-proxy 36m Normal Killing pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-qmsfx Stopping container istio-proxy 36m Normal Killing pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-c55xx Stopping container istio-proxy 36m Normal Killing pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-kl5fq Stopping container istio-proxy 36m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 36m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 36m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-kzzph Stopping container discovery 36m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-kzzph 36m Normal Killing pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-p8ljt Stopping container istio-proxy 36m Warning Unhealthy pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-wvq8z Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning ProbeError pod/e2e-mt-a251fdfb-openshift-default-7f7f49cc57-wvq8z Readiness probe error: HTTP probe failed with statuscode: 503... 36m Normal Killing pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-rb7qp Stopping container istio-proxy 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-35a187-openshift-default New size: 2; reason: All metrics below target 36m Normal ScalingReplicaSet deployment/e2e-worker-w6-35a187-openshift-default Scaled down replica set e2e-worker-w6-35a187-openshift-default-5b54c65f75 from 3 to 2 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-27e82b-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 36m Normal ScalingReplicaSet deployment/e2e-worker-w2-27e82b-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 from 3 to 2 36m Normal SuccessfulDelete replicaset/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77 Deleted pod: e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-qc65q 36m Normal Killing pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-qc65q Stopping container istio-proxy 36m Normal SuccessfulDelete replicaset/e2e-worker-w6-35a187-openshift-default-5b54c65f75 Deleted pod: e2e-worker-w6-35a187-openshift-default-5b54c65f75-rb7qp 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-ae3cd0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-ae3cd0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Normal Killing pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-n2wcf Stopping container istio-proxy 36m Normal Killing pod/e2e-worker-w2-27e82b-openshift-default-c6c7c4b77-4dqrc Stopping container istio-proxy 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-34f1ce-openshift-default New size: 3; reason: All metrics below target 36m Normal SuccessfulDelete replicaset/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf Deleted pod: e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-r8xrl 36m Normal Killing pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-r8xrl Stopping container istio-proxy 36m Normal ScalingReplicaSet deployment/e2e-worker-w4-34f1ce-openshift-default Scaled down replica set e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf from 4 to 3 36m Normal Killing pod/e2e-api-b-39e3a8-openshift-default-6b6658797-ghqtx Stopping container istio-proxy 36m Normal SuccessfulCreate replicaset/e2e-inf-a-ddf61a-openshift-default-f57f54bc Created pod: e2e-inf-a-ddf61a-openshift-default-f57f54bc-4gvfk 36m Normal ScalingReplicaSet deployment/e2e-inf-a-ddf61a-openshift-default Scaled up replica set e2e-inf-a-ddf61a-openshift-default-f57f54bc from 0 to 1 36m Normal AddedLabel gateway/e2e-inf-a-ddf61a Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-ddf61a 36m Normal AddedInterface pod/e2e-inf-a-ddf61a-openshift-default-f57f54bc-4gvfk Add eth0 [10.132.0.242/23] from ovn-kubernetes 36m Normal SuccessfulDelete replicaset/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc Deleted pod: e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-pg48c 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-986d37-openshift-default New size: 3; reason: All metrics below target 36m Normal Killing pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-pg48c Stopping container istio-proxy 36m Normal Killing pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-w6bmh Stopping container istio-proxy 36m Normal Killing pod/maas-default-gateway-openshift-default-8f6f87f59-wv5pf Stopping container istio-proxy 36m Normal ScalingReplicaSet deployment/e2e-worker-w5-fe01fa-openshift-default Scaled down replica set e2e-worker-w5-fe01fa-openshift-default-564f5c6485 from 5 to 3 36m Warning ProbeError pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-pg48c Readiness probe error: HTTP probe failed with statuscode: 503... 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-fe01fa-openshift-default New size: 3; reason: All metrics below target 36m Normal Killing pod/maas-default-gateway-openshift-default-8f6f87f59-thj8z Stopping container istio-proxy 36m Normal Killing pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-qxg67 Stopping container istio-proxy 36m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-8f6f87f59 from 7 to 5 36m Normal SuccessfulDelete replicaset/e2e-worker-w5-fe01fa-openshift-default-564f5c6485 Deleted pod: e2e-worker-w5-fe01fa-openshift-default-564f5c6485-w6bmh 36m Warning Unhealthy pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-pg48c Readiness probe failed: HTTP probe failed with statuscode: 503 36m Normal SuccessfulDelete replicaset/e2e-worker-w5-fe01fa-openshift-default-564f5c6485 Deleted pod: e2e-worker-w5-fe01fa-openshift-default-564f5c6485-qxg67 36m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-thj8z Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-thj8z Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-wv5pf Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-wv5pf Readiness probe failed: HTTP probe failed with statuscode: 503 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-ddf61a Scaled up replica set payload-processing-e2e-inf-a-ddf61a-5f445c7548 from 0 to 1 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-ddf61a Scaled up replica set payload-pre-processing-e2e-inf-a-ddf61a-67ff4567bc from 0 to 1 36m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-ddf61a-5f445c7548 Error creating: pods "payload-processing-e2e-inf-a-ddf61a-5f445c7548-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-ddf61a: serviceaccount "payload-processing-e2e-inf-a-ddf61a" not found 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-ddf61a-67ff4567bc Error creating: pods "payload-pre-processing-e2e-inf-a-ddf61a-67ff4567bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-ddf61a: serviceaccount "payload-processing-e2e-inf-a-ddf61a" not found 36m Normal Started pod/e2e-api-a-92e375-openshift-default-6987df58bc-jzxns Container started 36m Normal Pulled pod/e2e-api-a-92e375-openshift-default-6987df58bc-jzxns Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-ddf61a-5f445c7548 Created pod: payload-processing-e2e-inf-a-ddf61a-5f445c7548-fwpmj 36m Normal Created pod/e2e-api-a-92e375-openshift-default-6987df58bc-jzxns Container created 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-ddf61a-67ff4567bc Created pod: payload-pre-processing-e2e-inf-a-ddf61a-67ff4567bc-mrc2x 36m Normal Started pod/payload-pre-processing-e2e-inf-a-ddf61a-67ff4567bc-mrc2x Container started 36m Normal Created pod/payload-pre-processing-e2e-inf-a-ddf61a-67ff4567bc-mrc2x Container created 36m Normal Pulled pod/payload-pre-processing-e2e-inf-a-ddf61a-67ff4567bc-mrc2x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Warning Unhealthy pod/e2e-inf-a-76b073-openshift-default-697d759dd5-ql7sd Readiness probe failed: Get "http://10.132.0.230:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 36m Warning ProbeError pod/e2e-inf-a-76b073-openshift-default-697d759dd5-ql7sd Readiness probe error: Get "http://10.132.0.230:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 36m Normal AddedInterface pod/payload-processing-e2e-inf-a-ddf61a-5f445c7548-fwpmj Add eth0 [10.134.0.141/23] from ovn-kubernetes 36m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-ddf61a-67ff4567bc-mrc2x Add eth0 [10.134.0.140/23] from ovn-kubernetes 36m Normal Started pod/e2e-inf-a-76b073-openshift-default-697d759dd5-ql7sd Container started 36m Normal Created pod/e2e-inf-a-76b073-openshift-default-697d759dd5-ql7sd Container created 36m Normal Pulled pod/payload-processing-e2e-inf-a-ddf61a-5f445c7548-fwpmj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Created pod/payload-processing-e2e-inf-a-ddf61a-5f445c7548-fwpmj Container created 36m Normal Pulled pod/e2e-inf-a-76b073-openshift-default-697d759dd5-ql7sd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/payload-processing-e2e-inf-a-ddf61a-5f445c7548-fwpmj Container started 36m Normal Killing pod/e2e-inf-b-460d4e-openshift-default-575d5fbfb7-ftjgz Stopping container istio-proxy 36m Warning Unhealthy pod/e2e-api-a-92e375-openshift-default-6987df58bc-jzxns Startup probe failed: Get "http://10.132.0.231:15021/healthz/ready": dial tcp 10.132.0.231:15021: connect: connection refused 36m Warning ProbeError pod/e2e-api-a-92e375-openshift-default-6987df58bc-jzxns Startup probe error: Get "http://10.132.0.231:15021/healthz/ready": dial tcp 10.132.0.231:15021: connect: connection refused... 36m Warning ProbeError pod/e2e-inf-a-76b073-openshift-default-697d759dd5-ql7sd Startup probe error: Get "http://10.132.0.230:15021/healthz/ready": dial tcp 10.132.0.230:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-inf-a-76b073-openshift-default-697d759dd5-ql7sd Startup probe failed: Get "http://10.132.0.230:15021/healthz/ready": dial tcp 10.132.0.230:15021: connect: connection refused 36m Warning ProbeError pod/e2e-inf-b-460d4e-openshift-default-575d5fbfb7-ftjgz Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning Unhealthy pod/e2e-inf-b-460d4e-openshift-default-575d5fbfb7-ftjgz Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-76b073-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-76b073-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-92e375-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-92e375-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal Killing pod/payload-pre-processing-e2e-shared-a-w6-f7b8d8-576dfbf9f9-d9d8l Stopping container payload-pre-processing 36m Normal Killing pod/payload-processing-e2e-shared-a-w6-f7b8d8-6b866d558b-hq859 Stopping container payload-processing 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-76b073-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-76b073-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-92e375-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-92e375-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Normal AddedLabel gateway/e2e-inf-b-49837c Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-49837c 35m Normal ScalingReplicaSet deployment/e2e-inf-b-49837c-openshift-default Scaled up replica set e2e-inf-b-49837c-openshift-default-566c644bb9 from 0 to 1 35m Normal SuccessfulCreate replicaset/e2e-inf-b-49837c-openshift-default-566c644bb9 Created pod: e2e-inf-b-49837c-openshift-default-566c644bb9-wb52d 35m Normal AddedInterface pod/e2e-inf-b-49837c-openshift-default-566c644bb9-wb52d Add eth0 [10.133.0.99/23] from ovn-kubernetes 35m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-hkgx7 Readiness probe failed: Get "http://10.133.0.30:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 35m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-hkgx7 Readiness probe error: Get "http://10.133.0.30:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 35m Normal AddedLabel gateway/e2e-api-b-d687ff Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-d687ff 35m Normal SuccessfulCreate replicaset/e2e-api-b-d687ff-openshift-default-5c75fcb6d5 Created pod: e2e-api-b-d687ff-openshift-default-5c75fcb6d5-25thf 35m Normal ScalingReplicaSet deployment/e2e-api-b-d687ff-openshift-default Scaled up replica set e2e-api-b-d687ff-openshift-default-5c75fcb6d5 from 0 to 1 35m Normal AddedInterface pod/e2e-api-b-d687ff-openshift-default-5c75fcb6d5-25thf Add eth0 [10.132.0.243/23] from ovn-kubernetes 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-49837c Scaled up replica set payload-pre-processing-e2e-inf-b-49837c-5c557bd847 from 0 to 1 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-49837c Scaled up replica set payload-processing-e2e-inf-b-49837c-79c55dc878 from 0 to 1 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-49837c-79c55dc878 Created pod: payload-processing-e2e-inf-b-49837c-79c55dc878-vd2k9 35m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-49837c-79c55dc878 Error creating: pods "payload-processing-e2e-inf-b-49837c-79c55dc878-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-49837c: serviceaccount "payload-processing-e2e-inf-b-49837c" not found 35m Normal Pulled pod/payload-pre-processing-e2e-inf-b-49837c-5c557bd847-pn6w5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-49837c-5c557bd847-pn6w5 Add eth0 [10.134.0.144/23] from ovn-kubernetes 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-49837c-5c557bd847 Error creating: pods "payload-pre-processing-e2e-inf-b-49837c-5c557bd847-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-49837c: serviceaccount "payload-processing-e2e-inf-b-49837c" not found 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-49837c-5c557bd847 Created pod: payload-pre-processing-e2e-inf-b-49837c-5c557bd847-pn6w5 35m Normal Created pod/payload-processing-e2e-inf-b-49837c-79c55dc878-vd2k9 Container created 35m Normal Created pod/payload-pre-processing-e2e-inf-b-49837c-5c557bd847-pn6w5 Container created 35m Normal AddedInterface pod/payload-processing-e2e-inf-b-49837c-79c55dc878-vd2k9 Add eth0 [10.134.0.145/23] from ovn-kubernetes 35m Normal Started pod/payload-processing-e2e-inf-b-49837c-79c55dc878-vd2k9 Container started 35m Normal Started pod/payload-pre-processing-e2e-inf-b-49837c-5c557bd847-pn6w5 Container started 35m Normal Pulled pod/payload-processing-e2e-inf-b-49837c-79c55dc878-vd2k9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-api-a-ae3cd0-openshift-default-5dbcd797d5-hvcv4 Container created 35m Normal Pulled pod/e2e-api-a-ae3cd0-openshift-default-5dbcd797d5-hvcv4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Killing pod/e2e-shared-a-w6-f7b8d8-openshift-default-6b8ffbcc97-cccbw Stopping container istio-proxy 35m Normal Started pod/e2e-api-a-ae3cd0-openshift-default-5dbcd797d5-hvcv4 Container started 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-d687ff Scaled up replica set payload-processing-e2e-api-b-d687ff-7666858cfd from 0 to 1 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-d687ff Scaled up replica set payload-pre-processing-e2e-api-b-d687ff-7c6c95ffc9 from 0 to 1 35m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-d687ff-7c6c95ffc9-4xkwz Add eth0 [10.134.0.148/23] from ovn-kubernetes 35m Normal Created pod/payload-pre-processing-e2e-api-b-d687ff-7c6c95ffc9-4xkwz Container created 35m Normal Pulled pod/payload-pre-processing-e2e-api-b-d687ff-7c6c95ffc9-4xkwz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-d687ff-7c6c95ffc9 Created pod: payload-pre-processing-e2e-api-b-d687ff-7c6c95ffc9-4xkwz 35m Warning FailedCreate replicaset/payload-processing-e2e-api-b-d687ff-7666858cfd Error creating: pods "payload-processing-e2e-api-b-d687ff-7666858cfd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-d687ff: serviceaccount "payload-processing-e2e-api-b-d687ff" not found 35m Normal AddedInterface pod/payload-processing-e2e-api-b-d687ff-7666858cfd-wh6tj Add eth0 [10.134.0.149/23] from ovn-kubernetes 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-d687ff-7c6c95ffc9 Error creating: pods "payload-pre-processing-e2e-api-b-d687ff-7c6c95ffc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-d687ff: serviceaccount "payload-processing-e2e-api-b-d687ff" not found 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-d687ff-7666858cfd Created pod: payload-processing-e2e-api-b-d687ff-7666858cfd-wh6tj 35m Normal Started pod/payload-pre-processing-e2e-api-b-d687ff-7c6c95ffc9-4xkwz Container started 35m Normal Created pod/payload-processing-e2e-api-b-d687ff-7666858cfd-wh6tj Container created 35m Normal Started pod/payload-processing-e2e-api-b-d687ff-7666858cfd-wh6tj Container started 35m Normal Pulled pod/payload-processing-e2e-api-b-d687ff-7666858cfd-wh6tj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Warning ProbeError pod/e2e-api-a-ae3cd0-openshift-default-5dbcd797d5-hvcv4 Startup probe error: Get "http://10.132.0.237:15021/healthz/ready": dial tcp 10.132.0.237:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-api-a-ae3cd0-openshift-default-5dbcd797d5-hvcv4 Startup probe failed: Get "http://10.132.0.237:15021/healthz/ready": dial tcp 10.132.0.237:15021: connect: connection refused 35m Normal Killing pod/payload-processing-e2e-shared-b-w6-0d75be-7b66b84856-j28w4 Stopping container payload-processing 35m Normal Killing pod/payload-pre-processing-e2e-shared-b-w6-0d75be-75f9855f5-kftq4 Stopping container payload-pre-processing 35m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-8f6f87f59 Created pod: maas-default-gateway-openshift-default-8f6f87f59-jpjkl 35m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-nn2nk Container created 35m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-49lgw Container started 35m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-jpjkl Container created 35m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-nn2nk Add eth0 [10.134.0.153/23] from ovn-kubernetes 35m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-jpjkl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-jpjkl Add eth0 [10.132.0.244/23] from ovn-kubernetes 35m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-nn2nk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-mz295 Container started 35m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-gtzrp Container started 35m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-mz295 Container created 35m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-mz295 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-nn2nk Container started 35m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-gtzrp Add eth0 [10.134.0.152/23] from ovn-kubernetes 35m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-gtzrp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-49lgw Add eth0 [10.133.0.100/23] from ovn-kubernetes 35m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-49lgw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-mz295 Add eth0 [10.132.0.245/23] from ovn-kubernetes 35m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-49lgw Container created 35m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-jpjkl Container started 35m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-gtzrp Container created 35m Warning Unhealthy pod/payload-processing-e2e-shared-b-w6-0d75be-7b66b84856-j28w4 Readiness probe failed: timeout: failed to connect service "10.132.0.187:50052" within 1s: context deadline exceeded 35m Normal Killing pod/payload-pre-processing-e2e-worker-w5-fe01fa-576c754f8c-8dm7l Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-worker-w5-fe01fa-698f78d94b-4wmlb Stopping container payload-processing 35m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-gtzrp Startup probe failed: Get "http://10.134.0.152:15021/healthz/ready": dial tcp 10.134.0.152:15021: connect: connection refused 35m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-nn2nk Startup probe failed: Get "http://10.134.0.153:15021/healthz/ready": dial tcp 10.134.0.153:15021: connect: connection refused 35m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-nn2nk Startup probe error: Get "http://10.134.0.153:15021/healthz/ready": dial tcp 10.134.0.153:15021: connect: connection refused... 35m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-gtzrp Startup probe error: Get "http://10.134.0.152:15021/healthz/ready": dial tcp 10.134.0.152:15021: connect: connection refused... 35m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-49lgw Startup probe failed: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused 35m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-jpjkl Startup probe failed: Get "http://10.132.0.244:15021/healthz/ready": dial tcp 10.132.0.244:15021: connect: connection refused 35m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-49lgw Startup probe error: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused... 35m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-mz295 Startup probe failed: Get "http://10.132.0.245:15021/healthz/ready": dial tcp 10.132.0.245:15021: connect: connection refused 35m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-jpjkl Startup probe error: Get "http://10.132.0.244:15021/healthz/ready": dial tcp 10.132.0.244:15021: connect: connection refused... 35m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-mz295 Startup probe error: Get "http://10.132.0.245:15021/healthz/ready": dial tcp 10.132.0.245:15021: connect: connection refused... 35m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-mz295 Startup probe failed: Get "http://10.132.0.245:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 35m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-mz295 Startup probe error: Get "http://10.132.0.245:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-ae3cd0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-ae3cd0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning Unhealthy pod/payload-processing-e2e-worker-w5-fe01fa-698f78d94b-4wmlb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Normal ScalingReplicaSet deployment/e2e-api-b-1e5c33-openshift-default Scaled up replica set e2e-api-b-1e5c33-openshift-default-6f5f6b8dc9 from 0 to 1 34m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w5-fe01fa-576c754f8c-8dm7l Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Normal AddedLabel gateway/e2e-api-b-1e5c33 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-1e5c33 34m Normal SuccessfulCreate replicaset/e2e-api-b-1e5c33-openshift-default-6f5f6b8dc9 Created pod: e2e-api-b-1e5c33-openshift-default-6f5f6b8dc9-lmv7w 34m Normal AddedInterface pod/e2e-api-b-1e5c33-openshift-default-6f5f6b8dc9-lmv7w Add eth0 [10.133.0.101/23] from ovn-kubernetes 34m Normal Pulled pod/e2e-inf-a-ddf61a-openshift-default-f57f54bc-4gvfk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-ddf61a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Normal Started pod/e2e-inf-a-ddf61a-openshift-default-f57f54bc-4gvfk Container started 34m Normal Created pod/e2e-inf-a-ddf61a-openshift-default-f57f54bc-4gvfk Container created 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-ddf61a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-1e5c33-78944dcc56 Created pod: payload-processing-e2e-api-b-1e5c33-78944dcc56-xzctn 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-1e5c33 Scaled up replica set payload-pre-processing-e2e-api-b-1e5c33-56fbff6867 from 0 to 1 34m Warning FailedCreate replicaset/payload-processing-e2e-api-b-1e5c33-78944dcc56 Error creating: pods "payload-processing-e2e-api-b-1e5c33-78944dcc56-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-1e5c33: serviceaccount "payload-processing-e2e-api-b-1e5c33" not found 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-1e5c33 Scaled up replica set payload-processing-e2e-api-b-1e5c33-78944dcc56 from 0 to 1 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-1e5c33-56fbff6867 Error creating: pods "payload-pre-processing-e2e-api-b-1e5c33-56fbff6867-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-1e5c33: serviceaccount "payload-processing-e2e-api-b-1e5c33" not found 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-1e5c33-56fbff6867 Created pod: payload-pre-processing-e2e-api-b-1e5c33-56fbff6867-ckkk9 34m Normal Pulled pod/payload-processing-e2e-api-b-1e5c33-78944dcc56-xzctn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Started pod/payload-pre-processing-e2e-api-b-1e5c33-56fbff6867-ckkk9 Container started 34m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-1e5c33-56fbff6867-ckkk9 Add eth0 [10.134.0.156/23] from ovn-kubernetes 34m Normal Pulled pod/payload-pre-processing-e2e-api-b-1e5c33-56fbff6867-ckkk9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/payload-processing-e2e-api-b-1e5c33-78944dcc56-xzctn Add eth0 [10.134.0.157/23] from ovn-kubernetes 34m Normal Created pod/payload-processing-e2e-api-b-1e5c33-78944dcc56-xzctn Container created 34m Normal Started pod/payload-processing-e2e-api-b-1e5c33-78944dcc56-xzctn Container started 34m Warning ProbeError pod/e2e-inf-a-ddf61a-openshift-default-f57f54bc-4gvfk Startup probe error: Get "http://10.132.0.242:15021/healthz/ready": dial tcp 10.132.0.242:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-inf-a-ddf61a-openshift-default-f57f54bc-4gvfk Startup probe failed: Get "http://10.132.0.242:15021/healthz/ready": dial tcp 10.132.0.242:15021: connect: connection refused 34m Normal Created pod/payload-pre-processing-e2e-api-b-1e5c33-56fbff6867-ckkk9 Container created 34m Normal Killing pod/e2e-shared-b-w6-0d75be-openshift-default-57f99d9cbc-smttn Stopping container istio-proxy 34m Normal AddedLabel gateway/e2e-inf-b-37abc0 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-37abc0 34m Normal ScalingReplicaSet deployment/e2e-inf-b-37abc0-openshift-default Scaled up replica set e2e-inf-b-37abc0-openshift-default-65b888884f from 0 to 1 34m Normal SuccessfulCreate replicaset/e2e-inf-b-37abc0-openshift-default-65b888884f Created pod: e2e-inf-b-37abc0-openshift-default-65b888884f-pqjfl 34m Normal AddedInterface pod/e2e-inf-b-37abc0-openshift-default-65b888884f-pqjfl Add eth0 [10.132.0.247/23] from ovn-kubernetes 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-ddf61a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-ddf61a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Normal Killing pod/payload-processing-e2e-worker-w6-35a187-59b76ccb5b-25xkv Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-worker-w6-35a187-7887c98dff-5h96p Stopping container payload-pre-processing 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-37abc0-54fc77c78 Created pod: payload-processing-e2e-inf-b-37abc0-54fc77c78-vvrhm 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-37abc0-69469c56c Error creating: pods "payload-pre-processing-e2e-inf-b-37abc0-69469c56c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-37abc0: serviceaccount "payload-processing-e2e-inf-b-37abc0" not found 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-37abc0-69469c56c Created pod: payload-pre-processing-e2e-inf-b-37abc0-69469c56c-6bzrh 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-37abc0 Scaled up replica set payload-pre-processing-e2e-inf-b-37abc0-69469c56c from 0 to 1 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-37abc0 Scaled up replica set payload-processing-e2e-inf-b-37abc0-54fc77c78 from 0 to 1 34m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-37abc0-54fc77c78 Error creating: pods "payload-processing-e2e-inf-b-37abc0-54fc77c78-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-37abc0: serviceaccount "payload-processing-e2e-inf-b-37abc0" not found 34m Normal Pulled pod/payload-pre-processing-e2e-inf-b-37abc0-69469c56c-6bzrh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Started pod/payload-pre-processing-e2e-inf-b-37abc0-69469c56c-6bzrh Container started 34m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-37abc0-69469c56c-6bzrh Add eth0 [10.134.0.162/23] from ovn-kubernetes 34m Normal Created pod/payload-processing-e2e-inf-b-37abc0-54fc77c78-vvrhm Container created 34m Normal Created pod/payload-pre-processing-e2e-inf-b-37abc0-69469c56c-6bzrh Container created 34m Normal Started pod/payload-processing-e2e-inf-b-37abc0-54fc77c78-vvrhm Container started 34m Normal Pulled pod/payload-processing-e2e-inf-b-37abc0-54fc77c78-vvrhm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/payload-processing-e2e-inf-b-37abc0-54fc77c78-vvrhm Add eth0 [10.133.0.102/23] from ovn-kubernetes 33m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w6-35a187-7887c98dff-5h96p Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Warning Unhealthy pod/payload-processing-e2e-worker-w6-35a187-59b76ccb5b-25xkv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Warning ProbeError pod/e2e-inf-b-49837c-openshift-default-566c644bb9-wb52d Readiness probe error: Get "http://10.133.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 33m Warning Unhealthy pod/e2e-inf-b-49837c-openshift-default-566c644bb9-wb52d Readiness probe failed: Get "http://10.133.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 33m Normal Pulled pod/e2e-inf-b-49837c-openshift-default-566c644bb9-wb52d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Started pod/e2e-inf-b-49837c-openshift-default-566c644bb9-wb52d Container started 33m Normal Created pod/e2e-inf-b-49837c-openshift-default-566c644bb9-wb52d Container created 33m Normal Killing pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-2mn85 Stopping container istio-proxy 33m Normal Killing pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-d44pq Stopping container istio-proxy 33m Normal Killing pod/e2e-worker-w5-fe01fa-openshift-default-564f5c6485-8hzf8 Stopping container istio-proxy 33m Warning ProbeError pod/e2e-inf-b-49837c-openshift-default-566c644bb9-wb52d Startup probe error: Get "http://10.133.0.99:15021/healthz/ready": dial tcp 10.133.0.99:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-inf-b-49837c-openshift-default-566c644bb9-wb52d Startup probe failed: Get "http://10.133.0.99:15021/healthz/ready": dial tcp 10.133.0.99:15021: connect: connection refused 33m Normal SuccessfulDelete replicaset/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc Deleted pod: e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-t28zs 33m Normal Killing pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-t28zs Stopping container istio-proxy 33m Warning Unhealthy pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-t28zs Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-49837c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning ProbeError pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-t28zs Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-49837c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Normal Killing pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-6hww6 Stopping container istio-proxy 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-46b586-openshift-default New size: 2; reason: All metrics below target 33m Normal ScalingReplicaSet deployment/e2e-worker-w0-46b586-openshift-default Scaled down replica set e2e-worker-w0-46b586-openshift-default-86c598c96c from 3 to 2 33m Normal SuccessfulDelete replicaset/e2e-worker-w0-46b586-openshift-default-86c598c96c Deleted pod: e2e-worker-w0-46b586-openshift-default-86c598c96c-6hww6 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-49837c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-49837c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Normal Started pod/e2e-api-b-1e5c33-openshift-default-6f5f6b8dc9-lmv7w Container started 33m Normal Pulled pod/e2e-api-b-1e5c33-openshift-default-6f5f6b8dc9-lmv7w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-api-b-1e5c33-openshift-default-6f5f6b8dc9-lmv7w Container created 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-1e5c33-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-1e5c33-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning Unhealthy pod/e2e-api-b-1e5c33-openshift-default-6f5f6b8dc9-lmv7w Startup probe failed: Get "http://10.133.0.101:15021/healthz/ready": dial tcp 10.133.0.101:15021: connect: connection refused 33m Warning ProbeError pod/e2e-api-b-1e5c33-openshift-default-6f5f6b8dc9-lmv7w Startup probe error: Get "http://10.133.0.101:15021/healthz/ready": dial tcp 10.133.0.101:15021: connect: connection refused... 33m Normal Pulled pod/e2e-api-b-d687ff-openshift-default-5c75fcb6d5-25thf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-api-b-d687ff-openshift-default-5c75fcb6d5-25thf Container created 33m Normal Started pod/e2e-api-b-d687ff-openshift-default-5c75fcb6d5-25thf Container started 32m Warning Unhealthy pod/e2e-api-b-d687ff-openshift-default-5c75fcb6d5-25thf Startup probe failed: Get "http://10.132.0.243:15021/healthz/ready": dial tcp 10.132.0.243:15021: connect: connection refused 32m Warning ProbeError pod/e2e-api-b-d687ff-openshift-default-5c75fcb6d5-25thf Startup probe error: Get "http://10.132.0.243:15021/healthz/ready": dial tcp 10.132.0.243:15021: connect: connection refused... 32m Normal Killing pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-ggkpj Stopping container istio-proxy 32m Normal SuccessfulDelete replicaset/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf Deleted pod: e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-ggkpj 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-34f1ce-openshift-default New size: 2; reason: All metrics below target 32m Normal ScalingReplicaSet deployment/e2e-worker-w4-34f1ce-openshift-default Scaled down replica set e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf from 3 to 2 32m Normal Killing pod/payload-processing-e2e-api-a-92e375-75c9fbbc84-nt42r Stopping container payload-processing 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-d687ff-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Normal Killing pod/payload-pre-processing-e2e-api-a-92e375-6d87d5b947-4249b Stopping container payload-pre-processing 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-d687ff-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning ProbeError pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-ggkpj Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning Unhealthy pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-ggkpj Readiness probe failed: HTTP probe failed with statuscode: 503 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-1e5c33-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-1e5c33-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Normal Killing pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-657lr Stopping container istio-proxy 32m Normal Killing pod/e2e-worker-w6-35a187-openshift-default-5b54c65f75-bsqhk Stopping container istio-proxy 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-d687ff-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-d687ff-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning ProbeError pod/e2e-inf-b-37abc0-openshift-default-65b888884f-pqjfl Readiness probe error: Get "http://10.132.0.247:15021/healthz/ready": read tcp 10.132.0.2:58978->10.132.0.247:15021: read: connection reset by peer... 32m Warning Unhealthy pod/e2e-inf-b-37abc0-openshift-default-65b888884f-pqjfl Readiness probe failed: Get "http://10.132.0.247:15021/healthz/ready": read tcp 10.132.0.2:58978->10.132.0.247:15021: read: connection reset by peer 32m Normal Started pod/e2e-inf-b-37abc0-openshift-default-65b888884f-pqjfl Container started 32m Normal Created pod/e2e-inf-b-37abc0-openshift-default-65b888884f-pqjfl Container created 32m Normal Pulled pod/e2e-inf-b-37abc0-openshift-default-65b888884f-pqjfl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Warning Unhealthy pod/e2e-inf-b-37abc0-openshift-default-65b888884f-pqjfl Startup probe failed: Get "http://10.132.0.247:15021/healthz/ready": dial tcp 10.132.0.247:15021: connect: connection refused 32m Warning ProbeError pod/e2e-inf-b-37abc0-openshift-default-65b888884f-pqjfl Startup probe error: Get "http://10.132.0.247:15021/healthz/ready": dial tcp 10.132.0.247:15021: connect: connection refused... 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-37abc0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-37abc0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Normal Killing pod/payload-processing-e2e-api-a-ae3cd0-9bb5478db-92qxb Stopping container payload-processing 32m Normal Killing pod/payload-pre-processing-e2e-api-a-ae3cd0-7ffbd9dc87-l54tr Stopping container payload-pre-processing 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-37abc0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-37abc0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Normal Killing pod/e2e-api-a-92e375-openshift-default-6987df58bc-jzxns Stopping container istio-proxy 31m Warning Unhealthy pod/e2e-api-a-92e375-openshift-default-6987df58bc-jzxns Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning ProbeError pod/e2e-api-a-92e375-openshift-default-6987df58bc-jzxns Readiness probe error: HTTP probe failed with statuscode: 503... 30m Normal Killing pod/payload-processing-e2e-api-b-d687ff-7666858cfd-wh6tj Stopping container payload-processing 30m Normal Killing pod/payload-pre-processing-e2e-api-b-d687ff-7c6c95ffc9-4xkwz Stopping container payload-pre-processing 30m Normal Killing pod/e2e-api-a-ae3cd0-openshift-default-5dbcd797d5-hvcv4 Stopping container istio-proxy 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-5c3374-openshift-default New size: 1; reason: All metrics below target 30m Normal SuccessfulDelete replicaset/e2e-worker-w1-5c3374-openshift-default-d6ff699c7 Deleted pod: e2e-worker-w1-5c3374-openshift-default-d6ff699c7-678tp 30m Normal ScalingReplicaSet deployment/e2e-worker-w1-5c3374-openshift-default Scaled down replica set e2e-worker-w1-5c3374-openshift-default-d6ff699c7 from 2 to 1 30m Normal Killing pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-678tp Stopping container istio-proxy 30m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 30m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 30m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-wqjzk 30m Normal Killing pod/payload-processing-e2e-api-b-1e5c33-78944dcc56-xzctn Stopping container payload-processing 30m Normal Killing pod/payload-pre-processing-e2e-api-b-1e5c33-56fbff6867-ckkk9 Stopping container payload-pre-processing 30m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-wqjzk Stopping container discovery 30m Normal SuccessfulDelete replicaset/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc Deleted pod: e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-jqc5p 30m Normal Killing pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-jqc5p Stopping container istio-proxy 29m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-8f6f87f59 Deleted pod: maas-default-gateway-openshift-default-8f6f87f59-sgc28 29m Normal Killing pod/maas-default-gateway-openshift-default-8f6f87f59-bd4hv Stopping container istio-proxy 29m Normal Killing pod/maas-default-gateway-openshift-default-8f6f87f59-49lgw Stopping container istio-proxy 29m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-8f6f87f59 from 10 to 5 29m Normal Killing pod/maas-default-gateway-openshift-default-8f6f87f59-sgc28 Stopping container istio-proxy 29m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: All metrics below target 29m Normal Killing pod/maas-default-gateway-openshift-default-8f6f87f59-gtzrp Stopping container istio-proxy 29m Normal Killing pod/maas-default-gateway-openshift-default-8f6f87f59-hkgx7 Stopping container istio-proxy 29m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-hkgx7 Readiness probe failed: HTTP probe failed with statuscode: 503 29m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-hkgx7 Readiness probe error: HTTP probe failed with statuscode: 503... 29m Normal Killing pod/e2e-api-b-d687ff-openshift-default-5c75fcb6d5-25thf Stopping container istio-proxy 28m Normal Killing pod/payload-processing-e2e-worker-w4-34f1ce-56d77c46fd-9wsff Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-worker-w4-34f1ce-9b77f84c9-jfnmg Stopping container payload-pre-processing 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-46b586-openshift-default New size: 1; reason: All metrics below target 28m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: All metrics below target 28m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-8f6f87f59 from 5 to 4 28m Normal Killing pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-8ckgt Stopping container istio-proxy 28m Normal Killing pod/maas-default-gateway-openshift-default-8f6f87f59-jpjkl Stopping container istio-proxy 28m Normal SuccessfulDelete replicaset/e2e-worker-w0-46b586-openshift-default-86c598c96c Deleted pod: e2e-worker-w0-46b586-openshift-default-86c598c96c-8ckgt 28m Normal ScalingReplicaSet deployment/e2e-worker-w0-46b586-openshift-default Scaled down replica set e2e-worker-w0-46b586-openshift-default-86c598c96c from 2 to 1 28m Normal ScalingReplicaSet deployment/e2e-worker-w4-34f1ce-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf from 2 to 1 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-34f1ce-openshift-default New size: 1; reason: All metrics below target 28m Normal SuccessfulDelete replicaset/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf Deleted pod: e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-m8pb2 28m Normal Killing pod/e2e-api-b-1e5c33-openshift-default-6f5f6b8dc9-lmv7w Stopping container istio-proxy 28m Normal Killing pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-m8pb2 Stopping container istio-proxy 28m Warning ProbeError pod/e2e-api-b-1e5c33-openshift-default-6f5f6b8dc9-lmv7w Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning Unhealthy pod/e2e-api-b-1e5c33-openshift-default-6f5f6b8dc9-lmv7w Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning Unhealthy pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-m8pb2 Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning ProbeError pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-m8pb2 Readiness probe error: HTTP probe failed with statuscode: 503... 28m Normal Killing pod/payload-processing-e2e-worker-w3-986d37-695476d5cd-99x6j Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-worker-w3-986d37-5d7d64658c-h2tgp Stopping container payload-pre-processing 28m Normal Killing pod/payload-pre-processing-e2e-inf-a-76b073-68467cb4b7-pf7qq Stopping container payload-pre-processing 28m Normal Killing pod/payload-processing-e2e-inf-a-76b073-78787bc78-d6l68 Stopping container payload-processing 27m Normal Killing pod/e2e-inf-a-76b073-openshift-default-697d759dd5-ql7sd Stopping container istio-proxy 27m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-76b073-68467cb4b7-pf7qq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal Killing pod/payload-processing-e2e-inf-b-49837c-79c55dc878-vd2k9 Stopping container payload-processing 27m Normal Killing pod/payload-pre-processing-e2e-inf-b-49837c-5c557bd847-pn6w5 Stopping container payload-pre-processing 27m Normal Killing pod/e2e-worker-w4-34f1ce-openshift-default-856fb7b5bf-vx2nt Stopping container istio-proxy 27m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-8f6f87f59 from 4 to 3 27m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: All metrics below target 27m Normal Killing pod/maas-default-gateway-openshift-default-8f6f87f59-nn2nk Stopping container istio-proxy 27m Normal ScalingReplicaSet deployment/e2e-worker-w3-986d37-openshift-default (combined from similar events): Scaled up replica set e2e-worker-w3-986d37-openshift-default-5bc8bd86fc from 1 to 2 27m Normal SuccessfulCreate replicaset/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc Created pod: e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-sz5n2 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-986d37-openshift-default (combined from similar events): New size: 2; reason: cpu resource utilization (percentage of request) above target 27m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-49837c-5c557bd847-pn6w5 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal AddedInterface pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-sz5n2 Add eth0 [10.132.1.2/23] from ovn-kubernetes 27m Normal Started pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-sz5n2 Container started 27m Normal Pulled pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-sz5n2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal Created pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-sz5n2 Container created 27m Warning ProbeError pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-sz5n2 Startup probe error: Get "http://10.132.1.2:15021/healthz/ready": dial tcp 10.132.1.2:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-sz5n2 Startup probe failed: Get "http://10.132.1.2:15021/healthz/ready": dial tcp 10.132.1.2:15021: connect: connection refused 27m Warning Unhealthy pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-sz5n2 Startup probe failed: Get "http://10.132.1.2:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 27m Warning ProbeError pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-sz5n2 Startup probe error: Get "http://10.132.1.2:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 27m Normal Killing pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-hrh5g Stopping container istio-proxy 27m Normal Killing pod/e2e-worker-w3-986d37-openshift-default-5bc8bd86fc-sz5n2 Stopping container istio-proxy 27m Warning FailedCreate replicaset/e2e-inf-b-49837c-openshift-default-566c644bb9 Error creating: pods "e2e-inf-b-49837c-openshift-default-566c644bb9-" is forbidden: error looking up service account openshift-ingress/e2e-inf-b-49837c-openshift-default: serviceaccount "e2e-inf-b-49837c-openshift-default" not found 27m Normal Killing pod/e2e-inf-b-49837c-openshift-default-566c644bb9-wb52d Stopping container istio-proxy 27m Normal ScalingReplicaSet deployment/e2e-inf-b-49837c-openshift-default Scaled up replica set e2e-inf-b-49837c-openshift-default-566c644bb9 from 0 to 1 26m Normal Killing pod/payload-processing-e2e-shared-a-w0-d5360e-854fc9bc5d-mc76v Stopping container payload-processing 26m Normal Killing pod/payload-pre-processing-e2e-shared-a-w0-d5360e-b6fb568dc-htsps Stopping container payload-pre-processing 26m Normal Killing pod/e2e-shared-a-w0-d5360e-openshift-default-fbccfc549-d828w Stopping container istio-proxy 26m Warning Unhealthy pod/e2e-shared-a-w0-d5360e-openshift-default-fbccfc549-d828w Readiness probe failed: HTTP probe failed with statuscode: 503 26m Warning ProbeError pod/e2e-shared-a-w0-d5360e-openshift-default-fbccfc549-d828w Readiness probe error: HTTP probe failed with statuscode: 503... 26m Normal Killing pod/payload-processing-e2e-shared-b-w0-957019-788c577f9b-2shs7 Stopping container payload-processing 26m Normal Killing pod/payload-pre-processing-e2e-shared-b-w0-957019-76d548dcf-bbqz5 Stopping container payload-pre-processing 25m Normal Killing pod/e2e-shared-b-w0-957019-openshift-default-59bc4d44f9-8xtcd Stopping container istio-proxy 25m Normal Killing pod/payload-pre-processing-e2e-worker-w0-46b586-669f464d79-fsqx4 Stopping container payload-pre-processing 25m Normal Killing pod/payload-processing-e2e-worker-w0-46b586-556df6d97b-dcfpj Stopping container payload-processing 25m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-46b586-669f464d79-fsqx4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Normal Killing pod/payload-processing-e2e-inf-a-ddf61a-5f445c7548-fwpmj Stopping container payload-processing 25m Normal Killing pod/payload-pre-processing-e2e-inf-a-ddf61a-67ff4567bc-mrc2x Stopping container payload-pre-processing 24m Normal Killing pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-f6k98 Stopping container istio-proxy 24m Normal Killing pod/e2e-inf-a-ddf61a-openshift-default-f57f54bc-4gvfk Stopping container istio-proxy 24m Warning ProbeError pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-f6k98 Readiness probe error: HTTP probe failed with statuscode: 503... 24m Warning Unhealthy pod/e2e-worker-w0-46b586-openshift-default-86c598c96c-f6k98 Readiness probe failed: HTTP probe failed with statuscode: 503 24m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-ddf61a-67ff4567bc-mrc2x Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Warning Unhealthy pod/payload-processing-e2e-inf-a-ddf61a-5f445c7548-fwpmj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-8f6f87f59 (combined from similar events): Deleted pod: maas-default-gateway-openshift-default-8f6f87f59-mz295 24m Normal Killing pod/maas-default-gateway-openshift-default-8f6f87f59-mz295 Stopping container istio-proxy 24m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 2; reason: All metrics below target 24m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-8f6f87f59 from 3 to 2 24m Normal Killing pod/payload-pre-processing-e2e-inf-b-37abc0-69469c56c-6bzrh Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-inf-b-37abc0-54fc77c78-vvrhm Stopping container payload-processing 24m Normal Killing pod/e2e-inf-b-37abc0-openshift-default-65b888884f-pqjfl Stopping container istio-proxy 24m Warning Unhealthy pod/payload-processing-e2e-inf-b-37abc0-54fc77c78-vvrhm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-37abc0-69469c56c-6bzrh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Normal Killing pod/payload-processing-e2e-worker-w1-5c3374-7969595c9d-gzhtd Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-worker-w1-5c3374-f56fc6f79-7b9pp Stopping container payload-pre-processing 23m Normal ScalingReplicaSet deployment/e2e-worker-w1-5c3374-openshift-default Scaled up replica set e2e-worker-w1-5c3374-openshift-default-d6ff699c7 from 1 to 2 23m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-5c3374-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 23m Normal SuccessfulCreate replicaset/e2e-worker-w1-5c3374-openshift-default-d6ff699c7 Created pod: e2e-worker-w1-5c3374-openshift-default-d6ff699c7-s4kvf 23m Normal Started pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-s4kvf Container started 23m Normal Created pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-s4kvf Container created 23m Normal Pulled pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-s4kvf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-s4kvf Add eth0 [10.133.0.110/23] from ovn-kubernetes 23m Warning Unhealthy pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-s4kvf Startup probe failed: Get "http://10.133.0.110:15021/healthz/ready": dial tcp 10.133.0.110:15021: connect: connection refused 23m Warning ProbeError pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-s4kvf Startup probe error: Get "http://10.133.0.110:15021/healthz/ready": dial tcp 10.133.0.110:15021: connect: connection refused... 23m Normal Killing pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-s4kvf Stopping container istio-proxy 23m Normal Killing pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-ppwlm Stopping container istio-proxy 23m Warning ProbeError pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-ppwlm Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning Unhealthy pod/e2e-worker-w1-5c3374-openshift-default-d6ff699c7-ppwlm Readiness probe failed: HTTP probe failed with statuscode: 503 23m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-x28pp Add eth0 [10.133.0.113/23] from ovn-kubernetes 23m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 23m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-8f6f87f59 from 2 to 4 23m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-875hj Add eth0 [10.133.0.112/23] from ovn-kubernetes 23m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-x28pp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-875hj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-x28pp Container created 23m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-x28pp Container started 23m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-875hj Container created 23m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-875hj Container started 23m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-x28pp Startup probe failed: Get "http://10.133.0.113:15021/healthz/ready": dial tcp 10.133.0.113:15021: connect: connection refused 23m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-x28pp Startup probe error: Get "http://10.133.0.113:15021/healthz/ready": dial tcp 10.133.0.113:15021: connect: connection refused... 23m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-875hj Startup probe error: Get "http://10.133.0.112:15021/healthz/ready": dial tcp 10.133.0.112:15021: connect: connection refused... 23m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-875hj Startup probe failed: Get "http://10.133.0.112:15021/healthz/ready": dial tcp 10.133.0.112:15021: connect: connection refused 23m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: 23m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-8f6f87f59 from 4 to 5 22m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-4sgzb Container created 22m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-4sgzb Add eth0 [10.132.1.16/23] from ovn-kubernetes 22m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-4sgzb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-4sgzb Container started 22m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-4sgzb Startup probe failed: Get "http://10.132.1.16:15021/healthz/ready": dial tcp 10.132.1.16:15021: connect: connection refused 22m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-4sgzb Startup probe error: Get "http://10.132.1.16:15021/healthz/ready": dial tcp 10.132.1.16:15021: connect: connection refused... 22m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-lwhxc Readiness probe error: Get "http://10.132.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 22m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-lwhxc Readiness probe failed: Get "http://10.132.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-zgt8d Add eth0 [10.133.0.115/23] from ovn-kubernetes 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-pqml2 Add eth0 [10.134.0.173/23] from ovn-kubernetes 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-znmqk Add eth0 [10.134.0.172/23] from ovn-kubernetes 21m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 21m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-8f6f87f59 from 5 to 10 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-2xfw2 Add eth0 [10.133.0.116/23] from ovn-kubernetes 21m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-56c5v Container created 21m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-zgt8d Container created 21m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-2xfw2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-2xfw2 Container created 21m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-pqml2 Container started 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-8f6f87f59-56c5v Add eth0 [10.132.1.18/23] from ovn-kubernetes 21m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-2xfw2 Container started 21m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-zgt8d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-56c5v Container started 21m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-znmqk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-zgt8d Container started 21m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-znmqk Container created 21m Normal Started pod/maas-default-gateway-openshift-default-8f6f87f59-znmqk Container started 21m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-56c5v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Pulled pod/maas-default-gateway-openshift-default-8f6f87f59-pqml2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Created pod/maas-default-gateway-openshift-default-8f6f87f59-pqml2 Container created 21m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-56c5v Startup probe error: Get "http://10.132.1.18:15021/healthz/ready": dial tcp 10.132.1.18:15021: connect: connection refused... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-56c5v Startup probe failed: Get "http://10.132.1.18:15021/healthz/ready": dial tcp 10.132.1.18:15021: connect: connection refused 21m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-2xfw2 Startup probe error: Get "http://10.133.0.116:15021/healthz/ready": dial tcp 10.133.0.116:15021: connect: connection refused... 21m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-pqml2 Startup probe error: Get "http://10.134.0.173:15021/healthz/ready": dial tcp 10.134.0.173:15021: connect: connection refused... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-pqml2 Startup probe failed: Get "http://10.134.0.173:15021/healthz/ready": dial tcp 10.134.0.173:15021: connect: connection refused 21m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-znmqk Startup probe error: Get "http://10.134.0.172:15021/healthz/ready": dial tcp 10.134.0.172:15021: connect: connection refused... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-2xfw2 Startup probe failed: Get "http://10.133.0.116:15021/healthz/ready": dial tcp 10.133.0.116:15021: connect: connection refused 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-znmqk Startup probe failed: Get "http://10.134.0.172:15021/healthz/ready": dial tcp 10.134.0.172:15021: connect: connection refused 21m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-zgt8d Startup probe error: Get "http://10.133.0.115:15021/healthz/ready": dial tcp 10.133.0.115:15021: connect: connection refused... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-zgt8d Startup probe failed: Get "http://10.133.0.115:15021/healthz/ready": dial tcp 10.133.0.115:15021: connect: connection refused 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-56c5v Startup probe failed: Get "http://10.132.1.18:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 21m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-56c5v Startup probe error: Get "http://10.132.1.18:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-znmqk Startup probe failed: Get "http://10.134.0.172:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-pqml2 Startup probe failed: Get "http://10.134.0.173:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 21m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-zgt8d Startup probe error: Get "http://10.133.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-2xfw2 Startup probe failed: Get "http://10.133.0.116:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 21m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-pqml2 Startup probe error: Get "http://10.134.0.173:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-zgt8d Startup probe failed: Get "http://10.133.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 21m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-znmqk Startup probe error: Get "http://10.134.0.172:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 21m Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-2xfw2 Startup probe error: Get "http://10.133.0.116:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 15m Normal AddedLabel gateway/e2e-worker-single-main-0e6572 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-0e6572 15m Normal ScalingReplicaSet deployment/e2e-worker-single-main-0e6572-openshift-default Scaled up replica set e2e-worker-single-main-0e6572-openshift-default-5c475f9cbf from 0 to 1 15m Normal SuccessfulCreate replicaset/e2e-worker-single-main-0e6572-openshift-default-5c475f9cbf Created pod: e2e-worker-single-main-0e6572-openshift-default-5c475f9cbf97qjl 15m Normal AddedInterface pod/e2e-worker-single-main-0e6572-openshift-default-5c475f9cbf97qjl Add eth0 [10.132.1.32/23] from ovn-kubernetes 15m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-0e6572 Scaled up replica set payload-processing-e2e-worker-single-main-0e6572-64778475db from 0 to 1 15m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-0e6572-84687b7747 Error creating: pods "payload-pre-processing-e2e-worker-single-main-0e6572-84687b7747-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-0e6572: serviceaccount "payload-processing-e2e-worker-single-main-0e6572" not found 15m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-0e6572 Scaled up replica set payload-pre-processing-e2e-worker-single-main-0e6572-84687b7747 from 0 to 1 15m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-0e6572-64778475db Error creating: pods "payload-processing-e2e-worker-single-main-0e6572-64778475db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-0e6572: serviceaccount "payload-processing-e2e-worker-single-main-0e6572" not found 15m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-0e6572-8468772hmv Add eth0 [10.133.0.117/23] from ovn-kubernetes 15m Normal Created pod/payload-pre-processing-e2e-worker-single-main-0e6572-8468772hmv Container created 15m Normal Started pod/payload-pre-processing-e2e-worker-single-main-0e6572-8468772hmv Container started 15m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-0e6572-64778475db Created pod: payload-processing-e2e-worker-single-main-0e6572-64778475dfmdr6 15m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-0e6572-84687b7747 Created pod: payload-pre-processing-e2e-worker-single-main-0e6572-8468772hmv 15m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-0e6572-64778475dfmdr6 Add eth0 [10.133.0.118/23] from ovn-kubernetes 15m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-0e6572-8468772hmv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 15m Normal Started pod/payload-processing-e2e-worker-single-main-0e6572-64778475dfmdr6 Container started 15m Normal Pulled pod/payload-processing-e2e-worker-single-main-0e6572-64778475dfmdr6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 15m Normal Created pod/payload-processing-e2e-worker-single-main-0e6572-64778475dfmdr6 Container created 15m Normal Started pod/e2e-worker-single-main-0e6572-openshift-default-5c475f9cbf97qjl Container started 15m Normal Pulled pod/e2e-worker-single-main-0e6572-openshift-default-5c475f9cbf97qjl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 15m Normal Created pod/e2e-worker-single-main-0e6572-openshift-default-5c475f9cbf97qjl Container created 15m Warning ProbeError pod/e2e-worker-single-main-0e6572-openshift-default-5c475f9cbf97qjl Startup probe error: Get "http://10.132.1.32:15021/healthz/ready": dial tcp 10.132.1.32:15021: connect: connection refused... 15m Warning Unhealthy pod/e2e-worker-single-main-0e6572-openshift-default-5c475f9cbf97qjl Startup probe failed: Get "http://10.132.1.32:15021/healthz/ready": dial tcp 10.132.1.32:15021: connect: connection refused 15m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-0e6572-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 15m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-0e6572-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 14m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-0e6572-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 14m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-0e6572-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 14m Normal Killing pod/payload-pre-processing-e2e-worker-single-main-0e6572-8468772hmv Stopping container payload-pre-processing 14m Normal Killing pod/payload-processing-e2e-worker-single-main-0e6572-64778475dfmdr6 Stopping container payload-processing 14m Normal Killing pod/e2e-worker-single-main-0e6572-openshift-default-5c475f9cbf97qjl Stopping container istio-proxy 13m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-0e6572-8468772hmv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 13m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-0e6572-64778475dfmdr6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 11m Normal ScalingReplicaSet deployment/e2e-worker-single-main-e98014-openshift-default Scaled up replica set e2e-worker-single-main-e98014-openshift-default-596b484cdf from 0 to 1 11m Normal AddedLabel gateway/e2e-worker-single-main-e98014 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-e98014 11m Normal SuccessfulCreate replicaset/e2e-worker-single-main-e98014-openshift-default-596b484cdf Created pod: e2e-worker-single-main-e98014-openshift-default-596b484cdf5xsgc 11m Normal AddedInterface pod/e2e-worker-single-main-e98014-openshift-default-596b484cdf5xsgc Add eth0 [10.132.1.39/23] from ovn-kubernetes 11m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-e98014-54d9bccbb6 Error creating: pods "payload-processing-e2e-worker-single-main-e98014-54d9bccbb6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-e98014: serviceaccount "payload-processing-e2e-worker-single-main-e98014" not found 11m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-e98014 Scaled up replica set payload-processing-e2e-worker-single-main-e98014-54d9bccbb6 from 0 to 1 11m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-e98014-6989c556cf Created pod: payload-pre-processing-e2e-worker-single-main-e98014-6989c9wz5h 11m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-e98014 Scaled up replica set payload-pre-processing-e2e-worker-single-main-e98014-6989c556cf from 0 to 1 11m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-e98014-6989c556cf Error creating: pods "payload-pre-processing-e2e-worker-single-main-e98014-6989c556cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-e98014: serviceaccount "payload-processing-e2e-worker-single-main-e98014" not found 11m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-e98014-6989c9wz5h Add eth0 [10.133.0.120/23] from ovn-kubernetes 11m Normal Started pod/payload-pre-processing-e2e-worker-single-main-e98014-6989c9wz5h Container started 11m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-e98014-6989c9wz5h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 11m Normal Started pod/payload-processing-e2e-worker-single-main-e98014-54d9bccbbjcft2 Container started 11m Normal Created pod/payload-processing-e2e-worker-single-main-e98014-54d9bccbbjcft2 Container created 11m Normal Pulled pod/payload-processing-e2e-worker-single-main-e98014-54d9bccbbjcft2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 11m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-e98014-54d9bccbbjcft2 Add eth0 [10.133.0.121/23] from ovn-kubernetes 11m Normal Created pod/payload-pre-processing-e2e-worker-single-main-e98014-6989c9wz5h Container created 11m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-e98014-54d9bccbb6 Created pod: payload-processing-e2e-worker-single-main-e98014-54d9bccbbjcft2 10m Normal Started pod/e2e-worker-single-main-e98014-openshift-default-596b484cdf5xsgc Container started 10m Normal Created pod/e2e-worker-single-main-e98014-openshift-default-596b484cdf5xsgc Container created 10m Normal Pulled pod/e2e-worker-single-main-e98014-openshift-default-596b484cdf5xsgc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 10m Warning Unhealthy pod/e2e-worker-single-main-e98014-openshift-default-596b484cdf5xsgc Startup probe failed: Get "http://10.132.1.39:15021/healthz/ready": dial tcp 10.132.1.39:15021: connect: connection refused 10m Warning ProbeError pod/e2e-worker-single-main-e98014-openshift-default-596b484cdf5xsgc Startup probe error: Get "http://10.132.1.39:15021/healthz/ready": dial tcp 10.132.1.39:15021: connect: connection refused... 10m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-e98014-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 10m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-e98014-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 9m31s Normal Killing pod/payload-processing-e2e-worker-single-main-e98014-54d9bccbbjcft2 Stopping container payload-processing 9m31s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-e98014-6989c9wz5h Stopping container payload-pre-processing 9m19s Normal Killing pod/e2e-worker-single-main-e98014-openshift-default-596b484cdf5xsgc Stopping container istio-proxy 9m12s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-e98014-54d9bccbbjcft2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 9m12s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-e98014-6989c9wz5h Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 8m20s Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-4sgzb Readiness probe error: Get "http://10.132.1.16:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 8m20s Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-4sgzb Readiness probe failed: Get "http://10.132.1.16:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 8m20s Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-875hj Readiness probe error: Get "http://10.133.0.112:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 8m20s Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-875hj Readiness probe failed: Get "http://10.133.0.112:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 6m41s Normal SuccessfulCreate replicaset/e2e-ait-8214dff9-openshift-default-7b77bdd57c Created pod: e2e-ait-8214dff9-openshift-default-7b77bdd57c-zm5nl 6m41s Normal ScalingReplicaSet deployment/e2e-ait-8214dff9-openshift-default Scaled up replica set e2e-ait-8214dff9-openshift-default-7b77bdd57c from 0 to 1 6m41s Normal AddedLabel gateway/e2e-ait-8214dff9 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-8214dff9 6m40s Normal AddedInterface pod/e2e-ait-8214dff9-openshift-default-7b77bdd57c-zm5nl Add eth0 [10.132.1.45/23] from ovn-kubernetes 6m34s Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-8214dff9 Scaled up replica set payload-processing-e2e-ait-8214dff9-85b686f698 from 0 to 1 6m34s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-8214dff9 Scaled up replica set payload-pre-processing-e2e-ait-8214dff9-854cb688f9 from 0 to 1 6m33s Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-8214dff9-85b686f698 Created pod: payload-processing-e2e-ait-8214dff9-85b686f698-krx74 6m33s Normal AddedInterface pod/payload-pre-processing-e2e-ait-8214dff9-854cb688f9-z4k5h Add eth0 [10.133.0.126/23] from ovn-kubernetes 6m33s Warning FailedCreate replicaset/payload-processing-e2e-ait-8214dff9-85b686f698 Error creating: pods "payload-processing-e2e-ait-8214dff9-85b686f698-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-8214dff9: serviceaccount "payload-processing-e2e-ait-8214dff9" not found 6m33s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-8214dff9-854cb688f9 Created pod: payload-pre-processing-e2e-ait-8214dff9-854cb688f9-z4k5h 6m33s Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-8214dff9-854cb688f9 Error creating: pods "payload-pre-processing-e2e-ait-8214dff9-854cb688f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-8214dff9: serviceaccount "payload-processing-e2e-ait-8214dff9" not found 6m32s Normal Started pod/payload-pre-processing-e2e-ait-8214dff9-854cb688f9-z4k5h Container started 6m32s Normal Pulled pod/payload-processing-e2e-ait-8214dff9-85b686f698-krx74 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 6m32s Normal Created pod/payload-processing-e2e-ait-8214dff9-85b686f698-krx74 Container created 6m32s Normal Started pod/payload-processing-e2e-ait-8214dff9-85b686f698-krx74 Container started 6m32s Normal Created pod/payload-pre-processing-e2e-ait-8214dff9-854cb688f9-z4k5h Container created 6m32s Normal AddedInterface pod/payload-processing-e2e-ait-8214dff9-85b686f698-krx74 Add eth0 [10.133.0.127/23] from ovn-kubernetes 6m32s Normal Pulled pod/payload-pre-processing-e2e-ait-8214dff9-854cb688f9-z4k5h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 6m13s Normal Pulled pod/e2e-ait-8214dff9-openshift-default-7b77bdd57c-zm5nl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 6m13s Normal Created pod/e2e-ait-8214dff9-openshift-default-7b77bdd57c-zm5nl Container created 6m13s Normal Started pod/e2e-ait-8214dff9-openshift-default-7b77bdd57c-zm5nl Container started 6m11s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-8214dff9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 6m11s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-8214dff9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 6m8s Warning Unhealthy pod/e2e-ait-8214dff9-openshift-default-7b77bdd57c-zm5nl Startup probe failed: Get "http://10.132.1.45:15021/healthz/ready": dial tcp 10.132.1.45:15021: connect: connection refused 6m8s Warning ProbeError pod/e2e-ait-8214dff9-openshift-default-7b77bdd57c-zm5nl Startup probe error: Get "http://10.132.1.45:15021/healthz/ready": dial tcp 10.132.1.45:15021: connect: connection refused... 5m49s Normal Killing pod/payload-pre-processing-e2e-ait-8214dff9-854cb688f9-z4k5h Stopping container payload-pre-processing 5m49s Normal Killing pod/payload-processing-e2e-ait-8214dff9-85b686f698-krx74 Stopping container payload-processing 5m40s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-8214dff9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 5m40s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-8214dff9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 5m34s Normal Killing pod/e2e-ait-8214dff9-openshift-default-7b77bdd57c-zm5nl Stopping container istio-proxy 5m33s Normal AddedLabel gateway/e2e-authcfg-http-a3c14c Added label istio.io/rev=openshift-gateway to gateway e2e-authcfg-http-a3c14c 5m32s Normal AddedInterface pod/e2e-authcfg-http-a3c14c-openshift-default-5b7bd5fc65-bztx4 Add eth0 [10.133.0.129/23] from ovn-kubernetes 5m32s Normal SuccessfulCreate replicaset/e2e-authcfg-http-a3c14c-openshift-default-5b7bd5fc65 Created pod: e2e-authcfg-http-a3c14c-openshift-default-5b7bd5fc65-bztx4 5m32s Normal ScalingReplicaSet deployment/e2e-authcfg-http-a3c14c-openshift-default Scaled up replica set e2e-authcfg-http-a3c14c-openshift-default-5b7bd5fc65 from 0 to 1 5m25s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-authcfg-http-a3c14c-775858b75 Created pod: payload-pre-processing-e2e-authcfg-http-a3c14c-775858b75-gthbc 5m25s Normal ScalingReplicaSet deployment/payload-processing-e2e-authcfg-http-a3c14c Scaled up replica set payload-processing-e2e-authcfg-http-a3c14c-69bc5c5c66 from 0 to 1 5m25s Warning FailedCreate replicaset/payload-processing-e2e-authcfg-http-a3c14c-69bc5c5c66 Error creating: pods "payload-processing-e2e-authcfg-http-a3c14c-69bc5c5c66-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-a3c14c: serviceaccount "payload-processing-e2e-authcfg-http-a3c14c" not found 5m25s Warning FailedCreate replicaset/payload-pre-processing-e2e-authcfg-http-a3c14c-775858b75 Error creating: pods "payload-pre-processing-e2e-authcfg-http-a3c14c-775858b75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-a3c14c: serviceaccount "payload-processing-e2e-authcfg-http-a3c14c" not found 5m25s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-authcfg-http-a3c14c Scaled up replica set payload-pre-processing-e2e-authcfg-http-a3c14c-775858b75 from 0 to 1 5m24s Normal SuccessfulCreate replicaset/payload-processing-e2e-authcfg-http-a3c14c-69bc5c5c66 Created pod: payload-processing-e2e-authcfg-http-a3c14c-69bc5c5c66-pg9zj 5m24s Normal Pulled pod/payload-processing-e2e-authcfg-http-a3c14c-69bc5c5c66-pg9zj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m24s Normal Pulled pod/payload-pre-processing-e2e-authcfg-http-a3c14c-775858b75-gthbc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m24s Normal Created pod/payload-processing-e2e-authcfg-http-a3c14c-69bc5c5c66-pg9zj Container created 5m24s Normal Started pod/payload-pre-processing-e2e-authcfg-http-a3c14c-775858b75-gthbc Container started 5m24s Normal Created pod/payload-pre-processing-e2e-authcfg-http-a3c14c-775858b75-gthbc Container created 5m24s Normal Started pod/payload-processing-e2e-authcfg-http-a3c14c-69bc5c5c66-pg9zj Container started 5m24s Normal AddedInterface pod/payload-processing-e2e-authcfg-http-a3c14c-69bc5c5c66-pg9zj Add eth0 [10.132.1.47/23] from ovn-kubernetes 5m24s Normal AddedInterface pod/payload-pre-processing-e2e-authcfg-http-a3c14c-775858b75-gthbc Add eth0 [10.133.0.130/23] from ovn-kubernetes 5m5s Normal Pulled pod/e2e-authcfg-http-a3c14c-openshift-default-5b7bd5fc65-bztx4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 5m5s Normal Created pod/e2e-authcfg-http-a3c14c-openshift-default-5b7bd5fc65-bztx4 Container created 5m5s Normal Started pod/e2e-authcfg-http-a3c14c-openshift-default-5b7bd5fc65-bztx4 Container started 5m2s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-a3c14c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 5m2s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-a3c14c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 4m59s Warning ProbeError pod/e2e-authcfg-http-a3c14c-openshift-default-5b7bd5fc65-bztx4 Startup probe error: Get "http://10.133.0.129:15021/healthz/ready": dial tcp 10.133.0.129:15021: connect: connection refused... 4m59s Warning Unhealthy pod/e2e-authcfg-http-a3c14c-openshift-default-5b7bd5fc65-bztx4 Startup probe failed: Get "http://10.133.0.129:15021/healthz/ready": dial tcp 10.133.0.129:15021: connect: connection refused 4m32s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-a3c14c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m32s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-a3c14c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 3m32s Normal Killing pod/payload-processing-e2e-authcfg-http-a3c14c-69bc5c5c66-pg9zj Stopping container payload-processing 3m31s Normal Killing pod/payload-pre-processing-e2e-authcfg-http-a3c14c-775858b75-gthbc Stopping container payload-pre-processing 3m11s Normal Killing pod/e2e-authcfg-http-a3c14c-openshift-default-5b7bd5fc65-bztx4 Stopping container istio-proxy 2m50s Warning ProbeError pod/maas-default-gateway-openshift-default-8f6f87f59-x28pp Readiness probe error: Get "http://10.133.0.113:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 2m50s Warning Unhealthy pod/maas-default-gateway-openshift-default-8f6f87f59-x28pp Readiness probe failed: Get "http://10.133.0.113:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)