=== events in openshift-ingress (2026-11-03T23:32:03Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 59m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-fsjfb Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fsjfb to ip-10-0-135-87.us-west-1.compute.internal 41m Normal Scheduled pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-b9x4s Successfully assigned openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-b9x4s to ip-10-0-143-132.us-west-1.compute.internal 22m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-l5859 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-l5859 to ip-10-0-132-187.us-west-1.compute.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-lxr98 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-lxr98 to ip-10-0-143-132.us-west-1.compute.internal 36m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-njmdj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-njmdj to ip-10-0-143-132.us-west-1.compute.internal 32m Normal Scheduled pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-8jb2h Successfully assigned openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-8jb2h to ip-10-0-135-87.us-west-1.compute.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-np57c Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-np57c to ip-10-0-132-187.us-west-1.compute.internal 65m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-nqmgk Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-nqmgk to ip-10-0-143-132.us-west-1.compute.internal 38m Normal Scheduled pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-5wnpt Successfully assigned openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-5wnpt to ip-10-0-143-132.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs Successfully assigned openshift-ingress/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs to ip-10-0-132-187.us-west-1.compute.internal 36m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-qk9bp Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qk9bp to ip-10-0-132-187.us-west-1.compute.internal 22m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-qp7hb Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-qp7hb to ip-10-0-132-187.us-west-1.compute.internal 59m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-vj28x Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-vj28x to ip-10-0-143-132.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-sm442 Successfully assigned openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-sm442 to ip-10-0-143-132.us-west-1.compute.internal 32m Normal Scheduled pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx Successfully assigned openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx to ip-10-0-132-187.us-west-1.compute.internal 43m Normal Scheduled pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r Successfully assigned openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r to ip-10-0-132-187.us-west-1.compute.internal 60m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-vs7zn Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-vs7zn to ip-10-0-143-132.us-west-1.compute.internal 32m Normal Scheduled pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-27kc9 Successfully assigned openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-27kc9 to ip-10-0-143-132.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp Successfully assigned openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp to ip-10-0-132-187.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-qk9xk Successfully assigned openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-qk9xk to ip-10-0-143-132.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-g6qdv Successfully assigned openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-g6qdv to ip-10-0-143-132.us-west-1.compute.internal 52m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-wtfpt Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-wtfpt to ip-10-0-143-132.us-west-1.compute.internal 60m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-kr95p Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kr95p to ip-10-0-132-187.us-west-1.compute.internal 36m Normal Scheduled pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-zktwm Successfully assigned openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-zktwm to ip-10-0-135-87.us-west-1.compute.internal 70m Normal Scheduled pod/router-default-79487cbf4c-klb6d Successfully assigned openshift-ingress/router-default-79487cbf4c-klb6d to ip-10-0-135-87.us-west-1.compute.internal 70m Warning FailedScheduling pod/router-default-79487cbf4c-klb6d no nodes available to schedule pods 44m Normal Scheduled pod/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8 Successfully assigned openshift-ingress/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8 to ip-10-0-132-187.us-west-1.compute.internal 76m Warning FailedScheduling pod/router-default-79487cbf4c-klb6d no nodes available to schedule pods 41m Normal Scheduled pod/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z Successfully assigned openshift-ingress/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z to ip-10-0-132-187.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-44frr Successfully assigned openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-44frr to ip-10-0-135-87.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql Successfully assigned openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql to ip-10-0-132-187.us-west-1.compute.internal 47m Normal Scheduled pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-vvn46 Successfully assigned openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-vvn46 to ip-10-0-143-132.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x Successfully assigned openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x to ip-10-0-132-187.us-west-1.compute.internal 45m Normal Scheduled pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr Successfully assigned openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr to ip-10-0-132-187.us-west-1.compute.internal 55m Normal Scheduled pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-tww6f Successfully assigned openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-tww6f to ip-10-0-143-132.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-nc2kj Successfully assigned openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-nc2kj to ip-10-0-143-132.us-west-1.compute.internal 57m Normal Scheduled pod/payload-processing-e2e-worker-w6-7540da-59844f9b7b-z68qt Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-7540da-59844f9b7b-z68qt to ip-10-0-135-87.us-west-1.compute.internal 57m Normal Scheduled pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx Successfully assigned openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx to ip-10-0-132-187.us-west-1.compute.internal 38m Normal Scheduled pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-lc9v2 Successfully assigned openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-lc9v2 to ip-10-0-143-132.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-wb2g5 Successfully assigned openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-wb2g5 to ip-10-0-143-132.us-west-1.compute.internal 55m Normal Scheduled pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-m7rjj Successfully assigned openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-m7rjj to ip-10-0-135-87.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt Successfully assigned openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt to ip-10-0-132-187.us-west-1.compute.internal 65m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-gwwlg Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-gwwlg to ip-10-0-132-187.us-west-1.compute.internal 47m Normal Scheduled pod/e2e-ait-468d8d1b-openshift-default-685f7b56f5-rkfh8 Successfully assigned openshift-ingress/e2e-ait-468d8d1b-openshift-default-685f7b56f5-rkfh8 to ip-10-0-135-87.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-wlh8v Successfully assigned openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-wlh8v to ip-10-0-143-132.us-west-1.compute.internal 36m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-kpmpx Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kpmpx to ip-10-0-143-132.us-west-1.compute.internal 57m Normal Scheduled pod/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8 to ip-10-0-132-187.us-west-1.compute.internal 43m Normal Scheduled pod/e2e-shared-a-w3-eb3c33-openshift-default-8687c6c447-x7w6q Successfully assigned openshift-ingress/e2e-shared-a-w3-eb3c33-openshift-default-8687c6c447-x7w6q to ip-10-0-135-87.us-west-1.compute.internal 47m Normal Scheduled pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr Successfully assigned openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr to ip-10-0-132-187.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl Successfully assigned openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl to ip-10-0-132-187.us-west-1.compute.internal 32m Normal Scheduled pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-hjhmd Successfully assigned openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-hjhmd to ip-10-0-135-87.us-west-1.compute.internal 57m Normal Scheduled pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-q95hh Successfully assigned openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-q95hh to ip-10-0-143-132.us-west-1.compute.internal 58m Normal Scheduled pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9 Successfully assigned openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9 to ip-10-0-132-187.us-west-1.compute.internal 47m Normal Scheduled pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj Successfully assigned openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj to ip-10-0-132-187.us-west-1.compute.internal 59m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-xtmcj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-xtmcj to ip-10-0-132-187.us-west-1.compute.internal 62m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-cwdgf Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-cwdgf to ip-10-0-132-187.us-west-1.compute.internal 58m Normal Scheduled pod/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h to ip-10-0-132-187.us-west-1.compute.internal 36m Normal Scheduled pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-7xpv9 Successfully assigned openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-7xpv9 to ip-10-0-135-87.us-west-1.compute.internal 44m Normal Scheduled pod/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb Successfully assigned openshift-ingress/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb to ip-10-0-132-187.us-west-1.compute.internal 33m Normal Scheduled pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-kc75t Successfully assigned openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-kc75t to ip-10-0-143-132.us-west-1.compute.internal 6m20s Normal Scheduled pod/e2e-ait-738d0c9a-openshift-default-6dd5576d67-lcmqs Successfully assigned openshift-ingress/e2e-ait-738d0c9a-openshift-default-6dd5576d67-lcmqs to ip-10-0-135-87.us-west-1.compute.internal 45m Normal Scheduled pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-8hmfx Successfully assigned openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-8hmfx to ip-10-0-135-87.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5 Successfully assigned openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5 to ip-10-0-132-187.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz Successfully assigned openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz to ip-10-0-132-187.us-west-1.compute.internal 58m Normal Scheduled pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r Successfully assigned openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r to ip-10-0-132-187.us-west-1.compute.internal 46m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-vndrh Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vndrh to ip-10-0-135-87.us-west-1.compute.internal 57m Normal Scheduled pod/payload-processing-e2e-worker-w3-1d31f8-64f6fcf754-48jnl Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-1d31f8-64f6fcf754-48jnl to ip-10-0-135-87.us-west-1.compute.internal 44m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-vsltn Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-vsltn to ip-10-0-143-132.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-nprdv Successfully assigned openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-nprdv to ip-10-0-135-87.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-8mkq4 Successfully assigned openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-8mkq4 to ip-10-0-135-87.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-gjgqz Successfully assigned openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-gjgqz to ip-10-0-135-87.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-7bggr Successfully assigned openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-7bggr to ip-10-0-135-87.us-west-1.compute.internal 54m Normal Scheduled pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-h59rg Successfully assigned openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-h59rg to ip-10-0-143-132.us-west-1.compute.internal 32m Normal Scheduled pod/e2e-api-a-1048fc-openshift-default-887fc8d4f-lq7gz Successfully assigned openshift-ingress/e2e-api-a-1048fc-openshift-default-887fc8d4f-lq7gz to ip-10-0-143-132.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg Successfully assigned openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg to ip-10-0-132-187.us-west-1.compute.internal 57m Normal Scheduled pod/payload-processing-e2e-worker-w2-164c36-74dc8ddf7b-wqgvc Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-164c36-74dc8ddf7b-wqgvc to ip-10-0-135-87.us-west-1.compute.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-zhbvw Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-zhbvw to ip-10-0-135-87.us-west-1.compute.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-zkrw7 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-zkrw7 to ip-10-0-135-87.us-west-1.compute.internal 45m Normal Scheduled pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-4hrkd Successfully assigned openshift-ingress/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-4hrkd to ip-10-0-143-132.us-west-1.compute.internal 62m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-k2njq Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-k2njq to ip-10-0-132-187.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs Successfully assigned openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs to ip-10-0-132-187.us-west-1.compute.internal 41m Normal Scheduled pod/e2e-shared-a-w0-ea1be0-openshift-default-6bfb8dcf95-wl6kg Successfully assigned openshift-ingress/e2e-shared-a-w0-ea1be0-openshift-default-6bfb8dcf95-wl6kg to ip-10-0-135-87.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-shfnk Successfully assigned openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-shfnk to ip-10-0-143-132.us-west-1.compute.internal 63m Normal Scheduled pod/kube-auth-proxy-586b8ddf6d-h9m6s Successfully assigned openshift-ingress/kube-auth-proxy-586b8ddf6d-h9m6s to ip-10-0-132-187.us-west-1.compute.internal 58m Normal Scheduled pod/payload-processing-e2e-worker-w1-7afb1c-5777db7584-dvxk5 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-7afb1c-5777db7584-dvxk5 to ip-10-0-135-87.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf Successfully assigned openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf to ip-10-0-132-187.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9p25h Successfully assigned openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9p25h to ip-10-0-143-132.us-west-1.compute.internal 10m Normal Scheduled pod/e2e-worker-single-main-60ef36-openshift-default-86dc6c59d5qhpc5 Successfully assigned openshift-ingress/e2e-worker-single-main-60ef36-openshift-default-86dc6c59d5qhpc5 to ip-10-0-135-87.us-west-1.compute.internal 39m Normal Scheduled pod/e2e-api-a-26444f-openshift-default-695897496f-mcdxk Successfully assigned openshift-ingress/e2e-api-a-26444f-openshift-default-695897496f-mcdxk to ip-10-0-143-132.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz Successfully assigned openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz to ip-10-0-132-187.us-west-1.compute.internal 32m Normal Scheduled pod/e2e-inf-b-e66d59-openshift-default-6b654555-8jvm4 Successfully assigned openshift-ingress/e2e-inf-b-e66d59-openshift-default-6b654555-8jvm4 to ip-10-0-143-132.us-west-1.compute.internal 63m Normal Scheduled pod/kube-auth-proxy-586b8ddf6d-sgzck Successfully assigned openshift-ingress/kube-auth-proxy-586b8ddf6d-sgzck to ip-10-0-143-132.us-west-1.compute.internal 32m Normal Scheduled pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-lj7c6 Successfully assigned openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-lj7c6 to ip-10-0-143-132.us-west-1.compute.internal 58m Normal Scheduled pod/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds to ip-10-0-132-187.us-west-1.compute.internal 15m Normal Scheduled pod/payload-processing-e2e-worker-single-main-8a134f-77cfb5fcfnt7zf Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-8a134f-77cfb5fcfnt7zf to ip-10-0-135-87.us-west-1.compute.internal 10m Normal Scheduled pod/payload-processing-e2e-worker-single-main-60ef36-656858f446mbq7 Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-60ef36-656858f446mbq7 to ip-10-0-135-87.us-west-1.compute.internal 38m Normal Scheduled pod/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq to ip-10-0-132-187.us-west-1.compute.internal 40m Normal Scheduled pod/payload-processing-e2e-shared-b-w3-433db8-685d97b9d8-b8cj9 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-433db8-685d97b9d8-b8cj9 to ip-10-0-135-87.us-west-1.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn to ip-10-0-132-187.us-west-1.compute.internal 38m Normal Scheduled pod/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr to ip-10-0-132-187.us-west-1.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-shared-a-w4-de5715-7c6c757c54-cr9p8 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-de5715-7c6c757c54-cr9p8 to ip-10-0-135-87.us-west-1.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-shared-a-w3-eb3c33-846894ccd8-mzt64 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-eb3c33-846894ccd8-mzt64 to ip-10-0-135-87.us-west-1.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-c93e70-5ffc55b946-vqsqn Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-c93e70-5ffc55b946-vqsqn to ip-10-0-135-87.us-west-1.compute.internal 32m Normal Scheduled pod/e2e-api-a-ea5c8f-openshift-default-54496b9b7c-d2vcm Successfully assigned openshift-ingress/e2e-api-a-ea5c8f-openshift-default-54496b9b7c-d2vcm to ip-10-0-143-132.us-west-1.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-shared-a-w0-ea1be0-74784f4fdd-5g9xp Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-ea1be0-74784f4fdd-5g9xp to ip-10-0-135-87.us-west-1.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4 Successfully assigned openshift-ingress/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4 to ip-10-0-132-187.us-west-1.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs Successfully assigned openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs to ip-10-0-132-187.us-west-1.compute.internal 35m Normal Scheduled pod/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-7f2l6 Successfully assigned openshift-ingress/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-7f2l6 to ip-10-0-135-87.us-west-1.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-migrate-fc129ccf-9f4d47cd6-bv8bj Successfully assigned openshift-ingress/payload-processing-e2e-migrate-fc129ccf-9f4d47cd6-bv8bj to ip-10-0-135-87.us-west-1.compute.internal 32m Normal Scheduled pod/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp to ip-10-0-132-187.us-west-1.compute.internal 31m Normal Scheduled pod/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5 to ip-10-0-132-187.us-west-1.compute.internal 30m Normal Scheduled pod/payload-processing-e2e-inf-b-78be2c-598f68f6f6-bdszd Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-78be2c-598f68f6f6-bdszd to ip-10-0-135-87.us-west-1.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-inf-a-f8cd0c-6b7c848d84-j265n Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-f8cd0c-6b7c848d84-j265n to ip-10-0-135-87.us-west-1.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg to ip-10-0-132-187.us-west-1.compute.internal 38m Normal Scheduled pod/payload-processing-e2e-inf-a-a91d2d-7b5d5fbfbb-k5gzl Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-a91d2d-7b5d5fbfbb-k5gzl to ip-10-0-135-87.us-west-1.compute.internal 33m Normal Scheduled pod/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6 to ip-10-0-132-187.us-west-1.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp Successfully assigned openshift-ingress/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp to ip-10-0-132-187.us-west-1.compute.internal 5m20s Normal Scheduled pod/payload-processing-e2e-authcfg-http-034b52-7c869c85b4-57j52 Successfully assigned openshift-ingress/payload-processing-e2e-authcfg-http-034b52-7c869c85b4-57j52 to ip-10-0-135-87.us-west-1.compute.internal 39m Normal Scheduled pod/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df Successfully assigned openshift-ingress/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df to ip-10-0-132-187.us-west-1.compute.internal 29m Normal Scheduled pod/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd Successfully assigned openshift-ingress/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd to ip-10-0-132-187.us-west-1.compute.internal 39m Normal Scheduled pod/payload-processing-e2e-api-a-efce83-84867f4d46-24l2f Successfully assigned openshift-ingress/payload-processing-e2e-api-a-efce83-84867f4d46-24l2f to ip-10-0-135-87.us-west-1.compute.internal 32m Normal Scheduled pod/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw Successfully assigned openshift-ingress/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw to ip-10-0-132-187.us-west-1.compute.internal 39m Normal Scheduled pod/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks Successfully assigned openshift-ingress/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks to ip-10-0-132-187.us-west-1.compute.internal 32m Normal Scheduled pod/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4 to ip-10-0-132-187.us-west-1.compute.internal 6m12s Normal Scheduled pod/payload-processing-e2e-ait-738d0c9a-57b49dd7f6-9vwgk Successfully assigned openshift-ingress/payload-processing-e2e-ait-738d0c9a-57b49dd7f6-9vwgk to ip-10-0-135-87.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-2hmfk Successfully assigned openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-2hmfk to ip-10-0-143-132.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4 Successfully assigned openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4 to ip-10-0-132-187.us-west-1.compute.internal 50m Normal Scheduled pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-cb25n Successfully assigned openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-cb25n to ip-10-0-143-132.us-west-1.compute.internal 47m Normal Scheduled pod/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz Successfully assigned openshift-ingress/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz to ip-10-0-132-187.us-west-1.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch Successfully assigned openshift-ingress/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch to ip-10-0-132-187.us-west-1.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6 Successfully assigned openshift-ingress/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6 to ip-10-0-132-187.us-west-1.compute.internal 45m Normal Scheduled pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-xfkst Successfully assigned openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-xfkst to ip-10-0-135-87.us-west-1.compute.internal 57m Normal Scheduled pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-rsqdc Successfully assigned openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-rsqdc to ip-10-0-135-87.us-west-1.compute.internal 54m Normal Scheduled pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-9qxhp Successfully assigned openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-9qxhp to ip-10-0-135-87.us-west-1.compute.internal 58m Normal Scheduled pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-8vndl Successfully assigned openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-8vndl to ip-10-0-135-87.us-west-1.compute.internal 29m Normal Scheduled pod/e2e-api-b-cfe1bd-openshift-default-7985f9c557-5kzht Successfully assigned openshift-ingress/e2e-api-b-cfe1bd-openshift-default-7985f9c557-5kzht to ip-10-0-135-87.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q Successfully assigned openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q to ip-10-0-132-187.us-west-1.compute.internal 65m Normal Scheduled pod/openshift-ai-inference-openshift-ai-inference-65dccbd94c-kkznv Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-65dccbd94c-kkznv to ip-10-0-135-87.us-west-1.compute.internal 31m Normal Scheduled pod/e2e-inf-b-dcf5a5-openshift-default-68bf4b7597-qtcnn Successfully assigned openshift-ingress/e2e-inf-b-dcf5a5-openshift-default-68bf4b7597-qtcnn to ip-10-0-143-132.us-west-1.compute.internal 59m Normal Scheduled pod/payload-processing-7bb9489676-wqbz4 Successfully assigned openshift-ingress/payload-processing-7bb9489676-wqbz4 to ip-10-0-132-187.us-west-1.compute.internal 59m Normal Scheduled pod/payload-pre-processing-7b98bb678c-vxr7w Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-vxr7w to ip-10-0-132-187.us-west-1.compute.internal 62m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-jr4dh Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-jr4dh to ip-10-0-135-87.us-west-1.compute.internal 43m Normal Scheduled pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-tg5g7 Successfully assigned openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-tg5g7 to ip-10-0-135-87.us-west-1.compute.internal 27m Normal Scheduled pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5xrfk Successfully assigned openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5xrfk to ip-10-0-143-132.us-west-1.compute.internal 58m Normal Scheduled pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-ldktd Successfully assigned openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-ldktd to ip-10-0-143-132.us-west-1.compute.internal 52m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-2jqg5 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2jqg5 to ip-10-0-135-87.us-west-1.compute.internal 58m Normal Scheduled pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf Successfully assigned openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf to ip-10-0-132-187.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-2jsvl Successfully assigned openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-2jsvl to ip-10-0-135-87.us-west-1.compute.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-7540da-598d566c67-ld8zm Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-7540da-598d566c67-ld8zm to ip-10-0-135-87.us-west-1.compute.internal 57m Normal Scheduled pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-2nqsk Successfully assigned openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-2nqsk to ip-10-0-135-87.us-west-1.compute.internal 5m27s Normal Scheduled pod/e2e-authcfg-http-034b52-openshift-default-546fbd7649-76chx Successfully assigned openshift-ingress/e2e-authcfg-http-034b52-openshift-default-546fbd7649-76chx to ip-10-0-135-87.us-west-1.compute.internal 42m Normal Scheduled pod/e2e-shared-b-w2-4a1480-openshift-default-75c6f8849c-vw8pq Successfully assigned openshift-ingress/e2e-shared-b-w2-4a1480-openshift-default-75c6f8849c-vw8pq to ip-10-0-135-87.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-hsn96 Successfully assigned openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-hsn96 to ip-10-0-135-87.us-west-1.compute.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-164c36-84d6767d47-wn2ct Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-164c36-84d6767d47-wn2ct to ip-10-0-135-87.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq Successfully assigned openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq to ip-10-0-132-187.us-west-1.compute.internal 58m Normal Scheduled pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-g7kbs Successfully assigned openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-g7kbs to ip-10-0-135-87.us-west-1.compute.internal 22m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-2z78m Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-2z78m to ip-10-0-132-187.us-west-1.compute.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg to ip-10-0-132-187.us-west-1.compute.internal 52m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-4bjss Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-4bjss to ip-10-0-135-87.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-8jkmq Successfully assigned openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-8jkmq to ip-10-0-135-87.us-west-1.compute.internal 57m Normal Scheduled pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2 Successfully assigned openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2 to ip-10-0-132-187.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-w6k2c Successfully assigned openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-w6k2c to ip-10-0-143-132.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-pwwqr Successfully assigned openshift-ingress/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-pwwqr to ip-10-0-143-132.us-west-1.compute.internal 41m Normal Scheduled pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb Successfully assigned openshift-ingress/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb to ip-10-0-132-187.us-west-1.compute.internal 47m Normal Scheduled pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-9v2fq Successfully assigned openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-9v2fq to ip-10-0-135-87.us-west-1.compute.internal 58m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf to ip-10-0-132-187.us-west-1.compute.internal 54m Normal Scheduled pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-cr5jb Successfully assigned openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-cr5jb to ip-10-0-135-87.us-west-1.compute.internal 44m Normal Scheduled pod/e2e-derive-9d7aa682-openshift-default-74bdd985b5-sv4td Successfully assigned openshift-ingress/e2e-derive-9d7aa682-openshift-default-74bdd985b5-sv4td to ip-10-0-143-132.us-west-1.compute.internal 40m Normal Scheduled pod/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs Successfully assigned openshift-ingress/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs to ip-10-0-132-187.us-west-1.compute.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-5wrhg Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-5wrhg to ip-10-0-132-187.us-west-1.compute.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-6jrgz Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-6jrgz to ip-10-0-143-132.us-west-1.compute.internal 36m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-8q2rz Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-8q2rz to ip-10-0-143-132.us-west-1.compute.internal 59m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-9gvgr Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-9gvgr to ip-10-0-135-87.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-54fm8 Successfully assigned openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-54fm8 to ip-10-0-143-132.us-west-1.compute.internal 33m Normal Scheduled pod/e2e-inf-a-924fba-openshift-default-dd49687c5-phg8v Successfully assigned openshift-ingress/e2e-inf-a-924fba-openshift-default-dd49687c5-phg8v to ip-10-0-143-132.us-west-1.compute.internal 22m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-cdxtr Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-cdxtr to ip-10-0-143-132.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-xvdvr Successfully assigned openshift-ingress/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-xvdvr to ip-10-0-135-87.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-nlgd9 Successfully assigned openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-nlgd9 to ip-10-0-143-132.us-west-1.compute.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-d2tgj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-d2tgj to ip-10-0-132-187.us-west-1.compute.internal 22m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-fn2dg Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-fn2dg to ip-10-0-132-187.us-west-1.compute.internal 15m Normal Scheduled pod/e2e-worker-single-main-8a134f-openshift-default-6cfbcbcbc7jr9zq Successfully assigned openshift-ingress/e2e-worker-single-main-8a134f-openshift-default-6cfbcbcbc7jr9zq to ip-10-0-135-87.us-west-1.compute.internal 58m Normal Scheduled pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-p4dg2 Successfully assigned openshift-ingress/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-p4dg2 to ip-10-0-135-87.us-west-1.compute.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-1d31f8-f85f6f659-t2mm6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-1d31f8-f85f6f659-t2mm6 to ip-10-0-135-87.us-west-1.compute.internal 63m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-7cfc97f8f7xkbnh Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-7cfc97f8f7xkbnh to ip-10-0-135-87.us-west-1.compute.internal 39m Normal Scheduled pod/e2e-shared-b-w0-0406a8-openshift-default-689d85c855-l64hj Successfully assigned openshift-ingress/e2e-shared-b-w0-0406a8-openshift-default-689d85c855-l64hj to ip-10-0-143-132.us-west-1.compute.internal 47m Normal Scheduled pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb Successfully assigned openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb to ip-10-0-132-187.us-west-1.compute.internal 60m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-hn8gd Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-hn8gd to ip-10-0-135-87.us-west-1.compute.internal 36m Normal Scheduled pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb Successfully assigned openshift-ingress/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb to ip-10-0-132-187.us-west-1.compute.internal 31m Normal Scheduled pod/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq Successfully assigned openshift-ingress/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq to ip-10-0-132-187.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-klsq2 Successfully assigned openshift-ingress/e2e-worker-w2-164c36-openshift-default-54b556b4c9-klsq2 to ip-10-0-135-87.us-west-1.compute.internal 32m Normal Scheduled pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp Successfully assigned openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp to ip-10-0-132-187.us-west-1.compute.internal 33m Normal Scheduled pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-sn5r8 Successfully assigned openshift-ingress/e2e-mt-a9ab23c4-openshift-default-766c495bb9-sn5r8 to ip-10-0-135-87.us-west-1.compute.internal 58m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-7afb1c-869549494c-272x5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-7afb1c-869549494c-272x5 to ip-10-0-135-87.us-west-1.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw to ip-10-0-132-187.us-west-1.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9 to ip-10-0-132-187.us-west-1.compute.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x to ip-10-0-132-187.us-west-1.compute.internal 45m Normal Scheduled pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-6jvmm Successfully assigned openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-6jvmm to ip-10-0-143-132.us-west-1.compute.internal 38m Normal Scheduled pod/e2e-inf-a-a91d2d-openshift-default-6b9b889877-szkd2 Successfully assigned openshift-ingress/e2e-inf-a-a91d2d-openshift-default-6b9b889877-szkd2 to ip-10-0-143-132.us-west-1.compute.internal 47m Normal Scheduled pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-7qx7j Successfully assigned openshift-ingress/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-7qx7j to ip-10-0-135-87.us-west-1.compute.internal 52m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-hn9sv Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-hn9sv to ip-10-0-135-87.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp Successfully assigned openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp to ip-10-0-132-187.us-west-1.compute.internal 58m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx to ip-10-0-132-187.us-west-1.compute.internal 15m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-8a134f-57db7wbxl4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-8a134f-57db7wbxl4 to ip-10-0-135-87.us-west-1.compute.internal 10m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-60ef36-77855qtqr8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-60ef36-77855qtqr8 to ip-10-0-135-87.us-west-1.compute.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn to ip-10-0-132-187.us-west-1.compute.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk to ip-10-0-132-187.us-west-1.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm to ip-10-0-132-187.us-west-1.compute.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w0-0406a8-648f9b947-9vn4j Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-0406a8-648f9b947-9vn4j to ip-10-0-135-87.us-west-1.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w4-de5715-89cc48587-2nf7c Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-de5715-89cc48587-2nf7c to ip-10-0-135-87.us-west-1.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w3-eb3c33-6ff459c8c9-jfqlz Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-eb3c33-6ff459c8c9-jfqlz to ip-10-0-135-87.us-west-1.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-c93e70-8fbcf64c-bvn98 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c93e70-8fbcf64c-bvn98 to ip-10-0-135-87.us-west-1.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w0-ea1be0-59585bc9f9-766gw Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ea1be0-59585bc9f9-766gw to ip-10-0-135-87.us-west-1.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-mt-ebffae22-76f58b75cf-kmqd4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-ebffae22-76f58b75cf-kmqd4 to ip-10-0-135-87.us-west-1.compute.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-s4t6f Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-s4t6f to ip-10-0-135-87.us-west-1.compute.internal 35m Normal Scheduled pod/e2e-inf-a-efd39a-openshift-default-5985657f6c-fp6r2 Successfully assigned openshift-ingress/e2e-inf-a-efd39a-openshift-default-5985657f6c-fp6r2 to ip-10-0-135-87.us-west-1.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb to ip-10-0-132-187.us-west-1.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-migrate-fc129ccf-75b6c8f7d7-qtmzt Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-fc129ccf-75b6c8f7d7-qtmzt to ip-10-0-135-87.us-west-1.compute.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7 to ip-10-0-132-187.us-west-1.compute.internal 31m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-dcf5a5-9f4df6cf5-pt975 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-dcf5a5-9f4df6cf5-pt975 to ip-10-0-135-87.us-west-1.compute.internal 30m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-78be2c-69859d788c-vtdqn Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-78be2c-69859d788c-vtdqn to ip-10-0-135-87.us-west-1.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55 to ip-10-0-132-187.us-west-1.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp to ip-10-0-132-187.us-west-1.compute.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-a91d2d-c8f444867-j8llw Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-a91d2d-c8f444867-j8llw to ip-10-0-135-87.us-west-1.compute.internal 33m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7 to ip-10-0-132-187.us-west-1.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b to ip-10-0-132-187.us-west-1.compute.internal 5m20s Normal Scheduled pod/payload-pre-processing-e2e-authcfg-http-034b52-555f758647-zv96x Successfully assigned openshift-ingress/payload-pre-processing-e2e-authcfg-http-034b52-555f758647-zv96x to ip-10-0-135-87.us-west-1.compute.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk to ip-10-0-132-187.us-west-1.compute.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-api-a-efce83-5c4b5c8f8c-xpbhs Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-efce83-5c4b5c8f8c-xpbhs to ip-10-0-135-87.us-west-1.compute.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-api-a-ea5c8f-7db7d5b55f-qnbc8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-ea5c8f-7db7d5b55f-qnbc8 to ip-10-0-143-132.us-west-1.compute.internal 34m Normal Scheduled pod/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5 Successfully assigned openshift-ingress/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5 to ip-10-0-132-187.us-west-1.compute.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-api-a-26444f-67b5fb89f-jsmbq Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-26444f-67b5fb89f-jsmbq to ip-10-0-135-87.us-west-1.compute.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd to ip-10-0-132-187.us-west-1.compute.internal 6m12s Normal Scheduled pod/payload-pre-processing-e2e-ait-738d0c9a-76ddd9d799-pnhcj Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-738d0c9a-76ddd9d799-pnhcj to ip-10-0-135-87.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2bdcq Successfully assigned openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2bdcq to ip-10-0-143-132.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-28w9g Successfully assigned openshift-ingress/e2e-worker-w6-7540da-openshift-default-bb989bcf9-28w9g to ip-10-0-143-132.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-59hxl Successfully assigned openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-59hxl to ip-10-0-135-87.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-5tnzh Successfully assigned openshift-ingress/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-5tnzh to ip-10-0-135-87.us-west-1.compute.internal 76m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-79487cbf4c from 0 to 1 76m Normal SuccessfulCreate replicaset/router-default-79487cbf4c Created pod: router-default-79487cbf4c-klb6d 76m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 76m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 76m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 70m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 70m Warning FailedMount pod/router-default-79487cbf4c-klb6d MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 70m Warning FailedMount pod/router-default-79487cbf4c-klb6d MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 69m Warning FailedMount pod/router-default-79487cbf4c-klb6d MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 69m Warning FailedMount pod/router-default-79487cbf4c-klb6d MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 69m Normal AddedInterface pod/router-default-79487cbf4c-klb6d Add eth0 [10.132.0.14/23] from ovn-kubernetes 69m Normal Pulled pod/router-default-79487cbf4c-klb6d Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 69m Normal Created pod/router-default-79487cbf4c-klb6d Container created 69m Normal Started pod/router-default-79487cbf4c-klb6d Container started 65m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 65m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-gwwlg Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 65m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-gwwlg Add eth0 [10.134.0.21/23] from ovn-kubernetes 65m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-gwwlg 65m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 65m Normal Created pod/istiod-openshift-gateway-7876bfcb74-gwwlg Container created 65m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-gwwlg Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.777s (3.777s including waiting). Image size: 185476477 bytes. 65m Normal Started pod/istiod-openshift-gateway-7876bfcb74-gwwlg Container started 65m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-gwwlg Readiness probe failed: HTTP probe failed with statuscode: 503 65m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-gwwlg Readiness probe error: HTTP probe failed with statuscode: 503... 65m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7f5667f86c from 0 to 1 65m Normal SuccessfulCreate replicaset/openshift-ai-inference-openshift-ai-inference-65dccbd94c Created pod: openshift-ai-inference-openshift-ai-inference-65dccbd94c-kkznv 65m Normal ScalingReplicaSet deployment/openshift-ai-inference-openshift-ai-inference Scaled up replica set openshift-ai-inference-openshift-ai-inference-65dccbd94c from 0 to 1 65m Normal EnsuringLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensuring load balancer 65m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c Created pod: maas-default-gateway-openshift-default-7f5667f86c-nqmgk 65m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-nqmgk Add eth0 [10.133.0.21/23] from ovn-kubernetes 65m Normal Pulling pod/openshift-ai-inference-openshift-ai-inference-65dccbd94c-kkznv Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 65m Normal AddedInterface pod/openshift-ai-inference-openshift-ai-inference-65dccbd94c-kkznv Add eth0 [10.132.0.30/23] from ovn-kubernetes 65m Normal Pulling pod/maas-default-gateway-openshift-default-7f5667f86c-nqmgk Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 65m Normal EnsuredLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensured load balancer 64m Normal Started pod/openshift-ai-inference-openshift-ai-inference-65dccbd94c-kkznv Container started 64m Normal Created pod/openshift-ai-inference-openshift-ai-inference-65dccbd94c-kkznv Container created 64m Normal Pulled pod/openshift-ai-inference-openshift-ai-inference-65dccbd94c-kkznv Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.801s (3.801s including waiting). Image size: 183555333 bytes. 64m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-nqmgk Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 7.931s (7.931s including waiting). Image size: 183555333 bytes. 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Normal SuccessfulCreate replicaset/kube-auth-proxy-586b8ddf6d Created pod: kube-auth-proxy-586b8ddf6d-sgzck 63m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-7cfc97f8f7 from 0 to 1 63m Normal SuccessfulCreate replicaset/kube-auth-proxy-586b8ddf6d Created pod: kube-auth-proxy-586b8ddf6d-h9m6s 63m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-586b8ddf6d from 0 to 2 63m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-7cfc97f8f7 Created pod: data-science-gateway-data-science-gateway-class-7cfc97f8f7xkbnh 63m Normal Started pod/data-science-gateway-data-science-gateway-class-7cfc97f8f7xkbnh Container started 63m Normal AddedInterface pod/kube-auth-proxy-586b8ddf6d-sgzck Add eth0 [10.133.0.29/23] from ovn-kubernetes 63m Normal Created pod/data-science-gateway-data-science-gateway-class-7cfc97f8f7xkbnh Container created 63m Normal Pulling pod/kube-auth-proxy-586b8ddf6d-sgzck Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 63m Normal Pulling pod/kube-auth-proxy-586b8ddf6d-h9m6s Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 63m Normal Pulled pod/data-science-gateway-data-science-gateway-class-7cfc97f8f7xkbnh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/kube-auth-proxy-586b8ddf6d-h9m6s Add eth0 [10.134.0.24/23] from ovn-kubernetes 63m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-7cfc97f8f7xkbnh Add eth0 [10.132.0.39/23] from ovn-kubernetes 63m Normal Pulled pod/kube-auth-proxy-586b8ddf6d-h9m6s Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.911s (2.911s including waiting). Image size: 188904677 bytes. 63m Normal Started pod/kube-auth-proxy-586b8ddf6d-h9m6s Container started 63m Normal Pulled pod/kube-auth-proxy-586b8ddf6d-sgzck Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.786s (2.786s including waiting). Image size: 188904677 bytes. 63m Normal Created pod/kube-auth-proxy-586b8ddf6d-h9m6s Container created 63m Normal Created pod/kube-auth-proxy-586b8ddf6d-sgzck Container created 63m Normal Started pod/kube-auth-proxy-586b8ddf6d-sgzck Container started 63m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 63m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 63m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 62m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-cwdgf Container started 62m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-cwdgf Container created 62m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-cwdgf Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 62m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-cwdgf Add eth0 [10.134.0.27/23] from ovn-kubernetes 62m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-cwdgf 62m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-cwdgf Readiness probe error: HTTP probe failed with statuscode: 503... 62m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-cwdgf Readiness probe failed: HTTP probe failed with statuscode: 503 62m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 62m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-gwwlg 62m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-gwwlg Stopping container discovery 62m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 62m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c Created pod: maas-default-gateway-openshift-default-7f5667f86c-jr4dh 62m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7f5667f86c from 1 to 3 62m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c Created pod: maas-default-gateway-openshift-default-7f5667f86c-k2njq 62m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-jr4dh Container started 62m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-jr4dh Container created 62m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-jr4dh Add eth0 [10.132.0.46/23] from ovn-kubernetes 62m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-jr4dh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-k2njq Add eth0 [10.134.0.31/23] from ovn-kubernetes 62m Normal Pulling pod/maas-default-gateway-openshift-default-7f5667f86c-k2njq Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 62m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-nqmgk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-nqmgk Container started 62m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-nqmgk Container created 62m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-k2njq Container started 62m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-k2njq Container created 62m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-k2njq Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.664s (3.664s including waiting). Image size: 183555333 bytes. 62m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-jr4dh Startup probe failed: Get "http://10.132.0.46:15021/healthz/ready": dial tcp 10.132.0.46:15021: connect: connection refused 62m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-jr4dh Startup probe error: Get "http://10.132.0.46:15021/healthz/ready": dial tcp 10.132.0.46:15021: connect: connection refused... 62m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-nqmgk Startup probe error: Get "http://10.133.0.21:15021/healthz/ready": dial tcp 10.133.0.21:15021: connect: connection refused... 62m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-nqmgk Startup probe failed: Get "http://10.133.0.21:15021/healthz/ready": dial tcp 10.133.0.21:15021: connect: connection refused 62m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-k2njq Startup probe failed: Get "http://10.134.0.31:15021/healthz/ready": dial tcp 10.134.0.31:15021: connect: connection refused 62m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-k2njq Startup probe error: Get "http://10.134.0.31:15021/healthz/ready": dial tcp 10.134.0.31:15021: connect: connection refused... 62m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 61m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c Created pod: maas-default-gateway-openshift-default-7f5667f86c-kr95p 60m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c Created pod: maas-default-gateway-openshift-default-7f5667f86c-vs7zn 60m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c Created pod: maas-default-gateway-openshift-default-7f5667f86c-hn8gd 60m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-vs7zn Container created 60m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-hn8gd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-hn8gd Container started 60m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-kr95p Add eth0 [10.134.0.34/23] from ovn-kubernetes 60m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-hn8gd Container created 60m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-kr95p Container started 60m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-vs7zn Add eth0 [10.133.0.40/23] from ovn-kubernetes 60m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-vs7zn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-kr95p Container created 60m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-vs7zn Container started 60m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-hn8gd Add eth0 [10.132.0.50/23] from ovn-kubernetes 60m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-kr95p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-vs7zn Startup probe failed: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused 59m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-vs7zn Startup probe error: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused... 59m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-kr95p Startup probe error: Get "http://10.134.0.34:15021/healthz/ready": dial tcp 10.134.0.34:15021: connect: connection refused... 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-kr95p Startup probe failed: Get "http://10.134.0.34:15021/healthz/ready": dial tcp 10.134.0.34:15021: connect: connection refused 59m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-hn8gd Startup probe error: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused... 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-hn8gd Startup probe failed: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused 59m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-vs7zn Startup probe error: Get "http://10.133.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-vs7zn Startup probe failed: Get "http://10.133.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-kr95p Startup probe error: Get "http://10.134.0.34:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-kr95p Startup probe failed: Get "http://10.134.0.34:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-hn8gd Startup probe error: Get "http://10.132.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-hn8gd Startup probe failed: Get "http://10.132.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7f5667f86c from 6 to 9 59m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 59m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c Created pod: maas-default-gateway-openshift-default-7f5667f86c-fsjfb 59m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c Created pod: maas-default-gateway-openshift-default-7f5667f86c-xtmcj 59m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c Created pod: maas-default-gateway-openshift-default-7f5667f86c-vj28x 59m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-xtmcj Add eth0 [10.134.0.37/23] from ovn-kubernetes 59m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-fsjfb Add eth0 [10.132.0.52/23] from ovn-kubernetes 59m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-xtmcj Container started 59m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-xtmcj Container created 59m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-vj28x Add eth0 [10.133.0.41/23] from ovn-kubernetes 59m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-fsjfb Container started 59m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-xtmcj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-fsjfb Container created 59m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-fsjfb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-vj28x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-vj28x Container started 59m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-vj28x Container created 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-vj28x Startup probe failed: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused 59m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-vj28x Startup probe error: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused... 59m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-fsjfb Startup probe error: Get "http://10.132.0.52:15021/healthz/ready": dial tcp 10.132.0.52:15021: connect: connection refused... 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-fsjfb Startup probe failed: Get "http://10.132.0.52:15021/healthz/ready": dial tcp 10.132.0.52:15021: connect: connection refused 59m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-xtmcj Startup probe error: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused... 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-xtmcj Startup probe failed: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused 59m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-xtmcj Startup probe error: Get "http://10.134.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-fsjfb Startup probe failed: Get "http://10.132.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-fsjfb Startup probe error: Get "http://10.132.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-xtmcj Startup probe failed: Get "http://10.134.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-vj28x Startup probe error: Get "http://10.133.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-vj28x Startup probe failed: Get "http://10.133.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7f5667f86c from 9 to 10 59m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-9gvgr Container started 59m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-9gvgr Container created 59m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-9gvgr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-9gvgr Add eth0 [10.132.0.53/23] from ovn-kubernetes 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-9gvgr Startup probe failed: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused 59m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-9gvgr Startup probe error: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused... 59m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 59m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-7bb9489676 from 0 to 1 59m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 59m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-vxr7w 59m Warning FailedCreate replicaset/payload-processing-7bb9489676 Error creating: pods "payload-processing-7bb9489676-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 59m Normal AddedInterface pod/payload-processing-7bb9489676-wqbz4 Add eth0 [10.134.0.40/23] from ovn-kubernetes 59m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-vxr7w Add eth0 [10.134.0.39/23] from ovn-kubernetes 59m Normal Pulling pod/payload-pre-processing-7b98bb678c-vxr7w Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 59m Normal Pulling pod/payload-processing-7bb9489676-wqbz4 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 59m Normal SuccessfulCreate replicaset/payload-processing-7bb9489676 Created pod: payload-processing-7bb9489676-wqbz4 59m Normal Created pod/payload-processing-7bb9489676-wqbz4 Container created 59m Normal Started pod/payload-pre-processing-7b98bb678c-vxr7w Container started 59m Normal Pulled pod/payload-processing-7bb9489676-wqbz4 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.627s (2.627s including waiting). Image size: 125353963 bytes. 59m Normal Started pod/payload-processing-7bb9489676-wqbz4 Container started 59m Normal Pulled pod/payload-pre-processing-7b98bb678c-vxr7w Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.785s (2.785s including waiting). Image size: 125353963 bytes. 59m Normal Created pod/payload-pre-processing-7b98bb678c-vxr7w Container created 58m Normal AddedLabel gateway/e2e-worker-w6-7540da Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-7540da 58m Normal AddedLabel gateway/e2e-worker-w2-164c36 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-164c36 58m Normal AddedLabel gateway/e2e-worker-w3-1d31f8 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-1d31f8 58m Normal SuccessfulCreate replicaset/e2e-worker-w0-b64e2e-openshift-default-64c944c45c Created pod: e2e-worker-w0-b64e2e-openshift-default-64c944c45c-p4dg2 58m Normal ScalingReplicaSet deployment/e2e-worker-w4-32449d-openshift-default Scaled up replica set e2e-worker-w4-32449d-openshift-default-756d4bb7c7 from 0 to 1 58m Normal ScalingReplicaSet deployment/e2e-worker-w0-b64e2e-openshift-default Scaled up replica set e2e-worker-w0-b64e2e-openshift-default-64c944c45c from 0 to 1 58m Normal AddedLabel gateway/e2e-worker-w1-7afb1c Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-7afb1c 58m Normal AddedLabel gateway/e2e-worker-w0-b64e2e Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-b64e2e 58m Normal AddedLabel gateway/e2e-worker-w4-32449d Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-32449d 58m Normal AddedLabel gateway/e2e-worker-w5-cbd26c Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-cbd26c 58m Normal SuccessfulCreate replicaset/e2e-worker-w4-32449d-openshift-default-756d4bb7c7 Created pod: e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r 58m Normal ScalingReplicaSet deployment/e2e-worker-w6-7540da-openshift-default Scaled up replica set e2e-worker-w6-7540da-openshift-default-bb989bcf9 from 0 to 1 58m Normal AddedInterface pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-p4dg2 Add eth0 [10.132.0.54/23] from ovn-kubernetes 58m Normal SuccessfulCreate replicaset/e2e-worker-w6-7540da-openshift-default-bb989bcf9 Created pod: e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf 58m Normal SuccessfulCreate replicaset/e2e-worker-w2-164c36-openshift-default-54b556b4c9 Created pod: e2e-worker-w2-164c36-openshift-default-54b556b4c9-ldktd 58m Normal AddedInterface pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r Add eth0 [10.134.0.42/23] from ovn-kubernetes 58m Normal ScalingReplicaSet deployment/e2e-worker-w1-7afb1c-openshift-default Scaled up replica set e2e-worker-w1-7afb1c-openshift-default-6d7b749b87 from 0 to 1 58m Normal ScalingReplicaSet deployment/e2e-worker-w2-164c36-openshift-default Scaled up replica set e2e-worker-w2-164c36-openshift-default-54b556b4c9 from 0 to 1 58m Normal SuccessfulCreate replicaset/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87 Created pod: e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-g7kbs 58m Normal AddedInterface pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-g7kbs Add eth0 [10.132.0.55/23] from ovn-kubernetes 58m Normal AddedInterface pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-ldktd Add eth0 [10.133.0.43/23] from ovn-kubernetes 58m Normal SuccessfulCreate replicaset/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c Created pod: e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9 58m Normal ScalingReplicaSet deployment/e2e-worker-w5-cbd26c-openshift-default Scaled up replica set e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 from 0 to 1 58m Normal AddedInterface pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-8vndl Add eth0 [10.132.0.56/23] from ovn-kubernetes 58m Normal AddedInterface pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf Add eth0 [10.134.0.43/23] from ovn-kubernetes 58m Normal SuccessfulCreate replicaset/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 Created pod: e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-8vndl 58m Normal ScalingReplicaSet deployment/e2e-worker-w3-1d31f8-openshift-default Scaled up replica set e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c from 0 to 1 58m Normal AddedInterface pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9 Add eth0 [10.134.0.44/23] from ovn-kubernetes 58m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-b64e2e Scaled up replica set payload-processing-e2e-worker-w0-b64e2e-55674f5fb6 from 0 to 1 58m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-b64e2e Scaled up replica set payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c from 0 to 1 58m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6 Error creating: pods "payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-b64e2e: serviceaccount "payload-processing-e2e-worker-w0-b64e2e" not found 58m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6 Created pod: payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds 58m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c Created pod: payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx 58m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c Error creating: pods "payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-b64e2e: serviceaccount "payload-processing-e2e-worker-w0-b64e2e" not found 58m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx Add eth0 [10.134.0.45/23] from ovn-kubernetes 58m Normal Started pod/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds Container started 58m Normal Created pod/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds Container created 58m Normal Started pod/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx Container started 58m Normal Created pod/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx Container created 58m Normal AddedInterface pod/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds Add eth0 [10.134.0.46/23] from ovn-kubernetes 58m Normal Pulled pod/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5 Error creating: pods "payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-32449d: serviceaccount "payload-processing-e2e-worker-w4-32449d" not found 58m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-32449d Scaled up replica set payload-processing-e2e-worker-w4-32449d-58d4f897c8 from 0 to 1 58m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-32449d Scaled up replica set payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5 from 0 to 1 58m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-32449d-58d4f897c8 Error creating: pods "payload-processing-e2e-worker-w4-32449d-58d4f897c8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-32449d: serviceaccount "payload-processing-e2e-worker-w4-32449d" not found 58m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal Created pod/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf Container created 58m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-32449d-58d4f897c8 Created pod: payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h 58m Normal AddedInterface pod/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h Add eth0 [10.134.0.48/23] from ovn-kubernetes 58m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf Add eth0 [10.134.0.47/23] from ovn-kubernetes 58m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5 Created pod: payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf 58m Normal Started pod/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf Container started 58m Normal Started pod/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h Container started 58m Normal Created pod/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h Container created 58m Normal Pulled pod/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-164c36-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-164c36-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-7afb1c-5777db7584 Error creating: pods "payload-processing-e2e-worker-w1-7afb1c-5777db7584-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-7afb1c: serviceaccount "payload-processing-e2e-worker-w1-7afb1c" not found 58m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-7afb1c-5777db7584 Created pod: payload-processing-e2e-worker-w1-7afb1c-5777db7584-dvxk5 58m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-7afb1c Scaled up replica set payload-processing-e2e-worker-w1-7afb1c-5777db7584 from 0 to 1 58m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-7afb1c Scaled up replica set payload-pre-processing-e2e-worker-w1-7afb1c-869549494c from 0 to 1 58m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-7afb1c-869549494c Created pod: payload-pre-processing-e2e-worker-w1-7afb1c-869549494c-272x5 58m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-7afb1c-869549494c Error creating: pods "payload-pre-processing-e2e-worker-w1-7afb1c-869549494c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-7afb1c: serviceaccount "payload-processing-e2e-worker-w1-7afb1c" not found 58m Normal AddedInterface pod/payload-processing-e2e-worker-w1-7afb1c-5777db7584-dvxk5 Add eth0 [10.132.0.58/23] from ovn-kubernetes 58m Normal Pulling pod/payload-processing-e2e-worker-w1-7afb1c-5777db7584-dvxk5 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 58m Normal Pulling pod/payload-pre-processing-e2e-worker-w1-7afb1c-869549494c-272x5 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 58m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-7afb1c-869549494c-272x5 Add eth0 [10.132.0.57/23] from ovn-kubernetes 57m Normal Pulled pod/payload-processing-e2e-worker-w1-7afb1c-5777db7584-dvxk5 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.936s (3.936s including waiting). Image size: 125353963 bytes. 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-7540da-59844f9b7b Created pod: payload-processing-e2e-worker-w6-7540da-59844f9b7b-z68qt 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-7540da-598d566c67 Created pod: payload-pre-processing-e2e-worker-w6-7540da-598d566c67-ld8zm 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-7540da-598d566c67 Error creating: pods "payload-pre-processing-e2e-worker-w6-7540da-598d566c67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-7540da: serviceaccount "payload-processing-e2e-worker-w6-7540da" not found 57m Normal Created pod/payload-pre-processing-e2e-worker-w1-7afb1c-869549494c-272x5 Container created 57m Normal Started pod/payload-pre-processing-e2e-worker-w1-7afb1c-869549494c-272x5 Container started 57m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-7540da-59844f9b7b Error creating: pods "payload-processing-e2e-worker-w6-7540da-59844f9b7b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-7540da: serviceaccount "payload-processing-e2e-worker-w6-7540da" not found 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-7540da Scaled up replica set payload-pre-processing-e2e-worker-w6-7540da-598d566c67 from 0 to 1 57m Normal Created pod/payload-processing-e2e-worker-w1-7afb1c-5777db7584-dvxk5 Container created 57m Normal Started pod/payload-processing-e2e-worker-w1-7afb1c-5777db7584-dvxk5 Container started 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-7540da Scaled up replica set payload-processing-e2e-worker-w6-7540da-59844f9b7b from 0 to 1 57m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-7afb1c-869549494c-272x5 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 4.234s (4.234s including waiting). Image size: 125353963 bytes. 57m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-7540da-598d566c67-ld8zm Add eth0 [10.132.0.59/23] from ovn-kubernetes 57m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-7540da-598d566c67-ld8zm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/payload-processing-e2e-worker-w6-7540da-59844f9b7b-z68qt Add eth0 [10.132.0.60/23] from ovn-kubernetes 57m Normal Pulled pod/payload-processing-e2e-worker-w6-7540da-59844f9b7b-z68qt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Created pod/payload-pre-processing-e2e-worker-w6-7540da-598d566c67-ld8zm Container created 57m Normal Created pod/payload-processing-e2e-worker-w6-7540da-59844f9b7b-z68qt Container created 57m Normal Started pod/payload-processing-e2e-worker-w6-7540da-59844f9b7b-z68qt Container started 57m Normal Started pod/payload-pre-processing-e2e-worker-w6-7540da-598d566c67-ld8zm Container started 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-1d31f8 Scaled up replica set payload-pre-processing-e2e-worker-w3-1d31f8-f85f6f659 from 0 to 1 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-1d31f8-f85f6f659 Error creating: pods "payload-pre-processing-e2e-worker-w3-1d31f8-f85f6f659-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-1d31f8: serviceaccount "payload-processing-e2e-worker-w3-1d31f8" not found 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-1d31f8 Scaled up replica set payload-processing-e2e-worker-w3-1d31f8-64f6fcf754 from 0 to 1 57m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-1d31f8-64f6fcf754 Error creating: pods "payload-processing-e2e-worker-w3-1d31f8-64f6fcf754-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-1d31f8: serviceaccount "payload-processing-e2e-worker-w3-1d31f8" not found 57m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-1d31f8-f85f6f659-t2mm6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-1d31f8-f85f6f659 Created pod: payload-pre-processing-e2e-worker-w3-1d31f8-f85f6f659-t2mm6 57m Normal Started pod/payload-pre-processing-e2e-worker-w3-1d31f8-f85f6f659-t2mm6 Container started 57m Normal Created pod/payload-pre-processing-e2e-worker-w3-1d31f8-f85f6f659-t2mm6 Container created 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-1d31f8-64f6fcf754 Created pod: payload-processing-e2e-worker-w3-1d31f8-64f6fcf754-48jnl 57m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-1d31f8-f85f6f659-t2mm6 Add eth0 [10.132.0.61/23] from ovn-kubernetes 57m Normal AddedInterface pod/payload-processing-e2e-worker-w3-1d31f8-64f6fcf754-48jnl Add eth0 [10.132.0.62/23] from ovn-kubernetes 57m Normal Created pod/payload-processing-e2e-worker-w3-1d31f8-64f6fcf754-48jnl Container created 57m Normal Started pod/payload-processing-e2e-worker-w3-1d31f8-64f6fcf754-48jnl Container started 57m Normal Pulled pod/payload-processing-e2e-worker-w3-1d31f8-64f6fcf754-48jnl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c Error creating: pods "payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-cbd26c: serviceaccount "payload-processing-e2e-worker-w5-cbd26c" not found 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c Created pod: payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg 57m Normal Pulled pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-cbd26c-845d9688fb Error creating: pods "payload-processing-e2e-worker-w5-cbd26c-845d9688fb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-cbd26c: serviceaccount "payload-processing-e2e-worker-w5-cbd26c" not found 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-cbd26c Scaled up replica set payload-processing-e2e-worker-w5-cbd26c-845d9688fb from 0 to 1 57m Normal Started pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r Container started 57m Normal Created pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r Container created 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-cbd26c Scaled up replica set payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c from 0 to 1 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-cbd26c-845d9688fb Created pod: payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8 57m Normal Pulled pod/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8 Add eth0 [10.134.0.51/23] from ovn-kubernetes 57m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg Add eth0 [10.134.0.50/23] from ovn-kubernetes 57m Normal Started pod/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg Container started 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-164c36 Scaled up replica set payload-processing-e2e-worker-w2-164c36-74dc8ddf7b from 0 to 1 57m Normal Created pod/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg Container created 57m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Created pod/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8 Container created 57m Normal Started pod/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8 Container started 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-164c36 Scaled up replica set payload-pre-processing-e2e-worker-w2-164c36-84d6767d47 from 0 to 1 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-1d31f8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-164c36-74dc8ddf7b Created pod: payload-processing-e2e-worker-w2-164c36-74dc8ddf7b-wqgvc 57m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-164c36-74dc8ddf7b Error creating: pods "payload-processing-e2e-worker-w2-164c36-74dc8ddf7b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-164c36: serviceaccount "payload-processing-e2e-worker-w2-164c36" not found 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-164c36-84d6767d47 Error creating: pods "payload-pre-processing-e2e-worker-w2-164c36-84d6767d47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-164c36: serviceaccount "payload-processing-e2e-worker-w2-164c36" not found 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-164c36-84d6767d47 Created pod: payload-pre-processing-e2e-worker-w2-164c36-84d6767d47-wn2ct 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-1d31f8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Normal Created pod/payload-pre-processing-e2e-worker-w2-164c36-84d6767d47-wn2ct Container created 57m Normal Started pod/payload-pre-processing-e2e-worker-w2-164c36-84d6767d47-wn2ct Container started 57m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-164c36-84d6767d47-wn2ct Add eth0 [10.132.0.63/23] from ovn-kubernetes 57m Normal Pulled pod/payload-processing-e2e-worker-w2-164c36-74dc8ddf7b-wqgvc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/payload-processing-e2e-worker-w2-164c36-74dc8ddf7b-wqgvc Add eth0 [10.132.0.64/23] from ovn-kubernetes 57m Normal Started pod/payload-processing-e2e-worker-w2-164c36-74dc8ddf7b-wqgvc Container started 57m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-164c36-84d6767d47-wn2ct Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Created pod/payload-processing-e2e-worker-w2-164c36-74dc8ddf7b-wqgvc Container created 57m Warning ProbeError pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r Startup probe error: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r Startup probe failed: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused 57m Normal Pulled pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-g7kbs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-g7kbs Container created 57m Normal Started pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-g7kbs Container started 57m Normal Pulled pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf Container created 57m Normal Started pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf Container started 57m Warning Unhealthy pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-g7kbs Startup probe failed: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-g7kbs Startup probe error: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf Startup probe failed: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf Startup probe error: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused... 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-32449d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-32449d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-7afb1c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-7afb1c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Normal SuccessfulCreate replicaset/e2e-worker-w2-164c36-openshift-default-54b556b4c9 Created pod: e2e-worker-w2-164c36-openshift-default-54b556b4c9-2nqsk 57m Normal SuccessfulCreate replicaset/e2e-worker-w2-164c36-openshift-default-54b556b4c9 Created pod: e2e-worker-w2-164c36-openshift-default-54b556b4c9-q95hh 57m Normal SuccessfulCreate replicaset/e2e-worker-w2-164c36-openshift-default-54b556b4c9 Created pod: e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-164c36-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 57m Normal ScalingReplicaSet deployment/e2e-worker-w2-164c36-openshift-default Scaled up replica set e2e-worker-w2-164c36-openshift-default-54b556b4c9 from 1 to 4 57m Normal Started pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-q95hh Container started 57m Normal Started pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-ldktd Container started 57m Normal Created pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-2nqsk Container created 57m Normal AddedInterface pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-2nqsk Add eth0 [10.132.0.65/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-q95hh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-q95hh Add eth0 [10.133.0.48/23] from ovn-kubernetes 57m Normal Started pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx Container started 57m Normal Created pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx Container created 57m Normal Started pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-2nqsk Container started 57m Normal Pulled pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-2nqsk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Pulled pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-ldktd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-ldktd Container created 57m Normal AddedInterface pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx Add eth0 [10.134.0.53/23] from ovn-kubernetes 57m Normal Created pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-q95hh Container created 57m Normal Pulled pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Warning ProbeError pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-q95hh Startup probe error: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-q95hh Startup probe failed: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx Startup probe error: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx Startup probe failed: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-2nqsk Startup probe error: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-2nqsk Startup probe failed: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-ldktd Startup probe failed: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-ldktd Startup probe error: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused... 57m Normal Started pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-p4dg2 Container started 57m Normal Pulled pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-p4dg2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-p4dg2 Container created 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-32449d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-b64e2e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-b64e2e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-32449d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning ProbeError pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-p4dg2 Startup probe error: Get "http://10.132.0.54:15021/healthz/ready": dial tcp 10.132.0.54:15021: connect: connection refused... 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-7540da-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-7540da-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Warning Unhealthy pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-p4dg2 Startup probe failed: Get "http://10.132.0.54:15021/healthz/ready": dial tcp 10.132.0.54:15021: connect: connection refused 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-164c36-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-164c36-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning ProbeError pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9 Readiness probe error: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9 Readiness probe failed: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused 57m Normal Pulled pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9 Container created 57m Normal Started pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9 Container started 57m Warning ProbeError pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9 Startup probe error: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9 Startup probe failed: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused 57m Normal Created pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-8vndl Container created 57m Normal Started pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-8vndl Container started 57m Normal Pulled pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-8vndl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-7afb1c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-7afb1c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Normal ScalingReplicaSet deployment/e2e-worker-w2-164c36-openshift-default Scaled up replica set e2e-worker-w2-164c36-openshift-default-54b556b4c9 from 4 to 6 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-164c36-openshift-default New size: 6; reason: 57m Normal SuccessfulCreate replicaset/e2e-worker-w2-164c36-openshift-default-54b556b4c9 Created pod: e2e-worker-w2-164c36-openshift-default-54b556b4c9-rsqdc 57m Normal SuccessfulCreate replicaset/e2e-worker-w2-164c36-openshift-default-54b556b4c9 Created pod: e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2 57m Normal AddedInterface pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-rsqdc Add eth0 [10.132.0.67/23] from ovn-kubernetes 57m Normal Created pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-rsqdc Container created 57m Normal Pulled pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2 Container created 57m Normal Started pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2 Container started 57m Normal Pulled pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-rsqdc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Warning Unhealthy pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-8vndl Startup probe failed: Get "http://10.132.0.56:15021/healthz/ready": dial tcp 10.132.0.56:15021: connect: connection refused 57m Normal AddedInterface pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2 Add eth0 [10.134.0.54/23] from ovn-kubernetes 57m Normal Started pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-rsqdc Container started 57m Warning ProbeError pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-8vndl Startup probe error: Get "http://10.132.0.56:15021/healthz/ready": dial tcp 10.132.0.56:15021: connect: connection refused... 56m Warning ProbeError pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-rsqdc Startup probe error: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused... 56m Warning ProbeError pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2 Startup probe error: Get "http://10.134.0.54:15021/healthz/ready": dial tcp 10.134.0.54:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2 Startup probe failed: Get "http://10.134.0.54:15021/healthz/ready": dial tcp 10.134.0.54:15021: connect: connection refused 56m Warning Unhealthy pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-rsqdc Startup probe failed: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-b64e2e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-b64e2e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-1d31f8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-1d31f8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-cbd26c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-cbd26c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-7afb1c-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 55m Normal SuccessfulCreate replicaset/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87 Created pod: e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-tww6f 55m Normal ScalingReplicaSet deployment/e2e-worker-w1-7afb1c-openshift-default Scaled up replica set e2e-worker-w1-7afb1c-openshift-default-6d7b749b87 from 1 to 2 55m Normal Started pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-tww6f Container started 55m Normal Created pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-tww6f Container created 55m Normal Pulled pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-tww6f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-tww6f Add eth0 [10.133.0.51/23] from ovn-kubernetes 55m Warning Unhealthy pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-tww6f Startup probe failed: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-tww6f Startup probe error: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused... 55m Normal ScalingReplicaSet deployment/e2e-worker-w6-7540da-openshift-default Scaled up replica set e2e-worker-w6-7540da-openshift-default-bb989bcf9 from 1 to 2 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-7540da-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 55m Normal SuccessfulCreate replicaset/e2e-worker-w6-7540da-openshift-default-bb989bcf9 Created pod: e2e-worker-w6-7540da-openshift-default-bb989bcf9-m7rjj 55m Normal AddedInterface pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-m7rjj Add eth0 [10.132.0.73/23] from ovn-kubernetes 55m Normal Pulled pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-m7rjj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Started pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-m7rjj Container started 55m Normal Created pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-m7rjj Container created 54m Warning ProbeError pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-m7rjj Startup probe error: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-m7rjj Startup probe failed: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-b64e2e-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulCreate replicaset/e2e-worker-w0-b64e2e-openshift-default-64c944c45c Created pod: e2e-worker-w0-b64e2e-openshift-default-64c944c45c-h59rg 54m Normal ScalingReplicaSet deployment/e2e-worker-w0-b64e2e-openshift-default Scaled up replica set e2e-worker-w0-b64e2e-openshift-default-64c944c45c from 1 to 2 54m Normal AddedInterface pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-h59rg Add eth0 [10.133.0.53/23] from ovn-kubernetes 54m Normal Started pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-h59rg Container started 54m Normal Created pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-h59rg Container created 54m Normal Pulled pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-h59rg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Warning Unhealthy pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-h59rg Startup probe failed: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-h59rg Startup probe error: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused... 54m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-jr4dh Stopping container istio-proxy 54m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-hn8gd Stopping container istio-proxy 54m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-9gvgr Stopping container istio-proxy 54m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7f5667f86c Deleted pod: maas-default-gateway-openshift-default-7f5667f86c-hn8gd 54m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7f5667f86c Deleted pod: maas-default-gateway-openshift-default-7f5667f86c-9gvgr 54m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7f5667f86c Deleted pod: maas-default-gateway-openshift-default-7f5667f86c-jr4dh 54m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7f5667f86c Deleted pod: maas-default-gateway-openshift-default-7f5667f86c-fsjfb 54m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-fsjfb Stopping container istio-proxy 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-jr4dh Readiness probe failed: HTTP probe failed with statuscode: 503 54m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-jr4dh Readiness probe error: HTTP probe failed with statuscode: 503... 54m Normal ScalingReplicaSet deployment/e2e-worker-w4-32449d-openshift-default Scaled up replica set e2e-worker-w4-32449d-openshift-default-756d4bb7c7 from 1 to 2 54m Normal SuccessfulCreate replicaset/e2e-worker-w4-32449d-openshift-default-756d4bb7c7 Created pod: e2e-worker-w4-32449d-openshift-default-756d4bb7c7-cr5jb 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-32449d-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal AddedInterface pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-cr5jb Add eth0 [10.132.0.78/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-cr5jb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-cr5jb Container created 54m Normal Started pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-cr5jb Container started 54m Normal SuccessfulCreate replicaset/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c Created pod: e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-9qxhp 54m Normal ScalingReplicaSet deployment/e2e-worker-w3-1d31f8-openshift-default Scaled up replica set e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c from 1 to 2 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-1d31f8-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal Started pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-9qxhp Container started 54m Normal AddedInterface pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-9qxhp Add eth0 [10.132.0.79/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-9qxhp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-9qxhp Container created 53m Warning ProbeError pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-cr5jb Startup probe error: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-cr5jb Startup probe failed: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-9qxhp Startup probe error: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-9qxhp Startup probe failed: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused 53m Normal Pulled pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-sm442 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal SuccessfulCreate replicaset/e2e-worker-w4-32449d-openshift-default-756d4bb7c7 Created pod: e2e-worker-w4-32449d-openshift-default-756d4bb7c7-sm442 53m Normal ScalingReplicaSet deployment/e2e-worker-w4-32449d-openshift-default Scaled up replica set e2e-worker-w4-32449d-openshift-default-756d4bb7c7 from 2 to 3 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-32449d-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 53m Normal Started pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-sm442 Container started 53m Normal Created pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-sm442 Container created 53m Normal AddedInterface pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-sm442 Add eth0 [10.133.0.54/23] from ovn-kubernetes 52m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-wtfpt Add eth0 [10.133.0.55/23] from ovn-kubernetes 52m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-wtfpt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-4bjss Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-2jqg5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-4bjss Add eth0 [10.132.0.91/23] from ovn-kubernetes 52m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-hn9sv Add eth0 [10.132.0.92/23] from ovn-kubernetes 52m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-2jqg5 Add eth0 [10.132.0.93/23] from ovn-kubernetes 52m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-2jqg5 Container started 52m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-hn9sv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-hn9sv Container started 52m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-4bjss Container created 52m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-wtfpt Container started 52m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-wtfpt Container created 52m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-4bjss Container started 52m Warning Unhealthy pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-sm442 Startup probe failed: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused 52m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-hn9sv Container created 52m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-2jqg5 Container created 52m Warning ProbeError pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-sm442 Startup probe error: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused... 52m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-2jqg5 Startup probe failed: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused 52m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-hn9sv Startup probe failed: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused 52m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-hn9sv Startup probe error: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused... 52m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-wtfpt Startup probe error: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused... 52m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-4bjss Startup probe failed: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused 52m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-4bjss Startup probe error: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused... 52m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-2jqg5 Startup probe error: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused... 52m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-wtfpt Startup probe failed: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused 52m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-2jqg5 Startup probe error: Get "http://10.132.0.93:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-4bjss Startup probe error: Get "http://10.132.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-4bjss Startup probe failed: Get "http://10.132.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-hn9sv Startup probe error: Get "http://10.132.0.92:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-hn9sv Startup probe failed: Get "http://10.132.0.92:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-2jqg5 Startup probe failed: Get "http://10.132.0.93:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-7540da-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 52m Normal ScalingReplicaSet deployment/e2e-worker-w6-7540da-openshift-default Scaled up replica set e2e-worker-w6-7540da-openshift-default-bb989bcf9 from 2 to 4 52m Normal AddedInterface pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-nc2kj Add eth0 [10.133.0.56/23] from ovn-kubernetes 52m Normal SuccessfulCreate replicaset/e2e-worker-w2-164c36-openshift-default-54b556b4c9 Created pod: e2e-worker-w2-164c36-openshift-default-54b556b4c9-wlh8v 52m Normal ScalingReplicaSet deployment/e2e-worker-w2-164c36-openshift-default Scaled up replica set e2e-worker-w2-164c36-openshift-default-54b556b4c9 from 6 to 8 52m Normal SuccessfulCreate replicaset/e2e-worker-w6-7540da-openshift-default-bb989bcf9 Created pod: e2e-worker-w6-7540da-openshift-default-bb989bcf9-nc2kj 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-164c36-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 52m Normal SuccessfulCreate replicaset/e2e-worker-w6-7540da-openshift-default-bb989bcf9 Created pod: e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp 52m Normal AddedInterface pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp Add eth0 [10.134.0.61/23] from ovn-kubernetes 52m Normal SuccessfulCreate replicaset/e2e-worker-w2-164c36-openshift-default-54b556b4c9 Created pod: e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl 52m Normal Pulled pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-nc2kj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Pulled pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Pulled pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp Container created 52m Normal Started pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp Container started 52m Normal Created pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl Container created 52m Normal Started pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl Container started 52m Normal Started pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-nc2kj Container started 52m Normal Created pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-wlh8v Container created 52m Normal Created pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-nc2kj Container created 52m Normal AddedInterface pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-wlh8v Add eth0 [10.133.0.57/23] from ovn-kubernetes 52m Normal AddedInterface pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl Add eth0 [10.134.0.62/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-wlh8v Container started 52m Normal Pulled pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-wlh8v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Warning Unhealthy pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp Startup probe failed: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp Startup probe error: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-wlh8v Startup probe failed: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-nc2kj Startup probe failed: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-wlh8v Startup probe error: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-nc2kj Startup probe error: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl Startup probe error: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl Startup probe failed: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp Startup probe error: Get "http://10.134.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp Startup probe failed: Get "http://10.134.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Normal SuccessfulCreate replicaset/e2e-worker-w0-b64e2e-openshift-default-64c944c45c Created pod: e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-b64e2e-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 52m Normal SuccessfulCreate replicaset/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87 Created pod: e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs 52m Normal ScalingReplicaSet deployment/e2e-worker-w0-b64e2e-openshift-default Scaled up replica set e2e-worker-w0-b64e2e-openshift-default-64c944c45c from 2 to 3 52m Normal Started pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs Container started 52m Normal Created pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs Container created 52m Normal Pulled pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Pulled pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs Add eth0 [10.134.0.64/23] from ovn-kubernetes 52m Normal Created pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg Container created 52m Normal Started pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg Container started 52m Normal AddedInterface pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg Add eth0 [10.134.0.63/23] from ovn-kubernetes 52m Warning Unhealthy pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg Startup probe failed: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg Startup probe error: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs Startup probe failed: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs Startup probe error: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused... 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-164c36-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 51m Normal AddedInterface pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt Add eth0 [10.134.0.65/23] from ovn-kubernetes 51m Normal ScalingReplicaSet deployment/e2e-worker-w2-164c36-openshift-default Scaled up replica set e2e-worker-w2-164c36-openshift-default-54b556b4c9 from 8 to 10 51m Normal SuccessfulCreate replicaset/e2e-worker-w2-164c36-openshift-default-54b556b4c9 Created pod: e2e-worker-w2-164c36-openshift-default-54b556b4c9-klsq2 51m Normal SuccessfulCreate replicaset/e2e-worker-w2-164c36-openshift-default-54b556b4c9 (combined from similar events): Created pod: e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt 51m Normal Created pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-klsq2 Container created 51m Normal Started pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-klsq2 Container started 51m Normal Started pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt Container started 51m Normal Pulled pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-klsq2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Pulled pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt Container created 51m Normal AddedInterface pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-klsq2 Add eth0 [10.132.0.105/23] from ovn-kubernetes 51m Warning Unhealthy pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-klsq2 Startup probe failed: Get "http://10.132.0.105:15021/healthz/ready": dial tcp 10.132.0.105:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-klsq2 Startup probe error: Get "http://10.132.0.105:15021/healthz/ready": dial tcp 10.132.0.105:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt Startup probe failed: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt Startup probe error: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused... 51m Normal ScalingReplicaSet deployment/e2e-worker-w5-cbd26c-openshift-default Scaled up replica set e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 from 1 to 2 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-cbd26c-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 Created pod: e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-qk9xk 51m Normal AddedInterface pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-qk9xk Add eth0 [10.133.0.60/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-qk9xk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-qk9xk Container created 51m Normal Started pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-qk9xk Container started 51m Warning Unhealthy pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-qk9xk Startup probe failed: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-qk9xk Startup probe error: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused... 51m Normal SuccessfulCreate replicaset/e2e-worker-w4-32449d-openshift-default-756d4bb7c7 Created pod: e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq 51m Normal SuccessfulCreate replicaset/e2e-worker-w4-32449d-openshift-default-756d4bb7c7 Created pod: e2e-worker-w4-32449d-openshift-default-756d4bb7c7-wb2g5 51m Normal ScalingReplicaSet deployment/e2e-worker-w4-32449d-openshift-default Scaled up replica set e2e-worker-w4-32449d-openshift-default-756d4bb7c7 from 3 to 6 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-32449d-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 51m Normal Started pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-wb2g5 Container started 51m Normal Pulled pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq Add eth0 [10.134.0.66/23] from ovn-kubernetes 51m Normal Created pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-wb2g5 Container created 51m Normal Pulled pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-wb2g5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Pulled pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-44frr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-44frr Add eth0 [10.132.0.107/23] from ovn-kubernetes 51m Normal SuccessfulCreate replicaset/e2e-worker-w4-32449d-openshift-default-756d4bb7c7 Created pod: e2e-worker-w4-32449d-openshift-default-756d4bb7c7-44frr 51m Normal AddedInterface pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-wb2g5 Add eth0 [10.133.0.61/23] from ovn-kubernetes 51m Normal Created pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq Container created 51m Normal ScalingReplicaSet deployment/e2e-worker-w6-7540da-openshift-default Scaled up replica set e2e-worker-w6-7540da-openshift-default-bb989bcf9 from 4 to 5 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-7540da-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 51m Normal Started pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-44frr Container started 51m Normal AddedInterface pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-8jkmq Add eth0 [10.132.0.108/23] from ovn-kubernetes 51m Normal SuccessfulCreate replicaset/e2e-worker-w6-7540da-openshift-default-bb989bcf9 Created pod: e2e-worker-w6-7540da-openshift-default-bb989bcf9-8jkmq 51m Normal Created pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-44frr Container created 51m Normal Started pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq Container started 51m Normal Created pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-8jkmq Container created 51m Normal Pulled pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-8jkmq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-8jkmq Container started 50m Warning Unhealthy pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-wb2g5 Startup probe failed: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq Startup probe error: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-44frr Startup probe error: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-wb2g5 Startup probe error: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-44frr Startup probe failed: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq Startup probe failed: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-8jkmq Startup probe failed: Get "http://10.132.0.108:15021/healthz/ready": dial tcp 10.132.0.108:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-8jkmq Startup probe error: Get "http://10.132.0.108:15021/healthz/ready": dial tcp 10.132.0.108:15021: connect: connection refused... 50m Normal AddedInterface pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-cb25n Add eth0 [10.133.0.62/23] from ovn-kubernetes 50m Normal ScalingReplicaSet deployment/e2e-worker-w3-1d31f8-openshift-default Scaled up replica set e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c from 2 to 3 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-1d31f8-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 50m Normal SuccessfulCreate replicaset/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c Created pod: e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-cb25n 50m Normal Pulled pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-cb25n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-cb25n Container created 50m Normal Started pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-cb25n Container started 50m Warning ProbeError pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-cb25n Startup probe error: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-cb25n Startup probe failed: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-32449d-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 49m Normal SuccessfulCreate replicaset/e2e-worker-w4-32449d-openshift-default-756d4bb7c7 Created pod: e2e-worker-w4-32449d-openshift-default-756d4bb7c7-2jsvl 49m Normal ScalingReplicaSet deployment/e2e-worker-w4-32449d-openshift-default Scaled up replica set e2e-worker-w4-32449d-openshift-default-756d4bb7c7 from 6 to 10 49m Normal ScalingReplicaSet deployment/e2e-worker-w0-b64e2e-openshift-default Scaled up replica set e2e-worker-w0-b64e2e-openshift-default-64c944c45c from 3 to 6 49m Normal AddedInterface pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-shfnk Add eth0 [10.133.0.65/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-shfnk Container started 49m Normal Created pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-xvdvr Container created 49m Normal Pulled pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-xvdvr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql Container created 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-b64e2e-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 49m Normal AddedInterface pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-xvdvr Add eth0 [10.132.0.115/23] from ovn-kubernetes 49m Normal AddedInterface pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql Add eth0 [10.134.0.71/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql Container started 49m Normal Pulled pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-shfnk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-shfnk Container created 49m Normal Pulled pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal SuccessfulCreate replicaset/e2e-worker-w4-32449d-openshift-default-756d4bb7c7 Created pod: e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql 49m Normal SuccessfulCreate replicaset/e2e-worker-w4-32449d-openshift-default-756d4bb7c7 Created pod: e2e-worker-w4-32449d-openshift-default-756d4bb7c7-shfnk 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-b64e2e-openshift-default-64c944c45c Created pod: e2e-worker-w0-b64e2e-openshift-default-64c944c45c-2hmfk 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-b64e2e-openshift-default-64c944c45c Created pod: e2e-worker-w0-b64e2e-openshift-default-64c944c45c-8mkq4 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-b64e2e-openshift-default-64c944c45c Created pod: e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf 49m Normal SuccessfulCreate replicaset/e2e-worker-w4-32449d-openshift-default-756d4bb7c7 (combined from similar events): Created pod: e2e-worker-w4-32449d-openshift-default-756d4bb7c7-xvdvr 49m Normal AddedInterface pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-2jsvl Add eth0 [10.132.0.114/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-xvdvr Container started 49m Normal Created pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-2jsvl Container created 49m Normal Started pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-2hmfk Container started 49m Normal AddedInterface pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-8mkq4 Add eth0 [10.132.0.116/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-8mkq4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-8mkq4 Container created 49m Normal Started pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-8mkq4 Container started 49m Normal Started pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-2jsvl Container started 49m Normal Pulled pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-2jsvl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-2hmfk Add eth0 [10.133.0.66/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf Container started 49m Normal Created pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf Container created 49m Normal Pulled pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-2hmfk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf Add eth0 [10.134.0.72/23] from ovn-kubernetes 49m Normal Created pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-2hmfk Container created 48m Warning ProbeError pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-2hmfk Startup probe error: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-2hmfk Startup probe failed: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf Startup probe failed: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9 Readiness probe failed: Get "http://10.134.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-8jkmq Readiness probe failed: Get "http://10.132.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-shfnk Startup probe failed: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9 Readiness probe error: Get "http://10.134.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-shfnk Startup probe error: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-8jkmq Readiness probe error: Get "http://10.132.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-8mkq4 Startup probe failed: Get "http://10.132.0.116:15021/healthz/ready": dial tcp 10.132.0.116:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-8mkq4 Startup probe error: Get "http://10.132.0.116:15021/healthz/ready": dial tcp 10.132.0.116:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-xvdvr Startup probe failed: Get "http://10.132.0.115:15021/healthz/ready": dial tcp 10.132.0.115:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-2jsvl Startup probe error: Get "http://10.132.0.114:15021/healthz/ready": dial tcp 10.132.0.114:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-2jsvl Startup probe failed: Get "http://10.132.0.114:15021/healthz/ready": dial tcp 10.132.0.114:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-xvdvr Startup probe error: Get "http://10.132.0.115:15021/healthz/ready": dial tcp 10.132.0.115:15021: connect: connection refused... 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-7540da-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 48m Normal SuccessfulCreate replicaset/e2e-worker-w6-7540da-openshift-default-bb989bcf9 Created pod: e2e-worker-w6-7540da-openshift-default-bb989bcf9-54fm8 48m Normal AddedInterface pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-54fm8 Add eth0 [10.133.0.67/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-54fm8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x Add eth0 [10.134.0.73/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal ScalingReplicaSet deployment/e2e-worker-w6-7540da-openshift-default Scaled up replica set e2e-worker-w6-7540da-openshift-default-bb989bcf9 from 5 to 7 48m Normal SuccessfulCreate replicaset/e2e-worker-w6-7540da-openshift-default-bb989bcf9 Created pod: e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x 48m Normal SuccessfulCreate replicaset/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c Created pod: e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-nprdv 48m Normal Created pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-54fm8 Container created 48m Normal Started pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x Container started 48m Normal Pulled pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-nprdv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x Container created 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-1d31f8-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 48m Normal ScalingReplicaSet deployment/e2e-worker-w3-1d31f8-openshift-default Scaled up replica set e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c from 3 to 4 48m Normal AddedInterface pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-nprdv Add eth0 [10.132.0.120/23] from ovn-kubernetes 48m Normal Started pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-54fm8 Container started 48m Normal Started pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-nprdv Container started 48m Normal Created pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-nprdv Container created 48m Warning ProbeError pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-54fm8 Startup probe error: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x Startup probe error: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x Startup probe failed: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-54fm8 Startup probe failed: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-nprdv Startup probe error: Get "http://10.132.0.120:15021/healthz/ready": dial tcp 10.132.0.120:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-nprdv Startup probe failed: Get "http://10.132.0.120:15021/healthz/ready": dial tcp 10.132.0.120:15021: connect: connection refused 48m Normal SuccessfulCreate replicaset/e2e-worker-w0-b64e2e-openshift-default-64c944c45c Created pod: e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4 48m Normal ScalingReplicaSet deployment/e2e-worker-w0-b64e2e-openshift-default Scaled up replica set e2e-worker-w0-b64e2e-openshift-default-64c944c45c from 6 to 7 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-b64e2e-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 48m Normal Pulled pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4 Add eth0 [10.134.0.74/23] from ovn-kubernetes 48m Normal SuccessfulCreate replicaset/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 Created pod: e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9p25h 48m Normal SuccessfulCreate replicaset/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 Created pod: e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5 48m Normal Created pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4 Container created 48m Normal ScalingReplicaSet deployment/e2e-worker-w5-cbd26c-openshift-default Scaled up replica set e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 from 2 to 4 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-cbd26c-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 48m Normal Started pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4 Container started 48m Normal ScalingReplicaSet deployment/e2e-worker-w3-1d31f8-openshift-default Scaled up replica set e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c from 4 to 6 48m Normal AddedInterface pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-w6k2c Add eth0 [10.133.0.69/23] from ovn-kubernetes 48m Normal AddedInterface pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9p25h Add eth0 [10.133.0.68/23] from ovn-kubernetes 48m Normal Started pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5 Container started 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-1d31f8-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 48m Normal Started pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9p25h Container started 48m Normal AddedInterface pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5 Add eth0 [10.134.0.75/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Pulled pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9p25h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9p25h Container created 48m Normal AddedInterface pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz Add eth0 [10.134.0.76/23] from ovn-kubernetes 48m Normal SuccessfulCreate replicaset/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c Created pod: e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz 48m Normal Created pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5 Container created 48m Normal SuccessfulCreate replicaset/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c Created pod: e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-w6k2c 48m Normal Pulled pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-w6k2c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Pulled pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-w6k2c Container started 48m Normal Created pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz Container created 48m Normal Created pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-w6k2c Container created 48m Normal Started pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz Container started 48m Warning Unhealthy pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4 Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4 Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz Startup probe error: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5 Startup probe error: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-w6k2c Startup probe failed: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-w6k2c Startup probe error: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9p25h Startup probe error: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9p25h Startup probe failed: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz Startup probe failed: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5 Startup probe failed: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-p4dg2 Readiness probe error: Get "http://10.132.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-p4dg2 Readiness probe failed: Get "http://10.132.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Normal SuccessfulCreate replicaset/e2e-worker-w0-b64e2e-openshift-default-64c944c45c Created pod: e2e-worker-w0-b64e2e-openshift-default-64c944c45c-nlgd9 48m Normal SuccessfulCreate replicaset/e2e-worker-w0-b64e2e-openshift-default-64c944c45c (combined from similar events): Created pod: e2e-worker-w0-b64e2e-openshift-default-64c944c45c-gjgqz 48m Normal SuccessfulCreate replicaset/e2e-worker-w0-b64e2e-openshift-default-64c944c45c Created pod: e2e-worker-w0-b64e2e-openshift-default-64c944c45c-7bggr 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-b64e2e-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 48m Normal ScalingReplicaSet deployment/e2e-worker-w0-b64e2e-openshift-default Scaled up replica set e2e-worker-w0-b64e2e-openshift-default-64c944c45c from 7 to 10 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-7afb1c-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 48m Normal ScalingReplicaSet deployment/e2e-worker-w1-7afb1c-openshift-default Scaled up replica set e2e-worker-w1-7afb1c-openshift-default-6d7b749b87 from 3 to 6 48m Normal Pulled pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-nlgd9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-nlgd9 Container created 48m Normal Started pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-7bggr Container started 48m Normal Started pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-nlgd9 Container started 48m Normal Created pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-7bggr Container created 48m Normal Pulled pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-7bggr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal SuccessfulCreate replicaset/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87 Created pod: e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-7qx7j 48m Normal SuccessfulCreate replicaset/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87 Created pod: e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr 48m Normal SuccessfulCreate replicaset/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87 Created pod: e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-vvn46 48m Normal AddedInterface pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-nlgd9 Add eth0 [10.133.0.70/23] from ovn-kubernetes 48m Normal Started pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-vvn46 Container started 48m Normal Created pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-vvn46 Container created 48m Normal Pulled pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-vvn46 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-vvn46 Add eth0 [10.133.0.71/23] from ovn-kubernetes 48m Normal AddedInterface pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr Add eth0 [10.134.0.77/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-7bggr Add eth0 [10.132.0.128/23] from ovn-kubernetes 48m Normal AddedInterface pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-gjgqz Add eth0 [10.132.0.129/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-gjgqz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Started pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr Container started 47m Normal Started pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-gjgqz Container started 47m Normal Created pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-7qx7j Container created 47m Normal Started pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-7qx7j Container started 47m Normal Created pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-gjgqz Container created 47m Normal Created pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr Container created 47m Normal Pulled pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-7qx7j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-7qx7j Add eth0 [10.132.0.130/23] from ovn-kubernetes 47m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7f5667f86c from 10 to 8 47m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7f5667f86c Deleted pod: maas-default-gateway-openshift-default-7f5667f86c-wtfpt 47m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7f5667f86c Deleted pod: maas-default-gateway-openshift-default-7f5667f86c-vs7zn 47m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-wtfpt Stopping container istio-proxy 47m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: All metrics below target 47m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-vs7zn Stopping container istio-proxy 47m Warning Unhealthy pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-gjgqz Startup probe failed: Get "http://10.132.0.129:15021/healthz/ready": dial tcp 10.132.0.129:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-7bggr Startup probe error: Get "http://10.132.0.128:15021/healthz/ready": dial tcp 10.132.0.128:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr Startup probe failed: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-7qx7j Startup probe error: Get "http://10.132.0.130:15021/healthz/ready": dial tcp 10.132.0.130:15021: connect: connection refused... 47m Warning ProbeError pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-vvn46 Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused... 47m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-vs7zn Readiness probe failed: HTTP probe failed with statuscode: 503 47m Warning ProbeError pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr Startup probe error: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused... 47m Warning ProbeError pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-gjgqz Startup probe error: Get "http://10.132.0.129:15021/healthz/ready": dial tcp 10.132.0.129:15021: connect: connection refused... 47m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-vs7zn Readiness probe error: HTTP probe failed with statuscode: 503... 47m Warning Unhealthy pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-7bggr Startup probe failed: Get "http://10.132.0.128:15021/healthz/ready": dial tcp 10.132.0.128:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-nlgd9 Startup probe error: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-7qx7j Startup probe failed: Get "http://10.132.0.130:15021/healthz/ready": dial tcp 10.132.0.130:15021: connect: connection refused 47m Warning Unhealthy pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-nlgd9 Startup probe failed: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused 47m Warning Unhealthy pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-vvn46 Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused 47m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7f5667f86c Deleted pod: maas-default-gateway-openshift-default-7f5667f86c-hn9sv 47m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7f5667f86c Deleted pod: maas-default-gateway-openshift-default-7f5667f86c-2jqg5 47m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7f5667f86c from 8 to 6 47m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-hn9sv Stopping container istio-proxy 47m Warning Unhealthy pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-cr5jb Readiness probe failed: Get "http://10.132.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-cr5jb Readiness probe error: Get "http://10.132.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-2jqg5 Stopping container istio-proxy 47m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: All metrics below target 47m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-k2njq Stopping container istio-proxy 47m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default (combined from similar events): Scaled down replica set maas-default-gateway-openshift-default-7f5667f86c from 6 to 3 47m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7f5667f86c Deleted pod: maas-default-gateway-openshift-default-7f5667f86c-xtmcj 47m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-kr95p Stopping container istio-proxy 47m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-xtmcj Stopping container istio-proxy 47m Normal ScalingReplicaSet deployment/e2e-ait-468d8d1b-openshift-default Scaled up replica set e2e-ait-468d8d1b-openshift-default-685f7b56f5 from 0 to 1 47m Normal AddedLabel gateway/e2e-ait-468d8d1b Added label istio.io/rev=openshift-gateway to gateway e2e-ait-468d8d1b 47m Normal AddedInterface pod/e2e-ait-468d8d1b-openshift-default-685f7b56f5-rkfh8 Add eth0 [10.132.0.143/23] from ovn-kubernetes 47m Normal SuccessfulCreate replicaset/e2e-ait-468d8d1b-openshift-default-685f7b56f5 Created pod: e2e-ait-468d8d1b-openshift-default-685f7b56f5-rkfh8 47m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-kr95p Readiness probe failed: HTTP probe failed with statuscode: 503 47m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-kr95p Readiness probe error: HTTP probe failed with statuscode: 503... 47m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-xtmcj Readiness probe error: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused... 47m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-xtmcj Readiness probe failed: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused 47m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-k2njq Readiness probe error: HTTP probe failed with statuscode: 503... 47m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-k2njq Readiness probe failed: HTTP probe failed with statuscode: 503 47m Normal SuccessfulCreate replicaset/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45 Created pod: e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj 47m Normal ScalingReplicaSet deployment/e2e-migrate-fc129ccf-gw-openshift-default Scaled up replica set e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45 from 0 to 1 47m Normal AddedLabel gateway/e2e-migrate-fc129ccf-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-fc129ccf-gw 47m Normal AddedInterface pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj Add eth0 [10.134.0.78/23] from ovn-kubernetes 47m Normal ScalingReplicaSet deployment/e2e-worker-w1-7afb1c-openshift-default Scaled up replica set e2e-worker-w1-7afb1c-openshift-default-6d7b749b87 from 6 to 8 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-7afb1c-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 47m Normal SuccessfulCreate replicaset/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87 Created pod: e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb 47m Normal SuccessfulCreate replicaset/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87 Created pod: e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-9v2fq 47m Normal Pulled pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-9v2fq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb Add eth0 [10.134.0.79/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb Container created 47m Normal Started pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-9v2fq Container started 47m Normal Created pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-9v2fq Container created 47m Normal Started pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb Container started 47m Normal AddedInterface pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-9v2fq Add eth0 [10.132.0.144/23] from ovn-kubernetes 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-468d8d1b Scaled up replica set payload-pre-processing-e2e-ait-468d8d1b-6689969c4c from 0 to 1 47m Warning ProbeError pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg Readiness probe error: Get "http://10.134.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-468d8d1b Scaled up replica set payload-processing-e2e-ait-468d8d1b-58646d466d from 0 to 1 47m Warning Unhealthy pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg Readiness probe failed: Get "http://10.134.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c Error creating: pods "payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-468d8d1b: serviceaccount "payload-processing-e2e-ait-468d8d1b" not found 47m Warning FailedCreate replicaset/payload-processing-e2e-ait-468d8d1b-58646d466d Error creating: pods "payload-processing-e2e-ait-468d8d1b-58646d466d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-468d8d1b: serviceaccount "payload-processing-e2e-ait-468d8d1b" not found 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c Created pod: payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-468d8d1b-58646d466d Created pod: payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz 47m Warning Unhealthy pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-54fm8 Readiness probe failed: Get "http://10.133.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-nc2kj Readiness probe failed: Get "http://10.133.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-54fm8 Readiness probe error: Get "http://10.133.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-qk9xk Readiness probe error: Get "http://10.133.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-7bggr Readiness probe error: Get "http://10.132.0.128:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr Readiness probe error: Get "http://10.134.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-nc2kj Readiness probe error: Get "http://10.133.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-qk9xk Readiness probe failed: Get "http://10.133.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr Readiness probe failed: Get "http://10.134.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-vvn46 Readiness probe error: Get "http://10.133.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-vvn46 Readiness probe failed: Get "http://10.133.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4 Readiness probe error: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4 Readiness probe failed: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-7bggr Readiness probe failed: Get "http://10.132.0.128:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5 Readiness probe error: Get "http://10.134.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf Readiness probe error: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-w6k2c Readiness probe error: Get "http://10.133.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5 Readiness probe failed: Get "http://10.134.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9p25h Readiness probe failed: Get "http://10.133.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9p25h Readiness probe error: Get "http://10.133.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-w6k2c Readiness probe failed: Get "http://10.133.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-g7kbs Readiness probe failed: Get "http://10.132.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-g7kbs Readiness probe error: Get "http://10.132.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-7qx7j Readiness probe failed: Get "http://10.132.0.130:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-gjgqz Readiness probe error: Get "http://10.132.0.129:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-gjgqz Readiness probe failed: Get "http://10.132.0.129:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-7qx7j Readiness probe error: Get "http://10.132.0.130:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf Readiness probe failed: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz Readiness probe error: Get "http://10.134.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-nprdv Readiness probe error: Get "http://10.132.0.120:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz Readiness probe failed: Get "http://10.134.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-nprdv Readiness probe failed: Get "http://10.132.0.120:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-xvdvr Readiness probe failed: Get "http://10.132.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-xvdvr Readiness probe error: Get "http://10.132.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-2jsvl Readiness probe failed: Get "http://10.132.0.114:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-2jsvl Readiness probe error: Get "http://10.132.0.114:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf Readiness probe failed: Get "http://10.134.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf Readiness probe error: Get "http://10.134.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-9v2fq Startup probe error: Get "http://10.132.0.144:15021/healthz/ready": dial tcp 10.132.0.144:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-9v2fq Startup probe failed: Get "http://10.132.0.144:15021/healthz/ready": dial tcp 10.132.0.144:15021: connect: connection refused 47m Normal Pulled pod/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x Add eth0 [10.134.0.80/23] from ovn-kubernetes 47m Normal Created pod/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x Container created 47m Warning Unhealthy pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb Startup probe failed: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused 47m Normal AddedInterface pod/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz Add eth0 [10.134.0.81/23] from ovn-kubernetes 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-fc129ccf Scaled up replica set payload-processing-e2e-migrate-fc129ccf-9f4d47cd6 from 0 to 1 47m Normal Pulled pod/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Warning FailedCreate replicaset/payload-processing-e2e-migrate-fc129ccf-9f4d47cd6 Error creating: pods "payload-processing-e2e-migrate-fc129ccf-9f4d47cd6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-fc129ccf: serviceaccount "payload-processing-e2e-migrate-fc129ccf" not found 47m Warning ProbeError pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb Startup probe error: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused... 47m Normal Started pod/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz Container started 47m Warning ProbeError pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-9v2fq Startup probe error: Get "http://10.132.0.144:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-fc129ccf Scaled up replica set payload-pre-processing-e2e-migrate-fc129ccf-75b6c8f7d7 from 0 to 1 47m Normal Started pod/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x Container started 47m Normal Created pod/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz Container created 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-fc129ccf-9f4d47cd6 Created pod: payload-processing-e2e-migrate-fc129ccf-9f4d47cd6-bv8bj 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-fc129ccf-75b6c8f7d7 Error creating: pods "payload-pre-processing-e2e-migrate-fc129ccf-75b6c8f7d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-fc129ccf: serviceaccount "payload-processing-e2e-migrate-fc129ccf" not found 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-fc129ccf-75b6c8f7d7 Created pod: payload-pre-processing-e2e-migrate-fc129ccf-75b6c8f7d7-qtmzt 46m Normal AddedInterface pod/payload-processing-e2e-migrate-fc129ccf-9f4d47cd6-bv8bj Add eth0 [10.132.0.153/23] from ovn-kubernetes 46m Normal Started pod/payload-processing-e2e-migrate-fc129ccf-9f4d47cd6-bv8bj Container started 46m Normal Pulled pod/payload-pre-processing-e2e-migrate-fc129ccf-75b6c8f7d7-qtmzt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Started pod/payload-pre-processing-e2e-migrate-fc129ccf-75b6c8f7d7-qtmzt Container started 46m Normal Pulled pod/payload-processing-e2e-migrate-fc129ccf-9f4d47cd6-bv8bj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-fc129ccf-75b6c8f7d7-qtmzt Add eth0 [10.132.0.152/23] from ovn-kubernetes 46m Normal Created pod/payload-pre-processing-e2e-migrate-fc129ccf-75b6c8f7d7-qtmzt Container created 46m Normal Created pod/payload-processing-e2e-migrate-fc129ccf-9f4d47cd6-bv8bj Container created 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-fc129ccf-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-fc129ccf-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal Pulled pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2bdcq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-28w9g Add eth0 [10.133.0.75/23] from ovn-kubernetes 46m Normal SuccessfulCreate replicaset/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 Created pod: e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-g6qdv 46m Normal SuccessfulCreate replicaset/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 Created pod: e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-59hxl 46m Normal SuccessfulCreate replicaset/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 Created pod: e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-cbd26c-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 46m Normal ScalingReplicaSet deployment/e2e-worker-w5-cbd26c-openshift-default Scaled up replica set e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 from 4 to 8 46m Normal SuccessfulCreate replicaset/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 Created pod: e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz 46m Normal Pulled pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-28w9g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal ScalingReplicaSet deployment/e2e-worker-w6-7540da-openshift-default Scaled up replica set e2e-worker-w6-7540da-openshift-default-bb989bcf9 from 7 to 10 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-7540da-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 46m Normal SuccessfulCreate replicaset/e2e-worker-w6-7540da-openshift-default-bb989bcf9 (combined from similar events): Created pod: e2e-worker-w6-7540da-openshift-default-bb989bcf9-hsn96 46m Normal SuccessfulCreate replicaset/e2e-worker-w6-7540da-openshift-default-bb989bcf9 Created pod: e2e-worker-w6-7540da-openshift-default-bb989bcf9-2bdcq 46m Normal SuccessfulCreate replicaset/e2e-worker-w6-7540da-openshift-default-bb989bcf9 Created pod: e2e-worker-w6-7540da-openshift-default-bb989bcf9-28w9g 46m Normal AddedInterface pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2bdcq Add eth0 [10.133.0.76/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Pulled pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-hsn96 Container created 46m Normal Pulled pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-hsn96 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-hsn96 Add eth0 [10.132.0.155/23] from ovn-kubernetes 46m Normal Started pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz Container started 46m Normal ScalingReplicaSet deployment/e2e-worker-w3-1d31f8-openshift-default Scaled up replica set e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c from 6 to 8 46m Normal Started pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2bdcq Container started 46m Normal Created pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2bdcq Container created 46m Normal AddedInterface pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-g6qdv Add eth0 [10.133.0.77/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-g6qdv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-59hxl Container started 46m Normal Created pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-g6qdv Container created 46m Normal SuccessfulCreate replicaset/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c Created pod: e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-pwwqr 46m Normal SuccessfulCreate replicaset/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c Created pod: e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp 46m Normal Created pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz Container created 46m Normal AddedInterface pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz Add eth0 [10.134.0.85/23] from ovn-kubernetes 46m Normal AddedInterface pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp Add eth0 [10.134.0.87/23] from ovn-kubernetes 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-1d31f8-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 46m Normal Started pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-28w9g Container started 46m Normal Started pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-hsn96 Container started 46m Normal Created pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-28w9g Container created 46m Normal Created pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-59hxl Container created 46m Normal AddedInterface pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q Add eth0 [10.134.0.86/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q Container created 46m Normal AddedInterface pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-59hxl Add eth0 [10.132.0.156/23] from ovn-kubernetes 46m Normal Started pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q Container started 46m Normal Pulled pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-59hxl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-pwwqr Container created 46m Normal AddedInterface pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-pwwqr Add eth0 [10.133.0.78/23] from ovn-kubernetes 46m Normal Started pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp Container started 46m Normal Started pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-g6qdv Container started 46m Normal Pulled pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-pwwqr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-pwwqr Container started 46m Normal Created pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp Container created 46m Warning ProbeError pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-hsn96 Startup probe error: Get "http://10.132.0.155:15021/healthz/ready": dial tcp 10.132.0.155:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-hsn96 Startup probe failed: Get "http://10.132.0.155:15021/healthz/ready": dial tcp 10.132.0.155:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-59hxl Startup probe error: Get "http://10.132.0.156:15021/healthz/ready": dial tcp 10.132.0.156:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-59hxl Startup probe failed: Get "http://10.132.0.156:15021/healthz/ready": dial tcp 10.132.0.156:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q Startup probe failed: Get "http://10.134.0.86:15021/healthz/ready": dial tcp 10.134.0.86:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q Startup probe error: Get "http://10.134.0.86:15021/healthz/ready": dial tcp 10.134.0.86:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz Startup probe failed: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2bdcq Startup probe error: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz Startup probe error: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2bdcq Startup probe failed: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-28w9g Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp Startup probe error: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-28w9g Startup probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp Startup probe failed: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-g6qdv Startup probe error: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-g6qdv Startup probe failed: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-pwwqr Startup probe error: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-pwwqr Startup probe failed: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused 46m Normal SuccessfulCreate replicaset/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 Created pod: e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-5tnzh 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-cbd26c-openshift-default New size: 9; reason: All metrics below target 46m Normal ScalingReplicaSet deployment/e2e-worker-w5-cbd26c-openshift-default Scaled up replica set e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 from 8 to 9 46m Normal AddedInterface pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-5tnzh Add eth0 [10.132.0.158/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-5tnzh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Warning Unhealthy pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx Readiness probe failed: Get "http://10.134.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx Readiness probe error: Get "http://10.134.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2 Readiness probe failed: Get "http://10.134.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-ldktd Readiness probe error: Get "http://10.133.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning ProbeError pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-2nqsk Readiness probe error: Get "http://10.132.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning ProbeError pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-q95hh Readiness probe error: Get "http://10.133.0.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-rsqdc Readiness probe failed: Get "http://10.132.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2 Readiness probe error: Get "http://10.134.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-q95hh Readiness probe failed: Get "http://10.133.0.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-rsqdc Readiness probe error: Get "http://10.132.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-ldktd Readiness probe failed: Get "http://10.133.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning Unhealthy pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-2nqsk Readiness probe failed: Get "http://10.132.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Normal Started pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-5tnzh Container started 46m Normal Created pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-5tnzh Container created 46m Warning Unhealthy pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-5tnzh Startup probe failed: Get "http://10.132.0.158:15021/healthz/ready": dial tcp 10.132.0.158:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-5tnzh Startup probe error: Get "http://10.132.0.158:15021/healthz/ready": dial tcp 10.132.0.158:15021: connect: connection refused... 46m Normal AddedLabel gateway/e2e-ait-1cb89500 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-1cb89500 46m Normal ScalingReplicaSet deployment/e2e-ait-1cb89500-openshift-default Scaled up replica set e2e-ait-1cb89500-openshift-default-7f99977f5c from 0 to 1 46m Normal SuccessfulCreate replicaset/e2e-ait-1cb89500-openshift-default-7f99977f5c Created pod: e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs 46m Normal AddedInterface pod/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs Add eth0 [10.134.0.88/23] from ovn-kubernetes 46m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 46m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 46m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-vndrh 46m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-vndrh Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 46m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-vndrh Add eth0 [10.132.0.160/23] from ovn-kubernetes 45m Normal Pulled pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-6jvmm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-6jvmm Add eth0 [10.133.0.81/23] from ovn-kubernetes 45m Normal Created pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-6jvmm Container created 45m Normal ScalingReplicaSet deployment/e2e-worker-w5-cbd26c-openshift-default Scaled up replica set e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 from 9 to 10 45m Normal Started pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-6jvmm Container started 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-cbd26c-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 45m Normal SuccessfulCreate replicaset/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 (combined from similar events): Created pod: e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-6jvmm 45m Normal ScalingReplicaSet deployment/e2e-worker-w3-1d31f8-openshift-default Scaled up replica set e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c from 8 to 10 45m Normal SuccessfulCreate replicaset/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c (combined from similar events): Created pod: e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-1d31f8-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 45m Normal SuccessfulCreate replicaset/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c Created pod: e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-xfkst 45m Normal Created pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-xfkst Container created 45m Normal Started pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-xfkst Container started 45m Normal Started pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr Container started 45m Normal Pulled pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr Add eth0 [10.134.0.89/23] from ovn-kubernetes 45m Normal Pulled pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-xfkst Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-xfkst Add eth0 [10.132.0.161/23] from ovn-kubernetes 45m Normal Created pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr Container created 45m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-vndrh Container created 45m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-vndrh Container started 45m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-vndrh Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 6.114s (6.114s including waiting). Image size: 185476477 bytes. 45m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7f5667f86c from 3 to 6 45m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 45m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c (combined from similar events): Created pod: maas-default-gateway-openshift-default-7f5667f86c-5wrhg 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-zhbvw Add eth0 [10.132.0.162/23] from ovn-kubernetes 45m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-zhbvw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-d2tgj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-d2tgj Container started 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-d2tgj Add eth0 [10.134.0.90/23] from ovn-kubernetes 45m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-zhbvw Container created 45m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-zhbvw Container started 45m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-d2tgj Container created 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-5wrhg Add eth0 [10.134.0.91/23] from ovn-kubernetes 45m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-5wrhg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-5wrhg Container created 45m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-5wrhg Container started 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-1cb89500 Scaled up replica set payload-pre-processing-e2e-ait-1cb89500-66f7868c55 from 0 to 1 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-1cb89500 Scaled up replica set payload-processing-e2e-ait-1cb89500-7846b7d4b6 from 0 to 1 45m Warning FailedCreate replicaset/payload-processing-e2e-ait-1cb89500-7846b7d4b6 Error creating: pods "payload-processing-e2e-ait-1cb89500-7846b7d4b6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-1cb89500: serviceaccount "payload-processing-e2e-ait-1cb89500" not found 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-1cb89500-66f7868c55 Error creating: pods "payload-pre-processing-e2e-ait-1cb89500-66f7868c55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-1cb89500: serviceaccount "payload-processing-e2e-ait-1cb89500" not found 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-1cb89500-7846b7d4b6 Created pod: payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-1cb89500-66f7868c55 Created pod: payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw 45m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-vndrh Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-vndrh Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning Unhealthy pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-6jvmm Startup probe failed: Get "http://10.133.0.81:15021/healthz/ready": dial tcp 10.133.0.81:15021: connect: connection refused 45m Warning ProbeError pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-6jvmm Startup probe error: Get "http://10.133.0.81:15021/healthz/ready": dial tcp 10.133.0.81:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-xfkst Startup probe failed: Get "http://10.132.0.161:15021/healthz/ready": dial tcp 10.132.0.161:15021: connect: connection refused 45m Warning ProbeError pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-xfkst Startup probe error: Get "http://10.132.0.161:15021/healthz/ready": dial tcp 10.132.0.161:15021: connect: connection refused... 45m Normal AddedInterface pod/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw Add eth0 [10.134.0.92/23] from ovn-kubernetes 45m Warning ProbeError pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp Readiness probe error: Get "http://10.134.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning Unhealthy pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp Readiness probe failed: Get "http://10.134.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Normal AddedInterface pod/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6 Add eth0 [10.134.0.93/23] from ovn-kubernetes 45m Normal Created pod/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw Container created 45m Normal Started pod/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw Container started 45m Normal Started pod/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6 Container started 45m Warning Unhealthy pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr Startup probe failed: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused 45m Normal Pulled pod/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Warning ProbeError pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr Startup probe error: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused... 45m Normal Pulled pod/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6 Container created 45m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-zhbvw Startup probe error: Get "http://10.132.0.162:15021/healthz/ready": dial tcp 10.132.0.162:15021: connect: connection refused... 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-zhbvw Startup probe failed: Get "http://10.132.0.162:15021/healthz/ready": dial tcp 10.132.0.162:15021: connect: connection refused 45m Warning Unhealthy pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr Startup probe failed: Get "http://10.134.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning ProbeError pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr Startup probe error: Get "http://10.134.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-5wrhg Startup probe failed: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused 45m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-5wrhg Startup probe error: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused... 45m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-d2tgj Startup probe error: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused... 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-d2tgj Startup probe failed: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused 45m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: 45m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-zkrw7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-zkrw7 Add eth0 [10.132.0.163/23] from ovn-kubernetes 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-468d8d1b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-6jrgz Container created 45m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-6jrgz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-6jrgz Add eth0 [10.133.0.83/23] from ovn-kubernetes 45m Normal Started pod/e2e-ait-468d8d1b-openshift-default-685f7b56f5-rkfh8 Container started 45m Normal Created pod/e2e-ait-468d8d1b-openshift-default-685f7b56f5-rkfh8 Container created 45m Normal Pulled pod/e2e-ait-468d8d1b-openshift-default-685f7b56f5-rkfh8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-6jrgz Container started 45m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-np57c Container created 45m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-np57c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-np57c Add eth0 [10.134.0.94/23] from ovn-kubernetes 45m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-zkrw7 Container started 45m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-zkrw7 Container created 45m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-lxr98 Container started 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-lxr98 Add eth0 [10.133.0.82/23] from ovn-kubernetes 45m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-lxr98 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-lxr98 Container created 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-468d8d1b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-np57c Container started 45m Warning ProbeError pod/e2e-ait-468d8d1b-openshift-default-685f7b56f5-rkfh8 Startup probe error: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-ait-468d8d1b-openshift-default-685f7b56f5-rkfh8 Startup probe failed: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-np57c Startup probe failed: Get "http://10.134.0.94:15021/healthz/ready": dial tcp 10.134.0.94:15021: connect: connection refused 45m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-lxr98 Startup probe error: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused... 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-lxr98 Startup probe failed: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-6jrgz Startup probe failed: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused 45m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-np57c Startup probe error: Get "http://10.134.0.94:15021/healthz/ready": dial tcp 10.134.0.94:15021: connect: connection refused... 45m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-6jrgz Startup probe error: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused... 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-zkrw7 Startup probe failed: Get "http://10.132.0.163:15021/healthz/ready": dial tcp 10.132.0.163:15021: connect: connection refused 45m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-zkrw7 Startup probe error: Get "http://10.132.0.163:15021/healthz/ready": dial tcp 10.132.0.163:15021: connect: connection refused... 45m Normal Killing pod/payload-processing-e2e-ait-468d8d1b-58646d466d-dqbjz Stopping container payload-processing 45m Normal Killing pod/payload-pre-processing-e2e-ait-468d8d1b-6689969c4c-t6l8x Stopping container payload-pre-processing 45m Normal Killing pod/payload-processing-e2e-migrate-fc129ccf-9f4d47cd6-bv8bj Stopping container payload-processing 45m Normal Killing pod/payload-pre-processing-e2e-migrate-fc129ccf-75b6c8f7d7-qtmzt Stopping container payload-pre-processing 45m Normal SuccessfulCreate replicaset/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45 Created pod: e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-4hrkd 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-migrate-fc129ccf-gw-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 45m Normal ScalingReplicaSet deployment/e2e-migrate-fc129ccf-gw-openshift-default Scaled up replica set e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45 from 1 to 3 45m Normal SuccessfulCreate replicaset/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45 Created pod: e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-8hmfx 45m Normal Created pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj Container created 45m Normal AddedInterface pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-4hrkd Add eth0 [10.133.0.85/23] from ovn-kubernetes 45m Normal AddedInterface pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-8hmfx Add eth0 [10.132.0.168/23] from ovn-kubernetes 45m Normal Pulled pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-8hmfx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-8hmfx Container created 45m Normal Started pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-8hmfx Container started 45m Normal Pulled pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-4hrkd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-4hrkd Container created 45m Normal Started pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-4hrkd Container started 45m Normal Pulled pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj Container started 44m Warning Unhealthy pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj Startup probe failed: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused 44m Warning ProbeError pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj Startup probe error: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused... 44m Warning ProbeError pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-8hmfx Startup probe error: Get "http://10.132.0.168:15021/healthz/ready": dial tcp 10.132.0.168:15021: connect: connection refused... 44m Warning ProbeError pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-4hrkd Startup probe error: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-8hmfx Startup probe failed: Get "http://10.132.0.168:15021/healthz/ready": dial tcp 10.132.0.168:15021: connect: connection refused 44m Warning Unhealthy pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-4hrkd Startup probe failed: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-468d8d1b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-468d8d1b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal Killing pod/e2e-ait-468d8d1b-openshift-default-685f7b56f5-rkfh8 Stopping container istio-proxy 44m Normal AddedLabel gateway/e2e-ait-26b8d994 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-26b8d994 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-fc129ccf-gw-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-fc129ccf-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal SuccessfulCreate replicaset/e2e-ait-26b8d994-openshift-default-78f469648f Created pod: e2e-ait-26b8d994-openshift-default-78f469648f-4psc8 44m Normal ScalingReplicaSet deployment/e2e-ait-26b8d994-openshift-default Scaled up replica set e2e-ait-26b8d994-openshift-default-78f469648f from 0 to 1 44m Warning Unhealthy pod/e2e-ait-468d8d1b-openshift-default-685f7b56f5-rkfh8 Readiness probe failed: HTTP probe failed with statuscode: 503 44m Warning ProbeError pod/e2e-ait-468d8d1b-openshift-default-685f7b56f5-rkfh8 Readiness probe error: HTTP probe failed with statuscode: 503... 44m Normal AddedInterface pod/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8 Add eth0 [10.134.0.95/23] from ovn-kubernetes 44m Normal Killing pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-8hmfx Stopping container istio-proxy 44m Normal Killing pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj Stopping container istio-proxy 44m Normal Killing pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-4hrkd Stopping container istio-proxy 44m Warning Unhealthy pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj Readiness probe failed: HTTP probe failed with statuscode: 503 44m Normal AddedInterface pod/e2e-derive-9d7aa682-openshift-default-74bdd985b5-sv4td Add eth0 [10.133.0.86/23] from ovn-kubernetes 44m Normal SuccessfulCreate replicaset/e2e-derive-9d7aa682-openshift-default-74bdd985b5 Created pod: e2e-derive-9d7aa682-openshift-default-74bdd985b5-sv4td 44m Warning ProbeError pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-t5ljj Readiness probe error: HTTP probe failed with statuscode: 503... 44m Normal AddedLabel gateway/e2e-derive-9d7aa682 Added label istio.io/rev=openshift-gateway to gateway e2e-derive-9d7aa682 44m Normal ScalingReplicaSet deployment/e2e-derive-9d7aa682-openshift-default Scaled up replica set e2e-derive-9d7aa682-openshift-default-74bdd985b5 from 0 to 1 44m Warning ProbeError pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-8hmfx Readiness probe error: HTTP probe failed with statuscode: 503... 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7 Error creating: pods "payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-26b8d994: serviceaccount "payload-processing-e2e-ait-26b8d994" not found 44m Warning Unhealthy pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-8hmfx Readiness probe failed: HTTP probe failed with statuscode: 503 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-26b8d994 Scaled up replica set payload-processing-e2e-ait-26b8d994-85f6dc5786 from 0 to 1 44m Warning Unhealthy pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-4hrkd Readiness probe failed: HTTP probe failed with statuscode: 503 44m Warning ProbeError pod/e2e-migrate-fc129ccf-gw-openshift-default-5cd657dd45-4hrkd Readiness probe error: HTTP probe failed with statuscode: 503... 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-26b8d994-85f6dc5786 Created pod: payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch 44m Warning FailedCreate replicaset/payload-processing-e2e-ait-26b8d994-85f6dc5786 Error creating: pods "payload-processing-e2e-ait-26b8d994-85f6dc5786-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-26b8d994: serviceaccount "payload-processing-e2e-ait-26b8d994" not found 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-26b8d994 Scaled up replica set payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7 from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7 Created pod: payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9 44m Normal AddedInterface pod/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch Add eth0 [10.134.0.96/23] from ovn-kubernetes 44m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-nqmgk Readiness probe error: Get "http://10.133.0.21:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-nqmgk Readiness probe failed: Get "http://10.133.0.21:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Normal Pulled pod/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Pulled pod/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9 Add eth0 [10.134.0.97/23] from ovn-kubernetes 44m Normal Created pod/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch Container created 44m Normal Started pod/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9 Container started 44m Normal Started pod/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch Container started 44m Normal Created pod/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9 Container created 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959 Created pod: payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959 Error creating: pods "payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-9d7aa682: serviceaccount "payload-processing-e2e-derive-9d7aa682" not found 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-9d7aa682 Scaled up replica set payload-processing-e2e-derive-9d7aa682-7f85d95b8d from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-9d7aa682-7f85d95b8d Created pod: payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-9d7aa682 Scaled up replica set payload-pre-processing-e2e-derive-9d7aa682-5576bd6959 from 0 to 1 44m Warning FailedCreate replicaset/payload-processing-e2e-derive-9d7aa682-7f85d95b8d Error creating: pods "payload-processing-e2e-derive-9d7aa682-7f85d95b8d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-9d7aa682: serviceaccount "payload-processing-e2e-derive-9d7aa682" not found 44m Normal Created pod/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp Container created 44m Normal Pulled pod/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Started pod/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b Container started 44m Normal Started pod/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp Container started 44m Normal AddedInterface pod/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b Add eth0 [10.134.0.98/23] from ovn-kubernetes 44m Normal Pulled pod/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp Add eth0 [10.134.0.99/23] from ovn-kubernetes 44m Normal Created pod/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b Container created 44m Normal Pulled pod/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Started pod/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs Container started 44m Normal Created pod/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs Container created 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-1cb89500-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-1cb89500-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning Unhealthy pod/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs Startup probe failed: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused 44m Warning ProbeError pod/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs Startup probe error: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused... 44m Normal AddedLabel gateway/e2e-shared-a-w2-c93e70 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-c93e70 44m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-c93e70-openshift-default Scaled up replica set e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69 from 0 to 1 44m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: cpu resource utilization (percentage of request) above target 44m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 3 44m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69 Created pod: e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb 44m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-vsltn 44m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-vsltn Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 44m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-vsltn Add eth0 [10.133.0.87/23] from ovn-kubernetes 44m Normal AddedInterface pod/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb Add eth0 [10.134.0.100/23] from ovn-kubernetes 44m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-vsltn Container created 44m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-vsltn Container started 44m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-vsltn Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 5.568s (5.568s including waiting). Image size: 185476477 bytes. 44m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-vsltn Readiness probe failed: HTTP probe failed with statuscode: 503 44m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-vsltn Readiness probe error: HTTP probe failed with statuscode: 503... 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-c93e70 Scaled up replica set payload-pre-processing-e2e-shared-a-w2-c93e70-8fbcf64c from 0 to 1 44m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-c93e70-5ffc55b946 Error creating: pods "payload-processing-e2e-shared-a-w2-c93e70-5ffc55b946-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-c93e70: serviceaccount "payload-processing-e2e-shared-a-w2-c93e70" not found 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-c93e70-5ffc55b946 Created pod: payload-processing-e2e-shared-a-w2-c93e70-5ffc55b946-vqsqn 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-c93e70 Scaled up replica set payload-processing-e2e-shared-a-w2-c93e70-5ffc55b946 from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-c93e70-8fbcf64c Created pod: payload-pre-processing-e2e-shared-a-w2-c93e70-8fbcf64c-bvn98 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-c93e70-8fbcf64c Error creating: pods "payload-pre-processing-e2e-shared-a-w2-c93e70-8fbcf64c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-c93e70: serviceaccount "payload-processing-e2e-shared-a-w2-c93e70" not found 44m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-c93e70-8fbcf64c-bvn98 Add eth0 [10.132.0.170/23] from ovn-kubernetes 44m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-c93e70-5ffc55b946-vqsqn Add eth0 [10.132.0.171/23] from ovn-kubernetes 44m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-c93e70-8fbcf64c-bvn98 Container started 44m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-c93e70-8fbcf64c-bvn98 Container created 44m Normal Pulled pod/payload-processing-e2e-shared-a-w2-c93e70-5ffc55b946-vqsqn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-processing-e2e-shared-a-w2-c93e70-5ffc55b946-vqsqn Container created 44m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-c93e70-8fbcf64c-bvn98 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Started pod/payload-processing-e2e-shared-a-w2-c93e70-5ffc55b946-vqsqn Container started 43m Normal Killing pod/payload-processing-e2e-ait-1cb89500-7846b7d4b6-sb5q6 Stopping container payload-processing 43m Normal Killing pod/payload-pre-processing-e2e-ait-1cb89500-66f7868c55-mj6mw Stopping container payload-pre-processing 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-1cb89500-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-1cb89500-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal AddedLabel gateway/e2e-mt-ebffae22 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-ebffae22 43m Normal ScalingReplicaSet deployment/e2e-mt-ebffae22-openshift-default Scaled up replica set e2e-mt-ebffae22-openshift-default-78c8d7888f from 0 to 1 43m Normal SuccessfulCreate replicaset/e2e-mt-ebffae22-openshift-default-78c8d7888f Created pod: e2e-mt-ebffae22-openshift-default-78c8d7888f-tg5g7 43m Normal AddedInterface pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-tg5g7 Add eth0 [10.132.0.172/23] from ovn-kubernetes 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-ebffae22 Scaled up replica set payload-processing-e2e-mt-ebffae22-675bf6d65d from 0 to 1 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-ebffae22 Scaled up replica set payload-pre-processing-e2e-mt-ebffae22-76f58b75cf from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-ebffae22-76f58b75cf Created pod: payload-pre-processing-e2e-mt-ebffae22-76f58b75cf-kmqd4 43m Warning FailedCreate replicaset/payload-processing-e2e-mt-ebffae22-675bf6d65d Error creating: pods "payload-processing-e2e-mt-ebffae22-675bf6d65d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-ebffae22: serviceaccount "payload-processing-e2e-mt-ebffae22" not found 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-ebffae22-76f58b75cf Error creating: pods "payload-pre-processing-e2e-mt-ebffae22-76f58b75cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-ebffae22: serviceaccount "payload-processing-e2e-mt-ebffae22" not found 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-ebffae22-675bf6d65d Created pod: payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4 43m Normal Pulled pod/payload-pre-processing-e2e-mt-ebffae22-76f58b75cf-kmqd4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4 Add eth0 [10.134.0.103/23] from ovn-kubernetes 43m Normal AddedInterface pod/payload-pre-processing-e2e-mt-ebffae22-76f58b75cf-kmqd4 Add eth0 [10.132.0.173/23] from ovn-kubernetes 43m Normal Pulled pod/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4 Container created 43m Normal Started pod/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4 Container started 43m Normal Started pod/payload-pre-processing-e2e-mt-ebffae22-76f58b75cf-kmqd4 Container started 43m Normal Created pod/payload-pre-processing-e2e-mt-ebffae22-76f58b75cf-kmqd4 Container created 43m Normal Killing pod/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs Stopping container istio-proxy 43m Warning Unhealthy pod/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs Readiness probe failed: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused 43m Warning ProbeError pod/e2e-ait-1cb89500-openshift-default-7f99977f5c-2bjjs Readiness probe error: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused... 43m Normal SuccessfulCreate replicaset/e2e-mt-a9ab23c4-openshift-default-766c495bb9 Created pod: e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r 43m Normal ScalingReplicaSet deployment/e2e-mt-a9ab23c4-openshift-default Scaled up replica set e2e-mt-a9ab23c4-openshift-default-766c495bb9 from 0 to 1 43m Normal AddedLabel gateway/e2e-mt-a9ab23c4 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-a9ab23c4 43m Normal AddedInterface pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r Add eth0 [10.134.0.104/23] from ovn-kubernetes 43m Warning Unhealthy pod/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8 Readiness probe failed: Get "http://10.134.0.95:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Warning ProbeError pod/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8 Readiness probe error: Get "http://10.134.0.95:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Normal AddedLabel gateway/e2e-shared-a-w3-eb3c33 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-eb3c33 43m Normal Created pod/e2e-derive-9d7aa682-openshift-default-74bdd985b5-sv4td Container created 43m Normal Started pod/e2e-derive-9d7aa682-openshift-default-74bdd985b5-sv4td Container started 43m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-eb3c33-openshift-default-8687c6c447 Created pod: e2e-shared-a-w3-eb3c33-openshift-default-8687c6c447-x7w6q 43m Normal Pulled pod/e2e-derive-9d7aa682-openshift-default-74bdd985b5-sv4td Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-eb3c33-openshift-default Scaled up replica set e2e-shared-a-w3-eb3c33-openshift-default-8687c6c447 from 0 to 1 43m Normal AddedInterface pod/e2e-shared-a-w3-eb3c33-openshift-default-8687c6c447-x7w6q Add eth0 [10.132.0.174/23] from ovn-kubernetes 43m Normal Created pod/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8 Container created 43m Normal Pulled pod/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8 Container started 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-a9ab23c4 Scaled up replica set payload-processing-e2e-mt-a9ab23c4-845f6c8c86 from 0 to 1 43m Warning ProbeError pod/e2e-derive-9d7aa682-openshift-default-74bdd985b5-sv4td Startup probe error: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused... 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-a9ab23c4-845f6c8c86 Created pod: payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-a9ab23c4 Scaled up replica set payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9 from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9 Created pod: payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9 Error creating: pods "payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-a9ab23c4: serviceaccount "payload-processing-e2e-mt-a9ab23c4" not found 43m Warning FailedCreate replicaset/payload-processing-e2e-mt-a9ab23c4-845f6c8c86 Error creating: pods "payload-processing-e2e-mt-a9ab23c4-845f6c8c86-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-a9ab23c4: serviceaccount "payload-processing-e2e-mt-a9ab23c4" not found 43m Warning Unhealthy pod/e2e-derive-9d7aa682-openshift-default-74bdd985b5-sv4td Startup probe failed: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused 43m Normal Pulled pod/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs Add eth0 [10.134.0.105/23] from ovn-kubernetes 43m Normal Pulled pod/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb Container started 43m Warning ProbeError pod/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8 Startup probe error: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8 Startup probe failed: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused 43m Normal Created pod/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs Container created 43m Normal Started pod/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs Container started 43m Normal Created pod/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb Container created 43m Normal AddedInterface pod/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb Add eth0 [10.134.0.106/23] from ovn-kubernetes 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w3-eb3c33 Scaled up replica set payload-pre-processing-e2e-shared-a-w3-eb3c33-6ff459c8c9 from 0 to 1 42m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w3-eb3c33-846894ccd8 Error creating: pods "payload-processing-e2e-shared-a-w3-eb3c33-846894ccd8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-eb3c33: serviceaccount "payload-processing-e2e-shared-a-w3-eb3c33" not found 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w3-eb3c33 Scaled up replica set payload-processing-e2e-shared-a-w3-eb3c33-846894ccd8 from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w3-eb3c33-846894ccd8 Created pod: payload-processing-e2e-shared-a-w3-eb3c33-846894ccd8-mzt64 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w3-eb3c33-6ff459c8c9 Error creating: pods "payload-pre-processing-e2e-shared-a-w3-eb3c33-6ff459c8c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-eb3c33: serviceaccount "payload-processing-e2e-shared-a-w3-eb3c33" not found 42m Normal Started pod/payload-processing-e2e-shared-a-w3-eb3c33-846894ccd8-mzt64 Container started 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-a9ab23c4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Normal Pulled pod/payload-processing-e2e-shared-a-w3-eb3c33-846894ccd8-mzt64 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-processing-e2e-shared-a-w3-eb3c33-846894ccd8-mzt64 Add eth0 [10.132.0.175/23] from ovn-kubernetes 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-a9ab23c4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w3-eb3c33-6ff459c8c9 Created pod: payload-pre-processing-e2e-shared-a-w3-eb3c33-6ff459c8c9-jfqlz 42m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w3-eb3c33-6ff459c8c9-jfqlz Add eth0 [10.132.0.176/23] from ovn-kubernetes 42m Normal Created pod/payload-processing-e2e-shared-a-w3-eb3c33-846894ccd8-mzt64 Container created 42m Normal Created pod/payload-pre-processing-e2e-shared-a-w3-eb3c33-6ff459c8c9-jfqlz Container created 42m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w3-eb3c33-6ff459c8c9-jfqlz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Started pod/payload-pre-processing-e2e-shared-a-w3-eb3c33-6ff459c8c9-jfqlz Container started 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-9d7aa682-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-9d7aa682-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-9d7aa682-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-9d7aa682-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal Killing pod/payload-processing-e2e-ait-26b8d994-85f6dc5786-2q4ch Stopping container payload-processing 42m Normal Killing pod/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9 Stopping container payload-pre-processing 42m Warning Unhealthy pod/payload-pre-processing-e2e-ait-26b8d994-68bbbbbfd7-ln9k9 Readiness probe failed: timeout: failed to connect service "10.134.0.97:50052" within 1s: context deadline exceeded 42m Normal Pulled pod/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb Container created 42m Normal Started pod/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb Container started 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-26b8d994-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-c93e70-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-c93e70-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-26b8d994-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Normal Killing pod/payload-pre-processing-e2e-derive-9d7aa682-5576bd6959-mp87b Stopping container payload-pre-processing 42m Normal Killing pod/payload-processing-e2e-derive-9d7aa682-7f85d95b8d-gfcvp Stopping container payload-processing 42m Warning ProbeError pod/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb Startup probe error: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb Startup probe failed: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused 42m Normal AddedLabel gateway/e2e-shared-b-w2-4a1480 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-4a1480 42m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-4a1480-openshift-default Scaled up replica set e2e-shared-b-w2-4a1480-openshift-default-75c6f8849c from 0 to 1 42m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-4a1480-openshift-default-75c6f8849c Created pod: e2e-shared-b-w2-4a1480-openshift-default-75c6f8849c-vw8pq 42m Normal AddedInterface pod/e2e-shared-b-w2-4a1480-openshift-default-75c6f8849c-vw8pq Add eth0 [10.132.0.179/23] from ovn-kubernetes 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-4a1480 Scaled up replica set payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58 from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58 Created pod: payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn 41m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58 Error creating: pods "payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480: serviceaccount "payload-processing-e2e-shared-b-w2-4a1480" not found 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87 Error creating: pods "payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-4a1480: serviceaccount "payload-processing-e2e-shared-b-w2-4a1480" not found 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87 Created pod: payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-4a1480 Scaled up replica set payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87 from 0 to 1 41m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm Add eth0 [10.134.0.109/23] from ovn-kubernetes 41m Normal Created pod/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn Container created 41m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn Add eth0 [10.134.0.108/23] from ovn-kubernetes 41m Normal Pulled pod/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm Container started 41m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm Container created 41m Normal Started pod/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn Container started 41m Normal ScalingReplicaSet deployment/e2e-mt-ebffae22-openshift-default Scaled up replica set e2e-mt-ebffae22-openshift-default-78c8d7888f from 1 to 3 41m Normal SuccessfulCreate replicaset/e2e-mt-ebffae22-openshift-default-78c8d7888f Created pod: e2e-mt-ebffae22-openshift-default-78c8d7888f-b9x4s 41m Normal SuccessfulCreate replicaset/e2e-mt-ebffae22-openshift-default-78c8d7888f Created pod: e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-ebffae22-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 41m Normal Started pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-b9x4s Container started 41m Normal Pulled pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-b9x4s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-b9x4s Container created 41m Normal AddedInterface pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-b9x4s Add eth0 [10.133.0.91/23] from ovn-kubernetes 41m Normal Created pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-tg5g7 Container created 41m Normal AddedInterface pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb Add eth0 [10.134.0.110/23] from ovn-kubernetes 41m Normal Killing pod/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8 Stopping container istio-proxy 41m Warning ProbeError pod/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8 Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-ait-26b8d994-openshift-default-78f469648f-4psc8 Readiness probe failed: HTTP probe failed with statuscode: 503 41m Normal Pulled pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-tg5g7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Started pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-tg5g7 Container started 41m Normal Started pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb Container started 41m Normal Created pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb Container created 41m Normal Pulled pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal AddedLabel gateway/e2e-shared-a-w0-ea1be0 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-ea1be0 41m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-ea1be0-openshift-default Scaled up replica set e2e-shared-a-w0-ea1be0-openshift-default-6bfb8dcf95 from 0 to 1 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-c93e70-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-c93e70-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-ea1be0-openshift-default-6bfb8dcf95 Created pod: e2e-shared-a-w0-ea1be0-openshift-default-6bfb8dcf95-wl6kg 41m Normal Killing pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-xvdvr Stopping container istio-proxy 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-b64e2e-openshift-default New size: 6; reason: All metrics below target 41m Normal SuccessfulDelete replicaset/e2e-worker-w4-32449d-openshift-default-756d4bb7c7 Deleted pod: e2e-worker-w4-32449d-openshift-default-756d4bb7c7-cr5jb 41m Normal SuccessfulDelete replicaset/e2e-worker-w4-32449d-openshift-default-756d4bb7c7 Deleted pod: e2e-worker-w4-32449d-openshift-default-756d4bb7c7-44frr 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-32449d-openshift-default New size: 6; reason: All metrics below target 41m Normal ScalingReplicaSet deployment/e2e-worker-w4-32449d-openshift-default Scaled down replica set e2e-worker-w4-32449d-openshift-default-756d4bb7c7 from 10 to 6 41m Normal Killing pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-2jsvl Stopping container istio-proxy 41m Normal AddedInterface pod/e2e-shared-a-w0-ea1be0-openshift-default-6bfb8dcf95-wl6kg Add eth0 [10.132.0.180/23] from ovn-kubernetes 41m Normal SuccessfulDelete replicaset/e2e-worker-w4-32449d-openshift-default-756d4bb7c7 Deleted pod: e2e-worker-w4-32449d-openshift-default-756d4bb7c7-xvdvr 41m Normal SuccessfulDelete replicaset/e2e-worker-w4-32449d-openshift-default-756d4bb7c7 Deleted pod: e2e-worker-w4-32449d-openshift-default-756d4bb7c7-2jsvl 41m Normal Killing pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-cr5jb Stopping container istio-proxy 41m Normal ScalingReplicaSet deployment/e2e-worker-w0-b64e2e-openshift-default Scaled down replica set e2e-worker-w0-b64e2e-openshift-default-64c944c45c from 10 to 6 41m Normal Killing pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-44frr Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w0-b64e2e-openshift-default-64c944c45c Deleted pod: e2e-worker-w0-b64e2e-openshift-default-64c944c45c-gjgqz 41m Normal SuccessfulDelete replicaset/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87 Deleted pod: e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb 41m Normal Killing pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-gjgqz Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w0-b64e2e-openshift-default-64c944c45c Deleted pod: e2e-worker-w0-b64e2e-openshift-default-64c944c45c-7bggr 41m Normal ScalingReplicaSet deployment/e2e-worker-w1-7afb1c-openshift-default Scaled down replica set e2e-worker-w1-7afb1c-openshift-default-6d7b749b87 from 8 to 4 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-7afb1c-openshift-default New size: 4; reason: All metrics below target 41m Normal Killing pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-7bggr Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-97cjb Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87 Deleted pod: e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-7qx7j 41m Normal Killing pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87 Deleted pod: e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr 41m Normal SuccessfulDelete replicaset/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87 Deleted pod: e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-9v2fq 41m Normal SuccessfulDelete replicaset/e2e-worker-w0-b64e2e-openshift-default-64c944c45c Deleted pod: e2e-worker-w0-b64e2e-openshift-default-64c944c45c-8mkq4 41m Normal Killing pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-p4dg2 Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-9v2fq Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-8mkq4 Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w0-b64e2e-openshift-default-64c944c45c Deleted pod: e2e-worker-w0-b64e2e-openshift-default-64c944c45c-p4dg2 41m Normal Killing pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-7qx7j Stopping container istio-proxy 41m Warning Unhealthy pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-cr5jb Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning ProbeError pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-b9x4s Startup probe error: Get "http://10.133.0.91:15021/healthz/ready": dial tcp 10.133.0.91:15021: connect: connection refused... 41m Warning ProbeError pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-cr5jb Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-b9x4s Startup probe failed: Get "http://10.133.0.91:15021/healthz/ready": dial tcp 10.133.0.91:15021: connect: connection refused 41m Warning ProbeError pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-tg5g7 Startup probe error: Get "http://10.132.0.172:15021/healthz/ready": dial tcp 10.132.0.172:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-tg5g7 Startup probe failed: Get "http://10.132.0.172:15021/healthz/ready": dial tcp 10.132.0.172:15021: connect: connection refused 41m Warning Unhealthy pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-44frr Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning ProbeError pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb Startup probe error: Get "http://10.134.0.110:15021/healthz/ready": dial tcp 10.134.0.110:15021: connect: connection refused... 41m Warning ProbeError pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-44frr Readiness probe error: HTTP probe failed with statuscode: 503... 41m Normal Killing pod/e2e-derive-9d7aa682-openshift-default-74bdd985b5-sv4td Stopping container istio-proxy 41m Warning Unhealthy pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb Startup probe failed: Get "http://10.134.0.110:15021/healthz/ready": dial tcp 10.134.0.110:15021: connect: connection refused 41m Normal AddedLabel gateway/e2e-shared-a-w4-de5715 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-de5715 41m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7 Created pod: e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z 41m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-de5715-openshift-default Scaled up replica set e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7 from 0 to 1 41m Warning Unhealthy pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-7bggr Readiness probe failed: Get "http://10.132.0.128:15021/healthz/ready": dial tcp 10.132.0.128:15021: connect: connection refused 41m Warning Unhealthy pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr Readiness probe failed: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused 41m Warning ProbeError pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-qfxxr Readiness probe error: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused... 41m Normal AddedInterface pod/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z Add eth0 [10.134.0.111/23] from ovn-kubernetes 41m Warning ProbeError pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-7bggr Readiness probe error: Get "http://10.132.0.128:15021/healthz/ready": dial tcp 10.132.0.128:15021: connect: connection refused... 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-4a1480-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-4a1480-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w0-ea1be0 Scaled up replica set payload-processing-e2e-shared-a-w0-ea1be0-74784f4fdd from 0 to 1 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w0-ea1be0 Scaled up replica set payload-pre-processing-e2e-shared-a-w0-ea1be0-59585bc9f9 from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w0-ea1be0-74784f4fdd Created pod: payload-processing-e2e-shared-a-w0-ea1be0-74784f4fdd-5g9xp 41m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w0-ea1be0-74784f4fdd Error creating: pods "payload-processing-e2e-shared-a-w0-ea1be0-74784f4fdd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-ea1be0: serviceaccount "payload-processing-e2e-shared-a-w0-ea1be0" not found 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w0-ea1be0-59585bc9f9 Created pod: payload-pre-processing-e2e-shared-a-w0-ea1be0-59585bc9f9-766gw 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w0-ea1be0-59585bc9f9 Error creating: pods "payload-pre-processing-e2e-shared-a-w0-ea1be0-59585bc9f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-ea1be0: serviceaccount "payload-processing-e2e-shared-a-w0-ea1be0" not found 41m Normal Created pod/payload-processing-e2e-shared-a-w0-ea1be0-74784f4fdd-5g9xp Container created 41m Normal Created pod/payload-pre-processing-e2e-shared-a-w0-ea1be0-59585bc9f9-766gw Container created 41m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w0-ea1be0-59585bc9f9-766gw Add eth0 [10.132.0.182/23] from ovn-kubernetes 41m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w0-ea1be0-59585bc9f9-766gw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/payload-processing-e2e-shared-a-w0-ea1be0-74784f4fdd-5g9xp Add eth0 [10.132.0.181/23] from ovn-kubernetes 41m Normal Started pod/payload-processing-e2e-shared-a-w0-ea1be0-74784f4fdd-5g9xp Container started 41m Normal Pulled pod/payload-processing-e2e-shared-a-w0-ea1be0-74784f4fdd-5g9xp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Started pod/payload-pre-processing-e2e-shared-a-w0-ea1be0-59585bc9f9-766gw Container started 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w4-de5715 Scaled up replica set payload-processing-e2e-shared-a-w4-de5715-7c6c757c54 from 0 to 1 41m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w4-de5715-7c6c757c54 Error creating: pods "payload-processing-e2e-shared-a-w4-de5715-7c6c757c54-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-de5715: serviceaccount "payload-processing-e2e-shared-a-w4-de5715" not found 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w4-de5715 Scaled up replica set payload-pre-processing-e2e-shared-a-w4-de5715-89cc48587 from 0 to 1 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w4-de5715-89cc48587 Error creating: pods "payload-pre-processing-e2e-shared-a-w4-de5715-89cc48587-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-de5715: serviceaccount "payload-processing-e2e-shared-a-w4-de5715" not found 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w4-de5715-89cc48587 Created pod: payload-pre-processing-e2e-shared-a-w4-de5715-89cc48587-2nf7c 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w4-de5715-7c6c757c54 Created pod: payload-processing-e2e-shared-a-w4-de5715-7c6c757c54-cr9p8 41m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w4-de5715-89cc48587-2nf7c Add eth0 [10.132.0.183/23] from ovn-kubernetes 41m Normal AddedInterface pod/payload-processing-e2e-shared-a-w4-de5715-7c6c757c54-cr9p8 Add eth0 [10.132.0.184/23] from ovn-kubernetes 41m Normal Pulled pod/payload-processing-e2e-shared-a-w4-de5715-7c6c757c54-cr9p8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Started pod/payload-processing-e2e-shared-a-w4-de5715-7c6c757c54-cr9p8 Container started 41m Normal Started pod/payload-pre-processing-e2e-shared-a-w4-de5715-89cc48587-2nf7c Container started 41m Normal Created pod/payload-processing-e2e-shared-a-w4-de5715-7c6c757c54-cr9p8 Container created 41m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w4-de5715-89cc48587-2nf7c Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-pre-processing-e2e-shared-a-w4-de5715-89cc48587-2nf7c Container created 41m Warning ProbeError pod/e2e-shared-a-w3-eb3c33-openshift-default-8687c6c447-x7w6q Readiness probe error: Get "http://10.132.0.174:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Warning Unhealthy pod/e2e-shared-a-w3-eb3c33-openshift-default-8687c6c447-x7w6q Readiness probe failed: Get "http://10.132.0.174:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Normal Created pod/e2e-shared-a-w3-eb3c33-openshift-default-8687c6c447-x7w6q Container created 41m Normal Pulled pod/e2e-shared-a-w3-eb3c33-openshift-default-8687c6c447-x7w6q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Started pod/e2e-shared-a-w3-eb3c33-openshift-default-8687c6c447-x7w6q Container started 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-ebffae22-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal Pulled pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Started pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r Container started 41m Normal Created pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r Container created 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-ebffae22-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning ProbeError pod/e2e-shared-a-w3-eb3c33-openshift-default-8687c6c447-x7w6q Startup probe error: Get "http://10.132.0.174:15021/healthz/ready": dial tcp 10.132.0.174:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-shared-a-w3-eb3c33-openshift-default-8687c6c447-x7w6q Startup probe failed: Get "http://10.132.0.174:15021/healthz/ready": dial tcp 10.132.0.174:15021: connect: connection refused 41m Warning Unhealthy pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r Startup probe failed: Get "http://10.134.0.104:15021/healthz/ready": dial tcp 10.134.0.104:15021: connect: connection refused 41m Warning ProbeError pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r Startup probe error: Get "http://10.134.0.104:15021/healthz/ready": dial tcp 10.134.0.104:15021: connect: connection refused... 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-eb3c33-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-eb3c33-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-a9ab23c4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal ScalingReplicaSet deployment/e2e-worker-w6-7540da-openshift-default Scaled down replica set e2e-worker-w6-7540da-openshift-default-bb989bcf9 from 10 to 6 40m Normal SuccessfulDelete replicaset/e2e-worker-w6-7540da-openshift-default-bb989bcf9 Deleted pod: e2e-worker-w6-7540da-openshift-default-bb989bcf9-28w9g 40m Normal Killing pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-nc2kj Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w6-7540da-openshift-default-bb989bcf9 Deleted pod: e2e-worker-w6-7540da-openshift-default-bb989bcf9-nc2kj 40m Normal Killing pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-28w9g Stopping container istio-proxy 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-a9ab23c4-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal SuccessfulDelete replicaset/e2e-worker-w6-7540da-openshift-default-bb989bcf9 Deleted pod: e2e-worker-w6-7540da-openshift-default-bb989bcf9-2bdcq 40m Normal SuccessfulDelete replicaset/e2e-worker-w6-7540da-openshift-default-bb989bcf9 Deleted pod: e2e-worker-w6-7540da-openshift-default-bb989bcf9-54fm8 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-7540da-openshift-default New size: 6; reason: All metrics below target 40m Normal Killing pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-54fm8 Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2bdcq Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w2-164c36-openshift-default-54b556b4c9 Deleted pod: e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx 40m Normal SuccessfulDelete replicaset/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 Deleted pod: e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-6jvmm 40m Normal SuccessfulDelete replicaset/e2e-worker-w2-164c36-openshift-default-54b556b4c9 Deleted pod: e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl 40m Normal SuccessfulDelete replicaset/e2e-worker-w2-164c36-openshift-default-54b556b4c9 Deleted pod: e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2 40m Normal SuccessfulDelete replicaset/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c Deleted pod: e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp 40m Normal SuccessfulDelete replicaset/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c Deleted pod: e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz 40m Normal Killing pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt Stopping container istio-proxy 40m Normal ScalingReplicaSet deployment/e2e-worker-w5-cbd26c-openshift-default Scaled down replica set e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 from 10 to 6 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-cbd26c-openshift-default New size: 6; reason: All metrics below target 40m Normal ScalingReplicaSet deployment/e2e-worker-w2-164c36-openshift-default Scaled down replica set e2e-worker-w2-164c36-openshift-default-54b556b4c9 from 10 to 6 40m Normal SuccessfulDelete replicaset/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 Deleted pod: e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-g6qdv 40m Normal SuccessfulDelete replicaset/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 Deleted pod: e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-qk9xk 40m Normal SuccessfulDelete replicaset/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 Deleted pod: e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9p25h 40m Normal Killing pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c Deleted pod: e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-164c36-openshift-default New size: 6; reason: All metrics below target 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-1d31f8-openshift-default New size: 7; reason: All metrics below target 40m Normal Killing pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-qk9xk Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-29wxx Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9p25h Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-6jvmm Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-g6qdv Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w2-164c36-openshift-default-54b556b4c9 Deleted pod: e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt 40m Normal ScalingReplicaSet deployment/e2e-worker-w3-1d31f8-openshift-default Scaled down replica set e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c from 10 to 7 40m Normal Killing pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-kfvp2 Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-f9hkr Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-8m8kp Stopping container istio-proxy 40m Warning Unhealthy pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-nc2kj Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning ProbeError pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-n2hcl Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning ProbeError pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-qk9xk Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-qk9xk Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning ProbeError pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-xxlzt Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning ProbeError pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-nc2kj Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning ProbeError pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-54fm8 Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-54fm8 Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning ProbeError pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9p25h Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning Unhealthy pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9p25h Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning ProbeError pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-d55lz Readiness probe error: HTTP probe failed with statuscode: 503... 40m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff Created pod: e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs 40m Normal AddedLabel gateway/e2e-shared-b-w3-433db8 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-433db8 40m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-433db8-openshift-default Scaled up replica set e2e-shared-b-w3-433db8-openshift-default-58ff6846ff from 0 to 1 40m Normal AddedInterface pod/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs Add eth0 [10.134.0.113/23] from ovn-kubernetes 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-eb3c33-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-eb3c33-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w3-433db8 Scaled up replica set payload-processing-e2e-shared-b-w3-433db8-685d97b9d8 from 0 to 1 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59 Created pod: payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk 40m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w3-433db8-685d97b9d8 Error creating: pods "payload-processing-e2e-shared-b-w3-433db8-685d97b9d8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-433db8: serviceaccount "payload-processing-e2e-shared-b-w3-433db8" not found 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w3-433db8-685d97b9d8 Created pod: payload-processing-e2e-shared-b-w3-433db8-685d97b9d8-b8cj9 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59 Error creating: pods "payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-433db8: serviceaccount "payload-processing-e2e-shared-b-w3-433db8" not found 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w3-433db8 Scaled up replica set payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59 from 0 to 1 40m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk Add eth0 [10.134.0.115/23] from ovn-kubernetes 40m Normal Created pod/payload-processing-e2e-shared-b-w3-433db8-685d97b9d8-b8cj9 Container created 40m Normal AddedInterface pod/payload-processing-e2e-shared-b-w3-433db8-685d97b9d8-b8cj9 Add eth0 [10.132.0.186/23] from ovn-kubernetes 40m Normal Pulled pod/payload-processing-e2e-shared-b-w3-433db8-685d97b9d8-b8cj9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Started pod/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk Container started 40m Normal Started pod/payload-processing-e2e-shared-b-w3-433db8-685d97b9d8-b8cj9 Container started 40m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Created pod/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk Container created 40m Normal ScalingReplicaSet deployment/e2e-worker-w3-1d31f8-openshift-default Scaled down replica set e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c from 7 to 6 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-1d31f8-openshift-default New size: 6; reason: All metrics below target 40m Normal SuccessfulDelete replicaset/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c Deleted pod: e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-nprdv 40m Normal Killing pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-nprdv Stopping container istio-proxy 40m Warning ProbeError pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-nprdv Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-nprdv Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning ProbeError pod/e2e-shared-b-w2-4a1480-openshift-default-75c6f8849c-vw8pq Readiness probe error: Get "http://10.132.0.179:15021/healthz/ready": dial tcp 10.132.0.179:15021: connect: connection refused... 40m Normal Pulled pod/e2e-shared-a-w0-ea1be0-openshift-default-6bfb8dcf95-wl6kg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Started pod/e2e-shared-b-w2-4a1480-openshift-default-75c6f8849c-vw8pq Container started 40m Normal Created pod/e2e-shared-a-w0-ea1be0-openshift-default-6bfb8dcf95-wl6kg Container created 40m Normal Created pod/e2e-shared-b-w2-4a1480-openshift-default-75c6f8849c-vw8pq Container created 40m Normal Pulled pod/e2e-shared-b-w2-4a1480-openshift-default-75c6f8849c-vw8pq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Started pod/e2e-shared-a-w0-ea1be0-openshift-default-6bfb8dcf95-wl6kg Container started 40m Warning Unhealthy pod/e2e-shared-b-w2-4a1480-openshift-default-75c6f8849c-vw8pq Readiness probe failed: Get "http://10.132.0.179:15021/healthz/ready": dial tcp 10.132.0.179:15021: connect: connection refused 39m Warning ProbeError pod/e2e-shared-a-w0-ea1be0-openshift-default-6bfb8dcf95-wl6kg Startup probe error: Get "http://10.132.0.180:15021/healthz/ready": dial tcp 10.132.0.180:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-shared-a-w0-ea1be0-openshift-default-6bfb8dcf95-wl6kg Startup probe failed: Get "http://10.132.0.180:15021/healthz/ready": dial tcp 10.132.0.180:15021: connect: connection refused 39m Warning ProbeError pod/e2e-shared-b-w2-4a1480-openshift-default-75c6f8849c-vw8pq Startup probe error: Get "http://10.132.0.179:15021/healthz/ready": dial tcp 10.132.0.179:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-shared-b-w2-4a1480-openshift-default-75c6f8849c-vw8pq Startup probe failed: Get "http://10.132.0.179:15021/healthz/ready": dial tcp 10.132.0.179:15021: connect: connection refused 39m Normal SuccessfulCreate replicaset/e2e-api-a-efce83-openshift-default-5b595bb589 Created pod: e2e-api-a-efce83-openshift-default-5b595bb589-8j7df 39m Normal AddedLabel gateway/e2e-api-a-efce83 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-efce83 39m Normal ScalingReplicaSet deployment/e2e-api-a-efce83-openshift-default Scaled up replica set e2e-api-a-efce83-openshift-default-5b595bb589 from 0 to 1 39m Normal AddedInterface pod/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df Add eth0 [10.134.0.116/23] from ovn-kubernetes 39m Warning FailedCreate replicaset/payload-processing-e2e-api-a-efce83-84867f4d46 Error creating: pods "payload-processing-e2e-api-a-efce83-84867f4d46-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-efce83: serviceaccount "payload-processing-e2e-api-a-efce83" not found 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-efce83 Scaled up replica set payload-processing-e2e-api-a-efce83-84867f4d46 from 0 to 1 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-efce83 Scaled up replica set payload-pre-processing-e2e-api-a-efce83-5c4b5c8f8c from 0 to 1 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-efce83-5c4b5c8f8c Created pod: payload-pre-processing-e2e-api-a-efce83-5c4b5c8f8c-xpbhs 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-efce83-5c4b5c8f8c Error creating: pods "payload-pre-processing-e2e-api-a-efce83-5c4b5c8f8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-efce83: serviceaccount "payload-processing-e2e-api-a-efce83" not found 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-efce83-84867f4d46 Created pod: payload-processing-e2e-api-a-efce83-84867f4d46-24l2f 39m Normal Started pod/payload-processing-e2e-api-a-efce83-84867f4d46-24l2f Container started 39m Normal Created pod/payload-pre-processing-e2e-api-a-efce83-5c4b5c8f8c-xpbhs Container created 39m Normal Started pod/payload-pre-processing-e2e-api-a-efce83-5c4b5c8f8c-xpbhs Container started 39m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-efce83-5c4b5c8f8c-xpbhs Add eth0 [10.132.0.188/23] from ovn-kubernetes 39m Normal AddedInterface pod/payload-processing-e2e-api-a-efce83-84867f4d46-24l2f Add eth0 [10.132.0.189/23] from ovn-kubernetes 39m Normal Pulled pod/payload-processing-e2e-api-a-efce83-84867f4d46-24l2f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Pulled pod/payload-pre-processing-e2e-api-a-efce83-5c4b5c8f8c-xpbhs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Created pod/payload-processing-e2e-api-a-efce83-84867f4d46-24l2f Container created 39m Normal Killing pod/payload-processing-e2e-mt-ebffae22-675bf6d65d-f4wm4 Stopping container payload-processing 39m Normal Killing pod/payload-pre-processing-e2e-mt-ebffae22-76f58b75cf-kmqd4 Stopping container payload-pre-processing 39m Normal SuccessfulCreate replicaset/e2e-api-a-26444f-openshift-default-695897496f Created pod: e2e-api-a-26444f-openshift-default-695897496f-mcdxk 39m Normal AddedLabel gateway/e2e-api-a-26444f Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-26444f 39m Normal ScalingReplicaSet deployment/e2e-api-a-26444f-openshift-default Scaled up replica set e2e-api-a-26444f-openshift-default-695897496f from 0 to 1 39m Normal AddedInterface pod/e2e-api-a-26444f-openshift-default-695897496f-mcdxk Add eth0 [10.133.0.93/23] from ovn-kubernetes 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-ea1be0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-ea1be0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-b64e2e-openshift-default New size: 5; reason: All metrics below target 39m Normal ScalingReplicaSet deployment/e2e-worker-w0-b64e2e-openshift-default Scaled down replica set e2e-worker-w0-b64e2e-openshift-default-64c944c45c from 6 to 5 39m Normal SuccessfulDelete replicaset/e2e-worker-w0-b64e2e-openshift-default-64c944c45c Deleted pod: e2e-worker-w0-b64e2e-openshift-default-64c944c45c-nlgd9 39m Normal Killing pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-nlgd9 Stopping container istio-proxy 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-26444f-67b5fb89f Error creating: pods "payload-pre-processing-e2e-api-a-26444f-67b5fb89f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-26444f: serviceaccount "payload-processing-e2e-api-a-26444f" not found 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-26444f Scaled up replica set payload-pre-processing-e2e-api-a-26444f-67b5fb89f from 0 to 1 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-26444f-67b5fb89f Created pod: payload-pre-processing-e2e-api-a-26444f-67b5fb89f-jsmbq 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-26444f Scaled up replica set payload-processing-e2e-api-a-26444f-7d9bd9f4d6 from 0 to 1 39m Warning FailedCreate replicaset/payload-processing-e2e-api-a-26444f-7d9bd9f4d6 Error creating: pods "payload-processing-e2e-api-a-26444f-7d9bd9f4d6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-26444f: serviceaccount "payload-processing-e2e-api-a-26444f" not found 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-26444f-7d9bd9f4d6 Created pod: payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks 39m Normal Started pod/payload-pre-processing-e2e-api-a-26444f-67b5fb89f-jsmbq Container started 39m Normal Pulled pod/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Warning ProbeError pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-nlgd9 Readiness probe error: HTTP probe failed with statuscode: 503... 39m Normal Started pod/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks Container started 39m Warning Unhealthy pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-nlgd9 Readiness probe failed: HTTP probe failed with statuscode: 503 39m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-26444f-67b5fb89f-jsmbq Add eth0 [10.132.0.190/23] from ovn-kubernetes 39m Normal Pulled pod/payload-pre-processing-e2e-api-a-26444f-67b5fb89f-jsmbq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Created pod/payload-pre-processing-e2e-api-a-26444f-67b5fb89f-jsmbq Container created 39m Normal AddedInterface pod/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks Add eth0 [10.134.0.117/23] from ovn-kubernetes 39m Normal Created pod/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks Container created 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-4a1480-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-4a1480-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning Unhealthy pod/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z Readiness probe failed: Get "http://10.134.0.111:15021/healthz/ready": read tcp 10.134.0.2:43426->10.134.0.111:15021: read: connection reset by peer 39m Warning ProbeError pod/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z Readiness probe error: Get "http://10.134.0.111:15021/healthz/ready": read tcp 10.134.0.2:43426->10.134.0.111:15021: read: connection reset by peer... 39m Normal Pulled pod/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z Container created 39m Normal Started pod/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z Container started 39m Normal SuccessfulDelete replicaset/e2e-worker-w4-32449d-openshift-default-756d4bb7c7 Deleted pod: e2e-worker-w4-32449d-openshift-default-756d4bb7c7-wb2g5 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-32449d-openshift-default New size: 5; reason: All metrics below target 39m Warning Unhealthy pod/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z Startup probe failed: Get "http://10.134.0.111:15021/healthz/ready": dial tcp 10.134.0.111:15021: connect: connection refused 39m Normal ScalingReplicaSet deployment/e2e-worker-w4-32449d-openshift-default Scaled down replica set e2e-worker-w4-32449d-openshift-default-756d4bb7c7 from 6 to 5 39m Normal Killing pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-wb2g5 Stopping container istio-proxy 39m Warning ProbeError pod/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z Startup probe error: Get "http://10.134.0.111:15021/healthz/ready": dial tcp 10.134.0.111:15021: connect: connection refused... 39m Warning ProbeError pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-wb2g5 Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-wb2g5 Readiness probe failed: HTTP probe failed with statuscode: 503 39m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7f5667f86c from 10 to 6 39m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-vj28x Stopping container istio-proxy 39m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7f5667f86c (combined from similar events): Deleted pod: maas-default-gateway-openshift-default-7f5667f86c-nqmgk 39m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: All metrics below target 39m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-6jrgz Stopping container istio-proxy 39m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-nqmgk Stopping container istio-proxy 39m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-lxr98 Stopping container istio-proxy 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-de5715-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-vj28x Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-de5715-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-vj28x Readiness probe failed: HTTP probe failed with statuscode: 503 39m Normal AddedLabel gateway/e2e-shared-b-w0-0406a8 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-0406a8 39m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-0406a8-openshift-default-689d85c855 Created pod: e2e-shared-b-w0-0406a8-openshift-default-689d85c855-l64hj 39m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-0406a8-openshift-default Scaled up replica set e2e-shared-b-w0-0406a8-openshift-default-689d85c855 from 0 to 1 39m Normal AddedInterface pod/e2e-shared-b-w0-0406a8-openshift-default-689d85c855-l64hj Add eth0 [10.133.0.95/23] from ovn-kubernetes 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w0-0406a8-648f9b947 Error creating: pods "payload-pre-processing-e2e-shared-b-w0-0406a8-648f9b947-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8: serviceaccount "payload-processing-e2e-shared-b-w0-0406a8" not found 38m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b Error creating: pods "payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-0406a8: serviceaccount "payload-processing-e2e-shared-b-w0-0406a8" not found 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b Created pod: payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w0-0406a8 Scaled up replica set payload-pre-processing-e2e-shared-b-w0-0406a8-648f9b947 from 0 to 1 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w0-0406a8-648f9b947 Created pod: payload-pre-processing-e2e-shared-b-w0-0406a8-648f9b947-9vn4j 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w0-0406a8 Scaled up replica set payload-processing-e2e-shared-b-w0-0406a8-67c994f59b from 0 to 1 38m Normal Pulled pod/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr Add eth0 [10.134.0.118/23] from ovn-kubernetes 38m Normal Created pod/payload-pre-processing-e2e-shared-b-w0-0406a8-648f9b947-9vn4j Container created 38m Normal Started pod/payload-pre-processing-e2e-shared-b-w0-0406a8-648f9b947-9vn4j Container started 38m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w0-0406a8-648f9b947-9vn4j Add eth0 [10.132.0.191/23] from ovn-kubernetes 38m Normal Started pod/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr Container started 38m Normal Created pod/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr Container created 38m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w0-0406a8-648f9b947-9vn4j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c Created pod: e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-lc9v2 38m Normal AddedLabel gateway/e2e-shared-b-w4-aacab9 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-aacab9 38m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-aacab9-openshift-default Scaled up replica set e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c from 0 to 1 38m Normal AddedInterface pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-lc9v2 Add eth0 [10.133.0.96/23] from ovn-kubernetes 38m Normal Killing pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-tg5g7 Stopping container istio-proxy 38m Normal Killing pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb Stopping container istio-proxy 38m Normal Killing pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-b9x4s Stopping container istio-proxy 38m Normal ScalingReplicaSet deployment/e2e-inf-a-a91d2d-openshift-default Scaled up replica set e2e-inf-a-a91d2d-openshift-default-6b9b889877 from 0 to 1 38m Normal SuccessfulCreate replicaset/e2e-inf-a-a91d2d-openshift-default-6b9b889877 Created pod: e2e-inf-a-a91d2d-openshift-default-6b9b889877-szkd2 38m Normal AddedLabel gateway/e2e-inf-a-a91d2d Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-a91d2d 38m Warning ProbeError pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-b9x4s Readiness probe error: HTTP probe failed with statuscode: 503... 38m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w4-aacab9-864f985548 Error creating: pods "payload-processing-e2e-shared-b-w4-aacab9-864f985548-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9: serviceaccount "payload-processing-e2e-shared-b-w4-aacab9" not found 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w4-aacab9 Scaled up replica set payload-processing-e2e-shared-b-w4-aacab9-864f985548 from 0 to 1 38m Normal AddedInterface pod/e2e-inf-a-a91d2d-openshift-default-6b9b889877-szkd2 Add eth0 [10.133.0.97/23] from ovn-kubernetes 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w4-aacab9 Scaled up replica set payload-pre-processing-e2e-shared-b-w4-aacab9-864895467 from 0 to 1 38m Warning Unhealthy pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-b9x4s Readiness probe failed: HTTP probe failed with statuscode: 503 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467 Created pod: payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467 Error creating: pods "payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-aacab9: serviceaccount "payload-processing-e2e-shared-b-w4-aacab9" not found 38m Warning ProbeError pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-tg5g7 Readiness probe error: HTTP probe failed with statuscode: 503... 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w4-aacab9-864f985548 Created pod: payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq 38m Warning Unhealthy pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-tg5g7 Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning Unhealthy pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb Readiness probe failed: HTTP probe failed with statuscode: 503 38m Normal AddedInterface pod/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq Add eth0 [10.134.0.120/23] from ovn-kubernetes 38m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn Add eth0 [10.134.0.119/23] from ovn-kubernetes 38m Normal Created pod/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn Container created 38m Warning ProbeError pod/e2e-mt-ebffae22-openshift-default-78c8d7888f-254mb Readiness probe error: HTTP probe failed with statuscode: 503... 38m Normal Started pod/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq Container started 38m Normal Created pod/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq Container created 38m Normal Pulled pod/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Started pod/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn Container started 38m Warning ProbeError pod/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs Readiness probe error: Get "http://10.134.0.113:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning Unhealthy pod/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs Readiness probe failed: Get "http://10.134.0.113:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Normal Pulled pod/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Started pod/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs Container started 38m Normal Created pod/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs Container created 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-de5715-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-de5715-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-a91d2d Scaled up replica set payload-processing-e2e-inf-a-a91d2d-7b5d5fbfbb from 0 to 1 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-a91d2d-c8f444867 Error creating: pods "payload-pre-processing-e2e-inf-a-a91d2d-c8f444867-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-a91d2d: serviceaccount "payload-processing-e2e-inf-a-a91d2d" not found 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-a91d2d Scaled up replica set payload-pre-processing-e2e-inf-a-a91d2d-c8f444867 from 0 to 1 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-a91d2d-7b5d5fbfbb Created pod: payload-processing-e2e-inf-a-a91d2d-7b5d5fbfbb-k5gzl 38m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-a91d2d-7b5d5fbfbb Error creating: pods "payload-processing-e2e-inf-a-a91d2d-7b5d5fbfbb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-a91d2d: serviceaccount "payload-processing-e2e-inf-a-a91d2d" not found 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-a91d2d-c8f444867 Created pod: payload-pre-processing-e2e-inf-a-a91d2d-c8f444867-j8llw 38m Normal AddedInterface pod/payload-processing-e2e-inf-a-a91d2d-7b5d5fbfbb-k5gzl Add eth0 [10.132.0.192/23] from ovn-kubernetes 38m Normal Pulled pod/payload-pre-processing-e2e-inf-a-a91d2d-c8f444867-j8llw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-a91d2d-c8f444867-j8llw Add eth0 [10.132.0.193/23] from ovn-kubernetes 38m Normal Started pod/payload-processing-e2e-inf-a-a91d2d-7b5d5fbfbb-k5gzl Container started 38m Normal Created pod/payload-processing-e2e-inf-a-a91d2d-7b5d5fbfbb-k5gzl Container created 38m Normal Pulled pod/payload-processing-e2e-inf-a-a91d2d-7b5d5fbfbb-k5gzl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Started pod/payload-pre-processing-e2e-inf-a-a91d2d-c8f444867-j8llw Container started 38m Normal Created pod/payload-pre-processing-e2e-inf-a-a91d2d-c8f444867-j8llw Container created 38m Warning Unhealthy pod/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs Startup probe failed: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused 38m Warning ProbeError pod/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs Startup probe error: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused... 38m Normal AddedInterface pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-5wnpt Add eth0 [10.133.0.99/23] from ovn-kubernetes 38m Normal ScalingReplicaSet deployment/e2e-mt-a9ab23c4-openshift-default Scaled up replica set e2e-mt-a9ab23c4-openshift-default-766c495bb9 from 1 to 2 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-a9ab23c4-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 38m Normal SuccessfulCreate replicaset/e2e-mt-a9ab23c4-openshift-default-766c495bb9 Created pod: e2e-mt-a9ab23c4-openshift-default-766c495bb9-5wnpt 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-433db8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-433db8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Normal Started pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-5wnpt Container started 38m Normal Created pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-5wnpt Container created 38m Normal Pulled pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-5wnpt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Warning ProbeError pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-5wnpt Startup probe error: Get "http://10.133.0.99:15021/healthz/ready": dial tcp 10.133.0.99:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-5wnpt Startup probe failed: Get "http://10.133.0.99:15021/healthz/ready": dial tcp 10.133.0.99:15021: connect: connection refused 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-433db8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-433db8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Normal Started pod/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df Container started 37m Normal Pulled pod/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Created pod/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df Container created 37m Warning Unhealthy pod/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df Startup probe failed: Get "http://10.134.0.116:15021/healthz/ready": dial tcp 10.134.0.116:15021: connect: connection refused 37m Warning ProbeError pod/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df Startup probe error: Get "http://10.134.0.116:15021/healthz/ready": dial tcp 10.134.0.116:15021: connect: connection refused... 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-efce83-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-efce83-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Normal SuccessfulDelete replicaset/e2e-worker-w4-32449d-openshift-default-756d4bb7c7 Deleted pod: e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-32449d-openshift-default New size: 4; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/e2e-worker-w4-32449d-openshift-default Scaled down replica set e2e-worker-w4-32449d-openshift-default-756d4bb7c7 from 5 to 4 37m Normal Killing pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq Stopping container istio-proxy 37m Normal Killing pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-tww6f Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87 Deleted pod: e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-tww6f 37m Normal ScalingReplicaSet deployment/e2e-worker-w1-7afb1c-openshift-default Scaled down replica set e2e-worker-w1-7afb1c-openshift-default-6d7b749b87 from 4 to 3 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-7afb1c-openshift-default New size: 3; reason: All metrics below target 37m Warning Unhealthy pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-5qztq Readiness probe error: HTTP probe failed with statuscode: 503... 37m Normal Killing pod/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-d9mnb Stopping container payload-pre-processing 37m Normal Killing pod/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-tsfqs Stopping container payload-processing 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-efce83-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-efce83-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning Unhealthy pod/e2e-api-a-26444f-openshift-default-695897496f-mcdxk Readiness probe failed: Get "http://10.133.0.93:15021/healthz/ready": dial tcp 10.133.0.93:15021: connect: connection refused 37m Normal Pulled pod/e2e-api-a-26444f-openshift-default-695897496f-mcdxk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Warning ProbeError pod/e2e-api-a-26444f-openshift-default-695897496f-mcdxk Readiness probe error: Get "http://10.133.0.93:15021/healthz/ready": dial tcp 10.133.0.93:15021: connect: connection refused... 37m Normal Started pod/e2e-api-a-26444f-openshift-default-695897496f-mcdxk Container started 37m Normal Created pod/e2e-api-a-26444f-openshift-default-695897496f-mcdxk Container created 37m Warning Unhealthy pod/e2e-api-a-26444f-openshift-default-695897496f-mcdxk Startup probe failed: Get "http://10.133.0.93:15021/healthz/ready": dial tcp 10.133.0.93:15021: connect: connection refused 37m Warning ProbeError pod/e2e-api-a-26444f-openshift-default-695897496f-mcdxk Startup probe error: Get "http://10.133.0.93:15021/healthz/ready": dial tcp 10.133.0.93:15021: connect: connection refused... 36m Normal Pulled pod/e2e-shared-b-w0-0406a8-openshift-default-689d85c855-l64hj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-0406a8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-0406a8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal Started pod/e2e-shared-b-w0-0406a8-openshift-default-689d85c855-l64hj Container started 36m Normal Created pod/e2e-shared-b-w0-0406a8-openshift-default-689d85c855-l64hj Container created 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-26444f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-26444f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning Unhealthy pod/e2e-shared-b-w0-0406a8-openshift-default-689d85c855-l64hj Startup probe failed: Get "http://10.133.0.95:15021/healthz/ready": dial tcp 10.133.0.95:15021: connect: connection refused 36m Warning ProbeError pod/e2e-shared-b-w0-0406a8-openshift-default-689d85c855-l64hj Startup probe error: Get "http://10.133.0.95:15021/healthz/ready": dial tcp 10.133.0.95:15021: connect: connection refused... 36m Normal ScalingReplicaSet deployment/e2e-worker-w0-b64e2e-openshift-default Scaled down replica set e2e-worker-w0-b64e2e-openshift-default-64c944c45c from 5 to 4 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-b64e2e-openshift-default New size: 4; reason: All metrics below target 36m Normal SuccessfulDelete replicaset/e2e-worker-w0-b64e2e-openshift-default-64c944c45c Deleted pod: e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg 36m Normal Killing pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg Stopping container istio-proxy 36m Warning ProbeError pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning Unhealthy pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-jf8qg Readiness probe failed: HTTP probe failed with statuscode: 503 36m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7f5667f86c from 6 to 10 36m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-qk9bp Container started 36m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-njmdj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-8q2rz Container started 36m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-njmdj Add eth0 [10.133.0.109/23] from ovn-kubernetes 36m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-qk9bp Container created 36m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-njmdj Container created 36m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-8q2rz Add eth0 [10.133.0.108/23] from ovn-kubernetes 36m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-njmdj Container started 36m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-8q2rz Container created 36m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-8q2rz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-qk9bp Add eth0 [10.134.0.123/23] from ovn-kubernetes 36m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-qk9bp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-kpmpx Container started 36m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-kpmpx Container created 36m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-kpmpx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-kpmpx Add eth0 [10.133.0.107/23] from ovn-kubernetes 36m Normal Killing pod/payload-pre-processing-e2e-api-a-efce83-5c4b5c8f8c-xpbhs Stopping container payload-pre-processing 36m Normal Killing pod/payload-processing-e2e-api-a-efce83-84867f4d46-24l2f Stopping container payload-processing 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-0406a8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-0406a8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-qk9bp Startup probe failed: Get "http://10.134.0.123:15021/healthz/ready": dial tcp 10.134.0.123:15021: connect: connection refused 36m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-qk9bp Startup probe error: Get "http://10.134.0.123:15021/healthz/ready": dial tcp 10.134.0.123:15021: connect: connection refused... 36m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-kpmpx Startup probe error: Get "http://10.133.0.107:15021/healthz/ready": dial tcp 10.133.0.107:15021: connect: connection refused... 36m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-kpmpx Startup probe failed: Get "http://10.133.0.107:15021/healthz/ready": dial tcp 10.133.0.107:15021: connect: connection refused 36m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-njmdj Startup probe error: Get "http://10.133.0.109:15021/healthz/ready": dial tcp 10.133.0.109:15021: connect: connection refused... 36m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-njmdj Startup probe failed: Get "http://10.133.0.109:15021/healthz/ready": dial tcp 10.133.0.109:15021: connect: connection refused 36m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-8q2rz Startup probe error: Get "http://10.133.0.108:15021/healthz/ready": dial tcp 10.133.0.108:15021: connect: connection refused... 36m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-8q2rz Startup probe failed: Get "http://10.133.0.108:15021/healthz/ready": dial tcp 10.133.0.108:15021: connect: connection refused 36m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-kpmpx Startup probe failed: Get "http://10.133.0.107:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 36m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-kpmpx Startup probe error: Get "http://10.133.0.107:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 36m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-njmdj Startup probe failed: Get "http://10.133.0.109:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 36m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-njmdj Startup probe error: Get "http://10.133.0.109:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 36m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c Created pod: e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-zktwm 36m Normal Killing pod/payload-pre-processing-e2e-api-a-26444f-67b5fb89f-jsmbq Stopping container payload-pre-processing 36m Normal Killing pod/payload-processing-e2e-api-a-26444f-7d9bd9f4d6-ztbks Stopping container payload-processing 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-aacab9-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 36m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c Created pod: e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb 36m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c Created pod: e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-7xpv9 36m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-aacab9-openshift-default Scaled up replica set e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c from 1 to 4 36m Normal AddedInterface pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-zktwm Add eth0 [10.132.0.199/23] from ovn-kubernetes 36m Normal Pulled pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-zktwm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-zktwm Container started 36m Normal Pulled pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-7xpv9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Created pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-7xpv9 Container created 36m Normal Started pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-7xpv9 Container started 36m Normal Pulled pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Created pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb Container created 36m Normal AddedInterface pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-7xpv9 Add eth0 [10.132.0.200/23] from ovn-kubernetes 36m Warning ProbeError pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-lc9v2 Readiness probe error: Get "http://10.133.0.96:15021/healthz/ready": read tcp 10.133.0.2:51210->10.133.0.96:15021: read: connection reset by peer... 36m Warning Unhealthy pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-lc9v2 Readiness probe failed: Get "http://10.133.0.96:15021/healthz/ready": read tcp 10.133.0.2:51210->10.133.0.96:15021: read: connection reset by peer 36m Normal Started pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb Container started 36m Normal AddedInterface pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb Add eth0 [10.134.0.124/23] from ovn-kubernetes 36m Normal Created pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-zktwm Container created 36m Normal Started pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-lc9v2 Container started 36m Normal Created pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-lc9v2 Container created 36m Normal Started pod/e2e-inf-a-a91d2d-openshift-default-6b9b889877-szkd2 Container started 36m Normal Created pod/e2e-inf-a-a91d2d-openshift-default-6b9b889877-szkd2 Container created 36m Normal Pulled pod/e2e-inf-a-a91d2d-openshift-default-6b9b889877-szkd2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Pulled pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-lc9v2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Warning ProbeError pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-zktwm Startup probe error: Get "http://10.132.0.199:15021/healthz/ready": dial tcp 10.132.0.199:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-zktwm Startup probe failed: Get "http://10.132.0.199:15021/healthz/ready": dial tcp 10.132.0.199:15021: connect: connection refused 36m Warning ProbeError pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb Startup probe error: Get "http://10.134.0.124:15021/healthz/ready": dial tcp 10.134.0.124:15021: connect: connection refused... 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-32449d-openshift-default New size: 3; reason: All metrics below target 36m Normal ScalingReplicaSet deployment/e2e-worker-w4-32449d-openshift-default Scaled down replica set e2e-worker-w4-32449d-openshift-default-756d4bb7c7 from 4 to 3 36m Warning Unhealthy pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb Startup probe failed: Get "http://10.134.0.124:15021/healthz/ready": dial tcp 10.134.0.124:15021: connect: connection refused 36m Normal SuccessfulDelete replicaset/e2e-worker-w4-32449d-openshift-default-756d4bb7c7 Deleted pod: e2e-worker-w4-32449d-openshift-default-756d4bb7c7-shfnk 36m Warning ProbeError pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-7xpv9 Startup probe error: Get "http://10.132.0.200:15021/healthz/ready": dial tcp 10.132.0.200:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-7xpv9 Startup probe failed: Get "http://10.132.0.200:15021/healthz/ready": dial tcp 10.132.0.200:15021: connect: connection refused 36m Normal Killing pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-shfnk Stopping container istio-proxy 36m Warning ProbeError pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-lc9v2 Startup probe error: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused... 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-7540da-openshift-default New size: 4; reason: All metrics below target 36m Normal ScalingReplicaSet deployment/e2e-worker-w6-7540da-openshift-default Scaled down replica set e2e-worker-w6-7540da-openshift-default-bb989bcf9 from 6 to 4 36m Normal Killing pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-5tnzh Stopping container istio-proxy 36m Warning Unhealthy pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-lc9v2 Startup probe failed: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused 36m Normal Killing pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x Stopping container istio-proxy 36m Normal Killing pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5 Stopping container istio-proxy 36m Warning Unhealthy pod/e2e-inf-a-a91d2d-openshift-default-6b9b889877-szkd2 Startup probe failed: Get "http://10.133.0.97:15021/healthz/ready": dial tcp 10.133.0.97:15021: connect: connection refused 36m Warning ProbeError pod/e2e-inf-a-a91d2d-openshift-default-6b9b889877-szkd2 Startup probe error: Get "http://10.133.0.97:15021/healthz/ready": dial tcp 10.133.0.97:15021: connect: connection refused... 36m Normal Killing pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-klsq2 Stopping container istio-proxy 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-164c36-openshift-default New size: 4; reason: All metrics below target 36m Normal ScalingReplicaSet deployment/e2e-worker-w2-164c36-openshift-default Scaled down replica set e2e-worker-w2-164c36-openshift-default-54b556b4c9 from 6 to 4 36m Normal SuccessfulDelete replicaset/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 Deleted pod: e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-5tnzh 36m Normal SuccessfulDelete replicaset/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 Deleted pod: e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5 36m Normal Killing pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-wlh8v Stopping container istio-proxy 36m Normal SuccessfulDelete replicaset/e2e-worker-w6-7540da-openshift-default-bb989bcf9 Deleted pod: e2e-worker-w6-7540da-openshift-default-bb989bcf9-hsn96 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-cbd26c-openshift-default New size: 4; reason: All metrics below target 36m Normal ScalingReplicaSet deployment/e2e-worker-w5-cbd26c-openshift-default Scaled down replica set e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 from 6 to 4 36m Normal SuccessfulDelete replicaset/e2e-worker-w6-7540da-openshift-default-bb989bcf9 Deleted pod: e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x 36m Normal SuccessfulDelete replicaset/e2e-worker-w2-164c36-openshift-default-54b556b4c9 Deleted pod: e2e-worker-w2-164c36-openshift-default-54b556b4c9-wlh8v 36m Normal SuccessfulDelete replicaset/e2e-worker-w2-164c36-openshift-default-54b556b4c9 Deleted pod: e2e-worker-w2-164c36-openshift-default-54b556b4c9-klsq2 36m Normal Killing pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-hsn96 Stopping container istio-proxy 36m Normal SuccessfulDelete replicaset/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c Deleted pod: e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-w6k2c 36m Warning Unhealthy pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-wlh8v Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning ProbeError pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-wlh8v Readiness probe error: HTTP probe failed with statuscode: 503... 36m Normal Killing pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-w6k2c Stopping container istio-proxy 36m Normal SuccessfulDelete replicaset/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c Deleted pod: e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-cb25n 36m Warning Unhealthy pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-klsq2 Readiness probe failed: HTTP probe failed with statuscode: 503 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-1d31f8-openshift-default New size: 4; reason: All metrics below target 36m Warning ProbeError pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-klsq2 Readiness probe error: HTTP probe failed with statuscode: 503... 36m Normal Killing pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-cb25n Stopping container istio-proxy 36m Warning Unhealthy pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning Unhealthy pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-cb25n Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning ProbeError pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-cb25n Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning ProbeError pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-qj45x Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning ProbeError pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-shfnk Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning Unhealthy pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-w6k2c Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning Unhealthy pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5 Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning ProbeError pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-w6k2c Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning ProbeError pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-bmfs5 Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning Unhealthy pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-shfnk Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning ProbeError pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-hsn96 Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning Unhealthy pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-hsn96 Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-a91d2d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-a91d2d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Normal Killing pod/payload-pre-processing-e2e-shared-a-w0-ea1be0-59585bc9f9-766gw Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-shared-a-w0-ea1be0-74784f4fdd-5g9xp Stopping container payload-processing 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-aacab9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-aacab9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-a91d2d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-a91d2d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Normal Killing pod/payload-pre-processing-e2e-inf-a-a91d2d-c8f444867-j8llw Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-inf-a-a91d2d-7b5d5fbfbb-k5gzl Stopping container payload-processing 35m Normal Killing pod/payload-pre-processing-e2e-shared-a-w4-de5715-89cc48587-2nf7c Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-shared-a-w4-de5715-7c6c757c54-cr9p8 Stopping container payload-processing 35m Normal Killing pod/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df Stopping container istio-proxy 35m Normal ScalingReplicaSet deployment/e2e-inf-a-efd39a-openshift-default Scaled up replica set e2e-inf-a-efd39a-openshift-default-5985657f6c from 0 to 1 35m Normal AddedLabel gateway/e2e-inf-a-efd39a Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-efd39a 35m Normal SuccessfulCreate replicaset/e2e-inf-a-efd39a-openshift-default-5985657f6c Created pod: e2e-inf-a-efd39a-openshift-default-5985657f6c-fp6r2 35m Normal AddedInterface pod/e2e-inf-a-efd39a-openshift-default-5985657f6c-fp6r2 Add eth0 [10.132.0.205/23] from ovn-kubernetes 35m Warning FailedCreate replicaset/payload-processing-e2e-mt-a9ab23c4-845f6c8c86 Error creating: pods "payload-processing-e2e-mt-a9ab23c4-845f6c8c86-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-a9ab23c4: serviceaccount "payload-processing-e2e-mt-a9ab23c4" not found 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-a9ab23c4 Scaled up replica set payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9 from 0 to 1 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-a9ab23c4 Scaled up replica set payload-processing-e2e-mt-a9ab23c4-845f6c8c86 from 0 to 1 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-a9ab23c4-845f6c8c86 Created pod: payload-processing-e2e-mt-a9ab23c4-845f6c8c86-7f2l6 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9 Error creating: pods "payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-a9ab23c4: serviceaccount "payload-processing-e2e-mt-a9ab23c4" not found 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9 Created pod: payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-s4t6f 35m Normal Created pod/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-s4t6f Container created 35m Normal Pulled pod/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-7f2l6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-s4t6f Add eth0 [10.132.0.206/23] from ovn-kubernetes 35m Normal Pulled pod/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-s4t6f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Started pod/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-s4t6f Container started 35m Normal Started pod/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-7f2l6 Container started 35m Normal Created pod/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-7f2l6 Container created 35m Normal AddedInterface pod/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-7f2l6 Add eth0 [10.132.0.207/23] from ovn-kubernetes 35m Warning ProbeError pod/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df Readiness probe error: Get "http://10.134.0.116:15021/healthz/ready": dial tcp 10.134.0.116:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-api-a-efce83-openshift-default-5b595bb589-8j7df Readiness probe failed: Get "http://10.134.0.116:15021/healthz/ready": dial tcp 10.134.0.116:15021: connect: connection refused 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-efd39a Scaled up replica set payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7 from 0 to 1 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7 Error creating: pods "payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-efd39a: serviceaccount "payload-processing-e2e-inf-a-efd39a" not found 34m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-efd39a-5c4cc85976 Error creating: pods "payload-processing-e2e-inf-a-efd39a-5c4cc85976-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-efd39a: serviceaccount "payload-processing-e2e-inf-a-efd39a" not found 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-efd39a Scaled up replica set payload-processing-e2e-inf-a-efd39a-5c4cc85976 from 0 to 1 34m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp Add eth0 [10.134.0.125/23] from ovn-kubernetes 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-efd39a-5c4cc85976 Created pod: payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7 Created pod: payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp 34m Normal Started pod/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg Container started 34m Normal Started pod/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp Container started 34m Normal Pulled pod/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Pulled pod/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg Add eth0 [10.134.0.126/23] from ovn-kubernetes 34m Normal Created pod/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg Container created 34m Normal Created pod/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp Container created 34m Normal Killing pod/e2e-api-a-26444f-openshift-default-695897496f-mcdxk Stopping container istio-proxy 34m Normal ScalingReplicaSet deployment/e2e-inf-a-f8cd0c-openshift-default Scaled up replica set e2e-inf-a-f8cd0c-openshift-default-554f94596c from 0 to 1 34m Normal AddedLabel gateway/e2e-inf-a-f8cd0c Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-f8cd0c 34m Normal AddedInterface pod/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5 Add eth0 [10.134.0.127/23] from ovn-kubernetes 34m Normal SuccessfulCreate replicaset/e2e-inf-a-f8cd0c-openshift-default-554f94596c Created pod: e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5 34m Normal Killing pod/e2e-shared-a-w0-ea1be0-openshift-default-6bfb8dcf95-wl6kg Stopping container istio-proxy 34m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-f8cd0c-6b7c848d84 Error creating: pods "payload-processing-e2e-inf-a-f8cd0c-6b7c848d84-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-f8cd0c: serviceaccount "payload-processing-e2e-inf-a-f8cd0c" not found 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c Created pod: payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c Error creating: pods "payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-f8cd0c: serviceaccount "payload-processing-e2e-inf-a-f8cd0c" not found 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-f8cd0c Scaled up replica set payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c from 0 to 1 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-f8cd0c-6b7c848d84 Created pod: payload-processing-e2e-inf-a-f8cd0c-6b7c848d84-j265n 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-f8cd0c Scaled up replica set payload-processing-e2e-inf-a-f8cd0c-6b7c848d84 from 0 to 1 34m Normal Created pod/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55 Container created 34m Normal Pulled pod/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Started pod/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55 Container started 34m Normal Started pod/payload-processing-e2e-inf-a-f8cd0c-6b7c848d84-j265n Container started 34m Normal Created pod/payload-processing-e2e-inf-a-f8cd0c-6b7c848d84-j265n Container created 34m Normal Pulled pod/payload-processing-e2e-inf-a-f8cd0c-6b7c848d84-j265n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/payload-processing-e2e-inf-a-f8cd0c-6b7c848d84-j265n Add eth0 [10.132.0.208/23] from ovn-kubernetes 34m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55 Add eth0 [10.134.0.128/23] from ovn-kubernetes 34m Normal Killing pod/payload-processing-e2e-shared-b-w0-0406a8-67c994f59b-9cmhr Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-shared-b-w0-0406a8-648f9b947-9vn4j Stopping container payload-pre-processing 34m Normal Killing pod/e2e-shared-a-w4-de5715-openshift-default-d69cdcfc7-p8v7z Stopping container istio-proxy 34m Normal Killing pod/payload-pre-processing-e2e-shared-b-w4-aacab9-864895467-zqffn Stopping container payload-pre-processing 34m Normal Killing pod/payload-processing-e2e-shared-b-w4-aacab9-864f985548-mdwdq Stopping container payload-processing 33m Normal AddedLabel gateway/e2e-inf-a-924fba Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-924fba 33m Normal Killing pod/e2e-inf-a-a91d2d-openshift-default-6b9b889877-szkd2 Stopping container istio-proxy 33m Normal SuccessfulCreate replicaset/e2e-inf-a-924fba-openshift-default-dd49687c5 Created pod: e2e-inf-a-924fba-openshift-default-dd49687c5-phg8v 33m Normal ScalingReplicaSet deployment/e2e-inf-a-924fba-openshift-default Scaled up replica set e2e-inf-a-924fba-openshift-default-dd49687c5 from 0 to 1 33m Normal AddedInterface pod/e2e-inf-a-924fba-openshift-default-dd49687c5-phg8v Add eth0 [10.133.0.110/23] from ovn-kubernetes 33m Warning ProbeError pod/e2e-inf-a-a91d2d-openshift-default-6b9b889877-szkd2 Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-inf-a-a91d2d-openshift-default-6b9b889877-szkd2 Readiness probe failed: HTTP probe failed with statuscode: 503 33m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-924fba Scaled up replica set payload-processing-e2e-inf-a-924fba-998c6c468 from 0 to 1 33m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c Error creating: pods "payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-924fba: serviceaccount "payload-processing-e2e-inf-a-924fba" not found 33m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-924fba-998c6c468 Error creating: pods "payload-processing-e2e-inf-a-924fba-998c6c468-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-924fba: serviceaccount "payload-processing-e2e-inf-a-924fba" not found 33m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-924fba Scaled up replica set payload-pre-processing-e2e-inf-a-924fba-5b6499b74c from 0 to 1 33m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c Created pod: payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7 33m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-924fba-998c6c468 Created pod: payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6 33m Normal Created pod/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6 Container created 33m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7 Add eth0 [10.134.0.130/23] from ovn-kubernetes 33m Normal AddedInterface pod/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6 Add eth0 [10.134.0.131/23] from ovn-kubernetes 33m Normal Pulled pod/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal Started pod/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6 Container started 33m Normal Created pod/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7 Container created 33m Normal Pulled pod/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal Started pod/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7 Container started 33m Normal Killing pod/payload-pre-processing-e2e-worker-w5-cbd26c-6fb5db446c-hk5gg Stopping container payload-pre-processing 33m Normal Killing pod/payload-processing-e2e-worker-w5-cbd26c-845d9688fb-gvkf8 Stopping container payload-processing 33m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-vsltn Stopping container discovery 33m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 33m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 33m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-vsltn 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-924fba-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-924fba-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Normal SuccessfulDelete replicaset/e2e-worker-w4-32449d-openshift-default-756d4bb7c7 Deleted pod: e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-b64e2e-openshift-default New size: 2; reason: All metrics below target 33m Normal SuccessfulDelete replicaset/e2e-worker-w0-b64e2e-openshift-default-64c944c45c Deleted pod: e2e-worker-w0-b64e2e-openshift-default-64c944c45c-2hmfk 33m Normal SuccessfulDelete replicaset/e2e-worker-w0-b64e2e-openshift-default-64c944c45c Deleted pod: e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf 33m Normal Killing pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql Stopping container istio-proxy 33m Normal Killing pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf Stopping container istio-proxy 33m Normal Killing pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-2hmfk Stopping container istio-proxy 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-32449d-openshift-default New size: 2; reason: All metrics below target 33m Normal Killing pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-xfkst Stopping container istio-proxy 33m Normal Killing pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-q95hh Stopping container istio-proxy 33m Normal Killing pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q Stopping container istio-proxy 33m Normal ScalingReplicaSet deployment/e2e-worker-w1-7afb1c-openshift-default Scaled down replica set e2e-worker-w1-7afb1c-openshift-default-6d7b749b87 from 3 to 2 33m Normal Killing pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-8jkmq Stopping container istio-proxy 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-7540da-openshift-default New size: 3; reason: All metrics below target 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-7afb1c-openshift-default New size: 2; reason: All metrics below target 33m Normal ScalingReplicaSet deployment/e2e-worker-w2-164c36-openshift-default Scaled down replica set e2e-worker-w2-164c36-openshift-default-54b556b4c9 from 4 to 3 33m Normal SuccessfulDelete replicaset/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87 Deleted pod: e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-vvn46 33m Normal SuccessfulDelete replicaset/e2e-worker-w6-7540da-openshift-default-bb989bcf9 Deleted pod: e2e-worker-w6-7540da-openshift-default-bb989bcf9-8jkmq 33m Normal SuccessfulDelete replicaset/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c Deleted pod: e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-xfkst 33m Normal ScalingReplicaSet deployment/e2e-worker-w6-7540da-openshift-default Scaled down replica set e2e-worker-w6-7540da-openshift-default-bb989bcf9 from 4 to 3 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-164c36-openshift-default New size: 3; reason: All metrics below target 33m Normal Killing pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-vvn46 Stopping container istio-proxy 33m Normal SuccessfulDelete replicaset/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 Deleted pod: e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q 33m Normal SuccessfulDelete replicaset/e2e-worker-w2-164c36-openshift-default-54b556b4c9 Deleted pod: e2e-worker-w2-164c36-openshift-default-54b556b4c9-q95hh 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-cbd26c-openshift-default New size: 3; reason: All metrics below target 33m Normal ScalingReplicaSet deployment/e2e-worker-w5-cbd26c-openshift-default Scaled down replica set e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 from 4 to 3 33m Warning ProbeError pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-vvn46 Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-vvn46 Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-2hmfk Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning ProbeError pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-2hmfk Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-wx7wf Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning Unhealthy pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-4d8ql Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning Unhealthy pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-7d64q Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning ProbeError pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-xfkst Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-xfkst Readiness probe failed: HTTP probe failed with statuscode: 503 33m Normal SuccessfulCreate replicaset/e2e-mt-a9ab23c4-openshift-default-766c495bb9 Created pod: e2e-mt-a9ab23c4-openshift-default-766c495bb9-sn5r8 33m Normal ScalingReplicaSet deployment/e2e-mt-a9ab23c4-openshift-default Scaled up replica set e2e-mt-a9ab23c4-openshift-default-766c495bb9 from 2 to 4 33m Normal SuccessfulCreate replicaset/e2e-mt-a9ab23c4-openshift-default-766c495bb9 Created pod: e2e-mt-a9ab23c4-openshift-default-766c495bb9-kc75t 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-a9ab23c4-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 33m Normal SuccessfulDelete replicaset/e2e-worker-w2-164c36-openshift-default-54b556b4c9 Deleted pod: e2e-worker-w2-164c36-openshift-default-54b556b4c9-rsqdc 33m Normal AddedInterface pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-sn5r8 Add eth0 [10.132.0.214/23] from ovn-kubernetes 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-cbd26c-openshift-default New size: 2; reason: All metrics below target 33m Normal ScalingReplicaSet deployment/e2e-worker-w5-cbd26c-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 from 3 to 2 33m Normal Created pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-sn5r8 Container created 33m Normal ScalingReplicaSet deployment/e2e-worker-w2-164c36-openshift-default Scaled down replica set e2e-worker-w2-164c36-openshift-default-54b556b4c9 from 3 to 2 33m Normal Killing pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-59hxl Stopping container istio-proxy 33m Normal Killing pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-rsqdc Stopping container istio-proxy 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-164c36-openshift-default New size: 2; reason: All metrics below target 33m Normal Started pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-sn5r8 Container started 33m Normal SuccessfulDelete replicaset/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75 Deleted pod: e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-59hxl 33m Normal Pulled pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-sn5r8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-kc75t Add eth0 [10.133.0.113/23] from ovn-kubernetes 33m Normal Pulled pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-kc75t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-kc75t Container created 33m Normal Started pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-kc75t Container started 33m Warning ProbeError pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-59hxl Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-59hxl Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-kc75t Startup probe error: Get "http://10.133.0.113:15021/healthz/ready": dial tcp 10.133.0.113:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-kc75t Startup probe failed: Get "http://10.133.0.113:15021/healthz/ready": dial tcp 10.133.0.113:15021: connect: connection refused 33m Normal Killing pod/e2e-shared-b-w0-0406a8-openshift-default-689d85c855-l64hj Stopping container istio-proxy 33m Warning ProbeError pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-sn5r8 Startup probe error: Get "http://10.132.0.214:15021/healthz/ready": dial tcp 10.132.0.214:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-sn5r8 Startup probe failed: Get "http://10.132.0.214:15021/healthz/ready": dial tcp 10.132.0.214:15021: connect: connection refused 33m Normal AddedLabel gateway/e2e-api-a-ea5c8f Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-ea5c8f 32m Normal SuccessfulCreate replicaset/e2e-api-a-ea5c8f-openshift-default-54496b9b7c Created pod: e2e-api-a-ea5c8f-openshift-default-54496b9b7c-d2vcm 32m Normal ScalingReplicaSet deployment/e2e-api-a-ea5c8f-openshift-default Scaled up replica set e2e-api-a-ea5c8f-openshift-default-54496b9b7c from 0 to 1 32m Warning ProbeError pod/e2e-shared-b-w0-0406a8-openshift-default-689d85c855-l64hj Readiness probe error: HTTP probe failed with statuscode: 503... 32m Normal AddedInterface pod/e2e-api-a-ea5c8f-openshift-default-54496b9b7c-d2vcm Add eth0 [10.133.0.114/23] from ovn-kubernetes 32m Warning Unhealthy pod/e2e-shared-b-w0-0406a8-openshift-default-689d85c855-l64hj Readiness probe failed: HTTP probe failed with statuscode: 503 32m Normal SuccessfulCreate replicaset/e2e-mt-a9ab23c4-openshift-default-766c495bb9 Created pod: e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp 32m Normal SuccessfulCreate replicaset/e2e-mt-a9ab23c4-openshift-default-766c495bb9 Created pod: e2e-mt-a9ab23c4-openshift-default-766c495bb9-hjhmd 32m Normal SuccessfulCreate replicaset/e2e-mt-a9ab23c4-openshift-default-766c495bb9 Created pod: e2e-mt-a9ab23c4-openshift-default-766c495bb9-27kc9 32m Normal SuccessfulCreate replicaset/e2e-mt-a9ab23c4-openshift-default-766c495bb9 Created pod: e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-a9ab23c4-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 32m Normal ScalingReplicaSet deployment/e2e-mt-a9ab23c4-openshift-default Scaled up replica set e2e-mt-a9ab23c4-openshift-default-766c495bb9 from 4 to 8 32m Normal Pulled pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Started pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp Container started 32m Normal AddedInterface pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-27kc9 Add eth0 [10.133.0.115/23] from ovn-kubernetes 32m Normal Created pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-hjhmd Container created 32m Normal Started pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx Container started 32m Normal Created pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx Container created 32m Normal AddedInterface pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx Add eth0 [10.134.0.133/23] from ovn-kubernetes 32m Normal AddedInterface pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-hjhmd Add eth0 [10.132.0.215/23] from ovn-kubernetes 32m Normal Created pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp Container created 32m Normal Pulled pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Pulled pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-hjhmd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp Add eth0 [10.134.0.132/23] from ovn-kubernetes 32m Normal Pulled pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-27kc9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Started pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-hjhmd Container started 32m Normal Created pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-27kc9 Container created 32m Normal Started pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-27kc9 Container started 32m Normal Started pod/e2e-inf-a-efd39a-openshift-default-5985657f6c-fp6r2 Container started 32m Normal Created pod/e2e-inf-a-efd39a-openshift-default-5985657f6c-fp6r2 Container created 32m Normal Pulled pod/e2e-inf-a-efd39a-openshift-default-5985657f6c-fp6r2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-ea5c8f Scaled up replica set payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd from 0 to 1 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-ea5c8f Scaled up replica set payload-pre-processing-e2e-api-a-ea5c8f-7db7d5b55f from 0 to 1 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-ea5c8f-7db7d5b55f Created pod: payload-pre-processing-e2e-api-a-ea5c8f-7db7d5b55f-qnbc8 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-ea5c8f-7db7d5b55f Error creating: pods "payload-pre-processing-e2e-api-a-ea5c8f-7db7d5b55f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-ea5c8f: serviceaccount "payload-processing-e2e-api-a-ea5c8f" not found 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd Created pod: payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw 32m Warning FailedCreate replicaset/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd Error creating: pods "payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-ea5c8f: serviceaccount "payload-processing-e2e-api-a-ea5c8f" not found 32m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-ea5c8f-7db7d5b55f-qnbc8 Add eth0 [10.133.0.116/23] from ovn-kubernetes 32m Normal Killing pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-zktwm Stopping container istio-proxy 32m Normal Killing pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-24ggb Stopping container istio-proxy 32m Normal Pulling pod/payload-pre-processing-e2e-api-a-ea5c8f-7db7d5b55f-qnbc8 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 32m Normal AddedInterface pod/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw Add eth0 [10.134.0.134/23] from ovn-kubernetes 32m Normal Killing pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-lc9v2 Stopping container istio-proxy 32m Normal Killing pod/e2e-shared-b-w4-aacab9-openshift-default-7c7c84964c-7xpv9 Stopping container istio-proxy 32m Normal Started pod/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw Container started 32m Warning Unhealthy pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-27kc9 Startup probe failed: Get "http://10.133.0.115:15021/healthz/ready": dial tcp 10.133.0.115:15021: connect: connection refused 32m Normal Pulled pod/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal AddedLabel gateway/e2e-api-a-1048fc Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-1048fc 32m Warning ProbeError pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-27kc9 Startup probe error: Get "http://10.133.0.115:15021/healthz/ready": dial tcp 10.133.0.115:15021: connect: connection refused... 32m Normal SuccessfulCreate replicaset/e2e-api-a-1048fc-openshift-default-887fc8d4f Created pod: e2e-api-a-1048fc-openshift-default-887fc8d4f-lq7gz 32m Normal ScalingReplicaSet deployment/e2e-api-a-1048fc-openshift-default Scaled up replica set e2e-api-a-1048fc-openshift-default-887fc8d4f from 0 to 1 32m Normal Created pod/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw Container created 32m Warning ProbeError pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-hjhmd Startup probe error: Get "http://10.132.0.215:15021/healthz/ready": dial tcp 10.132.0.215:15021: connect: connection refused... 32m Warning ProbeError pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx Startup probe error: Get "http://10.134.0.133:15021/healthz/ready": dial tcp 10.134.0.133:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp Startup probe failed: Get "http://10.134.0.132:15021/healthz/ready": dial tcp 10.134.0.132:15021: connect: connection refused 32m Normal AddedInterface pod/e2e-api-a-1048fc-openshift-default-887fc8d4f-lq7gz Add eth0 [10.133.0.117/23] from ovn-kubernetes 32m Warning Unhealthy pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-hjhmd Startup probe failed: Get "http://10.132.0.215:15021/healthz/ready": dial tcp 10.132.0.215:15021: connect: connection refused 32m Warning ProbeError pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp Startup probe error: Get "http://10.134.0.132:15021/healthz/ready": dial tcp 10.134.0.132:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx Startup probe failed: Get "http://10.134.0.133:15021/healthz/ready": dial tcp 10.134.0.133:15021: connect: connection refused 32m Warning Unhealthy pod/e2e-inf-a-efd39a-openshift-default-5985657f6c-fp6r2 Startup probe failed: Get "http://10.132.0.205:15021/healthz/ready": dial tcp 10.132.0.205:15021: connect: connection refused 32m Warning ProbeError pod/e2e-inf-a-efd39a-openshift-default-5985657f6c-fp6r2 Startup probe error: Get "http://10.132.0.205:15021/healthz/ready": dial tcp 10.132.0.205:15021: connect: connection refused... 32m Normal Created pod/payload-pre-processing-e2e-api-a-ea5c8f-7db7d5b55f-qnbc8 Container created 32m Normal Started pod/payload-pre-processing-e2e-api-a-ea5c8f-7db7d5b55f-qnbc8 Container started 32m Normal Pulled pod/payload-pre-processing-e2e-api-a-ea5c8f-7db7d5b55f-qnbc8 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.488s (3.488s including waiting). Image size: 125353963 bytes. 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-a9ab23c4-openshift-default New size: 10; reason: 32m Normal SuccessfulCreate replicaset/e2e-mt-a9ab23c4-openshift-default-766c495bb9 Created pod: e2e-mt-a9ab23c4-openshift-default-766c495bb9-8jb2h 32m Normal ScalingReplicaSet deployment/e2e-mt-a9ab23c4-openshift-default Scaled up replica set e2e-mt-a9ab23c4-openshift-default-766c495bb9 from 8 to 10 32m Normal AddedInterface pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-8jb2h Add eth0 [10.132.0.216/23] from ovn-kubernetes 32m Normal Started pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-lj7c6 Container started 32m Normal Pulled pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-8jb2h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-8jb2h Container created 32m Normal Pulled pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-lj7c6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-lj7c6 Container created 32m Normal AddedInterface pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-lj7c6 Add eth0 [10.133.0.118/23] from ovn-kubernetes 32m Normal SuccessfulCreate replicaset/e2e-mt-a9ab23c4-openshift-default-766c495bb9 (combined from similar events): Created pod: e2e-mt-a9ab23c4-openshift-default-766c495bb9-lj7c6 32m Normal Started pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-8jb2h Container started 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-efd39a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-efd39a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning Unhealthy pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-8jb2h Startup probe failed: Get "http://10.132.0.216:15021/healthz/ready": dial tcp 10.132.0.216:15021: connect: connection refused 32m Warning ProbeError pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-8jb2h Startup probe error: Get "http://10.132.0.216:15021/healthz/ready": dial tcp 10.132.0.216:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-lj7c6 Startup probe failed: Get "http://10.133.0.118:15021/healthz/ready": dial tcp 10.133.0.118:15021: connect: connection refused 32m Warning ProbeError pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-lj7c6 Startup probe error: Get "http://10.133.0.118:15021/healthz/ready": dial tcp 10.133.0.118:15021: connect: connection refused... 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-1048fc-567986d79 Created pod: payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-1048fc-567986d79 Error creating: pods "payload-pre-processing-e2e-api-a-1048fc-567986d79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-1048fc: serviceaccount "payload-processing-e2e-api-a-1048fc" not found 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-1048fc Scaled up replica set payload-pre-processing-e2e-api-a-1048fc-567986d79 from 0 to 1 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-1048fc Scaled up replica set payload-processing-e2e-api-a-1048fc-dcf578678 from 0 to 1 32m Warning FailedCreate replicaset/payload-processing-e2e-api-a-1048fc-dcf578678 Error creating: pods "payload-processing-e2e-api-a-1048fc-dcf578678-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-1048fc: serviceaccount "payload-processing-e2e-api-a-1048fc" not found 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-1048fc-dcf578678 Created pod: payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4 32m Normal AddedInterface pod/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4 Add eth0 [10.134.0.136/23] from ovn-kubernetes 32m Normal Created pod/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4 Container created 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-ea5c8f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd Add eth0 [10.134.0.135/23] from ovn-kubernetes 32m Normal Pulled pod/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Created pod/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd Container created 32m Normal Started pod/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4 Container started 32m Normal Started pod/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd Container started 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-ea5c8f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Normal Pulled pod/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Killing pod/payload-pre-processing-e2e-mt-a9ab23c4-7f6b8568c9-s4t6f Stopping container payload-pre-processing 32m Normal Killing pod/payload-processing-e2e-mt-a9ab23c4-845f6c8c86-7f2l6 Stopping container payload-processing 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-efd39a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-efd39a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Normal ScalingReplicaSet deployment/e2e-inf-b-e66d59-openshift-default Scaled up replica set e2e-inf-b-e66d59-openshift-default-6b654555 from 0 to 1 32m Normal SuccessfulCreate replicaset/e2e-inf-b-e66d59-openshift-default-6b654555 Created pod: e2e-inf-b-e66d59-openshift-default-6b654555-8jvm4 32m Normal AddedLabel gateway/e2e-inf-b-e66d59 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-e66d59 32m Normal AddedInterface pod/e2e-inf-b-e66d59-openshift-default-6b654555-8jvm4 Add eth0 [10.133.0.121/23] from ovn-kubernetes 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-e66d59 Scaled up replica set payload-processing-e2e-inf-b-e66d59-58ffcfcbd from 0 to 1 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5 Error creating: pods "payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-e66d59: serviceaccount "payload-processing-e2e-inf-b-e66d59" not found 32m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-e66d59-58ffcfcbd Error creating: pods "payload-processing-e2e-inf-b-e66d59-58ffcfcbd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-e66d59: serviceaccount "payload-processing-e2e-inf-b-e66d59" not found 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-e66d59-58ffcfcbd Created pod: payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5 Created pod: payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-e66d59 Scaled up replica set payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5 from 0 to 1 32m Normal Pulled pod/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Started pod/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7 Container started 32m Normal Created pod/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp Container created 32m Normal Started pod/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp Container started 32m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7 Add eth0 [10.134.0.143/23] from ovn-kubernetes 32m Normal Pulled pod/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Created pod/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7 Container created 32m Normal AddedInterface pod/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp Add eth0 [10.134.0.144/23] from ovn-kubernetes 31m Normal SuccessfulCreate replicaset/e2e-inf-b-dcf5a5-openshift-default-68bf4b7597 Created pod: e2e-inf-b-dcf5a5-openshift-default-68bf4b7597-qtcnn 31m Normal AddedLabel gateway/e2e-inf-b-dcf5a5 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-dcf5a5 31m Normal ScalingReplicaSet deployment/e2e-inf-b-dcf5a5-openshift-default Scaled up replica set e2e-inf-b-dcf5a5-openshift-default-68bf4b7597 from 0 to 1 31m Normal AddedInterface pod/e2e-inf-b-dcf5a5-openshift-default-68bf4b7597-qtcnn Add eth0 [10.133.0.122/23] from ovn-kubernetes 31m Normal Pulled pod/e2e-inf-a-924fba-openshift-default-dd49687c5-phg8v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-e66d59-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-e66d59-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning Unhealthy pod/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5 Readiness probe failed: Get "http://10.134.0.127:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 31m Warning ProbeError pod/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5 Readiness probe error: Get "http://10.134.0.127:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 31m Normal Started pod/e2e-inf-a-924fba-openshift-default-dd49687c5-phg8v Container started 31m Normal Created pod/e2e-inf-a-924fba-openshift-default-dd49687c5-phg8v Container created 31m Normal Killing pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz Stopping container istio-proxy 31m Normal Created pod/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5 Container created 31m Normal Started pod/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5 Container started 31m Normal Pulled pod/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Killing pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-8vndl Stopping container istio-proxy 31m Warning Unhealthy pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning ProbeError pod/e2e-worker-w5-cbd26c-openshift-default-586cfdcc75-9pdpz Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning Unhealthy pod/e2e-inf-a-924fba-openshift-default-dd49687c5-phg8v Startup probe failed: Get "http://10.133.0.110:15021/healthz/ready": dial tcp 10.133.0.110:15021: connect: connection refused 31m Warning ProbeError pod/e2e-inf-a-924fba-openshift-default-dd49687c5-phg8v Startup probe error: Get "http://10.133.0.110:15021/healthz/ready": dial tcp 10.133.0.110:15021: connect: connection refused... 31m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-dcf5a5 Scaled up replica set payload-processing-e2e-inf-b-dcf5a5-75f99579d6 from 0 to 1 31m Warning ProbeError pod/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5 Startup probe error: Get "http://10.134.0.127:15021/healthz/ready": dial tcp 10.134.0.127:15021: connect: connection refused... 31m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-dcf5a5 Scaled up replica set payload-pre-processing-e2e-inf-b-dcf5a5-9f4df6cf5 from 0 to 1 31m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-dcf5a5-9f4df6cf5 Created pod: payload-pre-processing-e2e-inf-b-dcf5a5-9f4df6cf5-pt975 31m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-dcf5a5-9f4df6cf5 Error creating: pods "payload-pre-processing-e2e-inf-b-dcf5a5-9f4df6cf5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-dcf5a5: serviceaccount "payload-processing-e2e-inf-b-dcf5a5" not found 31m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-dcf5a5-75f99579d6 Error creating: pods "payload-processing-e2e-inf-b-dcf5a5-75f99579d6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-dcf5a5: serviceaccount "payload-processing-e2e-inf-b-dcf5a5" not found 31m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-dcf5a5-75f99579d6 Created pod: payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5 31m Warning Unhealthy pod/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5 Startup probe failed: Get "http://10.134.0.127:15021/healthz/ready": dial tcp 10.134.0.127:15021: connect: connection refused 31m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-dcf5a5-9f4df6cf5-pt975 Add eth0 [10.132.0.228/23] from ovn-kubernetes 31m Normal AddedInterface pod/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5 Add eth0 [10.134.0.145/23] from ovn-kubernetes 31m Normal Pulled pod/payload-pre-processing-e2e-inf-b-dcf5a5-9f4df6cf5-pt975 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal Started pod/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5 Container started 31m Normal Created pod/payload-pre-processing-e2e-inf-b-dcf5a5-9f4df6cf5-pt975 Container created 31m Normal Created pod/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5 Container created 31m Normal Started pod/payload-pre-processing-e2e-inf-b-dcf5a5-9f4df6cf5-pt975 Container started 31m Normal Pulled pod/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-f8cd0c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-f8cd0c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-7540da-openshift-default New size: 2; reason: All metrics below target 31m Normal Killing pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-pwwqr Stopping container istio-proxy 31m Normal SuccessfulDelete replicaset/e2e-worker-w6-7540da-openshift-default-bb989bcf9 Deleted pod: e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf 31m Normal SuccessfulDelete replicaset/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c Deleted pod: e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-pwwqr 31m Warning ProbeError pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf Readiness probe error: HTTP probe failed with statuscode: 503... 31m Normal Pulled pod/e2e-api-a-ea5c8f-openshift-default-54496b9b7c-d2vcm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Created pod/e2e-api-a-ea5c8f-openshift-default-54496b9b7c-d2vcm Container created 31m Warning Unhealthy pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-kh6bf Readiness probe failed: HTTP probe failed with statuscode: 503 31m Normal Started pod/e2e-api-a-ea5c8f-openshift-default-54496b9b7c-d2vcm Container started 31m Warning ProbeError pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-pwwqr Readiness probe error: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-pwwqr Readiness probe failed: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused 31m Normal SuccessfulCreate replicaset/e2e-inf-b-78be2c-openshift-default-5b668467b7 Created pod: e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq 31m Normal ScalingReplicaSet deployment/e2e-inf-b-78be2c-openshift-default Scaled up replica set e2e-inf-b-78be2c-openshift-default-5b668467b7 from 0 to 1 31m Normal AddedLabel gateway/e2e-inf-b-78be2c Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-78be2c 31m Normal AddedInterface pod/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq Add eth0 [10.134.0.147/23] from ovn-kubernetes 31m Warning ProbeError pod/e2e-api-a-ea5c8f-openshift-default-54496b9b7c-d2vcm Startup probe error: Get "http://10.133.0.114:15021/healthz/ready": dial tcp 10.133.0.114:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-api-a-ea5c8f-openshift-default-54496b9b7c-d2vcm Startup probe failed: Get "http://10.133.0.114:15021/healthz/ready": dial tcp 10.133.0.114:15021: connect: connection refused 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-924fba-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-924fba-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-78be2c-598f68f6f6 Error creating: pods "payload-processing-e2e-inf-b-78be2c-598f68f6f6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-78be2c: serviceaccount "payload-processing-e2e-inf-b-78be2c" not found 30m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-78be2c Scaled up replica set payload-pre-processing-e2e-inf-b-78be2c-69859d788c from 0 to 1 30m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-78be2c-69859d788c Created pod: payload-pre-processing-e2e-inf-b-78be2c-69859d788c-vtdqn 30m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-78be2c Scaled up replica set payload-processing-e2e-inf-b-78be2c-598f68f6f6 from 0 to 1 30m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-78be2c-598f68f6f6 Created pod: payload-processing-e2e-inf-b-78be2c-598f68f6f6-bdszd 30m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-78be2c-69859d788c Error creating: pods "payload-pre-processing-e2e-inf-b-78be2c-69859d788c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-78be2c: serviceaccount "payload-processing-e2e-inf-b-78be2c" not found 30m Normal Pulled pod/payload-pre-processing-e2e-inf-b-78be2c-69859d788c-vtdqn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal Started pod/payload-processing-e2e-inf-b-78be2c-598f68f6f6-bdszd Container started 30m Normal Created pod/payload-processing-e2e-inf-b-78be2c-598f68f6f6-bdszd Container created 30m Normal Pulled pod/payload-processing-e2e-inf-b-78be2c-598f68f6f6-bdszd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-78be2c-69859d788c-vtdqn Add eth0 [10.132.0.229/23] from ovn-kubernetes 30m Normal Started pod/payload-pre-processing-e2e-inf-b-78be2c-69859d788c-vtdqn Container started 30m Normal Created pod/payload-pre-processing-e2e-inf-b-78be2c-69859d788c-vtdqn Container created 30m Normal AddedInterface pod/payload-processing-e2e-inf-b-78be2c-598f68f6f6-bdszd Add eth0 [10.132.0.230/23] from ovn-kubernetes 30m Normal Killing pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-f6g2r Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-hjhmd Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-5wnpt Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-sn5r8 Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-27kc9 Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-kc75t Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-8jb2h Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-lj7c6 Stopping container istio-proxy 30m Warning Unhealthy pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-8jb2h Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning Unhealthy pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-kc75t Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning ProbeError pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-lj7c6 Readiness probe error: HTTP probe failed with statuscode: 503... 30m Normal Started pod/e2e-api-a-1048fc-openshift-default-887fc8d4f-lq7gz Container started 30m Warning Unhealthy pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-lj7c6 Readiness probe failed: HTTP probe failed with statuscode: 503 30m Normal Created pod/e2e-api-a-1048fc-openshift-default-887fc8d4f-lq7gz Container created 30m Warning ProbeError pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-8jb2h Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning ProbeError pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-27kc9 Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning ProbeError pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-kc75t Readiness probe error: HTTP probe failed with statuscode: 503... 30m Normal Pulled pod/e2e-api-a-1048fc-openshift-default-887fc8d4f-lq7gz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Warning Unhealthy pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-27kc9 Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning ProbeError pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-sn5r8 Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-1048fc-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Warning Unhealthy pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-sn5r8 Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning ProbeError pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning ProbeError pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-hjhmd Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning Unhealthy pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning Unhealthy pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-rshnp Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning Unhealthy pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-hjhmd Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning ProbeError pod/e2e-mt-a9ab23c4-openshift-default-766c495bb9-2cwtx Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-1048fc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-ea5c8f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-ea5c8f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Warning Unhealthy pod/e2e-api-a-1048fc-openshift-default-887fc8d4f-lq7gz Startup probe failed: Get "http://10.133.0.117:15021/healthz/ready": dial tcp 10.133.0.117:15021: connect: connection refused 30m Warning ProbeError pod/e2e-api-a-1048fc-openshift-default-887fc8d4f-lq7gz Startup probe error: Get "http://10.133.0.117:15021/healthz/ready": dial tcp 10.133.0.117:15021: connect: connection refused... 30m Warning ProbeError pod/e2e-inf-b-e66d59-openshift-default-6b654555-8jvm4 Readiness probe error: Get "http://10.133.0.121:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 30m Warning Unhealthy pod/e2e-inf-b-e66d59-openshift-default-6b654555-8jvm4 Readiness probe failed: Get "http://10.133.0.121:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 30m Normal Pulled pod/e2e-inf-b-e66d59-openshift-default-6b654555-8jvm4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Created pod/e2e-inf-b-e66d59-openshift-default-6b654555-8jvm4 Container created 30m Normal Started pod/e2e-inf-b-e66d59-openshift-default-6b654555-8jvm4 Container started 30m Warning ProbeError pod/e2e-inf-b-e66d59-openshift-default-6b654555-8jvm4 Startup probe error: Get "http://10.133.0.121:15021/healthz/ready": dial tcp 10.133.0.121:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-inf-b-e66d59-openshift-default-6b654555-8jvm4 Startup probe failed: Get "http://10.133.0.121:15021/healthz/ready": dial tcp 10.133.0.121:15021: connect: connection refused 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-1048fc-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-1048fc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Normal AddedLabel gateway/e2e-api-b-cfe1bd Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-cfe1bd 29m Normal ScalingReplicaSet deployment/e2e-api-b-cfe1bd-openshift-default Scaled up replica set e2e-api-b-cfe1bd-openshift-default-7985f9c557 from 0 to 1 29m Normal SuccessfulCreate replicaset/e2e-api-b-cfe1bd-openshift-default-7985f9c557 Created pod: e2e-api-b-cfe1bd-openshift-default-7985f9c557-5kzht 29m Normal Killing pod/payload-pre-processing-e2e-api-a-ea5c8f-7db7d5b55f-qnbc8 Stopping container payload-pre-processing 29m Normal Killing pod/payload-processing-e2e-api-a-ea5c8f-69fc9cd5fd-6q5dw Stopping container payload-processing 29m Normal AddedInterface pod/e2e-api-b-cfe1bd-openshift-default-7985f9c557-5kzht Add eth0 [10.132.0.232/23] from ovn-kubernetes 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-cfe1bd Scaled up replica set payload-processing-e2e-api-b-cfe1bd-74ffbccdd6 from 0 to 1 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655 Error creating: pods "payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-cfe1bd: serviceaccount "payload-processing-e2e-api-b-cfe1bd" not found 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655 Created pod: payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk 29m Warning FailedCreate replicaset/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6 Error creating: pods "payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-cfe1bd: serviceaccount "payload-processing-e2e-api-b-cfe1bd" not found 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-cfe1bd Scaled up replica set payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655 from 0 to 1 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6 Created pod: payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd 29m Normal Pulled pod/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk Add eth0 [10.134.0.148/23] from ovn-kubernetes 29m Normal Started pod/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk Container started 29m Normal Created pod/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk Container created 29m Normal Started pod/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd Container started 29m Normal AddedInterface pod/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd Add eth0 [10.134.0.149/23] from ovn-kubernetes 29m Normal Pulled pod/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal Created pod/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd Container created 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-e66d59-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-e66d59-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Normal Created pod/e2e-inf-b-dcf5a5-openshift-default-68bf4b7597-qtcnn Container created 29m Normal Pulled pod/e2e-inf-b-dcf5a5-openshift-default-68bf4b7597-qtcnn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Started pod/e2e-inf-b-dcf5a5-openshift-default-68bf4b7597-qtcnn Container started 29m Warning ProbeError pod/e2e-inf-b-dcf5a5-openshift-default-68bf4b7597-qtcnn Startup probe error: Get "http://10.133.0.122:15021/healthz/ready": dial tcp 10.133.0.122:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-inf-b-dcf5a5-openshift-default-68bf4b7597-qtcnn Startup probe failed: Get "http://10.133.0.122:15021/healthz/ready": dial tcp 10.133.0.122:15021: connect: connection refused 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-32449d-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 29m Normal ScalingReplicaSet deployment/e2e-worker-w4-32449d-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w4-32449d-openshift-default-756d4bb7c7 from 2 to 1 29m Normal SuccessfulDelete replicaset/e2e-worker-w4-32449d-openshift-default-756d4bb7c7 Deleted pod: e2e-worker-w4-32449d-openshift-default-756d4bb7c7-sm442 29m Normal Killing pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-sm442 Stopping container istio-proxy 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-dcf5a5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-dcf5a5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Normal Killing pod/payload-pre-processing-e2e-inf-a-efd39a-64b77b4d7-8zqdp Stopping container payload-pre-processing 28m Normal Killing pod/payload-processing-e2e-inf-a-efd39a-5c4cc85976-4dkzg Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-worker-w1-7afb1c-869549494c-272x5 Stopping container payload-pre-processing 28m Normal Killing pod/payload-processing-e2e-worker-w1-7afb1c-5777db7584-dvxk5 Stopping container payload-processing 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-dcf5a5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-dcf5a5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Normal Created pod/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq Container created 28m Normal Pulled pod/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Started pod/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq Container started 28m Normal Killing pod/payload-processing-e2e-inf-a-f8cd0c-6b7c848d84-j265n Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-inf-a-f8cd0c-75f9b8df4c-lbw55 Stopping container payload-pre-processing 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-78be2c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-78be2c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning Unhealthy pod/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq Startup probe failed: Get "http://10.134.0.147:15021/healthz/ready": dial tcp 10.134.0.147:15021: connect: connection refused 28m Warning ProbeError pod/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq Startup probe error: Get "http://10.134.0.147:15021/healthz/ready": dial tcp 10.134.0.147:15021: connect: connection refused... 28m Normal Killing pod/e2e-api-a-ea5c8f-openshift-default-54496b9b7c-d2vcm Stopping container istio-proxy 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-164c36-openshift-default New size: 1; reason: All metrics below target 28m Normal SuccessfulDelete replicaset/e2e-worker-w2-164c36-openshift-default-54b556b4c9 Deleted pod: e2e-worker-w2-164c36-openshift-default-54b556b4c9-ldktd 28m Normal ScalingReplicaSet deployment/e2e-worker-w2-164c36-openshift-default Scaled down replica set e2e-worker-w2-164c36-openshift-default-54b556b4c9 from 2 to 1 28m Normal Killing pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-ldktd Stopping container istio-proxy 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-78be2c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-78be2c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Normal Killing pod/payload-processing-e2e-worker-w0-b64e2e-55674f5fb6-b7sds Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-worker-w0-b64e2e-59488dfb6c-hbtbx Stopping container payload-pre-processing 27m Normal Killing pod/payload-processing-e2e-inf-a-924fba-998c6c468-wgmr6 Stopping container payload-processing 27m Normal Killing pod/payload-pre-processing-e2e-inf-a-924fba-5b6499b74c-vj2w7 Stopping container payload-pre-processing 27m Normal Killing pod/e2e-inf-a-efd39a-openshift-default-5985657f6c-fp6r2 Stopping container istio-proxy 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-7afb1c-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 27m Normal ScalingReplicaSet deployment/e2e-worker-w1-7afb1c-openshift-default Scaled up replica set e2e-worker-w1-7afb1c-openshift-default-6d7b749b87 from 2 to 3 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-b64e2e-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 27m Normal ScalingReplicaSet deployment/e2e-worker-w0-b64e2e-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w0-b64e2e-openshift-default-64c944c45c from 2 to 1 27m Normal Created pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5xrfk Container created 27m Normal Killing pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4 Stopping container istio-proxy 27m Normal SuccessfulDelete replicaset/e2e-worker-w0-b64e2e-openshift-default-64c944c45c Deleted pod: e2e-worker-w0-b64e2e-openshift-default-64c944c45c-mbsp4 27m Normal Pulled pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5xrfk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal SuccessfulCreate replicaset/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87 Created pod: e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5xrfk 27m Normal AddedInterface pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5xrfk Add eth0 [10.133.0.128/23] from ovn-kubernetes 27m Normal Started pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5xrfk Container started 27m Warning Unhealthy pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5xrfk Startup probe failed: Get "http://10.133.0.128:15021/healthz/ready": dial tcp 10.133.0.128:15021: connect: connection refused 27m Warning ProbeError pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5xrfk Startup probe error: Get "http://10.133.0.128:15021/healthz/ready": dial tcp 10.133.0.128:15021: connect: connection refused... 27m Normal Killing pod/payload-pre-processing-e2e-inf-b-e66d59-5c55fdbfc5-qg4z7 Stopping container payload-pre-processing 27m Normal Killing pod/payload-processing-e2e-inf-b-e66d59-58ffcfcbd-s9wqp Stopping container payload-processing 27m Normal Pulled pod/e2e-api-b-cfe1bd-openshift-default-7985f9c557-5kzht Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal Created pod/e2e-api-b-cfe1bd-openshift-default-7985f9c557-5kzht Container created 27m Normal Started pod/e2e-api-b-cfe1bd-openshift-default-7985f9c557-5kzht Container started 27m Normal Killing pod/e2e-inf-a-f8cd0c-openshift-default-554f94596c-gq2n5 Stopping container istio-proxy 27m Warning ProbeError pod/e2e-api-b-cfe1bd-openshift-default-7985f9c557-5kzht Startup probe error: Get "http://10.132.0.232:15021/healthz/ready": dial tcp 10.132.0.232:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-api-b-cfe1bd-openshift-default-7985f9c557-5kzht Startup probe failed: Get "http://10.132.0.232:15021/healthz/ready": dial tcp 10.132.0.232:15021: connect: connection refused 27m Normal Killing pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5xrfk Stopping container istio-proxy 27m Normal Killing pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-g7kbs Stopping container istio-proxy 27m Normal Killing pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs Stopping container istio-proxy 27m Warning ProbeError pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs Readiness probe error: HTTP probe failed with statuscode: 503... 27m Warning Unhealthy pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-5jvcs Readiness probe failed: HTTP probe failed with statuscode: 503 27m Warning Unhealthy pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-g7kbs Readiness probe failed: HTTP probe failed with statuscode: 503 27m Warning ProbeError pod/e2e-worker-w1-7afb1c-openshift-default-6d7b749b87-g7kbs Readiness probe error: HTTP probe failed with statuscode: 503... 27m Normal Killing pod/payload-processing-e2e-inf-b-dcf5a5-75f99579d6-dpbp5 Stopping container payload-processing 27m Normal Killing pod/payload-pre-processing-e2e-inf-b-dcf5a5-9f4df6cf5-pt975 Stopping container payload-pre-processing 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-cfe1bd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-cfe1bd-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Normal Killing pod/e2e-inf-a-924fba-openshift-default-dd49687c5-phg8v Stopping container istio-proxy 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-cfe1bd-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-cfe1bd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Normal Killing pod/payload-pre-processing-e2e-inf-b-78be2c-69859d788c-vtdqn Stopping container payload-pre-processing 26m Normal Killing pod/payload-processing-e2e-inf-b-78be2c-598f68f6f6-bdszd Stopping container payload-processing 26m Normal Killing pod/payload-pre-processing-e2e-api-a-1048fc-567986d79-g8kvd Stopping container payload-pre-processing 26m Normal Killing pod/payload-processing-e2e-api-a-1048fc-dcf578678-cgsz4 Stopping container payload-processing 26m Normal Killing pod/e2e-worker-w0-b64e2e-openshift-default-64c944c45c-h59rg Stopping container istio-proxy 26m Normal Killing pod/e2e-inf-b-e66d59-openshift-default-6b654555-8jvm4 Stopping container istio-proxy 26m Warning Unhealthy pod/e2e-inf-b-e66d59-openshift-default-6b654555-8jvm4 Readiness probe failed: HTTP probe failed with statuscode: 503 26m Warning ProbeError pod/e2e-inf-b-e66d59-openshift-default-6b654555-8jvm4 Readiness probe error: HTTP probe failed with statuscode: 503... 26m Normal SuccessfulDelete replicaset/e2e-worker-w6-7540da-openshift-default-bb989bcf9 Deleted pod: e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp 26m Normal ScalingReplicaSet deployment/e2e-worker-w6-7540da-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w6-7540da-openshift-default-bb989bcf9 from 2 to 1 26m Normal Killing pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-9qxhp Stopping container istio-proxy 26m Normal SuccessfulDelete replicaset/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c Deleted pod: e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-9qxhp 26m Normal ScalingReplicaSet deployment/e2e-worker-w3-1d31f8-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c from 2 to 1 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-1d31f8-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 26m Normal Killing pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-2ghqp Stopping container istio-proxy 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-7540da-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 26m Normal Killing pod/payload-processing-e2e-worker-w6-7540da-59844f9b7b-z68qt Stopping container payload-processing 26m Normal Killing pod/payload-pre-processing-e2e-worker-w6-7540da-598d566c67-ld8zm Stopping container payload-pre-processing 26m Normal Killing pod/e2e-inf-b-dcf5a5-openshift-default-68bf4b7597-qtcnn Stopping container istio-proxy 25m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-c93e70-8fbcf64c-bvn98 Stopping container payload-pre-processing 25m Normal Killing pod/payload-processing-e2e-shared-a-w2-c93e70-5ffc55b946-vqsqn Stopping container payload-processing 25m Warning Unhealthy pod/payload-processing-e2e-worker-w6-7540da-59844f9b7b-z68qt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w6-7540da-598d566c67-ld8zm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Warning ProbeError pod/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq Readiness probe failed: HTTP probe failed with statuscode: 503 25m Normal Killing pod/e2e-inf-b-78be2c-openshift-default-5b668467b7-7wkxq Stopping container istio-proxy 25m Normal Killing pod/payload-processing-e2e-shared-a-w3-eb3c33-846894ccd8-mzt64 Stopping container payload-processing 25m Normal Killing pod/payload-pre-processing-e2e-shared-a-w3-eb3c33-6ff459c8c9-jfqlz Stopping container payload-pre-processing 25m Normal Killing pod/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb Stopping container istio-proxy 25m Warning ProbeError pod/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/e2e-shared-a-w2-c93e70-openshift-default-5768dbcd69-g9gqb Readiness probe failed: HTTP probe failed with statuscode: 503 25m Normal Killing pod/e2e-api-a-1048fc-openshift-default-887fc8d4f-lq7gz Stopping container istio-proxy 25m Normal Killing pod/payload-processing-e2e-shared-b-w2-4a1480-7b96ff9d58-v8dwn Stopping container payload-processing 25m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-4a1480-5f47d84f87-d54hm Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-api-b-cfe1bd-74ffbccdd6-mhxsd Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-api-b-cfe1bd-5566d6c655-c8lwk Stopping container payload-pre-processing 24m Normal Killing pod/e2e-shared-a-w3-eb3c33-openshift-default-8687c6c447-x7w6q Stopping container istio-proxy 24m Normal Killing pod/payload-pre-processing-e2e-shared-b-w3-433db8-f46f8b59-cpthk Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-shared-b-w3-433db8-685d97b9d8-b8cj9 Stopping container payload-processing 24m Normal Killing pod/e2e-worker-w6-7540da-openshift-default-bb989bcf9-m7rjj Stopping container istio-proxy 24m Normal Killing pod/e2e-shared-b-w2-4a1480-openshift-default-75c6f8849c-vw8pq Stopping container istio-proxy 24m Normal Killing pod/payload-processing-e2e-worker-w2-164c36-74dc8ddf7b-wqgvc Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-worker-w2-164c36-84d6767d47-wn2ct Stopping container payload-pre-processing 24m Normal Killing pod/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs Stopping container istio-proxy 24m Warning Unhealthy pod/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs Readiness probe failed: HTTP probe failed with statuscode: 503 24m Warning ProbeError pod/e2e-shared-b-w3-433db8-openshift-default-58ff6846ff-vlfxs Readiness probe error: HTTP probe failed with statuscode: 503... 24m Normal Killing pod/e2e-api-b-cfe1bd-openshift-default-7985f9c557-5kzht Stopping container istio-proxy 24m Normal Killing pod/payload-processing-e2e-worker-w3-1d31f8-64f6fcf754-48jnl Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-worker-w3-1d31f8-f85f6f659-t2mm6 Stopping container payload-pre-processing 24m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-np57c Readiness probe error: HTTP probe failed with statuscode: 503... 24m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-np57c Readiness probe failed: HTTP probe failed with statuscode: 503 24m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-kpmpx Stopping container istio-proxy 24m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-np57c Stopping container istio-proxy 24m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: All metrics below target 24m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-qk9bp Stopping container istio-proxy 24m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7f5667f86c Deleted pod: maas-default-gateway-openshift-default-7f5667f86c-kpmpx 24m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-5wrhg Stopping container istio-proxy 24m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7f5667f86c Deleted pod: maas-default-gateway-openshift-default-7f5667f86c-d2tgj 24m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7f5667f86c from 10 to 5 24m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-d2tgj Stopping container istio-proxy 24m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7f5667f86c Deleted pod: maas-default-gateway-openshift-default-7f5667f86c-np57c 23m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-d2tgj Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-5wrhg Readiness probe failed: HTTP probe failed with statuscode: 503 23m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-5wrhg Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-d2tgj Readiness probe failed: HTTP probe failed with statuscode: 503 23m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-qk9bp Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-qk9bp Readiness probe failed: HTTP probe failed with statuscode: 503 23m Normal Killing pod/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf Stopping container payload-pre-processing 23m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w4-32449d-5bb4fc95f5-hnssf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning Unhealthy pod/payload-processing-e2e-worker-w4-32449d-58d4f897c8-2dw8h Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Normal Killing pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-2nqsk Stopping container istio-proxy 23m Normal Killing pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9 Stopping container istio-proxy 23m Warning Unhealthy pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-2nqsk Readiness probe failed: HTTP probe failed with statuscode: 503 23m Warning ProbeError pod/e2e-worker-w2-164c36-openshift-default-54b556b4c9-2nqsk Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning ProbeError pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9 Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning Unhealthy pod/e2e-worker-w3-1d31f8-openshift-default-69fb4dfb5c-dcln9 Readiness probe failed: HTTP probe failed with statuscode: 503 23m Normal Killing pod/e2e-worker-w4-32449d-openshift-default-756d4bb7c7-x4c8r Stopping container istio-proxy 23m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-zkrw7 Readiness probe error: Get "http://10.132.0.163:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 23m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-zkrw7 Readiness probe failed: Get "http://10.132.0.163:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 23m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7f5667f86c from 5 to 10 23m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 22m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-2z78m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-qp7hb Container created 22m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-cdxtr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-l5859 Container created 22m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-l5859 Container started 22m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-qp7hb Add eth0 [10.134.0.154/23] from ovn-kubernetes 22m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-cdxtr Container created 22m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-cdxtr Container started 22m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-2z78m Container created 22m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-qp7hb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-l5859 Add eth0 [10.134.0.156/23] from ovn-kubernetes 22m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-fn2dg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-2z78m Container started 22m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-qp7hb Container started 22m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-l5859 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-cdxtr Add eth0 [10.133.0.129/23] from ovn-kubernetes 22m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-fn2dg Container started 22m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-fn2dg Container created 22m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-2z78m Add eth0 [10.134.0.157/23] from ovn-kubernetes 22m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-fn2dg Add eth0 [10.134.0.155/23] from ovn-kubernetes 22m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-cdxtr Startup probe failed: Get "http://10.133.0.129:15021/healthz/ready": dial tcp 10.133.0.129:15021: connect: connection refused 22m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-cdxtr Startup probe error: Get "http://10.133.0.129:15021/healthz/ready": dial tcp 10.133.0.129:15021: connect: connection refused... 22m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-qp7hb Startup probe error: Get "http://10.134.0.154:15021/healthz/ready": dial tcp 10.134.0.154:15021: connect: connection refused... 22m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-qp7hb Startup probe failed: Get "http://10.134.0.154:15021/healthz/ready": dial tcp 10.134.0.154:15021: connect: connection refused 22m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-fn2dg Startup probe failed: Get "http://10.134.0.155:15021/healthz/ready": dial tcp 10.134.0.155:15021: connect: connection refused 22m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-l5859 Startup probe error: Get "http://10.134.0.156:15021/healthz/ready": dial tcp 10.134.0.156:15021: connect: connection refused... 22m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-fn2dg Startup probe error: Get "http://10.134.0.155:15021/healthz/ready": dial tcp 10.134.0.155:15021: connect: connection refused... 22m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-l5859 Startup probe failed: Get "http://10.134.0.156:15021/healthz/ready": dial tcp 10.134.0.156:15021: connect: connection refused 22m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-2z78m Startup probe failed: Get "http://10.134.0.157:15021/healthz/ready": dial tcp 10.134.0.157:15021: connect: connection refused 22m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-2z78m Startup probe error: Get "http://10.134.0.157:15021/healthz/ready": dial tcp 10.134.0.157:15021: connect: connection refused... 22m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-2z78m Startup probe failed: Get "http://10.134.0.157:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 22m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-qp7hb Startup probe failed: Get "http://10.134.0.154:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 22m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-2z78m Startup probe error: Get "http://10.134.0.157:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 22m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-fn2dg Startup probe error: Get "http://10.134.0.155:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 22m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-l5859 Startup probe error: Get "http://10.134.0.156:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 22m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-fn2dg Startup probe failed: Get "http://10.134.0.155:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 22m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-l5859 Startup probe failed: Get "http://10.134.0.156:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 22m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-qp7hb Startup probe error: Get "http://10.134.0.154:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 22m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 22m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-vndrh Stopping container discovery 22m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-vndrh 22m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 16m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-4bjss Readiness probe error: Get "http://10.132.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-4bjss Readiness probe failed: Get "http://10.132.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 15m Normal AddedLabel gateway/e2e-worker-single-main-8a134f Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-8a134f 15m Normal AddedInterface pod/e2e-worker-single-main-8a134f-openshift-default-6cfbcbcbc7jr9zq Add eth0 [10.132.1.21/23] from ovn-kubernetes 15m Normal SuccessfulCreate replicaset/e2e-worker-single-main-8a134f-openshift-default-6cfbcbcbc7 Created pod: e2e-worker-single-main-8a134f-openshift-default-6cfbcbcbc7jr9zq 15m Normal ScalingReplicaSet deployment/e2e-worker-single-main-8a134f-openshift-default Scaled up replica set e2e-worker-single-main-8a134f-openshift-default-6cfbcbcbc7 from 0 to 1 15m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-8a134f Scaled up replica set payload-processing-e2e-worker-single-main-8a134f-77cfb5fcf8 from 0 to 1 15m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-8a134f-77cfb5fcf8 Error creating: pods "payload-processing-e2e-worker-single-main-8a134f-77cfb5fcf8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-8a134f: serviceaccount "payload-processing-e2e-worker-single-main-8a134f" not found 15m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-8a134f Scaled up replica set payload-pre-processing-e2e-worker-single-main-8a134f-57db7bb847 from 0 to 1 15m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-8a134f-57db7bb847 Error creating: pods "payload-pre-processing-e2e-worker-single-main-8a134f-57db7bb847-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-8a134f: serviceaccount "payload-processing-e2e-worker-single-main-8a134f" not found 15m Normal Pulled pod/payload-processing-e2e-worker-single-main-8a134f-77cfb5fcfnt7zf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 15m Normal Started pod/payload-processing-e2e-worker-single-main-8a134f-77cfb5fcfnt7zf Container started 15m Normal Created pod/payload-processing-e2e-worker-single-main-8a134f-77cfb5fcfnt7zf Container created 15m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-8a134f-77cfb5fcfnt7zf Add eth0 [10.132.1.22/23] from ovn-kubernetes 15m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-8a134f-77cfb5fcf8 Created pod: payload-processing-e2e-worker-single-main-8a134f-77cfb5fcfnt7zf 15m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-8a134f-57db7wbxl4 Add eth0 [10.132.1.23/23] from ovn-kubernetes 15m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-8a134f-57db7bb847 Created pod: payload-pre-processing-e2e-worker-single-main-8a134f-57db7wbxl4 15m Normal Started pod/payload-pre-processing-e2e-worker-single-main-8a134f-57db7wbxl4 Container started 15m Normal Created pod/payload-pre-processing-e2e-worker-single-main-8a134f-57db7wbxl4 Container created 15m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-8a134f-57db7wbxl4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 15m Normal Started pod/e2e-worker-single-main-8a134f-openshift-default-6cfbcbcbc7jr9zq Container started 15m Normal Created pod/e2e-worker-single-main-8a134f-openshift-default-6cfbcbcbc7jr9zq Container created 15m Normal Pulled pod/e2e-worker-single-main-8a134f-openshift-default-6cfbcbcbc7jr9zq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 15m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-8a134f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 15m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-8a134f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 15m Warning ProbeError pod/e2e-worker-single-main-8a134f-openshift-default-6cfbcbcbc7jr9zq Startup probe error: Get "http://10.132.1.21:15021/healthz/ready": dial tcp 10.132.1.21:15021: connect: connection refused... 15m Warning Unhealthy pod/e2e-worker-single-main-8a134f-openshift-default-6cfbcbcbc7jr9zq Startup probe failed: Get "http://10.132.1.21:15021/healthz/ready": dial tcp 10.132.1.21:15021: connect: connection refused 15m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-8a134f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 15m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-8a134f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 14m Normal Killing pod/payload-processing-e2e-worker-single-main-8a134f-77cfb5fcfnt7zf Stopping container payload-processing 14m Normal Killing pod/payload-pre-processing-e2e-worker-single-main-8a134f-57db7wbxl4 Stopping container payload-pre-processing 13m Normal Killing pod/e2e-worker-single-main-8a134f-openshift-default-6cfbcbcbc7jr9zq Stopping container istio-proxy 13m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-8a134f-57db7wbxl4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 13m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-8a134f-77cfb5fcfnt7zf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 11m Normal AddedInterface pod/e2e-worker-single-main-60ef36-openshift-default-86dc6c59d5qhpc5 Add eth0 [10.132.1.30/23] from ovn-kubernetes 11m Normal AddedLabel gateway/e2e-worker-single-main-60ef36 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-60ef36 11m Normal ScalingReplicaSet deployment/e2e-worker-single-main-60ef36-openshift-default Scaled up replica set e2e-worker-single-main-60ef36-openshift-default-86dc6c59d5 from 0 to 1 11m Normal SuccessfulCreate replicaset/e2e-worker-single-main-60ef36-openshift-default-86dc6c59d5 Created pod: e2e-worker-single-main-60ef36-openshift-default-86dc6c59d5qhpc5 10m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-60ef36 Scaled up replica set payload-processing-e2e-worker-single-main-60ef36-656858f44 from 0 to 1 10m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-60ef36-656858f44 Error creating: pods "payload-processing-e2e-worker-single-main-60ef36-656858f44-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-60ef36: serviceaccount "payload-processing-e2e-worker-single-main-60ef36" not found 10m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-60ef36-656858f44 Created pod: payload-processing-e2e-worker-single-main-60ef36-656858f446mbq7 10m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-60ef36 Scaled up replica set payload-pre-processing-e2e-worker-single-main-60ef36-77855d4c6c from 0 to 1 10m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-60ef36-77855d4c6c Error creating: pods "payload-pre-processing-e2e-worker-single-main-60ef36-77855d4c6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-60ef36: serviceaccount "payload-processing-e2e-worker-single-main-60ef36" not found 10m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-60ef36-77855d4c6c Created pod: payload-pre-processing-e2e-worker-single-main-60ef36-77855qtqr8 10m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-60ef36-656858f446mbq7 Add eth0 [10.132.1.31/23] from ovn-kubernetes 10m Normal Pulled pod/payload-processing-e2e-worker-single-main-60ef36-656858f446mbq7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 10m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-60ef36-77855qtqr8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 10m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-60ef36-77855qtqr8 Add eth0 [10.132.1.32/23] from ovn-kubernetes 10m Normal Created pod/payload-pre-processing-e2e-worker-single-main-60ef36-77855qtqr8 Container created 10m Normal Started pod/payload-pre-processing-e2e-worker-single-main-60ef36-77855qtqr8 Container started 10m Normal Started pod/payload-processing-e2e-worker-single-main-60ef36-656858f446mbq7 Container started 10m Normal Created pod/payload-processing-e2e-worker-single-main-60ef36-656858f446mbq7 Container created 10m Normal Started pod/e2e-worker-single-main-60ef36-openshift-default-86dc6c59d5qhpc5 Container started 10m Normal Pulled pod/e2e-worker-single-main-60ef36-openshift-default-86dc6c59d5qhpc5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 10m Normal Created pod/e2e-worker-single-main-60ef36-openshift-default-86dc6c59d5qhpc5 Container created 10m Warning ProbeError pod/e2e-worker-single-main-60ef36-openshift-default-86dc6c59d5qhpc5 Startup probe error: Get "http://10.132.1.30:15021/healthz/ready": dial tcp 10.132.1.30:15021: connect: connection refused... 10m Warning Unhealthy pod/e2e-worker-single-main-60ef36-openshift-default-86dc6c59d5qhpc5 Startup probe failed: Get "http://10.132.1.30:15021/healthz/ready": dial tcp 10.132.1.30:15021: connect: connection refused 10m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-60ef36-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 10m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-60ef36-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 10m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-60ef36-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 10m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-60ef36-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 9m19s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-60ef36-77855qtqr8 Stopping container payload-pre-processing 9m19s Normal Killing pod/payload-processing-e2e-worker-single-main-60ef36-656858f446mbq7 Stopping container payload-processing 9m4s Normal Killing pod/e2e-worker-single-main-60ef36-openshift-default-86dc6c59d5qhpc5 Stopping container istio-proxy 8m49s Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-cdxtr Readiness probe failed: Get "http://10.133.0.129:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 8m49s Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-cdxtr Readiness probe error: Get "http://10.133.0.129:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 7m21s Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-njmdj Readiness probe error: Get "http://10.133.0.109:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 7m21s Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-njmdj Readiness probe failed: Get "http://10.133.0.109:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 7m21s Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-8q2rz Readiness probe error: Get "http://10.133.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 7m21s Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-8q2rz Readiness probe failed: Get "http://10.133.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 7m3s Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-qp7hb Readiness probe error: Get "http://10.134.0.154:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 7m3s Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-qp7hb Readiness probe failed: Get "http://10.134.0.154:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 7m3s Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-l5859 Readiness probe failed: Get "http://10.134.0.156:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 7m3s Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-l5859 Readiness probe error: Get "http://10.134.0.156:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 7m3s Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-fn2dg Readiness probe failed: Get "http://10.134.0.155:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 7m3s Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-fn2dg Readiness probe error: Get "http://10.134.0.155:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 6m21s Normal SuccessfulCreate replicaset/e2e-ait-738d0c9a-openshift-default-6dd5576d67 Created pod: e2e-ait-738d0c9a-openshift-default-6dd5576d67-lcmqs 6m21s Normal ScalingReplicaSet deployment/e2e-ait-738d0c9a-openshift-default Scaled up replica set e2e-ait-738d0c9a-openshift-default-6dd5576d67 from 0 to 1 6m21s Normal AddedLabel gateway/e2e-ait-738d0c9a Added label istio.io/rev=openshift-gateway to gateway e2e-ait-738d0c9a 6m20s Normal AddedInterface pod/e2e-ait-738d0c9a-openshift-default-6dd5576d67-lcmqs Add eth0 [10.132.1.39/23] from ovn-kubernetes 6m14s Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-738d0c9a Scaled up replica set payload-processing-e2e-ait-738d0c9a-57b49dd7f6 from 0 to 1 6m14s Warning FailedCreate replicaset/payload-processing-e2e-ait-738d0c9a-57b49dd7f6 Error creating: pods "payload-processing-e2e-ait-738d0c9a-57b49dd7f6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-738d0c9a: serviceaccount "payload-processing-e2e-ait-738d0c9a" not found 6m14s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-738d0c9a Scaled up replica set payload-pre-processing-e2e-ait-738d0c9a-76ddd9d799 from 0 to 1 6m14s Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-738d0c9a-76ddd9d799 Error creating: pods "payload-pre-processing-e2e-ait-738d0c9a-76ddd9d799-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-738d0c9a: serviceaccount "payload-processing-e2e-ait-738d0c9a" not found 6m13s Normal Pulled pod/payload-pre-processing-e2e-ait-738d0c9a-76ddd9d799-pnhcj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 6m13s Normal Pulled pod/payload-processing-e2e-ait-738d0c9a-57b49dd7f6-9vwgk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 6m13s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-738d0c9a-76ddd9d799 Created pod: payload-pre-processing-e2e-ait-738d0c9a-76ddd9d799-pnhcj 6m13s Normal Started pod/payload-processing-e2e-ait-738d0c9a-57b49dd7f6-9vwgk Container started 6m13s Normal Created pod/payload-processing-e2e-ait-738d0c9a-57b49dd7f6-9vwgk Container created 6m13s Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-738d0c9a-57b49dd7f6 Created pod: payload-processing-e2e-ait-738d0c9a-57b49dd7f6-9vwgk 6m13s Normal AddedInterface pod/payload-pre-processing-e2e-ait-738d0c9a-76ddd9d799-pnhcj Add eth0 [10.132.1.41/23] from ovn-kubernetes 6m13s Normal AddedInterface pod/payload-processing-e2e-ait-738d0c9a-57b49dd7f6-9vwgk Add eth0 [10.132.1.40/23] from ovn-kubernetes 6m12s Normal Started pod/payload-pre-processing-e2e-ait-738d0c9a-76ddd9d799-pnhcj Container started 6m12s Normal Created pod/payload-pre-processing-e2e-ait-738d0c9a-76ddd9d799-pnhcj Container created 5m53s Normal Created pod/e2e-ait-738d0c9a-openshift-default-6dd5576d67-lcmqs Container created 5m53s Normal Started pod/e2e-ait-738d0c9a-openshift-default-6dd5576d67-lcmqs Container started 5m53s Normal Pulled pod/e2e-ait-738d0c9a-openshift-default-6dd5576d67-lcmqs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 5m47s Warning Unhealthy pod/e2e-ait-738d0c9a-openshift-default-6dd5576d67-lcmqs Startup probe failed: Get "http://10.132.1.39:15021/healthz/ready": dial tcp 10.132.1.39:15021: connect: connection refused 5m47s Warning ProbeError pod/e2e-ait-738d0c9a-openshift-default-6dd5576d67-lcmqs Startup probe error: Get "http://10.132.1.39:15021/healthz/ready": dial tcp 10.132.1.39:15021: connect: connection refused... 5m39s Normal Killing pod/payload-processing-e2e-ait-738d0c9a-57b49dd7f6-9vwgk Stopping container payload-processing 5m39s Normal Killing pod/payload-pre-processing-e2e-ait-738d0c9a-76ddd9d799-pnhcj Stopping container payload-pre-processing 5m35s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-738d0c9a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 5m35s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-738d0c9a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 5m30s Normal Killing pod/e2e-ait-738d0c9a-openshift-default-6dd5576d67-lcmqs Stopping container istio-proxy 5m29s Normal AddedLabel gateway/e2e-authcfg-http-034b52 Added label istio.io/rev=openshift-gateway to gateway e2e-authcfg-http-034b52 5m28s Normal ScalingReplicaSet deployment/e2e-authcfg-http-034b52-openshift-default Scaled up replica set e2e-authcfg-http-034b52-openshift-default-546fbd7649 from 0 to 1 5m28s Normal SuccessfulCreate replicaset/e2e-authcfg-http-034b52-openshift-default-546fbd7649 Created pod: e2e-authcfg-http-034b52-openshift-default-546fbd7649-76chx 5m27s Normal AddedInterface pod/e2e-authcfg-http-034b52-openshift-default-546fbd7649-76chx Add eth0 [10.132.1.43/23] from ovn-kubernetes 5m21s Warning FailedCreate replicaset/payload-pre-processing-e2e-authcfg-http-034b52-555f758647 Error creating: pods "payload-pre-processing-e2e-authcfg-http-034b52-555f758647-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-034b52: serviceaccount "payload-processing-e2e-authcfg-http-034b52" not found 5m21s Warning FailedCreate replicaset/payload-processing-e2e-authcfg-http-034b52-7c869c85b4 Error creating: pods "payload-processing-e2e-authcfg-http-034b52-7c869c85b4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-034b52: serviceaccount "payload-processing-e2e-authcfg-http-034b52" not found 5m21s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-authcfg-http-034b52 Scaled up replica set payload-pre-processing-e2e-authcfg-http-034b52-555f758647 from 0 to 1 5m21s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-authcfg-http-034b52-555f758647 Created pod: payload-pre-processing-e2e-authcfg-http-034b52-555f758647-zv96x 5m21s Normal ScalingReplicaSet deployment/payload-processing-e2e-authcfg-http-034b52 Scaled up replica set payload-processing-e2e-authcfg-http-034b52-7c869c85b4 from 0 to 1 5m20s Normal Created pod/payload-processing-e2e-authcfg-http-034b52-7c869c85b4-57j52 Container created 5m20s Normal Pulled pod/payload-pre-processing-e2e-authcfg-http-034b52-555f758647-zv96x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m20s Normal Started pod/payload-processing-e2e-authcfg-http-034b52-7c869c85b4-57j52 Container started 5m20s Normal Started pod/payload-pre-processing-e2e-authcfg-http-034b52-555f758647-zv96x Container started 5m20s Normal AddedInterface pod/payload-pre-processing-e2e-authcfg-http-034b52-555f758647-zv96x Add eth0 [10.132.1.44/23] from ovn-kubernetes 5m20s Normal SuccessfulCreate replicaset/payload-processing-e2e-authcfg-http-034b52-7c869c85b4 Created pod: payload-processing-e2e-authcfg-http-034b52-7c869c85b4-57j52 5m20s Normal Pulled pod/payload-processing-e2e-authcfg-http-034b52-7c869c85b4-57j52 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m20s Normal AddedInterface pod/payload-processing-e2e-authcfg-http-034b52-7c869c85b4-57j52 Add eth0 [10.132.1.45/23] from ovn-kubernetes 5m20s Normal Created pod/payload-pre-processing-e2e-authcfg-http-034b52-555f758647-zv96x Container created 5m4s Normal Created pod/e2e-authcfg-http-034b52-openshift-default-546fbd7649-76chx Container created 5m4s Normal Started pod/e2e-authcfg-http-034b52-openshift-default-546fbd7649-76chx Container started 5m4s Normal Pulled pod/e2e-authcfg-http-034b52-openshift-default-546fbd7649-76chx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 4m59s Warning ProbeError pod/e2e-authcfg-http-034b52-openshift-default-546fbd7649-76chx Startup probe error: Get "http://10.132.1.43:15021/healthz/ready": dial tcp 10.132.1.43:15021: connect: connection refused... 4m59s Warning Unhealthy pod/e2e-authcfg-http-034b52-openshift-default-546fbd7649-76chx Startup probe failed: Get "http://10.132.1.43:15021/healthz/ready": dial tcp 10.132.1.43:15021: connect: connection refused 4m58s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-034b52-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m58s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-034b52-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m43s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-034b52-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 4m43s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-034b52-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 3m17s Normal Killing pod/payload-processing-e2e-authcfg-http-034b52-7c869c85b4-57j52 Stopping container payload-processing 3m17s Normal Killing pod/payload-pre-processing-e2e-authcfg-http-034b52-555f758647-zv96x Stopping container payload-pre-processing 3m1s Normal Killing pod/e2e-authcfg-http-034b52-openshift-default-546fbd7649-76chx Stopping container istio-proxy 2m58s Warning ProbeError pod/e2e-authcfg-http-034b52-openshift-default-546fbd7649-76chx Readiness probe error: HTTP probe failed with statuscode: 503... 2m58s Warning Unhealthy pod/e2e-authcfg-http-034b52-openshift-default-546fbd7649-76chx Readiness probe failed: HTTP probe failed with statuscode: 503 108s Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-2z78m Readiness probe error: Get "http://10.134.0.157:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 108s Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-2z78m Readiness probe failed: Get "http://10.134.0.157:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)