=== events in openshift-ingress (2026-10-29T23:19:27Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 43m Normal Scheduled pod/maas-default-gateway-openshift-default-5db77fbd9f-82d8r Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-82d8r to ip-10-0-128-31.ec2.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-mt-1fa4b8b6-675f9bbd4c-xlqfw Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-1fa4b8b6-675f9bbd4c-xlqfw to ip-10-0-138-2.ec2.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-mt-1fa4b8b6-675f9bbd4c-4jhnf Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-1fa4b8b6-675f9bbd4c-4jhnf to ip-10-0-128-31.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-mt-1131a29d-79fbb5fd4c-rqqk2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-1131a29d-79fbb5fd4c-rqqk2 to ip-10-0-128-31.ec2.internal 25m Normal Scheduled pod/e2e-ipp-a-7be71d-openshift-default-8dd8b5bdc-4hx2h Successfully assigned openshift-ingress/e2e-ipp-a-7be71d-openshift-default-8dd8b5bdc-4hx2h to ip-10-0-128-31.ec2.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-migrate-e385da9b-64fcf8b6bc-zjwpw Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-e385da9b-64fcf8b6bc-zjwpw to ip-10-0-128-31.ec2.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-a962d7-688dd94bc-xptcq Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-a962d7-688dd94bc-xptcq to ip-10-0-128-31.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-a962d7-64fcd788bc-vc928 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-a962d7-64fcd788bc-vc928 to ip-10-0-138-2.ec2.internal 16m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-9d6d4e-767bdcc655-qhqnw Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-9d6d4e-767bdcc655-qhqnw to ip-10-0-138-2.ec2.internal 17m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-9d6d4e-5fd9b5769f-xgn77 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-9d6d4e-5fd9b5769f-xgn77 to ip-10-0-128-31.ec2.internal 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-85862e-6c4464c9-9xtht Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-85862e-6c4464c9-9xtht to ip-10-0-128-31.ec2.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w0-920032-85fc75f8c5-fsg62 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-920032-85fc75f8c5-fsg62 to ip-10-0-138-2.ec2.internal 50m Normal Scheduled pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-vnz5k Successfully assigned openshift-ingress/e2e-worker-w1-fc684b-openshift-default-7fcf678997-vnz5k to ip-10-0-138-2.ec2.internal 43m Normal Scheduled pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-v56n2 Successfully assigned openshift-ingress/e2e-worker-w1-fc684b-openshift-default-7fcf678997-v56n2 to ip-10-0-143-15.ec2.internal 43m Normal Scheduled pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-ghbxb Successfully assigned openshift-ingress/e2e-worker-w1-fc684b-openshift-default-7fcf678997-ghbxb to ip-10-0-128-31.ec2.internal 30m Normal Scheduled pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-wghsp Successfully assigned openshift-ingress/e2e-inf-b-e619b6-openshift-default-6d4c644df9-wghsp to ip-10-0-138-2.ec2.internal 49m Normal Scheduled pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-g26hx Successfully assigned openshift-ingress/e2e-worker-w1-fc684b-openshift-default-7fcf678997-g26hx to ip-10-0-143-15.ec2.internal 48m Normal Scheduled pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-448t7 Successfully assigned openshift-ingress/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-448t7 to ip-10-0-138-2.ec2.internal 43m Normal Scheduled pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-dnzl8 Successfully assigned openshift-ingress/e2e-worker-w1-fc684b-openshift-default-7fcf678997-dnzl8 to ip-10-0-138-2.ec2.internal 43m Normal Scheduled pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-cwggq Successfully assigned openshift-ingress/e2e-worker-w1-fc684b-openshift-default-7fcf678997-cwggq to ip-10-0-128-31.ec2.internal 33m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w1-af260f-7cfc9454ff-9c5lq Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w1-af260f-7cfc9454ff-9c5lq to ip-10-0-138-2.ec2.internal 52m Normal Scheduled pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-988qz Successfully assigned openshift-ingress/e2e-worker-w1-fc684b-openshift-default-7fcf678997-988qz to ip-10-0-128-31.ec2.internal 44m Normal Scheduled pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-8wvv6 Successfully assigned openshift-ingress/e2e-worker-w1-fc684b-openshift-default-7fcf678997-8wvv6 to ip-10-0-128-31.ec2.internal 46m Normal Scheduled pod/e2e-ait-f7dea32d-openshift-default-5cf5f998fc-dt4cr Successfully assigned openshift-ingress/e2e-ait-f7dea32d-openshift-default-5cf5f998fc-dt4cr to ip-10-0-128-31.ec2.internal 49m Normal Scheduled pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-8npq2 Successfully assigned openshift-ingress/e2e-worker-w1-fc684b-openshift-default-7fcf678997-8npq2 to ip-10-0-138-2.ec2.internal 44m Normal Scheduled pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-7qslq Successfully assigned openshift-ingress/e2e-worker-w1-fc684b-openshift-default-7fcf678997-7qslq to ip-10-0-143-15.ec2.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-eb02df-54d8869579-kckzx Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-eb02df-54d8869579-kckzx to ip-10-0-138-2.ec2.internal 52m Normal Scheduled pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-5lkhc Successfully assigned openshift-ingress/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-5lkhc to ip-10-0-138-2.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w3-aaa3c6-64445b56bc-9wn9m Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-aaa3c6-64445b56bc-9wn9m to ip-10-0-138-2.ec2.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-85862e-597bbb855-5jrdf Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-85862e-597bbb855-5jrdf to ip-10-0-128-31.ec2.internal 49m Normal Scheduled pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-787nl Successfully assigned openshift-ingress/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-787nl to ip-10-0-138-2.ec2.internal 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-75b68d-77657dc59-xt7mh Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-75b68d-77657dc59-xt7mh to ip-10-0-138-2.ec2.internal 48m Normal Scheduled pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-zbfz4 Successfully assigned openshift-ingress/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-zbfz4 to ip-10-0-128-31.ec2.internal 48m Normal Scheduled pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9hfvc Successfully assigned openshift-ingress/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9hfvc to ip-10-0-128-31.ec2.internal 20m Normal Scheduled pod/e2e-ipp-a-80248b-openshift-default-5677b677f5-lx6cs Successfully assigned openshift-ingress/e2e-ipp-a-80248b-openshift-default-5677b677f5-lx6cs to ip-10-0-138-2.ec2.internal 52m Normal Scheduled pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-tczdj Successfully assigned openshift-ingress/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-tczdj to ip-10-0-138-2.ec2.internal 50m Normal Scheduled pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-rkgz4 Successfully assigned openshift-ingress/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-rkgz4 to ip-10-0-128-31.ec2.internal 49m Normal Scheduled pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-ntt82 Successfully assigned openshift-ingress/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-ntt82 to ip-10-0-138-2.ec2.internal 42m Normal Scheduled pod/e2e-api-a-616da7-openshift-default-85b49d64c5-6ctxr Successfully assigned openshift-ingress/e2e-api-a-616da7-openshift-default-85b49d64c5-6ctxr to ip-10-0-138-2.ec2.internal 49m Normal Scheduled pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-nf9qq Successfully assigned openshift-ingress/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-nf9qq to ip-10-0-128-31.ec2.internal 49m Normal Scheduled pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-lks7l Successfully assigned openshift-ingress/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-lks7l to ip-10-0-143-15.ec2.internal 50m Normal Scheduled pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-f7th7 Successfully assigned openshift-ingress/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-f7th7 to ip-10-0-143-15.ec2.internal 48m Normal Scheduled pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-9vrlz Successfully assigned openshift-ingress/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-9vrlz to ip-10-0-143-15.ec2.internal 49m Normal Scheduled pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9rm2s Successfully assigned openshift-ingress/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9rm2s to ip-10-0-128-31.ec2.internal 48m Normal Scheduled pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-8656t Successfully assigned openshift-ingress/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-8656t to ip-10-0-138-2.ec2.internal 48m Normal Scheduled pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-7glgv Successfully assigned openshift-ingress/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-7glgv to ip-10-0-138-2.ec2.internal 48m Normal Scheduled pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-dccl7 Successfully assigned openshift-ingress/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-dccl7 to ip-10-0-143-15.ec2.internal 8m58s Normal Scheduled pod/e2e-worker-single-main-e5236b-openshift-default-547784c65-qg68z Successfully assigned openshift-ingress/e2e-worker-single-main-e5236b-openshift-default-547784c65-qg68z to ip-10-0-128-31.ec2.internal 50m Normal Scheduled pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-jjlwt Successfully assigned openshift-ingress/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-jjlwt to ip-10-0-128-31.ec2.internal 17m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-75b68d-59747b7fb7-7n6zb Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-75b68d-59747b7fb7-7n6zb to ip-10-0-128-31.ec2.internal 49m Normal Scheduled pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-rppfp Successfully assigned openshift-ingress/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-rppfp to ip-10-0-143-15.ec2.internal 47m Normal Scheduled pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-vt95w Successfully assigned openshift-ingress/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-vt95w to ip-10-0-128-31.ec2.internal 33m Normal Scheduled pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-ml8sr Successfully assigned openshift-ingress/e2e-inf-b-e619b6-openshift-default-6d4c644df9-ml8sr to ip-10-0-138-2.ec2.internal 19m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-63e89c-784f845595-7vhc9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-63e89c-784f845595-7vhc9 to ip-10-0-138-2.ec2.internal 5m31s Normal Scheduled pod/e2e-worker-single-main-063169-openshift-default-786ffd57b5prdhk Successfully assigned openshift-ingress/e2e-worker-single-main-063169-openshift-default-786ffd57b5prdhk to ip-10-0-128-31.ec2.internal 44m Normal Scheduled pod/e2e-shared-b-w6-db7419-openshift-default-9dd55d5bc-q2xvw Successfully assigned openshift-ingress/e2e-shared-b-w6-db7419-openshift-default-9dd55d5bc-q2xvw to ip-10-0-128-31.ec2.internal 47m Normal Scheduled pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-zsn6g Successfully assigned openshift-ingress/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-zsn6g to ip-10-0-143-15.ec2.internal 41m Normal Scheduled pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-6x2tx Successfully assigned openshift-ingress/e2e-api-a-cae55b-openshift-default-f795f5d7f-6x2tx to ip-10-0-128-31.ec2.internal 17m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-63e89c-744cbf57c9-k8rvr Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-63e89c-744cbf57c9-k8rvr to ip-10-0-138-2.ec2.internal 29m Normal Scheduled pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-qpsqc Successfully assigned openshift-ingress/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-qpsqc to ip-10-0-143-15.ec2.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-4796a9-d6c677c75-9zj9b Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-4796a9-d6c677c75-9zj9b to ip-10-0-138-2.ec2.internal 36m Normal Scheduled pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-82f2v Successfully assigned openshift-ingress/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-82f2v to ip-10-0-138-2.ec2.internal 31m Normal Scheduled pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-2dhcj Successfully assigned openshift-ingress/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-2dhcj to ip-10-0-128-31.ec2.internal 31m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w4-f8ca96-696c58bc6c-4dw72 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-f8ca96-696c58bc6c-4dw72 to ip-10-0-128-31.ec2.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w5-9aafe6-6f8f97cff9-4cmpr Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w5-9aafe6-6f8f97cff9-4cmpr to ip-10-0-138-2.ec2.internal 41m Normal Scheduled pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-lv8h9 Successfully assigned openshift-ingress/e2e-api-a-cae55b-openshift-default-f795f5d7f-lv8h9 to ip-10-0-143-15.ec2.internal 48m Normal Scheduled pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-89tnt Successfully assigned openshift-ingress/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-89tnt to ip-10-0-138-2.ec2.internal 50m Normal Scheduled pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-95zbb Successfully assigned openshift-ingress/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-95zbb to ip-10-0-128-31.ec2.internal 28m Normal Scheduled pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-jtp6w Successfully assigned openshift-ingress/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-jtp6w to ip-10-0-128-31.ec2.internal 49m Normal Scheduled pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-9txdf Successfully assigned openshift-ingress/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-9txdf to ip-10-0-143-15.ec2.internal 23m Normal Scheduled pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-cs6kx Successfully assigned openshift-ingress/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-cs6kx to ip-10-0-128-31.ec2.internal 48m Normal Scheduled pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-bkgjl Successfully assigned openshift-ingress/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-bkgjl to ip-10-0-138-2.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w6-06a95f-7c5bb67c55-dkr8j Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w6-06a95f-7c5bb67c55-dkr8j to ip-10-0-138-2.ec2.internal 42m Normal Scheduled pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-x82q6 Successfully assigned openshift-ingress/e2e-api-a-cae55b-openshift-default-f795f5d7f-x82q6 to ip-10-0-138-2.ec2.internal 28m Normal Scheduled pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-55kkj Successfully assigned openshift-ingress/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-55kkj to ip-10-0-143-15.ec2.internal 52m Normal Scheduled pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-cfx67 Successfully assigned openshift-ingress/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-cfx67 to ip-10-0-138-2.ec2.internal 21m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-4796a9-58585d8495-rt4r8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-4796a9-58585d8495-rt4r8 to ip-10-0-138-2.ec2.internal 30m Normal Scheduled pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-2kr84 Successfully assigned openshift-ingress/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-2kr84 to ip-10-0-138-2.ec2.internal 48m Normal Scheduled pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-fqhkm Successfully assigned openshift-ingress/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-fqhkm to ip-10-0-143-15.ec2.internal 48m Normal Scheduled pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-h947w Successfully assigned openshift-ingress/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-h947w to ip-10-0-128-31.ec2.internal 26m Normal Scheduled pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-shmwc Successfully assigned openshift-ingress/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-shmwc to ip-10-0-143-15.ec2.internal 43m Normal Scheduled pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-p9dds Successfully assigned openshift-ingress/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-p9dds to ip-10-0-143-15.ec2.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-c87bf1-84c7b6485f-cxhjm Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-c87bf1-84c7b6485f-cxhjm to ip-10-0-138-2.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-c87bf1-6bc965df8c-79jh7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-c87bf1-6bc965df8c-79jh7 to ip-10-0-143-15.ec2.internal 36m Normal Scheduled pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-kv7vp Successfully assigned openshift-ingress/e2e-inf-b-e619b6-openshift-default-6d4c644df9-kv7vp to ip-10-0-128-31.ec2.internal 47m Normal Scheduled pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-jp7np Successfully assigned openshift-ingress/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-jp7np to ip-10-0-128-31.ec2.internal 24m Normal Scheduled pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-nn958 Successfully assigned openshift-ingress/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-nn958 to ip-10-0-138-2.ec2.internal 43m Normal Scheduled pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-74t4k Successfully assigned openshift-ingress/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-74t4k to ip-10-0-128-31.ec2.internal 44m Normal Scheduled pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-6mjhh Successfully assigned openshift-ingress/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-6mjhh to ip-10-0-138-2.ec2.internal 48m Normal Scheduled pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-jwrdl Successfully assigned openshift-ingress/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-jwrdl to ip-10-0-143-15.ec2.internal 48m Normal Scheduled pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-s5cq4 Successfully assigned openshift-ingress/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-s5cq4 to ip-10-0-128-31.ec2.internal 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-b8f54c-7c969c8947-9wpb5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f54c-7c969c8947-9wpb5 to ip-10-0-138-2.ec2.internal 24m Normal Scheduled pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-ht9j2 Successfully assigned openshift-ingress/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-ht9j2 to ip-10-0-138-2.ec2.internal 60m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-6c786f9b57gktwl Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-6c786f9b57gktwl to ip-10-0-143-15.ec2.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-b8f54c-59d54cdb67-t4sjz Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-b8f54c-59d54cdb67-t4sjz to ip-10-0-128-31.ec2.internal 28m Normal Scheduled pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-hjvtw Successfully assigned openshift-ingress/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-hjvtw to ip-10-0-128-31.ec2.internal 42m Normal Scheduled pod/e2e-api-a-ee886f-openshift-default-7f66f8dfb5-m9m5s Successfully assigned openshift-ingress/e2e-api-a-ee886f-openshift-default-7f66f8dfb5-m9m5s to ip-10-0-128-31.ec2.internal 22m Normal Scheduled pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-hdcz8 Successfully assigned openshift-ingress/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-hdcz8 to ip-10-0-143-15.ec2.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-8c4b02-c7cf79579-lj49f Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-8c4b02-c7cf79579-lj49f to ip-10-0-138-2.ec2.internal 48m Normal Scheduled pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8gswt Successfully assigned openshift-ingress/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8gswt to ip-10-0-138-2.ec2.internal 21m Normal Scheduled pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-9m5dt Successfully assigned openshift-ingress/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-9m5dt to ip-10-0-138-2.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-8c4b02-696f4f544c-xbnh8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-8c4b02-696f4f544c-xbnh8 to ip-10-0-128-31.ec2.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w0-02720a-54d9b599c9-q5xcw Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-02720a-54d9b599c9-q5xcw to ip-10-0-128-31.ec2.internal 31m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w1-d68c2c-7d66774ccf-wvf5r Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d68c2c-7d66774ccf-wvf5r to ip-10-0-138-2.ec2.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-3ee017-b5cc8c7ff-nnl9p Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3ee017-b5cc8c7ff-nnl9p to ip-10-0-128-31.ec2.internal 30m Normal Scheduled pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-59rv7 Successfully assigned openshift-ingress/e2e-inf-b-e619b6-openshift-default-6d4c644df9-59rv7 to ip-10-0-143-15.ec2.internal 47m Normal Scheduled pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8n7fg Successfully assigned openshift-ingress/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8n7fg to ip-10-0-128-31.ec2.internal 47m Normal Scheduled pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-cb9wn Successfully assigned openshift-ingress/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-cb9wn to ip-10-0-138-2.ec2.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w3-3bd0f6-858c665c67-btrf6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3bd0f6-858c665c67-btrf6 to ip-10-0-128-31.ec2.internal 65m Normal Scheduled pod/router-default-7b6b64f55f-9l4xp Successfully assigned openshift-ingress/router-default-7b6b64f55f-9l4xp to ip-10-0-128-31.ec2.internal 65m Warning FailedScheduling pod/router-default-7b6b64f55f-9l4xp no nodes available to schedule pods 70m Warning FailedScheduling pod/router-default-7b6b64f55f-9l4xp no nodes available to schedule pods 21m Normal Scheduled pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-xpz6c Successfully assigned openshift-ingress/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-xpz6c to ip-10-0-128-31.ec2.internal 41m Normal Scheduled pod/e2e-api-b-39322f-openshift-default-78f75cb6d5-np7f7 Successfully assigned openshift-ingress/e2e-api-b-39322f-openshift-default-78f75cb6d5-np7f7 to ip-10-0-128-31.ec2.internal 47m Normal Scheduled pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-dd4x7 Successfully assigned openshift-ingress/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-dd4x7 to ip-10-0-138-2.ec2.internal 19m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-83f120-75b867c88c-dcmh5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-83f120-75b867c88c-dcmh5 to ip-10-0-128-31.ec2.internal 21m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-83f120-57b979dcd7-zqf6r Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-83f120-57b979dcd7-zqf6r to ip-10-0-128-31.ec2.internal 18m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-80248b-5c58666f59-csjml Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-80248b-5c58666f59-csjml to ip-10-0-138-2.ec2.internal 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-80248b-5bd96b4c59-zqqfc Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-80248b-5bd96b4c59-zqqfc to ip-10-0-138-2.ec2.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-7be71d-865cd9bc7f-52m9d Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-7be71d-865cd9bc7f-52m9d to ip-10-0-138-2.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-7be71d-648bf674d7-kqdsm Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-7be71d-648bf674d7-kqdsm to ip-10-0-138-2.ec2.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-e619b6-868f869f9f-j8pwm Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-e619b6-868f869f9f-j8pwm to ip-10-0-128-31.ec2.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w4-c933cb-786d7ff8b7-zhqxg Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-c933cb-786d7ff8b7-zhqxg to ip-10-0-138-2.ec2.internal 23m Normal Scheduled pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-wrrbr Successfully assigned openshift-ingress/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-wrrbr to ip-10-0-143-15.ec2.internal 48m Normal Scheduled pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-h7dlg Successfully assigned openshift-ingress/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-h7dlg to ip-10-0-143-15.ec2.internal 52m Normal Scheduled pod/payload-processing-e2e-worker-w6-784e7a-79d748c8bb-fbgzx Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-784e7a-79d748c8bb-fbgzx to ip-10-0-128-31.ec2.internal 21m Normal Scheduled pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-n69t2 Successfully assigned openshift-ingress/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-n69t2 to ip-10-0-143-15.ec2.internal 48m Normal Scheduled pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-lqhnf Successfully assigned openshift-ingress/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-lqhnf to ip-10-0-128-31.ec2.internal 27m Normal Scheduled pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-j4gn7 Successfully assigned openshift-ingress/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-j4gn7 to ip-10-0-128-31.ec2.internal 47m Normal Scheduled pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-nr58r Successfully assigned openshift-ingress/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-nr58r to ip-10-0-143-15.ec2.internal 41m Normal Scheduled pod/e2e-api-b-3d1777-openshift-default-56f984b757-knrnv Successfully assigned openshift-ingress/e2e-api-b-3d1777-openshift-default-56f984b757-knrnv to ip-10-0-138-2.ec2.internal 31m Normal Scheduled pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-6xvfq Successfully assigned openshift-ingress/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-6xvfq to ip-10-0-138-2.ec2.internal 52m Normal Scheduled pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-t8fq4 Successfully assigned openshift-ingress/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-t8fq4 to ip-10-0-128-31.ec2.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w5-6fc6bd-78c4d8b54c-qg69n Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w5-6fc6bd-78c4d8b54c-qg69n to ip-10-0-138-2.ec2.internal 50m Normal Scheduled pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-vjfcn Successfully assigned openshift-ingress/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-vjfcn to ip-10-0-138-2.ec2.internal 23m Normal Scheduled pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-sgz2w Successfully assigned openshift-ingress/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-sgz2w to ip-10-0-128-31.ec2.internal 52m Normal Scheduled pod/payload-processing-e2e-worker-w5-22d634-7db4fc9554-56z6p Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-22d634-7db4fc9554-56z6p to ip-10-0-128-31.ec2.internal 49m Normal Scheduled pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-w9vdp Successfully assigned openshift-ingress/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-w9vdp to ip-10-0-143-15.ec2.internal 28m Normal Scheduled pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-kwffs Successfully assigned openshift-ingress/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-kwffs to ip-10-0-138-2.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w6-db7419-79447f7f59-h9nmc Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w6-db7419-79447f7f59-h9nmc to ip-10-0-128-31.ec2.internal 22m Normal Scheduled pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-bkj9z Successfully assigned openshift-ingress/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-bkj9z to ip-10-0-143-15.ec2.internal 44m Normal Scheduled pod/e2e-shared-a-w6-06a95f-openshift-default-74f57588cc-xlqvh Successfully assigned openshift-ingress/e2e-shared-a-w6-06a95f-openshift-default-74f57588cc-xlqvh to ip-10-0-138-2.ec2.internal 21m Normal Scheduled pod/e2e-ipp-a-83f120-openshift-default-78b78cf899-x822r Successfully assigned openshift-ingress/e2e-ipp-a-83f120-openshift-default-78b78cf899-x822r to ip-10-0-138-2.ec2.internal 27m Normal Scheduled pod/e2e-shared-a-w6-06a95f-openshift-default-74f57588cc-f4bjj Successfully assigned openshift-ingress/e2e-shared-a-w6-06a95f-openshift-default-74f57588cc-f4bjj to ip-10-0-128-31.ec2.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-c296a3-778459576c-5mnqg Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-c296a3-778459576c-5mnqg to ip-10-0-138-2.ec2.internal 41m Normal Scheduled pod/e2e-api-b-aeda8c-openshift-default-57d7447bc-6nmrv Successfully assigned openshift-ingress/e2e-api-b-aeda8c-openshift-default-57d7447bc-6nmrv to ip-10-0-128-31.ec2.internal 50m Normal Scheduled pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-4jtkn Successfully assigned openshift-ingress/e2e-worker-w5-22d634-openshift-default-89dbbf78f-4jtkn to ip-10-0-128-31.ec2.internal 46m Normal Scheduled pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-bb657 Successfully assigned openshift-ingress/e2e-worker-w5-22d634-openshift-default-89dbbf78f-bb657 to ip-10-0-143-15.ec2.internal 52m Normal Scheduled pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-hz9sp Successfully assigned openshift-ingress/e2e-worker-w5-22d634-openshift-default-89dbbf78f-hz9sp to ip-10-0-138-2.ec2.internal 28m Normal Scheduled pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-r9lfc Successfully assigned openshift-ingress/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-r9lfc to ip-10-0-138-2.ec2.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-8d985a-6b59cc474c-5vth4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-8d985a-6b59cc474c-5vth4 to ip-10-0-128-31.ec2.internal 52m Normal Scheduled pod/payload-processing-e2e-worker-w4-08ebee-5f548676d4-st67q Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-08ebee-5f548676d4-st67q to ip-10-0-128-31.ec2.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-8147a8-7f7ff5cc5-4fxdk Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-8147a8-7f7ff5cc5-4fxdk to ip-10-0-128-31.ec2.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-2a3ef0-8648759887-mhb5v Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-2a3ef0-8648759887-mhb5v to ip-10-0-138-2.ec2.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-05417d-5cbcfc559-jjksv Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-05417d-5cbcfc559-jjksv to ip-10-0-128-31.ec2.internal 37m Normal Scheduled pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-fhq59 Successfully assigned openshift-ingress/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-fhq59 to ip-10-0-128-31.ec2.internal 49m Normal Scheduled pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-nbn5j Successfully assigned openshift-ingress/e2e-worker-w5-22d634-openshift-default-89dbbf78f-nbn5j to ip-10-0-143-15.ec2.internal 25m Normal Scheduled pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-2t2kf Successfully assigned openshift-ingress/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-2t2kf to ip-10-0-143-15.ec2.internal 46m Normal Scheduled pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-p6swd Successfully assigned openshift-ingress/e2e-worker-w5-22d634-openshift-default-89dbbf78f-p6swd to ip-10-0-128-31.ec2.internal 46m Normal Scheduled pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-q5z2q Successfully assigned openshift-ingress/e2e-worker-w5-22d634-openshift-default-89dbbf78f-q5z2q to ip-10-0-138-2.ec2.internal 49m Normal Scheduled pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-wsjz8 Successfully assigned openshift-ingress/e2e-worker-w5-22d634-openshift-default-89dbbf78f-wsjz8 to ip-10-0-138-2.ec2.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-fee294-c499455c5-ds72t Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-fee294-c499455c5-ds72t to ip-10-0-128-31.ec2.internal 45m Normal Scheduled pod/e2e-derive-6c6e65e5-openshift-default-7586c4c97c-p8csz Successfully assigned openshift-ingress/e2e-derive-6c6e65e5-openshift-default-7586c4c97c-p8csz to ip-10-0-128-31.ec2.internal 23m Normal Scheduled pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-x9bxc Successfully assigned openshift-ingress/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-x9bxc to ip-10-0-138-2.ec2.internal 5m25s Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-063169-78d58mpvtg Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-063169-78d58mpvtg to ip-10-0-128-31.ec2.internal 26m Normal Scheduled pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-vpr7b Successfully assigned openshift-ingress/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-vpr7b to ip-10-0-128-31.ec2.internal 39m Normal Scheduled pod/e2e-inf-b-c296a3-openshift-default-79f7f78789-fdkc4 Successfully assigned openshift-ingress/e2e-inf-b-c296a3-openshift-default-79f7f78789-fdkc4 to ip-10-0-138-2.ec2.internal 23m Normal Scheduled pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-rt6nd Successfully assigned openshift-ingress/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-rt6nd to ip-10-0-143-15.ec2.internal 45m Normal Scheduled pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-4h4z5 Successfully assigned openshift-ingress/e2e-worker-w6-784e7a-openshift-default-595898bd5c-4h4z5 to ip-10-0-128-31.ec2.internal 27m Normal Scheduled pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-ns4gv Successfully assigned openshift-ingress/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-ns4gv to ip-10-0-128-31.ec2.internal 45m Normal Scheduled pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-8cvvq Successfully assigned openshift-ingress/e2e-worker-w6-784e7a-openshift-default-595898bd5c-8cvvq to ip-10-0-128-31.ec2.internal 23m Normal Scheduled pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-n2ttv Successfully assigned openshift-ingress/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-n2ttv to ip-10-0-138-2.ec2.internal 50m Normal Scheduled pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-kw5tw Successfully assigned openshift-ingress/e2e-worker-w6-784e7a-openshift-default-595898bd5c-kw5tw to ip-10-0-138-2.ec2.internal 37m Normal Scheduled pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-4fcgv Successfully assigned openshift-ingress/e2e-inf-a-4d3168-openshift-default-96c48bc6c-4fcgv to ip-10-0-128-31.ec2.internal 52m Normal Scheduled pod/payload-processing-e2e-worker-w3-43b37d-fd685c7db-qkngc Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-43b37d-fd685c7db-qkngc to ip-10-0-128-31.ec2.internal 27m Normal Scheduled pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-mvw5s Successfully assigned openshift-ingress/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-mvw5s to ip-10-0-143-15.ec2.internal 45m Normal Scheduled pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-lx7p6 Successfully assigned openshift-ingress/e2e-worker-w6-784e7a-openshift-default-595898bd5c-lx7p6 to ip-10-0-143-15.ec2.internal 31m Normal Scheduled pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-mb2dl Successfully assigned openshift-ingress/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-mb2dl to ip-10-0-138-2.ec2.internal 46m Normal Scheduled pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-nkds4 Successfully assigned openshift-ingress/e2e-worker-w6-784e7a-openshift-default-595898bd5c-nkds4 to ip-10-0-128-31.ec2.internal 26m Normal Scheduled pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-glsxl Successfully assigned openshift-ingress/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-glsxl to ip-10-0-143-15.ec2.internal 45m Normal Scheduled pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-pgwf2 Successfully assigned openshift-ingress/e2e-worker-w6-784e7a-openshift-default-595898bd5c-pgwf2 to ip-10-0-138-2.ec2.internal 36m Normal Scheduled pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-4g7sz Successfully assigned openshift-ingress/e2e-inf-a-4d3168-openshift-default-96c48bc6c-4g7sz to ip-10-0-143-15.ec2.internal 26m Normal Scheduled pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-ctqgd Successfully assigned openshift-ingress/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-ctqgd to ip-10-0-138-2.ec2.internal 52m Normal Scheduled pod/payload-processing-e2e-worker-w2-2ba10a-78fb8d6fd4-qw5lv Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-2ba10a-78fb8d6fd4-qw5lv to ip-10-0-128-31.ec2.internal 46m Normal Scheduled pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-r7bbc Successfully assigned openshift-ingress/e2e-worker-w6-784e7a-openshift-default-595898bd5c-r7bbc to ip-10-0-143-15.ec2.internal 27m Normal Scheduled pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-bn6sm Successfully assigned openshift-ingress/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-bn6sm to ip-10-0-128-31.ec2.internal 49m Normal Scheduled pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-ssfqt Successfully assigned openshift-ingress/e2e-worker-w6-784e7a-openshift-default-595898bd5c-ssfqt to ip-10-0-143-15.ec2.internal 52m Normal Scheduled pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-tf78l Successfully assigned openshift-ingress/e2e-worker-w6-784e7a-openshift-default-595898bd5c-tf78l to ip-10-0-128-31.ec2.internal 8m53s Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-e5236b-7548fnkkbf Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-e5236b-7548fnkkbf to ip-10-0-128-31.ec2.internal 41m Normal Scheduled pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-l4blm Successfully assigned openshift-ingress/e2e-inf-a-4d3168-openshift-default-96c48bc6c-l4blm to ip-10-0-138-2.ec2.internal 28m Normal Scheduled pod/e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9-x9gf6 Successfully assigned openshift-ingress/e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9-x9gf6 to ip-10-0-128-31.ec2.internal 44m Normal Scheduled pod/e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9-qmc4m Successfully assigned openshift-ingress/e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9-qmc4m to ip-10-0-138-2.ec2.internal 49m Normal Scheduled pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-vwlks Successfully assigned openshift-ingress/e2e-worker-w6-784e7a-openshift-default-595898bd5c-vwlks to ip-10-0-138-2.ec2.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-2e046a-74bbd64fc9-7nrm7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-2e046a-74bbd64fc9-7nrm7 to ip-10-0-138-2.ec2.internal 52m Normal Scheduled pod/payload-processing-e2e-worker-w1-fc684b-5db88966b-rwg2s Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-fc684b-5db88966b-rwg2s to ip-10-0-138-2.ec2.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-c7d192-fcf758f79-fd7hp Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-c7d192-fcf758f79-fd7hp to ip-10-0-138-2.ec2.internal 29m Normal Scheduled pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-mshrv Successfully assigned openshift-ingress/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-mshrv to ip-10-0-138-2.ec2.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-c1750b-65b7cbb459-gt6wn Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-c1750b-65b7cbb459-gt6wn to ip-10-0-128-31.ec2.internal 25m Normal Scheduled pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-mbzm5 Successfully assigned openshift-ingress/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-mbzm5 to ip-10-0-128-31.ec2.internal 60m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-szfl8 Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-szfl8 to ip-10-0-138-2.ec2.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-a5af18-84d9b694d7-lzhcq Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-a5af18-84d9b694d7-lzhcq to ip-10-0-128-31.ec2.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-5a69c9-95cc9866c-bsqgg Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-5a69c9-95cc9866c-bsqgg to ip-10-0-128-31.ec2.internal 25m Normal Scheduled pod/e2e-ipp-a-8c4b02-openshift-default-679994b96f-7wssh Successfully assigned openshift-ingress/e2e-ipp-a-8c4b02-openshift-default-679994b96f-7wssh to ip-10-0-128-31.ec2.internal 37m Normal Scheduled pod/e2e-inf-b-c296a3-openshift-default-79f7f78789-6gwx9 Successfully assigned openshift-ingress/e2e-inf-b-c296a3-openshift-default-79f7f78789-6gwx9 to ip-10-0-128-31.ec2.internal 52m Normal Scheduled pod/payload-processing-e2e-worker-w0-2e046a-6888946c9d-w2lbg Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-2e046a-6888946c9d-w2lbg to ip-10-0-128-31.ec2.internal 8m53s Normal Scheduled pod/payload-processing-e2e-worker-single-main-e5236b-fcf66d4f44xjqg Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-e5236b-fcf66d4f44xjqg to ip-10-0-128-31.ec2.internal 5m25s Normal Scheduled pod/payload-processing-e2e-worker-single-main-063169-fbc96c98b5zgjw Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-063169-fbc96c98b5zgjw to ip-10-0-128-31.ec2.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-4d3168-5cf56f8fd7-dbn2k Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-4d3168-5cf56f8fd7-dbn2k to ip-10-0-138-2.ec2.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-derive-6c6e65e5-78df9fdc95-qrgvn Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-6c6e65e5-78df9fdc95-qrgvn to ip-10-0-138-2.ec2.internal 23m Normal Scheduled pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-8j4dd Successfully assigned openshift-ingress/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-8j4dd to ip-10-0-143-15.ec2.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-fc684b-7fb5fd79c5-hb2rn Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-fc684b-7fb5fd79c5-hb2rn to ip-10-0-128-31.ec2.internal 22m Normal Scheduled pod/e2e-ipp-a-b8f54c-openshift-default-69ccbf5c6c-mwkmt Successfully assigned openshift-ingress/e2e-ipp-a-b8f54c-openshift-default-69ccbf5c6c-mwkmt to ip-10-0-138-2.ec2.internal 37m Normal Scheduled pod/e2e-inf-a-5a69c9-openshift-default-cbff5c97-bfffh Successfully assigned openshift-ingress/e2e-inf-a-5a69c9-openshift-default-cbff5c97-bfffh to ip-10-0-138-2.ec2.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-api-b-aeda8c-5cc9444875-zqf4n Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-aeda8c-5cc9444875-zqf4n to ip-10-0-128-31.ec2.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-api-b-3d1777-5d6bcd9959-52r9w Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-3d1777-5d6bcd9959-52r9w to ip-10-0-138-2.ec2.internal 45m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-267vm Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-267vm to ip-10-0-143-15.ec2.internal 44m Normal Scheduled pod/payload-processing-e2e-shared-b-w6-db7419-84c96f7884-pqpn2 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w6-db7419-84c96f7884-pqpn2 to ip-10-0-128-31.ec2.internal 36m Normal Scheduled pod/payload-processing-e2e-shared-b-w5-6fc6bd-cf68d45f8-22lvh Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w5-6fc6bd-cf68d45f8-22lvh to ip-10-0-138-2.ec2.internal 29m Normal Scheduled pod/payload-processing-e2e-shared-b-w4-c933cb-fc7cf9456-g8t8z Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-c933cb-fc7cf9456-g8t8z to ip-10-0-138-2.ec2.internal 19m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-9jn89 Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9jn89 to ip-10-0-143-15.ec2.internal 32m Normal Scheduled pod/e2e-inf-a-5a69c9-openshift-default-cbff5c97-dkq5d Successfully assigned openshift-ingress/e2e-inf-a-5a69c9-openshift-default-cbff5c97-dkq5d to ip-10-0-128-31.ec2.internal 59m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-bhpfs Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bhpfs to ip-10-0-138-2.ec2.internal 27m Normal Scheduled pod/e2e-ipp-a-c87bf1-openshift-default-648d4845d9-qx65t Successfully assigned openshift-ingress/e2e-ipp-a-c87bf1-openshift-default-648d4845d9-qx65t to ip-10-0-138-2.ec2.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-api-b-39322f-6bd999dc7f-dcj8q Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-39322f-6bd999dc7f-dcj8q to ip-10-0-138-2.ec2.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-api-a-ee886f-5979bf8887-hr469 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-ee886f-5979bf8887-hr469 to ip-10-0-128-31.ec2.internal 23m Normal Scheduled pod/e2e-ipp-b-4796a9-openshift-default-75cd4d45f7-gm2vv Successfully assigned openshift-ingress/e2e-ipp-b-4796a9-openshift-default-75cd4d45f7-gm2vv to ip-10-0-138-2.ec2.internal 43m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-z794f Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-z794f to ip-10-0-128-31.ec2.internal 43m Normal Scheduled pod/payload-processing-e2e-shared-b-w3-3bd0f6-c58c9678-lhc7t Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-3bd0f6-c58c9678-lhc7t to ip-10-0-128-31.ec2.internal 27m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-3ee017-5f8b7645fb-477wq Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-3ee017-5f8b7645fb-477wq to ip-10-0-138-2.ec2.internal 31m Normal Scheduled pod/payload-processing-e2e-shared-b-w1-d68c2c-5c949bdf9b-52x5q Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w1-d68c2c-5c949bdf9b-52x5q to ip-10-0-138-2.ec2.internal 28m Normal Scheduled pod/payload-processing-e2e-shared-b-w0-02720a-78cdf6c4cd-w4jmh Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-02720a-78cdf6c4cd-w4jmh to ip-10-0-138-2.ec2.internal 21m Normal Scheduled pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-wkhfz Successfully assigned openshift-ingress/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-wkhfz to ip-10-0-143-15.ec2.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-2ba10a-6d55f55df9-gfdpx Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-2ba10a-6d55f55df9-gfdpx to ip-10-0-138-2.ec2.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-api-a-cae55b-84d5658d5f-gzwp2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-cae55b-84d5658d5f-gzwp2 to ip-10-0-138-2.ec2.internal 25m Normal Scheduled pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-wk57v Successfully assigned openshift-ingress/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-wk57v to ip-10-0-138-2.ec2.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-api-a-616da7-7747fb4947-5lbcj Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-616da7-7747fb4947-5lbcj to ip-10-0-138-2.ec2.internal 19m Normal Scheduled pod/e2e-ipp-b-63e89c-openshift-default-6d97c8847c-8bqfm Successfully assigned openshift-ingress/e2e-ipp-b-63e89c-openshift-default-6d97c8847c-8bqfm to ip-10-0-128-31.ec2.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-ait-f7dea32d-79976f6559-8pgqr Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-f7dea32d-79976f6559-8pgqr to ip-10-0-128-31.ec2.internal 32m Normal Scheduled pod/e2e-inf-a-a5af18-openshift-default-58d79f487f-7zgh9 Successfully assigned openshift-ingress/e2e-inf-a-a5af18-openshift-default-58d79f487f-7zgh9 to ip-10-0-138-2.ec2.internal 44m Normal Scheduled pod/payload-processing-e2e-shared-a-w6-06a95f-6c459f96db-6pb7q Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w6-06a95f-6c459f96db-6pb7q to ip-10-0-138-2.ec2.internal 37m Normal Scheduled pod/payload-processing-e2e-shared-a-w5-9aafe6-788454f66b-t8w2h Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w5-9aafe6-788454f66b-t8w2h to ip-10-0-138-2.ec2.internal 31m Normal Scheduled pod/payload-processing-e2e-shared-a-w4-f8ca96-576b496c86-rkx2m Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-f8ca96-576b496c86-rkx2m to ip-10-0-128-31.ec2.internal 20m Normal Scheduled pod/e2e-ipp-b-75b68d-openshift-default-5f58db85f7-74lsn Successfully assigned openshift-ingress/e2e-ipp-b-75b68d-openshift-default-5f58db85f7-74lsn to ip-10-0-138-2.ec2.internal 39m Normal Scheduled pod/e2e-inf-b-8d985a-openshift-default-856f5f5bf5-w2k2c Successfully assigned openshift-ingress/e2e-inf-b-8d985a-openshift-default-856f5f5bf5-w2k2c to ip-10-0-128-31.ec2.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-ait-dc369ed0-558776d6ff-bhzch Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-dc369ed0-558776d6ff-bhzch to ip-10-0-128-31.ec2.internal 21m Normal Scheduled pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-ftzdt Successfully assigned openshift-ingress/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-ftzdt to ip-10-0-128-31.ec2.internal 60m Normal Scheduled pod/kube-auth-proxy-7b84b87b7c-gk9x6 Successfully assigned openshift-ingress/kube-auth-proxy-7b84b87b7c-gk9x6 to ip-10-0-138-2.ec2.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-43b37d-74d9bdb487-qtfzw Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-43b37d-74d9bdb487-qtfzw to ip-10-0-128-31.ec2.internal 38m Normal Scheduled pod/e2e-inf-a-a5af18-openshift-default-58d79f487f-tw782 Successfully assigned openshift-ingress/e2e-inf-a-a5af18-openshift-default-58d79f487f-tw782 to ip-10-0-128-31.ec2.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-08ebee-7bcfbb7487-9x5xj Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-08ebee-7bcfbb7487-9x5xj to ip-10-0-138-2.ec2.internal 44m Normal Scheduled pod/payload-processing-e2e-shared-a-w3-aaa3c6-845674cd68-8fgr7 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-aaa3c6-845674cd68-8fgr7 to ip-10-0-128-31.ec2.internal 29m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-eb02df-7b6dfb5df8-sw5mn Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-eb02df-7b6dfb5df8-sw5mn to ip-10-0-138-2.ec2.internal 33m Normal Scheduled pod/payload-processing-e2e-shared-a-w1-af260f-fd896cf88-rb5bz Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w1-af260f-fd896cf88-rb5bz to ip-10-0-138-2.ec2.internal 29m Normal Scheduled pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-q8qxp Successfully assigned openshift-ingress/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-q8qxp to ip-10-0-138-2.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-7b98bb678c-64t9f Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-64t9f to ip-10-0-128-31.ec2.internal 37m Normal Scheduled pod/e2e-inf-b-8147a8-openshift-default-855c99786c-k9ckd Successfully assigned openshift-ingress/e2e-inf-b-8147a8-openshift-default-855c99786c-k9ckd to ip-10-0-138-2.ec2.internal 23m Normal Scheduled pod/e2e-ipp-b-85862e-openshift-default-7b794c9f49-6wxwm Successfully assigned openshift-ingress/e2e-ipp-b-85862e-openshift-default-7b794c9f49-6wxwm to ip-10-0-138-2.ec2.internal 60m Normal Scheduled pod/kube-auth-proxy-7b84b87b7c-vx6bc Successfully assigned openshift-ingress/kube-auth-proxy-7b84b87b7c-vx6bc to ip-10-0-128-31.ec2.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-22d634-6768765995-phnhr Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-22d634-6768765995-phnhr to ip-10-0-128-31.ec2.internal 33m Normal Scheduled pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-cxjmw Successfully assigned openshift-ingress/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-cxjmw to ip-10-0-128-31.ec2.internal 16m Normal Scheduled pod/e2e-ipp-b-9d6d4e-openshift-default-5b7b554885-5j85l Successfully assigned openshift-ingress/e2e-ipp-b-9d6d4e-openshift-default-5b7b554885-5j85l to ip-10-0-128-31.ec2.internal 33m Normal Scheduled pod/e2e-inf-b-8147a8-openshift-default-855c99786c-4sv5h Successfully assigned openshift-ingress/e2e-inf-b-8147a8-openshift-default-855c99786c-4sv5h to ip-10-0-128-31.ec2.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-784e7a-5bd4bc7d47-cddfw Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-784e7a-5bd4bc7d47-cddfw to ip-10-0-128-31.ec2.internal 43m Normal Scheduled pod/payload-processing-e2e-mt-1fa4b8b6-68f88d6f68-c9869 Successfully assigned openshift-ingress/payload-processing-e2e-mt-1fa4b8b6-68f88d6f68-c9869 to ip-10-0-138-2.ec2.internal 40m Normal Scheduled pod/payload-processing-e2e-mt-1fa4b8b6-68f88d6f68-2sqfg Successfully assigned openshift-ingress/payload-processing-e2e-mt-1fa4b8b6-68f88d6f68-2sqfg to ip-10-0-128-31.ec2.internal 53m Normal Scheduled pod/payload-processing-5f78687d5b-8wn8h Successfully assigned openshift-ingress/payload-processing-5f78687d5b-8wn8h to ip-10-0-128-31.ec2.internal 40m Normal Scheduled pod/e2e-inf-a-c1750b-openshift-default-85d9759655-956xt Successfully assigned openshift-ingress/e2e-inf-a-c1750b-openshift-default-85d9759655-956xt to ip-10-0-138-2.ec2.internal 31m Normal Scheduled pod/e2e-inf-b-2a3ef0-openshift-default-5cdf89c9-rvbgs Successfully assigned openshift-ingress/e2e-inf-b-2a3ef0-openshift-default-5cdf89c9-rvbgs to ip-10-0-128-31.ec2.internal 46m Normal Scheduled pod/payload-processing-e2e-ait-dc369ed0-5b7fd658fd-pj5gf Successfully assigned openshift-ingress/payload-processing-e2e-ait-dc369ed0-5b7fd658fd-pj5gf to ip-10-0-138-2.ec2.internal 17m Normal Scheduled pod/e2e-ipp-b-9d6d4e-openshift-default-5b7b554885-kvnwl Successfully assigned openshift-ingress/e2e-ipp-b-9d6d4e-openshift-default-5b7b554885-kvnwl to ip-10-0-138-2.ec2.internal 13m Normal Scheduled pod/maas-default-gateway-openshift-default-5db77fbd9f-zzv89 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-zzv89 to ip-10-0-128-31.ec2.internal 43m Normal Scheduled pod/maas-default-gateway-openshift-default-5db77fbd9f-z7hjw Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-z7hjw to ip-10-0-138-2.ec2.internal 42m Normal Scheduled pod/maas-default-gateway-openshift-default-5db77fbd9f-xvdlm Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-xvdlm to ip-10-0-128-31.ec2.internal 46m Normal Scheduled pod/payload-processing-e2e-ait-f7dea32d-747d8fcc7d-42zlx Successfully assigned openshift-ingress/payload-processing-e2e-ait-f7dea32d-747d8fcc7d-42zlx to ip-10-0-128-31.ec2.internal 29m Normal Scheduled pod/payload-processing-e2e-shared-a-w0-920032-77dc47dc96-xh765 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-920032-77dc47dc96-xh765 to ip-10-0-138-2.ec2.internal 46m Normal Scheduled pod/e2e-ait-dc369ed0-openshift-default-858c684c57-npckw Successfully assigned openshift-ingress/e2e-ait-dc369ed0-openshift-default-858c684c57-npckw to ip-10-0-138-2.ec2.internal 43m Normal Scheduled pod/maas-default-gateway-openshift-default-5db77fbd9f-2qx7m Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2qx7m to ip-10-0-143-15.ec2.internal 44m Normal Scheduled pod/payload-processing-e2e-mt-1131a29d-6d7ccc5c4d-mdml5 Successfully assigned openshift-ingress/payload-processing-e2e-mt-1131a29d-6d7ccc5c4d-mdml5 to ip-10-0-128-31.ec2.internal 57m Normal Scheduled pod/maas-default-gateway-openshift-default-5db77fbd9f-8d8sf Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-8d8sf to ip-10-0-143-15.ec2.internal 13m Normal Scheduled pod/maas-default-gateway-openshift-default-5db77fbd9f-vkx5n Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-vkx5n to ip-10-0-128-31.ec2.internal 36m Normal Scheduled pod/e2e-inf-b-2a3ef0-openshift-default-5cdf89c9-fwcr6 Successfully assigned openshift-ingress/e2e-inf-b-2a3ef0-openshift-default-5cdf89c9-fwcr6 to ip-10-0-138-2.ec2.internal 33m Normal Scheduled pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-4krkx Successfully assigned openshift-ingress/e2e-inf-a-c7d192-openshift-default-c4f554bb7-4krkx to ip-10-0-138-2.ec2.internal 25m Normal Scheduled pod/e2e-ipp-b-a962d7-openshift-default-6dbc7cd94c-625mm Successfully assigned openshift-ingress/e2e-ipp-b-a962d7-openshift-default-6dbc7cd94c-625mm to ip-10-0-138-2.ec2.internal 12m Normal Scheduled pod/maas-default-gateway-openshift-default-5db77fbd9f-tm9cb Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-tm9cb to ip-10-0-138-2.ec2.internal 43m Normal Scheduled pod/maas-default-gateway-openshift-default-5db77fbd9f-r8hqd Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-r8hqd to ip-10-0-138-2.ec2.internal 25m Normal Scheduled pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-jzqt4 Successfully assigned openshift-ingress/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-jzqt4 to ip-10-0-128-31.ec2.internal 45m Normal Scheduled pod/e2e-migrate-e385da9b-gw-openshift-default-6d84d6474f-mljgv Successfully assigned openshift-ingress/e2e-migrate-e385da9b-gw-openshift-default-6d84d6474f-mljgv to ip-10-0-138-2.ec2.internal 41m Normal Scheduled pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-d4sqk Successfully assigned openshift-ingress/e2e-mt-1131a29d-openshift-default-54ccb47c65-d4sqk to ip-10-0-143-15.ec2.internal 45m Normal Scheduled pod/payload-processing-e2e-migrate-e385da9b-78d97b4574-h5mcr Successfully assigned openshift-ingress/payload-processing-e2e-migrate-e385da9b-78d97b4574-h5mcr to ip-10-0-128-31.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-b-a962d7-7cf57bb744-49pdj Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-a962d7-7cf57bb744-49pdj to ip-10-0-138-2.ec2.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-b-a962d7-55d9d5c8c4-dfl49 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-a962d7-55d9d5c8c4-dfl49 to ip-10-0-128-31.ec2.internal 36m Normal Scheduled pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-5gpj4 Successfully assigned openshift-ingress/e2e-inf-a-c7d192-openshift-default-c4f554bb7-5gpj4 to ip-10-0-128-31.ec2.internal 16m Normal Scheduled pod/payload-processing-e2e-ipp-b-9d6d4e-65f69f95b8-qb57q Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-9d6d4e-65f69f95b8-qb57q to ip-10-0-138-2.ec2.internal 17m Normal Scheduled pod/payload-processing-e2e-ipp-b-9d6d4e-5c49d979db-z5zzn Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-9d6d4e-5c49d979db-z5zzn to ip-10-0-128-31.ec2.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-b-85862e-69678cb8f6-hkxqw Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-85862e-69678cb8f6-hkxqw to ip-10-0-138-2.ec2.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-b-85862e-5759d9df58-t4vrn Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-85862e-5759d9df58-t4vrn to ip-10-0-128-31.ec2.internal 18m Normal Scheduled pod/payload-processing-e2e-ipp-b-75b68d-ddbc7885b-wmsp7 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-75b68d-ddbc7885b-wmsp7 to ip-10-0-138-2.ec2.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-b-75b68d-9f999d45d-9qwbt Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-75b68d-9f999d45d-9qwbt to ip-10-0-138-2.ec2.internal 19m Normal Scheduled pod/payload-processing-e2e-ipp-b-63e89c-745667d778-c8x5k Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-63e89c-745667d778-c8x5k to ip-10-0-138-2.ec2.internal 17m Normal Scheduled pod/payload-processing-e2e-ipp-b-63e89c-68b459b96-7hqcv Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-63e89c-68b459b96-7hqcv to ip-10-0-138-2.ec2.internal 21m Normal Scheduled pod/payload-processing-e2e-ipp-b-4796a9-7fc7d467d8-nkkxj Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-4796a9-7fc7d467d8-nkkxj to ip-10-0-138-2.ec2.internal 36m Normal Scheduled pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-cjl27 Successfully assigned openshift-ingress/e2e-inf-a-c7d192-openshift-default-c4f554bb7-cjl27 to ip-10-0-143-15.ec2.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-b-4796a9-6c55db49cd-kkpxj Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-4796a9-6c55db49cd-kkpxj to ip-10-0-128-31.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-a-c87bf1-85b4d8ffd8-jz456 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-c87bf1-85b4d8ffd8-jz456 to ip-10-0-143-15.ec2.internal 27m Normal Scheduled pod/payload-processing-e2e-ipp-a-c87bf1-5cf8c8df7d-xwbvk Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-c87bf1-5cf8c8df7d-xwbvk to ip-10-0-138-2.ec2.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-a-b8f54c-77b9597456-2htkw Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-b8f54c-77b9597456-2htkw to ip-10-0-138-2.ec2.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-a-b8f54c-6dd8f8fcbb-8v49p Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-b8f54c-6dd8f8fcbb-8v49p to ip-10-0-138-2.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-a-8c4b02-86d6df458-f5kjj Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-8c4b02-86d6df458-f5kjj to ip-10-0-128-31.ec2.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-a-8c4b02-68c646b4dd-rx4cf Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-8c4b02-68c646b4dd-rx4cf to ip-10-0-138-2.ec2.internal 19m Normal Scheduled pod/payload-processing-e2e-ipp-a-83f120-f467f9994-b9pr9 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-83f120-f467f9994-b9pr9 to ip-10-0-128-31.ec2.internal 21m Normal Scheduled pod/payload-processing-e2e-ipp-a-83f120-59d8965d64-mksct Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-83f120-59d8965d64-mksct to ip-10-0-128-31.ec2.internal 35m Normal Scheduled pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-kl464 Successfully assigned openshift-ingress/e2e-inf-a-c7d192-openshift-default-c4f554bb7-kl464 to ip-10-0-138-2.ec2.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-a-80248b-79b8c55464-d7k5d Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-80248b-79b8c55464-d7k5d to ip-10-0-138-2.ec2.internal 18m Normal Scheduled pod/payload-processing-e2e-ipp-a-80248b-5dfc48b658-bwlk6 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-80248b-5dfc48b658-bwlk6 to ip-10-0-138-2.ec2.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-a-7be71d-85b85b8ffd-d77xb Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-7be71d-85b85b8ffd-d77xb to ip-10-0-128-31.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-a-7be71d-59d98d4c64-5gtp5 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-7be71d-59d98d4c64-5gtp5 to ip-10-0-138-2.ec2.internal 36m Normal Scheduled pod/payload-processing-e2e-inf-b-e619b6-7f44c56884-ctrpq Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-e619b6-7f44c56884-ctrpq to ip-10-0-128-31.ec2.internal 39m Normal Scheduled pod/payload-processing-e2e-inf-b-c296a3-544676fc5d-fvr68 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-c296a3-544676fc5d-fvr68 to ip-10-0-128-31.ec2.internal 39m Normal Scheduled pod/payload-processing-e2e-inf-b-8d985a-78b6b6596-5tj4s Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-8d985a-78b6b6596-5tj4s to ip-10-0-128-31.ec2.internal 36m Normal Scheduled pod/payload-processing-e2e-inf-b-8147a8-f8bf948b4-575w2 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-8147a8-f8bf948b4-575w2 to ip-10-0-128-31.ec2.internal 36m Normal Scheduled pod/payload-processing-e2e-inf-b-2a3ef0-9d775496-fs4gv Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-2a3ef0-9d775496-fs4gv to ip-10-0-138-2.ec2.internal 33m Normal Scheduled pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-nbz7s Successfully assigned openshift-ingress/e2e-inf-a-c7d192-openshift-default-c4f554bb7-nbz7s to ip-10-0-143-15.ec2.internal 36m Normal Scheduled pod/payload-processing-e2e-inf-b-05417d-754c7b55dd-kclz4 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-05417d-754c7b55dd-kclz4 to ip-10-0-138-2.ec2.internal 38m Normal Scheduled pod/payload-processing-e2e-inf-a-fee294-79797dc7c4-4q6tx Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-fee294-79797dc7c4-4q6tx to ip-10-0-128-31.ec2.internal 37m Normal Scheduled pod/payload-processing-e2e-inf-a-c7d192-7f56447bb4-wq8sm Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-c7d192-7f56447bb4-wq8sm to ip-10-0-138-2.ec2.internal 44m Normal Scheduled pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-fmfnq Successfully assigned openshift-ingress/e2e-mt-1131a29d-openshift-default-54ccb47c65-fmfnq to ip-10-0-138-2.ec2.internal 12m Normal Scheduled pod/maas-default-gateway-openshift-default-5db77fbd9f-96ppp Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-96ppp to ip-10-0-128-31.ec2.internal 40m Normal Scheduled pod/payload-processing-e2e-inf-a-c1750b-85f79d758d-n8sxl Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-c1750b-85f79d758d-n8sxl to ip-10-0-128-31.ec2.internal 37m Normal Scheduled pod/payload-processing-e2e-inf-a-a5af18-5786f6f7f6-v8znz Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-a5af18-5786f6f7f6-v8znz to ip-10-0-128-31.ec2.internal 37m Normal Scheduled pod/payload-processing-e2e-inf-a-5a69c9-7779f4d998-xrblq Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-5a69c9-7779f4d998-xrblq to ip-10-0-138-2.ec2.internal 12m Normal Scheduled pod/maas-default-gateway-openshift-default-5db77fbd9f-9g4hg Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-9g4hg to ip-10-0-138-2.ec2.internal 38m Normal Scheduled pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-rgzsj Successfully assigned openshift-ingress/e2e-inf-a-c7d192-openshift-default-c4f554bb7-rgzsj to ip-10-0-128-31.ec2.internal 43m Normal Scheduled pod/maas-default-gateway-openshift-default-5db77fbd9f-cr7nq Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-cr7nq to ip-10-0-128-31.ec2.internal 24m Normal Scheduled pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-jmfgs Successfully assigned openshift-ingress/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-jmfgs to ip-10-0-143-15.ec2.internal 12m Normal Scheduled pod/maas-default-gateway-openshift-default-5db77fbd9f-dn5qq Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-dn5qq to ip-10-0-143-15.ec2.internal 54m Normal Scheduled pod/maas-default-gateway-openshift-default-5db77fbd9f-fhw57 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-fhw57 to ip-10-0-128-31.ec2.internal 24m Normal Scheduled pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-fzz2k Successfully assigned openshift-ingress/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-fzz2k to ip-10-0-138-2.ec2.internal 41m Normal Scheduled pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-jlnzb Successfully assigned openshift-ingress/e2e-mt-1131a29d-openshift-default-54ccb47c65-jlnzb to ip-10-0-128-31.ec2.internal 41m Normal Scheduled pod/payload-processing-e2e-inf-a-4d3168-d94579868-m6l26 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-4d3168-d94579868-m6l26 to ip-10-0-138-2.ec2.internal 36m Normal Scheduled pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-xw24m Successfully assigned openshift-ingress/e2e-inf-a-c7d192-openshift-default-c4f554bb7-xw24m to ip-10-0-138-2.ec2.internal 42m Normal Scheduled pod/payload-processing-e2e-api-a-616da7-8485684f4d-b7ww6 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-616da7-8485684f4d-b7ww6 to ip-10-0-138-2.ec2.internal 41m Normal Scheduled pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-mx7rd Successfully assigned openshift-ingress/e2e-mt-1131a29d-openshift-default-54ccb47c65-mx7rd to ip-10-0-128-31.ec2.internal 43m Normal Scheduled pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-47v7d Successfully assigned openshift-ingress/e2e-mt-1fa4b8b6-openshift-default-6765496769-47v7d to ip-10-0-128-31.ec2.internal 42m Normal Scheduled pod/maas-default-gateway-openshift-default-5db77fbd9f-r5b9v Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-r5b9v to ip-10-0-138-2.ec2.internal 38m Normal Scheduled pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-bmbm4 Successfully assigned openshift-ingress/e2e-mt-1fa4b8b6-openshift-default-6765496769-bmbm4 to ip-10-0-138-2.ec2.internal 36m Normal Scheduled pod/e2e-inf-b-05417d-openshift-default-6d96dbcbc5-nq8jg Successfully assigned openshift-ingress/e2e-inf-b-05417d-openshift-default-6d96dbcbc5-nq8jg to ip-10-0-128-31.ec2.internal 42m Normal Scheduled pod/maas-default-gateway-openshift-default-5db77fbd9f-hrms9 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-hrms9 to ip-10-0-143-15.ec2.internal 54m Normal Scheduled pod/maas-default-gateway-openshift-default-5db77fbd9f-k9fn7 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-k9fn7 to ip-10-0-138-2.ec2.internal 45m Normal Scheduled pod/payload-processing-e2e-derive-6c6e65e5-6c89f86574-szfbh Successfully assigned openshift-ingress/payload-processing-e2e-derive-6c6e65e5-6c89f86574-szfbh to ip-10-0-138-2.ec2.internal 13m Normal Scheduled pod/maas-default-gateway-openshift-default-5db77fbd9f-nw2zt Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-nw2zt to ip-10-0-143-15.ec2.internal 42m Normal Scheduled pod/payload-processing-e2e-api-a-cae55b-9587db5fb-8nx59 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-cae55b-9587db5fb-8nx59 to ip-10-0-138-2.ec2.internal 38m Normal Scheduled pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-mjcl9 Successfully assigned openshift-ingress/e2e-mt-1fa4b8b6-openshift-default-6765496769-mjcl9 to ip-10-0-128-31.ec2.internal 38m Normal Scheduled pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-zqtg8 Successfully assigned openshift-ingress/e2e-mt-1fa4b8b6-openshift-default-6765496769-zqtg8 to ip-10-0-138-2.ec2.internal 42m Normal Scheduled pod/payload-processing-e2e-api-a-ee886f-588fb4f8b-rqldn Successfully assigned openshift-ingress/payload-processing-e2e-api-a-ee886f-588fb4f8b-rqldn to ip-10-0-138-2.ec2.internal 54m Normal Scheduled pod/maas-default-gateway-openshift-default-5db77fbd9f-nqsf5 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-nqsf5 to ip-10-0-128-31.ec2.internal 39m Normal Scheduled pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-ms7lr Successfully assigned openshift-ingress/e2e-mt-1fa4b8b6-openshift-default-6765496769-ms7lr to ip-10-0-143-15.ec2.internal 41m Normal Scheduled pod/payload-processing-e2e-api-b-aeda8c-75cdbfc96-xx8st Successfully assigned openshift-ingress/payload-processing-e2e-api-b-aeda8c-75cdbfc96-xx8st to ip-10-0-138-2.ec2.internal 38m Normal Scheduled pod/e2e-inf-a-fee294-openshift-default-644cdf5d45-cf2vc Successfully assigned openshift-ingress/e2e-inf-a-fee294-openshift-default-644cdf5d45-cf2vc to ip-10-0-128-31.ec2.internal 13m Normal Scheduled pod/maas-default-gateway-openshift-default-5db77fbd9f-kgx4g Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-kgx4g to ip-10-0-138-2.ec2.internal 53m Normal Scheduled pod/maas-default-gateway-openshift-default-5db77fbd9f-m55mt Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-m55mt to ip-10-0-138-2.ec2.internal 38m Normal Scheduled pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-xgh4q Successfully assigned openshift-ingress/e2e-mt-1fa4b8b6-openshift-default-6765496769-xgh4q to ip-10-0-143-15.ec2.internal 41m Normal Scheduled pod/payload-processing-e2e-api-b-39322f-5c74d47cd-88276 Successfully assigned openshift-ingress/payload-processing-e2e-api-b-39322f-5c74d47cd-88276 to ip-10-0-138-2.ec2.internal 53m Normal Scheduled pod/maas-default-gateway-openshift-default-5db77fbd9f-mg4hv Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-mg4hv to ip-10-0-143-15.ec2.internal 39m Normal Scheduled pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-wwxlh Successfully assigned openshift-ingress/e2e-mt-1fa4b8b6-openshift-default-6765496769-wwxlh to ip-10-0-138-2.ec2.internal 41m Normal Scheduled pod/payload-processing-e2e-api-b-3d1777-6dfd6d6694-vmg4d Successfully assigned openshift-ingress/payload-processing-e2e-api-b-3d1777-6dfd6d6694-vmg4d to ip-10-0-138-2.ec2.internal 33m Normal Scheduled pod/e2e-inf-a-fee294-openshift-default-644cdf5d45-4dbqs Successfully assigned openshift-ingress/e2e-inf-a-fee294-openshift-default-644cdf5d45-4dbqs to ip-10-0-138-2.ec2.internal 70m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-7b6b64f55f from 0 to 1 70m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 70m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 70m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 70m Normal SuccessfulCreate replicaset/router-default-7b6b64f55f Created pod: router-default-7b6b64f55f-9l4xp 65m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 65m Warning FailedMount pod/router-default-7b6b64f55f-9l4xp MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 65m Warning FailedMount pod/router-default-7b6b64f55f-9l4xp MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 65m Warning FailedMount pod/router-default-7b6b64f55f-9l4xp MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 65m Warning FailedMount pod/router-default-7b6b64f55f-9l4xp MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 64m Normal Pulled pod/router-default-7b6b64f55f-9l4xp Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 64m Normal Started pod/router-default-7b6b64f55f-9l4xp Container started 64m Normal Created pod/router-default-7b6b64f55f-9l4xp Container created 64m Normal AddedInterface pod/router-default-7b6b64f55f-9l4xp Add eth0 [10.132.0.19/23] from ovn-kubernetes 60m Normal AddedInterface pod/kube-auth-proxy-7b84b87b7c-gk9x6 Add eth0 [10.134.0.21/23] from ovn-kubernetes 60m Normal AddedInterface pod/kube-auth-proxy-7b84b87b7c-vx6bc Add eth0 [10.132.0.32/23] from ovn-kubernetes 60m Normal Pulling pod/kube-auth-proxy-7b84b87b7c-vx6bc Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 60m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-7b84b87b7c from 0 to 2 60m Normal SuccessfulCreate replicaset/kube-auth-proxy-7b84b87b7c Created pod: kube-auth-proxy-7b84b87b7c-vx6bc 60m Normal SuccessfulCreate replicaset/kube-auth-proxy-7b84b87b7c Created pod: kube-auth-proxy-7b84b87b7c-gk9x6 60m Normal Pulling pod/kube-auth-proxy-7b84b87b7c-gk9x6 Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 60m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 60m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-szfl8 60m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-szfl8 Add eth0 [10.134.0.22/23] from ovn-kubernetes 60m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-szfl8 Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 60m Normal Pulled pod/kube-auth-proxy-7b84b87b7c-gk9x6 Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.566s (2.566s including waiting). Image size: 188904677 bytes. 60m Normal Started pod/kube-auth-proxy-7b84b87b7c-vx6bc Container started 60m Normal Created pod/kube-auth-proxy-7b84b87b7c-vx6bc Container created 60m Normal Pulled pod/kube-auth-proxy-7b84b87b7c-vx6bc Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.484s (2.484s including waiting). Image size: 188904677 bytes. 60m Normal Started pod/kube-auth-proxy-7b84b87b7c-gk9x6 Container started 60m Normal Created pod/kube-auth-proxy-7b84b87b7c-gk9x6 Container created 60m Normal Created pod/istiod-openshift-gateway-7876bfcb74-szfl8 Container created 60m Normal Started pod/istiod-openshift-gateway-7876bfcb74-szfl8 Container started 60m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-szfl8 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.459s (2.459s including waiting). Image size: 185476477 bytes. 60m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-szfl8 Readiness probe failed: HTTP probe failed with statuscode: 503 60m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-6c786f9b57 from 0 to 1 60m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-6c786f9b57 Created pod: data-science-gateway-data-science-gateway-class-6c786f9b57gktwl 60m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-szfl8 Readiness probe error: HTTP probe failed with statuscode: 503... 60m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-6c786f9b57gktwl Add eth0 [10.133.0.25/23] from ovn-kubernetes 60m Normal Pulling pod/data-science-gateway-data-science-gateway-class-6c786f9b57gktwl Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 60m Normal Started pod/data-science-gateway-data-science-gateway-class-6c786f9b57gktwl Container started 60m Normal Pulled pod/data-science-gateway-data-science-gateway-class-6c786f9b57gktwl Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.73s (2.73s including waiting). Image size: 183555333 bytes. 60m Normal Created pod/data-science-gateway-data-science-gateway-class-6c786f9b57gktwl Container created 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 59m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-bhpfs 59m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 59m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-bhpfs Container started 59m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-bhpfs Container created 59m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-bhpfs Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-bhpfs Add eth0 [10.134.0.23/23] from ovn-kubernetes 59m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-szfl8 Stopping container discovery 59m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 59m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-szfl8 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Normal AddedInterface pod/maas-default-gateway-openshift-default-5db77fbd9f-8d8sf Add eth0 [10.133.0.33/23] from ovn-kubernetes 57m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5db77fbd9f Created pod: maas-default-gateway-openshift-default-5db77fbd9f-8d8sf 57m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 57m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-5db77fbd9f from 0 to 1 55m Normal Pulled pod/maas-default-gateway-openshift-default-5db77fbd9f-8d8sf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-8d8sf Readiness probe error: Get "http://10.133.0.33:15021/healthz/ready": read tcp 10.133.0.2:39418->10.133.0.33:15021: read: connection reset by peer... 55m Normal Created pod/maas-default-gateway-openshift-default-5db77fbd9f-8d8sf Container created 55m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-8d8sf Readiness probe failed: Get "http://10.133.0.33:15021/healthz/ready": read tcp 10.133.0.2:39418->10.133.0.33:15021: read: connection reset by peer 55m Normal Started pod/maas-default-gateway-openshift-default-5db77fbd9f-8d8sf Container started 55m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-8d8sf Startup probe failed: Get "http://10.133.0.33:15021/healthz/ready": dial tcp 10.133.0.33:15021: connect: connection refused 55m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-8d8sf Startup probe error: Get "http://10.133.0.33:15021/healthz/ready": dial tcp 10.133.0.33:15021: connect: connection refused... 55m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-5db77fbd9f from 1 to 4 54m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5db77fbd9f Created pod: maas-default-gateway-openshift-default-5db77fbd9f-fhw57 54m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5db77fbd9f Created pod: maas-default-gateway-openshift-default-5db77fbd9f-k9fn7 54m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5db77fbd9f Created pod: maas-default-gateway-openshift-default-5db77fbd9f-nqsf5 54m Normal AddedInterface pod/maas-default-gateway-openshift-default-5db77fbd9f-k9fn7 Add eth0 [10.134.0.35/23] from ovn-kubernetes 54m Normal Pulling pod/maas-default-gateway-openshift-default-5db77fbd9f-k9fn7 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 54m Normal Pulling pod/maas-default-gateway-openshift-default-5db77fbd9f-fhw57 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 54m Normal AddedInterface pod/maas-default-gateway-openshift-default-5db77fbd9f-fhw57 Add eth0 [10.132.0.48/23] from ovn-kubernetes 54m Normal Pulling pod/maas-default-gateway-openshift-default-5db77fbd9f-nqsf5 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 54m Normal AddedInterface pod/maas-default-gateway-openshift-default-5db77fbd9f-nqsf5 Add eth0 [10.132.0.49/23] from ovn-kubernetes 53m Normal Pulled pod/maas-default-gateway-openshift-default-5db77fbd9f-k9fn7 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.591s (2.591s including waiting). Image size: 183555333 bytes. 53m Normal Started pod/maas-default-gateway-openshift-default-5db77fbd9f-nqsf5 Container started 53m Normal Created pod/maas-default-gateway-openshift-default-5db77fbd9f-fhw57 Container created 53m Normal Pulled pod/maas-default-gateway-openshift-default-5db77fbd9f-fhw57 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.952s (2.952s including waiting). Image size: 183555333 bytes. 53m Normal Started pod/maas-default-gateway-openshift-default-5db77fbd9f-fhw57 Container started 53m Normal Started pod/maas-default-gateway-openshift-default-5db77fbd9f-k9fn7 Container started 53m Normal Created pod/maas-default-gateway-openshift-default-5db77fbd9f-k9fn7 Container created 53m Normal Pulled pod/maas-default-gateway-openshift-default-5db77fbd9f-nqsf5 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.873s (2.873s including waiting). Image size: 183555333 bytes. 53m Normal Created pod/maas-default-gateway-openshift-default-5db77fbd9f-nqsf5 Container created 53m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-nqsf5 Startup probe failed: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused 53m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-nqsf5 Startup probe error: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused... 53m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-fhw57 Startup probe failed: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused 53m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-fhw57 Startup probe error: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused... 53m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-k9fn7 Startup probe error: Get "http://10.134.0.35:15021/healthz/ready": dial tcp 10.134.0.35:15021: connect: connection refused... 53m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-k9fn7 Startup probe failed: Get "http://10.134.0.35:15021/healthz/ready": dial tcp 10.134.0.35:15021: connect: connection refused 53m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-nqsf5 Startup probe error: Get "http://10.132.0.49:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 53m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-nqsf5 Startup probe failed: Get "http://10.132.0.49:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 53m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 53m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-5f78687d5b from 0 to 1 53m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5db77fbd9f Created pod: maas-default-gateway-openshift-default-5db77fbd9f-mg4hv 53m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5db77fbd9f Created pod: maas-default-gateway-openshift-default-5db77fbd9f-m55mt 53m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 53m Normal SuccessfulCreate replicaset/payload-processing-5f78687d5b Created pod: payload-processing-5f78687d5b-8wn8h 53m Normal Created pod/maas-default-gateway-openshift-default-5db77fbd9f-mg4hv Container created 53m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 53m Normal AddedInterface pod/payload-processing-5f78687d5b-8wn8h Add eth0 [10.132.0.51/23] from ovn-kubernetes 53m Normal Started pod/maas-default-gateway-openshift-default-5db77fbd9f-m55mt Container started 53m Normal Pulling pod/payload-processing-5f78687d5b-8wn8h Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 53m Normal Created pod/maas-default-gateway-openshift-default-5db77fbd9f-m55mt Container created 53m Normal Pulled pod/maas-default-gateway-openshift-default-5db77fbd9f-m55mt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/maas-default-gateway-openshift-default-5db77fbd9f-mg4hv Add eth0 [10.133.0.41/23] from ovn-kubernetes 53m Warning FailedCreate replicaset/payload-processing-5f78687d5b Error creating: pods "payload-processing-5f78687d5b-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 53m Normal Pulled pod/maas-default-gateway-openshift-default-5db77fbd9f-mg4hv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/maas-default-gateway-openshift-default-5db77fbd9f-m55mt Add eth0 [10.134.0.37/23] from ovn-kubernetes 53m Normal Pulling pod/payload-pre-processing-7b98bb678c-64t9f Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 53m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-64t9f 53m Normal Started pod/maas-default-gateway-openshift-default-5db77fbd9f-mg4hv Container started 53m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-64t9f Add eth0 [10.132.0.52/23] from ovn-kubernetes 53m Normal Created pod/payload-processing-5f78687d5b-8wn8h Container created 53m Normal Pulled pod/payload-pre-processing-7b98bb678c-64t9f Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.041s (3.041s including waiting). Image size: 125317101 bytes. 53m Normal Pulled pod/payload-processing-5f78687d5b-8wn8h Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.18s (3.18s including waiting). Image size: 125317101 bytes. 53m Normal Created pod/payload-pre-processing-7b98bb678c-64t9f Container created 53m Normal Started pod/payload-pre-processing-7b98bb678c-64t9f Container started 53m Normal Started pod/payload-processing-5f78687d5b-8wn8h Container started 53m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-mg4hv Startup probe error: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused... 53m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-m55mt Startup probe error: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused... 53m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-mg4hv Startup probe failed: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused 53m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-m55mt Startup probe failed: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused 52m Normal SuccessfulCreate replicaset/e2e-worker-w5-22d634-openshift-default-89dbbf78f Created pod: e2e-worker-w5-22d634-openshift-default-89dbbf78f-hz9sp 52m Normal AddedLabel gateway/e2e-worker-w3-43b37d Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-43b37d 52m Normal AddedLabel gateway/e2e-worker-w6-784e7a Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-784e7a 52m Normal ScalingReplicaSet deployment/e2e-worker-w6-784e7a-openshift-default Scaled up replica set e2e-worker-w6-784e7a-openshift-default-595898bd5c from 0 to 1 52m Normal SuccessfulCreate replicaset/e2e-worker-w1-fc684b-openshift-default-7fcf678997 Created pod: e2e-worker-w1-fc684b-openshift-default-7fcf678997-988qz 52m Normal AddedLabel gateway/e2e-worker-w1-fc684b Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-fc684b 52m Normal AddedLabel gateway/e2e-worker-w0-2e046a Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-2e046a 52m Normal SuccessfulCreate replicaset/e2e-worker-w6-784e7a-openshift-default-595898bd5c Created pod: e2e-worker-w6-784e7a-openshift-default-595898bd5c-tf78l 52m Normal AddedLabel gateway/e2e-worker-w5-22d634 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-22d634 52m Normal AddedLabel gateway/e2e-worker-w2-2ba10a Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-2ba10a 52m Normal ScalingReplicaSet deployment/e2e-worker-w5-22d634-openshift-default Scaled up replica set e2e-worker-w5-22d634-openshift-default-89dbbf78f from 0 to 1 52m Normal SuccessfulCreate replicaset/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc Created pod: e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-cfx67 52m Normal ScalingReplicaSet deployment/e2e-worker-w1-fc684b-openshift-default Scaled up replica set e2e-worker-w1-fc684b-openshift-default-7fcf678997 from 0 to 1 52m Normal AddedLabel gateway/e2e-worker-w4-08ebee Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-08ebee 52m Normal ScalingReplicaSet deployment/e2e-worker-w3-43b37d-openshift-default Scaled up replica set e2e-worker-w3-43b37d-openshift-default-5dcf864fcc from 0 to 1 52m Normal AddedInterface pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-5lkhc Add eth0 [10.134.0.40/23] from ovn-kubernetes 52m Normal SuccessfulCreate replicaset/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 Created pod: e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-tczdj 52m Normal AddedInterface pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-hz9sp Add eth0 [10.134.0.39/23] from ovn-kubernetes 52m Normal AddedInterface pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-988qz Add eth0 [10.132.0.55/23] from ovn-kubernetes 52m Normal AddedInterface pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-t8fq4 Add eth0 [10.132.0.56/23] from ovn-kubernetes 52m Normal ScalingReplicaSet deployment/e2e-worker-w0-2e046a-openshift-default Scaled up replica set e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 from 0 to 1 52m Normal AddedInterface pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-tf78l Add eth0 [10.132.0.54/23] from ovn-kubernetes 52m Normal AddedInterface pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-cfx67 Add eth0 [10.134.0.38/23] from ovn-kubernetes 52m Normal SuccessfulCreate replicaset/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 Created pod: e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-5lkhc 52m Normal ScalingReplicaSet deployment/e2e-worker-w4-08ebee-openshift-default Scaled up replica set e2e-worker-w4-08ebee-openshift-default-5f5bf46967 from 0 to 1 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-08ebee-openshift-default-5f5bf46967 Created pod: e2e-worker-w4-08ebee-openshift-default-5f5bf46967-t8fq4 52m Normal ScalingReplicaSet deployment/e2e-worker-w2-2ba10a-openshift-default Scaled up replica set e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 from 0 to 1 52m Normal AddedInterface pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-tczdj Add eth0 [10.134.0.41/23] from ovn-kubernetes 52m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-784e7a-79d748c8bb Error creating: pods "payload-processing-e2e-worker-w6-784e7a-79d748c8bb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-784e7a: serviceaccount "payload-processing-e2e-worker-w6-784e7a" not found 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-784e7a Scaled up replica set payload-processing-e2e-worker-w6-784e7a-79d748c8bb from 0 to 1 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-784e7a Scaled up replica set payload-pre-processing-e2e-worker-w6-784e7a-5bd4bc7d47 from 0 to 1 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-784e7a-79d748c8bb Created pod: payload-processing-e2e-worker-w6-784e7a-79d748c8bb-fbgzx 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-784e7a-5bd4bc7d47 Created pod: payload-pre-processing-e2e-worker-w6-784e7a-5bd4bc7d47-cddfw 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-784e7a-5bd4bc7d47 Error creating: pods "payload-pre-processing-e2e-worker-w6-784e7a-5bd4bc7d47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-784e7a: serviceaccount "payload-processing-e2e-worker-w6-784e7a" not found 52m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-784e7a-5bd4bc7d47-cddfw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-784e7a-5bd4bc7d47-cddfw Add eth0 [10.132.0.57/23] from ovn-kubernetes 52m Normal Started pod/payload-pre-processing-e2e-worker-w6-784e7a-5bd4bc7d47-cddfw Container started 52m Normal Started pod/payload-processing-e2e-worker-w6-784e7a-79d748c8bb-fbgzx Container started 52m Normal Created pod/payload-processing-e2e-worker-w6-784e7a-79d748c8bb-fbgzx Container created 52m Normal Pulled pod/payload-processing-e2e-worker-w6-784e7a-79d748c8bb-fbgzx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/payload-processing-e2e-worker-w6-784e7a-79d748c8bb-fbgzx Add eth0 [10.132.0.58/23] from ovn-kubernetes 52m Normal Created pod/payload-pre-processing-e2e-worker-w6-784e7a-5bd4bc7d47-cddfw Container created 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-43b37d-74d9bdb487 Error creating: pods "payload-pre-processing-e2e-worker-w3-43b37d-74d9bdb487-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-43b37d: serviceaccount "payload-processing-e2e-worker-w3-43b37d" not found 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-43b37d-fd685c7db Created pod: payload-processing-e2e-worker-w3-43b37d-fd685c7db-qkngc 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-43b37d-74d9bdb487 Created pod: payload-pre-processing-e2e-worker-w3-43b37d-74d9bdb487-qtfzw 52m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-43b37d-fd685c7db Error creating: pods "payload-processing-e2e-worker-w3-43b37d-fd685c7db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-43b37d: serviceaccount "payload-processing-e2e-worker-w3-43b37d" not found 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-43b37d Scaled up replica set payload-pre-processing-e2e-worker-w3-43b37d-74d9bdb487 from 0 to 1 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-43b37d Scaled up replica set payload-processing-e2e-worker-w3-43b37d-fd685c7db from 0 to 1 52m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-43b37d-74d9bdb487-qtfzw Add eth0 [10.132.0.59/23] from ovn-kubernetes 52m Normal Created pod/payload-processing-e2e-worker-w3-43b37d-fd685c7db-qkngc Container created 52m Normal Started pod/payload-pre-processing-e2e-worker-w3-43b37d-74d9bdb487-qtfzw Container started 52m Normal Created pod/payload-pre-processing-e2e-worker-w3-43b37d-74d9bdb487-qtfzw Container created 52m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-43b37d-74d9bdb487-qtfzw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/payload-processing-e2e-worker-w3-43b37d-fd685c7db-qkngc Add eth0 [10.132.0.60/23] from ovn-kubernetes 52m Normal Pulled pod/payload-processing-e2e-worker-w3-43b37d-fd685c7db-qkngc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Started pod/payload-processing-e2e-worker-w3-43b37d-fd685c7db-qkngc Container started 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-2ba10a Scaled up replica set payload-pre-processing-e2e-worker-w2-2ba10a-6d55f55df9 from 0 to 1 52m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-2ba10a-78fb8d6fd4 Error creating: pods "payload-processing-e2e-worker-w2-2ba10a-78fb8d6fd4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-2ba10a: serviceaccount "payload-processing-e2e-worker-w2-2ba10a" not found 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-2ba10a Scaled up replica set payload-processing-e2e-worker-w2-2ba10a-78fb8d6fd4 from 0 to 1 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-2ba10a-6d55f55df9 Error creating: pods "payload-pre-processing-e2e-worker-w2-2ba10a-6d55f55df9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-2ba10a: serviceaccount "payload-processing-e2e-worker-w2-2ba10a" not found 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-2ba10a-6d55f55df9 Created pod: payload-pre-processing-e2e-worker-w2-2ba10a-6d55f55df9-gfdpx 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-2ba10a-78fb8d6fd4 Created pod: payload-processing-e2e-worker-w2-2ba10a-78fb8d6fd4-qw5lv 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-08ebee-7bcfbb7487 Error creating: pods "payload-pre-processing-e2e-worker-w4-08ebee-7bcfbb7487-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-08ebee: serviceaccount "payload-processing-e2e-worker-w4-08ebee" not found 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-08ebee-5f548676d4 Created pod: payload-processing-e2e-worker-w4-08ebee-5f548676d4-st67q 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-08ebee Scaled up replica set payload-processing-e2e-worker-w4-08ebee-5f548676d4 from 0 to 1 52m Normal Pulling pod/payload-pre-processing-e2e-worker-w2-2ba10a-6d55f55df9-gfdpx Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 52m Normal AddedInterface pod/payload-processing-e2e-worker-w2-2ba10a-78fb8d6fd4-qw5lv Add eth0 [10.132.0.61/23] from ovn-kubernetes 52m Normal Pulled pod/payload-processing-e2e-worker-w2-2ba10a-78fb8d6fd4-qw5lv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-08ebee-7bcfbb7487 Created pod: payload-pre-processing-e2e-worker-w4-08ebee-7bcfbb7487-9x5xj 52m Normal Created pod/payload-processing-e2e-worker-w2-2ba10a-78fb8d6fd4-qw5lv Container created 52m Normal Started pod/payload-processing-e2e-worker-w2-2ba10a-78fb8d6fd4-qw5lv Container started 52m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-08ebee-5f548676d4 Error creating: pods "payload-processing-e2e-worker-w4-08ebee-5f548676d4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-08ebee: serviceaccount "payload-processing-e2e-worker-w4-08ebee" not found 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-08ebee Scaled up replica set payload-pre-processing-e2e-worker-w4-08ebee-7bcfbb7487 from 0 to 1 52m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-2ba10a-6d55f55df9-gfdpx Add eth0 [10.134.0.42/23] from ovn-kubernetes 52m Normal Started pod/payload-processing-e2e-worker-w4-08ebee-5f548676d4-st67q Container started 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-fc684b-7fb5fd79c5 Created pod: payload-pre-processing-e2e-worker-w1-fc684b-7fb5fd79c5-hb2rn 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-fc684b-7fb5fd79c5 Error creating: pods "payload-pre-processing-e2e-worker-w1-fc684b-7fb5fd79c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-fc684b: serviceaccount "payload-processing-e2e-worker-w1-fc684b" not found 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-fc684b Scaled up replica set payload-pre-processing-e2e-worker-w1-fc684b-7fb5fd79c5 from 0 to 1 52m Normal Created pod/payload-processing-e2e-worker-w4-08ebee-5f548676d4-st67q Container created 52m Normal Pulled pod/payload-processing-e2e-worker-w4-08ebee-5f548676d4-st67q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/payload-processing-e2e-worker-w4-08ebee-5f548676d4-st67q Add eth0 [10.132.0.62/23] from ovn-kubernetes 52m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-fc684b-5db88966b Error creating: pods "payload-processing-e2e-worker-w1-fc684b-5db88966b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-fc684b: serviceaccount "payload-processing-e2e-worker-w1-fc684b" not found 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-fc684b-5db88966b Created pod: payload-processing-e2e-worker-w1-fc684b-5db88966b-rwg2s 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-fc684b Scaled up replica set payload-processing-e2e-worker-w1-fc684b-5db88966b from 0 to 1 52m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-08ebee-7bcfbb7487-9x5xj Add eth0 [10.134.0.43/23] from ovn-kubernetes 52m Normal Pulling pod/payload-pre-processing-e2e-worker-w4-08ebee-7bcfbb7487-9x5xj Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 52m Normal AddedInterface pod/payload-processing-e2e-worker-w1-fc684b-5db88966b-rwg2s Add eth0 [10.134.0.44/23] from ovn-kubernetes 52m Normal Created pod/payload-pre-processing-e2e-worker-w1-fc684b-7fb5fd79c5-hb2rn Container created 52m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-fc684b-7fb5fd79c5-hb2rn Add eth0 [10.132.0.63/23] from ovn-kubernetes 52m Normal Started pod/payload-pre-processing-e2e-worker-w1-fc684b-7fb5fd79c5-hb2rn Container started 52m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-fc684b-7fb5fd79c5-hb2rn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Pulling pod/payload-processing-e2e-worker-w1-fc684b-5db88966b-rwg2s Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-2e046a Scaled up replica set payload-processing-e2e-worker-w0-2e046a-6888946c9d from 0 to 1 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-2e046a-6888946c9d Created pod: payload-processing-e2e-worker-w0-2e046a-6888946c9d-w2lbg 52m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-2e046a-6888946c9d Error creating: pods "payload-processing-e2e-worker-w0-2e046a-6888946c9d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-2e046a: serviceaccount "payload-processing-e2e-worker-w0-2e046a" not found 52m Normal Started pod/payload-pre-processing-e2e-worker-w2-2ba10a-6d55f55df9-gfdpx Container started 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-2e046a-74bbd64fc9 Created pod: payload-pre-processing-e2e-worker-w0-2e046a-74bbd64fc9-7nrm7 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-2e046a Scaled up replica set payload-pre-processing-e2e-worker-w0-2e046a-74bbd64fc9 from 0 to 1 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-2e046a-74bbd64fc9 Error creating: pods "payload-pre-processing-e2e-worker-w0-2e046a-74bbd64fc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-2e046a: serviceaccount "payload-processing-e2e-worker-w0-2e046a" not found 52m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-2ba10a-6d55f55df9-gfdpx Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.827s (2.827s including waiting). Image size: 125317101 bytes. 52m Normal Created pod/payload-pre-processing-e2e-worker-w2-2ba10a-6d55f55df9-gfdpx Container created 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-22d634 Scaled up replica set payload-processing-e2e-worker-w5-22d634-7db4fc9554 from 0 to 1 52m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-22d634-7db4fc9554 Error creating: pods "payload-processing-e2e-worker-w5-22d634-7db4fc9554-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-22d634: serviceaccount "payload-processing-e2e-worker-w5-22d634" not found 52m Normal AddedInterface pod/payload-processing-e2e-worker-w0-2e046a-6888946c9d-w2lbg Add eth0 [10.132.0.64/23] from ovn-kubernetes 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-22d634-6768765995 Created pod: payload-pre-processing-e2e-worker-w5-22d634-6768765995-phnhr 52m Normal Started pod/payload-pre-processing-e2e-worker-w0-2e046a-74bbd64fc9-7nrm7 Container started 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-22d634 Scaled up replica set payload-pre-processing-e2e-worker-w5-22d634-6768765995 from 0 to 1 52m Normal Created pod/payload-pre-processing-e2e-worker-w0-2e046a-74bbd64fc9-7nrm7 Container created 52m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-2e046a-74bbd64fc9-7nrm7 Add eth0 [10.134.0.45/23] from ovn-kubernetes 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-22d634-6768765995 Error creating: pods "payload-pre-processing-e2e-worker-w5-22d634-6768765995-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-22d634: serviceaccount "payload-processing-e2e-worker-w5-22d634" not found 52m Normal Pulled pod/payload-processing-e2e-worker-w0-2e046a-6888946c9d-w2lbg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-08ebee-7bcfbb7487-9x5xj Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.695s (2.695s including waiting). Image size: 125317101 bytes. 52m Normal Started pod/payload-pre-processing-e2e-worker-w4-08ebee-7bcfbb7487-9x5xj Container started 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-22d634-7db4fc9554 Created pod: payload-processing-e2e-worker-w5-22d634-7db4fc9554-56z6p 52m Normal Created pod/payload-pre-processing-e2e-worker-w4-08ebee-7bcfbb7487-9x5xj Container created 52m Normal Pulled pod/payload-processing-e2e-worker-w1-fc684b-5db88966b-rwg2s Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 1.586s (1.586s including waiting). Image size: 125317101 bytes. 52m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-2e046a-74bbd64fc9-7nrm7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-22d634-6768765995-phnhr Add eth0 [10.132.0.65/23] from ovn-kubernetes 52m Normal Started pod/payload-processing-e2e-worker-w0-2e046a-6888946c9d-w2lbg Container started 52m Normal Created pod/payload-processing-e2e-worker-w1-fc684b-5db88966b-rwg2s Container created 52m Normal Started pod/payload-processing-e2e-worker-w1-fc684b-5db88966b-rwg2s Container started 52m Normal Created pod/payload-processing-e2e-worker-w0-2e046a-6888946c9d-w2lbg Container created 52m Normal Pulled pod/payload-processing-e2e-worker-w5-22d634-7db4fc9554-56z6p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-22d634-6768765995-phnhr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/payload-processing-e2e-worker-w5-22d634-7db4fc9554-56z6p Add eth0 [10.132.0.66/23] from ovn-kubernetes 52m Normal Created pod/payload-pre-processing-e2e-worker-w5-22d634-6768765995-phnhr Container created 52m Normal Started pod/payload-processing-e2e-worker-w5-22d634-7db4fc9554-56z6p Container started 52m Normal Started pod/payload-pre-processing-e2e-worker-w5-22d634-6768765995-phnhr Container started 52m Normal Created pod/payload-processing-e2e-worker-w5-22d634-7db4fc9554-56z6p Container created 52m Warning ProbeError pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-tf78l Readiness probe error: Get "http://10.132.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-tf78l Readiness probe failed: Get "http://10.132.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Normal Started pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-tf78l Container started 52m Normal Created pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-tf78l Container created 52m Normal Pulled pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-tf78l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-988qz Container created 52m Normal Pulled pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-988qz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Warning Unhealthy pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-tf78l Startup probe failed: Get "http://10.132.0.54:15021/healthz/ready": dial tcp 10.132.0.54:15021: connect: connection refused 52m Normal Started pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-988qz Container started 52m Warning ProbeError pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-tf78l Startup probe error: Get "http://10.132.0.54:15021/healthz/ready": dial tcp 10.132.0.54:15021: connect: connection refused... 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-784e7a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-784e7a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning Unhealthy pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-988qz Startup probe failed: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-hz9sp Readiness probe error: Get "http://10.134.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Normal Started pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-tczdj Container started 52m Warning Unhealthy pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-hz9sp Readiness probe failed: Get "http://10.134.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-988qz Startup probe error: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused... 52m Normal Created pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-tczdj Container created 52m Normal Pulled pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-tczdj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Pulled pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-hz9sp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-hz9sp Container created 52m Normal Started pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-hz9sp Container started 52m Normal Started pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-cfx67 Container started 52m Normal Pulled pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-cfx67 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-cfx67 Container created 52m Normal Created pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-5lkhc Container created 52m Normal Started pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-5lkhc Container started 52m Normal Pulled pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-5lkhc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Pulled pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-t8fq4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Warning Unhealthy pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-tczdj Startup probe failed: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused 52m Normal Started pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-t8fq4 Container started 52m Normal Created pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-t8fq4 Container created 52m Warning ProbeError pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-tczdj Startup probe error: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-hz9sp Startup probe failed: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-cfx67 Startup probe failed: Get "http://10.134.0.38:15021/healthz/ready": dial tcp 10.134.0.38:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-cfx67 Startup probe error: Get "http://10.134.0.38:15021/healthz/ready": dial tcp 10.134.0.38:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-hz9sp Startup probe error: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-5lkhc Startup probe error: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-5lkhc Startup probe failed: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-t8fq4 Startup probe failed: Get "http://10.132.0.56:15021/healthz/ready": dial tcp 10.132.0.56:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-t8fq4 Startup probe error: Get "http://10.132.0.56:15021/healthz/ready": dial tcp 10.132.0.56:15021: connect: connection refused... 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-2e046a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-08ebee-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-08ebee-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-2e046a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-2ba10a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-2ba10a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-43b37d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-784e7a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-43b37d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-784e7a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-2e046a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-2e046a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-2ba10a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-08ebee-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-fc684b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-2ba10a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-fc684b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-08ebee-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-22d634-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-22d634-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Normal Pulled pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-95zbb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal ScalingReplicaSet deployment/e2e-worker-w1-fc684b-openshift-default Scaled up replica set e2e-worker-w1-fc684b-openshift-default-7fcf678997 from 1 to 2 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-43b37d-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 50m Normal Started pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-95zbb Container started 50m Normal SuccessfulCreate replicaset/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc Created pod: e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-95zbb 50m Normal AddedInterface pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-vnz5k Add eth0 [10.134.0.53/23] from ovn-kubernetes 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-fc684b-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 50m Normal ScalingReplicaSet deployment/e2e-worker-w3-43b37d-openshift-default Scaled up replica set e2e-worker-w3-43b37d-openshift-default-5dcf864fcc from 1 to 2 50m Normal Pulled pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-vnz5k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal SuccessfulCreate replicaset/e2e-worker-w1-fc684b-openshift-default-7fcf678997 Created pod: e2e-worker-w1-fc684b-openshift-default-7fcf678997-vnz5k 50m Normal AddedInterface pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-95zbb Add eth0 [10.132.0.76/23] from ovn-kubernetes 50m Normal Created pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-95zbb Container created 50m Normal Started pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-vnz5k Container started 50m Normal Created pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-vnz5k Container created 50m Warning Unhealthy pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-95zbb Startup probe failed: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-95zbb Startup probe error: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-vnz5k Startup probe error: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-vnz5k Startup probe failed: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-784e7a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 50m Normal ScalingReplicaSet deployment/e2e-worker-w6-784e7a-openshift-default Scaled up replica set e2e-worker-w6-784e7a-openshift-default-595898bd5c from 1 to 2 50m Normal SuccessfulCreate replicaset/e2e-worker-w6-784e7a-openshift-default-595898bd5c Created pod: e2e-worker-w6-784e7a-openshift-default-595898bd5c-kw5tw 50m Normal Created pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-kw5tw Container created 50m Normal Started pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-4jtkn Container started 50m Normal SuccessfulCreate replicaset/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 Created pod: e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-f7th7 50m Normal AddedInterface pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-kw5tw Add eth0 [10.134.0.54/23] from ovn-kubernetes 50m Normal ScalingReplicaSet deployment/e2e-worker-w5-22d634-openshift-default Scaled up replica set e2e-worker-w5-22d634-openshift-default-89dbbf78f from 1 to 2 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-22d634-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 50m Normal Pulled pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-kw5tw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal SuccessfulCreate replicaset/e2e-worker-w4-08ebee-openshift-default-5f5bf46967 Created pod: e2e-worker-w4-08ebee-openshift-default-5f5bf46967-vjfcn 50m Normal Started pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-kw5tw Container started 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-08ebee-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 50m Normal ScalingReplicaSet deployment/e2e-worker-w4-08ebee-openshift-default Scaled up replica set e2e-worker-w4-08ebee-openshift-default-5f5bf46967 from 1 to 2 50m Normal AddedInterface pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-4jtkn Add eth0 [10.132.0.80/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-4jtkn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-4jtkn Container created 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2e046a-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 50m Normal ScalingReplicaSet deployment/e2e-worker-w0-2e046a-openshift-default Scaled up replica set e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 from 1 to 3 50m Normal SuccessfulCreate replicaset/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 Created pod: e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-rkgz4 50m Normal ScalingReplicaSet deployment/e2e-worker-w2-2ba10a-openshift-default Scaled up replica set e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 from 1 to 2 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-2ba10a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 50m Normal SuccessfulCreate replicaset/e2e-worker-w5-22d634-openshift-default-89dbbf78f Created pod: e2e-worker-w5-22d634-openshift-default-89dbbf78f-4jtkn 50m Normal SuccessfulCreate replicaset/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 Created pod: e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-jjlwt 50m Normal Started pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-f7th7 Container started 50m Normal AddedInterface pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-rkgz4 Add eth0 [10.132.0.82/23] from ovn-kubernetes 50m Normal Created pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-jjlwt Container created 50m Normal Started pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-jjlwt Container started 50m Normal Pulled pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-vjfcn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-f7th7 Add eth0 [10.133.0.49/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-jjlwt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Pulled pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-f7th7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-jjlwt Add eth0 [10.132.0.81/23] from ovn-kubernetes 50m Normal Created pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-f7th7 Container created 50m Normal Pulled pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-rkgz4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-vjfcn Container created 50m Normal Started pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-vjfcn Container started 50m Normal AddedInterface pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-vjfcn Add eth0 [10.134.0.55/23] from ovn-kubernetes 50m Normal Started pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-rkgz4 Container started 50m Normal Created pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-rkgz4 Container created 50m Warning ProbeError pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-jjlwt Startup probe error: Get "http://10.132.0.81:15021/healthz/ready": dial tcp 10.132.0.81:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-kw5tw Startup probe failed: Get "http://10.134.0.54:15021/healthz/ready": dial tcp 10.134.0.54:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-kw5tw Startup probe error: Get "http://10.134.0.54:15021/healthz/ready": dial tcp 10.134.0.54:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-4jtkn Startup probe failed: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-4jtkn Startup probe error: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-vjfcn Startup probe failed: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-vjfcn Startup probe error: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-jjlwt Startup probe failed: Get "http://10.132.0.81:15021/healthz/ready": dial tcp 10.132.0.81:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-f7th7 Startup probe error: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-f7th7 Startup probe failed: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-rkgz4 Startup probe error: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-rkgz4 Startup probe failed: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-2ba10a-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 Created pod: e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-ntt82 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-fc684b-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 49m Normal ScalingReplicaSet deployment/e2e-worker-w1-fc684b-openshift-default Scaled up replica set e2e-worker-w1-fc684b-openshift-default-7fcf678997 from 2 to 4 49m Normal SuccessfulCreate replicaset/e2e-worker-w1-fc684b-openshift-default-7fcf678997 Created pod: e2e-worker-w1-fc684b-openshift-default-7fcf678997-8npq2 49m Normal SuccessfulCreate replicaset/e2e-worker-w1-fc684b-openshift-default-7fcf678997 Created pod: e2e-worker-w1-fc684b-openshift-default-7fcf678997-g26hx 49m Normal ScalingReplicaSet deployment/e2e-worker-w6-784e7a-openshift-default Scaled up replica set e2e-worker-w6-784e7a-openshift-default-595898bd5c from 2 to 4 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 Created pod: e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-rppfp 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 Created pod: e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9rm2s 49m Normal ScalingReplicaSet deployment/e2e-worker-w2-2ba10a-openshift-default Scaled up replica set e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 from 2 to 4 49m Normal AddedInterface pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-9txdf Add eth0 [10.133.0.53/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-9txdf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal SuccessfulCreate replicaset/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc Created pod: e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-9txdf 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-784e7a-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 49m Normal SuccessfulCreate replicaset/e2e-worker-w6-784e7a-openshift-default-595898bd5c Created pod: e2e-worker-w6-784e7a-openshift-default-595898bd5c-vwlks 49m Normal SuccessfulCreate replicaset/e2e-worker-w6-784e7a-openshift-default-595898bd5c Created pod: e2e-worker-w6-784e7a-openshift-default-595898bd5c-ssfqt 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-43b37d-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 49m Normal ScalingReplicaSet deployment/e2e-worker-w3-43b37d-openshift-default Scaled up replica set e2e-worker-w3-43b37d-openshift-default-5dcf864fcc from 2 to 3 49m Normal AddedInterface pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-nbn5j Add eth0 [10.133.0.54/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-nbn5j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-wsjz8 Add eth0 [10.134.0.59/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-vwlks Container started 49m Normal Created pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-vwlks Container created 49m Normal Pulled pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-vwlks Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-vwlks Add eth0 [10.134.0.58/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 Created pod: e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-nf9qq 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 Created pod: e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-lks7l 49m Normal Started pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-ssfqt Container started 49m Normal ScalingReplicaSet deployment/e2e-worker-w0-2e046a-openshift-default Scaled up replica set e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 from 3 to 6 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2e046a-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 49m Normal Created pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-ssfqt Container created 49m Normal Pulled pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-ssfqt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-ssfqt Add eth0 [10.133.0.52/23] from ovn-kubernetes 49m Normal ScalingReplicaSet deployment/e2e-worker-w5-22d634-openshift-default Scaled up replica set e2e-worker-w5-22d634-openshift-default-89dbbf78f from 2 to 4 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-22d634-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 49m Normal SuccessfulCreate replicaset/e2e-worker-w5-22d634-openshift-default-89dbbf78f Created pod: e2e-worker-w5-22d634-openshift-default-89dbbf78f-wsjz8 49m Normal SuccessfulCreate replicaset/e2e-worker-w5-22d634-openshift-default-89dbbf78f Created pod: e2e-worker-w5-22d634-openshift-default-89dbbf78f-nbn5j 49m Normal Pulled pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-wsjz8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-g26hx Add eth0 [10.133.0.55/23] from ovn-kubernetes 49m Normal Created pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9rm2s Container created 49m Normal Started pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9rm2s Container started 49m Normal AddedInterface pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9rm2s Add eth0 [10.132.0.91/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-wsjz8 Container started 49m Normal Created pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-wsjz8 Container created 49m Normal Pulled pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9rm2s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-rppfp Add eth0 [10.133.0.56/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-rppfp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-rppfp Container created 49m Normal Started pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-nbn5j Container started 49m Normal Created pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-nbn5j Container created 49m Normal Started pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-rppfp Container started 49m Normal AddedInterface pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-8npq2 Add eth0 [10.134.0.60/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-8npq2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-g26hx Container started 49m Normal Created pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-g26hx Container created 49m Normal Started pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-ntt82 Container started 49m Normal Created pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-ntt82 Container created 49m Normal Pulled pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-ntt82 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-ntt82 Add eth0 [10.134.0.61/23] from ovn-kubernetes 49m Normal Created pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-8npq2 Container created 49m Normal Pulled pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-g26hx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-lks7l Add eth0 [10.133.0.57/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-lks7l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-9txdf Container started 49m Normal Created pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-lks7l Container created 49m Normal Started pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-lks7l Container started 49m Normal Started pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-nf9qq Container started 49m Normal Created pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-nf9qq Container created 49m Normal Pulled pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-nf9qq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-nf9qq Add eth0 [10.132.0.92/23] from ovn-kubernetes 49m Normal Created pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-9txdf Container created 49m Normal Started pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-8npq2 Container started 49m Warning ProbeError pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9rm2s Startup probe error: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9rm2s Startup probe failed: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-nf9qq Startup probe error: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-nf9qq Startup probe failed: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-vwlks Startup probe error: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-wsjz8 Startup probe failed: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-wsjz8 Startup probe error: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-8npq2 Startup probe error: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-8npq2 Startup probe failed: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-vwlks Startup probe failed: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-ntt82 Startup probe failed: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-ntt82 Startup probe error: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-nbn5j Startup probe failed: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-nbn5j Startup probe error: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-ssfqt Startup probe failed: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-9txdf Startup probe failed: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-ssfqt Startup probe error: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-9txdf Startup probe error: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-rppfp Startup probe error: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-g26hx Startup probe failed: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-g26hx Startup probe error: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-rppfp Startup probe failed: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-lks7l Startup probe failed: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-lks7l Startup probe error: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused... 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 Created pod: e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-787nl 49m Normal ScalingReplicaSet deployment/e2e-worker-w2-2ba10a-openshift-default Scaled up replica set e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 from 4 to 5 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-2ba10a-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 49m Normal AddedInterface pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-787nl Add eth0 [10.134.0.62/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-787nl Container started 49m Normal Created pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-787nl Container created 49m Normal Pulled pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-787nl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Warning Unhealthy pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-787nl Startup probe failed: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-787nl Startup probe error: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused... 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-08ebee-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 49m Normal SuccessfulCreate replicaset/e2e-worker-w4-08ebee-openshift-default-5f5bf46967 Created pod: e2e-worker-w4-08ebee-openshift-default-5f5bf46967-w9vdp 49m Normal ScalingReplicaSet deployment/e2e-worker-w4-08ebee-openshift-default Scaled up replica set e2e-worker-w4-08ebee-openshift-default-5f5bf46967 from 2 to 3 49m Normal AddedInterface pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-w9vdp Add eth0 [10.133.0.60/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-w9vdp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-w9vdp Container created 49m Normal Started pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-w9vdp Container started 49m Warning ProbeError pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-w9vdp Startup probe error: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-w9vdp Startup probe failed: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused 48m Normal ScalingReplicaSet deployment/e2e-worker-w0-2e046a-openshift-default Scaled up replica set e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 from 6 to 10 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2e046a-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 48m Normal Started pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-8656t Container started 48m Normal AddedInterface pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-7glgv Add eth0 [10.134.0.69/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-8656t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-9vrlz Add eth0 [10.133.0.61/23] from ovn-kubernetes 48m Normal Created pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-8656t Container created 48m Normal Pulled pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-9vrlz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-9vrlz Container created 48m Normal Created pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-zbfz4 Container created 48m Normal Started pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-9vrlz Container started 48m Normal AddedInterface pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-8656t Add eth0 [10.134.0.68/23] from ovn-kubernetes 48m Normal Started pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-7glgv Container started 48m Normal Created pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-7glgv Container created 48m Normal SuccessfulCreate replicaset/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 (combined from similar events): Created pod: e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-7glgv 48m Normal SuccessfulCreate replicaset/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 Created pod: e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-9vrlz 48m Normal SuccessfulCreate replicaset/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 Created pod: e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-zbfz4 48m Normal SuccessfulCreate replicaset/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 Created pod: e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-8656t 48m Normal Pulled pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-7glgv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-zbfz4 Add eth0 [10.132.0.102/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-zbfz4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-zbfz4 Container started 48m Warning ProbeError pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-7glgv Startup probe error: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-zbfz4 Startup probe failed: Get "http://10.132.0.102:15021/healthz/ready": dial tcp 10.132.0.102:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-9vrlz Startup probe error: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-8656t Startup probe error: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-8656t Startup probe failed: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-7glgv Startup probe failed: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-zbfz4 Startup probe error: Get "http://10.132.0.102:15021/healthz/ready": dial tcp 10.132.0.102:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-9vrlz Startup probe failed: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-43b37d-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 48m Normal SuccessfulCreate replicaset/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc Created pod: e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-s5cq4 48m Normal SuccessfulCreate replicaset/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc Created pod: e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-fqhkm 48m Normal SuccessfulCreate replicaset/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc Created pod: e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-bkgjl 48m Normal ScalingReplicaSet deployment/e2e-worker-w3-43b37d-openshift-default Scaled up replica set e2e-worker-w3-43b37d-openshift-default-5dcf864fcc from 3 to 6 48m Normal AddedInterface pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-bkgjl Add eth0 [10.134.0.70/23] from ovn-kubernetes 48m Normal AddedInterface pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-s5cq4 Add eth0 [10.132.0.104/23] from ovn-kubernetes 48m Normal Started pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-s5cq4 Container started 48m Normal Created pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-s5cq4 Container created 48m Normal Pulled pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-s5cq4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Pulled pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-fqhkm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-fqhkm Container created 48m Normal Started pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-fqhkm Container started 48m Normal AddedInterface pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-fqhkm Add eth0 [10.133.0.62/23] from ovn-kubernetes 48m Normal Started pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-bkgjl Container started 48m Normal Created pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-bkgjl Container created 48m Normal Pulled pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-bkgjl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Warning ProbeError pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-s5cq4 Startup probe error: Get "http://10.132.0.104:15021/healthz/ready": dial tcp 10.132.0.104:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-s5cq4 Startup probe failed: Get "http://10.132.0.104:15021/healthz/ready": dial tcp 10.132.0.104:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-bkgjl Startup probe error: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-fqhkm Startup probe error: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-fqhkm Startup probe failed: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-bkgjl Startup probe failed: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused 48m Normal SuccessfulCreate replicaset/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 Created pod: e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-dccl7 48m Normal ScalingReplicaSet deployment/e2e-worker-w3-43b37d-openshift-default Scaled up replica set e2e-worker-w3-43b37d-openshift-default-5dcf864fcc from 6 to 9 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-43b37d-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 48m Normal SuccessfulCreate replicaset/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 Created pod: e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9hfvc 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-2ba10a-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 48m Normal ScalingReplicaSet deployment/e2e-worker-w2-2ba10a-openshift-default Scaled up replica set e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 from 5 to 8 48m Normal SuccessfulCreate replicaset/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc Created pod: e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-jwrdl 48m Normal SuccessfulCreate replicaset/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 Created pod: e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-448t7 48m Normal SuccessfulCreate replicaset/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc Created pod: e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-89tnt 48m Normal SuccessfulCreate replicaset/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc Created pod: e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-h947w 48m Normal Created pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-h947w Container created 48m Normal Started pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-dccl7 Container started 48m Normal Pulled pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-h947w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9hfvc Container started 48m Normal Started pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-448t7 Container started 48m Normal Created pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9hfvc Container created 48m Normal Pulled pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9hfvc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-89tnt Container started 48m Normal Started pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-h947w Container started 48m Normal Created pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-448t7 Container created 48m Normal Pulled pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-448t7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-h947w Add eth0 [10.132.0.108/23] from ovn-kubernetes 48m Normal Created pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-dccl7 Container created 48m Normal Pulled pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-dccl7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-dccl7 Add eth0 [10.133.0.64/23] from ovn-kubernetes 48m Normal AddedInterface pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-jwrdl Add eth0 [10.133.0.63/23] from ovn-kubernetes 48m Normal AddedInterface pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-448t7 Add eth0 [10.134.0.72/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-jwrdl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-jwrdl Container started 48m Normal Created pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-jwrdl Container created 48m Normal AddedInterface pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9hfvc Add eth0 [10.132.0.109/23] from ovn-kubernetes 48m Normal AddedInterface pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-89tnt Add eth0 [10.134.0.71/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-89tnt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-89tnt Container created 48m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-5db77fbd9f from 6 to 5 48m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5db77fbd9f Deleted pod: maas-default-gateway-openshift-default-5db77fbd9f-fhw57 48m Normal Killing pod/maas-default-gateway-openshift-default-5db77fbd9f-fhw57 Stopping container istio-proxy 48m Warning ProbeError pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-jwrdl Startup probe error: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-jwrdl Startup probe failed: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-h947w Startup probe failed: Get "http://10.132.0.108:15021/healthz/ready": dial tcp 10.132.0.108:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-h947w Startup probe error: Get "http://10.132.0.108:15021/healthz/ready": dial tcp 10.132.0.108:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9hfvc Startup probe error: Get "http://10.132.0.109:15021/healthz/ready": dial tcp 10.132.0.109:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-dccl7 Startup probe failed: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-dccl7 Startup probe error: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9hfvc Startup probe failed: Get "http://10.132.0.109:15021/healthz/ready": dial tcp 10.132.0.109:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-448t7 Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-s5cq4 Readiness probe error: Get "http://10.132.0.104:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-nf9qq Readiness probe error: Get "http://10.132.0.92:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-89tnt Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-zbfz4 Readiness probe error: Get "http://10.132.0.102:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-nf9qq Readiness probe failed: Get "http://10.132.0.92:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-zbfz4 Readiness probe failed: Get "http://10.132.0.102:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-s5cq4 Readiness probe failed: Get "http://10.132.0.104:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-89tnt Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9hfvc Startup probe error: Get "http://10.132.0.109:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-448t7 Startup probe failed: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9hfvc Startup probe failed: Get "http://10.132.0.109:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-dccl7 Startup probe failed: Get "http://10.133.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-dccl7 Startup probe error: Get "http://10.133.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-448t7 Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-448t7 Startup probe failed: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-89tnt Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-89tnt Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Normal Created pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-lqhnf Container created 48m Normal AddedInterface pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-lqhnf Add eth0 [10.132.0.112/23] from ovn-kubernetes 48m Normal AddedInterface pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-h7dlg Add eth0 [10.133.0.65/23] from ovn-kubernetes 48m Normal Started pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8gswt Container started 48m Normal ScalingReplicaSet deployment/e2e-worker-w4-08ebee-openshift-default Scaled up replica set e2e-worker-w4-08ebee-openshift-default-5f5bf46967 from 3 to 6 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-08ebee-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 48m Normal SuccessfulCreate replicaset/e2e-worker-w4-08ebee-openshift-default-5f5bf46967 Created pod: e2e-worker-w4-08ebee-openshift-default-5f5bf46967-h7dlg 48m Normal SuccessfulCreate replicaset/e2e-worker-w4-08ebee-openshift-default-5f5bf46967 Created pod: e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8gswt 48m Normal SuccessfulCreate replicaset/e2e-worker-w4-08ebee-openshift-default-5f5bf46967 Created pod: e2e-worker-w4-08ebee-openshift-default-5f5bf46967-lqhnf 48m Normal Created pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8gswt Container created 48m Normal Pulled pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8gswt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-lqhnf Container started 48m Normal Pulled pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-lqhnf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8gswt Add eth0 [10.134.0.73/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-h7dlg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-h7dlg Container created 48m Normal Started pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-h7dlg Container started 47m Warning ProbeError pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-lqhnf Startup probe error: Get "http://10.132.0.112:15021/healthz/ready": dial tcp 10.132.0.112:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-lqhnf Startup probe failed: Get "http://10.132.0.112:15021/healthz/ready": dial tcp 10.132.0.112:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-h7dlg Startup probe error: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-h7dlg Startup probe failed: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused 47m Warning Unhealthy pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-7glgv Readiness probe failed: Get "http://10.134.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-7glgv Readiness probe error: Get "http://10.134.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8gswt Startup probe failed: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8gswt Startup probe error: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused... 47m Normal Started pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-nr58r Container started 47m Normal SuccessfulCreate replicaset/e2e-worker-w4-08ebee-openshift-default-5f5bf46967 Created pod: e2e-worker-w4-08ebee-openshift-default-5f5bf46967-dd4x7 47m Normal AddedInterface pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-nr58r Add eth0 [10.133.0.66/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-nr58r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal ScalingReplicaSet deployment/e2e-worker-w4-08ebee-openshift-default Scaled up replica set e2e-worker-w4-08ebee-openshift-default-5f5bf46967 from 6 to 9 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-08ebee-openshift-default New size: 9; reason: All metrics below target 47m Normal Created pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-nr58r Container created 47m Normal AddedInterface pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8n7fg Add eth0 [10.132.0.118/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8n7fg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8n7fg Container created 47m Normal Started pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8n7fg Container started 47m Normal Started pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-dd4x7 Container started 47m Normal Created pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-dd4x7 Container created 47m Normal SuccessfulCreate replicaset/e2e-worker-w4-08ebee-openshift-default-5f5bf46967 Created pod: e2e-worker-w4-08ebee-openshift-default-5f5bf46967-nr58r 47m Normal Pulled pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-dd4x7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-dd4x7 Add eth0 [10.134.0.74/23] from ovn-kubernetes 47m Normal SuccessfulCreate replicaset/e2e-worker-w4-08ebee-openshift-default-5f5bf46967 Created pod: e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8n7fg 47m Warning Unhealthy pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8n7fg Startup probe failed: Get "http://10.132.0.118:15021/healthz/ready": dial tcp 10.132.0.118:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-nr58r Startup probe error: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-nr58r Startup probe failed: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8n7fg Startup probe error: Get "http://10.132.0.118:15021/healthz/ready": dial tcp 10.132.0.118:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-dd4x7 Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-dd4x7 Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-rkgz4 Readiness probe failed: Get "http://10.132.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-tczdj Readiness probe failed: Get "http://10.134.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-tczdj Readiness probe error: Get "http://10.134.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-rkgz4 Readiness probe error: Get "http://10.132.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-fqhkm Readiness probe failed: Get "http://10.133.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-fqhkm Readiness probe error: Get "http://10.133.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-lqhnf Readiness probe failed: Get "http://10.132.0.112:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-lqhnf Readiness probe error: Get "http://10.132.0.112:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9hfvc Readiness probe error: Get "http://10.132.0.109:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-h7dlg Readiness probe error: Get "http://10.133.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-h7dlg Readiness probe failed: Get "http://10.133.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9hfvc Readiness probe failed: Get "http://10.132.0.109:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8n7fg Startup probe error: Get "http://10.132.0.118:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-dccl7 Readiness probe failed: Get "http://10.133.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-dccl7 Readiness probe error: Get "http://10.133.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-nr58r Startup probe error: Get "http://10.133.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-nr58r Startup probe failed: Get "http://10.133.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8n7fg Startup probe failed: Get "http://10.132.0.118:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8gswt Readiness probe failed: Get "http://10.134.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-cfx67 Readiness probe error: Get "http://10.134.0.38:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-cfx67 Readiness probe failed: Get "http://10.134.0.38:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8gswt Readiness probe error: Get "http://10.134.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-dd4x7 Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-dd4x7 Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Normal SuccessfulCreate replicaset/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 (combined from similar events): Created pod: e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-vt95w 47m Normal SuccessfulCreate replicaset/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc (combined from similar events): Created pod: e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-jp7np 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-43b37d-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-2ba10a-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 47m Normal ScalingReplicaSet deployment/e2e-worker-w3-43b37d-openshift-default Scaled up replica set e2e-worker-w3-43b37d-openshift-default-5dcf864fcc from 9 to 10 47m Normal SuccessfulCreate replicaset/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 Created pod: e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-zsn6g 47m Normal ScalingReplicaSet deployment/e2e-worker-w2-2ba10a-openshift-default Scaled up replica set e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 from 8 to 10 47m Normal Pulled pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-vt95w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-zsn6g Container created 47m Normal Created pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-vt95w Container created 47m Normal Started pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-zsn6g Container started 47m Normal Started pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-jp7np Container started 47m Normal AddedInterface pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-zsn6g Add eth0 [10.133.0.67/23] from ovn-kubernetes 47m Normal AddedInterface pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-jp7np Add eth0 [10.132.0.121/23] from ovn-kubernetes 47m Normal Created pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-jp7np Container created 47m Normal Pulled pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-jp7np Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-vt95w Add eth0 [10.132.0.122/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-zsn6g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Started pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-vt95w Container started 47m Warning ProbeError pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-jp7np Startup probe error: Get "http://10.132.0.121:15021/healthz/ready": dial tcp 10.132.0.121:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-jp7np Startup probe failed: Get "http://10.132.0.121:15021/healthz/ready": dial tcp 10.132.0.121:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-vt95w Startup probe error: Get "http://10.132.0.122:15021/healthz/ready": dial tcp 10.132.0.122:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-vt95w Startup probe failed: Get "http://10.132.0.122:15021/healthz/ready": dial tcp 10.132.0.122:15021: connect: connection refused 47m Warning Unhealthy pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-zsn6g Startup probe failed: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-zsn6g Startup probe error: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused... 47m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5db77fbd9f Deleted pod: maas-default-gateway-openshift-default-5db77fbd9f-k9fn7 47m Warning ProbeError pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-95zbb Readiness probe error: Get "http://10.132.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-95zbb Readiness probe failed: Get "http://10.132.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Normal Killing pod/maas-default-gateway-openshift-default-5db77fbd9f-k9fn7 Stopping container istio-proxy 47m Warning ProbeError pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9rm2s Readiness probe error: Get "http://10.132.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9rm2s Readiness probe failed: Get "http://10.132.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Normal AddedInterface pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-cb9wn Add eth0 [10.134.0.79/23] from ovn-kubernetes 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-08ebee-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 47m Normal ScalingReplicaSet deployment/e2e-worker-w4-08ebee-openshift-default Scaled up replica set e2e-worker-w4-08ebee-openshift-default-5f5bf46967 from 9 to 10 47m Normal SuccessfulCreate replicaset/e2e-worker-w4-08ebee-openshift-default-5f5bf46967 (combined from similar events): Created pod: e2e-worker-w4-08ebee-openshift-default-5f5bf46967-cb9wn 47m Normal Pulled pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-cb9wn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-cb9wn Container created 47m Normal Started pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-cb9wn Container started 46m Warning ProbeError pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-cb9wn Startup probe error: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused... 46m Normal Killing pod/maas-default-gateway-openshift-default-5db77fbd9f-mg4hv Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5db77fbd9f Deleted pod: maas-default-gateway-openshift-default-5db77fbd9f-mg4hv 46m Warning Unhealthy pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-cb9wn Startup probe failed: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-5lkhc Readiness probe failed: Get "http://10.134.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-5lkhc Readiness probe error: Get "http://10.134.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning ProbeError pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-t8fq4 Readiness probe error: Get "http://10.132.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-t8fq4 Readiness probe failed: Get "http://10.132.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning Unhealthy pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-448t7 Readiness probe failed: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-448t7 Readiness probe error: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5db77fbd9f Deleted pod: maas-default-gateway-openshift-default-5db77fbd9f-nqsf5 46m Normal Killing pod/maas-default-gateway-openshift-default-5db77fbd9f-nqsf5 Stopping container istio-proxy 46m Normal AddedLabel gateway/e2e-ait-f7dea32d Added label istio.io/rev=openshift-gateway to gateway e2e-ait-f7dea32d 46m Normal SuccessfulCreate replicaset/e2e-ait-f7dea32d-openshift-default-5cf5f998fc Created pod: e2e-ait-f7dea32d-openshift-default-5cf5f998fc-dt4cr 46m Normal ScalingReplicaSet deployment/e2e-ait-f7dea32d-openshift-default Scaled up replica set e2e-ait-f7dea32d-openshift-default-5cf5f998fc from 0 to 1 46m Normal AddedInterface pod/e2e-ait-f7dea32d-openshift-default-5cf5f998fc-dt4cr Add eth0 [10.132.0.138/23] from ovn-kubernetes 46m Normal AddedLabel gateway/e2e-ait-dc369ed0 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-dc369ed0 46m Normal ScalingReplicaSet deployment/e2e-ait-dc369ed0-openshift-default Scaled up replica set e2e-ait-dc369ed0-openshift-default-858c684c57 from 0 to 1 46m Normal SuccessfulCreate replicaset/e2e-ait-dc369ed0-openshift-default-858c684c57 Created pod: e2e-ait-dc369ed0-openshift-default-858c684c57-npckw 46m Normal AddedInterface pod/e2e-ait-dc369ed0-openshift-default-858c684c57-npckw Add eth0 [10.134.0.80/23] from ovn-kubernetes 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-f7dea32d-79976f6559 Created pod: payload-pre-processing-e2e-ait-f7dea32d-79976f6559-8pgqr 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-f7dea32d Scaled up replica set payload-processing-e2e-ait-f7dea32d-747d8fcc7d from 0 to 1 46m Warning FailedCreate replicaset/payload-processing-e2e-ait-f7dea32d-747d8fcc7d Error creating: pods "payload-processing-e2e-ait-f7dea32d-747d8fcc7d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-f7dea32d: serviceaccount "payload-processing-e2e-ait-f7dea32d" not found 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-f7dea32d-79976f6559 Error creating: pods "payload-pre-processing-e2e-ait-f7dea32d-79976f6559-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-f7dea32d: serviceaccount "payload-processing-e2e-ait-f7dea32d" not found 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-f7dea32d Scaled up replica set payload-pre-processing-e2e-ait-f7dea32d-79976f6559 from 0 to 1 46m Warning ProbeError pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-h947w Readiness probe error: Get "http://10.132.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-h947w Readiness probe failed: Get "http://10.132.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning Unhealthy pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-jwrdl Readiness probe failed: Get "http://10.133.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-jwrdl Readiness probe error: Get "http://10.133.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-f7dea32d-747d8fcc7d Created pod: payload-processing-e2e-ait-f7dea32d-747d8fcc7d-42zlx 46m Warning ProbeError pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-bkgjl Readiness probe error: Get "http://10.134.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning ProbeError pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-ntt82 Readiness probe error: Get "http://10.134.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-ntt82 Readiness probe failed: Get "http://10.134.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning Unhealthy pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-bkgjl Readiness probe failed: Get "http://10.134.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Normal AddedInterface pod/payload-processing-e2e-ait-f7dea32d-747d8fcc7d-42zlx Add eth0 [10.132.0.140/23] from ovn-kubernetes 46m Normal Created pod/payload-processing-e2e-ait-f7dea32d-747d8fcc7d-42zlx Container created 46m Normal Pulled pod/payload-pre-processing-e2e-ait-f7dea32d-79976f6559-8pgqr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/payload-pre-processing-e2e-ait-f7dea32d-79976f6559-8pgqr Container created 46m Normal Pulled pod/payload-processing-e2e-ait-f7dea32d-747d8fcc7d-42zlx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Started pod/payload-pre-processing-e2e-ait-f7dea32d-79976f6559-8pgqr Container started 46m Normal Started pod/payload-processing-e2e-ait-f7dea32d-747d8fcc7d-42zlx Container started 46m Normal AddedInterface pod/payload-pre-processing-e2e-ait-f7dea32d-79976f6559-8pgqr Add eth0 [10.132.0.139/23] from ovn-kubernetes 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-dc369ed0-558776d6ff Error creating: pods "payload-pre-processing-e2e-ait-dc369ed0-558776d6ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-dc369ed0: serviceaccount "payload-processing-e2e-ait-dc369ed0" not found 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-dc369ed0-558776d6ff Created pod: payload-pre-processing-e2e-ait-dc369ed0-558776d6ff-bhzch 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-dc369ed0 Scaled up replica set payload-pre-processing-e2e-ait-dc369ed0-558776d6ff from 0 to 1 46m Warning FailedCreate replicaset/payload-processing-e2e-ait-dc369ed0-5b7fd658fd Error creating: pods "payload-processing-e2e-ait-dc369ed0-5b7fd658fd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-dc369ed0: serviceaccount "payload-processing-e2e-ait-dc369ed0" not found 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-dc369ed0-5b7fd658fd Created pod: payload-processing-e2e-ait-dc369ed0-5b7fd658fd-pj5gf 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-dc369ed0 Scaled up replica set payload-processing-e2e-ait-dc369ed0-5b7fd658fd from 0 to 1 46m Normal AddedInterface pod/payload-processing-e2e-ait-dc369ed0-5b7fd658fd-pj5gf Add eth0 [10.134.0.81/23] from ovn-kubernetes 46m Normal SuccessfulCreate replicaset/e2e-worker-w6-784e7a-openshift-default-595898bd5c Created pod: e2e-worker-w6-784e7a-openshift-default-595898bd5c-nkds4 46m Normal Started pod/payload-processing-e2e-ait-dc369ed0-5b7fd658fd-pj5gf Container started 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-784e7a-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 46m Normal AddedInterface pod/payload-pre-processing-e2e-ait-dc369ed0-558776d6ff-bhzch Add eth0 [10.132.0.141/23] from ovn-kubernetes 46m Normal Pulled pod/payload-pre-processing-e2e-ait-dc369ed0-558776d6ff-bhzch Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/payload-pre-processing-e2e-ait-dc369ed0-558776d6ff-bhzch Container created 46m Normal SuccessfulCreate replicaset/e2e-worker-w6-784e7a-openshift-default-595898bd5c Created pod: e2e-worker-w6-784e7a-openshift-default-595898bd5c-r7bbc 46m Normal ScalingReplicaSet deployment/e2e-worker-w6-784e7a-openshift-default Scaled up replica set e2e-worker-w6-784e7a-openshift-default-595898bd5c from 4 to 6 46m Normal Pulled pod/payload-processing-e2e-ait-dc369ed0-5b7fd658fd-pj5gf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Started pod/payload-pre-processing-e2e-ait-dc369ed0-558776d6ff-bhzch Container started 46m Normal Created pod/payload-processing-e2e-ait-dc369ed0-5b7fd658fd-pj5gf Container created 46m Normal Created pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-r7bbc Container created 46m Normal SuccessfulCreate replicaset/e2e-worker-w5-22d634-openshift-default-89dbbf78f Created pod: e2e-worker-w5-22d634-openshift-default-89dbbf78f-q5z2q 46m Normal AddedInterface pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-nkds4 Add eth0 [10.132.0.142/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-nkds4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-nkds4 Container created 46m Normal Started pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-nkds4 Container started 46m Normal SuccessfulCreate replicaset/e2e-worker-w5-22d634-openshift-default-89dbbf78f Created pod: e2e-worker-w5-22d634-openshift-default-89dbbf78f-p6swd 46m Normal Started pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-r7bbc Container started 46m Normal AddedInterface pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-bb657 Add eth0 [10.133.0.71/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-bb657 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal SuccessfulCreate replicaset/e2e-worker-w5-22d634-openshift-default-89dbbf78f Created pod: e2e-worker-w5-22d634-openshift-default-89dbbf78f-bb657 46m Normal Created pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-bb657 Container created 46m Normal Started pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-bb657 Container started 46m Normal ScalingReplicaSet deployment/e2e-worker-w5-22d634-openshift-default Scaled up replica set e2e-worker-w5-22d634-openshift-default-89dbbf78f from 4 to 7 46m Normal Started pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-q5z2q Container started 46m Normal Created pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-q5z2q Container created 46m Normal Pulled pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-q5z2q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-q5z2q Add eth0 [10.134.0.82/23] from ovn-kubernetes 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-22d634-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 46m Normal AddedInterface pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-r7bbc Add eth0 [10.133.0.70/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-r7bbc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-p6swd Container started 46m Normal Created pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-p6swd Container created 46m Normal Pulled pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-p6swd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-p6swd Add eth0 [10.132.0.143/23] from ovn-kubernetes 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-f7dea32d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-f7dea32d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning Unhealthy pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-r7bbc Startup probe failed: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-dc369ed0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning Unhealthy pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-nkds4 Startup probe failed: Get "http://10.132.0.142:15021/healthz/ready": dial tcp 10.132.0.142:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-r7bbc Startup probe error: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused... 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-dc369ed0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning ProbeError pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-nkds4 Startup probe error: Get "http://10.132.0.142:15021/healthz/ready": dial tcp 10.132.0.142:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-p6swd Startup probe error: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-q5z2q Startup probe failed: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-p6swd Startup probe failed: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-bb657 Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-q5z2q Startup probe error: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-bb657 Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-dc369ed0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-dc369ed0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-784e7a-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 45m Normal ScalingReplicaSet deployment/e2e-worker-w6-784e7a-openshift-default Scaled up replica set e2e-worker-w6-784e7a-openshift-default-595898bd5c from 6 to 8 45m Normal SuccessfulCreate replicaset/e2e-worker-w6-784e7a-openshift-default-595898bd5c Created pod: e2e-worker-w6-784e7a-openshift-default-595898bd5c-pgwf2 45m Normal SuccessfulCreate replicaset/e2e-worker-w6-784e7a-openshift-default-595898bd5c Created pod: e2e-worker-w6-784e7a-openshift-default-595898bd5c-8cvvq 45m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-267vm 45m Normal AddedInterface pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-8cvvq Add eth0 [10.132.0.146/23] from ovn-kubernetes 45m Normal Created pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-8cvvq Container created 45m Normal Pulled pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-pgwf2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-pgwf2 Add eth0 [10.134.0.85/23] from ovn-kubernetes 45m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 45m Normal Created pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-pgwf2 Container created 45m Normal Started pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-pgwf2 Container started 45m Normal Started pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-8cvvq Container started 45m Normal Pulled pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-8cvvq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 45m Normal Created pod/e2e-ait-f7dea32d-openshift-default-5cf5f998fc-dt4cr Container created 45m Normal Pulled pod/e2e-ait-f7dea32d-openshift-default-5cf5f998fc-dt4cr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-267vm Add eth0 [10.133.0.73/23] from ovn-kubernetes 45m Normal Started pod/e2e-ait-f7dea32d-openshift-default-5cf5f998fc-dt4cr Container started 45m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-267vm Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 45m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-267vm Container created 45m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-267vm Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.814s (2.814s including waiting). Image size: 185476477 bytes. 45m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-267vm Container started 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-f7dea32d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-267vm Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-267vm Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-f7dea32d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning ProbeError pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-8cvvq Startup probe error: Get "http://10.132.0.146:15021/healthz/ready": dial tcp 10.132.0.146:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-8cvvq Startup probe failed: Get "http://10.132.0.146:15021/healthz/ready": dial tcp 10.132.0.146:15021: connect: connection refused 45m Warning Unhealthy pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-pgwf2 Startup probe failed: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused 45m Warning ProbeError pod/e2e-ait-f7dea32d-openshift-default-5cf5f998fc-dt4cr Startup probe error: Get "http://10.132.0.138:15021/healthz/ready": dial tcp 10.132.0.138:15021: connect: connection refused... 45m Warning ProbeError pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-pgwf2 Startup probe error: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-ait-f7dea32d-openshift-default-5cf5f998fc-dt4cr Startup probe failed: Get "http://10.132.0.138:15021/healthz/ready": dial tcp 10.132.0.138:15021: connect: connection refused 45m Normal Killing pod/payload-processing-e2e-ait-f7dea32d-747d8fcc7d-42zlx Stopping container payload-processing 45m Normal Killing pod/payload-pre-processing-e2e-ait-f7dea32d-79976f6559-8pgqr Stopping container payload-pre-processing 45m Normal Killing pod/e2e-ait-f7dea32d-openshift-default-5cf5f998fc-dt4cr Stopping container istio-proxy 45m Normal ScalingReplicaSet deployment/e2e-migrate-e385da9b-gw-openshift-default Scaled up replica set e2e-migrate-e385da9b-gw-openshift-default-6d84d6474f from 0 to 1 45m Normal SuccessfulCreate replicaset/e2e-migrate-e385da9b-gw-openshift-default-6d84d6474f Created pod: e2e-migrate-e385da9b-gw-openshift-default-6d84d6474f-mljgv 45m Normal AddedLabel gateway/e2e-migrate-e385da9b-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-e385da9b-gw 45m Normal AddedInterface pod/e2e-migrate-e385da9b-gw-openshift-default-6d84d6474f-mljgv Add eth0 [10.134.0.86/23] from ovn-kubernetes 45m Normal Killing pod/payload-processing-e2e-ait-dc369ed0-5b7fd658fd-pj5gf Stopping container payload-processing 45m Normal Killing pod/payload-pre-processing-e2e-ait-dc369ed0-558776d6ff-bhzch Stopping container payload-pre-processing 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-e385da9b Scaled up replica set payload-processing-e2e-migrate-e385da9b-78d97b4574 from 0 to 1 45m Normal Started pod/payload-pre-processing-e2e-migrate-e385da9b-64fcf8b6bc-zjwpw Container started 45m Normal Pulled pod/payload-pre-processing-e2e-migrate-e385da9b-64fcf8b6bc-zjwpw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-e385da9b-64fcf8b6bc-zjwpw Add eth0 [10.132.0.148/23] from ovn-kubernetes 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-e385da9b-64fcf8b6bc Created pod: payload-pre-processing-e2e-migrate-e385da9b-64fcf8b6bc-zjwpw 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-e385da9b Scaled up replica set payload-pre-processing-e2e-migrate-e385da9b-64fcf8b6bc from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-e385da9b-78d97b4574 Created pod: payload-processing-e2e-migrate-e385da9b-78d97b4574-h5mcr 45m Warning FailedCreate replicaset/payload-processing-e2e-migrate-e385da9b-78d97b4574 Error creating: pods "payload-processing-e2e-migrate-e385da9b-78d97b4574-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-e385da9b: serviceaccount "payload-processing-e2e-migrate-e385da9b" not found 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-e385da9b-64fcf8b6bc Error creating: pods "payload-pre-processing-e2e-migrate-e385da9b-64fcf8b6bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-e385da9b: serviceaccount "payload-processing-e2e-migrate-e385da9b" not found 45m Normal Created pod/payload-pre-processing-e2e-migrate-e385da9b-64fcf8b6bc-zjwpw Container created 45m Normal Started pod/payload-processing-e2e-migrate-e385da9b-78d97b4574-h5mcr Container started 45m Normal Pulled pod/payload-processing-e2e-migrate-e385da9b-78d97b4574-h5mcr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-processing-e2e-migrate-e385da9b-78d97b4574-h5mcr Container created 45m Normal AddedInterface pod/payload-processing-e2e-migrate-e385da9b-78d97b4574-h5mcr Add eth0 [10.132.0.149/23] from ovn-kubernetes 45m Normal SuccessfulCreate replicaset/e2e-worker-w6-784e7a-openshift-default-595898bd5c Created pod: e2e-worker-w6-784e7a-openshift-default-595898bd5c-lx7p6 45m Normal ScalingReplicaSet deployment/e2e-worker-w6-784e7a-openshift-default Scaled up replica set e2e-worker-w6-784e7a-openshift-default-595898bd5c from 8 to 10 45m Normal SuccessfulCreate replicaset/e2e-worker-w6-784e7a-openshift-default-595898bd5c (combined from similar events): Created pod: e2e-worker-w6-784e7a-openshift-default-595898bd5c-4h4z5 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-784e7a-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 45m Normal Started pod/e2e-ait-dc369ed0-openshift-default-858c684c57-npckw Container started 45m Normal Pulled pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-lx7p6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-ait-dc369ed0-openshift-default-858c684c57-npckw Container created 45m Normal Created pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-lx7p6 Container created 45m Normal AddedInterface pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-lx7p6 Add eth0 [10.133.0.74/23] from ovn-kubernetes 45m Normal AddedInterface pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-4h4z5 Add eth0 [10.132.0.151/23] from ovn-kubernetes 45m Normal Created pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-4h4z5 Container created 45m Normal Pulled pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-4h4z5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Pulled pod/e2e-ait-dc369ed0-openshift-default-858c684c57-npckw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-lx7p6 Container started 45m Normal Started pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-4h4z5 Container started 45m Normal SuccessfulCreate replicaset/e2e-derive-6c6e65e5-openshift-default-7586c4c97c Created pod: e2e-derive-6c6e65e5-openshift-default-7586c4c97c-p8csz 45m Warning ProbeError pod/e2e-ait-dc369ed0-openshift-default-858c684c57-npckw Startup probe error: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-ait-dc369ed0-openshift-default-858c684c57-npckw Startup probe failed: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused 45m Normal ScalingReplicaSet deployment/e2e-derive-6c6e65e5-openshift-default Scaled up replica set e2e-derive-6c6e65e5-openshift-default-7586c4c97c from 0 to 1 45m Normal Killing pod/e2e-ait-dc369ed0-openshift-default-858c684c57-npckw Stopping container istio-proxy 45m Normal AddedLabel gateway/e2e-derive-6c6e65e5 Added label istio.io/rev=openshift-gateway to gateway e2e-derive-6c6e65e5 45m Normal AddedInterface pod/e2e-derive-6c6e65e5-openshift-default-7586c4c97c-p8csz Add eth0 [10.132.0.153/23] from ovn-kubernetes 45m Normal Pulled pod/e2e-derive-6c6e65e5-openshift-default-7586c4c97c-p8csz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/e2e-derive-6c6e65e5-openshift-default-7586c4c97c-p8csz Container started 45m Normal Created pod/e2e-derive-6c6e65e5-openshift-default-7586c4c97c-p8csz Container created 45m Warning Unhealthy pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-4h4z5 Startup probe failed: Get "http://10.132.0.151:15021/healthz/ready": dial tcp 10.132.0.151:15021: connect: connection refused 45m Warning Unhealthy pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-lx7p6 Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused 45m Warning ProbeError pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-4h4z5 Startup probe error: Get "http://10.132.0.151:15021/healthz/ready": dial tcp 10.132.0.151:15021: connect: connection refused... 45m Warning ProbeError pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-lx7p6 Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused... 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-6c6e65e5 Scaled up replica set payload-pre-processing-e2e-derive-6c6e65e5-78df9fdc95 from 0 to 1 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-6c6e65e5-78df9fdc95 Error creating: pods "payload-pre-processing-e2e-derive-6c6e65e5-78df9fdc95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-6c6e65e5: serviceaccount "payload-processing-e2e-derive-6c6e65e5" not found 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-6c6e65e5 Scaled up replica set payload-processing-e2e-derive-6c6e65e5-6c89f86574 from 0 to 1 45m Warning FailedCreate replicaset/payload-processing-e2e-derive-6c6e65e5-6c89f86574 Error creating: pods "payload-processing-e2e-derive-6c6e65e5-6c89f86574-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-6c6e65e5: serviceaccount "payload-processing-e2e-derive-6c6e65e5" not found 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-6c6e65e5-78df9fdc95 Created pod: payload-pre-processing-e2e-derive-6c6e65e5-78df9fdc95-qrgvn 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-6c6e65e5-6c89f86574 Created pod: payload-processing-e2e-derive-6c6e65e5-6c89f86574-szfbh 45m Normal Pulled pod/payload-processing-e2e-derive-6c6e65e5-6c89f86574-szfbh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-processing-e2e-derive-6c6e65e5-6c89f86574-szfbh Container created 45m Normal AddedInterface pod/payload-processing-e2e-derive-6c6e65e5-6c89f86574-szfbh Add eth0 [10.134.0.89/23] from ovn-kubernetes 45m Normal Started pod/payload-pre-processing-e2e-derive-6c6e65e5-78df9fdc95-qrgvn Container started 45m Normal Created pod/payload-pre-processing-e2e-derive-6c6e65e5-78df9fdc95-qrgvn Container created 45m Normal AddedInterface pod/payload-pre-processing-e2e-derive-6c6e65e5-78df9fdc95-qrgvn Add eth0 [10.134.0.88/23] from ovn-kubernetes 45m Normal Pulled pod/payload-pre-processing-e2e-derive-6c6e65e5-78df9fdc95-qrgvn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Started pod/payload-processing-e2e-derive-6c6e65e5-6c89f86574-szfbh Container started 44m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-06a95f-openshift-default-74f57588cc Created pod: e2e-shared-a-w6-06a95f-openshift-default-74f57588cc-xlqvh 44m Normal AddedLabel gateway/e2e-shared-a-w6-06a95f Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w6-06a95f 44m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-06a95f-openshift-default Scaled up replica set e2e-shared-a-w6-06a95f-openshift-default-74f57588cc from 0 to 1 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-e385da9b-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-e385da9b-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal AddedInterface pod/e2e-shared-a-w6-06a95f-openshift-default-74f57588cc-xlqvh Add eth0 [10.134.0.91/23] from ovn-kubernetes 44m Warning Unhealthy pod/e2e-migrate-e385da9b-gw-openshift-default-6d84d6474f-mljgv Readiness probe failed: Get "http://10.134.0.86:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-6c6e65e5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-6c6e65e5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal Killing pod/payload-pre-processing-e2e-migrate-e385da9b-64fcf8b6bc-zjwpw Stopping container payload-pre-processing 44m Normal Killing pod/payload-processing-e2e-migrate-e385da9b-78d97b4574-h5mcr Stopping container payload-processing 44m Warning ProbeError pod/e2e-migrate-e385da9b-gw-openshift-default-6d84d6474f-mljgv Readiness probe error: Get "http://10.134.0.86:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w6-06a95f-7c5bb67c55 Error creating: pods "payload-pre-processing-e2e-shared-a-w6-06a95f-7c5bb67c55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-06a95f: serviceaccount "payload-processing-e2e-shared-a-w6-06a95f" not found 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w6-06a95f Scaled up replica set payload-pre-processing-e2e-shared-a-w6-06a95f-7c5bb67c55 from 0 to 1 44m Normal ScalingReplicaSet deployment/e2e-migrate-e385da9b-gw-openshift-default Scaled up replica set e2e-migrate-e385da9b-gw-openshift-default-6d84d6474f from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w6-06a95f-6c459f96db Created pod: payload-processing-e2e-shared-a-w6-06a95f-6c459f96db-6pb7q 44m Warning FailedCreate replicaset/e2e-migrate-e385da9b-gw-openshift-default-6d84d6474f Error creating: pods "e2e-migrate-e385da9b-gw-openshift-default-6d84d6474f-" is forbidden: error looking up service account openshift-ingress/e2e-migrate-e385da9b-gw-openshift-default: serviceaccount "e2e-migrate-e385da9b-gw-openshift-default" not found 44m Normal Created pod/e2e-migrate-e385da9b-gw-openshift-default-6d84d6474f-mljgv Container created 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w6-06a95f-7c5bb67c55 Created pod: payload-pre-processing-e2e-shared-a-w6-06a95f-7c5bb67c55-dkr8j 44m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w6-06a95f-6c459f96db Error creating: pods "payload-processing-e2e-shared-a-w6-06a95f-6c459f96db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-06a95f: serviceaccount "payload-processing-e2e-shared-a-w6-06a95f" not found 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w6-06a95f Scaled up replica set payload-processing-e2e-shared-a-w6-06a95f-6c459f96db from 0 to 1 44m Normal Pulled pod/e2e-migrate-e385da9b-gw-openshift-default-6d84d6474f-mljgv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Killing pod/e2e-migrate-e385da9b-gw-openshift-default-6d84d6474f-mljgv Stopping container istio-proxy 44m Normal AddedInterface pod/payload-processing-e2e-shared-a-w6-06a95f-6c459f96db-6pb7q Add eth0 [10.134.0.93/23] from ovn-kubernetes 44m Normal Started pod/payload-processing-e2e-shared-a-w6-06a95f-6c459f96db-6pb7q Container started 44m Normal Started pod/payload-pre-processing-e2e-shared-a-w6-06a95f-7c5bb67c55-dkr8j Container started 44m Normal Created pod/payload-processing-e2e-shared-a-w6-06a95f-6c459f96db-6pb7q Container created 44m Normal Pulled pod/payload-processing-e2e-shared-a-w6-06a95f-6c459f96db-6pb7q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Started pod/e2e-migrate-e385da9b-gw-openshift-default-6d84d6474f-mljgv Container started 44m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w6-06a95f-7c5bb67c55-dkr8j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-pre-processing-e2e-shared-a-w6-06a95f-7c5bb67c55-dkr8j Container created 44m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w6-06a95f-7c5bb67c55-dkr8j Add eth0 [10.134.0.92/23] from ovn-kubernetes 44m Normal AddedLabel gateway/e2e-shared-a-w3-aaa3c6 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-aaa3c6 44m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-aaa3c6-openshift-default Scaled up replica set e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9 from 0 to 1 44m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9 Created pod: e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9-qmc4m 44m Normal AddedInterface pod/e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9-qmc4m Add eth0 [10.134.0.94/23] from ovn-kubernetes 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w3-aaa3c6-845674cd68 Created pod: payload-processing-e2e-shared-a-w3-aaa3c6-845674cd68-8fgr7 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w3-aaa3c6-64445b56bc Error creating: pods "payload-pre-processing-e2e-shared-a-w3-aaa3c6-64445b56bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-aaa3c6: serviceaccount "payload-processing-e2e-shared-a-w3-aaa3c6" not found 44m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w3-aaa3c6-845674cd68 Error creating: pods "payload-processing-e2e-shared-a-w3-aaa3c6-845674cd68-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-aaa3c6: serviceaccount "payload-processing-e2e-shared-a-w3-aaa3c6" not found 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w3-aaa3c6-64445b56bc Created pod: payload-pre-processing-e2e-shared-a-w3-aaa3c6-64445b56bc-9wn9m 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w3-aaa3c6 Scaled up replica set payload-processing-e2e-shared-a-w3-aaa3c6-845674cd68 from 0 to 1 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w3-aaa3c6 Scaled up replica set payload-pre-processing-e2e-shared-a-w3-aaa3c6-64445b56bc from 0 to 1 44m Normal Created pod/payload-processing-e2e-shared-a-w3-aaa3c6-845674cd68-8fgr7 Container created 44m Normal Pulled pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-8wvv6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-7qslq Add eth0 [10.133.0.75/23] from ovn-kubernetes 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-fc684b-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 44m Normal Pulled pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-7qslq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal ScalingReplicaSet deployment/e2e-worker-w1-fc684b-openshift-default Scaled up replica set e2e-worker-w1-fc684b-openshift-default-7fcf678997 from 4 to 6 44m Normal SuccessfulCreate replicaset/e2e-worker-w1-fc684b-openshift-default-7fcf678997 Created pod: e2e-worker-w1-fc684b-openshift-default-7fcf678997-8wvv6 44m Normal Pulled pod/payload-processing-e2e-shared-a-w3-aaa3c6-845674cd68-8fgr7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-processing-e2e-shared-a-w3-aaa3c6-845674cd68-8fgr7 Add eth0 [10.132.0.158/23] from ovn-kubernetes 44m Normal Started pod/payload-processing-e2e-shared-a-w3-aaa3c6-845674cd68-8fgr7 Container started 44m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w3-aaa3c6-64445b56bc-9wn9m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w3-aaa3c6-64445b56bc-9wn9m Add eth0 [10.134.0.95/23] from ovn-kubernetes 44m Normal SuccessfulCreate replicaset/e2e-worker-w1-fc684b-openshift-default-7fcf678997 Created pod: e2e-worker-w1-fc684b-openshift-default-7fcf678997-7qslq 44m Normal AddedInterface pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-8wvv6 Add eth0 [10.132.0.159/23] from ovn-kubernetes 44m Normal Started pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-7qslq Container started 44m Normal Created pod/payload-pre-processing-e2e-shared-a-w3-aaa3c6-64445b56bc-9wn9m Container created 44m Normal Created pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-7qslq Container created 44m Normal Started pod/payload-pre-processing-e2e-shared-a-w3-aaa3c6-64445b56bc-9wn9m Container started 44m Normal Started pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-8wvv6 Container started 44m Normal Created pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-8wvv6 Container created 44m Warning ProbeError pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-7qslq Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused... 44m Normal Killing pod/payload-pre-processing-e2e-derive-6c6e65e5-78df9fdc95-qrgvn Stopping container payload-pre-processing 44m Warning Unhealthy pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-8wvv6 Startup probe failed: Get "http://10.132.0.159:15021/healthz/ready": dial tcp 10.132.0.159:15021: connect: connection refused 44m Warning ProbeError pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-8wvv6 Startup probe error: Get "http://10.132.0.159:15021/healthz/ready": dial tcp 10.132.0.159:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-7qslq Startup probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused 44m Normal Killing pod/payload-processing-e2e-derive-6c6e65e5-6c89f86574-szfbh Stopping container payload-processing 44m Normal Killing pod/e2e-derive-6c6e65e5-openshift-default-7586c4c97c-p8csz Stopping container istio-proxy 44m Normal AddedLabel gateway/e2e-mt-1131a29d Added label istio.io/rev=openshift-gateway to gateway e2e-mt-1131a29d 44m Normal SuccessfulCreate replicaset/e2e-mt-1131a29d-openshift-default-54ccb47c65 Created pod: e2e-mt-1131a29d-openshift-default-54ccb47c65-fmfnq 44m Normal ScalingReplicaSet deployment/e2e-mt-1131a29d-openshift-default Scaled up replica set e2e-mt-1131a29d-openshift-default-54ccb47c65 from 0 to 1 44m Normal AddedInterface pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-fmfnq Add eth0 [10.134.0.98/23] from ovn-kubernetes 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-aaa3c6-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-1131a29d-6d7ccc5c4d Created pod: payload-processing-e2e-mt-1131a29d-6d7ccc5c4d-mdml5 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-aaa3c6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-1131a29d Scaled up replica set payload-pre-processing-e2e-mt-1131a29d-79fbb5fd4c from 0 to 1 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-1131a29d Scaled up replica set payload-processing-e2e-mt-1131a29d-6d7ccc5c4d from 0 to 1 44m Warning FailedCreate replicaset/payload-processing-e2e-mt-1131a29d-6d7ccc5c4d Error creating: pods "payload-processing-e2e-mt-1131a29d-6d7ccc5c4d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-1131a29d: serviceaccount "payload-processing-e2e-mt-1131a29d" not found 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-1131a29d-79fbb5fd4c Created pod: payload-pre-processing-e2e-mt-1131a29d-79fbb5fd4c-rqqk2 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-1131a29d-79fbb5fd4c Error creating: pods "payload-pre-processing-e2e-mt-1131a29d-79fbb5fd4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-1131a29d: serviceaccount "payload-processing-e2e-mt-1131a29d" not found 44m Normal Started pod/payload-pre-processing-e2e-mt-1131a29d-79fbb5fd4c-rqqk2 Container started 44m Normal Started pod/payload-processing-e2e-mt-1131a29d-6d7ccc5c4d-mdml5 Container started 44m Normal AddedInterface pod/payload-pre-processing-e2e-mt-1131a29d-79fbb5fd4c-rqqk2 Add eth0 [10.132.0.163/23] from ovn-kubernetes 44m Normal Pulled pod/payload-pre-processing-e2e-mt-1131a29d-79fbb5fd4c-rqqk2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-processing-e2e-mt-1131a29d-6d7ccc5c4d-mdml5 Container created 44m Normal Pulled pod/payload-processing-e2e-mt-1131a29d-6d7ccc5c4d-mdml5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-processing-e2e-mt-1131a29d-6d7ccc5c4d-mdml5 Add eth0 [10.132.0.162/23] from ovn-kubernetes 44m Normal Created pod/payload-pre-processing-e2e-mt-1131a29d-79fbb5fd4c-rqqk2 Container created 44m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-db7419-openshift-default-9dd55d5bc Created pod: e2e-shared-b-w6-db7419-openshift-default-9dd55d5bc-q2xvw 44m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-db7419-openshift-default Scaled up replica set e2e-shared-b-w6-db7419-openshift-default-9dd55d5bc from 0 to 1 44m Normal AddedLabel gateway/e2e-shared-b-w6-db7419 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w6-db7419 44m Normal AddedInterface pod/e2e-shared-b-w6-db7419-openshift-default-9dd55d5bc-q2xvw Add eth0 [10.132.0.164/23] from ovn-kubernetes 44m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-3bd0f6-openshift-default Scaled up replica set e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77 from 0 to 1 44m Normal AddedLabel gateway/e2e-shared-b-w3-3bd0f6 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-3bd0f6 44m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77 Created pod: e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-6mjhh 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w6-db7419 Scaled up replica set payload-pre-processing-e2e-shared-b-w6-db7419-79447f7f59 from 0 to 1 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w6-db7419-79447f7f59 Error creating: pods "payload-pre-processing-e2e-shared-b-w6-db7419-79447f7f59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-db7419: serviceaccount "payload-processing-e2e-shared-b-w6-db7419" not found 44m Normal AddedInterface pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-6mjhh Add eth0 [10.134.0.99/23] from ovn-kubernetes 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w6-db7419 Scaled up replica set payload-processing-e2e-shared-b-w6-db7419-84c96f7884 from 0 to 1 44m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w6-db7419-84c96f7884 Error creating: pods "payload-processing-e2e-shared-b-w6-db7419-84c96f7884-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-db7419: serviceaccount "payload-processing-e2e-shared-b-w6-db7419" not found 44m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w6-db7419-79447f7f59-h9nmc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w6-db7419-79447f7f59 Created pod: payload-pre-processing-e2e-shared-b-w6-db7419-79447f7f59-h9nmc 44m Normal AddedInterface pod/payload-processing-e2e-shared-b-w6-db7419-84c96f7884-pqpn2 Add eth0 [10.132.0.166/23] from ovn-kubernetes 44m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w6-db7419-79447f7f59-h9nmc Add eth0 [10.132.0.165/23] from ovn-kubernetes 44m Normal Created pod/payload-pre-processing-e2e-shared-b-w6-db7419-79447f7f59-h9nmc Container created 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w6-db7419-84c96f7884 Created pod: payload-processing-e2e-shared-b-w6-db7419-84c96f7884-pqpn2 44m Normal Started pod/payload-pre-processing-e2e-shared-b-w6-db7419-79447f7f59-h9nmc Container started 44m Normal Created pod/payload-processing-e2e-shared-b-w6-db7419-84c96f7884-pqpn2 Container created 44m Normal Pulled pod/payload-processing-e2e-shared-b-w6-db7419-84c96f7884-pqpn2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-processing-e2e-shared-b-w6-db7419-84c96f7884-pqpn2 Container started 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w3-3bd0f6 Scaled up replica set payload-pre-processing-e2e-shared-b-w3-3bd0f6-858c665c67 from 0 to 1 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w3-3bd0f6-858c665c67 Error creating: pods "payload-pre-processing-e2e-shared-b-w3-3bd0f6-858c665c67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-3bd0f6: serviceaccount "payload-processing-e2e-shared-b-w3-3bd0f6" not found 43m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w3-3bd0f6-c58c9678 Error creating: pods "payload-processing-e2e-shared-b-w3-3bd0f6-c58c9678-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-3bd0f6: serviceaccount "payload-processing-e2e-shared-b-w3-3bd0f6" not found 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w3-3bd0f6 Scaled up replica set payload-processing-e2e-shared-b-w3-3bd0f6-c58c9678 from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w3-3bd0f6-858c665c67 Created pod: payload-pre-processing-e2e-shared-b-w3-3bd0f6-858c665c67-btrf6 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w3-3bd0f6-c58c9678 Created pod: payload-processing-e2e-shared-b-w3-3bd0f6-c58c9678-lhc7t 43m Normal Pulled pod/payload-processing-e2e-shared-b-w3-3bd0f6-c58c9678-lhc7t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w3-3bd0f6-858c665c67-btrf6 Add eth0 [10.132.0.168/23] from ovn-kubernetes 43m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w3-3bd0f6-858c665c67-btrf6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-processing-e2e-shared-b-w3-3bd0f6-c58c9678-lhc7t Add eth0 [10.132.0.169/23] from ovn-kubernetes 43m Normal Started pod/payload-pre-processing-e2e-shared-b-w3-3bd0f6-858c665c67-btrf6 Container started 43m Normal Created pod/payload-processing-e2e-shared-b-w3-3bd0f6-c58c9678-lhc7t Container created 43m Normal Created pod/payload-pre-processing-e2e-shared-b-w3-3bd0f6-858c665c67-btrf6 Container created 43m Normal Started pod/payload-processing-e2e-shared-b-w3-3bd0f6-c58c9678-lhc7t Container started 43m Normal SuccessfulCreate replicaset/e2e-worker-w1-fc684b-openshift-default-7fcf678997 Created pod: e2e-worker-w1-fc684b-openshift-default-7fcf678997-v56n2 43m Normal ScalingReplicaSet deployment/e2e-worker-w1-fc684b-openshift-default Scaled up replica set e2e-worker-w1-fc684b-openshift-default-7fcf678997 from 6 to 10 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-fc684b-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 43m Normal SuccessfulCreate replicaset/e2e-worker-w1-fc684b-openshift-default-7fcf678997 Created pod: e2e-worker-w1-fc684b-openshift-default-7fcf678997-ghbxb 43m Normal SuccessfulCreate replicaset/e2e-worker-w1-fc684b-openshift-default-7fcf678997 Created pod: e2e-worker-w1-fc684b-openshift-default-7fcf678997-dnzl8 43m Normal SuccessfulCreate replicaset/e2e-worker-w1-fc684b-openshift-default-7fcf678997 (combined from similar events): Created pod: e2e-worker-w1-fc684b-openshift-default-7fcf678997-cwggq 43m Normal Created pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-dnzl8 Container created 43m Normal Created pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-cwggq Container created 43m Normal Pulled pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-v56n2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Created pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-v56n2 Container created 43m Normal AddedInterface pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-cwggq Add eth0 [10.132.0.171/23] from ovn-kubernetes 43m Normal Started pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-v56n2 Container started 43m Normal Pulled pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-cwggq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Pulled pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-dnzl8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-cwggq Container started 43m Normal Started pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-ghbxb Container started 43m Normal Created pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-ghbxb Container created 43m Normal Pulled pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-ghbxb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-ghbxb Add eth0 [10.132.0.170/23] from ovn-kubernetes 43m Normal Started pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-dnzl8 Container started 43m Normal AddedInterface pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-dnzl8 Add eth0 [10.134.0.101/23] from ovn-kubernetes 43m Normal AddedInterface pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-v56n2 Add eth0 [10.133.0.76/23] from ovn-kubernetes 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-1131a29d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-1131a29d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-3bd0f6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-3bd0f6-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal SuccessfulCreate replicaset/e2e-mt-1fa4b8b6-openshift-default-6765496769 Created pod: e2e-mt-1fa4b8b6-openshift-default-6765496769-47v7d 43m Normal ScalingReplicaSet deployment/e2e-mt-1fa4b8b6-openshift-default Scaled up replica set e2e-mt-1fa4b8b6-openshift-default-6765496769 from 0 to 1 43m Normal AddedLabel gateway/e2e-mt-1fa4b8b6 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-1fa4b8b6 43m Normal AddedInterface pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-47v7d Add eth0 [10.132.0.172/23] from ovn-kubernetes 43m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5db77fbd9f Created pod: maas-default-gateway-openshift-default-5db77fbd9f-cr7nq 43m Warning Unhealthy pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-ghbxb Startup probe failed: Get "http://10.132.0.170:15021/healthz/ready": dial tcp 10.132.0.170:15021: connect: connection refused 43m Warning ProbeError pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-v56n2 Startup probe error: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-v56n2 Startup probe failed: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused 43m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5db77fbd9f Created pod: maas-default-gateway-openshift-default-5db77fbd9f-82d8r 43m Warning ProbeError pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-ghbxb Startup probe error: Get "http://10.132.0.170:15021/healthz/ready": dial tcp 10.132.0.170:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-cwggq Startup probe failed: Get "http://10.132.0.171:15021/healthz/ready": dial tcp 10.132.0.171:15021: connect: connection refused 43m Normal Pulled pod/maas-default-gateway-openshift-default-5db77fbd9f-82d8r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/maas-default-gateway-openshift-default-5db77fbd9f-82d8r Add eth0 [10.132.0.173/23] from ovn-kubernetes 43m Warning ProbeError pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-cwggq Startup probe error: Get "http://10.132.0.171:15021/healthz/ready": dial tcp 10.132.0.171:15021: connect: connection refused... 43m Normal Created pod/maas-default-gateway-openshift-default-5db77fbd9f-82d8r Container created 43m Normal Started pod/maas-default-gateway-openshift-default-5db77fbd9f-82d8r Container started 43m Normal Started pod/maas-default-gateway-openshift-default-5db77fbd9f-cr7nq Container started 43m Normal Created pod/maas-default-gateway-openshift-default-5db77fbd9f-cr7nq Container created 43m Normal Pulled pod/maas-default-gateway-openshift-default-5db77fbd9f-cr7nq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/maas-default-gateway-openshift-default-5db77fbd9f-cr7nq Add eth0 [10.132.0.174/23] from ovn-kubernetes 43m Warning Unhealthy pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-ghbxb Startup probe failed: Get "http://10.132.0.170:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Warning ProbeError pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-ghbxb Startup probe error: Get "http://10.132.0.170:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Warning ProbeError pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-dnzl8 Startup probe error: Get "http://10.134.0.101:15021/healthz/ready": dial tcp 10.134.0.101:15021: connect: connection refused... 43m Normal Started pod/e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9-qmc4m Container started 43m Warning Unhealthy pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-dnzl8 Startup probe failed: Get "http://10.134.0.101:15021/healthz/ready": dial tcp 10.134.0.101:15021: connect: connection refused 43m Normal Pulled pod/e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9-qmc4m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Created pod/e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9-qmc4m Container created 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-1fa4b8b6 Scaled up replica set payload-pre-processing-e2e-mt-1fa4b8b6-675f9bbd4c from 0 to 1 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-1fa4b8b6 Scaled up replica set payload-processing-e2e-mt-1fa4b8b6-68f88d6f68 from 0 to 1 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-1fa4b8b6-675f9bbd4c Error creating: pods "payload-pre-processing-e2e-mt-1fa4b8b6-675f9bbd4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-1fa4b8b6: serviceaccount "payload-processing-e2e-mt-1fa4b8b6" not found 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-1fa4b8b6-675f9bbd4c Created pod: payload-pre-processing-e2e-mt-1fa4b8b6-675f9bbd4c-xlqfw 43m Warning FailedCreate replicaset/payload-processing-e2e-mt-1fa4b8b6-68f88d6f68 Error creating: pods "payload-processing-e2e-mt-1fa4b8b6-68f88d6f68-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-1fa4b8b6: serviceaccount "payload-processing-e2e-mt-1fa4b8b6" not found 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-1fa4b8b6-68f88d6f68 Created pod: payload-processing-e2e-mt-1fa4b8b6-68f88d6f68-c9869 43m Normal Pulled pod/e2e-shared-a-w6-06a95f-openshift-default-74f57588cc-xlqvh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Created pod/e2e-shared-a-w6-06a95f-openshift-default-74f57588cc-xlqvh Container created 43m Normal Started pod/e2e-shared-a-w6-06a95f-openshift-default-74f57588cc-xlqvh Container started 43m Normal Pulled pod/payload-processing-e2e-mt-1fa4b8b6-68f88d6f68-c9869 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-pre-processing-e2e-mt-1fa4b8b6-675f9bbd4c-xlqfw Add eth0 [10.134.0.103/23] from ovn-kubernetes 43m Normal Started pod/payload-pre-processing-e2e-mt-1fa4b8b6-675f9bbd4c-xlqfw Container started 43m Normal Created pod/payload-pre-processing-e2e-mt-1fa4b8b6-675f9bbd4c-xlqfw Container created 43m Normal Started pod/payload-processing-e2e-mt-1fa4b8b6-68f88d6f68-c9869 Container started 43m Normal Created pod/payload-processing-e2e-mt-1fa4b8b6-68f88d6f68-c9869 Container created 43m Normal AddedInterface pod/payload-processing-e2e-mt-1fa4b8b6-68f88d6f68-c9869 Add eth0 [10.134.0.104/23] from ovn-kubernetes 43m Normal Pulled pod/payload-pre-processing-e2e-mt-1fa4b8b6-675f9bbd4c-xlqfw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-z794f 43m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-82d8r Startup probe error: Get "http://10.132.0.173:15021/healthz/ready": dial tcp 10.132.0.173:15021: connect: connection refused... 43m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-z794f Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 43m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-z794f Add eth0 [10.132.0.175/23] from ovn-kubernetes 43m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-82d8r Startup probe failed: Get "http://10.132.0.173:15021/healthz/ready": dial tcp 10.132.0.173:15021: connect: connection refused 43m Warning Unhealthy pod/e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9-qmc4m Startup probe failed: Get "http://10.134.0.94:15021/healthz/ready": dial tcp 10.134.0.94:15021: connect: connection refused 43m Warning ProbeError pod/e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9-qmc4m Startup probe error: Get "http://10.134.0.94:15021/healthz/ready": dial tcp 10.134.0.94:15021: connect: connection refused... 43m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-cr7nq Startup probe error: Get "http://10.132.0.174:15021/healthz/ready": dial tcp 10.132.0.174:15021: connect: connection refused... 43m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-cr7nq Startup probe failed: Get "http://10.132.0.174:15021/healthz/ready": dial tcp 10.132.0.174:15021: connect: connection refused 43m Warning Unhealthy pod/e2e-shared-a-w6-06a95f-openshift-default-74f57588cc-xlqvh Startup probe failed: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused 43m Warning ProbeError pod/e2e-shared-a-w6-06a95f-openshift-default-74f57588cc-xlqvh Startup probe error: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused... 43m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-z794f Container started 43m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-z794f Container created 43m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-z794f Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.258s (3.258s including waiting). Image size: 185476477 bytes. 43m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-z794f Readiness probe failed: HTTP probe failed with statuscode: 503 43m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-z794f Readiness probe error: HTTP probe failed with statuscode: 503... 43m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5db77fbd9f Created pod: maas-default-gateway-openshift-default-5db77fbd9f-r8hqd 43m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-5db77fbd9f from 4 to 7 43m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 43m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5db77fbd9f Created pod: maas-default-gateway-openshift-default-5db77fbd9f-z7hjw 43m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5db77fbd9f Created pod: maas-default-gateway-openshift-default-5db77fbd9f-2qx7m 43m Normal AddedInterface pod/maas-default-gateway-openshift-default-5db77fbd9f-z7hjw Add eth0 [10.134.0.106/23] from ovn-kubernetes 43m Normal AddedInterface pod/maas-default-gateway-openshift-default-5db77fbd9f-r8hqd Add eth0 [10.134.0.105/23] from ovn-kubernetes 43m Normal Created pod/maas-default-gateway-openshift-default-5db77fbd9f-r8hqd Container created 43m Normal Started pod/maas-default-gateway-openshift-default-5db77fbd9f-z7hjw Container started 43m Normal Created pod/maas-default-gateway-openshift-default-5db77fbd9f-z7hjw Container created 43m Normal Pulled pod/maas-default-gateway-openshift-default-5db77fbd9f-z7hjw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/maas-default-gateway-openshift-default-5db77fbd9f-r8hqd Container started 43m Normal Started pod/maas-default-gateway-openshift-default-5db77fbd9f-2qx7m Container started 43m Normal Created pod/maas-default-gateway-openshift-default-5db77fbd9f-2qx7m Container created 43m Normal Pulled pod/maas-default-gateway-openshift-default-5db77fbd9f-r8hqd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Pulled pod/maas-default-gateway-openshift-default-5db77fbd9f-2qx7m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/maas-default-gateway-openshift-default-5db77fbd9f-2qx7m Add eth0 [10.133.0.78/23] from ovn-kubernetes 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-06a95f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-06a95f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-2qx7m Startup probe error: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused... 43m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-r8hqd Startup probe error: Get "http://10.134.0.105:15021/healthz/ready": dial tcp 10.134.0.105:15021: connect: connection refused... 43m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-r8hqd Startup probe failed: Get "http://10.134.0.105:15021/healthz/ready": dial tcp 10.134.0.105:15021: connect: connection refused 43m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-2qx7m Startup probe failed: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused 43m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-z7hjw Startup probe failed: Get "http://10.134.0.106:15021/healthz/ready": dial tcp 10.134.0.106:15021: connect: connection refused 43m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-z7hjw Startup probe error: Get "http://10.134.0.106:15021/healthz/ready": dial tcp 10.134.0.106:15021: connect: connection refused... 43m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-r8hqd Startup probe error: Get "http://10.134.0.105:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-r8hqd Startup probe failed: Get "http://10.134.0.105:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-06a95f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-06a95f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal Pulled pod/e2e-shared-b-w6-db7419-openshift-default-9dd55d5bc-q2xvw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Created pod/e2e-shared-b-w6-db7419-openshift-default-9dd55d5bc-q2xvw Container created 43m Normal Started pod/e2e-shared-b-w6-db7419-openshift-default-9dd55d5bc-q2xvw Container started 43m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-3bd0f6-openshift-default Scaled up replica set e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77 from 1 to 3 43m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77 Created pod: e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-p9dds 43m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77 Created pod: e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-74t4k 43m Normal AddedInterface pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-74t4k Add eth0 [10.132.0.177/23] from ovn-kubernetes 43m Normal Pulled pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-74t4k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-p9dds Container started 43m Normal Created pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-p9dds Container created 43m Normal Pulled pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-p9dds Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-p9dds Add eth0 [10.133.0.80/23] from ovn-kubernetes 43m Normal Started pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-74t4k Container started 43m Normal Created pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-74t4k Container created 43m Normal Pulled pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-6mjhh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Warning ProbeError pod/e2e-shared-b-w6-db7419-openshift-default-9dd55d5bc-q2xvw Startup probe error: Get "http://10.132.0.164:15021/healthz/ready": dial tcp 10.132.0.164:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-shared-b-w6-db7419-openshift-default-9dd55d5bc-q2xvw Startup probe failed: Get "http://10.132.0.164:15021/healthz/ready": dial tcp 10.132.0.164:15021: connect: connection refused 42m Normal Created pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-6mjhh Container created 42m Normal Started pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-6mjhh Container started 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-aaa3c6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-aaa3c6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning ProbeError pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-p9dds Startup probe error: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused... 42m Normal AddedLabel gateway/e2e-api-a-cae55b Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-cae55b 42m Warning ProbeError pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-74t4k Startup probe error: Get "http://10.132.0.177:15021/healthz/ready": dial tcp 10.132.0.177:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-74t4k Startup probe failed: Get "http://10.132.0.177:15021/healthz/ready": dial tcp 10.132.0.177:15021: connect: connection refused 42m Warning Unhealthy pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-p9dds Startup probe failed: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused 42m Warning ProbeError pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-6mjhh Startup probe error: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused... 42m Normal SuccessfulCreate replicaset/e2e-api-a-cae55b-openshift-default-f795f5d7f Created pod: e2e-api-a-cae55b-openshift-default-f795f5d7f-x82q6 42m Warning Unhealthy pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-6mjhh Startup probe failed: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused 42m Normal ScalingReplicaSet deployment/e2e-api-a-cae55b-openshift-default Scaled up replica set e2e-api-a-cae55b-openshift-default-f795f5d7f from 0 to 1 42m Normal AddedInterface pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-x82q6 Add eth0 [10.134.0.111/23] from ovn-kubernetes 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-db7419-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-db7419-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Normal SuccessfulCreate replicaset/e2e-api-a-616da7-openshift-default-85b49d64c5 Created pod: e2e-api-a-616da7-openshift-default-85b49d64c5-6ctxr 42m Normal AddedLabel gateway/e2e-api-a-616da7 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-616da7 42m Normal ScalingReplicaSet deployment/e2e-api-a-616da7-openshift-default Scaled up replica set e2e-api-a-616da7-openshift-default-85b49d64c5 from 0 to 1 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-cae55b-84d5658d5f Error creating: pods "payload-pre-processing-e2e-api-a-cae55b-84d5658d5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-cae55b: serviceaccount "payload-processing-e2e-api-a-cae55b" not found 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-cae55b-84d5658d5f Created pod: payload-pre-processing-e2e-api-a-cae55b-84d5658d5f-gzwp2 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-cae55b Scaled up replica set payload-pre-processing-e2e-api-a-cae55b-84d5658d5f from 0 to 1 42m Warning FailedCreate replicaset/payload-processing-e2e-api-a-cae55b-9587db5fb Error creating: pods "payload-processing-e2e-api-a-cae55b-9587db5fb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-cae55b: serviceaccount "payload-processing-e2e-api-a-cae55b" not found 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-cae55b-9587db5fb Created pod: payload-processing-e2e-api-a-cae55b-9587db5fb-8nx59 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-cae55b Scaled up replica set payload-processing-e2e-api-a-cae55b-9587db5fb from 0 to 1 42m Normal Created pod/payload-processing-e2e-api-a-cae55b-9587db5fb-8nx59 Container created 42m Normal AddedInterface pod/e2e-api-a-616da7-openshift-default-85b49d64c5-6ctxr Add eth0 [10.134.0.113/23] from ovn-kubernetes 42m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-cae55b-84d5658d5f-gzwp2 Add eth0 [10.134.0.114/23] from ovn-kubernetes 42m Normal Pulled pod/payload-pre-processing-e2e-api-a-cae55b-84d5658d5f-gzwp2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Started pod/payload-processing-e2e-api-a-cae55b-9587db5fb-8nx59 Container started 42m Normal Pulled pod/payload-processing-e2e-api-a-cae55b-9587db5fb-8nx59 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-processing-e2e-api-a-cae55b-9587db5fb-8nx59 Add eth0 [10.134.0.115/23] from ovn-kubernetes 42m Normal Started pod/payload-pre-processing-e2e-api-a-cae55b-84d5658d5f-gzwp2 Container started 42m Normal Created pod/payload-pre-processing-e2e-api-a-cae55b-84d5658d5f-gzwp2 Container created 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-616da7 Scaled up replica set payload-processing-e2e-api-a-616da7-8485684f4d from 0 to 1 42m Normal SuccessfulCreate replicaset/e2e-api-a-ee886f-openshift-default-7f66f8dfb5 Created pod: e2e-api-a-ee886f-openshift-default-7f66f8dfb5-m9m5s 42m Normal ScalingReplicaSet deployment/e2e-api-a-ee886f-openshift-default Scaled up replica set e2e-api-a-ee886f-openshift-default-7f66f8dfb5 from 0 to 1 42m Normal AddedLabel gateway/e2e-api-a-ee886f Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-ee886f 42m Warning FailedCreate replicaset/payload-processing-e2e-api-a-616da7-8485684f4d Error creating: pods "payload-processing-e2e-api-a-616da7-8485684f4d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-616da7: serviceaccount "payload-processing-e2e-api-a-616da7" not found 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-616da7-7747fb4947 Error creating: pods "payload-pre-processing-e2e-api-a-616da7-7747fb4947-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-616da7: serviceaccount "payload-processing-e2e-api-a-616da7" not found 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-616da7-8485684f4d Created pod: payload-processing-e2e-api-a-616da7-8485684f4d-b7ww6 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-616da7-7747fb4947 Created pod: payload-pre-processing-e2e-api-a-616da7-7747fb4947-5lbcj 42m Normal AddedInterface pod/e2e-api-a-ee886f-openshift-default-7f66f8dfb5-m9m5s Add eth0 [10.132.0.178/23] from ovn-kubernetes 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-616da7 Scaled up replica set payload-pre-processing-e2e-api-a-616da7-7747fb4947 from 0 to 1 42m Normal AddedInterface pod/payload-processing-e2e-api-a-616da7-8485684f4d-b7ww6 Add eth0 [10.134.0.117/23] from ovn-kubernetes 42m Normal Pulled pod/payload-processing-e2e-api-a-616da7-8485684f4d-b7ww6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-616da7-7747fb4947-5lbcj Add eth0 [10.134.0.116/23] from ovn-kubernetes 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-cae55b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Normal Started pod/payload-pre-processing-e2e-api-a-616da7-7747fb4947-5lbcj Container started 42m Normal Created pod/payload-pre-processing-e2e-api-a-616da7-7747fb4947-5lbcj Container created 42m Normal Pulled pod/payload-pre-processing-e2e-api-a-616da7-7747fb4947-5lbcj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-cae55b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Normal Created pod/payload-processing-e2e-api-a-616da7-8485684f4d-b7ww6 Container created 42m Normal Started pod/payload-processing-e2e-api-a-616da7-8485684f4d-b7ww6 Container started 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-db7419-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-ee886f Scaled up replica set payload-processing-e2e-api-a-ee886f-588fb4f8b from 0 to 1 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-db7419-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-ee886f Scaled up replica set payload-pre-processing-e2e-api-a-ee886f-5979bf8887 from 0 to 1 42m Warning FailedCreate replicaset/payload-processing-e2e-api-a-ee886f-588fb4f8b Error creating: pods "payload-processing-e2e-api-a-ee886f-588fb4f8b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-ee886f: serviceaccount "payload-processing-e2e-api-a-ee886f" not found 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-ee886f-588fb4f8b Created pod: payload-processing-e2e-api-a-ee886f-588fb4f8b-rqldn 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-ee886f-5979bf8887 Error creating: pods "payload-pre-processing-e2e-api-a-ee886f-5979bf8887-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-ee886f: serviceaccount "payload-processing-e2e-api-a-ee886f" not found 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-ee886f-5979bf8887 Created pod: payload-pre-processing-e2e-api-a-ee886f-5979bf8887-hr469 42m Normal AddedInterface pod/payload-processing-e2e-api-a-ee886f-588fb4f8b-rqldn Add eth0 [10.134.0.118/23] from ovn-kubernetes 42m Normal Pulled pod/payload-processing-e2e-api-a-ee886f-588fb4f8b-rqldn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Started pod/payload-pre-processing-e2e-api-a-ee886f-5979bf8887-hr469 Container started 42m Normal Started pod/payload-processing-e2e-api-a-ee886f-588fb4f8b-rqldn Container started 42m Normal Created pod/payload-pre-processing-e2e-api-a-ee886f-5979bf8887-hr469 Container created 42m Normal Created pod/payload-processing-e2e-api-a-ee886f-588fb4f8b-rqldn Container created 42m Normal Pulled pod/payload-pre-processing-e2e-api-a-ee886f-5979bf8887-hr469 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-ee886f-5979bf8887-hr469 Add eth0 [10.132.0.180/23] from ovn-kubernetes 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-3bd0f6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-3bd0f6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Normal Created pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-47v7d Container created 42m Normal Pulled pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-47v7d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Started pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-47v7d Container started 42m Warning Unhealthy pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-47v7d Startup probe failed: Get "http://10.132.0.172:15021/healthz/ready": dial tcp 10.132.0.172:15021: connect: connection refused 42m Warning ProbeError pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-47v7d Startup probe error: Get "http://10.132.0.172:15021/healthz/ready": dial tcp 10.132.0.172:15021: connect: connection refused... 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-1fa4b8b6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-1fa4b8b6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5db77fbd9f Created pod: maas-default-gateway-openshift-default-5db77fbd9f-xvdlm 42m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5db77fbd9f Created pod: maas-default-gateway-openshift-default-5db77fbd9f-hrms9 42m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5db77fbd9f Created pod: maas-default-gateway-openshift-default-5db77fbd9f-r5b9v 42m Normal Started pod/maas-default-gateway-openshift-default-5db77fbd9f-xvdlm Container started 42m Normal Started pod/maas-default-gateway-openshift-default-5db77fbd9f-hrms9 Container started 42m Normal Created pod/maas-default-gateway-openshift-default-5db77fbd9f-hrms9 Container created 42m Normal Pulled pod/maas-default-gateway-openshift-default-5db77fbd9f-xvdlm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/maas-default-gateway-openshift-default-5db77fbd9f-xvdlm Container created 42m Normal AddedInterface pod/maas-default-gateway-openshift-default-5db77fbd9f-hrms9 Add eth0 [10.133.0.83/23] from ovn-kubernetes 42m Normal AddedInterface pod/maas-default-gateway-openshift-default-5db77fbd9f-xvdlm Add eth0 [10.132.0.181/23] from ovn-kubernetes 42m Normal AddedInterface pod/maas-default-gateway-openshift-default-5db77fbd9f-r5b9v Add eth0 [10.134.0.119/23] from ovn-kubernetes 42m Normal Pulled pod/maas-default-gateway-openshift-default-5db77fbd9f-hrms9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Pulled pod/maas-default-gateway-openshift-default-5db77fbd9f-r5b9v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/maas-default-gateway-openshift-default-5db77fbd9f-r5b9v Container created 42m Normal Started pod/maas-default-gateway-openshift-default-5db77fbd9f-r5b9v Container started 42m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-hrms9 Startup probe failed: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused 42m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-hrms9 Startup probe error: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused... 42m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-xvdlm Startup probe failed: Get "http://10.132.0.181:15021/healthz/ready": dial tcp 10.132.0.181:15021: connect: connection refused 42m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-r5b9v Startup probe error: Get "http://10.134.0.119:15021/healthz/ready": dial tcp 10.134.0.119:15021: connect: connection refused... 42m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-xvdlm Startup probe error: Get "http://10.132.0.181:15021/healthz/ready": dial tcp 10.132.0.181:15021: connect: connection refused... 42m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-r5b9v Startup probe failed: Get "http://10.134.0.119:15021/healthz/ready": dial tcp 10.134.0.119:15021: connect: connection refused 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-1fa4b8b6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-1fa4b8b6-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal AddedLabel gateway/e2e-api-b-aeda8c Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-aeda8c 41m Normal ScalingReplicaSet deployment/e2e-api-b-aeda8c-openshift-default Scaled up replica set e2e-api-b-aeda8c-openshift-default-57d7447bc from 0 to 1 41m Normal SuccessfulCreate replicaset/e2e-api-b-aeda8c-openshift-default-57d7447bc Created pod: e2e-api-b-aeda8c-openshift-default-57d7447bc-6nmrv 41m Normal AddedInterface pod/e2e-api-b-aeda8c-openshift-default-57d7447bc-6nmrv Add eth0 [10.132.0.183/23] from ovn-kubernetes 41m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-m55mt Readiness probe failed: Get "http://10.134.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-m55mt Readiness probe error: Get "http://10.134.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Normal Killing pod/payload-processing-e2e-mt-1131a29d-6d7ccc5c4d-mdml5 Stopping container payload-processing 41m Normal Killing pod/payload-pre-processing-e2e-mt-1131a29d-79fbb5fd4c-rqqk2 Stopping container payload-pre-processing 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-aeda8c Scaled up replica set payload-processing-e2e-api-b-aeda8c-75cdbfc96 from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-aeda8c-5cc9444875 Created pod: payload-pre-processing-e2e-api-b-aeda8c-5cc9444875-zqf4n 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-aeda8c-75cdbfc96 Created pod: payload-processing-e2e-api-b-aeda8c-75cdbfc96-xx8st 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-aeda8c Scaled up replica set payload-pre-processing-e2e-api-b-aeda8c-5cc9444875 from 0 to 1 41m Warning FailedCreate replicaset/payload-processing-e2e-api-b-aeda8c-75cdbfc96 Error creating: pods "payload-processing-e2e-api-b-aeda8c-75cdbfc96-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-aeda8c: serviceaccount "payload-processing-e2e-api-b-aeda8c" not found 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-aeda8c-5cc9444875 Error creating: pods "payload-pre-processing-e2e-api-b-aeda8c-5cc9444875-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-aeda8c: serviceaccount "payload-processing-e2e-api-b-aeda8c" not found 41m Normal Created pod/payload-pre-processing-e2e-api-b-aeda8c-5cc9444875-zqf4n Container created 41m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-aeda8c-5cc9444875-zqf4n Add eth0 [10.132.0.184/23] from ovn-kubernetes 41m Normal AddedInterface pod/payload-processing-e2e-api-b-aeda8c-75cdbfc96-xx8st Add eth0 [10.134.0.121/23] from ovn-kubernetes 41m Normal Started pod/payload-pre-processing-e2e-api-b-aeda8c-5cc9444875-zqf4n Container started 41m Normal Pulled pod/payload-pre-processing-e2e-api-b-aeda8c-5cc9444875-zqf4n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Warning ProbeError pod/e2e-api-a-616da7-openshift-default-85b49d64c5-6ctxr Readiness probe error: Get "http://10.134.0.113:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Warning Unhealthy pod/e2e-api-a-616da7-openshift-default-85b49d64c5-6ctxr Readiness probe failed: Get "http://10.134.0.113:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Normal Created pod/payload-processing-e2e-api-b-aeda8c-75cdbfc96-xx8st Container created 41m Normal Started pod/payload-processing-e2e-api-b-aeda8c-75cdbfc96-xx8st Container started 41m Normal Pulled pod/payload-processing-e2e-api-b-aeda8c-75cdbfc96-xx8st Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal SuccessfulCreate replicaset/e2e-api-b-39322f-openshift-default-78f75cb6d5 Created pod: e2e-api-b-39322f-openshift-default-78f75cb6d5-np7f7 41m Normal ScalingReplicaSet deployment/e2e-api-b-39322f-openshift-default Scaled up replica set e2e-api-b-39322f-openshift-default-78f75cb6d5 from 0 to 1 41m Normal AddedLabel gateway/e2e-api-b-39322f Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-39322f 41m Normal ScalingReplicaSet deployment/e2e-api-a-cae55b-openshift-default Scaled up replica set e2e-api-a-cae55b-openshift-default-f795f5d7f from 1 to 3 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-a-cae55b-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 41m Normal SuccessfulCreate replicaset/e2e-api-a-cae55b-openshift-default-f795f5d7f Created pod: e2e-api-a-cae55b-openshift-default-f795f5d7f-6x2tx 41m Normal AddedInterface pod/e2e-api-b-39322f-openshift-default-78f75cb6d5-np7f7 Add eth0 [10.132.0.186/23] from ovn-kubernetes 41m Normal SuccessfulCreate replicaset/e2e-api-a-cae55b-openshift-default-f795f5d7f Created pod: e2e-api-a-cae55b-openshift-default-f795f5d7f-lv8h9 41m Normal Pulled pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-lv8h9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-lv8h9 Container created 41m Normal AddedInterface pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-6x2tx Add eth0 [10.132.0.187/23] from ovn-kubernetes 41m Normal Started pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-6x2tx Container started 41m Normal AddedInterface pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-lv8h9 Add eth0 [10.133.0.85/23] from ovn-kubernetes 41m Normal Pulled pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-6x2tx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-6x2tx Container created 41m Normal Started pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-lv8h9 Container started 41m Warning Unhealthy pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-x82q6 Readiness probe failed: Get "http://10.134.0.111:15021/healthz/ready": read tcp 10.134.0.2:50652->10.134.0.111:15021: read: connection reset by peer 41m Normal ScalingReplicaSet deployment/e2e-api-b-3d1777-openshift-default Scaled up replica set e2e-api-b-3d1777-openshift-default-56f984b757 from 0 to 1 41m Warning ProbeError pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-x82q6 Readiness probe error: Get "http://10.134.0.111:15021/healthz/ready": read tcp 10.134.0.2:50652->10.134.0.111:15021: read: connection reset by peer... 41m Normal SuccessfulCreate replicaset/e2e-api-b-3d1777-openshift-default-56f984b757 Created pod: e2e-api-b-3d1777-openshift-default-56f984b757-knrnv 41m Normal AddedLabel gateway/e2e-api-b-3d1777 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-3d1777 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-39322f Scaled up replica set payload-processing-e2e-api-b-39322f-5c74d47cd from 0 to 1 41m Normal Started pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-x82q6 Container started 41m Normal Started pod/e2e-api-a-616da7-openshift-default-85b49d64c5-6ctxr Container started 41m Normal Pulled pod/e2e-api-a-616da7-openshift-default-85b49d64c5-6ctxr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-api-a-616da7-openshift-default-85b49d64c5-6ctxr Container created 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-39322f-6bd999dc7f Error creating: pods "payload-pre-processing-e2e-api-b-39322f-6bd999dc7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-39322f: serviceaccount "payload-processing-e2e-api-b-39322f" not found 41m Normal Pulled pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-x82q6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-x82q6 Container created 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-39322f-6bd999dc7f Created pod: payload-pre-processing-e2e-api-b-39322f-6bd999dc7f-dcj8q 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-39322f Scaled up replica set payload-pre-processing-e2e-api-b-39322f-6bd999dc7f from 0 to 1 41m Warning FailedCreate replicaset/payload-processing-e2e-api-b-39322f-5c74d47cd Error creating: pods "payload-processing-e2e-api-b-39322f-5c74d47cd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-39322f: serviceaccount "payload-processing-e2e-api-b-39322f" not found 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-39322f-5c74d47cd Created pod: payload-processing-e2e-api-b-39322f-5c74d47cd-88276 41m Normal Pulled pod/payload-pre-processing-e2e-api-b-39322f-6bd999dc7f-dcj8q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Pulled pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-d4sqk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal SuccessfulCreate replicaset/e2e-mt-1131a29d-openshift-default-54ccb47c65 Created pod: e2e-mt-1131a29d-openshift-default-54ccb47c65-mx7rd 41m Normal SuccessfulCreate replicaset/e2e-mt-1131a29d-openshift-default-54ccb47c65 Created pod: e2e-mt-1131a29d-openshift-default-54ccb47c65-d4sqk 41m Normal SuccessfulCreate replicaset/e2e-mt-1131a29d-openshift-default-54ccb47c65 Created pod: e2e-mt-1131a29d-openshift-default-54ccb47c65-jlnzb 41m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-39322f-6bd999dc7f-dcj8q Add eth0 [10.134.0.123/23] from ovn-kubernetes 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-1131a29d-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 41m Normal AddedInterface pod/e2e-api-b-3d1777-openshift-default-56f984b757-knrnv Add eth0 [10.134.0.122/23] from ovn-kubernetes 41m Normal ScalingReplicaSet deployment/e2e-mt-1131a29d-openshift-default Scaled up replica set e2e-mt-1131a29d-openshift-default-54ccb47c65 from 1 to 4 41m Normal AddedInterface pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-d4sqk Add eth0 [10.133.0.86/23] from ovn-kubernetes 41m Normal Created pod/payload-pre-processing-e2e-api-b-39322f-6bd999dc7f-dcj8q Container created 41m Normal Created pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-mx7rd Container created 41m Normal Created pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-jlnzb Container created 41m Normal AddedInterface pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-mx7rd Add eth0 [10.132.0.188/23] from ovn-kubernetes 41m Normal AddedInterface pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-jlnzb Add eth0 [10.132.0.189/23] from ovn-kubernetes 41m Warning ProbeError pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-6x2tx Startup probe error: Get "http://10.132.0.187:15021/healthz/ready": dial tcp 10.132.0.187:15021: connect: connection refused... 41m Normal Pulled pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-jlnzb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Pulled pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-mx7rd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Warning Unhealthy pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-6x2tx Startup probe failed: Get "http://10.132.0.187:15021/healthz/ready": dial tcp 10.132.0.187:15021: connect: connection refused 41m Normal Started pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-jlnzb Container started 41m Normal Killing pod/payload-pre-processing-e2e-mt-1fa4b8b6-675f9bbd4c-xlqfw Stopping container payload-pre-processing 41m Normal Started pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-d4sqk Container started 41m Normal Created pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-d4sqk Container created 41m Warning ProbeError pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-lv8h9 Startup probe error: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused... 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-616da7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal Killing pod/payload-processing-e2e-mt-1fa4b8b6-68f88d6f68-c9869 Stopping container payload-processing 41m Warning Unhealthy pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-lv8h9 Startup probe failed: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused 41m Normal Started pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-mx7rd Container started 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-616da7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal AddedInterface pod/payload-processing-e2e-api-b-39322f-5c74d47cd-88276 Add eth0 [10.134.0.124/23] from ovn-kubernetes 41m Normal Pulled pod/payload-processing-e2e-api-b-39322f-5c74d47cd-88276 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Started pod/payload-pre-processing-e2e-api-b-39322f-6bd999dc7f-dcj8q Container started 41m Normal Started pod/payload-processing-e2e-api-b-39322f-5c74d47cd-88276 Container started 41m Normal Created pod/payload-processing-e2e-api-b-39322f-5c74d47cd-88276 Container created 41m Warning Unhealthy pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-fmfnq Readiness probe failed: Get "http://10.134.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Warning ProbeError pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-fmfnq Readiness probe error: Get "http://10.134.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Warning FailedCreate replicaset/payload-processing-e2e-api-b-3d1777-6dfd6d6694 Error creating: pods "payload-processing-e2e-api-b-3d1777-6dfd6d6694-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-3d1777: serviceaccount "payload-processing-e2e-api-b-3d1777" not found 41m Warning Unhealthy pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-d4sqk Startup probe failed: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-3d1777-5d6bcd9959 Error creating: pods "payload-pre-processing-e2e-api-b-3d1777-5d6bcd9959-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-3d1777: serviceaccount "payload-processing-e2e-api-b-3d1777" not found 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-3d1777-5d6bcd9959 Created pod: payload-pre-processing-e2e-api-b-3d1777-5d6bcd9959-52r9w 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-3d1777 Scaled up replica set payload-pre-processing-e2e-api-b-3d1777-5d6bcd9959 from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-3d1777-6dfd6d6694 Created pod: payload-processing-e2e-api-b-3d1777-6dfd6d6694-vmg4d 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-3d1777 Scaled up replica set payload-processing-e2e-api-b-3d1777-6dfd6d6694 from 0 to 1 41m Warning ProbeError pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-d4sqk Startup probe error: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused... 41m Warning ProbeError pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-jlnzb Startup probe error: Get "http://10.132.0.189:15021/healthz/ready": dial tcp 10.132.0.189:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-jlnzb Startup probe failed: Get "http://10.132.0.189:15021/healthz/ready": dial tcp 10.132.0.189:15021: connect: connection refused 41m Warning Unhealthy pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-x82q6 Startup probe failed: Get "http://10.134.0.111:15021/healthz/ready": dial tcp 10.134.0.111:15021: connect: connection refused 41m Normal AddedInterface pod/payload-processing-e2e-api-b-3d1777-6dfd6d6694-vmg4d Add eth0 [10.134.0.126/23] from ovn-kubernetes 41m Warning ProbeError pod/e2e-api-a-616da7-openshift-default-85b49d64c5-6ctxr Startup probe error: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused... 41m Warning ProbeError pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-x82q6 Startup probe error: Get "http://10.134.0.111:15021/healthz/ready": dial tcp 10.134.0.111:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-api-a-616da7-openshift-default-85b49d64c5-6ctxr Startup probe failed: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused 41m Normal Pulled pod/payload-processing-e2e-api-b-3d1777-6dfd6d6694-vmg4d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-processing-e2e-api-b-3d1777-6dfd6d6694-vmg4d Container created 41m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-3d1777-5d6bcd9959-52r9w Add eth0 [10.134.0.125/23] from ovn-kubernetes 41m Warning ProbeError pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-mx7rd Startup probe error: Get "http://10.132.0.188:15021/healthz/ready": dial tcp 10.132.0.188:15021: connect: connection refused... 41m Normal Pulled pod/payload-pre-processing-e2e-api-b-3d1777-5d6bcd9959-52r9w Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Warning Unhealthy pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-mx7rd Startup probe failed: Get "http://10.132.0.188:15021/healthz/ready": dial tcp 10.132.0.188:15021: connect: connection refused 41m Normal Created pod/payload-pre-processing-e2e-api-b-3d1777-5d6bcd9959-52r9w Container created 41m Normal Started pod/payload-pre-processing-e2e-api-b-3d1777-5d6bcd9959-52r9w Container started 41m Normal Started pod/payload-processing-e2e-api-b-3d1777-6dfd6d6694-vmg4d Container started 41m Normal Started pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-fmfnq Container started 41m Normal Pulled pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-fmfnq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-fmfnq Container created 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-2ba10a-openshift-default New size: 6; reason: All metrics below target 41m Normal Killing pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9rm2s Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 Deleted pod: e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9rm2s 41m Normal Killing pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-dd4x7 Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9hfvc Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-jjlwt Stopping container istio-proxy 41m Normal ScalingReplicaSet deployment/e2e-worker-w4-08ebee-openshift-default Scaled down replica set e2e-worker-w4-08ebee-openshift-default-5f5bf46967 from 10 to 6 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-08ebee-openshift-default New size: 6; reason: All metrics below target 41m Normal Killing pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-vjfcn Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-cb9wn Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8gswt Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-q5z2q Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w5-22d634-openshift-default-89dbbf78f Deleted pod: e2e-worker-w5-22d634-openshift-default-89dbbf78f-q5z2q 41m Warning ProbeError pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-dd4x7 Readiness probe error: HTTP probe failed with statuscode: 503... 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-22d634-openshift-default New size: 6; reason: All metrics below target 41m Normal ScalingReplicaSet deployment/e2e-worker-w5-22d634-openshift-default Scaled down replica set e2e-worker-w5-22d634-openshift-default-89dbbf78f from 7 to 6 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-3d1777-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal SuccessfulDelete replicaset/e2e-worker-w4-08ebee-openshift-default-5f5bf46967 Deleted pod: e2e-worker-w4-08ebee-openshift-default-5f5bf46967-vjfcn 41m Normal SuccessfulDelete replicaset/e2e-worker-w4-08ebee-openshift-default-5f5bf46967 Deleted pod: e2e-worker-w4-08ebee-openshift-default-5f5bf46967-cb9wn 41m Normal SuccessfulDelete replicaset/e2e-worker-w4-08ebee-openshift-default-5f5bf46967 Deleted pod: e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8gswt 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-3d1777-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal SuccessfulDelete replicaset/e2e-worker-w4-08ebee-openshift-default-5f5bf46967 Deleted pod: e2e-worker-w4-08ebee-openshift-default-5f5bf46967-dd4x7 41m Normal Killing pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-vt95w Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 Deleted pod: e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-jjlwt 41m Normal ScalingReplicaSet deployment/e2e-worker-w2-2ba10a-openshift-default Scaled down replica set e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 from 10 to 6 41m Normal SuccessfulDelete replicaset/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 Deleted pod: e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-vt95w 41m Normal SuccessfulDelete replicaset/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 Deleted pod: e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-9hfvc 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-1131a29d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-1131a29d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning Unhealthy pod/payload-pre-processing-e2e-mt-1fa4b8b6-675f9bbd4c-xlqfw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 41m Warning Unhealthy pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-fmfnq Startup probe failed: Get "http://10.134.0.98:15021/healthz/ready": dial tcp 10.134.0.98:15021: connect: connection refused 41m Warning ProbeError pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-fmfnq Startup probe error: Get "http://10.134.0.98:15021/healthz/ready": dial tcp 10.134.0.98:15021: connect: connection refused... 41m Normal Killing pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-fmfnq Stopping container istio-proxy 41m Normal Killing pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-mx7rd Stopping container istio-proxy 41m Normal Killing pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-d4sqk Stopping container istio-proxy 41m Normal Killing pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-jlnzb Stopping container istio-proxy 41m Normal AddedLabel gateway/e2e-inf-a-4d3168 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-4d3168 41m Warning ProbeError pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-d4sqk Readiness probe error: HTTP probe failed with statuscode: 503... 41m Normal SuccessfulCreate replicaset/e2e-inf-a-4d3168-openshift-default-96c48bc6c Created pod: e2e-inf-a-4d3168-openshift-default-96c48bc6c-l4blm 41m Normal ScalingReplicaSet deployment/e2e-inf-a-4d3168-openshift-default Scaled up replica set e2e-inf-a-4d3168-openshift-default-96c48bc6c from 0 to 1 41m Warning Unhealthy pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-d4sqk Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning ProbeError pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-jlnzb Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-jlnzb Readiness probe failed: HTTP probe failed with statuscode: 503 41m Normal AddedInterface pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-l4blm Add eth0 [10.134.0.127/23] from ovn-kubernetes 41m Warning Unhealthy pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-mx7rd Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-cae55b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-cae55b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning ProbeError pod/e2e-mt-1131a29d-openshift-default-54ccb47c65-mx7rd Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/payload-processing-e2e-mt-1fa4b8b6-68f88d6f68-c9869 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-4d3168-5cf56f8fd7 Created pod: payload-pre-processing-e2e-inf-a-4d3168-5cf56f8fd7-dbn2k 41m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-4d3168-d94579868 Error creating: pods "payload-processing-e2e-inf-a-4d3168-d94579868-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-4d3168: serviceaccount "payload-processing-e2e-inf-a-4d3168" not found 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-4d3168-d94579868 Created pod: payload-processing-e2e-inf-a-4d3168-d94579868-m6l26 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-4d3168 Scaled up replica set payload-processing-e2e-inf-a-4d3168-d94579868 from 0 to 1 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-4d3168 Scaled up replica set payload-pre-processing-e2e-inf-a-4d3168-5cf56f8fd7 from 0 to 1 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-4d3168-5cf56f8fd7 Error creating: pods "payload-pre-processing-e2e-inf-a-4d3168-5cf56f8fd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-4d3168: serviceaccount "payload-processing-e2e-inf-a-4d3168" not found 41m Normal Started pod/payload-processing-e2e-inf-a-4d3168-d94579868-m6l26 Container started 41m Normal Started pod/payload-pre-processing-e2e-inf-a-4d3168-5cf56f8fd7-dbn2k Container started 41m Normal Created pod/payload-pre-processing-e2e-inf-a-4d3168-5cf56f8fd7-dbn2k Container created 41m Normal Created pod/payload-processing-e2e-inf-a-4d3168-d94579868-m6l26 Container created 41m Normal Pulled pod/payload-processing-e2e-inf-a-4d3168-d94579868-m6l26 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/payload-processing-e2e-inf-a-4d3168-d94579868-m6l26 Add eth0 [10.134.0.130/23] from ovn-kubernetes 41m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-4d3168-5cf56f8fd7-dbn2k Add eth0 [10.134.0.129/23] from ovn-kubernetes 41m Normal Pulled pod/payload-pre-processing-e2e-inf-a-4d3168-5cf56f8fd7-dbn2k Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal ScalingReplicaSet deployment/e2e-inf-a-c1750b-openshift-default Scaled up replica set e2e-inf-a-c1750b-openshift-default-85d9759655 from 0 to 1 40m Normal AddedLabel gateway/e2e-inf-a-c1750b Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-c1750b 40m Normal SuccessfulCreate replicaset/e2e-inf-a-c1750b-openshift-default-85d9759655 Created pod: e2e-inf-a-c1750b-openshift-default-85d9759655-956xt 40m Normal AddedInterface pod/e2e-inf-a-c1750b-openshift-default-85d9759655-956xt Add eth0 [10.134.0.131/23] from ovn-kubernetes 40m Normal Created pod/e2e-api-a-ee886f-openshift-default-7f66f8dfb5-m9m5s Container created 40m Normal Pulled pod/e2e-api-a-ee886f-openshift-default-7f66f8dfb5-m9m5s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Started pod/e2e-api-a-ee886f-openshift-default-7f66f8dfb5-m9m5s Container started 40m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-c1750b-85f79d758d Error creating: pods "payload-processing-e2e-inf-a-c1750b-85f79d758d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-c1750b: serviceaccount "payload-processing-e2e-inf-a-c1750b" not found 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-c1750b Scaled up replica set payload-pre-processing-e2e-inf-a-c1750b-65b7cbb459 from 0 to 1 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-c1750b Scaled up replica set payload-processing-e2e-inf-a-c1750b-85f79d758d from 0 to 1 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-c1750b-65b7cbb459 Error creating: pods "payload-pre-processing-e2e-inf-a-c1750b-65b7cbb459-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-c1750b: serviceaccount "payload-processing-e2e-inf-a-c1750b" not found 40m Normal SuccessfulDelete replicaset/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc Deleted pod: e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-h947w 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-c1750b-85f79d758d Created pod: payload-processing-e2e-inf-a-c1750b-85f79d758d-n8sxl 40m Normal ScalingReplicaSet deployment/e2e-worker-w3-43b37d-openshift-default Scaled down replica set e2e-worker-w3-43b37d-openshift-default-5dcf864fcc from 10 to 6 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-43b37d-openshift-default New size: 6; reason: All metrics below target 40m Normal SuccessfulDelete replicaset/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc Deleted pod: e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-s5cq4 40m Normal SuccessfulDelete replicaset/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc Deleted pod: e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-95zbb 40m Normal SuccessfulDelete replicaset/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc Deleted pod: e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-jp7np 40m Normal Killing pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-jp7np Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-95zbb Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-h947w Stopping container istio-proxy 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-c1750b-65b7cbb459 Created pod: payload-pre-processing-e2e-inf-a-c1750b-65b7cbb459-gt6wn 40m Normal Killing pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-s5cq4 Stopping container istio-proxy 40m Normal Pulled pod/payload-pre-processing-e2e-inf-a-c1750b-65b7cbb459-gt6wn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Created pod/payload-pre-processing-e2e-inf-a-c1750b-65b7cbb459-gt6wn Container created 40m Normal Started pod/payload-pre-processing-e2e-inf-a-c1750b-65b7cbb459-gt6wn Container started 40m Normal AddedInterface pod/payload-processing-e2e-inf-a-c1750b-85f79d758d-n8sxl Add eth0 [10.132.0.194/23] from ovn-kubernetes 40m Normal Started pod/payload-processing-e2e-inf-a-c1750b-85f79d758d-n8sxl Container started 40m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-c1750b-65b7cbb459-gt6wn Add eth0 [10.132.0.193/23] from ovn-kubernetes 40m Normal Created pod/payload-processing-e2e-inf-a-c1750b-85f79d758d-n8sxl Container created 40m Normal Pulled pod/payload-processing-e2e-inf-a-c1750b-85f79d758d-n8sxl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Warning Unhealthy pod/e2e-api-a-ee886f-openshift-default-7f66f8dfb5-m9m5s Startup probe failed: Get "http://10.132.0.178:15021/healthz/ready": dial tcp 10.132.0.178:15021: connect: connection refused 40m Warning ProbeError pod/e2e-api-a-ee886f-openshift-default-7f66f8dfb5-m9m5s Startup probe error: Get "http://10.132.0.178:15021/healthz/ready": dial tcp 10.132.0.178:15021: connect: connection refused... 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-616da7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-616da7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedCreate replicaset/payload-processing-e2e-mt-1fa4b8b6-68f88d6f68 Error creating: pods "payload-processing-e2e-mt-1fa4b8b6-68f88d6f68-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-1fa4b8b6: serviceaccount "payload-processing-e2e-mt-1fa4b8b6" not found 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-1fa4b8b6-68f88d6f68 Created pod: payload-processing-e2e-mt-1fa4b8b6-68f88d6f68-2sqfg 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-1fa4b8b6 Scaled up replica set payload-pre-processing-e2e-mt-1fa4b8b6-675f9bbd4c from 0 to 1 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-1fa4b8b6-675f9bbd4c Created pod: payload-pre-processing-e2e-mt-1fa4b8b6-675f9bbd4c-4jhnf 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-1fa4b8b6-675f9bbd4c Error creating: pods "payload-pre-processing-e2e-mt-1fa4b8b6-675f9bbd4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-1fa4b8b6: serviceaccount "payload-processing-e2e-mt-1fa4b8b6" not found 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-1fa4b8b6 Scaled up replica set payload-processing-e2e-mt-1fa4b8b6-68f88d6f68 from 0 to 1 40m Normal Created pod/payload-pre-processing-e2e-mt-1fa4b8b6-675f9bbd4c-4jhnf Container created 40m Normal AddedInterface pod/payload-processing-e2e-mt-1fa4b8b6-68f88d6f68-2sqfg Add eth0 [10.132.0.196/23] from ovn-kubernetes 40m Normal Started pod/payload-processing-e2e-mt-1fa4b8b6-68f88d6f68-2sqfg Container started 40m Normal Created pod/payload-processing-e2e-mt-1fa4b8b6-68f88d6f68-2sqfg Container created 40m Normal Pulled pod/payload-processing-e2e-mt-1fa4b8b6-68f88d6f68-2sqfg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/payload-pre-processing-e2e-mt-1fa4b8b6-675f9bbd4c-4jhnf Add eth0 [10.132.0.195/23] from ovn-kubernetes 40m Normal Started pod/payload-pre-processing-e2e-mt-1fa4b8b6-675f9bbd4c-4jhnf Container started 40m Normal Pulled pod/payload-pre-processing-e2e-mt-1fa4b8b6-675f9bbd4c-4jhnf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-ee886f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-ee886f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal Killing pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-7glgv Stopping container istio-proxy 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2e046a-openshift-default New size: 7; reason: All metrics below target 40m Normal ScalingReplicaSet deployment/e2e-worker-w0-2e046a-openshift-default Scaled down replica set e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 from 10 to 7 40m Normal Killing pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-8656t Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-ntt82 Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 Deleted pod: e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-8656t 40m Normal SuccessfulDelete replicaset/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 Deleted pod: e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-7glgv 40m Normal SuccessfulDelete replicaset/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 Deleted pod: e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-ntt82 40m Warning Unhealthy pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-7glgv Readiness probe failed: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-7glgv Readiness probe error: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused... 40m Normal Killing pod/payload-processing-e2e-api-a-cae55b-9587db5fb-8nx59 Stopping container payload-processing 40m Normal Killing pod/payload-pre-processing-e2e-api-a-cae55b-84d5658d5f-gzwp2 Stopping container payload-pre-processing 40m Warning Unhealthy pod/payload-pre-processing-e2e-api-b-39322f-6bd999dc7f-dcj8q Readiness probe failed: timeout: health rpc did not complete within 1s 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-ee886f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning ProbeError pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-8656t Readiness probe error: Get "http://10.134.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Warning Unhealthy pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-8656t Readiness probe failed: Get "http://10.134.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-ee886f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning ProbeError pod/e2e-api-b-39322f-openshift-default-78f75cb6d5-np7f7 Readiness probe error: Get "http://10.132.0.186:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Warning Unhealthy pod/e2e-api-b-39322f-openshift-default-78f75cb6d5-np7f7 Readiness probe failed: Get "http://10.132.0.186:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Warning Unhealthy pod/e2e-api-b-aeda8c-openshift-default-57d7447bc-6nmrv Readiness probe failed: Get "http://10.132.0.183:15021/healthz/ready": dial tcp 10.132.0.183:15021: connect: connection refused 40m Normal SuccessfulDelete replicaset/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 Deleted pod: e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-zbfz4 40m Warning ProbeError pod/e2e-api-b-aeda8c-openshift-default-57d7447bc-6nmrv Readiness probe error: Get "http://10.132.0.183:15021/healthz/ready": dial tcp 10.132.0.183:15021: connect: connection refused... 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2e046a-openshift-default New size: 6; reason: All metrics below target 40m Normal Killing pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-zbfz4 Stopping container istio-proxy 40m Normal ScalingReplicaSet deployment/e2e-worker-w0-2e046a-openshift-default Scaled down replica set e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 from 7 to 6 40m Normal Pulled pod/e2e-api-b-aeda8c-openshift-default-57d7447bc-6nmrv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Started pod/e2e-api-b-39322f-openshift-default-78f75cb6d5-np7f7 Container started 40m Normal Created pod/e2e-api-b-39322f-openshift-default-78f75cb6d5-np7f7 Container created 40m Normal Pulled pod/e2e-api-b-39322f-openshift-default-78f75cb6d5-np7f7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-api-b-aeda8c-openshift-default-57d7447bc-6nmrv Container created 40m Normal Started pod/e2e-api-b-aeda8c-openshift-default-57d7447bc-6nmrv Container started 40m Normal Pulled pod/e2e-api-b-3d1777-openshift-default-56f984b757-knrnv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-api-b-3d1777-openshift-default-56f984b757-knrnv Container created 40m Normal Started pod/e2e-api-b-3d1777-openshift-default-56f984b757-knrnv Container started 40m Normal Killing pod/payload-pre-processing-e2e-api-a-616da7-7747fb4947-5lbcj Stopping container payload-pre-processing 40m Normal Killing pod/payload-processing-e2e-api-a-616da7-8485684f4d-b7ww6 Stopping container payload-processing 40m Warning Unhealthy pod/e2e-api-b-39322f-openshift-default-78f75cb6d5-np7f7 Startup probe failed: Get "http://10.132.0.186:15021/healthz/ready": dial tcp 10.132.0.186:15021: connect: connection refused 40m Warning ProbeError pod/e2e-api-b-39322f-openshift-default-78f75cb6d5-np7f7 Startup probe error: Get "http://10.132.0.186:15021/healthz/ready": dial tcp 10.132.0.186:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-api-b-aeda8c-openshift-default-57d7447bc-6nmrv Startup probe failed: Get "http://10.132.0.183:15021/healthz/ready": dial tcp 10.132.0.183:15021: connect: connection refused 40m Warning ProbeError pod/e2e-api-b-aeda8c-openshift-default-57d7447bc-6nmrv Startup probe error: Get "http://10.132.0.183:15021/healthz/ready": dial tcp 10.132.0.183:15021: connect: connection refused... 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-39322f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-39322f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning Unhealthy pod/e2e-api-b-3d1777-openshift-default-56f984b757-knrnv Startup probe failed: Get "http://10.134.0.122:15021/healthz/ready": dial tcp 10.134.0.122:15021: connect: connection refused 40m Warning ProbeError pod/e2e-api-b-3d1777-openshift-default-56f984b757-knrnv Startup probe error: Get "http://10.134.0.122:15021/healthz/ready": dial tcp 10.134.0.122:15021: connect: connection refused... 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-aeda8c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-aeda8c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal Killing pod/payload-pre-processing-e2e-api-a-ee886f-5979bf8887-hr469 Stopping container payload-pre-processing 40m Normal Killing pod/payload-processing-e2e-api-a-ee886f-588fb4f8b-rqldn Stopping container payload-processing 39m Normal Started pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-l4blm Container started 39m Normal Pulled pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-l4blm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-l4blm Container created 39m Normal AddedLabel gateway/e2e-inf-b-c296a3 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-c296a3 39m Normal ScalingReplicaSet deployment/e2e-inf-b-c296a3-openshift-default Scaled up replica set e2e-inf-b-c296a3-openshift-default-79f7f78789 from 0 to 1 39m Normal SuccessfulCreate replicaset/e2e-inf-b-c296a3-openshift-default-79f7f78789 Created pod: e2e-inf-b-c296a3-openshift-default-79f7f78789-fdkc4 39m Normal AddedInterface pod/e2e-inf-b-c296a3-openshift-default-79f7f78789-fdkc4 Add eth0 [10.134.0.135/23] from ovn-kubernetes 39m Normal Killing pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-4h4z5 Stopping container istio-proxy 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-784e7a-openshift-default New size: 8; reason: All metrics below target 39m Normal ScalingReplicaSet deployment/e2e-worker-w6-784e7a-openshift-default Scaled down replica set e2e-worker-w6-784e7a-openshift-default-595898bd5c from 10 to 8 39m Normal Killing pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-nkds4 Stopping container istio-proxy 39m Normal SuccessfulDelete replicaset/e2e-worker-w6-784e7a-openshift-default-595898bd5c Deleted pod: e2e-worker-w6-784e7a-openshift-default-595898bd5c-4h4z5 39m Normal SuccessfulDelete replicaset/e2e-worker-w6-784e7a-openshift-default-595898bd5c Deleted pod: e2e-worker-w6-784e7a-openshift-default-595898bd5c-nkds4 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-3d1777-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-3d1777-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-c296a3-778459576c Created pod: payload-pre-processing-e2e-inf-b-c296a3-778459576c-5mnqg 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-c296a3 Scaled up replica set payload-processing-e2e-inf-b-c296a3-544676fc5d from 0 to 1 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-c296a3 Scaled up replica set payload-pre-processing-e2e-inf-b-c296a3-778459576c from 0 to 1 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-c296a3-544676fc5d Created pod: payload-processing-e2e-inf-b-c296a3-544676fc5d-fvr68 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-c296a3-778459576c Error creating: pods "payload-pre-processing-e2e-inf-b-c296a3-778459576c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-c296a3: serviceaccount "payload-processing-e2e-inf-b-c296a3" not found 39m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-c296a3-544676fc5d Error creating: pods "payload-processing-e2e-inf-b-c296a3-544676fc5d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-c296a3: serviceaccount "payload-processing-e2e-inf-b-c296a3" not found 39m Normal Started pod/payload-processing-e2e-inf-b-c296a3-544676fc5d-fvr68 Container started 39m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-c296a3-778459576c-5mnqg Add eth0 [10.134.0.136/23] from ovn-kubernetes 39m Normal AddedInterface pod/payload-processing-e2e-inf-b-c296a3-544676fc5d-fvr68 Add eth0 [10.132.0.201/23] from ovn-kubernetes 39m Normal Created pod/payload-processing-e2e-inf-b-c296a3-544676fc5d-fvr68 Container created 39m Normal Pulled pod/payload-processing-e2e-inf-b-c296a3-544676fc5d-fvr68 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Warning ProbeError pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-l4blm Startup probe error: Get "http://10.134.0.127:15021/healthz/ready": dial tcp 10.134.0.127:15021: connect: connection refused... 39m Normal Pulled pod/payload-pre-processing-e2e-inf-b-c296a3-778459576c-5mnqg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Warning Unhealthy pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-l4blm Startup probe failed: Get "http://10.134.0.127:15021/healthz/ready": dial tcp 10.134.0.127:15021: connect: connection refused 39m Normal Created pod/payload-pre-processing-e2e-inf-b-c296a3-778459576c-5mnqg Container created 39m Normal Started pod/payload-pre-processing-e2e-inf-b-c296a3-778459576c-5mnqg Container started 39m Normal AddedLabel gateway/e2e-inf-b-8d985a Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-8d985a 39m Normal ScalingReplicaSet deployment/e2e-inf-b-8d985a-openshift-default Scaled up replica set e2e-inf-b-8d985a-openshift-default-856f5f5bf5 from 0 to 1 39m Normal SuccessfulCreate replicaset/e2e-inf-b-8d985a-openshift-default-856f5f5bf5 Created pod: e2e-inf-b-8d985a-openshift-default-856f5f5bf5-w2k2c 39m Normal AddedInterface pod/e2e-inf-b-8d985a-openshift-default-856f5f5bf5-w2k2c Add eth0 [10.132.0.202/23] from ovn-kubernetes 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-4d3168-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-39322f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-39322f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-4d3168-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-aeda8c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-aeda8c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-8d985a Scaled up replica set payload-processing-e2e-inf-b-8d985a-78b6b6596 from 0 to 1 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-8d985a Scaled up replica set payload-pre-processing-e2e-inf-b-8d985a-6b59cc474c from 0 to 1 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-8d985a-78b6b6596 Created pod: payload-processing-e2e-inf-b-8d985a-78b6b6596-5tj4s 39m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-8d985a-78b6b6596 Error creating: pods "payload-processing-e2e-inf-b-8d985a-78b6b6596-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-8d985a: serviceaccount "payload-processing-e2e-inf-b-8d985a" not found 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-8d985a-6b59cc474c Created pod: payload-pre-processing-e2e-inf-b-8d985a-6b59cc474c-5vth4 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-8d985a-6b59cc474c Error creating: pods "payload-pre-processing-e2e-inf-b-8d985a-6b59cc474c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-8d985a: serviceaccount "payload-processing-e2e-inf-b-8d985a" not found 39m Normal Started pod/payload-processing-e2e-inf-b-8d985a-78b6b6596-5tj4s Container started 39m Normal Pulled pod/payload-processing-e2e-inf-b-8d985a-78b6b6596-5tj4s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Started pod/payload-pre-processing-e2e-inf-b-8d985a-6b59cc474c-5vth4 Container started 39m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-8d985a-6b59cc474c-5vth4 Add eth0 [10.132.0.204/23] from ovn-kubernetes 39m Normal AddedInterface pod/payload-processing-e2e-inf-b-8d985a-78b6b6596-5tj4s Add eth0 [10.132.0.205/23] from ovn-kubernetes 39m Normal Pulled pod/payload-pre-processing-e2e-inf-b-8d985a-6b59cc474c-5vth4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Created pod/payload-pre-processing-e2e-inf-b-8d985a-6b59cc474c-5vth4 Container created 39m Normal Created pod/payload-processing-e2e-inf-b-8d985a-78b6b6596-5tj4s Container created 39m Normal SuccessfulCreate replicaset/e2e-mt-1fa4b8b6-openshift-default-6765496769 Created pod: e2e-mt-1fa4b8b6-openshift-default-6765496769-ms7lr 39m Normal SuccessfulCreate replicaset/e2e-mt-1fa4b8b6-openshift-default-6765496769 Created pod: e2e-mt-1fa4b8b6-openshift-default-6765496769-wwxlh 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-1fa4b8b6-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 39m Normal ScalingReplicaSet deployment/e2e-mt-1fa4b8b6-openshift-default Scaled up replica set e2e-mt-1fa4b8b6-openshift-default-6765496769 from 1 to 3 39m Normal Pulled pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-wwxlh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Started pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-ms7lr Container started 39m Normal Created pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-ms7lr Container created 39m Normal Pulled pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-ms7lr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-wwxlh Add eth0 [10.134.0.138/23] from ovn-kubernetes 39m Normal Started pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-wwxlh Container started 39m Normal Created pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-wwxlh Container created 39m Normal AddedInterface pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-ms7lr Add eth0 [10.133.0.87/23] from ovn-kubernetes 39m Normal Pulled pod/e2e-inf-a-c1750b-openshift-default-85d9759655-956xt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-inf-a-c1750b-openshift-default-85d9759655-956xt Container created 39m Normal Started pod/e2e-inf-a-c1750b-openshift-default-85d9759655-956xt Container started 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-c1750b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-c1750b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Normal Killing pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-6x2tx Stopping container istio-proxy 39m Normal Killing pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-x82q6 Stopping container istio-proxy 39m Normal Killing pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-lv8h9 Stopping container istio-proxy 39m Normal Killing pod/payload-pre-processing-e2e-mt-1fa4b8b6-675f9bbd4c-4jhnf Stopping container payload-pre-processing 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-4d3168-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Normal Killing pod/payload-processing-e2e-mt-1fa4b8b6-68f88d6f68-2sqfg Stopping container payload-processing 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-4d3168-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning ProbeError pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-x82q6 Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-api-a-cae55b-openshift-default-f795f5d7f-x82q6 Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning Unhealthy pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-ms7lr Startup probe failed: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused 39m Warning ProbeError pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-ms7lr Startup probe error: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused... 39m Normal Killing pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-r7bbc Stopping container istio-proxy 39m Warning ProbeError pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-wwxlh Startup probe error: Get "http://10.134.0.138:15021/healthz/ready": dial tcp 10.134.0.138:15021: connect: connection refused... 39m Normal Killing pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-pgwf2 Stopping container istio-proxy 39m Normal ScalingReplicaSet deployment/e2e-worker-w6-784e7a-openshift-default Scaled down replica set e2e-worker-w6-784e7a-openshift-default-595898bd5c from 8 to 6 39m Warning Unhealthy pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-wwxlh Startup probe failed: Get "http://10.134.0.138:15021/healthz/ready": dial tcp 10.134.0.138:15021: connect: connection refused 39m Normal SuccessfulDelete replicaset/e2e-worker-w6-784e7a-openshift-default-595898bd5c Deleted pod: e2e-worker-w6-784e7a-openshift-default-595898bd5c-pgwf2 39m Normal SuccessfulDelete replicaset/e2e-worker-w6-784e7a-openshift-default-595898bd5c Deleted pod: e2e-worker-w6-784e7a-openshift-default-595898bd5c-r7bbc 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-784e7a-openshift-default New size: 6; reason: All metrics below target 39m Warning ProbeError pod/e2e-inf-a-c1750b-openshift-default-85d9759655-956xt Startup probe error: Get "http://10.134.0.131:15021/healthz/ready": dial tcp 10.134.0.131:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-inf-a-c1750b-openshift-default-85d9759655-956xt Startup probe failed: Get "http://10.134.0.131:15021/healthz/ready": dial tcp 10.134.0.131:15021: connect: connection refused 39m Normal Killing pod/e2e-api-a-616da7-openshift-default-85b49d64c5-6ctxr Stopping container istio-proxy 39m Normal Killing pod/e2e-api-a-ee886f-openshift-default-7f66f8dfb5-m9m5s Stopping container istio-proxy 39m Warning ProbeError pod/e2e-api-a-ee886f-openshift-default-7f66f8dfb5-m9m5s Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-api-a-ee886f-openshift-default-7f66f8dfb5-m9m5s Readiness probe failed: HTTP probe failed with statuscode: 503 39m Normal Killing pod/payload-pre-processing-e2e-api-b-aeda8c-5cc9444875-zqf4n Stopping container payload-pre-processing 39m Normal Killing pod/payload-processing-e2e-api-b-aeda8c-75cdbfc96-xx8st Stopping container payload-processing 39m Normal Killing pod/payload-processing-e2e-api-b-39322f-5c74d47cd-88276 Stopping container payload-processing 39m Normal Killing pod/payload-pre-processing-e2e-api-b-39322f-6bd999dc7f-dcj8q Stopping container payload-pre-processing 39m Normal Killing pod/payload-processing-e2e-api-b-3d1777-6dfd6d6694-vmg4d Stopping container payload-processing 39m Normal Killing pod/payload-pre-processing-e2e-api-b-3d1777-5d6bcd9959-52r9w Stopping container payload-pre-processing 39m Normal SuccessfulCreate replicaset/e2e-mt-1fa4b8b6-openshift-default-6765496769 Created pod: e2e-mt-1fa4b8b6-openshift-default-6765496769-xgh4q 39m Normal ScalingReplicaSet deployment/e2e-mt-1fa4b8b6-openshift-default Scaled up replica set e2e-mt-1fa4b8b6-openshift-default-6765496769 from 3 to 6 39m Normal SuccessfulCreate replicaset/e2e-mt-1fa4b8b6-openshift-default-6765496769 Created pod: e2e-mt-1fa4b8b6-openshift-default-6765496769-bmbm4 39m Normal SuccessfulCreate replicaset/e2e-mt-1fa4b8b6-openshift-default-6765496769 Created pod: e2e-mt-1fa4b8b6-openshift-default-6765496769-mjcl9 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-1fa4b8b6-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 38m Normal AddedInterface pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-xgh4q Add eth0 [10.133.0.88/23] from ovn-kubernetes 38m Normal Started pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-mjcl9 Container started 38m Normal Created pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-bmbm4 Container created 38m Normal Pulled pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-xgh4q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-xgh4q Container created 38m Normal Pulled pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-mjcl9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-mjcl9 Container created 38m Normal AddedInterface pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-bmbm4 Add eth0 [10.134.0.140/23] from ovn-kubernetes 38m Normal Pulled pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-bmbm4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Started pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-bmbm4 Container started 38m Normal AddedInterface pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-mjcl9 Add eth0 [10.132.0.210/23] from ovn-kubernetes 38m Normal Started pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-xgh4q Container started 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-c1750b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-c1750b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning ProbeError pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-xgh4q Startup probe error: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-xgh4q Startup probe failed: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused 38m Warning ProbeError pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-mjcl9 Startup probe error: Get "http://10.132.0.210:15021/healthz/ready": dial tcp 10.132.0.210:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-mjcl9 Startup probe failed: Get "http://10.132.0.210:15021/healthz/ready": dial tcp 10.132.0.210:15021: connect: connection refused 38m Warning ProbeError pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-bmbm4 Startup probe error: Get "http://10.134.0.140:15021/healthz/ready": dial tcp 10.134.0.140:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-bmbm4 Startup probe failed: Get "http://10.134.0.140:15021/healthz/ready": dial tcp 10.134.0.140:15021: connect: connection refused 38m Normal ScalingReplicaSet deployment/e2e-worker-w2-2ba10a-openshift-default Scaled down replica set e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 from 6 to 5 38m Normal Killing pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-zsn6g Stopping container istio-proxy 38m Normal ScalingReplicaSet deployment/e2e-worker-w1-fc684b-openshift-default Scaled down replica set e2e-worker-w1-fc684b-openshift-default-7fcf678997 from 10 to 8 38m Normal Killing pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-cwggq Stopping container istio-proxy 38m Normal SuccessfulDelete replicaset/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 Deleted pod: e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-zsn6g 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-08ebee-openshift-default New size: 5; reason: All metrics below target 38m Normal Killing pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8n7fg Stopping container istio-proxy 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-2ba10a-openshift-default New size: 5; reason: All metrics below target 38m Normal Killing pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-8wvv6 Stopping container istio-proxy 38m Normal SuccessfulDelete replicaset/e2e-worker-w1-fc684b-openshift-default-7fcf678997 Deleted pod: e2e-worker-w1-fc684b-openshift-default-7fcf678997-8wvv6 38m Normal SuccessfulDelete replicaset/e2e-worker-w1-fc684b-openshift-default-7fcf678997 Deleted pod: e2e-worker-w1-fc684b-openshift-default-7fcf678997-cwggq 38m Normal SuccessfulDelete replicaset/e2e-worker-w4-08ebee-openshift-default-5f5bf46967 Deleted pod: e2e-worker-w4-08ebee-openshift-default-5f5bf46967-8n7fg 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-fc684b-openshift-default New size: 8; reason: All metrics below target 38m Normal ScalingReplicaSet deployment/e2e-worker-w4-08ebee-openshift-default Scaled down replica set e2e-worker-w4-08ebee-openshift-default-5f5bf46967 from 6 to 5 38m Normal ScalingReplicaSet deployment/e2e-mt-1fa4b8b6-openshift-default Scaled up replica set e2e-mt-1fa4b8b6-openshift-default-6765496769 from 6 to 7 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-1fa4b8b6-openshift-default New size: 7; reason: 38m Normal SuccessfulCreate replicaset/e2e-mt-1fa4b8b6-openshift-default-6765496769 Created pod: e2e-mt-1fa4b8b6-openshift-default-6765496769-zqtg8 38m Normal Pulled pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-zqtg8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Started pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-zqtg8 Container started 38m Normal AddedInterface pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-zqtg8 Add eth0 [10.134.0.141/23] from ovn-kubernetes 38m Normal Created pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-zqtg8 Container created 38m Warning Unhealthy pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-zqtg8 Startup probe failed: Get "http://10.134.0.141:15021/healthz/ready": dial tcp 10.134.0.141:15021: connect: connection refused 38m Warning ProbeError pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-zqtg8 Startup probe error: Get "http://10.134.0.141:15021/healthz/ready": dial tcp 10.134.0.141:15021: connect: connection refused... 38m Normal SuccessfulDelete replicaset/e2e-worker-w1-fc684b-openshift-default-7fcf678997 Deleted pod: e2e-worker-w1-fc684b-openshift-default-7fcf678997-dnzl8 38m Normal Killing pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-dnzl8 Stopping container istio-proxy 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-fc684b-openshift-default New size: 6; reason: All metrics below target 38m Normal SuccessfulDelete replicaset/e2e-worker-w1-fc684b-openshift-default-7fcf678997 Deleted pod: e2e-worker-w1-fc684b-openshift-default-7fcf678997-7qslq 38m Normal ScalingReplicaSet deployment/e2e-worker-w1-fc684b-openshift-default Scaled down replica set e2e-worker-w1-fc684b-openshift-default-7fcf678997 from 8 to 6 38m Normal Killing pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-7qslq Stopping container istio-proxy 38m Warning Unhealthy pod/e2e-inf-b-8d985a-openshift-default-856f5f5bf5-w2k2c Readiness probe failed: Get "http://10.132.0.202:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/e2e-inf-b-8d985a-openshift-default-856f5f5bf5-w2k2c Readiness probe error: Get "http://10.132.0.202:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Normal Started pod/e2e-inf-b-8d985a-openshift-default-856f5f5bf5-w2k2c Container started 38m Normal Created pod/e2e-inf-b-8d985a-openshift-default-856f5f5bf5-w2k2c Container created 38m Warning Unhealthy pod/e2e-inf-b-c296a3-openshift-default-79f7f78789-fdkc4 Readiness probe failed: Get "http://10.134.0.135:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/e2e-inf-b-c296a3-openshift-default-79f7f78789-fdkc4 Readiness probe error: Get "http://10.134.0.135:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Normal Pulled pod/e2e-inf-b-8d985a-openshift-default-856f5f5bf5-w2k2c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Started pod/e2e-inf-b-c296a3-openshift-default-79f7f78789-fdkc4 Container started 38m Normal Created pod/e2e-inf-b-c296a3-openshift-default-79f7f78789-fdkc4 Container created 38m Normal Killing pod/e2e-api-b-aeda8c-openshift-default-57d7447bc-6nmrv Stopping container istio-proxy 38m Normal Pulled pod/e2e-inf-b-c296a3-openshift-default-79f7f78789-fdkc4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal SuccessfulCreate replicaset/e2e-inf-a-fee294-openshift-default-644cdf5d45 Created pod: e2e-inf-a-fee294-openshift-default-644cdf5d45-cf2vc 38m Normal AddedLabel gateway/e2e-inf-a-fee294 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-fee294 38m Normal ScalingReplicaSet deployment/e2e-inf-a-fee294-openshift-default Scaled up replica set e2e-inf-a-fee294-openshift-default-644cdf5d45 from 0 to 1 38m Normal AddedInterface pod/e2e-inf-a-fee294-openshift-default-644cdf5d45-cf2vc Add eth0 [10.132.0.211/23] from ovn-kubernetes 38m Warning Unhealthy pod/e2e-inf-b-8d985a-openshift-default-856f5f5bf5-w2k2c Startup probe failed: Get "http://10.132.0.202:15021/healthz/ready": dial tcp 10.132.0.202:15021: connect: connection refused 38m Warning ProbeError pod/e2e-inf-b-8d985a-openshift-default-856f5f5bf5-w2k2c Startup probe error: Get "http://10.132.0.202:15021/healthz/ready": dial tcp 10.132.0.202:15021: connect: connection refused... 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-8d985a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-fee294-79797dc7c4 Created pod: payload-processing-e2e-inf-a-fee294-79797dc7c4-4q6tx 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-fee294 Scaled up replica set payload-processing-e2e-inf-a-fee294-79797dc7c4 from 0 to 1 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-fee294-c499455c5 Error creating: pods "payload-pre-processing-e2e-inf-a-fee294-c499455c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-fee294: serviceaccount "payload-processing-e2e-inf-a-fee294" not found 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-fee294-c499455c5 Created pod: payload-pre-processing-e2e-inf-a-fee294-c499455c5-ds72t 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-fee294 Scaled up replica set payload-pre-processing-e2e-inf-a-fee294-c499455c5 from 0 to 1 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-8d985a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-fee294-79797dc7c4 Error creating: pods "payload-processing-e2e-inf-a-fee294-79797dc7c4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-fee294: serviceaccount "payload-processing-e2e-inf-a-fee294" not found 38m Warning ProbeError pod/e2e-inf-b-c296a3-openshift-default-79f7f78789-fdkc4 Startup probe error: Get "http://10.134.0.135:15021/healthz/ready": dial tcp 10.134.0.135:15021: connect: connection refused... 38m Normal Created pod/payload-processing-e2e-inf-a-fee294-79797dc7c4-4q6tx Container created 38m Normal Started pod/payload-pre-processing-e2e-inf-a-fee294-c499455c5-ds72t Container started 38m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-fee294-c499455c5-ds72t Add eth0 [10.132.0.212/23] from ovn-kubernetes 38m Normal Started pod/payload-processing-e2e-inf-a-fee294-79797dc7c4-4q6tx Container started 38m Warning Unhealthy pod/e2e-inf-b-c296a3-openshift-default-79f7f78789-fdkc4 Startup probe failed: Get "http://10.134.0.135:15021/healthz/ready": dial tcp 10.134.0.135:15021: connect: connection refused 38m Normal Pulled pod/payload-processing-e2e-inf-a-fee294-79797dc7c4-4q6tx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Created pod/payload-pre-processing-e2e-inf-a-fee294-c499455c5-ds72t Container created 38m Normal Pulled pod/payload-pre-processing-e2e-inf-a-fee294-c499455c5-ds72t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/payload-processing-e2e-inf-a-fee294-79797dc7c4-4q6tx Add eth0 [10.132.0.213/23] from ovn-kubernetes 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-c296a3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-c296a3-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Normal Killing pod/e2e-api-b-39322f-openshift-default-78f75cb6d5-np7f7 Stopping container istio-proxy 38m Normal AddedLabel gateway/e2e-inf-a-a5af18 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-a5af18 38m Normal ScalingReplicaSet deployment/e2e-inf-a-a5af18-openshift-default Scaled up replica set e2e-inf-a-a5af18-openshift-default-58d79f487f from 0 to 1 38m Normal SuccessfulCreate replicaset/e2e-inf-a-a5af18-openshift-default-58d79f487f Created pod: e2e-inf-a-a5af18-openshift-default-58d79f487f-tw782 38m Normal AddedInterface pod/e2e-inf-a-a5af18-openshift-default-58d79f487f-tw782 Add eth0 [10.132.0.214/23] from ovn-kubernetes 38m Normal Killing pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-47v7d Stopping container istio-proxy 38m Normal Killing pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-zqtg8 Stopping container istio-proxy 38m Normal Killing pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-xgh4q Stopping container istio-proxy 38m Normal Killing pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-mjcl9 Stopping container istio-proxy 38m Normal Killing pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-wwxlh Stopping container istio-proxy 38m Normal Killing pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-ms7lr Stopping container istio-proxy 38m Normal Killing pod/e2e-mt-1fa4b8b6-openshift-default-6765496769-bmbm4 Stopping container istio-proxy 38m Normal Killing pod/e2e-api-b-3d1777-openshift-default-56f984b757-knrnv Stopping container istio-proxy 38m Normal ScalingReplicaSet deployment/e2e-inf-a-c7d192-openshift-default Scaled up replica set e2e-inf-a-c7d192-openshift-default-c4f554bb7 from 0 to 1 38m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-3bd0f6-openshift-default Scaled down replica set e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77 from 3 to 2 38m Normal AddedLabel gateway/e2e-inf-a-c7d192 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-c7d192 38m Normal SuccessfulCreate replicaset/e2e-inf-a-c7d192-openshift-default-c4f554bb7 Created pod: e2e-inf-a-c7d192-openshift-default-c4f554bb7-rgzsj 38m Normal AddedInterface pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-rgzsj Add eth0 [10.132.0.215/23] from ovn-kubernetes 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-3bd0f6-openshift-default New size: 2; reason: All metrics below target 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-a5af18 Scaled up replica set payload-processing-e2e-inf-a-a5af18-5786f6f7f6 from 0 to 1 38m Normal Killing pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-p9dds Stopping container istio-proxy 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-a5af18-84d9b694d7 Error creating: pods "payload-pre-processing-e2e-inf-a-a5af18-84d9b694d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-a5af18: serviceaccount "payload-processing-e2e-inf-a-a5af18" not found 38m Normal SuccessfulDelete replicaset/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77 Deleted pod: e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-p9dds 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-a5af18 Scaled up replica set payload-pre-processing-e2e-inf-a-a5af18-84d9b694d7 from 0 to 1 37m Normal Pulled pod/payload-pre-processing-e2e-inf-a-a5af18-84d9b694d7-lzhcq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Created pod/payload-processing-e2e-inf-a-a5af18-5786f6f7f6-v8znz Container created 37m Normal AddedInterface pod/payload-processing-e2e-inf-a-a5af18-5786f6f7f6-v8znz Add eth0 [10.132.0.217/23] from ovn-kubernetes 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-a5af18-5786f6f7f6 Created pod: payload-processing-e2e-inf-a-a5af18-5786f6f7f6-v8znz 37m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-a5af18-5786f6f7f6 Error creating: pods "payload-processing-e2e-inf-a-a5af18-5786f6f7f6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-a5af18: serviceaccount "payload-processing-e2e-inf-a-a5af18" not found 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-a5af18-84d9b694d7 Created pod: payload-pre-processing-e2e-inf-a-a5af18-84d9b694d7-lzhcq 37m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-a5af18-84d9b694d7-lzhcq Add eth0 [10.132.0.216/23] from ovn-kubernetes 37m Normal Started pod/payload-processing-e2e-inf-a-a5af18-5786f6f7f6-v8znz Container started 37m Normal Started pod/payload-pre-processing-e2e-inf-a-a5af18-84d9b694d7-lzhcq Container started 37m Normal Created pod/payload-pre-processing-e2e-inf-a-a5af18-84d9b694d7-lzhcq Container created 37m Normal Pulled pod/payload-processing-e2e-inf-a-a5af18-5786f6f7f6-v8znz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-c7d192-fcf758f79 Error creating: pods "payload-pre-processing-e2e-inf-a-c7d192-fcf758f79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-c7d192: serviceaccount "payload-processing-e2e-inf-a-c7d192" not found 37m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-c7d192-7f56447bb4 Error creating: pods "payload-processing-e2e-inf-a-c7d192-7f56447bb4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-c7d192: serviceaccount "payload-processing-e2e-inf-a-c7d192" not found 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-c7d192 Scaled up replica set payload-pre-processing-e2e-inf-a-c7d192-fcf758f79 from 0 to 1 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-c7d192 Scaled up replica set payload-processing-e2e-inf-a-c7d192-7f56447bb4 from 0 to 1 37m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-c7d192-fcf758f79-fd7hp Add eth0 [10.134.0.145/23] from ovn-kubernetes 37m Normal AddedInterface pod/payload-processing-e2e-inf-a-c7d192-7f56447bb4-wq8sm Add eth0 [10.134.0.146/23] from ovn-kubernetes 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-c7d192-fcf758f79 Created pod: payload-pre-processing-e2e-inf-a-c7d192-fcf758f79-fd7hp 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-8d985a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-8d985a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-c7d192-7f56447bb4 Created pod: payload-processing-e2e-inf-a-c7d192-7f56447bb4-wq8sm 37m Normal Pulled pod/payload-pre-processing-e2e-inf-a-c7d192-fcf758f79-fd7hp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Created pod/payload-pre-processing-e2e-inf-a-c7d192-fcf758f79-fd7hp Container created 37m Normal Started pod/payload-pre-processing-e2e-inf-a-c7d192-fcf758f79-fd7hp Container started 37m Normal Pulled pod/payload-processing-e2e-inf-a-c7d192-7f56447bb4-wq8sm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Started pod/payload-processing-e2e-inf-a-c7d192-7f56447bb4-wq8sm Container started 37m Normal Created pod/payload-processing-e2e-inf-a-c7d192-7f56447bb4-wq8sm Container created 37m Normal ScalingReplicaSet deployment/e2e-worker-w5-22d634-openshift-default Scaled down replica set e2e-worker-w5-22d634-openshift-default-89dbbf78f from 6 to 5 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-22d634-openshift-default New size: 5; reason: All metrics below target 37m Normal Killing pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-p6swd Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/e2e-worker-w5-22d634-openshift-default-89dbbf78f Deleted pod: e2e-worker-w5-22d634-openshift-default-89dbbf78f-p6swd 37m Normal SuccessfulCreate replicaset/e2e-inf-a-5a69c9-openshift-default-cbff5c97 Created pod: e2e-inf-a-5a69c9-openshift-default-cbff5c97-bfffh 37m Normal ScalingReplicaSet deployment/e2e-inf-a-5a69c9-openshift-default Scaled up replica set e2e-inf-a-5a69c9-openshift-default-cbff5c97 from 0 to 1 37m Normal AddedLabel gateway/e2e-inf-a-5a69c9 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-5a69c9 37m Normal AddedInterface pod/e2e-inf-a-5a69c9-openshift-default-cbff5c97-bfffh Add eth0 [10.134.0.147/23] from ovn-kubernetes 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-fee294-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-fee294-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-9aafe6-openshift-default Scaled up replica set e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85 from 0 to 1 37m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85 Created pod: e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-fhq59 37m Normal AddedLabel gateway/e2e-shared-a-w5-9aafe6 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w5-9aafe6 37m Normal AddedInterface pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-fhq59 Add eth0 [10.132.0.219/23] from ovn-kubernetes 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-5a69c9-95cc9866c Created pod: payload-pre-processing-e2e-inf-a-5a69c9-95cc9866c-bsqgg 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-5a69c9-95cc9866c Error creating: pods "payload-pre-processing-e2e-inf-a-5a69c9-95cc9866c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-5a69c9: serviceaccount "payload-processing-e2e-inf-a-5a69c9" not found 37m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-5a69c9-7779f4d998 Error creating: pods "payload-processing-e2e-inf-a-5a69c9-7779f4d998-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-5a69c9: serviceaccount "payload-processing-e2e-inf-a-5a69c9" not found 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-5a69c9 Scaled up replica set payload-processing-e2e-inf-a-5a69c9-7779f4d998 from 0 to 1 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-5a69c9 Scaled up replica set payload-pre-processing-e2e-inf-a-5a69c9-95cc9866c from 0 to 1 37m Normal Created pod/payload-pre-processing-e2e-inf-a-5a69c9-95cc9866c-bsqgg Container created 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-5a69c9-7779f4d998 Created pod: payload-processing-e2e-inf-a-5a69c9-7779f4d998-xrblq 37m Normal Started pod/payload-pre-processing-e2e-inf-a-5a69c9-95cc9866c-bsqgg Container started 37m Normal Pulled pod/payload-pre-processing-e2e-inf-a-5a69c9-95cc9866c-bsqgg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-5a69c9-95cc9866c-bsqgg Add eth0 [10.132.0.220/23] from ovn-kubernetes 37m Normal AddedInterface pod/payload-processing-e2e-inf-a-5a69c9-7779f4d998-xrblq Add eth0 [10.134.0.148/23] from ovn-kubernetes 37m Normal Pulled pod/payload-processing-e2e-inf-a-5a69c9-7779f4d998-xrblq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Started pod/payload-processing-e2e-inf-a-5a69c9-7779f4d998-xrblq Container started 37m Normal Created pod/payload-processing-e2e-inf-a-5a69c9-7779f4d998-xrblq Container created 37m Normal ScalingReplicaSet deployment/e2e-inf-a-4d3168-openshift-default Scaled up replica set e2e-inf-a-4d3168-openshift-default-96c48bc6c from 1 to 2 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-4d3168-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 37m Normal SuccessfulCreate replicaset/e2e-inf-a-4d3168-openshift-default-96c48bc6c Created pod: e2e-inf-a-4d3168-openshift-default-96c48bc6c-4fcgv 37m Normal Started pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-4fcgv Container started 37m Normal Created pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-4fcgv Container created 37m Normal Pulled pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-4fcgv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-4fcgv Add eth0 [10.132.0.221/23] from ovn-kubernetes 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w5-9aafe6 Scaled up replica set payload-processing-e2e-shared-a-w5-9aafe6-788454f66b from 0 to 1 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w5-9aafe6 Scaled up replica set payload-pre-processing-e2e-shared-a-w5-9aafe6-6f8f97cff9 from 0 to 1 37m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w5-9aafe6-788454f66b Error creating: pods "payload-processing-e2e-shared-a-w5-9aafe6-788454f66b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-9aafe6: serviceaccount "payload-processing-e2e-shared-a-w5-9aafe6" not found 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w5-9aafe6-6f8f97cff9 Error creating: pods "payload-pre-processing-e2e-shared-a-w5-9aafe6-6f8f97cff9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-9aafe6: serviceaccount "payload-processing-e2e-shared-a-w5-9aafe6" not found 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w5-9aafe6-6f8f97cff9 Created pod: payload-pre-processing-e2e-shared-a-w5-9aafe6-6f8f97cff9-4cmpr 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w5-9aafe6-788454f66b Created pod: payload-processing-e2e-shared-a-w5-9aafe6-788454f66b-t8w2h 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-c296a3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-c296a3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Normal Started pod/payload-pre-processing-e2e-shared-a-w5-9aafe6-6f8f97cff9-4cmpr Container started 37m Normal Created pod/payload-processing-e2e-shared-a-w5-9aafe6-788454f66b-t8w2h Container created 37m Normal Started pod/payload-processing-e2e-shared-a-w5-9aafe6-788454f66b-t8w2h Container started 37m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w5-9aafe6-6f8f97cff9-4cmpr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Created pod/payload-pre-processing-e2e-shared-a-w5-9aafe6-6f8f97cff9-4cmpr Container created 37m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w5-9aafe6-6f8f97cff9-4cmpr Add eth0 [10.134.0.151/23] from ovn-kubernetes 37m Normal Pulled pod/payload-processing-e2e-shared-a-w5-9aafe6-788454f66b-t8w2h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/payload-processing-e2e-shared-a-w5-9aafe6-788454f66b-t8w2h Add eth0 [10.134.0.150/23] from ovn-kubernetes 37m Normal ScalingReplicaSet deployment/e2e-worker-w5-22d634-openshift-default Scaled down replica set e2e-worker-w5-22d634-openshift-default-89dbbf78f from 5 to 4 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-22d634-openshift-default New size: 4; reason: All metrics below target 37m Normal Killing pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-bb657 Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/e2e-worker-w5-22d634-openshift-default-89dbbf78f Deleted pod: e2e-worker-w5-22d634-openshift-default-89dbbf78f-bb657 37m Warning Unhealthy pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-4fcgv Startup probe failed: Get "http://10.132.0.221:15021/healthz/ready": dial tcp 10.132.0.221:15021: connect: connection refused 37m Warning ProbeError pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-4fcgv Startup probe error: Get "http://10.132.0.221:15021/healthz/ready": dial tcp 10.132.0.221:15021: connect: connection refused... 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-5a69c9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-5a69c9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-c296a3-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 37m Normal SuccessfulCreate replicaset/e2e-inf-b-c296a3-openshift-default-79f7f78789 Created pod: e2e-inf-b-c296a3-openshift-default-79f7f78789-6gwx9 37m Normal ScalingReplicaSet deployment/e2e-inf-b-c296a3-openshift-default Scaled up replica set e2e-inf-b-c296a3-openshift-default-79f7f78789 from 1 to 2 37m Normal AddedInterface pod/e2e-inf-b-c296a3-openshift-default-79f7f78789-6gwx9 Add eth0 [10.132.0.222/23] from ovn-kubernetes 37m Normal Started pod/e2e-inf-b-c296a3-openshift-default-79f7f78789-6gwx9 Container started 37m Normal Created pod/e2e-inf-b-c296a3-openshift-default-79f7f78789-6gwx9 Container created 37m Normal Pulled pod/e2e-inf-b-c296a3-openshift-default-79f7f78789-6gwx9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Warning ProbeError pod/e2e-inf-b-c296a3-openshift-default-79f7f78789-6gwx9 Startup probe error: Get "http://10.132.0.222:15021/healthz/ready": dial tcp 10.132.0.222:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-inf-b-c296a3-openshift-default-79f7f78789-6gwx9 Startup probe failed: Get "http://10.132.0.222:15021/healthz/ready": dial tcp 10.132.0.222:15021: connect: connection refused 37m Normal AddedLabel gateway/e2e-inf-b-8147a8 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-8147a8 37m Normal SuccessfulCreate replicaset/e2e-inf-b-8147a8-openshift-default-855c99786c Created pod: e2e-inf-b-8147a8-openshift-default-855c99786c-k9ckd 37m Normal ScalingReplicaSet deployment/e2e-inf-b-8147a8-openshift-default Scaled up replica set e2e-inf-b-8147a8-openshift-default-855c99786c from 0 to 1 36m Normal AddedInterface pod/e2e-inf-b-8147a8-openshift-default-855c99786c-k9ckd Add eth0 [10.134.0.153/23] from ovn-kubernetes 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-8147a8-7f7ff5cc5 Error creating: pods "payload-pre-processing-e2e-inf-b-8147a8-7f7ff5cc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-8147a8: serviceaccount "payload-processing-e2e-inf-b-8147a8" not found 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-8147a8 Scaled up replica set payload-pre-processing-e2e-inf-b-8147a8-7f7ff5cc5 from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-8147a8-7f7ff5cc5 Created pod: payload-pre-processing-e2e-inf-b-8147a8-7f7ff5cc5-4fxdk 36m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-8147a8-f8bf948b4 Error creating: pods "payload-processing-e2e-inf-b-8147a8-f8bf948b4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-8147a8: serviceaccount "payload-processing-e2e-inf-b-8147a8" not found 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-8147a8 Scaled up replica set payload-processing-e2e-inf-b-8147a8-f8bf948b4 from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-8147a8-f8bf948b4 Created pod: payload-processing-e2e-inf-b-8147a8-f8bf948b4-575w2 36m Normal Started pod/payload-processing-e2e-inf-b-8147a8-f8bf948b4-575w2 Container started 36m Normal Pulled pod/payload-processing-e2e-inf-b-8147a8-f8bf948b4-575w2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Pulled pod/payload-pre-processing-e2e-inf-b-8147a8-7f7ff5cc5-4fxdk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Started pod/payload-pre-processing-e2e-inf-b-8147a8-7f7ff5cc5-4fxdk Container started 36m Normal Created pod/payload-processing-e2e-inf-b-8147a8-f8bf948b4-575w2 Container created 36m Normal AddedInterface pod/payload-processing-e2e-inf-b-8147a8-f8bf948b4-575w2 Add eth0 [10.132.0.224/23] from ovn-kubernetes 36m Normal Created pod/payload-pre-processing-e2e-inf-b-8147a8-7f7ff5cc5-4fxdk Container created 36m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-8147a8-7f7ff5cc5-4fxdk Add eth0 [10.132.0.223/23] from ovn-kubernetes 36m Normal SuccessfulCreate replicaset/e2e-inf-b-e619b6-openshift-default-6d4c644df9 Created pod: e2e-inf-b-e619b6-openshift-default-6d4c644df9-kv7vp 36m Normal AddedLabel gateway/e2e-inf-b-e619b6 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-e619b6 36m Normal ScalingReplicaSet deployment/e2e-inf-b-e619b6-openshift-default Scaled up replica set e2e-inf-b-e619b6-openshift-default-6d4c644df9 from 0 to 1 36m Normal AddedInterface pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-kv7vp Add eth0 [10.132.0.225/23] from ovn-kubernetes 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-e619b6 Scaled up replica set payload-processing-e2e-inf-b-e619b6-7f44c56884 from 0 to 1 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-e619b6 Scaled up replica set payload-pre-processing-e2e-inf-b-e619b6-868f869f9f from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-e619b6-868f869f9f Created pod: payload-pre-processing-e2e-inf-b-e619b6-868f869f9f-j8pwm 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-e619b6-7f44c56884 Created pod: payload-processing-e2e-inf-b-e619b6-7f44c56884-ctrpq 36m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-e619b6-7f44c56884 Error creating: pods "payload-processing-e2e-inf-b-e619b6-7f44c56884-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-e619b6: serviceaccount "payload-processing-e2e-inf-b-e619b6" not found 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-e619b6-868f869f9f Error creating: pods "payload-pre-processing-e2e-inf-b-e619b6-868f869f9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-e619b6: serviceaccount "payload-processing-e2e-inf-b-e619b6" not found 36m Normal Created pod/payload-pre-processing-e2e-inf-b-e619b6-868f869f9f-j8pwm Container created 36m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-e619b6-868f869f9f-j8pwm Add eth0 [10.132.0.226/23] from ovn-kubernetes 36m Normal Pulled pod/payload-pre-processing-e2e-inf-b-e619b6-868f869f9f-j8pwm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/payload-processing-e2e-inf-b-e619b6-7f44c56884-ctrpq Add eth0 [10.132.0.227/23] from ovn-kubernetes 36m Normal Started pod/payload-processing-e2e-inf-b-e619b6-7f44c56884-ctrpq Container started 36m Normal Started pod/payload-pre-processing-e2e-inf-b-e619b6-868f869f9f-j8pwm Container started 36m Normal SuccessfulCreate replicaset/e2e-inf-b-2a3ef0-openshift-default-5cdf89c9 Created pod: e2e-inf-b-2a3ef0-openshift-default-5cdf89c9-fwcr6 36m Normal ScalingReplicaSet deployment/e2e-inf-b-2a3ef0-openshift-default Scaled up replica set e2e-inf-b-2a3ef0-openshift-default-5cdf89c9 from 0 to 1 36m Normal Created pod/payload-processing-e2e-inf-b-e619b6-7f44c56884-ctrpq Container created 36m Normal AddedLabel gateway/e2e-inf-b-2a3ef0 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-2a3ef0 36m Normal Pulled pod/payload-processing-e2e-inf-b-e619b6-7f44c56884-ctrpq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/e2e-inf-b-2a3ef0-openshift-default-5cdf89c9-fwcr6 Add eth0 [10.134.0.154/23] from ovn-kubernetes 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-2a3ef0-8648759887 Error creating: pods "payload-pre-processing-e2e-inf-b-2a3ef0-8648759887-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-2a3ef0: serviceaccount "payload-processing-e2e-inf-b-2a3ef0" not found 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-2a3ef0 Scaled up replica set payload-processing-e2e-inf-b-2a3ef0-9d775496 from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-2a3ef0-9d775496 Created pod: payload-processing-e2e-inf-b-2a3ef0-9d775496-fs4gv 36m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-2a3ef0-9d775496 Error creating: pods "payload-processing-e2e-inf-b-2a3ef0-9d775496-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-2a3ef0: serviceaccount "payload-processing-e2e-inf-b-2a3ef0" not found 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-2a3ef0 Scaled up replica set payload-pre-processing-e2e-inf-b-2a3ef0-8648759887 from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-2a3ef0-8648759887 Created pod: payload-pre-processing-e2e-inf-b-2a3ef0-8648759887-mhb5v 36m Normal AddedInterface pod/payload-processing-e2e-inf-b-2a3ef0-9d775496-fs4gv Add eth0 [10.134.0.156/23] from ovn-kubernetes 36m Normal Pulled pod/payload-processing-e2e-inf-b-2a3ef0-9d775496-fs4gv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Pulled pod/payload-pre-processing-e2e-inf-b-2a3ef0-8648759887-mhb5v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Created pod/payload-pre-processing-e2e-inf-b-2a3ef0-8648759887-mhb5v Container created 36m Normal Started pod/payload-pre-processing-e2e-inf-b-2a3ef0-8648759887-mhb5v Container started 36m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-2a3ef0-8648759887-mhb5v Add eth0 [10.134.0.155/23] from ovn-kubernetes 36m Normal Created pod/e2e-inf-a-5a69c9-openshift-default-cbff5c97-bfffh Container created 36m Normal Started pod/e2e-inf-a-5a69c9-openshift-default-cbff5c97-bfffh Container started 36m Normal Created pod/payload-processing-e2e-inf-b-2a3ef0-9d775496-fs4gv Container created 36m Normal Started pod/payload-processing-e2e-inf-b-2a3ef0-9d775496-fs4gv Container started 36m Normal Pulled pod/e2e-inf-a-5a69c9-openshift-default-cbff5c97-bfffh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-c7d192-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 36m Normal SuccessfulCreate replicaset/e2e-inf-a-c7d192-openshift-default-c4f554bb7 Created pod: e2e-inf-a-c7d192-openshift-default-c4f554bb7-xw24m 36m Normal ScalingReplicaSet deployment/e2e-inf-a-c7d192-openshift-default Scaled up replica set e2e-inf-a-c7d192-openshift-default-c4f554bb7 from 1 to 4 36m Normal SuccessfulCreate replicaset/e2e-inf-a-c7d192-openshift-default-c4f554bb7 Created pod: e2e-inf-a-c7d192-openshift-default-c4f554bb7-5gpj4 36m Normal SuccessfulCreate replicaset/e2e-inf-a-c7d192-openshift-default-c4f554bb7 Created pod: e2e-inf-a-c7d192-openshift-default-c4f554bb7-cjl27 36m Normal AddedInterface pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-cjl27 Add eth0 [10.133.0.93/23] from ovn-kubernetes 36m Normal Pulled pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-cjl27 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Pulled pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-fhq59 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Created pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-fhq59 Container created 36m Normal Started pod/e2e-inf-a-fee294-openshift-default-644cdf5d45-cf2vc Container started 36m Normal Created pod/e2e-inf-a-fee294-openshift-default-644cdf5d45-cf2vc Container created 36m Normal Pulled pod/e2e-inf-a-fee294-openshift-default-644cdf5d45-cf2vc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-5gpj4 Add eth0 [10.132.0.228/23] from ovn-kubernetes 36m Normal Started pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-fhq59 Container started 36m Normal Pulled pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-5gpj4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-xw24m Add eth0 [10.134.0.157/23] from ovn-kubernetes 36m Normal Pulled pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-xw24m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Created pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-xw24m Container created 36m Normal Started pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-xw24m Container started 36m Normal Created pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-5gpj4 Container created 36m Normal Started pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-5gpj4 Container started 36m Normal Created pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-cjl27 Container created 36m Normal Started pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-cjl27 Container started 36m Normal Pulled pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-rgzsj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Created pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-rgzsj Container created 36m Normal Started pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-rgzsj Container started 36m Normal AddedLabel gateway/e2e-inf-b-05417d Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-05417d 36m Normal SuccessfulCreate replicaset/e2e-inf-b-05417d-openshift-default-6d96dbcbc5 Created pod: e2e-inf-b-05417d-openshift-default-6d96dbcbc5-nq8jg 36m Normal ScalingReplicaSet deployment/e2e-inf-b-05417d-openshift-default Scaled up replica set e2e-inf-b-05417d-openshift-default-6d96dbcbc5 from 0 to 1 36m Warning ProbeError pod/e2e-inf-a-5a69c9-openshift-default-cbff5c97-bfffh Startup probe error: Get "http://10.134.0.147:15021/healthz/ready": dial tcp 10.134.0.147:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-inf-a-5a69c9-openshift-default-cbff5c97-bfffh Startup probe failed: Get "http://10.134.0.147:15021/healthz/ready": dial tcp 10.134.0.147:15021: connect: connection refused 36m Normal AddedLabel gateway/e2e-shared-b-w5-6fc6bd Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w5-6fc6bd 36m Warning ProbeError pod/e2e-inf-a-fee294-openshift-default-644cdf5d45-cf2vc Startup probe error: Get "http://10.132.0.211:15021/healthz/ready": dial tcp 10.132.0.211:15021: connect: connection refused... 36m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89 Created pod: e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-82f2v 36m Warning Unhealthy pod/e2e-inf-a-fee294-openshift-default-644cdf5d45-cf2vc Startup probe failed: Get "http://10.132.0.211:15021/healthz/ready": dial tcp 10.132.0.211:15021: connect: connection refused 36m Warning Unhealthy pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-cjl27 Startup probe failed: Get "http://10.133.0.93:15021/healthz/ready": dial tcp 10.133.0.93:15021: connect: connection refused 36m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-6fc6bd-openshift-default Scaled up replica set e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89 from 0 to 1 36m Warning ProbeError pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-cjl27 Startup probe error: Get "http://10.133.0.93:15021/healthz/ready": dial tcp 10.133.0.93:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-fhq59 Startup probe failed: Get "http://10.132.0.219:15021/healthz/ready": dial tcp 10.132.0.219:15021: connect: connection refused 36m Warning ProbeError pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-fhq59 Startup probe error: Get "http://10.132.0.219:15021/healthz/ready": dial tcp 10.132.0.219:15021: connect: connection refused... 36m Normal AddedInterface pod/e2e-inf-b-05417d-openshift-default-6d96dbcbc5-nq8jg Add eth0 [10.132.0.229/23] from ovn-kubernetes 36m Warning Unhealthy pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-xw24m Startup probe failed: Get "http://10.134.0.157:15021/healthz/ready": dial tcp 10.134.0.157:15021: connect: connection refused 36m Warning Unhealthy pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-5gpj4 Startup probe failed: Get "http://10.132.0.228:15021/healthz/ready": dial tcp 10.132.0.228:15021: connect: connection refused 36m Warning ProbeError pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-xw24m Startup probe error: Get "http://10.134.0.157:15021/healthz/ready": dial tcp 10.134.0.157:15021: connect: connection refused... 36m Warning ProbeError pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-5gpj4 Startup probe error: Get "http://10.132.0.228:15021/healthz/ready": dial tcp 10.132.0.228:15021: connect: connection refused... 36m Normal AddedInterface pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-82f2v Add eth0 [10.134.0.158/23] from ovn-kubernetes 36m Normal ScalingReplicaSet deployment/e2e-worker-w2-2ba10a-openshift-default Scaled down replica set e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 from 5 to 4 36m Warning Unhealthy pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-rgzsj Startup probe failed: Get "http://10.132.0.215:15021/healthz/ready": dial tcp 10.132.0.215:15021: connect: connection refused 36m Warning ProbeError pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-rgzsj Startup probe error: Get "http://10.132.0.215:15021/healthz/ready": dial tcp 10.132.0.215:15021: connect: connection refused... 36m Normal SuccessfulDelete replicaset/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 Deleted pod: e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-5lkhc 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-2ba10a-openshift-default New size: 4; reason: All metrics below target 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-08ebee-openshift-default New size: 4; reason: All metrics below target 36m Normal SuccessfulDelete replicaset/e2e-worker-w4-08ebee-openshift-default-5f5bf46967 Deleted pod: e2e-worker-w4-08ebee-openshift-default-5f5bf46967-nr58r 36m Normal Killing pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-nr58r Stopping container istio-proxy 36m Normal Killing pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-5lkhc Stopping container istio-proxy 36m Normal ScalingReplicaSet deployment/e2e-worker-w4-08ebee-openshift-default Scaled down replica set e2e-worker-w4-08ebee-openshift-default-5f5bf46967 from 5 to 4 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-05417d Scaled up replica set payload-processing-e2e-inf-b-05417d-754c7b55dd from 0 to 1 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-c7d192-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-c7d192-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-05417d-754c7b55dd Created pod: payload-processing-e2e-inf-b-05417d-754c7b55dd-kclz4 36m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-05417d-754c7b55dd Error creating: pods "payload-processing-e2e-inf-b-05417d-754c7b55dd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-05417d: serviceaccount "payload-processing-e2e-inf-b-05417d" not found 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-05417d-5cbcfc559 Error creating: pods "payload-pre-processing-e2e-inf-b-05417d-5cbcfc559-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-05417d: serviceaccount "payload-processing-e2e-inf-b-05417d" not found 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-05417d-5cbcfc559 Created pod: payload-pre-processing-e2e-inf-b-05417d-5cbcfc559-jjksv 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-05417d Scaled up replica set payload-pre-processing-e2e-inf-b-05417d-5cbcfc559 from 0 to 1 36m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-05417d-5cbcfc559-jjksv Add eth0 [10.132.0.230/23] from ovn-kubernetes 36m Normal AddedInterface pod/payload-processing-e2e-inf-b-05417d-754c7b55dd-kclz4 Add eth0 [10.134.0.159/23] from ovn-kubernetes 36m Normal Pulled pod/payload-processing-e2e-inf-b-05417d-754c7b55dd-kclz4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Created pod/payload-processing-e2e-inf-b-05417d-754c7b55dd-kclz4 Container created 36m Normal Started pod/payload-processing-e2e-inf-b-05417d-754c7b55dd-kclz4 Container started 36m Normal Started pod/payload-pre-processing-e2e-inf-b-05417d-5cbcfc559-jjksv Container started 36m Normal Created pod/payload-pre-processing-e2e-inf-b-05417d-5cbcfc559-jjksv Container created 36m Normal Pulled pod/payload-pre-processing-e2e-inf-b-05417d-5cbcfc559-jjksv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-9aafe6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-9aafe6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w5-6fc6bd-78c4d8b54c Error creating: pods "payload-pre-processing-e2e-shared-b-w5-6fc6bd-78c4d8b54c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-6fc6bd: serviceaccount "payload-processing-e2e-shared-b-w5-6fc6bd" not found 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w5-6fc6bd Scaled up replica set payload-processing-e2e-shared-b-w5-6fc6bd-cf68d45f8 from 0 to 1 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w5-6fc6bd Scaled up replica set payload-pre-processing-e2e-shared-b-w5-6fc6bd-78c4d8b54c from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w5-6fc6bd-78c4d8b54c Created pod: payload-pre-processing-e2e-shared-b-w5-6fc6bd-78c4d8b54c-qg69n 36m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w5-6fc6bd-cf68d45f8 Error creating: pods "payload-processing-e2e-shared-b-w5-6fc6bd-cf68d45f8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-6fc6bd: serviceaccount "payload-processing-e2e-shared-b-w5-6fc6bd" not found 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w5-6fc6bd-cf68d45f8 Created pod: payload-processing-e2e-shared-b-w5-6fc6bd-cf68d45f8-22lvh 36m Normal Created pod/payload-pre-processing-e2e-shared-b-w5-6fc6bd-78c4d8b54c-qg69n Container created 36m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w5-6fc6bd-78c4d8b54c-qg69n Add eth0 [10.134.0.161/23] from ovn-kubernetes 36m Normal Started pod/payload-processing-e2e-shared-b-w5-6fc6bd-cf68d45f8-22lvh Container started 36m Normal Pulled pod/payload-processing-e2e-shared-b-w5-6fc6bd-cf68d45f8-22lvh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/payload-processing-e2e-shared-b-w5-6fc6bd-cf68d45f8-22lvh Add eth0 [10.134.0.160/23] from ovn-kubernetes 36m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w5-6fc6bd-78c4d8b54c-qg69n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Started pod/payload-pre-processing-e2e-shared-b-w5-6fc6bd-78c4d8b54c-qg69n Container started 36m Normal Created pod/payload-processing-e2e-shared-b-w5-6fc6bd-cf68d45f8-22lvh Container created 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-4d3168-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 36m Normal AddedInterface pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-4g7sz Add eth0 [10.133.0.95/23] from ovn-kubernetes 36m Normal SuccessfulCreate replicaset/e2e-inf-a-4d3168-openshift-default-96c48bc6c Created pod: e2e-inf-a-4d3168-openshift-default-96c48bc6c-4g7sz 36m Normal ScalingReplicaSet deployment/e2e-inf-a-4d3168-openshift-default Scaled up replica set e2e-inf-a-4d3168-openshift-default-96c48bc6c from 2 to 3 36m Normal Pulled pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-4g7sz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-4g7sz Container started 36m Normal Created pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-4g7sz Container created 36m Normal ScalingReplicaSet deployment/e2e-worker-w6-784e7a-openshift-default Scaled down replica set e2e-worker-w6-784e7a-openshift-default-595898bd5c from 6 to 5 36m Normal SuccessfulDelete replicaset/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc Deleted pod: e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-bkgjl 36m Normal Killing pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-bkgjl Stopping container istio-proxy 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-43b37d-openshift-default New size: 5; reason: All metrics below target 36m Normal ScalingReplicaSet deployment/e2e-worker-w3-43b37d-openshift-default Scaled down replica set e2e-worker-w3-43b37d-openshift-default-5dcf864fcc from 6 to 5 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-784e7a-openshift-default New size: 5; reason: All metrics below target 36m Normal SuccessfulDelete replicaset/e2e-worker-w6-784e7a-openshift-default-595898bd5c Deleted pod: e2e-worker-w6-784e7a-openshift-default-595898bd5c-lx7p6 36m Normal Killing pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-lx7p6 Stopping container istio-proxy 36m Warning ProbeError pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-4g7sz Startup probe error: Get "http://10.133.0.95:15021/healthz/ready": dial tcp 10.133.0.95:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-4g7sz Startup probe failed: Get "http://10.133.0.95:15021/healthz/ready": dial tcp 10.133.0.95:15021: connect: connection refused 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-5a69c9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-5a69c9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-fee294-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-fee294-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-c7d192-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-c7d192-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-9aafe6-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-9aafe6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Normal Killing pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-9vrlz Stopping container istio-proxy 35m Normal SuccessfulDelete replicaset/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 Deleted pod: e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-9vrlz 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2e046a-openshift-default New size: 5; reason: All metrics below target 35m Normal ScalingReplicaSet deployment/e2e-worker-w0-2e046a-openshift-default Scaled down replica set e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 from 6 to 5 35m Normal SuccessfulCreate replicaset/e2e-inf-a-c7d192-openshift-default-c4f554bb7 Created pod: e2e-inf-a-c7d192-openshift-default-c4f554bb7-kl464 35m Normal ScalingReplicaSet deployment/e2e-inf-a-c7d192-openshift-default Scaled up replica set e2e-inf-a-c7d192-openshift-default-c4f554bb7 from 4 to 5 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-c7d192-openshift-default New size: 5; reason: All metrics below target 35m Normal AddedInterface pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-kl464 Add eth0 [10.134.0.162/23] from ovn-kubernetes 35m Normal Pulled pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-kl464 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-kl464 Container created 35m Normal Started pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-kl464 Container started 35m Normal Created pod/e2e-inf-b-8147a8-openshift-default-855c99786c-k9ckd Container created 35m Normal Started pod/e2e-inf-b-8147a8-openshift-default-855c99786c-k9ckd Container started 35m Normal Pulled pod/e2e-inf-b-8147a8-openshift-default-855c99786c-k9ckd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Warning ProbeError pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-kl464 Startup probe error: Get "http://10.134.0.162:15021/healthz/ready": dial tcp 10.134.0.162:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-kl464 Startup probe failed: Get "http://10.134.0.162:15021/healthz/ready": dial tcp 10.134.0.162:15021: connect: connection refused 35m Warning Unhealthy pod/e2e-inf-b-8147a8-openshift-default-855c99786c-k9ckd Startup probe failed: Get "http://10.134.0.153:15021/healthz/ready": dial tcp 10.134.0.153:15021: connect: connection refused 35m Warning ProbeError pod/e2e-inf-b-8147a8-openshift-default-855c99786c-k9ckd Startup probe error: Get "http://10.134.0.153:15021/healthz/ready": dial tcp 10.134.0.153:15021: connect: connection refused... 35m Normal SuccessfulDelete replicaset/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 Deleted pod: e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-f7th7 35m Normal ScalingReplicaSet deployment/e2e-worker-w0-2e046a-openshift-default Scaled down replica set e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 from 5 to 4 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2e046a-openshift-default New size: 4; reason: All metrics below target 35m Normal Killing pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-f7th7 Stopping container istio-proxy 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-8147a8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-8147a8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning Unhealthy pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-f7th7 Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-f7th7 Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-8147a8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-8147a8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Normal Killing pod/maas-default-gateway-openshift-default-5db77fbd9f-r5b9v Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5db77fbd9f Deleted pod: maas-default-gateway-openshift-default-5db77fbd9f-z7hjw 34m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: All metrics below target 34m Normal Killing pod/maas-default-gateway-openshift-default-5db77fbd9f-z7hjw Stopping container istio-proxy 34m Normal Killing pod/maas-default-gateway-openshift-default-5db77fbd9f-r8hqd Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5db77fbd9f Deleted pod: maas-default-gateway-openshift-default-5db77fbd9f-r8hqd 34m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5db77fbd9f Deleted pod: maas-default-gateway-openshift-default-5db77fbd9f-r5b9v 34m Normal Started pod/e2e-inf-b-05417d-openshift-default-6d96dbcbc5-nq8jg Container started 34m Normal Pulled pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-82f2v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Started pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-kv7vp Container started 34m Normal Pulled pod/e2e-inf-b-05417d-openshift-default-6d96dbcbc5-nq8jg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Pulled pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-kv7vp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Created pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-kv7vp Container created 34m Normal Created pod/e2e-inf-b-05417d-openshift-default-6d96dbcbc5-nq8jg Container created 34m Normal Started pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-82f2v Container started 34m Normal Created pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-82f2v Container created 34m Warning ProbeError pod/e2e-inf-b-2a3ef0-openshift-default-5cdf89c9-fwcr6 Readiness probe error: Get "http://10.134.0.154:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 34m Warning Unhealthy pod/e2e-inf-b-2a3ef0-openshift-default-5cdf89c9-fwcr6 Readiness probe failed: Get "http://10.134.0.154:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 34m Normal SuccessfulDelete replicaset/e2e-worker-w6-784e7a-openshift-default-595898bd5c Deleted pod: e2e-worker-w6-784e7a-openshift-default-595898bd5c-kw5tw 34m Normal Killing pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-kw5tw Stopping container istio-proxy 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-784e7a-openshift-default New size: 4; reason: All metrics below target 34m Normal Started pod/e2e-inf-b-2a3ef0-openshift-default-5cdf89c9-fwcr6 Container started 34m Warning Unhealthy pod/e2e-inf-b-05417d-openshift-default-6d96dbcbc5-nq8jg Startup probe failed: Get "http://10.132.0.229:15021/healthz/ready": dial tcp 10.132.0.229:15021: connect: connection refused 34m Warning ProbeError pod/e2e-inf-b-05417d-openshift-default-6d96dbcbc5-nq8jg Startup probe error: Get "http://10.132.0.229:15021/healthz/ready": dial tcp 10.132.0.229:15021: connect: connection refused... 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-e619b6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Normal Pulled pod/e2e-inf-b-2a3ef0-openshift-default-5cdf89c9-fwcr6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-e619b6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Normal Created pod/e2e-inf-b-2a3ef0-openshift-default-5cdf89c9-fwcr6 Container created 34m Warning ProbeError pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-82f2v Startup probe error: Get "http://10.134.0.158:15021/healthz/ready": dial tcp 10.134.0.158:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-82f2v Startup probe failed: Get "http://10.134.0.158:15021/healthz/ready": dial tcp 10.134.0.158:15021: connect: connection refused 34m Warning Unhealthy pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-kv7vp Startup probe failed: Get "http://10.132.0.225:15021/healthz/ready": dial tcp 10.132.0.225:15021: connect: connection refused 34m Warning ProbeError pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-kv7vp Startup probe error: Get "http://10.132.0.225:15021/healthz/ready": dial tcp 10.132.0.225:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-inf-b-2a3ef0-openshift-default-5cdf89c9-fwcr6 Startup probe failed: Get "http://10.134.0.154:15021/healthz/ready": dial tcp 10.134.0.154:15021: connect: connection refused 34m Warning ProbeError pod/e2e-inf-b-2a3ef0-openshift-default-5cdf89c9-fwcr6 Startup probe error: Get "http://10.134.0.154:15021/healthz/ready": dial tcp 10.134.0.154:15021: connect: connection refused... 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-2a3ef0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-2a3ef0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-6fc6bd-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-6fc6bd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-e619b6-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-e619b6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Normal Killing pod/payload-processing-e2e-inf-a-4d3168-d94579868-m6l26 Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-inf-a-4d3168-5cf56f8fd7-dbn2k Stopping container payload-pre-processing 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-6fc6bd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-6fc6bd-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-05417d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-05417d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-4d3168-5cf56f8fd7-dbn2k Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning Unhealthy pod/payload-processing-e2e-inf-a-4d3168-d94579868-m6l26 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-2a3ef0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-2a3ef0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Normal Killing pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-4g7sz Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-4fcgv Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-l4blm Stopping container istio-proxy 34m Normal Killing pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-g26hx Stopping container istio-proxy 34m Normal Killing pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-8npq2 Stopping container istio-proxy 34m Normal ScalingReplicaSet deployment/e2e-worker-w1-fc684b-openshift-default Scaled down replica set e2e-worker-w1-fc684b-openshift-default-7fcf678997 from 6 to 4 34m Normal SuccessfulDelete replicaset/e2e-worker-w1-fc684b-openshift-default-7fcf678997 Deleted pod: e2e-worker-w1-fc684b-openshift-default-7fcf678997-g26hx 34m Normal SuccessfulDelete replicaset/e2e-worker-w1-fc684b-openshift-default-7fcf678997 Deleted pod: e2e-worker-w1-fc684b-openshift-default-7fcf678997-8npq2 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-fc684b-openshift-default New size: 4; reason: All metrics below target 34m Warning ProbeError pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-l4blm Readiness probe error: HTTP probe failed with statuscode: 503... 34m Warning Unhealthy pod/e2e-inf-a-4d3168-openshift-default-96c48bc6c-l4blm Readiness probe failed: HTTP probe failed with statuscode: 503 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-8147a8-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 34m Normal ScalingReplicaSet deployment/e2e-inf-b-8147a8-openshift-default Scaled up replica set e2e-inf-b-8147a8-openshift-default-855c99786c from 1 to 2 34m Normal SuccessfulCreate replicaset/e2e-inf-b-8147a8-openshift-default-855c99786c Created pod: e2e-inf-b-8147a8-openshift-default-855c99786c-4sv5h 33m Normal Created pod/e2e-inf-b-8147a8-openshift-default-855c99786c-4sv5h Container created 33m Normal AddedInterface pod/e2e-inf-b-8147a8-openshift-default-855c99786c-4sv5h Add eth0 [10.132.0.249/23] from ovn-kubernetes 33m Normal Started pod/e2e-inf-b-8147a8-openshift-default-855c99786c-4sv5h Container started 33m Normal Pulled pod/e2e-inf-b-8147a8-openshift-default-855c99786c-4sv5h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Killing pod/payload-pre-processing-e2e-inf-b-c296a3-778459576c-5mnqg Stopping container payload-pre-processing 33m Normal Killing pod/payload-processing-e2e-inf-b-c296a3-544676fc5d-fvr68 Stopping container payload-processing 33m Normal Started pod/e2e-inf-a-a5af18-openshift-default-58d79f487f-tw782 Container started 33m Normal Created pod/e2e-inf-a-a5af18-openshift-default-58d79f487f-tw782 Container created 33m Normal Pulled pod/e2e-inf-a-a5af18-openshift-default-58d79f487f-tw782 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Warning ProbeError pod/e2e-inf-b-8147a8-openshift-default-855c99786c-4sv5h Startup probe error: Get "http://10.132.0.249:15021/healthz/ready": dial tcp 10.132.0.249:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-inf-b-8147a8-openshift-default-855c99786c-4sv5h Startup probe failed: Get "http://10.132.0.249:15021/healthz/ready": dial tcp 10.132.0.249:15021: connect: connection refused 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-a5af18-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-a5af18-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning ProbeError pod/e2e-inf-a-a5af18-openshift-default-58d79f487f-tw782 Startup probe error: Get "http://10.132.0.214:15021/healthz/ready": dial tcp 10.132.0.214:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-inf-a-a5af18-openshift-default-58d79f487f-tw782 Startup probe failed: Get "http://10.132.0.214:15021/healthz/ready": dial tcp 10.132.0.214:15021: connect: connection refused 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-e619b6-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 33m Normal SuccessfulCreate replicaset/e2e-inf-b-e619b6-openshift-default-6d4c644df9 Created pod: e2e-inf-b-e619b6-openshift-default-6d4c644df9-ml8sr 33m Normal ScalingReplicaSet deployment/e2e-inf-b-e619b6-openshift-default Scaled up replica set e2e-inf-b-e619b6-openshift-default-6d4c644df9 from 1 to 2 33m Normal Created pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-ml8sr Container created 33m Normal Started pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-ml8sr Container started 33m Normal AddedInterface pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-ml8sr Add eth0 [10.134.0.179/23] from ovn-kubernetes 33m Normal Pulled pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-ml8sr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-fee294-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 33m Normal ScalingReplicaSet deployment/e2e-inf-a-fee294-openshift-default Scaled up replica set e2e-inf-a-fee294-openshift-default-644cdf5d45 from 1 to 2 33m Normal SuccessfulCreate replicaset/e2e-inf-a-fee294-openshift-default-644cdf5d45 Created pod: e2e-inf-a-fee294-openshift-default-644cdf5d45-4dbqs 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-c7d192-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 33m Normal Started pod/e2e-inf-a-fee294-openshift-default-644cdf5d45-4dbqs Container started 33m Normal ScalingReplicaSet deployment/e2e-inf-a-c7d192-openshift-default Scaled up replica set e2e-inf-a-c7d192-openshift-default-c4f554bb7 from 5 to 7 33m Normal AddedInterface pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-nbz7s Add eth0 [10.133.0.105/23] from ovn-kubernetes 33m Normal AddedInterface pod/e2e-inf-a-fee294-openshift-default-644cdf5d45-4dbqs Add eth0 [10.134.0.180/23] from ovn-kubernetes 33m Normal SuccessfulCreate replicaset/e2e-inf-a-c7d192-openshift-default-c4f554bb7 Created pod: e2e-inf-a-c7d192-openshift-default-c4f554bb7-4krkx 33m Normal Pulled pod/e2e-inf-a-fee294-openshift-default-644cdf5d45-4dbqs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-inf-a-fee294-openshift-default-644cdf5d45-4dbqs Container created 33m Normal Pulled pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-nbz7s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal SuccessfulCreate replicaset/e2e-inf-a-c7d192-openshift-default-c4f554bb7 Created pod: e2e-inf-a-c7d192-openshift-default-c4f554bb7-nbz7s 33m Normal Created pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-nbz7s Container created 33m Normal AddedInterface pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-4krkx Add eth0 [10.134.0.181/23] from ovn-kubernetes 33m Normal Pulled pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-4krkx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-4krkx Container created 33m Normal Started pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-nbz7s Container started 33m Normal Started pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-4krkx Container started 33m Warning ProbeError pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-ml8sr Startup probe error: Get "http://10.134.0.179:15021/healthz/ready": dial tcp 10.134.0.179:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-ml8sr Startup probe failed: Get "http://10.134.0.179:15021/healthz/ready": dial tcp 10.134.0.179:15021: connect: connection refused 33m Warning ProbeError pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-nbz7s Startup probe error: Get "http://10.133.0.105:15021/healthz/ready": dial tcp 10.133.0.105:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-nbz7s Startup probe failed: Get "http://10.133.0.105:15021/healthz/ready": dial tcp 10.133.0.105:15021: connect: connection refused 33m Warning ProbeError pod/e2e-inf-a-fee294-openshift-default-644cdf5d45-4dbqs Startup probe error: Get "http://10.134.0.180:15021/healthz/ready": dial tcp 10.134.0.180:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-4krkx Startup probe failed: Get "http://10.134.0.181:15021/healthz/ready": dial tcp 10.134.0.181:15021: connect: connection refused 33m Warning ProbeError pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-4krkx Startup probe error: Get "http://10.134.0.181:15021/healthz/ready": dial tcp 10.134.0.181:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-inf-a-fee294-openshift-default-644cdf5d45-4dbqs Startup probe failed: Get "http://10.134.0.180:15021/healthz/ready": dial tcp 10.134.0.180:15021: connect: connection refused 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-a5af18-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning Unhealthy pod/payload-processing-e2e-inf-b-c296a3-544676fc5d-fvr68 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-a5af18-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-c296a3-778459576c-5mnqg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Normal Killing pod/e2e-inf-b-c296a3-openshift-default-79f7f78789-6gwx9 Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-b-c296a3-openshift-default-79f7f78789-fdkc4 Stopping container istio-proxy 33m Normal AddedLabel gateway/e2e-shared-a-w1-af260f Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w1-af260f 33m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-af260f-openshift-default-846bb69f67 Created pod: e2e-shared-a-w1-af260f-openshift-default-846bb69f67-cxjmw 33m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-af260f-openshift-default Scaled up replica set e2e-shared-a-w1-af260f-openshift-default-846bb69f67 from 0 to 1 33m Normal AddedInterface pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-cxjmw Add eth0 [10.132.0.252/23] from ovn-kubernetes 33m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w1-af260f-fd896cf88 Error creating: pods "payload-processing-e2e-shared-a-w1-af260f-fd896cf88-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-af260f: serviceaccount "payload-processing-e2e-shared-a-w1-af260f" not found 33m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w1-af260f Scaled up replica set payload-processing-e2e-shared-a-w1-af260f-fd896cf88 from 0 to 1 33m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w1-af260f Scaled up replica set payload-pre-processing-e2e-shared-a-w1-af260f-7cfc9454ff from 0 to 1 33m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w1-af260f-7cfc9454ff Created pod: payload-pre-processing-e2e-shared-a-w1-af260f-7cfc9454ff-9c5lq 33m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w1-af260f-7cfc9454ff Error creating: pods "payload-pre-processing-e2e-shared-a-w1-af260f-7cfc9454ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-af260f: serviceaccount "payload-processing-e2e-shared-a-w1-af260f" not found 33m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w1-af260f-fd896cf88 Created pod: payload-processing-e2e-shared-a-w1-af260f-fd896cf88-rb5bz 33m Normal Started pod/payload-pre-processing-e2e-shared-a-w1-af260f-7cfc9454ff-9c5lq Container started 33m Normal AddedInterface pod/payload-processing-e2e-shared-a-w1-af260f-fd896cf88-rb5bz Add eth0 [10.134.0.183/23] from ovn-kubernetes 33m Normal Started pod/payload-processing-e2e-shared-a-w1-af260f-fd896cf88-rb5bz Container started 33m Normal Created pod/payload-pre-processing-e2e-shared-a-w1-af260f-7cfc9454ff-9c5lq Container created 33m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w1-af260f-7cfc9454ff-9c5lq Add eth0 [10.134.0.182/23] from ovn-kubernetes 33m Normal Created pod/payload-processing-e2e-shared-a-w1-af260f-fd896cf88-rb5bz Container created 33m Normal Pulled pod/payload-processing-e2e-shared-a-w1-af260f-fd896cf88-rb5bz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w1-af260f-7cfc9454ff-9c5lq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-2ba10a-openshift-default New size: 3; reason: All metrics below target 32m Normal Killing pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-rppfp Stopping container istio-proxy 32m Normal Killing pod/payload-processing-e2e-inf-a-c1750b-85f79d758d-n8sxl Stopping container payload-processing 32m Normal Killing pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-wsjz8 Stopping container istio-proxy 32m Normal ScalingReplicaSet deployment/e2e-worker-w5-22d634-openshift-default Scaled down replica set e2e-worker-w5-22d634-openshift-default-89dbbf78f from 4 to 3 32m Normal Killing pod/payload-pre-processing-e2e-inf-a-c1750b-65b7cbb459-gt6wn Stopping container payload-pre-processing 32m Normal SuccessfulDelete replicaset/e2e-worker-w5-22d634-openshift-default-89dbbf78f Deleted pod: e2e-worker-w5-22d634-openshift-default-89dbbf78f-wsjz8 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-22d634-openshift-default New size: 3; reason: All metrics below target 32m Normal SuccessfulDelete replicaset/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 Deleted pod: e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-rppfp 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-5a69c9-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 32m Normal SuccessfulCreate replicaset/e2e-inf-a-5a69c9-openshift-default-cbff5c97 Created pod: e2e-inf-a-5a69c9-openshift-default-cbff5c97-dkq5d 32m Normal ScalingReplicaSet deployment/e2e-inf-a-5a69c9-openshift-default Scaled up replica set e2e-inf-a-5a69c9-openshift-default-cbff5c97 from 1 to 2 32m Normal Created pod/e2e-inf-a-5a69c9-openshift-default-cbff5c97-dkq5d Container created 32m Normal Pulled pod/e2e-inf-a-5a69c9-openshift-default-cbff5c97-dkq5d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Started pod/e2e-inf-a-5a69c9-openshift-default-cbff5c97-dkq5d Container started 32m Normal AddedInterface pod/e2e-inf-a-5a69c9-openshift-default-cbff5c97-dkq5d Add eth0 [10.132.0.255/23] from ovn-kubernetes 32m Warning ProbeError pod/e2e-inf-a-5a69c9-openshift-default-cbff5c97-dkq5d Startup probe error: Get "http://10.132.0.255:15021/healthz/ready": dial tcp 10.132.0.255:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-inf-a-5a69c9-openshift-default-cbff5c97-dkq5d Startup probe failed: Get "http://10.132.0.255:15021/healthz/ready": dial tcp 10.132.0.255:15021: connect: connection refused 32m Normal SuccessfulDelete replicaset/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc Deleted pod: e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-fqhkm 32m Normal Killing pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-fqhkm Stopping container istio-proxy 32m Normal ScalingReplicaSet deployment/e2e-worker-w3-43b37d-openshift-default Scaled down replica set e2e-worker-w3-43b37d-openshift-default-5dcf864fcc from 5 to 4 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-43b37d-openshift-default New size: 4; reason: All metrics below target 32m Normal Killing pod/e2e-inf-a-c1750b-openshift-default-85d9759655-956xt Stopping container istio-proxy 32m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-c1750b-65b7cbb459-gt6wn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Warning Unhealthy pod/payload-processing-e2e-inf-a-c1750b-85f79d758d-n8sxl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Normal Killing pod/payload-pre-processing-e2e-inf-b-8d985a-6b59cc474c-5vth4 Stopping container payload-pre-processing 32m Normal Killing pod/payload-processing-e2e-inf-b-8d985a-78b6b6596-5tj4s Stopping container payload-processing 32m Normal ScalingReplicaSet deployment/e2e-inf-a-a5af18-openshift-default Scaled up replica set e2e-inf-a-a5af18-openshift-default-58d79f487f from 1 to 2 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-a5af18-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 32m Normal SuccessfulCreate replicaset/e2e-inf-a-a5af18-openshift-default-58d79f487f Created pod: e2e-inf-a-a5af18-openshift-default-58d79f487f-7zgh9 32m Normal Created pod/e2e-inf-a-a5af18-openshift-default-58d79f487f-7zgh9 Container created 32m Normal AddedInterface pod/e2e-inf-a-a5af18-openshift-default-58d79f487f-7zgh9 Add eth0 [10.134.0.184/23] from ovn-kubernetes 32m Normal Pulled pod/e2e-inf-a-a5af18-openshift-default-58d79f487f-7zgh9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Started pod/e2e-inf-a-a5af18-openshift-default-58d79f487f-7zgh9 Container started 32m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-8d985a-6b59cc474c-5vth4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Warning Unhealthy pod/payload-processing-e2e-inf-b-8d985a-78b6b6596-5tj4s Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Warning ProbeError pod/e2e-inf-a-a5af18-openshift-default-58d79f487f-7zgh9 Startup probe error: Get "http://10.134.0.184:15021/healthz/ready": dial tcp 10.134.0.184:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-inf-a-a5af18-openshift-default-58d79f487f-7zgh9 Startup probe failed: Get "http://10.134.0.184:15021/healthz/ready": dial tcp 10.134.0.184:15021: connect: connection refused 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-6fc6bd-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 31m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89 Created pod: e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-2dhcj 31m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-6fc6bd-openshift-default Scaled up replica set e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89 from 1 to 2 31m Normal AddedInterface pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-2dhcj Add eth0 [10.132.1.2/23] from ovn-kubernetes 31m Normal Pulled pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-2dhcj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Created pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-2dhcj Container created 31m Normal Started pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-2dhcj Container started 31m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-d68c2c-openshift-default Scaled up replica set e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f from 0 to 1 31m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f Created pod: e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-6xvfq 31m Normal AddedLabel gateway/e2e-shared-b-w1-d68c2c Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w1-d68c2c 31m Normal AddedInterface pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-6xvfq Add eth0 [10.134.0.185/23] from ovn-kubernetes 31m Warning ProbeError pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-2dhcj Startup probe error: Get "http://10.132.1.2:15021/healthz/ready": dial tcp 10.132.1.2:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-2dhcj Startup probe failed: Get "http://10.132.1.2:15021/healthz/ready": dial tcp 10.132.1.2:15021: connect: connection refused 31m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w1-d68c2c-7d66774ccf Error creating: pods "payload-pre-processing-e2e-shared-b-w1-d68c2c-7d66774ccf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-d68c2c: serviceaccount "payload-processing-e2e-shared-b-w1-d68c2c" not found 31m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w1-d68c2c Scaled up replica set payload-processing-e2e-shared-b-w1-d68c2c-5c949bdf9b from 0 to 1 31m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w1-d68c2c Scaled up replica set payload-pre-processing-e2e-shared-b-w1-d68c2c-7d66774ccf from 0 to 1 31m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w1-d68c2c-5c949bdf9b Error creating: pods "payload-processing-e2e-shared-b-w1-d68c2c-5c949bdf9b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-d68c2c: serviceaccount "payload-processing-e2e-shared-b-w1-d68c2c" not found 31m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w1-d68c2c-5c949bdf9b Created pod: payload-processing-e2e-shared-b-w1-d68c2c-5c949bdf9b-52x5q 31m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w1-d68c2c-7d66774ccf Created pod: payload-pre-processing-e2e-shared-b-w1-d68c2c-7d66774ccf-wvf5r 31m Normal AddedInterface pod/payload-processing-e2e-shared-b-w1-d68c2c-5c949bdf9b-52x5q Add eth0 [10.134.0.187/23] from ovn-kubernetes 31m Normal Created pod/payload-pre-processing-e2e-shared-b-w1-d68c2c-7d66774ccf-wvf5r Container created 31m Normal Started pod/payload-pre-processing-e2e-shared-b-w1-d68c2c-7d66774ccf-wvf5r Container started 31m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w1-d68c2c-7d66774ccf-wvf5r Add eth0 [10.134.0.186/23] from ovn-kubernetes 31m Normal Started pod/payload-processing-e2e-shared-b-w1-d68c2c-5c949bdf9b-52x5q Container started 31m Normal Created pod/payload-processing-e2e-shared-b-w1-d68c2c-5c949bdf9b-52x5q Container created 31m Normal Pulled pod/payload-processing-e2e-shared-b-w1-d68c2c-5c949bdf9b-52x5q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w1-d68c2c-7d66774ccf-wvf5r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal Killing pod/e2e-inf-b-8d985a-openshift-default-856f5f5bf5-w2k2c Stopping container istio-proxy 31m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65 Created pod: e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-mb2dl 31m Normal SuccessfulDelete replicaset/e2e-worker-w6-784e7a-openshift-default-595898bd5c Deleted pod: e2e-worker-w6-784e7a-openshift-default-595898bd5c-8cvvq 31m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-f8ca96-openshift-default Scaled up replica set e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65 from 0 to 1 31m Normal AddedLabel gateway/e2e-shared-a-w4-f8ca96 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-f8ca96 31m Normal Killing pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-8cvvq Stopping container istio-proxy 31m Normal SuccessfulDelete replicaset/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 Deleted pod: e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-nf9qq 31m Normal Killing pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-lqhnf Stopping container istio-proxy 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-08ebee-openshift-default New size: 3; reason: All metrics below target 31m Normal Killing pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-nf9qq Stopping container istio-proxy 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2e046a-openshift-default New size: 3; reason: All metrics below target 31m Normal ScalingReplicaSet deployment/e2e-worker-w0-2e046a-openshift-default Scaled down replica set e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 from 4 to 3 31m Normal SuccessfulDelete replicaset/e2e-worker-w4-08ebee-openshift-default-5f5bf46967 Deleted pod: e2e-worker-w4-08ebee-openshift-default-5f5bf46967-lqhnf 31m Normal AddedInterface pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-mb2dl Add eth0 [10.134.0.188/23] from ovn-kubernetes 31m Warning ProbeError pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-nf9qq Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning Unhealthy pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-nf9qq Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w4-f8ca96-696c58bc6c Error creating: pods "payload-pre-processing-e2e-shared-a-w4-f8ca96-696c58bc6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-f8ca96: serviceaccount "payload-processing-e2e-shared-a-w4-f8ca96" not found 31m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w4-f8ca96-576b496c86 Error creating: pods "payload-processing-e2e-shared-a-w4-f8ca96-576b496c86-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-f8ca96: serviceaccount "payload-processing-e2e-shared-a-w4-f8ca96" not found 31m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w4-f8ca96 Scaled up replica set payload-pre-processing-e2e-shared-a-w4-f8ca96-696c58bc6c from 0 to 1 31m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w4-f8ca96-576b496c86 Created pod: payload-processing-e2e-shared-a-w4-f8ca96-576b496c86-rkx2m 31m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w4-f8ca96 Scaled up replica set payload-processing-e2e-shared-a-w4-f8ca96-576b496c86 from 0 to 1 31m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w4-f8ca96-696c58bc6c Created pod: payload-pre-processing-e2e-shared-a-w4-f8ca96-696c58bc6c-4dw72 31m Normal Pulled pod/payload-processing-e2e-shared-a-w4-f8ca96-576b496c86-rkx2m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5db77fbd9f Deleted pod: maas-default-gateway-openshift-default-5db77fbd9f-82d8r 31m Normal Created pod/payload-pre-processing-e2e-shared-a-w4-f8ca96-696c58bc6c-4dw72 Container created 31m Normal Killing pod/maas-default-gateway-openshift-default-5db77fbd9f-xvdlm Stopping container istio-proxy 31m Normal Started pod/payload-pre-processing-e2e-shared-a-w4-f8ca96-696c58bc6c-4dw72 Container started 31m Normal Killing pod/maas-default-gateway-openshift-default-5db77fbd9f-82d8r Stopping container istio-proxy 31m Normal Started pod/payload-processing-e2e-shared-a-w4-f8ca96-576b496c86-rkx2m Container started 31m Normal Created pod/payload-processing-e2e-shared-a-w4-f8ca96-576b496c86-rkx2m Container created 31m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5db77fbd9f Deleted pod: maas-default-gateway-openshift-default-5db77fbd9f-xvdlm 31m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: All metrics below target 31m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w4-f8ca96-696c58bc6c-4dw72 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/payload-processing-e2e-shared-a-w4-f8ca96-576b496c86-rkx2m Add eth0 [10.132.1.4/23] from ovn-kubernetes 31m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w4-f8ca96-696c58bc6c-4dw72 Add eth0 [10.132.1.5/23] from ovn-kubernetes 31m Normal Killing pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-ghbxb Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-89tnt Stopping container istio-proxy 31m Normal SuccessfulDelete replicaset/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc Deleted pod: e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-89tnt 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-43b37d-openshift-default New size: 3; reason: All metrics below target 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-f8ca96-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-fc684b-openshift-default New size: 3; reason: All metrics below target 31m Normal ScalingReplicaSet deployment/e2e-worker-w1-fc684b-openshift-default Scaled down replica set e2e-worker-w1-fc684b-openshift-default-7fcf678997 from 4 to 3 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-f8ca96-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Normal SuccessfulDelete replicaset/e2e-worker-w1-fc684b-openshift-default-7fcf678997 Deleted pod: e2e-worker-w1-fc684b-openshift-default-7fcf678997-ghbxb 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-d68c2c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-d68c2c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Normal Killing pod/payload-processing-e2e-inf-a-fee294-79797dc7c4-4q6tx Stopping container payload-processing 31m Normal Killing pod/payload-pre-processing-e2e-inf-a-fee294-c499455c5-ds72t Stopping container payload-pre-processing 31m Normal ScalingReplicaSet deployment/e2e-inf-b-2a3ef0-openshift-default Scaled up replica set e2e-inf-b-2a3ef0-openshift-default-5cdf89c9 from 1 to 2 31m Normal SuccessfulCreate replicaset/e2e-inf-b-2a3ef0-openshift-default-5cdf89c9 Created pod: e2e-inf-b-2a3ef0-openshift-default-5cdf89c9-rvbgs 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-2a3ef0-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 31m Normal AddedInterface pod/e2e-inf-b-2a3ef0-openshift-default-5cdf89c9-rvbgs Add eth0 [10.132.1.10/23] from ovn-kubernetes 31m Normal Pulled pod/e2e-inf-b-2a3ef0-openshift-default-5cdf89c9-rvbgs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Created pod/e2e-inf-b-2a3ef0-openshift-default-5cdf89c9-rvbgs Container created 31m Normal Started pod/e2e-inf-b-2a3ef0-openshift-default-5cdf89c9-rvbgs Container started 31m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-267vm Stopping container discovery 31m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-267vm 31m Warning ProbeError pod/e2e-inf-b-2a3ef0-openshift-default-5cdf89c9-rvbgs Startup probe error: Get "http://10.132.1.10:15021/healthz/ready": dial tcp 10.132.1.10:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-inf-b-2a3ef0-openshift-default-5cdf89c9-rvbgs Startup probe failed: Get "http://10.132.1.10:15021/healthz/ready": dial tcp 10.132.1.10:15021: connect: connection refused 30m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-fee294-c499455c5-ds72t Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Warning Unhealthy pod/payload-processing-e2e-inf-a-fee294-79797dc7c4-4q6tx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Normal Killing pod/payload-processing-e2e-inf-a-a5af18-5786f6f7f6-v8znz Stopping container payload-processing 30m Normal Killing pod/payload-pre-processing-e2e-inf-a-a5af18-84d9b694d7-lzhcq Stopping container payload-pre-processing 30m Normal Killing pod/e2e-inf-a-fee294-openshift-default-644cdf5d45-4dbqs Stopping container istio-proxy 30m Normal Killing pod/e2e-inf-a-fee294-openshift-default-644cdf5d45-cf2vc Stopping container istio-proxy 30m Normal Killing pod/payload-pre-processing-e2e-inf-a-5a69c9-95cc9866c-bsqgg Stopping container payload-pre-processing 30m Normal Killing pod/payload-processing-e2e-inf-a-5a69c9-7779f4d998-xrblq Stopping container payload-processing 30m Normal Killing pod/payload-pre-processing-e2e-inf-a-c7d192-fcf758f79-fd7hp Stopping container payload-pre-processing 30m Normal Killing pod/payload-processing-e2e-inf-a-c7d192-7f56447bb4-wq8sm Stopping container payload-processing 30m Normal Killing pod/payload-pre-processing-e2e-inf-b-8147a8-7f7ff5cc5-4fxdk Stopping container payload-pre-processing 30m Normal Killing pod/payload-processing-e2e-inf-b-8147a8-f8bf948b4-575w2 Stopping container payload-processing 30m Normal Created pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-wghsp Container created 30m Normal Pulled pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-wghsp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-wghsp Add eth0 [10.134.0.189/23] from ovn-kubernetes 30m Normal Created pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-59rv7 Container created 30m Normal Started pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-wghsp Container started 30m Normal SuccessfulCreate replicaset/e2e-inf-b-e619b6-openshift-default-6d4c644df9 Created pod: e2e-inf-b-e619b6-openshift-default-6d4c644df9-59rv7 30m Normal Started pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-59rv7 Container started 30m Normal SuccessfulCreate replicaset/e2e-inf-b-e619b6-openshift-default-6d4c644df9 Created pod: e2e-inf-b-e619b6-openshift-default-6d4c644df9-wghsp 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-e619b6-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 30m Normal ScalingReplicaSet deployment/e2e-inf-b-e619b6-openshift-default Scaled up replica set e2e-inf-b-e619b6-openshift-default-6d4c644df9 from 2 to 4 30m Normal AddedInterface pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-59rv7 Add eth0 [10.133.0.108/23] from ovn-kubernetes 30m Normal Pulled pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-59rv7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Killing pod/e2e-inf-a-a5af18-openshift-default-58d79f487f-tw782 Stopping container istio-proxy 30m Normal Killing pod/e2e-inf-a-a5af18-openshift-default-58d79f487f-7zgh9 Stopping container istio-proxy 30m Normal Killing pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-5gpj4 Stopping container istio-proxy 30m Warning ProbeError pod/e2e-inf-a-a5af18-openshift-default-58d79f487f-7zgh9 Readiness probe error: HTTP probe failed with statuscode: 503... 30m Normal Killing pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-xw24m Stopping container istio-proxy 30m Normal Killing pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-rgzsj Stopping container istio-proxy 30m Warning Unhealthy pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-59rv7 Startup probe failed: Get "http://10.133.0.108:15021/healthz/ready": dial tcp 10.133.0.108:15021: connect: connection refused 30m Normal Killing pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-kl464 Stopping container istio-proxy 30m Normal Killing pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-cjl27 Stopping container istio-proxy 30m Normal Killing pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-nbz7s Stopping container istio-proxy 30m Normal Killing pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-4krkx Stopping container istio-proxy 30m Warning Unhealthy pod/e2e-inf-a-a5af18-openshift-default-58d79f487f-7zgh9 Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning ProbeError pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-59rv7 Startup probe error: Get "http://10.133.0.108:15021/healthz/ready": dial tcp 10.133.0.108:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-wghsp Startup probe failed: Get "http://10.134.0.189:15021/healthz/ready": dial tcp 10.134.0.189:15021: connect: connection refused 30m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-5a69c9-95cc9866c-bsqgg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Warning ProbeError pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-wghsp Startup probe error: Get "http://10.134.0.189:15021/healthz/ready": dial tcp 10.134.0.189:15021: connect: connection refused... 30m Warning ProbeError pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-nbz7s Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning Unhealthy pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-nbz7s Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning ProbeError pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-4krkx Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-a5af18-84d9b694d7-lzhcq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Warning Unhealthy pod/payload-processing-e2e-inf-a-a5af18-5786f6f7f6-v8znz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Warning Unhealthy pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-kl464 Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning Unhealthy pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-4krkx Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning ProbeError pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-cjl27 Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning Unhealthy pod/payload-processing-e2e-inf-a-5a69c9-7779f4d998-xrblq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Warning ProbeError pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-kl464 Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning Unhealthy pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-cjl27 Readiness probe failed: HTTP probe failed with statuscode: 503 30m Normal Created pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-6xvfq Container created 30m Warning ProbeError pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-5gpj4 Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning Unhealthy pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-xw24m Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning Unhealthy pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-5gpj4 Readiness probe failed: HTTP probe failed with statuscode: 503 30m Normal Started pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-6xvfq Container started 30m Normal Pulled pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-6xvfq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Warning ProbeError pod/e2e-inf-a-c7d192-openshift-default-c4f554bb7-xw24m Readiness probe error: HTTP probe failed with statuscode: 503... 30m Normal Killing pod/payload-pre-processing-e2e-inf-b-e619b6-868f869f9f-j8pwm Stopping container payload-pre-processing 30m Normal Killing pod/payload-processing-e2e-inf-b-e619b6-7f44c56884-ctrpq Stopping container payload-processing 30m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-c7d192-fcf758f79-fd7hp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-8147a8-7f7ff5cc5-4fxdk Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Warning Unhealthy pod/payload-processing-e2e-inf-b-8147a8-f8bf948b4-575w2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Normal AddedLabel gateway/e2e-shared-b-w4-c933cb Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-c933cb 30m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-c933cb-openshift-default Scaled up replica set e2e-shared-b-w4-c933cb-openshift-default-9986db7d7 from 0 to 1 30m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7 Created pod: e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-2kr84 30m Normal Killing pod/payload-pre-processing-e2e-inf-b-2a3ef0-8648759887-mhb5v Stopping container payload-pre-processing 30m Normal Killing pod/payload-processing-e2e-inf-b-2a3ef0-9d775496-fs4gv Stopping container payload-processing 30m Normal AddedInterface pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-2kr84 Add eth0 [10.134.0.190/23] from ovn-kubernetes 29m Warning ProbeError pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-6xvfq Startup probe error: Get "http://10.134.0.185:15021/healthz/ready": dial tcp 10.134.0.185:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-6xvfq Startup probe failed: Get "http://10.134.0.185:15021/healthz/ready": dial tcp 10.134.0.185:15021: connect: connection refused 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w4-c933cb-786d7ff8b7 Error creating: pods "payload-pre-processing-e2e-shared-b-w4-c933cb-786d7ff8b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-c933cb: serviceaccount "payload-processing-e2e-shared-b-w4-c933cb" not found 29m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w4-c933cb-fc7cf9456 Error creating: pods "payload-processing-e2e-shared-b-w4-c933cb-fc7cf9456-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-c933cb: serviceaccount "payload-processing-e2e-shared-b-w4-c933cb" not found 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w4-c933cb-fc7cf9456 Created pod: payload-processing-e2e-shared-b-w4-c933cb-fc7cf9456-g8t8z 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w4-c933cb Scaled up replica set payload-processing-e2e-shared-b-w4-c933cb-fc7cf9456 from 0 to 1 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w4-c933cb-786d7ff8b7 Created pod: payload-pre-processing-e2e-shared-b-w4-c933cb-786d7ff8b7-zhqxg 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w4-c933cb Scaled up replica set payload-pre-processing-e2e-shared-b-w4-c933cb-786d7ff8b7 from 0 to 1 29m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w4-c933cb-786d7ff8b7-zhqxg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/payload-processing-e2e-shared-b-w4-c933cb-fc7cf9456-g8t8z Add eth0 [10.134.0.192/23] from ovn-kubernetes 29m Normal Pulled pod/payload-processing-e2e-shared-b-w4-c933cb-fc7cf9456-g8t8z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal Started pod/payload-processing-e2e-shared-b-w4-c933cb-fc7cf9456-g8t8z Container started 29m Normal Started pod/payload-pre-processing-e2e-shared-b-w4-c933cb-786d7ff8b7-zhqxg Container started 29m Normal Created pod/payload-processing-e2e-shared-b-w4-c933cb-fc7cf9456-g8t8z Container created 29m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w4-c933cb-786d7ff8b7-zhqxg Add eth0 [10.134.0.191/23] from ovn-kubernetes 29m Normal Created pod/payload-pre-processing-e2e-shared-b-w4-c933cb-786d7ff8b7-zhqxg Container created 29m Normal Killing pod/e2e-inf-a-5a69c9-openshift-default-cbff5c97-dkq5d Stopping container istio-proxy 29m Normal Killing pod/e2e-inf-a-5a69c9-openshift-default-cbff5c97-bfffh Stopping container istio-proxy 29m Normal Killing pod/payload-pre-processing-e2e-inf-b-05417d-5cbcfc559-jjksv Stopping container payload-pre-processing 29m Normal Killing pod/payload-processing-e2e-inf-b-05417d-754c7b55dd-kclz4 Stopping container payload-processing 29m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-2a3ef0-8648759887-mhb5v Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-e619b6-868f869f9f-j8pwm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Normal Killing pod/e2e-inf-b-8147a8-openshift-default-855c99786c-k9ckd Stopping container istio-proxy 29m Normal Killing pod/e2e-inf-b-8147a8-openshift-default-855c99786c-4sv5h Stopping container istio-proxy 29m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-6fc6bd-openshift-default Scaled up replica set e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89 from 2 to 3 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-6fc6bd-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 29m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89 Created pod: e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-qpsqc 29m Warning ProbeError pod/e2e-inf-b-8147a8-openshift-default-855c99786c-k9ckd Readiness probe error: HTTP probe failed with statuscode: 503... 29m Warning Unhealthy pod/e2e-inf-b-8147a8-openshift-default-855c99786c-4sv5h Readiness probe failed: HTTP probe failed with statuscode: 503 29m Warning Unhealthy pod/e2e-inf-b-8147a8-openshift-default-855c99786c-k9ckd Readiness probe failed: HTTP probe failed with statuscode: 503 29m Normal Started pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-qpsqc Container started 29m Normal Created pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-qpsqc Container created 29m Normal Pulled pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-qpsqc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-qpsqc Add eth0 [10.133.0.111/23] from ovn-kubernetes 29m Warning ProbeError pod/e2e-inf-b-8147a8-openshift-default-855c99786c-4sv5h Readiness probe error: HTTP probe failed with statuscode: 503... 29m Normal Killing pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-wghsp Stopping container istio-proxy 29m Normal Killing pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-ml8sr Stopping container istio-proxy 29m Normal Killing pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-59rv7 Stopping container istio-proxy 29m Normal Killing pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-kv7vp Stopping container istio-proxy 29m Normal AddedInterface pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-q8qxp Add eth0 [10.134.0.193/23] from ovn-kubernetes 29m Normal AddedLabel gateway/e2e-shared-a-w0-920032 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-920032 29m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-920032-openshift-default Scaled up replica set e2e-shared-a-w0-920032-openshift-default-5b5f9cc67 from 0 to 1 29m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67 Created pod: e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-q8qxp 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-d68c2c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-d68c2c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Warning Unhealthy pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-kv7vp Readiness probe failed: HTTP probe failed with statuscode: 503 29m Warning Unhealthy pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-qpsqc Startup probe failed: Get "http://10.133.0.111:15021/healthz/ready": dial tcp 10.133.0.111:15021: connect: connection refused 29m Warning ProbeError pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-qpsqc Startup probe error: Get "http://10.133.0.111:15021/healthz/ready": dial tcp 10.133.0.111:15021: connect: connection refused... 29m Warning ProbeError pod/e2e-inf-b-e619b6-openshift-default-6d4c644df9-kv7vp Readiness probe error: HTTP probe failed with statuscode: 503... 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w0-920032-85fc75f8c5 Created pod: payload-pre-processing-e2e-shared-a-w0-920032-85fc75f8c5-fsg62 29m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w0-920032-77dc47dc96 Error creating: pods "payload-processing-e2e-shared-a-w0-920032-77dc47dc96-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-920032: serviceaccount "payload-processing-e2e-shared-a-w0-920032" not found 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w0-920032-77dc47dc96 Created pod: payload-processing-e2e-shared-a-w0-920032-77dc47dc96-xh765 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w0-920032 Scaled up replica set payload-pre-processing-e2e-shared-a-w0-920032-85fc75f8c5 from 0 to 1 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w0-920032-85fc75f8c5 Error creating: pods "payload-pre-processing-e2e-shared-a-w0-920032-85fc75f8c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-920032: serviceaccount "payload-processing-e2e-shared-a-w0-920032" not found 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w0-920032 Scaled up replica set payload-processing-e2e-shared-a-w0-920032-77dc47dc96 from 0 to 1 29m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w0-920032-85fc75f8c5-fsg62 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/payload-processing-e2e-shared-a-w0-920032-77dc47dc96-xh765 Add eth0 [10.134.0.195/23] from ovn-kubernetes 29m Normal Killing pod/e2e-inf-b-2a3ef0-openshift-default-5cdf89c9-rvbgs Stopping container istio-proxy 29m Normal Killing pod/e2e-inf-b-2a3ef0-openshift-default-5cdf89c9-fwcr6 Stopping container istio-proxy 29m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w0-920032-85fc75f8c5-fsg62 Add eth0 [10.134.0.194/23] from ovn-kubernetes 29m Normal Started pod/payload-pre-processing-e2e-shared-a-w0-920032-85fc75f8c5-fsg62 Container started 29m Normal Created pod/payload-pre-processing-e2e-shared-a-w0-920032-85fc75f8c5-fsg62 Container created 29m Normal Pulled pod/payload-processing-e2e-shared-a-w0-920032-77dc47dc96-xh765 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal Created pod/payload-processing-e2e-shared-a-w0-920032-77dc47dc96-xh765 Container created 29m Normal Started pod/payload-processing-e2e-shared-a-w0-920032-77dc47dc96-xh765 Container started 29m Warning ProbeError pod/e2e-inf-b-2a3ef0-openshift-default-5cdf89c9-fwcr6 Readiness probe error: HTTP probe failed with statuscode: 503... 29m Warning Unhealthy pod/e2e-inf-b-2a3ef0-openshift-default-5cdf89c9-fwcr6 Readiness probe failed: HTTP probe failed with statuscode: 503 29m Normal AddedLabel gateway/e2e-shared-a-w2-eb02df Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-eb02df 29m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c Created pod: e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-mshrv 29m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-05417d-5cbcfc559-jjksv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Normal AddedInterface pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-mshrv Add eth0 [10.134.0.196/23] from ovn-kubernetes 29m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-eb02df-openshift-default Scaled up replica set e2e-shared-a-w2-eb02df-openshift-default-5c45c545c from 0 to 1 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-eb02df-54d8869579 Created pod: payload-pre-processing-e2e-shared-a-w2-eb02df-54d8869579-kckzx 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-eb02df Scaled up replica set payload-pre-processing-e2e-shared-a-w2-eb02df-54d8869579 from 0 to 1 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-eb02df Scaled up replica set payload-processing-e2e-shared-a-w2-eb02df-7b6dfb5df8 from 0 to 1 29m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-eb02df-7b6dfb5df8 Error creating: pods "payload-processing-e2e-shared-a-w2-eb02df-7b6dfb5df8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-eb02df: serviceaccount "payload-processing-e2e-shared-a-w2-eb02df" not found 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-eb02df-54d8869579 Error creating: pods "payload-pre-processing-e2e-shared-a-w2-eb02df-54d8869579-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-eb02df: serviceaccount "payload-processing-e2e-shared-a-w2-eb02df" not found 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-eb02df-7b6dfb5df8 Created pod: payload-processing-e2e-shared-a-w2-eb02df-7b6dfb5df8-sw5mn 29m Normal Started pod/payload-processing-e2e-shared-a-w2-eb02df-7b6dfb5df8-sw5mn Container started 29m Normal Pulled pod/payload-processing-e2e-shared-a-w2-eb02df-7b6dfb5df8-sw5mn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-eb02df-7b6dfb5df8-sw5mn Add eth0 [10.134.0.198/23] from ovn-kubernetes 29m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-eb02df-54d8869579-kckzx Add eth0 [10.134.0.197/23] from ovn-kubernetes 29m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-eb02df-54d8869579-kckzx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-eb02df-54d8869579-kckzx Container started 29m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-eb02df-54d8869579-kckzx Container created 29m Normal Created pod/payload-processing-e2e-shared-a-w2-eb02df-7b6dfb5df8-sw5mn Container created 29m Normal Killing pod/e2e-inf-b-05417d-openshift-default-6d96dbcbc5-nq8jg Stopping container istio-proxy 29m Warning Unhealthy pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-cxjmw Readiness probe failed: Get "http://10.132.0.252:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 29m Warning ProbeError pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-cxjmw Readiness probe error: Get "http://10.132.0.252:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 29m Normal Started pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-cxjmw Container started 29m Normal Created pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-cxjmw Container created 29m Normal Pulled pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-cxjmw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Warning Unhealthy pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-cxjmw Startup probe failed: Get "http://10.132.0.252:15021/healthz/ready": dial tcp 10.132.0.252:15021: connect: connection refused 29m Warning ProbeError pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-cxjmw Startup probe error: Get "http://10.132.0.252:15021/healthz/ready": dial tcp 10.132.0.252:15021: connect: connection refused... 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-af260f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-af260f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85 Created pod: e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-r9lfc 28m Normal Created pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-r9lfc Container created 28m Normal AddedInterface pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-r9lfc Add eth0 [10.134.0.199/23] from ovn-kubernetes 28m Normal Pulled pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-r9lfc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Started pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-r9lfc Container started 28m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-9aafe6-openshift-default Scaled up replica set e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85 from 1 to 2 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w5-9aafe6-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-aaa3c6-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 28m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9 Created pod: e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9-x9gf6 28m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-aaa3c6-openshift-default Scaled up replica set e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9 from 1 to 2 28m Normal Created pod/e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9-x9gf6 Container created 28m Normal AddedInterface pod/e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9-x9gf6 Add eth0 [10.132.1.26/23] from ovn-kubernetes 28m Normal Pulled pod/e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9-x9gf6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Started pod/e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9-x9gf6 Container started 28m Warning ProbeError pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-r9lfc Startup probe error: Get "http://10.134.0.199:15021/healthz/ready": dial tcp 10.134.0.199:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-r9lfc Startup probe failed: Get "http://10.134.0.199:15021/healthz/ready": dial tcp 10.134.0.199:15021: connect: connection refused 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-08ebee-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 28m Normal Killing pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-h7dlg Stopping container istio-proxy 28m Normal Killing pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-v56n2 Stopping container istio-proxy 28m Warning Unhealthy pod/e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9-x9gf6 Startup probe failed: Get "http://10.132.1.26:15021/healthz/ready": dial tcp 10.132.1.26:15021: connect: connection refused 28m Warning ProbeError pod/e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9-x9gf6 Startup probe error: Get "http://10.132.1.26:15021/healthz/ready": dial tcp 10.132.1.26:15021: connect: connection refused... 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-fc684b-openshift-default New size: 2; reason: All metrics below target 28m Normal ScalingReplicaSet deployment/e2e-worker-w1-fc684b-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w1-fc684b-openshift-default-7fcf678997 from 3 to 2 28m Normal ScalingReplicaSet deployment/e2e-worker-w0-2e046a-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 from 3 to 2 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-2e046a-openshift-default New size: 2; reason: All metrics below target 28m Normal ScalingReplicaSet deployment/e2e-worker-w4-08ebee-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w4-08ebee-openshift-default-5f5bf46967 from 3 to 2 28m Normal SuccessfulDelete replicaset/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49 Deleted pod: e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-tczdj 28m Normal SuccessfulDelete replicaset/e2e-worker-w1-fc684b-openshift-default-7fcf678997 Deleted pod: e2e-worker-w1-fc684b-openshift-default-7fcf678997-v56n2 28m Normal Killing pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-tczdj Stopping container istio-proxy 28m Normal SuccessfulDelete replicaset/e2e-worker-w4-08ebee-openshift-default-5f5bf46967 Deleted pod: e2e-worker-w4-08ebee-openshift-default-5f5bf46967-h7dlg 28m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7 Created pod: e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-55kkj 28m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-c933cb-openshift-default Scaled up replica set e2e-shared-b-w4-c933cb-openshift-default-9986db7d7 from 1 to 3 28m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7 Created pod: e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-jtp6w 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-c933cb-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 28m Normal Started pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-2kr84 Container started 28m Normal Started pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-jtp6w Container started 28m Normal Pulled pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-2kr84 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Created pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-jtp6w Container created 28m Normal AddedInterface pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-jtp6w Add eth0 [10.132.1.27/23] from ovn-kubernetes 28m Normal Started pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-55kkj Container started 28m Normal Created pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-55kkj Container created 28m Normal Pulled pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-55kkj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-55kkj Add eth0 [10.133.0.116/23] from ovn-kubernetes 28m Normal Created pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-2kr84 Container created 28m Normal Pulled pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-jtp6w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-af260f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-af260f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: All metrics below target 28m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5db77fbd9f Deleted pod: maas-default-gateway-openshift-default-5db77fbd9f-2qx7m 28m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-5db77fbd9f from 5 to 4 28m Normal Killing pod/maas-default-gateway-openshift-default-5db77fbd9f-2qx7m Stopping container istio-proxy 28m Warning ProbeError pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-jtp6w Startup probe error: Get "http://10.132.1.27:15021/healthz/ready": dial tcp 10.132.1.27:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-jtp6w Startup probe failed: Get "http://10.132.1.27:15021/healthz/ready": dial tcp 10.132.1.27:15021: connect: connection refused 28m Warning ProbeError pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-55kkj Startup probe error: Get "http://10.133.0.116:15021/healthz/ready": dial tcp 10.133.0.116:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-55kkj Startup probe failed: Get "http://10.133.0.116:15021/healthz/ready": dial tcp 10.133.0.116:15021: connect: connection refused 28m Warning Unhealthy pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-2kr84 Startup probe failed: Get "http://10.134.0.190:15021/healthz/ready": dial tcp 10.134.0.190:15021: connect: connection refused 28m Warning ProbeError pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-2kr84 Startup probe error: Get "http://10.134.0.190:15021/healthz/ready": dial tcp 10.134.0.190:15021: connect: connection refused... 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-c933cb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Normal AddedLabel gateway/e2e-shared-b-w0-02720a Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-02720a 28m Normal AddedInterface pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-kwffs Add eth0 [10.134.0.201/23] from ovn-kubernetes 28m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-02720a-openshift-default Scaled up replica set e2e-shared-b-w0-02720a-openshift-default-6b64cb69 from 0 to 1 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-c933cb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-02720a-openshift-default-6b64cb69 Created pod: e2e-shared-b-w0-02720a-openshift-default-6b64cb69-kwffs 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w0-02720a Scaled up replica set payload-processing-e2e-shared-b-w0-02720a-78cdf6c4cd from 0 to 1 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w0-02720a-54d9b599c9 Error creating: pods "payload-pre-processing-e2e-shared-b-w0-02720a-54d9b599c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-02720a: serviceaccount "payload-processing-e2e-shared-b-w0-02720a" not found 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w0-02720a-54d9b599c9 Created pod: payload-pre-processing-e2e-shared-b-w0-02720a-54d9b599c9-q5xcw 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w0-02720a Scaled up replica set payload-pre-processing-e2e-shared-b-w0-02720a-54d9b599c9 from 0 to 1 28m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w0-02720a-78cdf6c4cd Error creating: pods "payload-processing-e2e-shared-b-w0-02720a-78cdf6c4cd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-02720a: serviceaccount "payload-processing-e2e-shared-b-w0-02720a" not found 28m Normal Started pod/payload-pre-processing-e2e-shared-b-w0-02720a-54d9b599c9-q5xcw Container started 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w0-02720a-78cdf6c4cd Created pod: payload-processing-e2e-shared-b-w0-02720a-78cdf6c4cd-w4jmh 28m Normal Created pod/payload-pre-processing-e2e-shared-b-w0-02720a-54d9b599c9-q5xcw Container created 28m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w0-02720a-54d9b599c9-q5xcw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w0-02720a-54d9b599c9-q5xcw Add eth0 [10.132.1.28/23] from ovn-kubernetes 28m Normal AddedInterface pod/payload-processing-e2e-shared-b-w0-02720a-78cdf6c4cd-w4jmh Add eth0 [10.134.0.202/23] from ovn-kubernetes 28m Normal Started pod/payload-processing-e2e-shared-b-w0-02720a-78cdf6c4cd-w4jmh Container started 28m Normal Created pod/payload-processing-e2e-shared-b-w0-02720a-78cdf6c4cd-w4jmh Container created 28m Normal Pulled pod/payload-processing-e2e-shared-b-w0-02720a-78cdf6c4cd-w4jmh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Warning ProbeError pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-mshrv Readiness probe error: Get "http://10.134.0.196:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 28m Warning Unhealthy pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-mshrv Readiness probe failed: Get "http://10.134.0.196:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 28m Normal Pulled pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-mshrv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7 Created pod: e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-hjvtw 28m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-3ee017-openshift-default Scaled up replica set e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7 from 0 to 1 28m Normal Created pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-q8qxp Container created 28m Normal AddedLabel gateway/e2e-shared-b-w2-3ee017 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-3ee017 28m Normal Pulled pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-q8qxp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Created pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-mshrv Container created 28m Normal Started pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-q8qxp Container started 28m Normal Started pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-mshrv Container started 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-920032-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Normal AddedInterface pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-hjvtw Add eth0 [10.132.1.29/23] from ovn-kubernetes 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-920032-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-3ee017 Scaled up replica set payload-pre-processing-e2e-shared-b-w2-3ee017-b5cc8c7ff from 0 to 1 27m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-3ee017-5f8b7645fb Error creating: pods "payload-processing-e2e-shared-b-w2-3ee017-5f8b7645fb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-3ee017: serviceaccount "payload-processing-e2e-shared-b-w2-3ee017" not found 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-3ee017 Scaled up replica set payload-processing-e2e-shared-b-w2-3ee017-5f8b7645fb from 0 to 1 27m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-3ee017-b5cc8c7ff-nnl9p Container started 27m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-3ee017-b5cc8c7ff-nnl9p Container created 27m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-3ee017-b5cc8c7ff-nnl9p Add eth0 [10.132.1.30/23] from ovn-kubernetes 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-3ee017-b5cc8c7ff Error creating: pods "payload-pre-processing-e2e-shared-b-w2-3ee017-b5cc8c7ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-3ee017: serviceaccount "payload-processing-e2e-shared-b-w2-3ee017" not found 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-3ee017-b5cc8c7ff Created pod: payload-pre-processing-e2e-shared-b-w2-3ee017-b5cc8c7ff-nnl9p 27m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-3ee017-b5cc8c7ff-nnl9p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-3ee017-5f8b7645fb Created pod: payload-processing-e2e-shared-b-w2-3ee017-5f8b7645fb-477wq 27m Warning ProbeError pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-mshrv Startup probe error: Get "http://10.134.0.196:15021/healthz/ready": dial tcp 10.134.0.196:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-mshrv Startup probe failed: Get "http://10.134.0.196:15021/healthz/ready": dial tcp 10.134.0.196:15021: connect: connection refused 27m Normal Started pod/payload-processing-e2e-shared-b-w2-3ee017-5f8b7645fb-477wq Container started 27m Normal Created pod/payload-processing-e2e-shared-b-w2-3ee017-5f8b7645fb-477wq Container created 27m Normal Pulled pod/payload-processing-e2e-shared-b-w2-3ee017-5f8b7645fb-477wq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-3ee017-5f8b7645fb-477wq Add eth0 [10.134.0.203/23] from ovn-kubernetes 27m Warning ProbeError pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-q8qxp Startup probe error: Get "http://10.134.0.193:15021/healthz/ready": dial tcp 10.134.0.193:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-q8qxp Startup probe failed: Get "http://10.134.0.193:15021/healthz/ready": dial tcp 10.134.0.193:15021: connect: connection refused 27m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-06a95f-openshift-default-74f57588cc Created pod: e2e-shared-a-w6-06a95f-openshift-default-74f57588cc-f4bjj 27m Normal Pulled pod/e2e-shared-a-w6-06a95f-openshift-default-74f57588cc-f4bjj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-06a95f-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 27m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-06a95f-openshift-default Scaled up replica set e2e-shared-a-w6-06a95f-openshift-default-74f57588cc from 1 to 2 27m Normal Started pod/e2e-shared-a-w6-06a95f-openshift-default-74f57588cc-f4bjj Container started 27m Normal AddedInterface pod/e2e-shared-a-w6-06a95f-openshift-default-74f57588cc-f4bjj Add eth0 [10.132.1.31/23] from ovn-kubernetes 27m Normal Created pod/e2e-shared-a-w6-06a95f-openshift-default-74f57588cc-f4bjj Container created 27m Normal ScalingReplicaSet deployment/e2e-worker-w3-43b37d-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w3-43b37d-openshift-default-5dcf864fcc from 3 to 2 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-43b37d-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 27m Normal Killing pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-9txdf Stopping container istio-proxy 27m Normal SuccessfulDelete replicaset/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc Deleted pod: e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-9txdf 27m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-d68c2c-openshift-default Scaled up replica set e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f from 1 to 2 27m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f Created pod: e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-j4gn7 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-d68c2c-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 27m Normal AddedInterface pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-j4gn7 Add eth0 [10.132.1.32/23] from ovn-kubernetes 27m Normal Started pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-j4gn7 Container started 27m Normal Created pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-j4gn7 Container created 27m Normal Pulled pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-j4gn7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Warning ProbeError pod/e2e-shared-a-w6-06a95f-openshift-default-74f57588cc-f4bjj Startup probe error: Get "http://10.132.1.31:15021/healthz/ready": dial tcp 10.132.1.31:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-shared-a-w6-06a95f-openshift-default-74f57588cc-f4bjj Startup probe failed: Get "http://10.132.1.31:15021/healthz/ready": dial tcp 10.132.1.31:15021: connect: connection refused 27m Normal SuccessfulCreate replicaset/e2e-ipp-a-c87bf1-openshift-default-648d4845d9 Created pod: e2e-ipp-a-c87bf1-openshift-default-648d4845d9-qx65t 27m Normal AddedLabel gateway/e2e-ipp-a-c87bf1 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-c87bf1 27m Warning Unhealthy pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-j4gn7 Startup probe failed: Get "http://10.132.1.32:15021/healthz/ready": dial tcp 10.132.1.32:15021: connect: connection refused 27m Normal ScalingReplicaSet deployment/e2e-ipp-a-c87bf1-openshift-default Scaled up replica set e2e-ipp-a-c87bf1-openshift-default-648d4845d9 from 0 to 1 27m Warning ProbeError pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-j4gn7 Startup probe error: Get "http://10.132.1.32:15021/healthz/ready": dial tcp 10.132.1.32:15021: connect: connection refused... 27m Normal AddedInterface pod/e2e-ipp-a-c87bf1-openshift-default-648d4845d9-qx65t Add eth0 [10.134.0.204/23] from ovn-kubernetes 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-eb02df-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-eb02df-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-c87bf1-84c7b6485f Error creating: pods "payload-pre-processing-e2e-ipp-a-c87bf1-84c7b6485f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c87bf1: serviceaccount "payload-processing-e2e-ipp-a-c87bf1" not found 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-c87bf1-5cf8c8df7d Error creating: pods "payload-processing-e2e-ipp-a-c87bf1-5cf8c8df7d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c87bf1: serviceaccount "payload-processing-e2e-ipp-a-c87bf1" not found 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-c87bf1-84c7b6485f Created pod: payload-pre-processing-e2e-ipp-a-c87bf1-84c7b6485f-cxhjm 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-c87bf1 Scaled up replica set payload-processing-e2e-ipp-a-c87bf1-5cf8c8df7d from 0 to 1 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-c87bf1 Scaled up replica set payload-pre-processing-e2e-ipp-a-c87bf1-84c7b6485f from 0 to 1 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-c87bf1-5cf8c8df7d Created pod: payload-processing-e2e-ipp-a-c87bf1-5cf8c8df7d-xwbvk 27m Normal AddedInterface pod/payload-processing-e2e-ipp-a-c87bf1-5cf8c8df7d-xwbvk Add eth0 [10.134.0.206/23] from ovn-kubernetes 27m Normal Started pod/payload-processing-e2e-ipp-a-c87bf1-5cf8c8df7d-xwbvk Container started 27m Normal Created pod/payload-processing-e2e-ipp-a-c87bf1-5cf8c8df7d-xwbvk Container created 27m Normal Pulled pod/payload-processing-e2e-ipp-a-c87bf1-5cf8c8df7d-xwbvk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-c87bf1-84c7b6485f-cxhjm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-c87bf1-84c7b6485f-cxhjm Add eth0 [10.134.0.205/23] from ovn-kubernetes 27m Normal Started pod/payload-pre-processing-e2e-ipp-a-c87bf1-84c7b6485f-cxhjm Container started 27m Normal Created pod/payload-pre-processing-e2e-ipp-a-c87bf1-84c7b6485f-cxhjm Container created 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-920032-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-920032-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-eb02df-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-eb02df-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-f8ca96-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 27m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65 Created pod: e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-bn6sm 27m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65 Created pod: e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-ns4gv 27m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65 Created pod: e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-mvw5s 27m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-f8ca96-openshift-default Scaled up replica set e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65 from 1 to 4 27m Normal Started pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-mb2dl Container started 27m Normal Pulled pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-ns4gv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal Started pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-bn6sm Container started 27m Normal Created pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-mb2dl Container created 27m Normal Pulled pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-mvw5s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal Started pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-ns4gv Container started 27m Normal Created pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-ns4gv Container created 27m Normal Pulled pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-mb2dl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-ns4gv Add eth0 [10.132.1.33/23] from ovn-kubernetes 27m Normal AddedInterface pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-mvw5s Add eth0 [10.133.0.122/23] from ovn-kubernetes 27m Normal Created pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-bn6sm Container created 27m Normal Started pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-mvw5s Container started 27m Normal AddedInterface pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-bn6sm Add eth0 [10.132.1.34/23] from ovn-kubernetes 27m Normal Pulled pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-bn6sm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal Created pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-mvw5s Container created 27m Warning ProbeError pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-mvw5s Startup probe error: Get "http://10.133.0.122:15021/healthz/ready": dial tcp 10.133.0.122:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-mvw5s Startup probe failed: Get "http://10.133.0.122:15021/healthz/ready": dial tcp 10.133.0.122:15021: connect: connection refused 27m Warning Unhealthy pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-mb2dl Startup probe failed: Get "http://10.134.0.188:15021/healthz/ready": dial tcp 10.134.0.188:15021: connect: connection refused 27m Warning ProbeError pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-mb2dl Startup probe error: Get "http://10.134.0.188:15021/healthz/ready": dial tcp 10.134.0.188:15021: connect: connection refused... 27m Warning ProbeError pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-ns4gv Startup probe error: Get "http://10.132.1.33:15021/healthz/ready": dial tcp 10.132.1.33:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-ns4gv Startup probe failed: Get "http://10.132.1.33:15021/healthz/ready": dial tcp 10.132.1.33:15021: connect: connection refused 27m Warning Unhealthy pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-bn6sm Startup probe failed: Get "http://10.132.1.34:15021/healthz/ready": dial tcp 10.132.1.34:15021: connect: connection refused 27m Warning ProbeError pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-bn6sm Startup probe error: Get "http://10.132.1.34:15021/healthz/ready": dial tcp 10.132.1.34:15021: connect: connection refused... 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-f8ca96-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-f8ca96-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65 Created pod: e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-glsxl 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-f8ca96-openshift-default New size: 7; reason: All metrics below target 26m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65 Created pod: e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-ctqgd 26m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65 Created pod: e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-vpr7b 26m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-f8ca96-openshift-default Scaled up replica set e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65 from 4 to 7 26m Normal Started pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-glsxl Container started 26m Normal Pulled pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-glsxl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Started pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-hjvtw Container started 26m Normal Created pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-hjvtw Container created 26m Normal AddedInterface pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-vpr7b Add eth0 [10.132.1.38/23] from ovn-kubernetes 26m Normal Pulled pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-vpr7b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-ctqgd Add eth0 [10.134.0.208/23] from ovn-kubernetes 26m Normal Created pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-vpr7b Container created 26m Normal Started pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-vpr7b Container started 26m Normal Pulled pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-ctqgd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Created pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-ctqgd Container created 26m Normal Started pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-ctqgd Container started 26m Normal Pulled pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-hjvtw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Created pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-glsxl Container created 26m Normal AddedInterface pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-glsxl Add eth0 [10.133.0.127/23] from ovn-kubernetes 26m Warning ProbeError pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-ctqgd Startup probe error: Get "http://10.134.0.208:15021/healthz/ready": dial tcp 10.134.0.208:15021: connect: connection refused... 26m Warning ProbeError pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-glsxl Startup probe error: Get "http://10.133.0.127:15021/healthz/ready": dial tcp 10.133.0.127:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-hjvtw Startup probe failed: Get "http://10.132.1.29:15021/healthz/ready": dial tcp 10.132.1.29:15021: connect: connection refused 26m Warning ProbeError pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-hjvtw Startup probe error: Get "http://10.132.1.29:15021/healthz/ready": dial tcp 10.132.1.29:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-ctqgd Startup probe failed: Get "http://10.134.0.208:15021/healthz/ready": dial tcp 10.134.0.208:15021: connect: connection refused 26m Warning ProbeError pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-vpr7b Startup probe error: Get "http://10.132.1.38:15021/healthz/ready": dial tcp 10.132.1.38:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-vpr7b Startup probe failed: Get "http://10.132.1.38:15021/healthz/ready": dial tcp 10.132.1.38:15021: connect: connection refused 26m Warning Unhealthy pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-glsxl Startup probe failed: Get "http://10.133.0.127:15021/healthz/ready": dial tcp 10.133.0.127:15021: connect: connection refused 26m Normal Started pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-shmwc Container started 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-3bd0f6-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 26m Normal Pulled pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-shmwc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Created pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-shmwc Container created 26m Normal AddedInterface pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-shmwc Add eth0 [10.133.0.129/23] from ovn-kubernetes 26m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-3bd0f6-openshift-default Scaled up replica set e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77 from 2 to 3 26m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77 Created pod: e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-shmwc 26m Normal Killing pod/payload-pre-processing-e2e-ipp-a-c87bf1-84c7b6485f-cxhjm Stopping container payload-pre-processing 26m Normal Killing pod/payload-processing-e2e-ipp-a-c87bf1-5cf8c8df7d-xwbvk Stopping container payload-processing 26m Warning Unhealthy pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-shmwc Startup probe failed: Get "http://10.133.0.129:15021/healthz/ready": dial tcp 10.133.0.129:15021: connect: connection refused 26m Warning ProbeError pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-shmwc Startup probe error: Get "http://10.133.0.129:15021/healthz/ready": dial tcp 10.133.0.129:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-ipp-a-c87bf1-openshift-default-648d4845d9-qx65t Readiness probe failed: Get "http://10.134.0.204:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 26m Warning ProbeError pod/e2e-ipp-a-c87bf1-openshift-default-648d4845d9-qx65t Readiness probe error: Get "http://10.134.0.204:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 26m Normal Pulled pod/e2e-ipp-a-c87bf1-openshift-default-648d4845d9-qx65t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Created pod/e2e-ipp-a-c87bf1-openshift-default-648d4845d9-qx65t Container created 26m Normal Started pod/e2e-ipp-a-c87bf1-openshift-default-648d4845d9-qx65t Container started 25m Warning Unhealthy pod/e2e-ipp-a-c87bf1-openshift-default-648d4845d9-qx65t Startup probe failed: Get "http://10.134.0.204:15021/healthz/ready": dial tcp 10.134.0.204:15021: connect: connection refused 25m Warning ProbeError pod/e2e-ipp-a-c87bf1-openshift-default-648d4845d9-qx65t Startup probe error: Get "http://10.134.0.204:15021/healthz/ready": dial tcp 10.134.0.204:15021: connect: connection refused... 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-c87bf1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-c87bf1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Normal SuccessfulCreate replicaset/e2e-ipp-a-7be71d-openshift-default-8dd8b5bdc Created pod: e2e-ipp-a-7be71d-openshift-default-8dd8b5bdc-4hx2h 25m Normal AddedLabel gateway/e2e-ipp-a-7be71d Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-7be71d 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-3ee017-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-3ee017-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Normal ScalingReplicaSet deployment/e2e-ipp-a-7be71d-openshift-default Scaled up replica set e2e-ipp-a-7be71d-openshift-default-8dd8b5bdc from 0 to 1 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-920032-openshift-default Scaled up replica set e2e-shared-a-w0-920032-openshift-default-5b5f9cc67 from 1 to 2 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67 Created pod: e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-jzqt4 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-920032-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 25m Normal AddedInterface pod/e2e-ipp-a-7be71d-openshift-default-8dd8b5bdc-4hx2h Add eth0 [10.132.1.44/23] from ovn-kubernetes 25m Normal AddedInterface pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-jzqt4 Add eth0 [10.132.1.45/23] from ovn-kubernetes 25m Normal Pulled pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-jzqt4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Created pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-jzqt4 Container created 25m Normal Started pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-jzqt4 Container started 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-7be71d Scaled up replica set payload-processing-e2e-ipp-a-7be71d-59d98d4c64 from 0 to 1 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-7be71d-59d98d4c64 Error creating: pods "payload-processing-e2e-ipp-a-7be71d-59d98d4c64-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-7be71d: serviceaccount "payload-processing-e2e-ipp-a-7be71d" not found 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-7be71d-648bf674d7 Error creating: pods "payload-pre-processing-e2e-ipp-a-7be71d-648bf674d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-7be71d: serviceaccount "payload-processing-e2e-ipp-a-7be71d" not found 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-7be71d Scaled up replica set payload-pre-processing-e2e-ipp-a-7be71d-648bf674d7 from 0 to 1 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-7be71d-59d98d4c64 Created pod: payload-processing-e2e-ipp-a-7be71d-59d98d4c64-5gtp5 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-7be71d-648bf674d7 Created pod: payload-pre-processing-e2e-ipp-a-7be71d-648bf674d7-kqdsm 25m Normal Started pod/payload-processing-e2e-ipp-a-7be71d-59d98d4c64-5gtp5 Container started 25m Normal Created pod/payload-processing-e2e-ipp-a-7be71d-59d98d4c64-5gtp5 Container created 25m Normal Started pod/payload-pre-processing-e2e-ipp-a-7be71d-648bf674d7-kqdsm Container started 25m Normal AddedInterface pod/payload-processing-e2e-ipp-a-7be71d-59d98d4c64-5gtp5 Add eth0 [10.134.0.211/23] from ovn-kubernetes 25m Normal Created pod/payload-pre-processing-e2e-ipp-a-7be71d-648bf674d7-kqdsm Container created 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-7be71d-648bf674d7-kqdsm Add eth0 [10.134.0.210/23] from ovn-kubernetes 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-7be71d-648bf674d7-kqdsm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal Pulled pod/payload-processing-e2e-ipp-a-7be71d-59d98d4c64-5gtp5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Warning Unhealthy pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-jzqt4 Startup probe failed: Get "http://10.132.1.45:15021/healthz/ready": dial tcp 10.132.1.45:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-jzqt4 Startup probe error: Get "http://10.132.1.45:15021/healthz/ready": dial tcp 10.132.1.45:15021: connect: connection refused... 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-c87bf1-6bc965df8c Error creating: pods "payload-pre-processing-e2e-ipp-a-c87bf1-6bc965df8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c87bf1: serviceaccount "payload-processing-e2e-ipp-a-c87bf1" not found 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-c87bf1 Scaled up replica set payload-pre-processing-e2e-ipp-a-c87bf1-6bc965df8c from 0 to 1 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-c87bf1-85b4d8ffd8 Error creating: pods "payload-processing-e2e-ipp-a-c87bf1-85b4d8ffd8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c87bf1: serviceaccount "payload-processing-e2e-ipp-a-c87bf1" not found 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-c87bf1-85b4d8ffd8 Created pod: payload-processing-e2e-ipp-a-c87bf1-85b4d8ffd8-jz456 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-c87bf1 Scaled up replica set payload-processing-e2e-ipp-a-c87bf1-85b4d8ffd8 from 0 to 1 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-c87bf1-6bc965df8c Created pod: payload-pre-processing-e2e-ipp-a-c87bf1-6bc965df8c-79jh7 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-c87bf1-6bc965df8c-79jh7 Add eth0 [10.133.0.130/23] from ovn-kubernetes 25m Normal Pulling pod/payload-pre-processing-e2e-ipp-a-c87bf1-6bc965df8c-79jh7 Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 25m Normal Pulling pod/payload-processing-e2e-ipp-a-c87bf1-85b4d8ffd8-jz456 Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 25m Normal AddedInterface pod/payload-processing-e2e-ipp-a-c87bf1-85b4d8ffd8-jz456 Add eth0 [10.133.0.131/23] from ovn-kubernetes 25m Normal Started pod/payload-processing-e2e-ipp-a-c87bf1-85b4d8ffd8-jz456 Container started 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-c87bf1-6bc965df8c-79jh7 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.435s (1.435s including waiting). Image size: 166612641 bytes. 25m Normal Created pod/payload-pre-processing-e2e-ipp-a-c87bf1-6bc965df8c-79jh7 Container created 25m Normal Started pod/payload-pre-processing-e2e-ipp-a-c87bf1-6bc965df8c-79jh7 Container started 25m Normal Pulled pod/payload-processing-e2e-ipp-a-c87bf1-85b4d8ffd8-jz456 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.34s (1.34s including waiting). Image size: 166612641 bytes. 25m Normal Created pod/payload-processing-e2e-ipp-a-c87bf1-85b4d8ffd8-jz456 Container created 25m Normal AddedInterface pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-wk57v Add eth0 [10.134.0.212/23] from ovn-kubernetes 25m Normal Pulled pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-wk57v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-9aafe6-openshift-default Scaled up replica set e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85 from 2 to 3 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w5-9aafe6-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w1-af260f-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-af260f-openshift-default-846bb69f67 Created pod: e2e-shared-a-w1-af260f-openshift-default-846bb69f67-wk57v 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85 Created pod: e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-2t2kf 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-af260f-openshift-default Scaled up replica set e2e-shared-a-w1-af260f-openshift-default-846bb69f67 from 1 to 2 25m Normal Created pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-2t2kf Container created 25m Normal Started pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-wk57v Container started 25m Normal Created pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-wk57v Container created 25m Normal Started pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-2t2kf Container started 25m Normal Pulled pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-2t2kf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-2t2kf Add eth0 [10.133.0.133/23] from ovn-kubernetes 25m Warning ProbeError pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-wk57v Startup probe error: Get "http://10.134.0.212:15021/healthz/ready": dial tcp 10.134.0.212:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-wk57v Startup probe failed: Get "http://10.134.0.212:15021/healthz/ready": dial tcp 10.134.0.212:15021: connect: connection refused 25m Warning Unhealthy pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-2t2kf Startup probe failed: Get "http://10.133.0.133:15021/healthz/ready": dial tcp 10.133.0.133:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-2t2kf Startup probe error: Get "http://10.133.0.133:15021/healthz/ready": dial tcp 10.133.0.133:15021: connect: connection refused... 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-7be71d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-7be71d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Normal AddedLabel gateway/e2e-ipp-a-8c4b02 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-8c4b02 25m Normal ScalingReplicaSet deployment/e2e-ipp-a-8c4b02-openshift-default Scaled up replica set e2e-ipp-a-8c4b02-openshift-default-679994b96f from 0 to 1 25m Normal AddedLabel gateway/e2e-ipp-b-a962d7 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-a962d7 25m Normal ScalingReplicaSet deployment/e2e-ipp-b-a962d7-openshift-default Scaled up replica set e2e-ipp-b-a962d7-openshift-default-6dbc7cd94c from 0 to 1 25m Normal SuccessfulCreate replicaset/e2e-ipp-a-8c4b02-openshift-default-679994b96f Created pod: e2e-ipp-a-8c4b02-openshift-default-679994b96f-7wssh 25m Normal SuccessfulCreate replicaset/e2e-ipp-b-a962d7-openshift-default-6dbc7cd94c Created pod: e2e-ipp-b-a962d7-openshift-default-6dbc7cd94c-625mm 25m Normal AddedInterface pod/e2e-ipp-b-a962d7-openshift-default-6dbc7cd94c-625mm Add eth0 [10.134.0.213/23] from ovn-kubernetes 25m Normal AddedInterface pod/e2e-ipp-a-8c4b02-openshift-default-679994b96f-7wssh Add eth0 [10.132.1.46/23] from ovn-kubernetes 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-8c4b02-86d6df458 Created pod: payload-processing-e2e-ipp-a-8c4b02-86d6df458-f5kjj 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-8c4b02-86d6df458 Error creating: pods "payload-processing-e2e-ipp-a-8c4b02-86d6df458-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-8c4b02: serviceaccount "payload-processing-e2e-ipp-a-8c4b02" not found 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-8c4b02 Scaled up replica set payload-pre-processing-e2e-ipp-a-8c4b02-696f4f544c from 0 to 1 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-c87bf1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-8c4b02-696f4f544c Created pod: payload-pre-processing-e2e-ipp-a-8c4b02-696f4f544c-xbnh8 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-8c4b02-696f4f544c Error creating: pods "payload-pre-processing-e2e-ipp-a-8c4b02-696f4f544c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-8c4b02: serviceaccount "payload-processing-e2e-ipp-a-8c4b02" not found 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-8c4b02 Scaled up replica set payload-processing-e2e-ipp-a-8c4b02-86d6df458 from 0 to 1 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-c87bf1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-8c4b02-696f4f544c-xbnh8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c Created pod: e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-mbzm5 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-8c4b02-696f4f544c-xbnh8 Add eth0 [10.132.1.47/23] from ovn-kubernetes 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-eb02df-openshift-default Scaled up replica set e2e-shared-a-w2-eb02df-openshift-default-5c45c545c from 1 to 2 25m Normal Created pod/payload-processing-e2e-ipp-a-8c4b02-86d6df458-f5kjj Container created 25m Normal Pulled pod/payload-processing-e2e-ipp-a-8c4b02-86d6df458-f5kjj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal Started pod/payload-processing-e2e-ipp-a-8c4b02-86d6df458-f5kjj Container started 25m Normal Created pod/payload-pre-processing-e2e-ipp-a-8c4b02-696f4f544c-xbnh8 Container created 25m Normal AddedInterface pod/payload-processing-e2e-ipp-a-8c4b02-86d6df458-f5kjj Add eth0 [10.132.1.48/23] from ovn-kubernetes 25m Normal Started pod/payload-pre-processing-e2e-ipp-a-8c4b02-696f4f544c-xbnh8 Container started 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-eb02df-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 25m Normal AddedInterface pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-mbzm5 Add eth0 [10.132.1.49/23] from ovn-kubernetes 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-a962d7 Scaled up replica set payload-processing-e2e-ipp-b-a962d7-7cf57bb744 from 0 to 1 25m Normal Created pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-mbzm5 Container created 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-a962d7-64fcd788bc Error creating: pods "payload-pre-processing-e2e-ipp-b-a962d7-64fcd788bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-a962d7: serviceaccount "payload-processing-e2e-ipp-b-a962d7" not found 25m Normal Pulled pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-mbzm5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Started pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-mbzm5 Container started 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-a962d7 Scaled up replica set payload-pre-processing-e2e-ipp-b-a962d7-64fcd788bc from 0 to 1 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-a962d7-7cf57bb744 Error creating: pods "payload-processing-e2e-ipp-b-a962d7-7cf57bb744-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-a962d7: serviceaccount "payload-processing-e2e-ipp-b-a962d7" not found 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-a962d7-7cf57bb744 Created pod: payload-processing-e2e-ipp-b-a962d7-7cf57bb744-49pdj 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-a962d7-64fcd788bc Created pod: payload-pre-processing-e2e-ipp-b-a962d7-64fcd788bc-vc928 25m Normal Created pod/payload-pre-processing-e2e-ipp-b-a962d7-64fcd788bc-vc928 Container created 25m Normal AddedInterface pod/payload-processing-e2e-ipp-b-a962d7-7cf57bb744-49pdj Add eth0 [10.134.0.215/23] from ovn-kubernetes 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-a962d7-64fcd788bc-vc928 Add eth0 [10.134.0.214/23] from ovn-kubernetes 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-a962d7-64fcd788bc-vc928 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal Pulled pod/payload-processing-e2e-ipp-b-a962d7-7cf57bb744-49pdj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal Created pod/payload-processing-e2e-ipp-b-a962d7-7cf57bb744-49pdj Container created 25m Normal Started pod/payload-processing-e2e-ipp-b-a962d7-7cf57bb744-49pdj Container started 25m Normal Started pod/payload-pre-processing-e2e-ipp-b-a962d7-64fcd788bc-vc928 Container started 25m Normal Killing pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-hz9sp Stopping container istio-proxy 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-2ba10a-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 25m Normal Started pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-kwffs Container started 25m Normal Created pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-kwffs Container created 25m Normal ScalingReplicaSet deployment/e2e-worker-w2-2ba10a-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 from 3 to 2 25m Normal Pulled pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-kwffs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-22d634-openshift-default New size: 2; reason: All metrics below target 25m Normal ScalingReplicaSet deployment/e2e-worker-w5-22d634-openshift-default Scaled down replica set e2e-worker-w5-22d634-openshift-default-89dbbf78f from 3 to 2 25m Normal SuccessfulDelete replicaset/e2e-worker-w5-22d634-openshift-default-89dbbf78f Deleted pod: e2e-worker-w5-22d634-openshift-default-89dbbf78f-hz9sp 25m Normal SuccessfulDelete replicaset/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99 Deleted pod: e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-448t7 25m Normal Killing pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-448t7 Stopping container istio-proxy 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-02720a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Warning ProbeError pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-mbzm5 Startup probe error: Get "http://10.132.1.49:15021/healthz/ready": dial tcp 10.132.1.49:15021: connect: connection refused... 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-02720a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Warning Unhealthy pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-mbzm5 Startup probe failed: Get "http://10.132.1.49:15021/healthz/ready": dial tcp 10.132.1.49:15021: connect: connection refused 24m Warning ProbeError pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-kwffs Startup probe error: Get "http://10.134.0.201:15021/healthz/ready": dial tcp 10.134.0.201:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-kwffs Startup probe failed: Get "http://10.134.0.201:15021/healthz/ready": dial tcp 10.134.0.201:15021: connect: connection refused 24m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-3bd0f6-openshift-default Scaled up replica set e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77 from 3 to 4 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77 Created pod: e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-nn958 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-3bd0f6-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 24m Normal AddedInterface pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-nn958 Add eth0 [10.134.0.216/23] from ovn-kubernetes 24m Normal Started pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-nn958 Container started 24m Normal Created pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-nn958 Container created 24m Normal Pulled pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-nn958 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Warning Unhealthy pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-nn958 Startup probe failed: Get "http://10.134.0.216:15021/healthz/ready": dial tcp 10.134.0.216:15021: connect: connection refused 24m Warning ProbeError pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-nn958 Startup probe error: Get "http://10.134.0.216:15021/healthz/ready": dial tcp 10.134.0.216:15021: connect: connection refused... 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-02720a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-02720a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning Unhealthy pod/e2e-ipp-a-7be71d-openshift-default-8dd8b5bdc-4hx2h Readiness probe failed: Get "http://10.132.1.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning ProbeError pod/e2e-ipp-a-7be71d-openshift-default-8dd8b5bdc-4hx2h Readiness probe error: Get "http://10.132.1.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Normal Pulled pod/e2e-ipp-a-7be71d-openshift-default-8dd8b5bdc-4hx2h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Killing pod/payload-processing-e2e-ipp-a-7be71d-59d98d4c64-5gtp5 Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-ipp-a-7be71d-648bf674d7-kqdsm Stopping container payload-pre-processing 24m Normal Created pod/e2e-ipp-a-7be71d-openshift-default-8dd8b5bdc-4hx2h Container created 24m Normal Started pod/e2e-ipp-a-7be71d-openshift-default-8dd8b5bdc-4hx2h Container started 24m Warning ProbeError pod/e2e-ipp-a-7be71d-openshift-default-8dd8b5bdc-4hx2h Startup probe error: Get "http://10.132.1.44:15021/healthz/ready": dial tcp 10.132.1.44:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-ipp-a-7be71d-openshift-default-8dd8b5bdc-4hx2h Startup probe failed: Get "http://10.132.1.44:15021/healthz/ready": dial tcp 10.132.1.44:15021: connect: connection refused 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-784e7a-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 24m Normal ScalingReplicaSet deployment/e2e-worker-w6-784e7a-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w6-784e7a-openshift-default-595898bd5c from 3 to 2 24m Normal Killing pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-vwlks Stopping container istio-proxy 24m Normal SuccessfulDelete replicaset/e2e-worker-w6-784e7a-openshift-default-595898bd5c Deleted pod: e2e-worker-w6-784e7a-openshift-default-595898bd5c-vwlks 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-3ee017-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7 Created pod: e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-ht9j2 24m Normal AddedInterface pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-ht9j2 Add eth0 [10.134.0.218/23] from ovn-kubernetes 24m Normal Started pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-ht9j2 Container started 24m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-3ee017-openshift-default Scaled up replica set e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7 from 1 to 2 24m Normal Created pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-ht9j2 Container created 24m Normal Pulled pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-ht9j2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal AddedInterface pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-fzz2k Add eth0 [10.134.0.219/23] from ovn-kubernetes 24m Normal Pulled pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-fzz2k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Pulled pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-jmfgs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-920032-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 24m Normal AddedInterface pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-jmfgs Add eth0 [10.133.0.140/23] from ovn-kubernetes 24m Normal Created pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-jmfgs Container created 24m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-920032-openshift-default Scaled up replica set e2e-shared-a-w0-920032-openshift-default-5b5f9cc67 from 2 to 4 24m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67 Created pod: e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-fzz2k 24m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67 Created pod: e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-jmfgs 24m Normal Started pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-jmfgs Container started 24m Normal Started pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-fzz2k Container started 24m Normal Created pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-fzz2k Container created 23m Warning ProbeError pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-jmfgs Startup probe error: Get "http://10.133.0.140:15021/healthz/ready": dial tcp 10.133.0.140:15021: connect: connection refused... 23m Normal Killing pod/maas-default-gateway-openshift-default-5db77fbd9f-hrms9 Stopping container istio-proxy 23m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5db77fbd9f Deleted pod: maas-default-gateway-openshift-default-5db77fbd9f-hrms9 23m Warning ProbeError pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-ht9j2 Startup probe error: Get "http://10.134.0.218:15021/healthz/ready": dial tcp 10.134.0.218:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-ht9j2 Startup probe failed: Get "http://10.134.0.218:15021/healthz/ready": dial tcp 10.134.0.218:15021: connect: connection refused 23m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: All metrics below target 23m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-5db77fbd9f from 4 to 3 23m Warning Unhealthy pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-jmfgs Startup probe failed: Get "http://10.133.0.140:15021/healthz/ready": dial tcp 10.133.0.140:15021: connect: connection refused 23m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c Created pod: e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-8j4dd 23m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-eb02df-openshift-default Scaled up replica set e2e-shared-a-w2-eb02df-openshift-default-5c45c545c from 2 to 3 23m Warning ProbeError pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-fzz2k Startup probe error: Get "http://10.134.0.219:15021/healthz/ready": dial tcp 10.134.0.219:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-fzz2k Startup probe failed: Get "http://10.134.0.219:15021/healthz/ready": dial tcp 10.134.0.219:15021: connect: connection refused 23m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-eb02df-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 23m Normal AddedInterface pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-8j4dd Add eth0 [10.133.0.142/23] from ovn-kubernetes 23m Normal Created pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-8j4dd Container created 23m Normal Pulled pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-8j4dd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Started pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-8j4dd Container started 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-7be71d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning ProbeError pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-8j4dd Startup probe error: Get "http://10.133.0.142:15021/healthz/ready": dial tcp 10.133.0.142:15021: connect: connection refused... 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-7be71d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning Unhealthy pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-8j4dd Startup probe failed: Get "http://10.133.0.142:15021/healthz/ready": dial tcp 10.133.0.142:15021: connect: connection refused 23m Normal Killing pod/payload-pre-processing-e2e-ipp-a-8c4b02-696f4f544c-xbnh8 Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-ipp-a-8c4b02-86d6df458-f5kjj Stopping container payload-processing 23m Normal AddedLabel gateway/e2e-ipp-b-4796a9 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-4796a9 23m Normal ScalingReplicaSet deployment/e2e-ipp-b-4796a9-openshift-default Scaled up replica set e2e-ipp-b-4796a9-openshift-default-75cd4d45f7 from 0 to 1 23m Normal SuccessfulCreate replicaset/e2e-ipp-b-4796a9-openshift-default-75cd4d45f7 Created pod: e2e-ipp-b-4796a9-openshift-default-75cd4d45f7-gm2vv 23m Normal AddedInterface pod/e2e-ipp-b-4796a9-openshift-default-75cd4d45f7-gm2vv Add eth0 [10.134.0.220/23] from ovn-kubernetes 23m Normal Killing pod/payload-processing-e2e-ipp-b-a962d7-7cf57bb744-49pdj Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-ipp-b-a962d7-64fcd788bc-vc928 Stopping container payload-pre-processing 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-4796a9-d6c677c75 Error creating: pods "payload-pre-processing-e2e-ipp-b-4796a9-d6c677c75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-4796a9: serviceaccount "payload-processing-e2e-ipp-b-4796a9" not found 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-4796a9-d6c677c75 Created pod: payload-pre-processing-e2e-ipp-b-4796a9-d6c677c75-9zj9b 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-4796a9 Scaled up replica set payload-pre-processing-e2e-ipp-b-4796a9-d6c677c75 from 0 to 1 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-4796a9 Scaled up replica set payload-processing-e2e-ipp-b-4796a9-6c55db49cd from 0 to 1 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-4796a9-6c55db49cd Created pod: payload-processing-e2e-ipp-b-4796a9-6c55db49cd-kkpxj 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-4796a9-6c55db49cd Error creating: pods "payload-processing-e2e-ipp-b-4796a9-6c55db49cd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-4796a9: serviceaccount "payload-processing-e2e-ipp-b-4796a9" not found 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-4796a9-d6c677c75-9zj9b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Normal Pulled pod/payload-processing-e2e-ipp-b-4796a9-6c55db49cd-kkpxj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Normal Created pod/payload-pre-processing-e2e-ipp-b-4796a9-d6c677c75-9zj9b Container created 23m Normal Started pod/payload-pre-processing-e2e-ipp-b-4796a9-d6c677c75-9zj9b Container started 23m Normal AddedInterface pod/payload-processing-e2e-ipp-b-4796a9-6c55db49cd-kkpxj Add eth0 [10.132.1.52/23] from ovn-kubernetes 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-4796a9-d6c677c75-9zj9b Add eth0 [10.134.0.221/23] from ovn-kubernetes 23m Normal Created pod/payload-processing-e2e-ipp-b-4796a9-6c55db49cd-kkpxj Container created 23m Normal Started pod/payload-processing-e2e-ipp-b-4796a9-6c55db49cd-kkpxj Container started 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-7be71d-85b85b8ffd Error creating: pods "payload-processing-e2e-ipp-a-7be71d-85b85b8ffd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-7be71d: serviceaccount "payload-processing-e2e-ipp-a-7be71d" not found 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-7be71d Scaled up replica set payload-processing-e2e-ipp-a-7be71d-85b85b8ffd from 0 to 1 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-7be71d-865cd9bc7f Error creating: pods "payload-pre-processing-e2e-ipp-a-7be71d-865cd9bc7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-7be71d: serviceaccount "payload-processing-e2e-ipp-a-7be71d" not found 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-7be71d Scaled up replica set payload-pre-processing-e2e-ipp-a-7be71d-865cd9bc7f from 0 to 1 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-7be71d-85b85b8ffd Created pod: payload-processing-e2e-ipp-a-7be71d-85b85b8ffd-d77xb 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-7be71d-865cd9bc7f Created pod: payload-pre-processing-e2e-ipp-a-7be71d-865cd9bc7f-52m9d 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-7be71d-865cd9bc7f-52m9d Add eth0 [10.134.0.222/23] from ovn-kubernetes 23m Normal Pulling pod/payload-pre-processing-e2e-ipp-a-7be71d-865cd9bc7f-52m9d Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 23m Normal AddedInterface pod/payload-processing-e2e-ipp-a-7be71d-85b85b8ffd-d77xb Add eth0 [10.132.1.54/23] from ovn-kubernetes 23m Normal Pulling pod/payload-processing-e2e-ipp-a-7be71d-85b85b8ffd-d77xb Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 23m Normal Started pod/payload-pre-processing-e2e-ipp-a-7be71d-865cd9bc7f-52m9d Container started 23m Normal Started pod/payload-processing-e2e-ipp-a-7be71d-85b85b8ffd-d77xb Container started 23m Normal Pulled pod/payload-processing-e2e-ipp-a-7be71d-85b85b8ffd-d77xb Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.521s (1.521s including waiting). Image size: 166612641 bytes. 23m Normal Created pod/payload-processing-e2e-ipp-a-7be71d-85b85b8ffd-d77xb Container created 23m Normal Created pod/payload-pre-processing-e2e-ipp-a-7be71d-865cd9bc7f-52m9d Container created 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-7be71d-865cd9bc7f-52m9d Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.649s (1.649s including waiting). Image size: 166612641 bytes. 23m Normal Pulled pod/e2e-ipp-a-8c4b02-openshift-default-679994b96f-7wssh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Created pod/e2e-ipp-a-8c4b02-openshift-default-679994b96f-7wssh Container created 23m Normal Started pod/e2e-ipp-a-8c4b02-openshift-default-679994b96f-7wssh Container started 23m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-f8ca96-openshift-default Scaled up replica set e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65 from 7 to 10 23m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-f8ca96-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 23m Normal Started pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-n2ttv Container started 23m Normal AddedInterface pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-rt6nd Add eth0 [10.133.0.143/23] from ovn-kubernetes 23m Normal AddedInterface pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-x9bxc Add eth0 [10.134.0.224/23] from ovn-kubernetes 23m Normal Created pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-x9bxc Container created 23m Normal Started pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-rt6nd Container started 23m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65 Created pod: e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-n2ttv 23m Normal Started pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-x9bxc Container started 23m Normal Pulled pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-x9bxc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Created pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-n2ttv Container created 23m Normal Pulled pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-n2ttv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-n2ttv Add eth0 [10.134.0.223/23] from ovn-kubernetes 23m Normal Pulled pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-rt6nd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65 (combined from similar events): Created pod: e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-x9bxc 23m Normal Created pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-rt6nd Container created 23m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65 Created pod: e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-rt6nd 23m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-02720a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 23m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-d68c2c-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 23m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f Created pod: e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-wrrbr 23m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-02720a-openshift-default Scaled up replica set e2e-shared-b-w0-02720a-openshift-default-6b64cb69 from 1 to 2 23m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-d68c2c-openshift-default Scaled up replica set e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f from 2 to 3 23m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-02720a-openshift-default-6b64cb69 Created pod: e2e-shared-b-w0-02720a-openshift-default-6b64cb69-sgz2w 23m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-c933cb-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 23m Normal Pulled pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-sgz2w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-wrrbr Add eth0 [10.133.0.144/23] from ovn-kubernetes 23m Normal Created pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-wrrbr Container created 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-8c4b02-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-8c4b02-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Normal Started pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-wrrbr Container started 23m Normal AddedInterface pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-sgz2w Add eth0 [10.132.1.56/23] from ovn-kubernetes 23m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7 Created pod: e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-cs6kx 23m Normal Pulled pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-wrrbr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Created pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-sgz2w Container created 23m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-c933cb-openshift-default Scaled up replica set e2e-shared-b-w4-c933cb-openshift-default-9986db7d7 from 3 to 4 23m Normal Started pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-sgz2w Container started 23m Normal Started pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-cs6kx Container started 23m Normal Created pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-cs6kx Container created 23m Normal Pulled pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-cs6kx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-cs6kx Add eth0 [10.132.1.57/23] from ovn-kubernetes 23m Warning Unhealthy pod/e2e-ipp-a-8c4b02-openshift-default-679994b96f-7wssh Startup probe failed: Get "http://10.132.1.46:15021/healthz/ready": dial tcp 10.132.1.46:15021: connect: connection refused 23m Warning ProbeError pod/e2e-ipp-a-8c4b02-openshift-default-679994b96f-7wssh Startup probe error: Get "http://10.132.1.46:15021/healthz/ready": dial tcp 10.132.1.46:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-x9bxc Startup probe failed: Get "http://10.134.0.224:15021/healthz/ready": dial tcp 10.134.0.224:15021: connect: connection refused 23m Warning ProbeError pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-n2ttv Startup probe error: Get "http://10.134.0.223:15021/healthz/ready": dial tcp 10.134.0.223:15021: connect: connection refused... 23m Warning ProbeError pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-rt6nd Startup probe error: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-rt6nd Startup probe failed: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused 23m Warning ProbeError pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-x9bxc Startup probe error: Get "http://10.134.0.224:15021/healthz/ready": dial tcp 10.134.0.224:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-n2ttv Startup probe failed: Get "http://10.134.0.223:15021/healthz/ready": dial tcp 10.134.0.223:15021: connect: connection refused 23m Warning ProbeError pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-wrrbr Startup probe error: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-wrrbr Startup probe failed: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused 23m Warning Unhealthy pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-cs6kx Startup probe failed: Get "http://10.132.1.57:15021/healthz/ready": dial tcp 10.132.1.57:15021: connect: connection refused 23m Warning ProbeError pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-sgz2w Startup probe error: Get "http://10.132.1.56:15021/healthz/ready": dial tcp 10.132.1.56:15021: connect: connection refused... 23m Normal AddedLabel gateway/e2e-ipp-b-85862e Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-85862e 23m Warning Unhealthy pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-sgz2w Startup probe failed: Get "http://10.132.1.56:15021/healthz/ready": dial tcp 10.132.1.56:15021: connect: connection refused 23m Warning ProbeError pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-cs6kx Startup probe error: Get "http://10.132.1.57:15021/healthz/ready": dial tcp 10.132.1.57:15021: connect: connection refused... 23m Normal ScalingReplicaSet deployment/e2e-ipp-b-85862e-openshift-default Scaled up replica set e2e-ipp-b-85862e-openshift-default-7b794c9f49 from 0 to 1 23m Normal SuccessfulCreate replicaset/e2e-ipp-b-85862e-openshift-default-7b794c9f49 Created pod: e2e-ipp-b-85862e-openshift-default-7b794c9f49-6wxwm 23m Normal AddedInterface pod/e2e-ipp-b-85862e-openshift-default-7b794c9f49-6wxwm Add eth0 [10.134.0.225/23] from ovn-kubernetes 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-85862e Scaled up replica set payload-pre-processing-e2e-ipp-b-85862e-597bbb855 from 0 to 1 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-85862e-597bbb855 Created pod: payload-pre-processing-e2e-ipp-b-85862e-597bbb855-5jrdf 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-85862e-5759d9df58 Created pod: payload-processing-e2e-ipp-b-85862e-5759d9df58-t4vrn 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-85862e-5759d9df58 Error creating: pods "payload-processing-e2e-ipp-b-85862e-5759d9df58-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-85862e: serviceaccount "payload-processing-e2e-ipp-b-85862e" not found 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-85862e Scaled up replica set payload-processing-e2e-ipp-b-85862e-5759d9df58 from 0 to 1 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-85862e-597bbb855 Error creating: pods "payload-pre-processing-e2e-ipp-b-85862e-597bbb855-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-85862e: serviceaccount "payload-processing-e2e-ipp-b-85862e" not found 23m Normal Created pod/payload-processing-e2e-ipp-b-85862e-5759d9df58-t4vrn Container created 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-8c4b02-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-85862e-597bbb855-5jrdf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Normal Created pod/payload-pre-processing-e2e-ipp-b-85862e-597bbb855-5jrdf Container created 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-85862e-597bbb855-5jrdf Add eth0 [10.132.1.58/23] from ovn-kubernetes 23m Normal Started pod/payload-pre-processing-e2e-ipp-b-85862e-597bbb855-5jrdf Container started 23m Normal Pulled pod/payload-processing-e2e-ipp-b-85862e-5759d9df58-t4vrn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/payload-processing-e2e-ipp-b-85862e-5759d9df58-t4vrn Add eth0 [10.132.1.59/23] from ovn-kubernetes 23m Normal Started pod/payload-processing-e2e-ipp-b-85862e-5759d9df58-t4vrn Container started 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-8c4b02-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-8c4b02 Scaled up replica set payload-pre-processing-e2e-ipp-a-8c4b02-c7cf79579 from 0 to 1 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-8c4b02 Scaled up replica set payload-processing-e2e-ipp-a-8c4b02-68c646b4dd from 0 to 1 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-8c4b02-68c646b4dd Created pod: payload-processing-e2e-ipp-a-8c4b02-68c646b4dd-rx4cf 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-8c4b02-68c646b4dd Error creating: pods "payload-processing-e2e-ipp-a-8c4b02-68c646b4dd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-8c4b02: serviceaccount "payload-processing-e2e-ipp-a-8c4b02" not found 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-8c4b02-c7cf79579 Created pod: payload-pre-processing-e2e-ipp-a-8c4b02-c7cf79579-lj49f 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-8c4b02-c7cf79579 Error creating: pods "payload-pre-processing-e2e-ipp-a-8c4b02-c7cf79579-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-8c4b02: serviceaccount "payload-processing-e2e-ipp-a-8c4b02" not found 22m Normal Pulled pod/payload-processing-e2e-ipp-a-8c4b02-68c646b4dd-rx4cf Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal Started pod/payload-processing-e2e-ipp-a-8c4b02-68c646b4dd-rx4cf Container started 22m Normal Started pod/payload-pre-processing-e2e-ipp-a-8c4b02-c7cf79579-lj49f Container started 22m Normal Created pod/payload-processing-e2e-ipp-a-8c4b02-68c646b4dd-rx4cf Container created 22m Normal AddedInterface pod/payload-processing-e2e-ipp-a-8c4b02-68c646b4dd-rx4cf Add eth0 [10.134.0.227/23] from ovn-kubernetes 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-8c4b02-c7cf79579-lj49f Add eth0 [10.134.0.226/23] from ovn-kubernetes 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-8c4b02-c7cf79579-lj49f Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal Created pod/payload-pre-processing-e2e-ipp-a-8c4b02-c7cf79579-lj49f Container created 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-a962d7 Scaled up replica set payload-processing-e2e-ipp-b-a962d7-55d9d5c8c4 from 0 to 1 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-a962d7-688dd94bc Error creating: pods "payload-pre-processing-e2e-ipp-b-a962d7-688dd94bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-a962d7: serviceaccount "payload-processing-e2e-ipp-b-a962d7" not found 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-a962d7-55d9d5c8c4 Error creating: pods "payload-processing-e2e-ipp-b-a962d7-55d9d5c8c4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-a962d7: serviceaccount "payload-processing-e2e-ipp-b-a962d7" not found 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-a962d7-688dd94bc Created pod: payload-pre-processing-e2e-ipp-b-a962d7-688dd94bc-xptcq 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-a962d7 Scaled up replica set payload-pre-processing-e2e-ipp-b-a962d7-688dd94bc from 0 to 1 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-a962d7-55d9d5c8c4 Created pod: payload-processing-e2e-ipp-b-a962d7-55d9d5c8c4-dfl49 22m Normal AddedInterface pod/payload-processing-e2e-ipp-b-a962d7-55d9d5c8c4-dfl49 Add eth0 [10.132.1.61/23] from ovn-kubernetes 22m Normal Pulled pod/payload-processing-e2e-ipp-b-a962d7-55d9d5c8c4-dfl49 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal Created pod/payload-processing-e2e-ipp-b-a962d7-55d9d5c8c4-dfl49 Container created 22m Normal Started pod/payload-processing-e2e-ipp-b-a962d7-55d9d5c8c4-dfl49 Container started 22m Normal Started pod/payload-pre-processing-e2e-ipp-b-a962d7-688dd94bc-xptcq Container started 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-a962d7-688dd94bc-xptcq Add eth0 [10.132.1.60/23] from ovn-kubernetes 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-a962d7-688dd94bc-xptcq Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal Created pod/payload-pre-processing-e2e-ipp-b-a962d7-688dd94bc-xptcq Container created 22m Normal AddedLabel gateway/e2e-ipp-a-b8f54c Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-b8f54c 22m Normal AddedInterface pod/e2e-ipp-a-b8f54c-openshift-default-69ccbf5c6c-mwkmt Add eth0 [10.134.0.228/23] from ovn-kubernetes 22m Normal SuccessfulCreate replicaset/e2e-ipp-a-b8f54c-openshift-default-69ccbf5c6c Created pod: e2e-ipp-a-b8f54c-openshift-default-69ccbf5c6c-mwkmt 22m Normal ScalingReplicaSet deployment/e2e-ipp-a-b8f54c-openshift-default Scaled up replica set e2e-ipp-a-b8f54c-openshift-default-69ccbf5c6c from 0 to 1 22m Normal Created pod/e2e-ipp-b-a962d7-openshift-default-6dbc7cd94c-625mm Container created 22m Normal Started pod/e2e-ipp-b-a962d7-openshift-default-6dbc7cd94c-625mm Container started 22m Normal Pulled pod/e2e-ipp-b-a962d7-openshift-default-6dbc7cd94c-625mm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-b8f54c Scaled up replica set payload-pre-processing-e2e-ipp-a-b8f54c-59d54cdb67 from 0 to 1 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-b8f54c-59d54cdb67 Created pod: payload-pre-processing-e2e-ipp-a-b8f54c-59d54cdb67-t4sjz 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-b8f54c-59d54cdb67 Error creating: pods "payload-pre-processing-e2e-ipp-a-b8f54c-59d54cdb67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-b8f54c: serviceaccount "payload-processing-e2e-ipp-a-b8f54c" not found 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-b8f54c Scaled up replica set payload-processing-e2e-ipp-a-b8f54c-77b9597456 from 0 to 1 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-b8f54c-77b9597456 Error creating: pods "payload-processing-e2e-ipp-a-b8f54c-77b9597456-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-b8f54c: serviceaccount "payload-processing-e2e-ipp-a-b8f54c" not found 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-b8f54c-77b9597456 Created pod: payload-processing-e2e-ipp-a-b8f54c-77b9597456-2htkw 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-b8f54c-59d54cdb67-t4sjz Add eth0 [10.132.1.62/23] from ovn-kubernetes 22m Normal Started pod/payload-pre-processing-e2e-ipp-a-b8f54c-59d54cdb67-t4sjz Container started 22m Normal Created pod/payload-pre-processing-e2e-ipp-a-b8f54c-59d54cdb67-t4sjz Container created 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-b8f54c-59d54cdb67-t4sjz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal Pulled pod/payload-processing-e2e-ipp-a-b8f54c-77b9597456-2htkw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal Created pod/payload-processing-e2e-ipp-a-b8f54c-77b9597456-2htkw Container created 22m Normal AddedInterface pod/payload-processing-e2e-ipp-a-b8f54c-77b9597456-2htkw Add eth0 [10.134.0.229/23] from ovn-kubernetes 22m Normal Started pod/payload-processing-e2e-ipp-a-b8f54c-77b9597456-2htkw Container started 22m Warning ProbeError pod/e2e-ipp-b-a962d7-openshift-default-6dbc7cd94c-625mm Startup probe error: Get "http://10.134.0.213:15021/healthz/ready": dial tcp 10.134.0.213:15021: connect: connection refused... 22m Warning Unhealthy pod/e2e-ipp-b-a962d7-openshift-default-6dbc7cd94c-625mm Startup probe failed: Get "http://10.134.0.213:15021/healthz/ready": dial tcp 10.134.0.213:15021: connect: connection refused 22m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7 Created pod: e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-hdcz8 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-3ee017-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 22m Normal AddedInterface pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-hdcz8 Add eth0 [10.133.0.146/23] from ovn-kubernetes 22m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-3ee017-openshift-default Scaled up replica set e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7 from 2 to 3 22m Normal Started pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-hdcz8 Container started 22m Normal Pulled pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-hdcz8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Created pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-hdcz8 Container created 22m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-a962d7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 22m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-a962d7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 22m Warning Unhealthy pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-hdcz8 Startup probe failed: Get "http://10.133.0.146:15021/healthz/ready": dial tcp 10.133.0.146:15021: connect: connection refused 22m Warning ProbeError pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-hdcz8 Startup probe error: Get "http://10.133.0.146:15021/healthz/ready": dial tcp 10.133.0.146:15021: connect: connection refused... 22m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-a962d7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Normal Created pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-bkj9z Container created 22m Normal Pulled pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-bkj9z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-02720a-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 22m Normal Started pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-bkj9z Container started 22m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-02720a-openshift-default-6b64cb69 Created pod: e2e-shared-b-w0-02720a-openshift-default-6b64cb69-bkj9z 22m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-02720a-openshift-default Scaled up replica set e2e-shared-b-w0-02720a-openshift-default-6b64cb69 from 2 to 3 22m Normal AddedInterface pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-bkj9z Add eth0 [10.133.0.148/23] from ovn-kubernetes 22m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-a962d7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Warning Unhealthy pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-bkj9z Startup probe failed: Get "http://10.133.0.148:15021/healthz/ready": dial tcp 10.133.0.148:15021: connect: connection refused 22m Warning ProbeError pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-bkj9z Startup probe error: Get "http://10.133.0.148:15021/healthz/ready": dial tcp 10.133.0.148:15021: connect: connection refused... 21m Normal AddedLabel gateway/e2e-ipp-a-83f120 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-83f120 21m Normal SuccessfulCreate replicaset/e2e-ipp-a-83f120-openshift-default-78b78cf899 Created pod: e2e-ipp-a-83f120-openshift-default-78b78cf899-x822r 21m Normal ScalingReplicaSet deployment/e2e-ipp-a-83f120-openshift-default Scaled up replica set e2e-ipp-a-83f120-openshift-default-78b78cf899 from 0 to 1 21m Normal Killing pod/payload-pre-processing-e2e-ipp-b-4796a9-d6c677c75-9zj9b Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-ipp-b-4796a9-6c55db49cd-kkpxj Stopping container payload-processing 21m Normal AddedInterface pod/e2e-ipp-a-83f120-openshift-default-78b78cf899-x822r Add eth0 [10.134.0.230/23] from ovn-kubernetes 21m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-83f120-57b979dcd7 Created pod: payload-pre-processing-e2e-ipp-a-83f120-57b979dcd7-zqf6r 21m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-83f120-57b979dcd7 Error creating: pods "payload-pre-processing-e2e-ipp-a-83f120-57b979dcd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-83f120: serviceaccount "payload-processing-e2e-ipp-a-83f120" not found 21m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-83f120 Scaled up replica set payload-processing-e2e-ipp-a-83f120-59d8965d64 from 0 to 1 21m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-83f120-59d8965d64 Created pod: payload-processing-e2e-ipp-a-83f120-59d8965d64-mksct 21m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-83f120-59d8965d64 Error creating: pods "payload-processing-e2e-ipp-a-83f120-59d8965d64-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-83f120: serviceaccount "payload-processing-e2e-ipp-a-83f120" not found 21m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-83f120 Scaled up replica set payload-pre-processing-e2e-ipp-a-83f120-57b979dcd7 from 0 to 1 21m Normal AddedInterface pod/payload-processing-e2e-ipp-a-83f120-59d8965d64-mksct Add eth0 [10.132.1.67/23] from ovn-kubernetes 21m Normal Pulled pod/payload-processing-e2e-ipp-a-83f120-59d8965d64-mksct Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-83f120-57b979dcd7-zqf6r Add eth0 [10.132.1.68/23] from ovn-kubernetes 21m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-83f120-57b979dcd7-zqf6r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 21m Normal Created pod/payload-pre-processing-e2e-ipp-a-83f120-57b979dcd7-zqf6r Container created 21m Normal Started pod/payload-processing-e2e-ipp-a-83f120-59d8965d64-mksct Container started 21m Normal Created pod/payload-processing-e2e-ipp-a-83f120-59d8965d64-mksct Container created 21m Normal Started pod/payload-pre-processing-e2e-ipp-a-83f120-57b979dcd7-zqf6r Container started 21m Normal Killing pod/payload-processing-e2e-ipp-a-c87bf1-85b4d8ffd8-jz456 Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-ipp-a-c87bf1-6bc965df8c-79jh7 Stopping container payload-pre-processing 21m Normal Pulled pod/e2e-ipp-b-85862e-openshift-default-7b794c9f49-6wxwm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Created pod/e2e-ipp-b-85862e-openshift-default-7b794c9f49-6wxwm Container created 21m Normal Started pod/e2e-ipp-b-85862e-openshift-default-7b794c9f49-6wxwm Container started 21m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7 Created pod: e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-9m5dt 21m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-3ee017-openshift-default Scaled up replica set e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7 from 3 to 4 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-3ee017-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-d68c2c-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 21m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-d68c2c-openshift-default Scaled up replica set e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f from 3 to 5 21m Normal AddedInterface pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-9m5dt Add eth0 [10.134.0.231/23] from ovn-kubernetes 21m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f Created pod: e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-n69t2 21m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f Created pod: e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-xpz6c 21m Normal Created pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-9m5dt Container created 21m Normal Started pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-9m5dt Container started 21m Normal Pulled pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-9m5dt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Started pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-n69t2 Container started 21m Normal AddedInterface pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-xpz6c Add eth0 [10.132.1.71/23] from ovn-kubernetes 21m Normal Pulled pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-n69t2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-n69t2 Add eth0 [10.133.0.150/23] from ovn-kubernetes 21m Normal Created pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-xpz6c Container created 21m Normal Created pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-n69t2 Container created 21m Normal Started pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-xpz6c Container started 21m Normal Pulled pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-xpz6c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Killing pod/e2e-ipp-a-c87bf1-openshift-default-648d4845d9-qx65t Stopping container istio-proxy 21m Warning Unhealthy pod/e2e-ipp-b-85862e-openshift-default-7b794c9f49-6wxwm Startup probe failed: Get "http://10.134.0.225:15021/healthz/ready": dial tcp 10.134.0.225:15021: connect: connection refused 21m Warning ProbeError pod/e2e-ipp-b-85862e-openshift-default-7b794c9f49-6wxwm Startup probe error: Get "http://10.134.0.225:15021/healthz/ready": dial tcp 10.134.0.225:15021: connect: connection refused... 21m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-af260f-openshift-default-846bb69f67 Created pod: e2e-shared-a-w1-af260f-openshift-default-846bb69f67-wkhfz 21m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-af260f-openshift-default-846bb69f67 Created pod: e2e-shared-a-w1-af260f-openshift-default-846bb69f67-ftzdt 21m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-af260f-openshift-default Scaled up replica set e2e-shared-a-w1-af260f-openshift-default-846bb69f67 from 2 to 4 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w1-af260f-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 21m Normal AddedInterface pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-wkhfz Add eth0 [10.133.0.151/23] from ovn-kubernetes 21m Normal Started pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-ftzdt Container started 21m Warning ProbeError pod/e2e-ipp-a-c87bf1-openshift-default-648d4845d9-qx65t Readiness probe error: HTTP probe failed with statuscode: 503... 21m Normal Pulled pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-wkhfz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Warning Unhealthy pod/e2e-ipp-a-c87bf1-openshift-default-648d4845d9-qx65t Readiness probe failed: HTTP probe failed with statuscode: 503 21m Normal Created pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-wkhfz Container created 21m Normal AddedInterface pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-ftzdt Add eth0 [10.132.1.73/23] from ovn-kubernetes 21m Normal Created pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-ftzdt Container created 21m Normal Pulled pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-ftzdt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Started pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-wkhfz Container started 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-83f120-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-83f120-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 21m Warning Unhealthy pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-xpz6c Startup probe failed: Get "http://10.132.1.71:15021/healthz/ready": dial tcp 10.132.1.71:15021: connect: connection refused 21m Warning ProbeError pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-n69t2 Startup probe error: Get "http://10.133.0.150:15021/healthz/ready": dial tcp 10.133.0.150:15021: connect: connection refused... 21m Warning ProbeError pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-xpz6c Startup probe error: Get "http://10.132.1.71:15021/healthz/ready": dial tcp 10.132.1.71:15021: connect: connection refused... 21m Warning ProbeError pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-9m5dt Startup probe error: Get "http://10.134.0.231:15021/healthz/ready": dial tcp 10.134.0.231:15021: connect: connection refused... 21m Warning Unhealthy pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-9m5dt Startup probe failed: Get "http://10.134.0.231:15021/healthz/ready": dial tcp 10.134.0.231:15021: connect: connection refused 21m Warning Unhealthy pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-n69t2 Startup probe failed: Get "http://10.133.0.150:15021/healthz/ready": dial tcp 10.133.0.150:15021: connect: connection refused 21m Normal Killing pod/payload-pre-processing-e2e-ipp-b-85862e-597bbb855-5jrdf Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-ipp-b-85862e-5759d9df58-t4vrn Stopping container payload-processing 21m Warning ProbeError pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-wkhfz Startup probe error: Get "http://10.133.0.151:15021/healthz/ready": dial tcp 10.133.0.151:15021: connect: connection refused... 21m Warning Unhealthy pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-wkhfz Startup probe failed: Get "http://10.133.0.151:15021/healthz/ready": dial tcp 10.133.0.151:15021: connect: connection refused 21m Warning ProbeError pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-ftzdt Startup probe error: Get "http://10.132.1.73:15021/healthz/ready": dial tcp 10.132.1.73:15021: connect: connection refused... 21m Warning Unhealthy pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-ftzdt Startup probe failed: Get "http://10.132.1.73:15021/healthz/ready": dial tcp 10.132.1.73:15021: connect: connection refused 21m Normal Killing pod/payload-processing-e2e-ipp-a-8c4b02-68c646b4dd-rx4cf Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-ipp-a-8c4b02-c7cf79579-lj49f Stopping container payload-pre-processing 21m Normal Killing pod/payload-pre-processing-e2e-ipp-b-a962d7-688dd94bc-xptcq Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-ipp-b-a962d7-55d9d5c8c4-dfl49 Stopping container payload-processing 21m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-4796a9-7fc7d467d8 Error creating: pods "payload-processing-e2e-ipp-b-4796a9-7fc7d467d8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-4796a9: serviceaccount "payload-processing-e2e-ipp-b-4796a9" not found 21m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-4796a9 Scaled up replica set payload-processing-e2e-ipp-b-4796a9-7fc7d467d8 from 0 to 1 21m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-4796a9-58585d8495 Error creating: pods "payload-pre-processing-e2e-ipp-b-4796a9-58585d8495-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-4796a9: serviceaccount "payload-processing-e2e-ipp-b-4796a9" not found 21m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-4796a9 Scaled up replica set payload-pre-processing-e2e-ipp-b-4796a9-58585d8495 from 0 to 1 21m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-4796a9-58585d8495 Created pod: payload-pre-processing-e2e-ipp-b-4796a9-58585d8495-rt4r8 21m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-4796a9-7fc7d467d8 Created pod: payload-processing-e2e-ipp-b-4796a9-7fc7d467d8-nkkxj 21m Normal Killing pod/e2e-ipp-a-8c4b02-openshift-default-679994b96f-7wssh Stopping container istio-proxy 21m Normal Started pod/payload-pre-processing-e2e-ipp-b-4796a9-58585d8495-rt4r8 Container started 21m Normal AddedInterface pod/payload-processing-e2e-ipp-b-4796a9-7fc7d467d8-nkkxj Add eth0 [10.134.0.233/23] from ovn-kubernetes 21m Normal Created pod/payload-pre-processing-e2e-ipp-b-4796a9-58585d8495-rt4r8 Container created 21m Normal Created pod/payload-processing-e2e-ipp-b-4796a9-7fc7d467d8-nkkxj Container created 21m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-4796a9-58585d8495-rt4r8 Add eth0 [10.134.0.232/23] from ovn-kubernetes 21m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-4796a9-58585d8495-rt4r8 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 21m Normal Started pod/payload-processing-e2e-ipp-b-4796a9-7fc7d467d8-nkkxj Container started 21m Normal Pulled pod/payload-processing-e2e-ipp-b-4796a9-7fc7d467d8-nkkxj Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 21m Normal Killing pod/e2e-ipp-b-a962d7-openshift-default-6dbc7cd94c-625mm Stopping container istio-proxy 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-85862e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-85862e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Warning ProbeError pod/e2e-ipp-b-a962d7-openshift-default-6dbc7cd94c-625mm Readiness probe error: Get "http://10.134.0.213:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 21m Warning Unhealthy pod/e2e-ipp-b-a962d7-openshift-default-6dbc7cd94c-625mm Readiness probe failed: Get "http://10.134.0.213:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 20m Normal Created pod/e2e-ipp-b-4796a9-openshift-default-75cd4d45f7-gm2vv Container created 20m Normal Killing pod/payload-pre-processing-e2e-ipp-a-b8f54c-59d54cdb67-t4sjz Stopping container payload-pre-processing 20m Normal Started pod/e2e-ipp-b-4796a9-openshift-default-75cd4d45f7-gm2vv Container started 20m Normal Killing pod/payload-processing-e2e-ipp-a-b8f54c-77b9597456-2htkw Stopping container payload-processing 20m Normal Pulled pod/e2e-ipp-b-4796a9-openshift-default-75cd4d45f7-gm2vv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Killing pod/payload-processing-e2e-shared-a-w6-06a95f-6c459f96db-6pb7q Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-shared-a-w6-06a95f-7c5bb67c55-dkr8j Stopping container payload-pre-processing 20m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-4796a9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 20m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-4796a9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 20m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-85862e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 20m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-85862e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 20m Warning Unhealthy pod/e2e-ipp-b-4796a9-openshift-default-75cd4d45f7-gm2vv Startup probe failed: Get "http://10.134.0.220:15021/healthz/ready": dial tcp 10.134.0.220:15021: connect: connection refused 20m Warning ProbeError pod/e2e-ipp-b-4796a9-openshift-default-75cd4d45f7-gm2vv Startup probe error: Get "http://10.134.0.220:15021/healthz/ready": dial tcp 10.134.0.220:15021: connect: connection refused... 20m Normal AddedLabel gateway/e2e-ipp-a-80248b Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-80248b 20m Normal ScalingReplicaSet deployment/e2e-ipp-a-80248b-openshift-default Scaled up replica set e2e-ipp-a-80248b-openshift-default-5677b677f5 from 0 to 1 20m Normal SuccessfulCreate replicaset/e2e-ipp-a-80248b-openshift-default-5677b677f5 Created pod: e2e-ipp-a-80248b-openshift-default-5677b677f5-lx6cs 20m Normal AddedInterface pod/e2e-ipp-a-80248b-openshift-default-5677b677f5-lx6cs Add eth0 [10.134.0.234/23] from ovn-kubernetes 20m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-85862e Scaled up replica set payload-processing-e2e-ipp-b-85862e-69678cb8f6 from 0 to 1 20m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-85862e-6c4464c9 Created pod: payload-pre-processing-e2e-ipp-b-85862e-6c4464c9-9xtht 20m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-85862e Scaled up replica set payload-pre-processing-e2e-ipp-b-85862e-6c4464c9 from 0 to 1 20m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-85862e-69678cb8f6 Created pod: payload-processing-e2e-ipp-b-85862e-69678cb8f6-hkxqw 20m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-85862e-6c4464c9 Error creating: pods "payload-pre-processing-e2e-ipp-b-85862e-6c4464c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-85862e: serviceaccount "payload-processing-e2e-ipp-b-85862e" not found 20m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-85862e-69678cb8f6 Error creating: pods "payload-processing-e2e-ipp-b-85862e-69678cb8f6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-85862e: serviceaccount "payload-processing-e2e-ipp-b-85862e" not found 20m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-80248b Scaled up replica set payload-processing-e2e-ipp-a-80248b-79b8c55464 from 0 to 1 20m Normal Started pod/payload-processing-e2e-ipp-b-85862e-69678cb8f6-hkxqw Container started 20m Normal AddedInterface pod/payload-processing-e2e-ipp-b-85862e-69678cb8f6-hkxqw Add eth0 [10.134.0.235/23] from ovn-kubernetes 20m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-80248b Scaled up replica set payload-pre-processing-e2e-ipp-a-80248b-5bd96b4c59 from 0 to 1 20m Normal Started pod/payload-pre-processing-e2e-ipp-b-85862e-6c4464c9-9xtht Container started 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-85862e-6c4464c9-9xtht Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-85862e-6c4464c9-9xtht Add eth0 [10.132.1.81/23] from ovn-kubernetes 20m Normal Created pod/payload-processing-e2e-ipp-b-85862e-69678cb8f6-hkxqw Container created 20m Normal Pulled pod/payload-processing-e2e-ipp-b-85862e-69678cb8f6-hkxqw Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal Created pod/payload-pre-processing-e2e-ipp-b-85862e-6c4464c9-9xtht Container created 20m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-80248b-79b8c55464 Error creating: pods "payload-processing-e2e-ipp-a-80248b-79b8c55464-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-80248b: serviceaccount "payload-processing-e2e-ipp-a-80248b" not found 20m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-80248b-5bd96b4c59 Created pod: payload-pre-processing-e2e-ipp-a-80248b-5bd96b4c59-zqqfc 20m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-80248b-5bd96b4c59 Error creating: pods "payload-pre-processing-e2e-ipp-a-80248b-5bd96b4c59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-80248b: serviceaccount "payload-processing-e2e-ipp-a-80248b" not found 20m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-80248b-79b8c55464 Created pod: payload-processing-e2e-ipp-a-80248b-79b8c55464-d7k5d 20m Normal Started pod/e2e-ipp-a-b8f54c-openshift-default-69ccbf5c6c-mwkmt Container started 20m Normal AddedInterface pod/payload-processing-e2e-ipp-a-80248b-79b8c55464-d7k5d Add eth0 [10.134.0.237/23] from ovn-kubernetes 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-80248b-5bd96b4c59-zqqfc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 20m Normal Created pod/payload-processing-e2e-ipp-a-80248b-79b8c55464-d7k5d Container created 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-80248b-5bd96b4c59-zqqfc Add eth0 [10.134.0.236/23] from ovn-kubernetes 20m Normal Created pod/e2e-ipp-a-b8f54c-openshift-default-69ccbf5c6c-mwkmt Container created 20m Normal Pulled pod/payload-processing-e2e-ipp-a-80248b-79b8c55464-d7k5d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 20m Normal Pulled pod/e2e-ipp-a-b8f54c-openshift-default-69ccbf5c6c-mwkmt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Started pod/payload-pre-processing-e2e-ipp-a-80248b-5bd96b4c59-zqqfc Container started 20m Normal Started pod/payload-processing-e2e-ipp-a-80248b-79b8c55464-d7k5d Container started 20m Normal Created pod/payload-pre-processing-e2e-ipp-a-80248b-5bd96b4c59-zqqfc Container created 20m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-b8f54c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 20m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-b8f54c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 20m Warning ProbeError pod/e2e-ipp-a-b8f54c-openshift-default-69ccbf5c6c-mwkmt Startup probe error: Get "http://10.134.0.228:15021/healthz/ready": dial tcp 10.134.0.228:15021: connect: connection refused... 20m Warning Unhealthy pod/e2e-ipp-a-b8f54c-openshift-default-69ccbf5c6c-mwkmt Startup probe failed: Get "http://10.134.0.228:15021/healthz/ready": dial tcp 10.134.0.228:15021: connect: connection refused 20m Normal Killing pod/payload-pre-processing-e2e-ipp-b-4796a9-58585d8495-rt4r8 Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-ipp-b-4796a9-7fc7d467d8-nkkxj Stopping container payload-processing 20m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-4796a9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-4796a9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Normal Killing pod/e2e-shared-a-w6-06a95f-openshift-default-74f57588cc-xlqvh Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-a-w6-06a95f-openshift-default-74f57588cc-f4bjj Stopping container istio-proxy 20m Warning ProbeError pod/e2e-shared-a-w6-06a95f-openshift-default-74f57588cc-xlqvh Readiness probe error: HTTP probe failed with statuscode: 503... 20m Warning Unhealthy pod/e2e-shared-a-w6-06a95f-openshift-default-74f57588cc-xlqvh Readiness probe failed: HTTP probe failed with statuscode: 503 20m Normal Killing pod/e2e-ipp-b-4796a9-openshift-default-75cd4d45f7-gm2vv Stopping container istio-proxy 20m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-b8f54c Scaled up replica set payload-pre-processing-e2e-ipp-a-b8f54c-7c969c8947 from 0 to 1 20m Warning Unhealthy pod/e2e-shared-a-w6-06a95f-openshift-default-74f57588cc-f4bjj Readiness probe failed: HTTP probe failed with statuscode: 503 20m Normal Killing pod/payload-pre-processing-e2e-ipp-a-83f120-57b979dcd7-zqf6r Stopping container payload-pre-processing 20m Warning ProbeError pod/e2e-shared-a-w6-06a95f-openshift-default-74f57588cc-f4bjj Readiness probe error: HTTP probe failed with statuscode: 503... 20m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-b8f54c Scaled up replica set payload-processing-e2e-ipp-a-b8f54c-6dd8f8fcbb from 0 to 1 20m Normal Killing pod/payload-processing-e2e-ipp-a-83f120-59d8965d64-mksct Stopping container payload-processing 20m Normal AddedLabel gateway/e2e-ipp-b-75b68d Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-75b68d 20m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-b8f54c-6dd8f8fcbb Error creating: pods "payload-processing-e2e-ipp-a-b8f54c-6dd8f8fcbb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-b8f54c: serviceaccount "payload-processing-e2e-ipp-a-b8f54c" not found 20m Normal ScalingReplicaSet deployment/e2e-ipp-b-75b68d-openshift-default Scaled up replica set e2e-ipp-b-75b68d-openshift-default-5f58db85f7 from 0 to 1 20m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-b8f54c-6dd8f8fcbb Created pod: payload-processing-e2e-ipp-a-b8f54c-6dd8f8fcbb-8v49p 20m Normal SuccessfulCreate replicaset/e2e-ipp-b-75b68d-openshift-default-5f58db85f7 Created pod: e2e-ipp-b-75b68d-openshift-default-5f58db85f7-74lsn 20m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-b8f54c-7c969c8947 Error creating: pods "payload-pre-processing-e2e-ipp-a-b8f54c-7c969c8947-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-b8f54c: serviceaccount "payload-processing-e2e-ipp-a-b8f54c" not found 20m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-b8f54c-7c969c8947 Created pod: payload-pre-processing-e2e-ipp-a-b8f54c-7c969c8947-9wpb5 20m Normal Started pod/payload-processing-e2e-ipp-a-b8f54c-6dd8f8fcbb-8v49p Container started 20m Normal Created pod/payload-processing-e2e-ipp-a-b8f54c-6dd8f8fcbb-8v49p Container created 20m Normal Pulled pod/payload-processing-e2e-ipp-a-b8f54c-6dd8f8fcbb-8v49p Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal AddedInterface pod/payload-processing-e2e-ipp-a-b8f54c-6dd8f8fcbb-8v49p Add eth0 [10.134.0.240/23] from ovn-kubernetes 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-b8f54c-7c969c8947-9wpb5 Add eth0 [10.134.0.239/23] from ovn-kubernetes 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-b8f54c-7c969c8947-9wpb5 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal Started pod/payload-pre-processing-e2e-ipp-a-b8f54c-7c969c8947-9wpb5 Container started 20m Normal AddedInterface pod/e2e-ipp-b-75b68d-openshift-default-5f58db85f7-74lsn Add eth0 [10.134.0.241/23] from ovn-kubernetes 20m Normal Created pod/payload-pre-processing-e2e-ipp-a-b8f54c-7c969c8947-9wpb5 Container created 20m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-6fc6bd-openshift-default New size: 2; reason: All metrics below target 20m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-6fc6bd-openshift-default Scaled down replica set e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89 from 3 to 2 20m Normal SuccessfulDelete replicaset/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89 Deleted pod: e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-2dhcj 20m Normal Killing pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-2dhcj Stopping container istio-proxy 20m Normal Killing pod/payload-pre-processing-e2e-shared-b-w6-db7419-79447f7f59-h9nmc Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-shared-b-w6-db7419-84c96f7884-pqpn2 Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-shared-a-w3-aaa3c6-64445b56bc-9wn9m Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-shared-a-w3-aaa3c6-845674cd68-8fgr7 Stopping container payload-processing 20m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-75b68d-9f999d45d Error creating: pods "payload-processing-e2e-ipp-b-75b68d-9f999d45d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-75b68d: serviceaccount "payload-processing-e2e-ipp-b-75b68d" not found 20m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-75b68d Scaled up replica set payload-pre-processing-e2e-ipp-b-75b68d-77657dc59 from 0 to 1 20m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-75b68d-77657dc59 Error creating: pods "payload-pre-processing-e2e-ipp-b-75b68d-77657dc59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-75b68d: serviceaccount "payload-processing-e2e-ipp-b-75b68d" not found 20m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-75b68d Scaled up replica set payload-processing-e2e-ipp-b-75b68d-9f999d45d from 0 to 1 20m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-75b68d-9f999d45d Created pod: payload-processing-e2e-ipp-b-75b68d-9f999d45d-9qwbt 20m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-75b68d-77657dc59 Created pod: payload-pre-processing-e2e-ipp-b-75b68d-77657dc59-xt7mh 20m Normal AddedInterface pod/payload-processing-e2e-ipp-b-75b68d-9f999d45d-9qwbt Add eth0 [10.134.0.243/23] from ovn-kubernetes 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-75b68d-77657dc59-xt7mh Add eth0 [10.134.0.242/23] from ovn-kubernetes 20m Normal Started pod/payload-processing-e2e-ipp-b-75b68d-9f999d45d-9qwbt Container started 20m Normal Created pod/payload-processing-e2e-ipp-b-75b68d-9f999d45d-9qwbt Container created 20m Normal Pulled pod/payload-processing-e2e-ipp-b-75b68d-9f999d45d-9qwbt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-75b68d-77657dc59-xt7mh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 20m Normal Started pod/payload-pre-processing-e2e-ipp-b-75b68d-77657dc59-xt7mh Container started 20m Normal Created pod/payload-pre-processing-e2e-ipp-b-75b68d-77657dc59-xt7mh Container created 19m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-b8f54c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 19m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-b8f54c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 19m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-9aafe6-openshift-default Scaled down replica set e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85 from 3 to 2 19m Normal Killing pod/payload-processing-e2e-shared-a-w4-f8ca96-576b496c86-rkx2m Stopping container payload-processing 19m Normal Killing pod/payload-pre-processing-e2e-shared-a-w4-f8ca96-696c58bc6c-4dw72 Stopping container payload-pre-processing 19m Normal Killing pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-2t2kf Stopping container istio-proxy 19m Normal SuccessfulDelete replicaset/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85 Deleted pod: e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-2t2kf 19m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w5-9aafe6-openshift-default New size: 2; reason: All metrics below target 19m Normal ScalingReplicaSet deployment/e2e-ipp-b-63e89c-openshift-default Scaled up replica set e2e-ipp-b-63e89c-openshift-default-6d97c8847c from 0 to 1 19m Normal SuccessfulCreate replicaset/e2e-ipp-b-63e89c-openshift-default-6d97c8847c Created pod: e2e-ipp-b-63e89c-openshift-default-6d97c8847c-8bqfm 19m Normal AddedLabel gateway/e2e-ipp-b-63e89c Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-63e89c 19m Normal Killing pod/e2e-shared-b-w6-db7419-openshift-default-9dd55d5bc-q2xvw Stopping container istio-proxy 19m Normal AddedInterface pod/e2e-ipp-b-63e89c-openshift-default-6d97c8847c-8bqfm Add eth0 [10.132.1.84/23] from ovn-kubernetes 19m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-63e89c Scaled up replica set payload-processing-e2e-ipp-b-63e89c-745667d778 from 0 to 1 19m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-63e89c-745667d778 Error creating: pods "payload-processing-e2e-ipp-b-63e89c-745667d778-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-63e89c: serviceaccount "payload-processing-e2e-ipp-b-63e89c" not found 19m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-63e89c Scaled up replica set payload-pre-processing-e2e-ipp-b-63e89c-784f845595 from 0 to 1 19m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-63e89c-784f845595 Error creating: pods "payload-pre-processing-e2e-ipp-b-63e89c-784f845595-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-63e89c: serviceaccount "payload-processing-e2e-ipp-b-63e89c" not found 19m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-63e89c-745667d778 Created pod: payload-processing-e2e-ipp-b-63e89c-745667d778-c8x5k 19m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-63e89c-784f845595 Created pod: payload-pre-processing-e2e-ipp-b-63e89c-784f845595-7vhc9 19m Normal AddedInterface pod/payload-processing-e2e-ipp-b-63e89c-745667d778-c8x5k Add eth0 [10.134.0.246/23] from ovn-kubernetes 19m Normal Started pod/payload-processing-e2e-ipp-b-63e89c-745667d778-c8x5k Container started 19m Normal Started pod/payload-pre-processing-e2e-ipp-b-63e89c-784f845595-7vhc9 Container started 19m Normal Created pod/payload-pre-processing-e2e-ipp-b-63e89c-784f845595-7vhc9 Container created 19m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-63e89c-784f845595-7vhc9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 19m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-63e89c-784f845595-7vhc9 Add eth0 [10.134.0.245/23] from ovn-kubernetes 19m Normal Created pod/payload-processing-e2e-ipp-b-63e89c-745667d778-c8x5k Container created 19m Normal Pulled pod/payload-processing-e2e-ipp-b-63e89c-745667d778-c8x5k Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 19m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-9jn89 Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 19m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-9jn89 Add eth0 [10.133.0.154/23] from ovn-kubernetes 19m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 3 19m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-9jn89 Container created 19m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-9jn89 Container started 19m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: cpu resource utilization (percentage of request) above target 19m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-9jn89 19m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-9jn89 Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-9jn89 Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w3-aaa3c6-64445b56bc-9wn9m Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-processing-e2e-shared-a-w3-aaa3c6-845674cd68-8fgr7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/payload-pre-processing-e2e-worker-w6-784e7a-5bd4bc7d47-cddfw Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-worker-w6-784e7a-79d748c8bb-fbgzx Stopping container payload-processing 19m Warning ProbeError pod/e2e-ipp-a-83f120-openshift-default-78b78cf899-x822r Readiness probe error: Get "http://10.134.0.230:15021/healthz/ready": read tcp 10.134.0.2:48166->10.134.0.230:15021: read: connection reset by peer... 19m Normal Pulled pod/e2e-ipp-a-83f120-openshift-default-78b78cf899-x822r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Warning Unhealthy pod/e2e-ipp-a-83f120-openshift-default-78b78cf899-x822r Readiness probe failed: Get "http://10.134.0.230:15021/healthz/ready": read tcp 10.134.0.2:48166->10.134.0.230:15021: read: connection reset by peer 19m Normal Created pod/e2e-ipp-a-83f120-openshift-default-78b78cf899-x822r Container created 19m Normal Started pod/e2e-ipp-a-83f120-openshift-default-78b78cf899-x822r Container started 19m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w4-f8ca96-696c58bc6c-4dw72 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-processing-e2e-shared-a-w4-f8ca96-576b496c86-rkx2m Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning ProbeError pod/e2e-ipp-a-83f120-openshift-default-78b78cf899-x822r Startup probe error: Get "http://10.134.0.230:15021/healthz/ready": dial tcp 10.134.0.230:15021: connect: connection refused... 19m Warning Unhealthy pod/e2e-ipp-a-83f120-openshift-default-78b78cf899-x822r Startup probe failed: Get "http://10.134.0.230:15021/healthz/ready": dial tcp 10.134.0.230:15021: connect: connection refused 19m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-83f120-75b867c88c Error creating: pods "payload-pre-processing-e2e-ipp-a-83f120-75b867c88c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-83f120: serviceaccount "payload-processing-e2e-ipp-a-83f120" not found 19m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-83f120-75b867c88c Created pod: payload-pre-processing-e2e-ipp-a-83f120-75b867c88c-dcmh5 19m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-83f120 Scaled up replica set payload-pre-processing-e2e-ipp-a-83f120-75b867c88c from 0 to 1 19m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-83f120-f467f9994 Error creating: pods "payload-processing-e2e-ipp-a-83f120-f467f9994-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-83f120: serviceaccount "payload-processing-e2e-ipp-a-83f120" not found 19m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-83f120-f467f9994 Created pod: payload-processing-e2e-ipp-a-83f120-f467f9994-b9pr9 19m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-83f120 Scaled up replica set payload-processing-e2e-ipp-a-83f120-f467f9994 from 0 to 1 19m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-83f120-75b867c88c-dcmh5 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 19m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-83f120-75b867c88c-dcmh5 Add eth0 [10.132.1.97/23] from ovn-kubernetes 19m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w6-784e7a-5bd4bc7d47-cddfw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal AddedInterface pod/payload-processing-e2e-ipp-a-83f120-f467f9994-b9pr9 Add eth0 [10.132.1.98/23] from ovn-kubernetes 19m Normal Pulled pod/payload-processing-e2e-ipp-a-83f120-f467f9994-b9pr9 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 19m Warning Unhealthy pod/payload-processing-e2e-worker-w6-784e7a-79d748c8bb-fbgzx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Created pod/payload-pre-processing-e2e-ipp-a-83f120-75b867c88c-dcmh5 Container created 19m Normal Started pod/payload-processing-e2e-ipp-a-83f120-f467f9994-b9pr9 Container started 19m Normal Started pod/payload-pre-processing-e2e-ipp-a-83f120-75b867c88c-dcmh5 Container started 19m Normal Created pod/payload-processing-e2e-ipp-a-83f120-f467f9994-b9pr9 Container created 19m Normal Killing pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-mb2dl Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-bn6sm Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-ns4gv Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-n2ttv Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-mvw5s Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-x9bxc Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-glsxl Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-rt6nd Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-vpr7b Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w4-f8ca96-openshift-default-7b49547d65-ctqgd Stopping container istio-proxy 19m Normal Killing pod/payload-pre-processing-e2e-shared-b-w4-c933cb-786d7ff8b7-zhqxg Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-shared-b-w4-c933cb-fc7cf9456-g8t8z Stopping container payload-processing 19m Normal Killing pod/e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9-qmc4m Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9-x9gf6 Stopping container istio-proxy 19m Warning Unhealthy pod/e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9-qmc4m Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning ProbeError pod/e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9-qmc4m Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning Unhealthy pod/e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9-x9gf6 Readiness probe failed: Get "http://10.132.1.26:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 19m Warning ProbeError pod/e2e-shared-a-w3-aaa3c6-openshift-default-7cf94db6f9-x9gf6 Readiness probe error: Get "http://10.132.1.26:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 18m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-3bd0f6-openshift-default New size: 3; reason: All metrics below target 18m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-3bd0f6-openshift-default Scaled down replica set e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77 from 4 to 3 18m Normal Killing pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-nn958 Stopping container istio-proxy 18m Normal SuccessfulDelete replicaset/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77 Deleted pod: e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-nn958 18m Normal Killing pod/payload-processing-e2e-shared-b-w3-3bd0f6-c58c9678-lhc7t Stopping container payload-processing 18m Normal Killing pod/payload-pre-processing-e2e-shared-b-w3-3bd0f6-858c665c67-btrf6 Stopping container payload-pre-processing 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-83f120-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-83f120-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Normal Killing pod/payload-pre-processing-e2e-ipp-a-80248b-5bd96b4c59-zqqfc Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-ipp-a-80248b-79b8c55464-d7k5d Stopping container payload-processing 18m Warning Unhealthy pod/e2e-ipp-a-80248b-openshift-default-5677b677f5-lx6cs Readiness probe failed: Get "http://10.134.0.234:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 18m Warning ProbeError pod/e2e-ipp-a-80248b-openshift-default-5677b677f5-lx6cs Readiness probe error: Get "http://10.134.0.234:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 18m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w4-c933cb-786d7ff8b7-zhqxg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-processing-e2e-shared-b-w4-c933cb-fc7cf9456-g8t8z Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Normal Started pod/e2e-ipp-a-80248b-openshift-default-5677b677f5-lx6cs Container started 18m Normal Created pod/e2e-ipp-a-80248b-openshift-default-5677b677f5-lx6cs Container created 18m Normal Pulled pod/e2e-ipp-a-80248b-openshift-default-5677b677f5-lx6cs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Normal Killing pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-6mjhh Stopping container istio-proxy 18m Normal Killing pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-shmwc Stopping container istio-proxy 18m Normal Killing pod/e2e-shared-b-w3-3bd0f6-openshift-default-69c4758d77-74t4k Stopping container istio-proxy 18m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-920032-openshift-default New size: 3; reason: All metrics below target 18m Normal SuccessfulDelete replicaset/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67 Deleted pod: e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-fzz2k 18m Normal Killing pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-fzz2k Stopping container istio-proxy 18m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-920032-openshift-default Scaled down replica set e2e-shared-a-w0-920032-openshift-default-5b5f9cc67 from 4 to 3 18m Warning Unhealthy pod/e2e-ipp-a-80248b-openshift-default-5677b677f5-lx6cs Startup probe failed: Get "http://10.134.0.234:15021/healthz/ready": dial tcp 10.134.0.234:15021: connect: connection refused 18m Warning ProbeError pod/e2e-ipp-a-80248b-openshift-default-5677b677f5-lx6cs Startup probe error: Get "http://10.134.0.234:15021/healthz/ready": dial tcp 10.134.0.234:15021: connect: connection refused... 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-80248b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-80248b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Normal Killing pod/payload-pre-processing-e2e-worker-w3-43b37d-74d9bdb487-qtfzw Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-ipp-b-75b68d-9f999d45d-9qwbt Stopping container payload-processing 18m Normal Killing pod/payload-processing-e2e-worker-w3-43b37d-fd685c7db-qkngc Stopping container payload-processing 18m Normal Killing pod/payload-pre-processing-e2e-ipp-b-75b68d-77657dc59-xt7mh Stopping container payload-pre-processing 18m Warning Unhealthy pod/payload-processing-e2e-shared-b-w3-3bd0f6-c58c9678-lhc7t Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w3-3bd0f6-858c665c67-btrf6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Normal Killing pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-tf78l Stopping container istio-proxy 18m Normal Killing pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-ssfqt Stopping container istio-proxy 18m Warning Unhealthy pod/payload-pre-processing-e2e-ipp-b-75b68d-77657dc59-xt7mh Readiness probe failed: timeout: failed to connect service "10.134.0.242:50052" within 1s: context deadline exceeded 18m Warning Unhealthy pod/payload-processing-e2e-ipp-b-75b68d-9f999d45d-9qwbt Readiness probe failed: timeout: failed to connect service "10.134.0.243:50052" within 1s: context deadline exceeded 18m Warning ProbeError pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-ssfqt Readiness probe error: HTTP probe failed with statuscode: 503... 18m Warning Unhealthy pod/e2e-worker-w6-784e7a-openshift-default-595898bd5c-ssfqt Readiness probe failed: HTTP probe failed with statuscode: 503 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-80248b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-80248b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 18m Normal Pulled pod/e2e-ipp-b-63e89c-openshift-default-6d97c8847c-8bqfm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Warning Unhealthy pod/e2e-ipp-b-75b68d-openshift-default-5f58db85f7-74lsn Readiness probe failed: Get "http://10.134.0.241:15021/healthz/ready": read tcp 10.134.0.2:44030->10.134.0.241:15021: read: connection reset by peer 18m Warning ProbeError pod/e2e-ipp-b-75b68d-openshift-default-5f58db85f7-74lsn Readiness probe error: Get "http://10.134.0.241:15021/healthz/ready": read tcp 10.134.0.2:44030->10.134.0.241:15021: read: connection reset by peer... 18m Normal Created pod/e2e-ipp-b-63e89c-openshift-default-6d97c8847c-8bqfm Container created 18m Normal Started pod/e2e-ipp-b-63e89c-openshift-default-6d97c8847c-8bqfm Container started 18m Normal Killing pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-55kkj Stopping container istio-proxy 18m Normal Created pod/e2e-ipp-b-75b68d-openshift-default-5f58db85f7-74lsn Container created 18m Normal Started pod/e2e-ipp-b-75b68d-openshift-default-5f58db85f7-74lsn Container started 18m Normal Pulled pod/e2e-ipp-b-75b68d-openshift-default-5f58db85f7-74lsn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Normal Killing pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-2kr84 Stopping container istio-proxy 18m Normal Killing pod/payload-pre-processing-e2e-ipp-b-63e89c-784f845595-7vhc9 Stopping container payload-pre-processing 18m Normal Killing pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-cs6kx Stopping container istio-proxy 18m Normal Killing pod/payload-processing-e2e-ipp-b-63e89c-745667d778-c8x5k Stopping container payload-processing 18m Normal Killing pod/e2e-shared-b-w4-c933cb-openshift-default-9986db7d7-jtp6w Stopping container istio-proxy 18m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-80248b-5c58666f59 Error creating: pods "payload-pre-processing-e2e-ipp-a-80248b-5c58666f59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-80248b: serviceaccount "payload-processing-e2e-ipp-a-80248b" not found 18m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-80248b-5c58666f59 Created pod: payload-pre-processing-e2e-ipp-a-80248b-5c58666f59-csjml 18m Warning ProbeError pod/e2e-ipp-b-63e89c-openshift-default-6d97c8847c-8bqfm Startup probe error: Get "http://10.132.1.84:15021/healthz/ready": dial tcp 10.132.1.84:15021: connect: connection refused... 18m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-80248b Scaled up replica set payload-pre-processing-e2e-ipp-a-80248b-5c58666f59 from 0 to 1 18m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-80248b-5dfc48b658 Error creating: pods "payload-processing-e2e-ipp-a-80248b-5dfc48b658-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-80248b: serviceaccount "payload-processing-e2e-ipp-a-80248b" not found 18m Warning Unhealthy pod/e2e-ipp-b-63e89c-openshift-default-6d97c8847c-8bqfm Startup probe failed: Get "http://10.132.1.84:15021/healthz/ready": dial tcp 10.132.1.84:15021: connect: connection refused 18m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-80248b Scaled up replica set payload-processing-e2e-ipp-a-80248b-5dfc48b658 from 0 to 1 18m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-80248b-5dfc48b658 Created pod: payload-processing-e2e-ipp-a-80248b-5dfc48b658-bwlk6 18m Warning ProbeError pod/e2e-ipp-b-75b68d-openshift-default-5f58db85f7-74lsn Startup probe error: Get "http://10.134.0.241:15021/healthz/ready": dial tcp 10.134.0.241:15021: connect: connection refused... 18m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-80248b-5c58666f59-csjml Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 18m Normal Created pod/payload-processing-e2e-ipp-a-80248b-5dfc48b658-bwlk6 Container created 18m Normal Pulled pod/payload-processing-e2e-ipp-a-80248b-5dfc48b658-bwlk6 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 18m Normal AddedInterface pod/payload-processing-e2e-ipp-a-80248b-5dfc48b658-bwlk6 Add eth0 [10.134.1.12/23] from ovn-kubernetes 18m Warning Unhealthy pod/e2e-ipp-b-75b68d-openshift-default-5f58db85f7-74lsn Startup probe failed: Get "http://10.134.0.241:15021/healthz/ready": dial tcp 10.134.0.241:15021: connect: connection refused 18m Normal Started pod/payload-pre-processing-e2e-ipp-a-80248b-5c58666f59-csjml Container started 18m Normal Started pod/payload-processing-e2e-ipp-a-80248b-5dfc48b658-bwlk6 Container started 18m Normal Created pod/payload-pre-processing-e2e-ipp-a-80248b-5c58666f59-csjml Container created 18m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-80248b-5c58666f59-csjml Add eth0 [10.134.1.11/23] from ovn-kubernetes 18m Warning Unhealthy pod/payload-processing-e2e-worker-w3-43b37d-fd685c7db-qkngc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w3-43b37d-74d9bdb487-qtfzw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-75b68d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-75b68d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Normal Killing pod/payload-pre-processing-e2e-worker-w4-08ebee-7bcfbb7487-9x5xj Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-worker-w4-08ebee-5f548676d4-st67q Stopping container payload-processing 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-63e89c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-63e89c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-eb02df-openshift-default Scaled down replica set e2e-shared-a-w2-eb02df-openshift-default-5c45c545c from 3 to 2 18m Normal SuccessfulDelete replicaset/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c Deleted pod: e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-mbzm5 18m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-eb02df-openshift-default New size: 2; reason: All metrics below target 18m Normal Killing pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-mbzm5 Stopping container istio-proxy 18m Normal Killing pod/payload-processing-e2e-ipp-a-b8f54c-6dd8f8fcbb-8v49p Stopping container payload-processing 18m Normal Killing pod/payload-processing-e2e-ipp-a-83f120-f467f9994-b9pr9 Stopping container payload-processing 18m Normal Killing pod/payload-pre-processing-e2e-ipp-a-83f120-75b867c88c-dcmh5 Stopping container payload-pre-processing 18m Normal Killing pod/payload-pre-processing-e2e-ipp-a-b8f54c-7c969c8947-9wpb5 Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-ipp-a-7be71d-85b85b8ffd-d77xb Stopping container payload-processing 18m Normal Killing pod/payload-pre-processing-e2e-ipp-a-7be71d-865cd9bc7f-52m9d Stopping container payload-pre-processing 18m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-75b68d-ddbc7885b Error creating: pods "payload-processing-e2e-ipp-b-75b68d-ddbc7885b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-75b68d: serviceaccount "payload-processing-e2e-ipp-b-75b68d" not found 18m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-75b68d Scaled up replica set payload-processing-e2e-ipp-b-75b68d-ddbc7885b from 0 to 1 18m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-75b68d Scaled up replica set payload-pre-processing-e2e-ipp-b-75b68d-59747b7fb7 from 0 to 1 18m Normal ScalingReplicaSet deployment/e2e-ipp-b-9d6d4e-openshift-default Scaled up replica set e2e-ipp-b-9d6d4e-openshift-default-5b7b554885 from 0 to 1 18m Normal AddedLabel gateway/e2e-ipp-b-9d6d4e Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-9d6d4e 18m Normal SuccessfulCreate replicaset/e2e-ipp-b-9d6d4e-openshift-default-5b7b554885 Created pod: e2e-ipp-b-9d6d4e-openshift-default-5b7b554885-kvnwl 18m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-75b68d-59747b7fb7 Created pod: payload-pre-processing-e2e-ipp-b-75b68d-59747b7fb7-7n6zb 18m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-75b68d-59747b7fb7 Error creating: pods "payload-pre-processing-e2e-ipp-b-75b68d-59747b7fb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-75b68d: serviceaccount "payload-processing-e2e-ipp-b-75b68d" not found 18m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-75b68d-ddbc7885b Created pod: payload-processing-e2e-ipp-b-75b68d-ddbc7885b-wmsp7 17m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-75b68d-59747b7fb7-7n6zb Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 17m Normal Started pod/payload-processing-e2e-ipp-b-75b68d-ddbc7885b-wmsp7 Container started 17m Normal AddedInterface pod/e2e-ipp-b-9d6d4e-openshift-default-5b7b554885-kvnwl Add eth0 [10.134.1.16/23] from ovn-kubernetes 17m Normal Created pod/payload-pre-processing-e2e-ipp-b-75b68d-59747b7fb7-7n6zb Container created 17m Normal AddedInterface pod/payload-processing-e2e-ipp-b-75b68d-ddbc7885b-wmsp7 Add eth0 [10.134.1.15/23] from ovn-kubernetes 17m Normal Pulled pod/payload-processing-e2e-ipp-b-75b68d-ddbc7885b-wmsp7 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 17m Normal Started pod/payload-pre-processing-e2e-ipp-b-75b68d-59747b7fb7-7n6zb Container started 17m Normal Created pod/payload-processing-e2e-ipp-b-75b68d-ddbc7885b-wmsp7 Container created 17m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-75b68d-59747b7fb7-7n6zb Add eth0 [10.132.1.102/23] from ovn-kubernetes 17m Normal Killing pod/e2e-ipp-a-83f120-openshift-default-78b78cf899-x822r Stopping container istio-proxy 17m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-75b68d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 17m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-75b68d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 17m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-9d6d4e-5fd9b5769f Created pod: payload-pre-processing-e2e-ipp-b-9d6d4e-5fd9b5769f-xgn77 17m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-9d6d4e-5c49d979db Error creating: pods "payload-processing-e2e-ipp-b-9d6d4e-5c49d979db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-9d6d4e: serviceaccount "payload-processing-e2e-ipp-b-9d6d4e" not found 17m Warning Unhealthy pod/e2e-ipp-a-b8f54c-openshift-default-69ccbf5c6c-mwkmt Readiness probe failed: HTTP probe failed with statuscode: 503 17m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-9d6d4e Scaled up replica set payload-processing-e2e-ipp-b-9d6d4e-5c49d979db from 0 to 1 17m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-9d6d4e Scaled up replica set payload-pre-processing-e2e-ipp-b-9d6d4e-5fd9b5769f from 0 to 1 17m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-9d6d4e-5fd9b5769f Error creating: pods "payload-pre-processing-e2e-ipp-b-9d6d4e-5fd9b5769f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-9d6d4e: serviceaccount "payload-processing-e2e-ipp-b-9d6d4e" not found 17m Normal Killing pod/e2e-ipp-a-b8f54c-openshift-default-69ccbf5c6c-mwkmt Stopping container istio-proxy 17m Warning ProbeError pod/e2e-ipp-a-b8f54c-openshift-default-69ccbf5c6c-mwkmt Readiness probe error: HTTP probe failed with statuscode: 503... 17m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-9d6d4e-5c49d979db Created pod: payload-processing-e2e-ipp-b-9d6d4e-5c49d979db-z5zzn 17m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-9d6d4e-5fd9b5769f-xgn77 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 17m Normal Created pod/payload-pre-processing-e2e-ipp-b-9d6d4e-5fd9b5769f-xgn77 Container created 17m Normal Started pod/payload-processing-e2e-ipp-b-9d6d4e-5c49d979db-z5zzn Container started 17m Normal Created pod/payload-processing-e2e-ipp-b-9d6d4e-5c49d979db-z5zzn Container created 17m Normal Pulled pod/payload-processing-e2e-ipp-b-9d6d4e-5c49d979db-z5zzn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 17m Normal AddedInterface pod/payload-processing-e2e-ipp-b-9d6d4e-5c49d979db-z5zzn Add eth0 [10.132.1.106/23] from ovn-kubernetes 17m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-9d6d4e-5fd9b5769f-xgn77 Add eth0 [10.132.1.105/23] from ovn-kubernetes 17m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-63e89c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 17m Normal Started pod/payload-pre-processing-e2e-ipp-b-9d6d4e-5fd9b5769f-xgn77 Container started 17m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-63e89c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 17m Warning Unhealthy pod/payload-processing-e2e-worker-w4-08ebee-5f548676d4-st67q Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-63e89c-68b459b96 Error creating: pods "payload-processing-e2e-ipp-b-63e89c-68b459b96-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-63e89c: serviceaccount "payload-processing-e2e-ipp-b-63e89c" not found 17m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-63e89c Scaled up replica set payload-pre-processing-e2e-ipp-b-63e89c-744cbf57c9 from 0 to 1 17m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-63e89c-744cbf57c9 Created pod: payload-pre-processing-e2e-ipp-b-63e89c-744cbf57c9-k8rvr 17m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-63e89c-744cbf57c9 Error creating: pods "payload-pre-processing-e2e-ipp-b-63e89c-744cbf57c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-63e89c: serviceaccount "payload-processing-e2e-ipp-b-63e89c" not found 17m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-63e89c Scaled up replica set payload-processing-e2e-ipp-b-63e89c-68b459b96 from 0 to 1 17m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-63e89c-744cbf57c9-k8rvr Add eth0 [10.134.1.19/23] from ovn-kubernetes 17m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-63e89c-68b459b96 Created pod: payload-processing-e2e-ipp-b-63e89c-68b459b96-7hqcv 17m Normal Started pod/payload-pre-processing-e2e-ipp-b-63e89c-744cbf57c9-k8rvr Container started 17m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w4-08ebee-7bcfbb7487-9x5xj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Normal Pulled pod/payload-processing-e2e-ipp-b-63e89c-68b459b96-7hqcv Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 17m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-63e89c-744cbf57c9-k8rvr Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 17m Normal Created pod/payload-pre-processing-e2e-ipp-b-63e89c-744cbf57c9-k8rvr Container created 17m Normal AddedInterface pod/payload-processing-e2e-ipp-b-63e89c-68b459b96-7hqcv Add eth0 [10.134.1.20/23] from ovn-kubernetes 17m Normal Started pod/payload-processing-e2e-ipp-b-63e89c-68b459b96-7hqcv Container started 17m Normal Created pod/payload-processing-e2e-ipp-b-63e89c-68b459b96-7hqcv Container created 17m Normal Killing pod/payload-processing-e2e-ipp-b-75b68d-ddbc7885b-wmsp7 Stopping container payload-processing 17m Normal Killing pod/payload-pre-processing-e2e-ipp-b-75b68d-59747b7fb7-7n6zb Stopping container payload-pre-processing 17m Normal Killing pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-cfx67 Stopping container istio-proxy 17m Normal Killing pod/e2e-worker-w3-43b37d-openshift-default-5dcf864fcc-jwrdl Stopping container istio-proxy 17m Normal Killing pod/e2e-ipp-b-75b68d-openshift-default-5f58db85f7-74lsn Stopping container istio-proxy 17m Normal Killing pod/payload-processing-e2e-ipp-b-63e89c-68b459b96-7hqcv Stopping container payload-processing 17m Normal Killing pod/payload-pre-processing-e2e-ipp-b-63e89c-744cbf57c9-k8rvr Stopping container payload-pre-processing 17m Warning Unhealthy pod/e2e-ipp-b-75b68d-openshift-default-5f58db85f7-74lsn Readiness probe failed: Get "http://10.134.0.241:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 17m Normal Killing pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-w9vdp Stopping container istio-proxy 17m Warning ProbeError pod/e2e-ipp-b-75b68d-openshift-default-5f58db85f7-74lsn Readiness probe error: Get "http://10.134.0.241:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 17m Normal Killing pod/e2e-worker-w4-08ebee-openshift-default-5f5bf46967-t8fq4 Stopping container istio-proxy 17m Normal Killing pod/e2e-ipp-a-7be71d-openshift-default-8dd8b5bdc-4hx2h Stopping container istio-proxy 17m Normal Killing pod/payload-processing-e2e-shared-a-w2-eb02df-7b6dfb5df8-sw5mn Stopping container payload-processing 17m Normal Killing pod/e2e-ipp-b-63e89c-openshift-default-6d97c8847c-8bqfm Stopping container istio-proxy 17m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-eb02df-54d8869579-kckzx Stopping container payload-pre-processing 16m Normal Killing pod/payload-processing-e2e-shared-a-w0-920032-77dc47dc96-xh765 Stopping container payload-processing 16m Normal Killing pod/payload-pre-processing-e2e-shared-a-w0-920032-85fc75f8c5-fsg62 Stopping container payload-pre-processing 16m Normal Killing pod/payload-pre-processing-e2e-ipp-b-9d6d4e-5fd9b5769f-xgn77 Stopping container payload-pre-processing 16m Normal Killing pod/payload-processing-e2e-ipp-b-9d6d4e-5c49d979db-z5zzn Stopping container payload-processing 16m Normal Killing pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-jzqt4 Stopping container istio-proxy 16m Normal Killing pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-q8qxp Stopping container istio-proxy 16m Normal Killing pod/e2e-shared-a-w0-920032-openshift-default-5b5f9cc67-jmfgs Stopping container istio-proxy 16m Normal Killing pod/payload-pre-processing-e2e-ipp-b-85862e-6c4464c9-9xtht Stopping container payload-pre-processing 16m Warning Unhealthy pod/payload-processing-e2e-shared-a-w2-eb02df-7b6dfb5df8-sw5mn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Normal Killing pod/payload-processing-e2e-ipp-b-85862e-69678cb8f6-hkxqw Stopping container payload-processing 16m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-b-9d6d4e-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 16m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w2-eb02df-54d8869579-kckzx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Normal ScalingReplicaSet deployment/e2e-ipp-b-9d6d4e-openshift-default Scaled up replica set e2e-ipp-b-9d6d4e-openshift-default-5b7b554885 from 1 to 2 16m Normal Killing pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-mshrv Stopping container istio-proxy 16m Normal Killing pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-8j4dd Stopping container istio-proxy 16m Normal SuccessfulCreate replicaset/e2e-ipp-b-9d6d4e-openshift-default-5b7b554885 Created pod: e2e-ipp-b-9d6d4e-openshift-default-5b7b554885-5j85l 16m Normal AddedInterface pod/e2e-ipp-b-9d6d4e-openshift-default-5b7b554885-5j85l Add eth0 [10.132.1.109/23] from ovn-kubernetes 16m Normal Created pod/e2e-ipp-b-9d6d4e-openshift-default-5b7b554885-5j85l Container created 16m Normal Started pod/e2e-ipp-b-9d6d4e-openshift-default-5b7b554885-5j85l Container started 16m Normal Pulled pod/e2e-ipp-b-9d6d4e-openshift-default-5b7b554885-5j85l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal Pulled pod/e2e-ipp-b-9d6d4e-openshift-default-5b7b554885-kvnwl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal Started pod/e2e-ipp-b-9d6d4e-openshift-default-5b7b554885-kvnwl Container started 16m Normal Created pod/e2e-ipp-b-9d6d4e-openshift-default-5b7b554885-kvnwl Container created 16m Warning Unhealthy pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-mshrv Readiness probe failed: Get "http://10.134.0.196:15021/healthz/ready": dial tcp 10.134.0.196:15021: connect: connection refused 16m Warning ProbeError pod/e2e-shared-a-w2-eb02df-openshift-default-5c45c545c-mshrv Readiness probe error: Get "http://10.134.0.196:15021/healthz/ready": dial tcp 10.134.0.196:15021: connect: connection refused... 16m Normal Killing pod/payload-processing-e2e-shared-b-w0-02720a-78cdf6c4cd-w4jmh Stopping container payload-processing 16m Warning ProbeError pod/e2e-ipp-b-9d6d4e-openshift-default-5b7b554885-5j85l Startup probe error: Get "http://10.132.1.109:15021/healthz/ready": dial tcp 10.132.1.109:15021: connect: connection refused... 16m Warning Unhealthy pod/e2e-ipp-b-9d6d4e-openshift-default-5b7b554885-5j85l Startup probe failed: Get "http://10.132.1.109:15021/healthz/ready": dial tcp 10.132.1.109:15021: connect: connection refused 16m Normal Killing pod/payload-pre-processing-e2e-shared-b-w0-02720a-54d9b599c9-q5xcw Stopping container payload-pre-processing 16m Warning Unhealthy pod/e2e-ipp-b-9d6d4e-openshift-default-5b7b554885-kvnwl Startup probe failed: Get "http://10.134.1.16:15021/healthz/ready": dial tcp 10.134.1.16:15021: connect: connection refused 16m Normal Killing pod/e2e-ipp-b-85862e-openshift-default-7b794c9f49-6wxwm Stopping container istio-proxy 16m Warning ProbeError pod/e2e-ipp-b-9d6d4e-openshift-default-5b7b554885-kvnwl Startup probe error: Get "http://10.134.1.16:15021/healthz/ready": dial tcp 10.134.1.16:15021: connect: connection refused... 16m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w0-920032-85fc75f8c5-fsg62 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Warning Unhealthy pod/payload-processing-e2e-shared-a-w0-920032-77dc47dc96-xh765 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Normal Killing pod/payload-processing-e2e-shared-b-w2-3ee017-5f8b7645fb-477wq Stopping container payload-processing 16m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-3ee017-b5cc8c7ff-nnl9p Stopping container payload-pre-processing 16m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-3ee017-openshift-default Scaled down replica set e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7 from 4 to 3 16m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-9d6d4e-767bdcc655 Error creating: pods "payload-pre-processing-e2e-ipp-b-9d6d4e-767bdcc655-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-9d6d4e: serviceaccount "payload-processing-e2e-ipp-b-9d6d4e" not found 16m Normal SuccessfulDelete replicaset/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7 Deleted pod: e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-ht9j2 16m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-9d6d4e-767bdcc655 Created pod: payload-pre-processing-e2e-ipp-b-9d6d4e-767bdcc655-qhqnw 16m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-9d6d4e Scaled up replica set payload-pre-processing-e2e-ipp-b-9d6d4e-767bdcc655 from 0 to 1 16m Normal Killing pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-ht9j2 Stopping container istio-proxy 16m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-9d6d4e-65f69f95b8 Error creating: pods "payload-processing-e2e-ipp-b-9d6d4e-65f69f95b8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-9d6d4e: serviceaccount "payload-processing-e2e-ipp-b-9d6d4e" not found 16m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-9d6d4e-65f69f95b8 Created pod: payload-processing-e2e-ipp-b-9d6d4e-65f69f95b8-qb57q 16m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-9d6d4e Scaled up replica set payload-processing-e2e-ipp-b-9d6d4e-65f69f95b8 from 0 to 1 16m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-3ee017-openshift-default New size: 3; reason: All metrics below target 16m Normal Created pod/payload-processing-e2e-ipp-b-9d6d4e-65f69f95b8-qb57q Container created 16m Normal Started pod/payload-processing-e2e-ipp-b-9d6d4e-65f69f95b8-qb57q Container started 16m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-9d6d4e-767bdcc655-qhqnw Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 16m Normal Started pod/payload-pre-processing-e2e-ipp-b-9d6d4e-767bdcc655-qhqnw Container started 16m Normal AddedInterface pod/payload-processing-e2e-ipp-b-9d6d4e-65f69f95b8-qb57q Add eth0 [10.134.1.26/23] from ovn-kubernetes 16m Normal Pulled pod/payload-processing-e2e-ipp-b-9d6d4e-65f69f95b8-qb57q Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 16m Normal Created pod/payload-pre-processing-e2e-ipp-b-9d6d4e-767bdcc655-qhqnw Container created 16m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-9d6d4e-767bdcc655-qhqnw Add eth0 [10.134.1.25/23] from ovn-kubernetes 16m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-9d6d4e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 16m Normal Killing pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-bkj9z Stopping container istio-proxy 16m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-02720a-openshift-default Scaled down replica set e2e-shared-b-w0-02720a-openshift-default-6b64cb69 from 3 to 2 16m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-9d6d4e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 16m Normal SuccessfulDelete replicaset/e2e-shared-b-w0-02720a-openshift-default-6b64cb69 Deleted pod: e2e-shared-b-w0-02720a-openshift-default-6b64cb69-bkj9z 16m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-02720a-openshift-default New size: 2; reason: All metrics below target 16m Normal Killing pod/payload-processing-e2e-shared-a-w5-9aafe6-788454f66b-t8w2h Stopping container payload-processing 16m Normal Killing pod/payload-pre-processing-e2e-shared-a-w5-9aafe6-6f8f97cff9-4cmpr Stopping container payload-pre-processing 16m Normal Killing pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-kwffs Stopping container istio-proxy 16m Normal Killing pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-sgz2w Stopping container istio-proxy 16m Warning Unhealthy pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-sgz2w Readiness probe failed: HTTP probe failed with statuscode: 503 16m Warning ProbeError pod/e2e-shared-b-w0-02720a-openshift-default-6b64cb69-sgz2w Readiness probe error: HTTP probe failed with statuscode: 503... 16m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w0-02720a-54d9b599c9-q5xcw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Warning Unhealthy pod/payload-processing-e2e-shared-b-w0-02720a-78cdf6c4cd-w4jmh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w2-3ee017-b5cc8c7ff-nnl9p Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Warning Unhealthy pod/payload-processing-e2e-shared-b-w2-3ee017-5f8b7645fb-477wq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-9d6d4e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 16m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-9d6d4e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 16m Normal Killing pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-hjvtw Stopping container istio-proxy 16m Normal Killing pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-9m5dt Stopping container istio-proxy 16m Normal Killing pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-hdcz8 Stopping container istio-proxy 16m Normal Killing pod/payload-pre-processing-e2e-worker-w0-2e046a-74bbd64fc9-7nrm7 Stopping container payload-pre-processing 16m Normal Killing pod/payload-processing-e2e-worker-w0-2e046a-6888946c9d-w2lbg Stopping container payload-processing 16m Warning ProbeError pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-hjvtw Readiness probe error: HTTP probe failed with statuscode: 503... 16m Warning Unhealthy pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-hjvtw Readiness probe failed: HTTP probe failed with statuscode: 503 16m Warning ProbeError pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-hdcz8 Readiness probe error: HTTP probe failed with statuscode: 503... 16m Warning Unhealthy pod/e2e-shared-b-w2-3ee017-openshift-default-86b45fc4d7-hdcz8 Readiness probe failed: HTTP probe failed with statuscode: 503 16m Normal Killing pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-fhq59 Stopping container istio-proxy 16m Normal Killing pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-r9lfc Stopping container istio-proxy 16m Warning Unhealthy pod/payload-processing-e2e-shared-a-w5-9aafe6-788454f66b-t8w2h Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Warning ProbeError pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-r9lfc Readiness probe error: HTTP probe failed with statuscode: 503... 16m Warning Unhealthy pod/e2e-shared-a-w5-9aafe6-openshift-default-6c9bf5dd85-r9lfc Readiness probe failed: HTTP probe failed with statuscode: 503 16m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w5-9aafe6-6f8f97cff9-4cmpr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Normal Killing pod/payload-pre-processing-e2e-worker-w2-2ba10a-6d55f55df9-gfdpx Stopping container payload-pre-processing 16m Normal Killing pod/payload-processing-e2e-worker-w2-2ba10a-78fb8d6fd4-qw5lv Stopping container payload-processing 15m Normal Killing pod/payload-pre-processing-e2e-shared-b-w5-6fc6bd-78c4d8b54c-qg69n Stopping container payload-pre-processing 15m Normal Killing pod/payload-processing-e2e-shared-b-w5-6fc6bd-cf68d45f8-22lvh Stopping container payload-processing 15m Normal Killing pod/payload-processing-e2e-ipp-a-80248b-5dfc48b658-bwlk6 Stopping container payload-processing 15m Normal Killing pod/payload-pre-processing-e2e-ipp-a-80248b-5c58666f59-csjml Stopping container payload-pre-processing 15m Warning Unhealthy pod/payload-processing-e2e-worker-w0-2e046a-6888946c9d-w2lbg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-2ba10a-6d55f55df9-gfdpx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-2e046a-74bbd64fc9-7nrm7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Warning Unhealthy pod/e2e-ipp-a-80248b-openshift-default-5677b677f5-lx6cs Readiness probe failed: HTTP probe failed with statuscode: 503 15m Warning ProbeError pod/e2e-ipp-a-80248b-openshift-default-5677b677f5-lx6cs Readiness probe error: HTTP probe failed with statuscode: 503... 15m Normal Killing pod/e2e-ipp-a-80248b-openshift-default-5677b677f5-lx6cs Stopping container istio-proxy 15m Warning Unhealthy pod/payload-processing-e2e-worker-w2-2ba10a-78fb8d6fd4-qw5lv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-2ba10a-6d55f55df9-gfdpx Readiness probe failed: timeout: failed to connect service "10.134.0.42:50052" within 1s: context deadline exceeded 15m Normal Killing pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-rkgz4 Stopping container istio-proxy 15m Normal Killing pod/e2e-worker-w0-2e046a-openshift-default-79c5dd5b49-lks7l Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-qpsqc Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-82f2v Stopping container istio-proxy 15m Normal Killing pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-787nl Stopping container istio-proxy 15m Normal Killing pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-dccl7 Stopping container istio-proxy 15m Warning Unhealthy pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-82f2v Readiness probe failed: HTTP probe failed with statuscode: 503 15m Warning ProbeError pod/e2e-shared-b-w5-6fc6bd-openshift-default-5665fddf89-82f2v Readiness probe error: HTTP probe failed with statuscode: 503... 15m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w5-6fc6bd-78c4d8b54c-qg69n Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Warning Unhealthy pod/payload-processing-e2e-shared-b-w5-6fc6bd-cf68d45f8-22lvh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Warning Unhealthy pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-787nl Readiness probe failed: HTTP probe failed with statuscode: 503 15m Warning ProbeError pod/e2e-worker-w2-2ba10a-openshift-default-6d84cb5d99-787nl Readiness probe error: HTTP probe failed with statuscode: 503... 15m Normal Killing pod/payload-pre-processing-e2e-worker-w5-22d634-6768765995-phnhr Stopping container payload-pre-processing 15m Normal Killing pod/payload-processing-e2e-worker-w5-22d634-7db4fc9554-56z6p Stopping container payload-processing 15m Normal Killing pod/payload-pre-processing-e2e-ipp-b-9d6d4e-767bdcc655-qhqnw Stopping container payload-pre-processing 15m Normal Killing pod/payload-processing-e2e-ipp-b-9d6d4e-65f69f95b8-qb57q Stopping container payload-processing 15m Normal Killing pod/e2e-ipp-b-9d6d4e-openshift-default-5b7b554885-kvnwl Stopping container istio-proxy 15m Normal Killing pod/e2e-ipp-b-9d6d4e-openshift-default-5b7b554885-5j85l Stopping container istio-proxy 15m Normal Killing pod/payload-processing-e2e-shared-a-w1-af260f-fd896cf88-rb5bz Stopping container payload-processing 15m Normal Killing pod/payload-pre-processing-e2e-shared-a-w1-af260f-7cfc9454ff-9c5lq Stopping container payload-pre-processing 14m Normal Killing pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-cxjmw Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-wk57v Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-ftzdt Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-wkhfz Stopping container istio-proxy 14m Normal Killing pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-4jtkn Stopping container istio-proxy 14m Normal Killing pod/e2e-worker-w5-22d634-openshift-default-89dbbf78f-nbn5j Stopping container istio-proxy 14m Warning ProbeError pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-wk57v Readiness probe error: Get "http://10.134.0.212:15021/healthz/ready": dial tcp 10.134.0.212:15021: connect: connection refused... 14m Warning Unhealthy pod/e2e-shared-a-w1-af260f-openshift-default-846bb69f67-wk57v Readiness probe failed: Get "http://10.134.0.212:15021/healthz/ready": dial tcp 10.134.0.212:15021: connect: connection refused 14m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w1-af260f-7cfc9454ff-9c5lq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-processing-e2e-shared-a-w1-af260f-fd896cf88-rb5bz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Normal Killing pod/payload-processing-e2e-shared-b-w1-d68c2c-5c949bdf9b-52x5q Stopping container payload-processing 14m Normal Killing pod/payload-pre-processing-e2e-shared-b-w1-d68c2c-7d66774ccf-wvf5r Stopping container payload-pre-processing 14m Normal Killing pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-6xvfq Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-n69t2 Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-xpz6c Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-j4gn7 Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-wrrbr Stopping container istio-proxy 14m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 14m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 14m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-9jn89 14m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-9jn89 Stopping container discovery 14m Warning Unhealthy pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-6xvfq Readiness probe failed: HTTP probe failed with statuscode: 503 14m Warning ProbeError pod/e2e-shared-b-w1-d68c2c-openshift-default-67bfdff69f-6xvfq Readiness probe error: HTTP probe failed with statuscode: 503... 14m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w1-d68c2c-7d66774ccf-wvf5r Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-processing-e2e-shared-b-w1-d68c2c-5c949bdf9b-52x5q Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 2; reason: All metrics below target 14m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-5db77fbd9f from 3 to 2 14m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5db77fbd9f Deleted pod: maas-default-gateway-openshift-default-5db77fbd9f-cr7nq 14m Normal Killing pod/maas-default-gateway-openshift-default-5db77fbd9f-cr7nq Stopping container istio-proxy 14m Normal Killing pod/payload-pre-processing-e2e-worker-w1-fc684b-7fb5fd79c5-hb2rn Stopping container payload-pre-processing 14m Normal Killing pod/payload-processing-e2e-worker-w1-fc684b-5db88966b-rwg2s Stopping container payload-processing 14m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-cr7nq Readiness probe error: Get "http://10.132.0.174:15021/healthz/ready": dial tcp 10.132.0.174:15021: connect: connection refused... 14m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-cr7nq Readiness probe failed: Get "http://10.132.0.174:15021/healthz/ready": dial tcp 10.132.0.174:15021: connect: connection refused 13m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w1-fc684b-7fb5fd79c5-hb2rn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 13m Normal Killing pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-vnz5k Stopping container istio-proxy 13m Warning Unhealthy pod/payload-processing-e2e-worker-w1-fc684b-5db88966b-rwg2s Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 13m Normal Killing pod/e2e-worker-w1-fc684b-openshift-default-7fcf678997-988qz Stopping container istio-proxy 13m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5db77fbd9f Created pod: maas-default-gateway-openshift-default-5db77fbd9f-kgx4g 13m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 13m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-5db77fbd9f from 2 to 4 13m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5db77fbd9f Created pod: maas-default-gateway-openshift-default-5db77fbd9f-vkx5n 13m Normal Pulled pod/maas-default-gateway-openshift-default-5db77fbd9f-vkx5n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Normal AddedInterface pod/maas-default-gateway-openshift-default-5db77fbd9f-vkx5n Add eth0 [10.132.1.117/23] from ovn-kubernetes 13m Normal Started pod/maas-default-gateway-openshift-default-5db77fbd9f-vkx5n Container started 13m Normal AddedInterface pod/maas-default-gateway-openshift-default-5db77fbd9f-kgx4g Add eth0 [10.134.1.40/23] from ovn-kubernetes 13m Normal Started pod/maas-default-gateway-openshift-default-5db77fbd9f-kgx4g Container started 13m Normal Created pod/maas-default-gateway-openshift-default-5db77fbd9f-vkx5n Container created 13m Normal Pulled pod/maas-default-gateway-openshift-default-5db77fbd9f-kgx4g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Normal Created pod/maas-default-gateway-openshift-default-5db77fbd9f-kgx4g Container created 13m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-vkx5n Startup probe error: Get "http://10.132.1.117:15021/healthz/ready": dial tcp 10.132.1.117:15021: connect: connection refused... 13m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-kgx4g Startup probe error: Get "http://10.134.1.40:15021/healthz/ready": dial tcp 10.134.1.40:15021: connect: connection refused... 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-vkx5n Startup probe failed: Get "http://10.132.1.117:15021/healthz/ready": dial tcp 10.132.1.117:15021: connect: connection refused 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-kgx4g Startup probe failed: Get "http://10.134.1.40:15021/healthz/ready": dial tcp 10.134.1.40:15021: connect: connection refused 13m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5db77fbd9f Created pod: maas-default-gateway-openshift-default-5db77fbd9f-nw2zt 13m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5db77fbd9f Created pod: maas-default-gateway-openshift-default-5db77fbd9f-zzv89 13m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-5db77fbd9f from 4 to 6 13m Normal AddedInterface pod/maas-default-gateway-openshift-default-5db77fbd9f-zzv89 Add eth0 [10.132.1.118/23] from ovn-kubernetes 13m Normal AddedInterface pod/maas-default-gateway-openshift-default-5db77fbd9f-nw2zt Add eth0 [10.133.0.156/23] from ovn-kubernetes 13m Normal Created pod/maas-default-gateway-openshift-default-5db77fbd9f-nw2zt Container created 13m Normal Pulled pod/maas-default-gateway-openshift-default-5db77fbd9f-zzv89 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Normal Started pod/maas-default-gateway-openshift-default-5db77fbd9f-zzv89 Container started 13m Normal Pulled pod/maas-default-gateway-openshift-default-5db77fbd9f-nw2zt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Normal Created pod/maas-default-gateway-openshift-default-5db77fbd9f-zzv89 Container created 13m Normal Started pod/maas-default-gateway-openshift-default-5db77fbd9f-nw2zt Container started 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-nw2zt Startup probe failed: Get "http://10.133.0.156:15021/healthz/ready": dial tcp 10.133.0.156:15021: connect: connection refused 13m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-nw2zt Startup probe error: Get "http://10.133.0.156:15021/healthz/ready": dial tcp 10.133.0.156:15021: connect: connection refused... 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-zzv89 Startup probe failed: Get "http://10.132.1.118:15021/healthz/ready": dial tcp 10.132.1.118:15021: connect: connection refused 13m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-zzv89 Startup probe error: Get "http://10.132.1.118:15021/healthz/ready": dial tcp 10.132.1.118:15021: connect: connection refused... 12m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default (combined from similar events): New size: 8; reason: cpu resource utilization (percentage of request) above target 12m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5db77fbd9f Created pod: maas-default-gateway-openshift-default-5db77fbd9f-96ppp 12m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5db77fbd9f Created pod: maas-default-gateway-openshift-default-5db77fbd9f-9g4hg 12m Normal AddedInterface pod/maas-default-gateway-openshift-default-5db77fbd9f-96ppp Add eth0 [10.132.1.119/23] from ovn-kubernetes 12m Normal Started pod/maas-default-gateway-openshift-default-5db77fbd9f-96ppp Container started 12m Normal AddedInterface pod/maas-default-gateway-openshift-default-5db77fbd9f-9g4hg Add eth0 [10.134.1.43/23] from ovn-kubernetes 12m Normal Pulled pod/maas-default-gateway-openshift-default-5db77fbd9f-9g4hg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 12m Normal Created pod/maas-default-gateway-openshift-default-5db77fbd9f-9g4hg Container created 12m Normal Started pod/maas-default-gateway-openshift-default-5db77fbd9f-9g4hg Container started 12m Normal Created pod/maas-default-gateway-openshift-default-5db77fbd9f-96ppp Container created 12m Normal Pulled pod/maas-default-gateway-openshift-default-5db77fbd9f-96ppp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-96ppp Startup probe failed: Get "http://10.132.1.119:15021/healthz/ready": dial tcp 10.132.1.119:15021: connect: connection refused 12m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-96ppp Startup probe error: Get "http://10.132.1.119:15021/healthz/ready": dial tcp 10.132.1.119:15021: connect: connection refused... 12m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-9g4hg Startup probe error: Get "http://10.134.1.43:15021/healthz/ready": dial tcp 10.134.1.43:15021: connect: connection refused... 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-9g4hg Startup probe failed: Get "http://10.134.1.43:15021/healthz/ready": dial tcp 10.134.1.43:15021: connect: connection refused 12m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default (combined from similar events): Scaled up replica set maas-default-gateway-openshift-default-5db77fbd9f from 8 to 10 12m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5db77fbd9f Created pod: maas-default-gateway-openshift-default-5db77fbd9f-dn5qq 12m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 12m Normal Created pod/maas-default-gateway-openshift-default-5db77fbd9f-tm9cb Container created 12m Normal Pulled pod/maas-default-gateway-openshift-default-5db77fbd9f-tm9cb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 12m Normal Started pod/maas-default-gateway-openshift-default-5db77fbd9f-tm9cb Container started 12m Normal AddedInterface pod/maas-default-gateway-openshift-default-5db77fbd9f-dn5qq Add eth0 [10.133.0.157/23] from ovn-kubernetes 12m Normal Pulled pod/maas-default-gateway-openshift-default-5db77fbd9f-dn5qq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 12m Normal Created pod/maas-default-gateway-openshift-default-5db77fbd9f-dn5qq Container created 12m Normal AddedInterface pod/maas-default-gateway-openshift-default-5db77fbd9f-tm9cb Add eth0 [10.134.1.46/23] from ovn-kubernetes 12m Normal Started pod/maas-default-gateway-openshift-default-5db77fbd9f-dn5qq Container started 12m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-dn5qq Startup probe error: Get "http://10.133.0.157:15021/healthz/ready": dial tcp 10.133.0.157:15021: connect: connection refused... 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-dn5qq Startup probe failed: Get "http://10.133.0.157:15021/healthz/ready": dial tcp 10.133.0.157:15021: connect: connection refused 11m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-tm9cb Startup probe error: Get "http://10.134.1.46:15021/healthz/ready": dial tcp 10.134.1.46:15021: connect: connection refused... 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-tm9cb Startup probe failed: Get "http://10.134.1.46:15021/healthz/ready": dial tcp 10.134.1.46:15021: connect: connection refused 11m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-dn5qq Startup probe error: Get "http://10.133.0.157:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-dn5qq Startup probe failed: Get "http://10.133.0.157:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 11m Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-tm9cb Startup probe error: Get "http://10.134.1.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-tm9cb Startup probe failed: Get "http://10.134.1.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 11m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 11m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-z794f Stopping container discovery 11m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-z794f 11m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 8m59s Normal ScalingReplicaSet deployment/e2e-worker-single-main-e5236b-openshift-default Scaled up replica set e2e-worker-single-main-e5236b-openshift-default-547784c65 from 0 to 1 8m59s Normal SuccessfulCreate replicaset/e2e-worker-single-main-e5236b-openshift-default-547784c65 Created pod: e2e-worker-single-main-e5236b-openshift-default-547784c65-qg68z 8m59s Normal AddedLabel gateway/e2e-worker-single-main-e5236b Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-e5236b 8m58s Normal AddedInterface pod/e2e-worker-single-main-e5236b-openshift-default-547784c65-qg68z Add eth0 [10.132.1.127/23] from ovn-kubernetes 8m54s Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-e5236b-7548fdb967 Error creating: pods "payload-pre-processing-e2e-worker-single-main-e5236b-7548fdb967-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-e5236b: serviceaccount "payload-processing-e2e-worker-single-main-e5236b" not found 8m54s Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-e5236b Scaled up replica set payload-processing-e2e-worker-single-main-e5236b-fcf66d4f4 from 0 to 1 8m54s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-e5236b Scaled up replica set payload-pre-processing-e2e-worker-single-main-e5236b-7548fdb967 from 0 to 1 8m53s Normal AddedInterface pod/payload-processing-e2e-worker-single-main-e5236b-fcf66d4f44xjqg Add eth0 [10.132.1.129/23] from ovn-kubernetes 8m53s Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-e5236b-7548fnkkbf Add eth0 [10.132.1.128/23] from ovn-kubernetes 8m53s Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-e5236b-fcf66d4f4 Created pod: payload-processing-e2e-worker-single-main-e5236b-fcf66d4f44xjqg 8m53s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-e5236b-7548fdb967 Created pod: payload-pre-processing-e2e-worker-single-main-e5236b-7548fnkkbf 8m53s Normal Started pod/payload-processing-e2e-worker-single-main-e5236b-fcf66d4f44xjqg Container started 8m53s Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-e5236b-7548fnkkbf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m53s Normal Pulled pod/payload-processing-e2e-worker-single-main-e5236b-fcf66d4f44xjqg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m53s Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-e5236b-fcf66d4f4 Error creating: pods "payload-processing-e2e-worker-single-main-e5236b-fcf66d4f4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-e5236b: serviceaccount "payload-processing-e2e-worker-single-main-e5236b" not found 8m53s Normal Created pod/payload-pre-processing-e2e-worker-single-main-e5236b-7548fnkkbf Container created 8m53s Normal Started pod/payload-pre-processing-e2e-worker-single-main-e5236b-7548fnkkbf Container started 8m53s Normal Created pod/payload-processing-e2e-worker-single-main-e5236b-fcf66d4f44xjqg Container created 8m41s Normal Pulled pod/e2e-worker-single-main-e5236b-openshift-default-547784c65-qg68z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 8m41s Warning Unhealthy pod/e2e-worker-single-main-e5236b-openshift-default-547784c65-qg68z Readiness probe failed: Get "http://10.132.1.127:15021/healthz/ready": read tcp 10.132.0.2:34500->10.132.1.127:15021: read: connection reset by peer 8m41s Warning ProbeError pod/e2e-worker-single-main-e5236b-openshift-default-547784c65-qg68z Readiness probe error: Get "http://10.132.1.127:15021/healthz/ready": read tcp 10.132.0.2:34500->10.132.1.127:15021: read: connection reset by peer... 8m40s Normal Started pod/e2e-worker-single-main-e5236b-openshift-default-547784c65-qg68z Container started 8m40s Normal Created pod/e2e-worker-single-main-e5236b-openshift-default-547784c65-qg68z Container created 8m35s Warning ProbeError pod/e2e-worker-single-main-e5236b-openshift-default-547784c65-qg68z Startup probe error: Get "http://10.132.1.127:15021/healthz/ready": dial tcp 10.132.1.127:15021: connect: connection refused... 8m35s Warning Unhealthy pod/e2e-worker-single-main-e5236b-openshift-default-547784c65-qg68z Startup probe failed: Get "http://10.132.1.127:15021/healthz/ready": dial tcp 10.132.1.127:15021: connect: connection refused 8m29s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-e5236b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 8m29s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-e5236b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 8m14s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-e5236b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 8m14s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-e5236b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 7m53s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-e5236b-7548fnkkbf Stopping container payload-pre-processing 7m53s Normal Killing pod/payload-processing-e2e-worker-single-main-e5236b-fcf66d4f44xjqg Stopping container payload-processing 7m43s Normal Killing pod/e2e-worker-single-main-e5236b-openshift-default-547784c65-qg68z Stopping container istio-proxy 7m32s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-e5236b-7548fnkkbf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 7m31s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-e5236b-fcf66d4f44xjqg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 5m32s Normal SuccessfulCreate replicaset/e2e-worker-single-main-063169-openshift-default-786ffd57b5 Created pod: e2e-worker-single-main-063169-openshift-default-786ffd57b5prdhk 5m32s Normal AddedLabel gateway/e2e-worker-single-main-063169 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-063169 5m32s Normal ScalingReplicaSet deployment/e2e-worker-single-main-063169-openshift-default Scaled up replica set e2e-worker-single-main-063169-openshift-default-786ffd57b5 from 0 to 1 5m31s Normal AddedInterface pod/e2e-worker-single-main-063169-openshift-default-786ffd57b5prdhk Add eth0 [10.132.1.135/23] from ovn-kubernetes 5m26s Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-063169-fbc96c98b Created pod: payload-processing-e2e-worker-single-main-063169-fbc96c98b5zgjw 5m26s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-063169 Scaled up replica set payload-pre-processing-e2e-worker-single-main-063169-78d58b79ff from 0 to 1 5m26s Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-063169-fbc96c98b Error creating: pods "payload-processing-e2e-worker-single-main-063169-fbc96c98b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-063169: serviceaccount "payload-processing-e2e-worker-single-main-063169" not found 5m26s Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-063169 Scaled up replica set payload-processing-e2e-worker-single-main-063169-fbc96c98b from 0 to 1 5m26s Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-063169-78d58b79ff Error creating: pods "payload-pre-processing-e2e-worker-single-main-063169-78d58b79ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-063169: serviceaccount "payload-processing-e2e-worker-single-main-063169" not found 5m26s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-063169-78d58b79ff Created pod: payload-pre-processing-e2e-worker-single-main-063169-78d58mpvtg 5m25s Normal Created pod/payload-processing-e2e-worker-single-main-063169-fbc96c98b5zgjw Container created 5m25s Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-063169-78d58mpvtg Add eth0 [10.132.1.136/23] from ovn-kubernetes 5m25s Normal Created pod/payload-pre-processing-e2e-worker-single-main-063169-78d58mpvtg Container created 5m25s Normal Started pod/payload-processing-e2e-worker-single-main-063169-fbc96c98b5zgjw Container started 5m25s Normal AddedInterface pod/payload-processing-e2e-worker-single-main-063169-fbc96c98b5zgjw Add eth0 [10.132.1.137/23] from ovn-kubernetes 5m25s Normal Pulled pod/payload-processing-e2e-worker-single-main-063169-fbc96c98b5zgjw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m25s Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-063169-78d58mpvtg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m25s Normal Started pod/payload-pre-processing-e2e-worker-single-main-063169-78d58mpvtg Container started 5m17s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-063169-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 5m17s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-063169-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 5m13s Warning ProbeError pod/e2e-worker-single-main-063169-openshift-default-786ffd57b5prdhk Readiness probe error: Get "http://10.132.1.135:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 5m13s Warning Unhealthy pod/e2e-worker-single-main-063169-openshift-default-786ffd57b5prdhk Readiness probe failed: Get "http://10.132.1.135:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 5m12s Normal Pulled pod/e2e-worker-single-main-063169-openshift-default-786ffd57b5prdhk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 5m12s Normal Created pod/e2e-worker-single-main-063169-openshift-default-786ffd57b5prdhk Container created 5m12s Normal Started pod/e2e-worker-single-main-063169-openshift-default-786ffd57b5prdhk Container started 5m7s Warning Unhealthy pod/e2e-worker-single-main-063169-openshift-default-786ffd57b5prdhk Startup probe failed: Get "http://10.132.1.135:15021/healthz/ready": dial tcp 10.132.1.135:15021: connect: connection refused 5m7s Warning ProbeError pod/e2e-worker-single-main-063169-openshift-default-786ffd57b5prdhk Startup probe error: Get "http://10.132.1.135:15021/healthz/ready": dial tcp 10.132.1.135:15021: connect: connection refused... 4m46s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-063169-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m46s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-063169-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m32s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-063169-78d58mpvtg Stopping container payload-pre-processing 4m32s Normal Killing pod/payload-processing-e2e-worker-single-main-063169-fbc96c98b5zgjw Stopping container payload-processing 4m22s Normal Killing pod/e2e-worker-single-main-063169-openshift-default-786ffd57b5prdhk Stopping container istio-proxy 4m21s Warning Unhealthy pod/e2e-worker-single-main-063169-openshift-default-786ffd57b5prdhk Readiness probe failed: HTTP probe failed with statuscode: 503 4m21s Warning ProbeError pod/e2e-worker-single-main-063169-openshift-default-786ffd57b5prdhk Readiness probe error: HTTP probe failed with statuscode: 503... 4m13s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-063169-fbc96c98b5zgjw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 4m13s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-063169-78d58mpvtg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 2m13s Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-nw2zt Readiness probe error: Get "http://10.133.0.156:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 2m13s Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-nw2zt Readiness probe failed: Get "http://10.133.0.156:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m13s Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-zzv89 Readiness probe error: Get "http://10.132.1.118:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 2m13s Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-96ppp Readiness probe failed: Get "http://10.132.1.119:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m13s Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-96ppp Readiness probe error: Get "http://10.132.1.119:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 2m13s Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-zzv89 Readiness probe failed: Get "http://10.132.1.118:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m13s Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-vkx5n Readiness probe error: Get "http://10.132.1.117:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 2m13s Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-vkx5n Readiness probe failed: Get "http://10.132.1.117:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m13s Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-kgx4g Readiness probe error: Get "http://10.134.1.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 2m13s Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-kgx4g Readiness probe failed: Get "http://10.134.1.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m12s Warning Unhealthy pod/maas-default-gateway-openshift-default-5db77fbd9f-9g4hg Readiness probe failed: Get "http://10.134.1.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m12s Warning ProbeError pod/maas-default-gateway-openshift-default-5db77fbd9f-9g4hg Readiness probe error: Get "http://10.134.1.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)...