=== events in openshift-ingress (2026-10-31T01:24:54Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 58m Normal Scheduled pod/payload-pre-processing-7b98bb678c-xhpxl Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-xhpxl to ip-10-0-140-47.us-west-2.compute.internal 32m Normal Scheduled pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-mwd5g Successfully assigned openshift-ingress/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-mwd5g to ip-10-0-137-56.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sf5lc Successfully assigned openshift-ingress/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sf5lc to ip-10-0-137-56.us-west-2.compute.internal 57m Normal Scheduled pod/payload-processing-e2e-worker-w2-e8ba06-bb9fb667f-x6fzk Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-e8ba06-bb9fb667f-x6fzk to ip-10-0-137-56.us-west-2.compute.internal 28m Normal Scheduled pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-rf2wq Successfully assigned openshift-ingress/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-rf2wq to ip-10-0-137-56.us-west-2.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-shared-a-w0-f32ec2-79c9797c5f-mllx6 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-f32ec2-79c9797c5f-mllx6 to ip-10-0-140-47.us-west-2.compute.internal 47m Normal Scheduled pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sxdqn Successfully assigned openshift-ingress/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sxdqn to ip-10-0-137-56.us-west-2.compute.internal 47m Normal Scheduled pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-tskrl Successfully assigned openshift-ingress/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-tskrl to ip-10-0-140-47.us-west-2.compute.internal 28m Normal Scheduled pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-xnr5x Successfully assigned openshift-ingress/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-xnr5x to ip-10-0-137-56.us-west-2.compute.internal 47m Normal Scheduled pod/e2e-ait-49bb6b1a-openshift-default-8bcd84ddc-9tqjs Successfully assigned openshift-ingress/e2e-ait-49bb6b1a-openshift-default-8bcd84ddc-9tqjs to ip-10-0-140-47.us-west-2.compute.internal 57m Normal Scheduled pod/payload-processing-e2e-worker-w1-982ae3-6c4b5dc9b9-mmctb Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-982ae3-6c4b5dc9b9-mmctb to ip-10-0-134-11.us-west-2.compute.internal 47m Normal Scheduled pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-qm9sf Successfully assigned openshift-ingress/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-qm9sf to ip-10-0-134-11.us-west-2.compute.internal 39m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-dcdbd0-bc7d86cfc-rs4qx Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-dcdbd0-bc7d86cfc-rs4qx to ip-10-0-137-56.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-ngfr5 Successfully assigned openshift-ingress/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-ngfr5 to ip-10-0-140-47.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-dz2cl Successfully assigned openshift-ingress/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-dz2cl to ip-10-0-134-11.us-west-2.compute.internal 46m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-xjq9h Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xjq9h to ip-10-0-140-47.us-west-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-vxqbg Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-vxqbg to ip-10-0-140-47.us-west-2.compute.internal 57m Normal Scheduled pod/payload-processing-e2e-worker-w3-1e5e75-ff6df9bb9-d888l Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-1e5e75-ff6df9bb9-d888l to ip-10-0-137-56.us-west-2.compute.internal 57m Normal Scheduled pod/payload-processing-e2e-worker-w0-101b9e-6c44b8947-88qn7 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-101b9e-6c44b8947-88qn7 to ip-10-0-134-11.us-west-2.compute.internal 63m Normal Scheduled pod/kube-auth-proxy-66d4777458-v9hv8 Successfully assigned openshift-ingress/kube-auth-proxy-66d4777458-v9hv8 to ip-10-0-134-11.us-west-2.compute.internal 11m Normal Scheduled pod/e2e-worker-single-main-abe7bd-openshift-default-7d97f8b7c75x2j7 Successfully assigned openshift-ingress/e2e-worker-single-main-abe7bd-openshift-default-7d97f8b7c75x2j7 to ip-10-0-137-56.us-west-2.compute.internal 57m Normal Scheduled pod/payload-processing-e2e-worker-w4-da9d07-5848fd449-s4wwm Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-da9d07-5848fd449-s4wwm to ip-10-0-137-56.us-west-2.compute.internal 57m Normal Scheduled pod/payload-processing-e2e-worker-w5-412743-688d66585f-vk5r7 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-412743-688d66585f-vk5r7 to ip-10-0-137-56.us-west-2.compute.internal 15m Normal Scheduled pod/e2e-worker-single-main-863fb0-openshift-default-86c56576ccdwkvd Successfully assigned openshift-ingress/e2e-worker-single-main-863fb0-openshift-default-86c56576ccdwkvd to ip-10-0-137-56.us-west-2.compute.internal 41m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-btpxb Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-btpxb to ip-10-0-134-11.us-west-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-mt-f0651cac-59bc6987b9-mh675 Successfully assigned openshift-ingress/payload-processing-e2e-mt-f0651cac-59bc6987b9-mh675 to ip-10-0-134-11.us-west-2.compute.internal 6m34s Normal Scheduled pod/e2e-ait-576e86c8-openshift-default-98874d9b5-w9brf Successfully assigned openshift-ingress/e2e-ait-576e86c8-openshift-default-98874d9b5-w9brf to ip-10-0-137-56.us-west-2.compute.internal 59m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-mk8f6 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-mk8f6 to ip-10-0-137-56.us-west-2.compute.internal 31m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-2rzpl Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-2rzpl to ip-10-0-137-56.us-west-2.compute.internal 10m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-abe7bd-545c4j4fxw Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-abe7bd-545c4j4fxw to ip-10-0-134-11.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-cg79s Successfully assigned openshift-ingress/e2e-worker-w5-412743-openshift-default-7bddc8b5df-cg79s to ip-10-0-140-47.us-west-2.compute.internal 31m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-6m7ds Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-6m7ds to ip-10-0-134-11.us-west-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-derive-74c6ba99-cd559cfbf-v6nns Successfully assigned openshift-ingress/payload-processing-e2e-derive-74c6ba99-cd559cfbf-v6nns to ip-10-0-134-11.us-west-2.compute.internal 5m18s Normal Scheduled pod/payload-processing-e2e-authcfg-http-44d22c-9c679cbbf-rmpq6 Successfully assigned openshift-ingress/payload-processing-e2e-authcfg-http-44d22c-9c679cbbf-rmpq6 to ip-10-0-137-56.us-west-2.compute.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-101b9e-688fd6f5d7-7nrcm Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-101b9e-688fd6f5d7-7nrcm to ip-10-0-137-56.us-west-2.compute.internal 35m Normal Scheduled pod/e2e-inf-a-4ab8e5-openshift-default-6c89ffbf45-mtckf Successfully assigned openshift-ingress/e2e-inf-a-4ab8e5-openshift-default-6c89ffbf45-mtckf to ip-10-0-137-56.us-west-2.compute.internal 32m Normal Scheduled pod/e2e-inf-b-bc2453-openshift-default-677bb79dc9-8pn9n Successfully assigned openshift-ingress/e2e-inf-b-bc2453-openshift-default-677bb79dc9-8pn9n to ip-10-0-134-11.us-west-2.compute.internal 31m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-9nrmp Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-9nrmp to ip-10-0-140-47.us-west-2.compute.internal 44m Normal Scheduled pod/e2e-ait-dd862feb-openshift-default-67b7c76895-zrjbk Successfully assigned openshift-ingress/e2e-ait-dd862feb-openshift-default-67b7c76895-zrjbk to ip-10-0-137-56.us-west-2.compute.internal 46m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-wspmm Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-wspmm to ip-10-0-134-11.us-west-2.compute.internal 44m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-thjff Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-thjff to ip-10-0-140-47.us-west-2.compute.internal 10m Normal Scheduled pod/payload-processing-e2e-worker-single-main-abe7bd-dff668477pzlvc Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-abe7bd-dff668477pzlvc to ip-10-0-137-56.us-west-2.compute.internal 62m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-8d22z Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8d22z to ip-10-0-137-56.us-west-2.compute.internal 15m Normal Scheduled pod/payload-processing-e2e-worker-single-main-863fb0-5ff7d48b7sg7fr Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-863fb0-5ff7d48b7sg7fr to ip-10-0-137-56.us-west-2.compute.internal 35m Normal Scheduled pod/payload-processing-e2e-inf-a-4ab8e5-b77879c55-qptzw Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-4ab8e5-b77879c55-qptzw to ip-10-0-137-56.us-west-2.compute.internal 15m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-863fb0-6cdb97h84f Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-863fb0-6cdb97h84f to ip-10-0-134-11.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-2jrfg Successfully assigned openshift-ingress/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-2jrfg to ip-10-0-137-56.us-west-2.compute.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w6-2594a9-6f4d5b64dc-s7mzb Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w6-2594a9-6f4d5b64dc-s7mzb to ip-10-0-134-11.us-west-2.compute.internal 38m Normal Scheduled pod/payload-processing-e2e-shared-b-w0-e9c3c1-57dd846c75-n5td5 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-e9c3c1-57dd846c75-n5td5 to ip-10-0-134-11.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-s66zc Successfully assigned openshift-ingress/e2e-worker-w5-412743-openshift-default-7bddc8b5df-s66zc to ip-10-0-134-11.us-west-2.compute.internal 22m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-256br Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-256br to ip-10-0-137-56.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-thfvg Successfully assigned openshift-ingress/e2e-worker-w5-412743-openshift-default-7bddc8b5df-thfvg to ip-10-0-137-56.us-west-2.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w4-0da668-dff997c59-8nrck Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-0da668-dff997c59-8nrck to ip-10-0-137-56.us-west-2.compute.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-dcdbd0-7cc8947c99-k4hnc Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-dcdbd0-7cc8947c99-k4hnc to ip-10-0-137-56.us-west-2.compute.internal 47m Normal Scheduled pod/e2e-ait-e7b593fa-openshift-default-655485f877-v8fd7 Successfully assigned openshift-ingress/e2e-ait-e7b593fa-openshift-default-655485f877-v8fd7 to ip-10-0-140-47.us-west-2.compute.internal 65m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-qbvvz Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-qbvvz to ip-10-0-137-56.us-west-2.compute.internal 38m Normal Scheduled pod/e2e-shared-b-w6-2594a9-openshift-default-8694ddf5f7-v4jsh Successfully assigned openshift-ingress/e2e-shared-b-w6-2594a9-openshift-default-8694ddf5f7-v4jsh to ip-10-0-137-56.us-west-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-zq2lm Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-zq2lm to ip-10-0-137-56.us-west-2.compute.internal 63m Normal Scheduled pod/kube-auth-proxy-66d4777458-wxv4x Successfully assigned openshift-ingress/kube-auth-proxy-66d4777458-wxv4x to ip-10-0-137-56.us-west-2.compute.internal 59m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-ztfwh Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-ztfwh to ip-10-0-137-56.us-west-2.compute.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w0-e9c3c1-57685bbfff-ddgqw Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-e9c3c1-57685bbfff-ddgqw to ip-10-0-137-56.us-west-2.compute.internal 41m Normal Scheduled pod/e2e-shared-b-w4-0da668-openshift-default-5467f58b7-tzrb7 Successfully assigned openshift-ingress/e2e-shared-b-w4-0da668-openshift-default-5467f58b7-tzrb7 to ip-10-0-137-56.us-west-2.compute.internal 46m Normal Scheduled pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-4hsbr Successfully assigned openshift-ingress/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-4hsbr to ip-10-0-137-56.us-west-2.compute.internal 46m Normal Scheduled pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-8d2dh Successfully assigned openshift-ingress/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-8d2dh to ip-10-0-140-47.us-west-2.compute.internal 44m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-r57tc Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-r57tc to ip-10-0-137-56.us-west-2.compute.internal 45m Normal Scheduled pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-bcc9n Successfully assigned openshift-ingress/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-bcc9n to ip-10-0-137-56.us-west-2.compute.internal 46m Normal Scheduled pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-f6p8m Successfully assigned openshift-ingress/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-f6p8m to ip-10-0-134-11.us-west-2.compute.internal 39m Normal Scheduled pod/e2e-shared-b-w2-dcdbd0-openshift-default-86cd956cc9-p8h4z Successfully assigned openshift-ingress/e2e-shared-b-w2-dcdbd0-openshift-default-86cd956cc9-p8h4z to ip-10-0-137-56.us-west-2.compute.internal 34m Normal Scheduled pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-gxxl8 Successfully assigned openshift-ingress/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-gxxl8 to ip-10-0-137-56.us-west-2.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-mt-7d5f4de7-cf6c94fdc-zvq97 Successfully assigned openshift-ingress/payload-processing-e2e-mt-7d5f4de7-cf6c94fdc-zvq97 to ip-10-0-137-56.us-west-2.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w6-e030e0-5686f5c779-sx2sx Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w6-e030e0-5686f5c779-sx2sx to ip-10-0-137-56.us-west-2.compute.internal 37m Normal Scheduled pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-c5pmx Successfully assigned openshift-ingress/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-c5pmx to ip-10-0-137-56.us-west-2.compute.internal 38m Normal Scheduled pod/payload-processing-e2e-api-a-c0f56c-679fccf4f-wqpxd Successfully assigned openshift-ingress/payload-processing-e2e-api-a-c0f56c-679fccf4f-wqpxd to ip-10-0-137-56.us-west-2.compute.internal 40m Normal Scheduled pod/payload-processing-e2e-api-a-5b4db8-78dcbd7945-mhhfh Successfully assigned openshift-ingress/payload-processing-e2e-api-a-5b4db8-78dcbd7945-mhhfh to ip-10-0-137-56.us-west-2.compute.internal 39m Normal Scheduled pod/e2e-shared-b-w0-e9c3c1-openshift-default-77447d98f7-8v6vx Successfully assigned openshift-ingress/e2e-shared-b-w0-e9c3c1-openshift-default-77447d98f7-8v6vx to ip-10-0-134-11.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-z5rd6 Successfully assigned openshift-ingress/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-z5rd6 to ip-10-0-140-47.us-west-2.compute.internal 59m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-qkkjs Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-qkkjs to ip-10-0-140-47.us-west-2.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-d58cd3-766b65cfd7-hjv58 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-d58cd3-766b65cfd7-hjv58 to ip-10-0-137-56.us-west-2.compute.internal 22m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-qfcgh Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-qfcgh to ip-10-0-140-47.us-west-2.compute.internal 37m Normal Scheduled pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-gvj7f Successfully assigned openshift-ingress/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-gvj7f to ip-10-0-134-11.us-west-2.compute.internal 45m Normal Scheduled pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-l56vr Successfully assigned openshift-ingress/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-l56vr to ip-10-0-140-47.us-west-2.compute.internal 47m Normal Scheduled pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-l64nw Successfully assigned openshift-ingress/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-l64nw to ip-10-0-134-11.us-west-2.compute.internal 47m Normal Scheduled pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-tlc4t Successfully assigned openshift-ingress/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-tlc4t to ip-10-0-140-47.us-west-2.compute.internal 47m Normal Scheduled pod/payload-processing-e2e-ait-e7b593fa-6cffcfbb4c-fr9cc Successfully assigned openshift-ingress/payload-processing-e2e-ait-e7b593fa-6cffcfbb4c-fr9cc to ip-10-0-134-11.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-9vg69 Successfully assigned openshift-ingress/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-9vg69 to ip-10-0-134-11.us-west-2.compute.internal 46m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-qbn7v Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-qbn7v to ip-10-0-137-56.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-8prk5 Successfully assigned openshift-ingress/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-8prk5 to ip-10-0-134-11.us-west-2.compute.internal 40m Normal Scheduled pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-tf2dv Successfully assigned openshift-ingress/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-tf2dv to ip-10-0-137-56.us-west-2.compute.internal 45m Normal Scheduled pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-vb78p Successfully assigned openshift-ingress/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-vb78p to ip-10-0-134-11.us-west-2.compute.internal 45m Normal Scheduled pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-vbgx5 Successfully assigned openshift-ingress/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-vbgx5 to ip-10-0-137-56.us-west-2.compute.internal 59m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-c2j7f Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-c2j7f to ip-10-0-134-11.us-west-2.compute.internal 64m Normal Scheduled pod/openshift-ai-inference-openshift-ai-inference-55f56bc75f-r4ncj Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-55f56bc75f-r4ncj to ip-10-0-134-11.us-west-2.compute.internal 47m Normal Scheduled pod/e2e-migrate-26fb7237-gw-openshift-default-85889546d5-9fm2n Successfully assigned openshift-ingress/e2e-migrate-26fb7237-gw-openshift-default-85889546d5-9fm2n to ip-10-0-140-47.us-west-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-ait-dd862feb-9c885f569-n957f Successfully assigned openshift-ingress/payload-processing-e2e-ait-dd862feb-9c885f569-n957f to ip-10-0-137-56.us-west-2.compute.internal 39m Normal Scheduled pod/payload-processing-e2e-api-a-f891e6-5b8bd8d58f-wpn4q Successfully assigned openshift-ingress/payload-processing-e2e-api-a-f891e6-5b8bd8d58f-wpn4q to ip-10-0-137-56.us-west-2.compute.internal 6m27s Normal Scheduled pod/payload-processing-e2e-ait-576e86c8-6f8546b5c9-txsdf Successfully assigned openshift-ingress/payload-processing-e2e-ait-576e86c8-6f8546b5c9-txsdf to ip-10-0-137-56.us-west-2.compute.internal 33m Normal Scheduled pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-2rtnt Successfully assigned openshift-ingress/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-2rtnt to ip-10-0-137-56.us-west-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w4-4b86e4-6c67bcf5f-8tb44 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4b86e4-6c67bcf5f-8tb44 to ip-10-0-137-56.us-west-2.compute.internal 33m Normal Scheduled pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-6cwlp Successfully assigned openshift-ingress/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-6cwlp to ip-10-0-134-11.us-west-2.compute.internal 37m Normal Scheduled pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-wqncc Successfully assigned openshift-ingress/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-wqncc to ip-10-0-140-47.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-tl6qf Successfully assigned openshift-ingress/e2e-worker-w4-da9d07-openshift-default-798d9948bc-tl6qf to ip-10-0-140-47.us-west-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-7lvcw Successfully assigned openshift-ingress/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-7lvcw to ip-10-0-137-56.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-56xs9 Successfully assigned openshift-ingress/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-56xs9 to ip-10-0-137-56.us-west-2.compute.internal 41m Normal Scheduled pod/e2e-shared-a-w6-e030e0-openshift-default-5c976d9f4f-lt8jf Successfully assigned openshift-ingress/e2e-shared-a-w6-e030e0-openshift-default-5c976d9f4f-lt8jf to ip-10-0-137-56.us-west-2.compute.internal 22m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-q4c82 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-q4c82 to ip-10-0-134-11.us-west-2.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-ait-49bb6b1a-775dfc4dd5-7jxf7 Successfully assigned openshift-ingress/payload-processing-e2e-ait-49bb6b1a-775dfc4dd5-7jxf7 to ip-10-0-137-56.us-west-2.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-d58cd3-6ff958d595-t9nfb Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-d58cd3-6ff958d595-t9nfb to ip-10-0-137-56.us-west-2.compute.internal 43m Normal Scheduled pod/e2e-shared-a-w4-4b86e4-openshift-default-79c864d79c-hz7fp Successfully assigned openshift-ingress/e2e-shared-a-w4-4b86e4-openshift-default-79c864d79c-hz7fp to ip-10-0-134-11.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-m746l Successfully assigned openshift-ingress/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-m746l to ip-10-0-140-47.us-west-2.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w0-f32ec2-85b9c79d79-kzhp6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-f32ec2-85b9c79d79-kzhp6 to ip-10-0-140-47.us-west-2.compute.internal 46m Normal Scheduled pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-dq5s7 Successfully assigned openshift-ingress/e2e-worker-w4-da9d07-openshift-default-798d9948bc-dq5s7 to ip-10-0-134-11.us-west-2.compute.internal 44m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-cdzdr Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-cdzdr to ip-10-0-134-11.us-west-2.compute.internal 33m Normal Scheduled pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-dln97 Successfully assigned openshift-ingress/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-dln97 to ip-10-0-140-47.us-west-2.compute.internal 58m Normal Scheduled pod/payload-processing-55fbbbf9dc-48lzq Successfully assigned openshift-ingress/payload-processing-55fbbbf9dc-48lzq to ip-10-0-140-47.us-west-2.compute.internal 5m26s Normal Scheduled pod/e2e-authcfg-http-44d22c-openshift-default-6b997dc7fc-dgpzm Successfully assigned openshift-ingress/e2e-authcfg-http-44d22c-openshift-default-6b997dc7fc-dgpzm to ip-10-0-137-56.us-west-2.compute.internal 63m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-7845f94dd5s8cl6 Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-7845f94dd5s8cl6 to ip-10-0-140-47.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-5wzlj Successfully assigned openshift-ingress/e2e-worker-w5-412743-openshift-default-7bddc8b5df-5wzlj to ip-10-0-140-47.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-4r6d7 Successfully assigned openshift-ingress/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-4r6d7 to ip-10-0-134-11.us-west-2.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-ait-49bb6b1a-6d969799c5-6lkcs Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-49bb6b1a-6d969799c5-6lkcs to ip-10-0-137-56.us-west-2.compute.internal 46m Normal Scheduled pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-5dxxk Successfully assigned openshift-ingress/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-5dxxk to ip-10-0-134-11.us-west-2.compute.internal 46m Normal Scheduled pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-97rfb Successfully assigned openshift-ingress/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-97rfb to ip-10-0-140-47.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-cml7r Successfully assigned openshift-ingress/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-cml7r to ip-10-0-137-56.us-west-2.compute.internal 38m Normal Scheduled pod/e2e-api-a-c0f56c-openshift-default-665cfcdf75-fbgz5 Successfully assigned openshift-ingress/e2e-api-a-c0f56c-openshift-default-665cfcdf75-fbgz5 to ip-10-0-134-11.us-west-2.compute.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-a6f4a5-7666bb86c-glmj2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-a6f4a5-7666bb86c-glmj2 to ip-10-0-137-56.us-west-2.compute.internal 41m Normal Scheduled pod/e2e-shared-a-w0-f32ec2-openshift-default-577fcdbf77-zd2vj Successfully assigned openshift-ingress/e2e-shared-a-w0-f32ec2-openshift-default-577fcdbf77-zd2vj to ip-10-0-134-11.us-west-2.compute.internal 44m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-mjxmq Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-mjxmq to ip-10-0-137-56.us-west-2.compute.internal 32m Normal Scheduled pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-d96gv Successfully assigned openshift-ingress/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-d96gv to ip-10-0-140-47.us-west-2.compute.internal 37m Normal Scheduled pod/payload-processing-e2e-api-b-50a19d-867c4c57b5-xknkf Successfully assigned openshift-ingress/payload-processing-e2e-api-b-50a19d-867c4c57b5-xknkf to ip-10-0-137-56.us-west-2.compute.internal 44m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-m47hc Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-m47hc to ip-10-0-140-47.us-west-2.compute.internal 46m Normal Scheduled pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-qwvpf Successfully assigned openshift-ingress/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-qwvpf to ip-10-0-134-11.us-west-2.compute.internal 47m Normal Scheduled pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-rgq28 Successfully assigned openshift-ingress/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-rgq28 to ip-10-0-140-47.us-west-2.compute.internal 46m Normal Scheduled pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-vkvcx Successfully assigned openshift-ingress/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-vkvcx to ip-10-0-137-56.us-west-2.compute.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-412743-6d5cb5bd7f-pzb7l Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-412743-6d5cb5bd7f-pzb7l to ip-10-0-137-56.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-xf6mc Successfully assigned openshift-ingress/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-xf6mc to ip-10-0-140-47.us-west-2.compute.internal 43m Normal Scheduled pod/e2e-mt-f0651cac-openshift-default-7fc6f9555c-pxtlv Successfully assigned openshift-ingress/e2e-mt-f0651cac-openshift-default-7fc6f9555c-pxtlv to ip-10-0-134-11.us-west-2.compute.internal 40m Normal Scheduled pod/e2e-api-a-f891e6-openshift-default-565fdf54d7-56xb6 Successfully assigned openshift-ingress/e2e-api-a-f891e6-openshift-default-565fdf54d7-56xb6 to ip-10-0-137-56.us-west-2.compute.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-lkfxl Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-lkfxl to ip-10-0-137-56.us-west-2.compute.internal 6m27s Normal Scheduled pod/payload-pre-processing-e2e-ait-576e86c8-6876945bcf-2shmj Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-576e86c8-6876945bcf-2shmj to ip-10-0-137-56.us-west-2.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-mt-7d5f4de7-cf6c94fdc-c7jqk Successfully assigned openshift-ingress/payload-processing-e2e-mt-7d5f4de7-cf6c94fdc-c7jqk to ip-10-0-134-11.us-west-2.compute.internal 31m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-l9jbt Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-l9jbt to ip-10-0-137-56.us-west-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-shared-a-w4-4b86e4-5f8755b487-kp5qc Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-4b86e4-5f8755b487-kp5qc to ip-10-0-137-56.us-west-2.compute.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-da9d07-69bb55d779-hg9l2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-da9d07-69bb55d779-hg9l2 to ip-10-0-137-56.us-west-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-ait-dd862feb-6c5d597495-9ftp6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-dd862feb-6c5d597495-9ftp6 to ip-10-0-137-56.us-west-2.compute.internal 22m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-jd2w5 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-jd2w5 to ip-10-0-137-56.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-65wmw Successfully assigned openshift-ingress/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-65wmw to ip-10-0-137-56.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-clnbw Successfully assigned openshift-ingress/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-clnbw to ip-10-0-140-47.us-west-2.compute.internal 29m Normal Scheduled pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-lzwsk Successfully assigned openshift-ingress/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-lzwsk to ip-10-0-137-56.us-west-2.compute.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-h4vr2 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-h4vr2 to ip-10-0-134-11.us-west-2.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-shared-a-w6-e030e0-57f56bb655-wfdmp Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w6-e030e0-57f56bb655-wfdmp to ip-10-0-137-56.us-west-2.compute.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-ait-e7b593fa-5c65dc776c-m8ptm Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-e7b593fa-5c65dc776c-m8ptm to ip-10-0-134-11.us-west-2.compute.internal 37m Normal Scheduled pod/e2e-api-b-50a19d-openshift-default-68c6d7d74c-24fwv Successfully assigned openshift-ingress/e2e-api-b-50a19d-openshift-default-68c6d7d74c-24fwv to ip-10-0-137-56.us-west-2.compute.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-1e5e75-7bf6cbcdcf-6b95p Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-1e5e75-7bf6cbcdcf-6b95p to ip-10-0-134-11.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-shared-a-w2-d58cd3-openshift-default-d9c46776f-vklx7 Successfully assigned openshift-ingress/e2e-shared-a-w2-d58cd3-openshift-default-d9c46776f-vklx7 to ip-10-0-137-56.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-spb2t Successfully assigned openshift-ingress/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-spb2t to ip-10-0-140-47.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-s5lkw Successfully assigned openshift-ingress/e2e-worker-w4-da9d07-openshift-default-798d9948bc-s5lkw to ip-10-0-134-11.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-wx4r4 Successfully assigned openshift-ingress/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-wx4r4 to ip-10-0-134-11.us-west-2.compute.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-api-a-5b4db8-5c87d84775-wgg4h Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-5b4db8-5c87d84775-wgg4h to ip-10-0-137-56.us-west-2.compute.internal 32m Normal Scheduled pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-lzt2n Successfully assigned openshift-ingress/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-lzt2n to ip-10-0-140-47.us-west-2.compute.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-api-a-c0f56c-6bbdcff4c-dswhj Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-c0f56c-6bbdcff4c-dswhj to ip-10-0-137-56.us-west-2.compute.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-api-a-f891e6-5cf4b56747-st4dd Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-f891e6-5cf4b56747-st4dd to ip-10-0-137-56.us-west-2.compute.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-e8ba06-7b548bc5f-m9r6m Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-e8ba06-7b548bc5f-m9r6m to ip-10-0-137-56.us-west-2.compute.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-api-b-50a19d-9874b478c-mqhwf Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-50a19d-9874b478c-mqhwf to ip-10-0-137-56.us-west-2.compute.internal 5m18s Normal Scheduled pod/payload-pre-processing-e2e-authcfg-http-44d22c-649bc5675f-zvc4z Successfully assigned openshift-ingress/payload-pre-processing-e2e-authcfg-http-44d22c-649bc5675f-zvc4z to ip-10-0-137-56.us-west-2.compute.internal 47m Normal Scheduled pod/payload-processing-e2e-migrate-26fb7237-75b5fbc687-xdh58 Successfully assigned openshift-ingress/payload-processing-e2e-migrate-26fb7237-75b5fbc687-xdh58 to ip-10-0-137-56.us-west-2.compute.internal 32m Normal Scheduled pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-lkp42 Successfully assigned openshift-ingress/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-lkp42 to ip-10-0-140-47.us-west-2.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-shared-b-w4-0da668-78b845796f-5srmm Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-0da668-78b845796f-5srmm to ip-10-0-137-56.us-west-2.compute.internal 70m Normal Scheduled pod/router-default-7c7d9879df-g4m8t Successfully assigned openshift-ingress/router-default-7c7d9879df-g4m8t to ip-10-0-140-47.us-west-2.compute.internal 71m Warning FailedScheduling pod/router-default-7c7d9879df-g4m8t no nodes available to schedule pods 75m Warning FailedScheduling pod/router-default-7c7d9879df-g4m8t no nodes available to schedule pods 44m Normal Scheduled pod/payload-pre-processing-e2e-derive-74c6ba99-794886fd59-cqrgc Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-74c6ba99-794886fd59-cqrgc to ip-10-0-134-11.us-west-2.compute.internal 38m Normal Scheduled pod/payload-processing-e2e-shared-b-w6-2594a9-54c85c6687-lnthc Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w6-2594a9-54c85c6687-lnthc to ip-10-0-134-11.us-west-2.compute.internal 46m Normal Scheduled pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-zvppf Successfully assigned openshift-ingress/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-zvppf to ip-10-0-137-56.us-west-2.compute.internal 32m Normal Scheduled pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-jztgt Successfully assigned openshift-ingress/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-jztgt to ip-10-0-134-11.us-west-2.compute.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-4ab8e5-79cc98db87-r95fp Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-4ab8e5-79cc98db87-r95fp to ip-10-0-137-56.us-west-2.compute.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-bc2453-7f45fcd947-x5bwz Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-bc2453-7f45fcd947-x5bwz to ip-10-0-134-11.us-west-2.compute.internal 32m Normal Scheduled pod/payload-processing-e2e-inf-b-bc2453-7658dcf9c7-smwvn Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-bc2453-7658dcf9c7-smwvn to ip-10-0-134-11.us-west-2.compute.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-migrate-26fb7237-7878f946bc-j9xgk Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-26fb7237-7878f946bc-j9xgk to ip-10-0-137-56.us-west-2.compute.internal 32m Normal Scheduled pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-hpj9w Successfully assigned openshift-ingress/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-hpj9w to ip-10-0-137-56.us-west-2.compute.internal 32m Normal Scheduled pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-fcctm Successfully assigned openshift-ingress/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-fcctm to ip-10-0-134-11.us-west-2.compute.internal 57m Normal Scheduled pod/payload-processing-e2e-worker-w6-a6f4a5-7b4d7b4dcc-v2hts Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-a6f4a5-7b4d7b4dcc-v2hts to ip-10-0-137-56.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-5ttl9 Successfully assigned openshift-ingress/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-5ttl9 to ip-10-0-137-56.us-west-2.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-mt-7d5f4de7-ffd6698f9-2r7sb Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-7d5f4de7-ffd6698f9-2r7sb to ip-10-0-137-56.us-west-2.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-mt-7d5f4de7-ffd6698f9-nwczt Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-7d5f4de7-ffd6698f9-nwczt to ip-10-0-134-11.us-west-2.compute.internal 46m Normal Scheduled pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-p7bzb Successfully assigned openshift-ingress/e2e-worker-w4-da9d07-openshift-default-798d9948bc-p7bzb to ip-10-0-140-47.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-j4tb5 Successfully assigned openshift-ingress/e2e-worker-w4-da9d07-openshift-default-798d9948bc-j4tb5 to ip-10-0-137-56.us-west-2.compute.internal 45m Normal Scheduled pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-hx9qr Successfully assigned openshift-ingress/e2e-worker-w4-da9d07-openshift-default-798d9948bc-hx9qr to ip-10-0-137-56.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-2f6wc Successfully assigned openshift-ingress/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-2f6wc to ip-10-0-140-47.us-west-2.compute.internal 64m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-cnm9b Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-cnm9b to ip-10-0-140-47.us-west-2.compute.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-982ae3-6b6857b7dc-92pzx Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-982ae3-6b6857b7dc-92pzx to ip-10-0-140-47.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-f9ghj Successfully assigned openshift-ingress/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-f9ghj to ip-10-0-134-11.us-west-2.compute.internal 44m Normal Scheduled pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-8hddn Successfully assigned openshift-ingress/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-8hddn to ip-10-0-134-11.us-west-2.compute.internal 31m Normal Scheduled pod/maas-default-gateway-openshift-default-6886b96f9-cmr8q Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6886b96f9-cmr8q to ip-10-0-134-11.us-west-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-mt-f0651cac-6dbbdb5867-fdt2g Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-f0651cac-6dbbdb5867-fdt2g to ip-10-0-134-11.us-west-2.compute.internal 76m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-7c7d9879df from 0 to 1 75m Normal SuccessfulCreate replicaset/router-default-7c7d9879df Created pod: router-default-7c7d9879df-g4m8t 75m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 75m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 75m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 71m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 70m Warning FailedMount pod/router-default-7c7d9879df-g4m8t MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 70m Warning FailedMount pod/router-default-7c7d9879df-g4m8t MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 70m Normal AddedInterface pod/router-default-7c7d9879df-g4m8t Add eth0 [10.132.0.9/23] from ovn-kubernetes 70m Normal Started pod/router-default-7c7d9879df-g4m8t Container started 70m Normal Created pod/router-default-7c7d9879df-g4m8t Container created 70m Normal Pulled pod/router-default-7c7d9879df-g4m8t Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 65m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 65m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-qbvvz 65m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 65m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-qbvvz Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 65m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-qbvvz Add eth0 [10.134.0.22/23] from ovn-kubernetes 64m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-qbvvz Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.67s (4.67s including waiting). Image size: 185476477 bytes. 64m Normal Created pod/istiod-openshift-gateway-7876bfcb74-qbvvz Container created 64m Normal Started pod/istiod-openshift-gateway-7876bfcb74-qbvvz Container started 64m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-qbvvz Readiness probe error: HTTP probe failed with statuscode: 503... 64m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-qbvvz Readiness probe failed: HTTP probe failed with statuscode: 503 64m Normal SuccessfulCreate replicaset/openshift-ai-inference-openshift-ai-inference-55f56bc75f Created pod: openshift-ai-inference-openshift-ai-inference-55f56bc75f-r4ncj 64m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-cnm9b Add eth0 [10.132.0.34/23] from ovn-kubernetes 64m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6886b96f9 from 0 to 1 64m Normal AddedInterface pod/openshift-ai-inference-openshift-ai-inference-55f56bc75f-r4ncj Add eth0 [10.133.0.17/23] from ovn-kubernetes 64m Normal Pulling pod/maas-default-gateway-openshift-default-6886b96f9-cnm9b Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 64m Normal EnsuringLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensuring load balancer 64m Normal ScalingReplicaSet deployment/openshift-ai-inference-openshift-ai-inference Scaled up replica set openshift-ai-inference-openshift-ai-inference-55f56bc75f from 0 to 1 64m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-cnm9b 64m Normal BackOff pod/openshift-ai-inference-openshift-ai-inference-55f56bc75f-r4ncj Back-off pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 64m Warning Failed pod/openshift-ai-inference-openshift-ai-inference-55f56bc75f-r4ncj Error: ImagePullBackOff 64m Warning Failed pod/openshift-ai-inference-openshift-ai-inference-55f56bc75f-r4ncj Error: ErrImagePull 64m Warning Failed pod/openshift-ai-inference-openshift-ai-inference-55f56bc75f-r4ncj Failed to pull image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55": unable to pull image: copying system image from manifest list: parsing image configuration: Get "https://cdn01.quay.io/quayio-production-s3/sha256/d4/d401fd69bdddf38670ec84be70c2da90a65270bc7595a1460d76ff52b1a25bea?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20261031%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20261031T002000Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=49c7087d5f335e5f2eec4084ea08eeb6263e9ae139274f7ef76fe74d722c44c2®ion=us-east-1&namespace=redhat-prod&username=redhat-prod+registry_proxy&repo_name=openshift-service-mesh----istio-proxyv2-rhel9&akamai_signature=exp=1793406900~hmac=c9f80f77cda8e66576ab022a8261764a6cdd544d6d47435885ea581de7e8e510": EOF 64m Normal EnsuredLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensured load balancer 64m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-cnm9b Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 9.041s (9.041s including waiting). Image size: 183555333 bytes. 64m Normal Pulling pod/openshift-ai-inference-openshift-ai-inference-55f56bc75f-r4ncj Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 64m Normal Pulled pod/openshift-ai-inference-openshift-ai-inference-55f56bc75f-r4ncj Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.373s (4.373s including waiting). Image size: 183555333 bytes. 64m Normal Created pod/openshift-ai-inference-openshift-ai-inference-55f56bc75f-r4ncj Container created 64m Normal Started pod/openshift-ai-inference-openshift-ai-inference-55f56bc75f-r4ncj Container started 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 64m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-66d4777458 from 0 to 2 63m Normal SuccessfulCreate replicaset/kube-auth-proxy-66d4777458 Created pod: kube-auth-proxy-66d4777458-v9hv8 63m Normal SuccessfulCreate replicaset/kube-auth-proxy-66d4777458 Created pod: kube-auth-proxy-66d4777458-wxv4x 63m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-7845f94dd5 from 0 to 1 63m Normal Pulling pod/kube-auth-proxy-66d4777458-wxv4x Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 63m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-7845f94dd5 Created pod: data-science-gateway-data-science-gateway-class-7845f94dd5s8cl6 63m Normal AddedInterface pod/kube-auth-proxy-66d4777458-wxv4x Add eth0 [10.134.0.24/23] from ovn-kubernetes 63m Normal AddedInterface pod/kube-auth-proxy-66d4777458-v9hv8 Add eth0 [10.133.0.24/23] from ovn-kubernetes 63m Normal Pulling pod/kube-auth-proxy-66d4777458-v9hv8 Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 63m Normal Started pod/data-science-gateway-data-science-gateway-class-7845f94dd5s8cl6 Container started 63m Normal Created pod/data-science-gateway-data-science-gateway-class-7845f94dd5s8cl6 Container created 63m Normal Pulled pod/data-science-gateway-data-science-gateway-class-7845f94dd5s8cl6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-7845f94dd5s8cl6 Add eth0 [10.132.0.42/23] from ovn-kubernetes 63m Normal Pulled pod/kube-auth-proxy-66d4777458-v9hv8 Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 3.517s (3.517s including waiting). Image size: 188904677 bytes. 63m Normal Started pod/kube-auth-proxy-66d4777458-v9hv8 Container started 63m Normal Created pod/kube-auth-proxy-66d4777458-v9hv8 Container created 63m Normal Created pod/kube-auth-proxy-66d4777458-wxv4x Container created 63m Normal Started pod/kube-auth-proxy-66d4777458-wxv4x Container started 63m Normal Pulled pod/kube-auth-proxy-66d4777458-wxv4x Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 4.907s (4.907s including waiting). Image size: 188904677 bytes. 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 63m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-8d22z 62m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-8d22z Add eth0 [10.134.0.26/23] from ovn-kubernetes 62m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 62m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-8d22z Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 62m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-8d22z Container created 62m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-8d22z Container started 62m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-qbvvz Stopping container discovery 62m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-qbvvz 62m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 61m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-cnm9b Readiness probe error: Get "http://10.132.0.34:15021/healthz/ready": read tcp 10.132.0.2:44108->10.132.0.34:15021: read: connection reset by peer... 61m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-cnm9b Container started 61m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-cnm9b Container created 61m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-cnm9b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-cnm9b Readiness probe failed: Get "http://10.132.0.34:15021/healthz/ready": read tcp 10.132.0.2:44108->10.132.0.34:15021: read: connection reset by peer 61m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-cnm9b Startup probe failed: Get "http://10.132.0.34:15021/healthz/ready": dial tcp 10.132.0.34:15021: connect: connection refused 61m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-cnm9b Startup probe error: Get "http://10.132.0.34:15021/healthz/ready": dial tcp 10.132.0.34:15021: connect: connection refused... 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-qkkjs 59m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 59m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-c2j7f 59m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-mk8f6 59m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6886b96f9 from 1 to 4 59m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-qkkjs Container started 59m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-qkkjs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-qkkjs Container created 59m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-qkkjs Add eth0 [10.132.0.61/23] from ovn-kubernetes 59m Normal Pulling pod/maas-default-gateway-openshift-default-6886b96f9-mk8f6 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 59m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-c2j7f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-c2j7f Add eth0 [10.133.0.30/23] from ovn-kubernetes 59m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-mk8f6 Add eth0 [10.134.0.31/23] from ovn-kubernetes 59m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-c2j7f Container created 59m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-c2j7f Container started 59m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-mk8f6 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.15s (4.15s including waiting). Image size: 183555333 bytes. 59m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-mk8f6 Container started 59m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-mk8f6 Container created 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-qkkjs Startup probe failed: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-c2j7f Startup probe failed: Get "http://10.133.0.30:15021/healthz/ready": dial tcp 10.133.0.30:15021: connect: connection refused 59m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-c2j7f Startup probe error: Get "http://10.133.0.30:15021/healthz/ready": dial tcp 10.133.0.30:15021: connect: connection refused... 59m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-qkkjs Startup probe error: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused... 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-mk8f6 Startup probe failed: Get "http://10.134.0.31:15021/healthz/ready": dial tcp 10.134.0.31:15021: connect: connection refused 59m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-mk8f6 Startup probe error: Get "http://10.134.0.31:15021/healthz/ready": dial tcp 10.134.0.31:15021: connect: connection refused... 59m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6886b96f9 from 4 to 5 59m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-ztfwh 59m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-ztfwh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-ztfwh Container started 59m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-ztfwh Container created 59m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-ztfwh Add eth0 [10.134.0.32/23] from ovn-kubernetes 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-ztfwh Startup probe failed: Get "http://10.134.0.32:15021/healthz/ready": dial tcp 10.134.0.32:15021: connect: connection refused 59m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-ztfwh Startup probe error: Get "http://10.134.0.32:15021/healthz/ready": dial tcp 10.134.0.32:15021: connect: connection refused... 58m Warning FailedCreate replicaset/payload-processing-55fbbbf9dc Error creating: pods "payload-processing-55fbbbf9dc-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 58m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 58m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 58m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-55fbbbf9dc from 0 to 1 58m Normal Pulling pod/payload-pre-processing-7b98bb678c-xhpxl Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 58m Normal Pulling pod/payload-processing-55fbbbf9dc-48lzq Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 58m Normal SuccessfulCreate replicaset/payload-processing-55fbbbf9dc Created pod: payload-processing-55fbbbf9dc-48lzq 58m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-xhpxl 58m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-xhpxl Add eth0 [10.132.0.64/23] from ovn-kubernetes 58m Normal AddedInterface pod/payload-processing-55fbbbf9dc-48lzq Add eth0 [10.132.0.65/23] from ovn-kubernetes 58m Normal Created pod/payload-pre-processing-7b98bb678c-xhpxl Container created 58m Normal Pulled pod/payload-processing-55fbbbf9dc-48lzq Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.189s (3.189s including waiting). Image size: 125353965 bytes. 58m Normal Created pod/payload-processing-55fbbbf9dc-48lzq Container created 58m Normal Started pod/payload-processing-55fbbbf9dc-48lzq Container started 58m Normal Started pod/payload-pre-processing-7b98bb678c-xhpxl Container started 58m Normal Pulled pod/payload-pre-processing-7b98bb678c-xhpxl Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.502s (3.502s including waiting). Image size: 125353965 bytes. 58m Normal AddedLabel gateway/e2e-worker-w3-1e5e75 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-1e5e75 58m Normal SuccessfulCreate replicaset/e2e-worker-w2-e8ba06-openshift-default-654dcd546f Created pod: e2e-worker-w2-e8ba06-openshift-default-654dcd546f-cml7r 58m Normal AddedLabel gateway/e2e-worker-w2-e8ba06 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-e8ba06 58m Normal AddedLabel gateway/e2e-worker-w1-982ae3 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-982ae3 58m Normal ScalingReplicaSet deployment/e2e-worker-w6-a6f4a5-openshift-default Scaled up replica set e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75 from 0 to 1 58m Normal AddedLabel gateway/e2e-worker-w4-da9d07 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-da9d07 58m Normal SuccessfulCreate replicaset/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75 Created pod: e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-2f6wc 58m Normal AddedLabel gateway/e2e-worker-w0-101b9e Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-101b9e 58m Normal ScalingReplicaSet deployment/e2e-worker-w2-e8ba06-openshift-default Scaled up replica set e2e-worker-w2-e8ba06-openshift-default-654dcd546f from 0 to 1 58m Normal AddedLabel gateway/e2e-worker-w5-412743 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-412743 58m Normal AddedLabel gateway/e2e-worker-w6-a6f4a5 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-a6f4a5 58m Normal SuccessfulCreate replicaset/e2e-worker-w1-982ae3-openshift-default-64d9fb649f Created pod: e2e-worker-w1-982ae3-openshift-default-64d9fb649f-2jrfg 58m Normal SuccessfulCreate replicaset/e2e-worker-w3-1e5e75-openshift-default-879c67bc5 Created pod: e2e-worker-w3-1e5e75-openshift-default-879c67bc5-wx4r4 58m Normal ScalingReplicaSet deployment/e2e-worker-w1-982ae3-openshift-default Scaled up replica set e2e-worker-w1-982ae3-openshift-default-64d9fb649f from 0 to 1 58m Normal ScalingReplicaSet deployment/e2e-worker-w3-1e5e75-openshift-default Scaled up replica set e2e-worker-w3-1e5e75-openshift-default-879c67bc5 from 0 to 1 58m Normal AddedInterface pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-cml7r Add eth0 [10.134.0.33/23] from ovn-kubernetes 58m Normal SuccessfulCreate replicaset/e2e-worker-w4-da9d07-openshift-default-798d9948bc Created pod: e2e-worker-w4-da9d07-openshift-default-798d9948bc-tl6qf 58m Normal AddedInterface pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-2f6wc Add eth0 [10.132.0.67/23] from ovn-kubernetes 58m Normal AddedInterface pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-wx4r4 Add eth0 [10.133.0.31/23] from ovn-kubernetes 58m Normal ScalingReplicaSet deployment/e2e-worker-w4-da9d07-openshift-default Scaled up replica set e2e-worker-w4-da9d07-openshift-default-798d9948bc from 0 to 1 58m Normal SuccessfulCreate replicaset/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5 Created pod: e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-dz2cl 58m Normal AddedInterface pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-dz2cl Add eth0 [10.133.0.32/23] from ovn-kubernetes 58m Normal ScalingReplicaSet deployment/e2e-worker-w0-101b9e-openshift-default Scaled up replica set e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5 from 0 to 1 58m Normal AddedInterface pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-tl6qf Add eth0 [10.132.0.68/23] from ovn-kubernetes 58m Normal ScalingReplicaSet deployment/e2e-worker-w5-412743-openshift-default Scaled up replica set e2e-worker-w5-412743-openshift-default-7bddc8b5df from 0 to 1 58m Normal AddedInterface pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-2jrfg Add eth0 [10.134.0.34/23] from ovn-kubernetes 58m Normal SuccessfulCreate replicaset/e2e-worker-w5-412743-openshift-default-7bddc8b5df Created pod: e2e-worker-w5-412743-openshift-default-7bddc8b5df-cg79s 57m Normal AddedInterface pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-cg79s Add eth0 [10.132.0.69/23] from ovn-kubernetes 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-e8ba06 Scaled up replica set payload-pre-processing-e2e-worker-w2-e8ba06-7b548bc5f from 0 to 1 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-e8ba06-7b548bc5f Error creating: pods "payload-pre-processing-e2e-worker-w2-e8ba06-7b548bc5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-e8ba06: serviceaccount "payload-processing-e2e-worker-w2-e8ba06" not found 57m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-e8ba06-bb9fb667f Error creating: pods "payload-processing-e2e-worker-w2-e8ba06-bb9fb667f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-e8ba06: serviceaccount "payload-processing-e2e-worker-w2-e8ba06" not found 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-e8ba06 Scaled up replica set payload-processing-e2e-worker-w2-e8ba06-bb9fb667f from 0 to 1 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-e8ba06-bb9fb667f Created pod: payload-processing-e2e-worker-w2-e8ba06-bb9fb667f-x6fzk 57m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-e8ba06-7b548bc5f-m9r6m Add eth0 [10.134.0.35/23] from ovn-kubernetes 57m Normal AddedInterface pod/payload-processing-e2e-worker-w2-e8ba06-bb9fb667f-x6fzk Add eth0 [10.134.0.36/23] from ovn-kubernetes 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-e8ba06-7b548bc5f Created pod: payload-pre-processing-e2e-worker-w2-e8ba06-7b548bc5f-m9r6m 57m Normal Pulling pod/payload-pre-processing-e2e-worker-w2-e8ba06-7b548bc5f-m9r6m Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 57m Normal Pulling pod/payload-processing-e2e-worker-w2-e8ba06-bb9fb667f-x6fzk Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 57m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-e8ba06-7b548bc5f-m9r6m Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.158s (3.158s including waiting). Image size: 125353965 bytes. 57m Normal Started pod/payload-processing-e2e-worker-w2-e8ba06-bb9fb667f-x6fzk Container started 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-1e5e75-7bf6cbcdcf Created pod: payload-pre-processing-e2e-worker-w3-1e5e75-7bf6cbcdcf-6b95p 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-1e5e75-7bf6cbcdcf Error creating: pods "payload-pre-processing-e2e-worker-w3-1e5e75-7bf6cbcdcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-1e5e75: serviceaccount "payload-processing-e2e-worker-w3-1e5e75" not found 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-1e5e75 Scaled up replica set payload-processing-e2e-worker-w3-1e5e75-ff6df9bb9 from 0 to 1 57m Normal Pulled pod/payload-processing-e2e-worker-w2-e8ba06-bb9fb667f-x6fzk Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3s (3s including waiting). Image size: 125353965 bytes. 57m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-1e5e75-ff6df9bb9 Error creating: pods "payload-processing-e2e-worker-w3-1e5e75-ff6df9bb9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-1e5e75: serviceaccount "payload-processing-e2e-worker-w3-1e5e75" not found 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-1e5e75 Scaled up replica set payload-pre-processing-e2e-worker-w3-1e5e75-7bf6cbcdcf from 0 to 1 57m Normal Created pod/payload-pre-processing-e2e-worker-w2-e8ba06-7b548bc5f-m9r6m Container created 57m Normal Started pod/payload-pre-processing-e2e-worker-w2-e8ba06-7b548bc5f-m9r6m Container started 57m Normal Created pod/payload-processing-e2e-worker-w2-e8ba06-bb9fb667f-x6fzk Container created 57m Normal Started pod/payload-processing-e2e-worker-w3-1e5e75-ff6df9bb9-d888l Container started 57m Normal Pulled pod/payload-processing-e2e-worker-w3-1e5e75-ff6df9bb9-d888l Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-1e5e75-7bf6cbcdcf-6b95p Add eth0 [10.133.0.33/23] from ovn-kubernetes 57m Normal Created pod/payload-processing-e2e-worker-w3-1e5e75-ff6df9bb9-d888l Container created 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-1e5e75-ff6df9bb9 Created pod: payload-processing-e2e-worker-w3-1e5e75-ff6df9bb9-d888l 57m Normal Pulling pod/payload-pre-processing-e2e-worker-w3-1e5e75-7bf6cbcdcf-6b95p Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 57m Normal AddedInterface pod/payload-processing-e2e-worker-w3-1e5e75-ff6df9bb9-d888l Add eth0 [10.134.0.37/23] from ovn-kubernetes 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-a6f4a5 Scaled up replica set payload-processing-e2e-worker-w6-a6f4a5-7b4d7b4dcc from 0 to 1 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-a6f4a5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-a6f4a5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-a6f4a5 Scaled up replica set payload-pre-processing-e2e-worker-w6-a6f4a5-7666bb86c from 0 to 1 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-a6f4a5-7666bb86c Created pod: payload-pre-processing-e2e-worker-w6-a6f4a5-7666bb86c-glmj2 57m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-a6f4a5-7b4d7b4dcc Error creating: pods "payload-processing-e2e-worker-w6-a6f4a5-7b4d7b4dcc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-a6f4a5: serviceaccount "payload-processing-e2e-worker-w6-a6f4a5" not found 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-a6f4a5-7b4d7b4dcc Created pod: payload-processing-e2e-worker-w6-a6f4a5-7b4d7b4dcc-v2hts 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-a6f4a5-7666bb86c Error creating: pods "payload-pre-processing-e2e-worker-w6-a6f4a5-7666bb86c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-a6f4a5: serviceaccount "payload-processing-e2e-worker-w6-a6f4a5" not found 57m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-a6f4a5-7666bb86c-glmj2 Add eth0 [10.134.0.38/23] from ovn-kubernetes 57m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-a6f4a5-7666bb86c-glmj2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-1e5e75-7bf6cbcdcf-6b95p Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.853s (2.853s including waiting). Image size: 125353965 bytes. 57m Normal Created pod/payload-pre-processing-e2e-worker-w3-1e5e75-7bf6cbcdcf-6b95p Container created 57m Normal Started pod/payload-pre-processing-e2e-worker-w3-1e5e75-7bf6cbcdcf-6b95p Container started 57m Normal Pulled pod/payload-processing-e2e-worker-w6-a6f4a5-7b4d7b4dcc-v2hts Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/payload-processing-e2e-worker-w6-a6f4a5-7b4d7b4dcc-v2hts Add eth0 [10.134.0.39/23] from ovn-kubernetes 57m Normal Started pod/payload-pre-processing-e2e-worker-w6-a6f4a5-7666bb86c-glmj2 Container started 57m Normal Created pod/payload-pre-processing-e2e-worker-w6-a6f4a5-7666bb86c-glmj2 Container created 57m Normal Started pod/payload-processing-e2e-worker-w6-a6f4a5-7b4d7b4dcc-v2hts Container started 57m Normal Created pod/payload-processing-e2e-worker-w6-a6f4a5-7b4d7b4dcc-v2hts Container created 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-982ae3 Scaled up replica set payload-pre-processing-e2e-worker-w1-982ae3-6b6857b7dc from 0 to 1 57m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-982ae3-6c4b5dc9b9 Error creating: pods "payload-processing-e2e-worker-w1-982ae3-6c4b5dc9b9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-982ae3: serviceaccount "payload-processing-e2e-worker-w1-982ae3" not found 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-982ae3-6b6857b7dc Error creating: pods "payload-pre-processing-e2e-worker-w1-982ae3-6b6857b7dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-982ae3: serviceaccount "payload-processing-e2e-worker-w1-982ae3" not found 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-982ae3 Scaled up replica set payload-processing-e2e-worker-w1-982ae3-6c4b5dc9b9 from 0 to 1 57m Normal Created pod/payload-pre-processing-e2e-worker-w1-982ae3-6b6857b7dc-92pzx Container created 57m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-982ae3-6b6857b7dc-92pzx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/payload-processing-e2e-worker-w1-982ae3-6c4b5dc9b9-mmctb Add eth0 [10.133.0.35/23] from ovn-kubernetes 57m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-982ae3-6b6857b7dc-92pzx Add eth0 [10.132.0.71/23] from ovn-kubernetes 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-982ae3-6b6857b7dc Created pod: payload-pre-processing-e2e-worker-w1-982ae3-6b6857b7dc-92pzx 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-982ae3-6c4b5dc9b9 Created pod: payload-processing-e2e-worker-w1-982ae3-6c4b5dc9b9-mmctb 57m Normal Started pod/payload-pre-processing-e2e-worker-w1-982ae3-6b6857b7dc-92pzx Container started 57m Normal Started pod/payload-processing-e2e-worker-w1-982ae3-6c4b5dc9b9-mmctb Container started 57m Normal Pulled pod/payload-processing-e2e-worker-w1-982ae3-6c4b5dc9b9-mmctb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Created pod/payload-processing-e2e-worker-w1-982ae3-6c4b5dc9b9-mmctb Container created 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-412743-688d66585f Created pod: payload-processing-e2e-worker-w5-412743-688d66585f-vk5r7 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-412743 Scaled up replica set payload-pre-processing-e2e-worker-w5-412743-6d5cb5bd7f from 0 to 1 57m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-412743-688d66585f Error creating: pods "payload-processing-e2e-worker-w5-412743-688d66585f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-412743: serviceaccount "payload-processing-e2e-worker-w5-412743" not found 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-412743-6d5cb5bd7f Created pod: payload-pre-processing-e2e-worker-w5-412743-6d5cb5bd7f-pzb7l 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-412743-6d5cb5bd7f Error creating: pods "payload-pre-processing-e2e-worker-w5-412743-6d5cb5bd7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-412743: serviceaccount "payload-processing-e2e-worker-w5-412743" not found 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-412743 Scaled up replica set payload-processing-e2e-worker-w5-412743-688d66585f from 0 to 1 57m Normal Created pod/payload-pre-processing-e2e-worker-w5-412743-6d5cb5bd7f-pzb7l Container created 57m Normal Created pod/payload-processing-e2e-worker-w5-412743-688d66585f-vk5r7 Container created 57m Normal Started pod/payload-pre-processing-e2e-worker-w5-412743-6d5cb5bd7f-pzb7l Container started 57m Normal Started pod/payload-processing-e2e-worker-w5-412743-688d66585f-vk5r7 Container started 57m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-412743-6d5cb5bd7f-pzb7l Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/payload-processing-e2e-worker-w5-412743-688d66585f-vk5r7 Add eth0 [10.134.0.41/23] from ovn-kubernetes 57m Normal Pulled pod/payload-processing-e2e-worker-w5-412743-688d66585f-vk5r7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-412743-6d5cb5bd7f-pzb7l Add eth0 [10.134.0.40/23] from ovn-kubernetes 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-da9d07 Scaled up replica set payload-processing-e2e-worker-w4-da9d07-5848fd449 from 0 to 1 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-da9d07 Scaled up replica set payload-pre-processing-e2e-worker-w4-da9d07-69bb55d779 from 0 to 1 57m Normal Started pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-wx4r4 Container started 57m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-da9d07-5848fd449 Error creating: pods "payload-processing-e2e-worker-w4-da9d07-5848fd449-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-da9d07: serviceaccount "payload-processing-e2e-worker-w4-da9d07" not found 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-da9d07-69bb55d779 Created pod: payload-pre-processing-e2e-worker-w4-da9d07-69bb55d779-hg9l2 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-da9d07-69bb55d779 Error creating: pods "payload-pre-processing-e2e-worker-w4-da9d07-69bb55d779-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-da9d07: serviceaccount "payload-processing-e2e-worker-w4-da9d07" not found 57m Normal Pulled pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-wx4r4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-da9d07-5848fd449 Created pod: payload-processing-e2e-worker-w4-da9d07-5848fd449-s4wwm 57m Normal Created pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-wx4r4 Container created 57m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-da9d07-69bb55d779-hg9l2 Add eth0 [10.134.0.42/23] from ovn-kubernetes 57m Normal Started pod/payload-processing-e2e-worker-w4-da9d07-5848fd449-s4wwm Container started 57m Normal Created pod/payload-pre-processing-e2e-worker-w4-da9d07-69bb55d779-hg9l2 Container created 57m Normal Started pod/payload-pre-processing-e2e-worker-w4-da9d07-69bb55d779-hg9l2 Container started 57m Normal Pulled pod/payload-processing-e2e-worker-w4-da9d07-5848fd449-s4wwm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Created pod/payload-processing-e2e-worker-w4-da9d07-5848fd449-s4wwm Container created 57m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-da9d07-69bb55d779-hg9l2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/payload-processing-e2e-worker-w4-da9d07-5848fd449-s4wwm Add eth0 [10.134.0.43/23] from ovn-kubernetes 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-101b9e-688fd6f5d7 Created pod: payload-pre-processing-e2e-worker-w0-101b9e-688fd6f5d7-7nrcm 57m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-101b9e-6c44b8947 Error creating: pods "payload-processing-e2e-worker-w0-101b9e-6c44b8947-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-101b9e: serviceaccount "payload-processing-e2e-worker-w0-101b9e" not found 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-101b9e Scaled up replica set payload-processing-e2e-worker-w0-101b9e-6c44b8947 from 0 to 1 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-101b9e-688fd6f5d7 Error creating: pods "payload-pre-processing-e2e-worker-w0-101b9e-688fd6f5d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-101b9e: serviceaccount "payload-processing-e2e-worker-w0-101b9e" not found 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-101b9e Scaled up replica set payload-pre-processing-e2e-worker-w0-101b9e-688fd6f5d7 from 0 to 1 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-101b9e-6c44b8947 Created pod: payload-processing-e2e-worker-w0-101b9e-6c44b8947-88qn7 57m Normal Created pod/payload-pre-processing-e2e-worker-w0-101b9e-688fd6f5d7-7nrcm Container created 57m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-101b9e-688fd6f5d7-7nrcm Add eth0 [10.134.0.44/23] from ovn-kubernetes 57m Normal AddedInterface pod/payload-processing-e2e-worker-w0-101b9e-6c44b8947-88qn7 Add eth0 [10.133.0.36/23] from ovn-kubernetes 57m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-101b9e-688fd6f5d7-7nrcm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Pulled pod/payload-processing-e2e-worker-w0-101b9e-6c44b8947-88qn7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Created pod/payload-processing-e2e-worker-w0-101b9e-6c44b8947-88qn7 Container created 57m Normal Started pod/payload-pre-processing-e2e-worker-w0-101b9e-688fd6f5d7-7nrcm Container started 57m Normal Started pod/payload-processing-e2e-worker-w0-101b9e-6c44b8947-88qn7 Container started 57m Warning ProbeError pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-wx4r4 Startup probe error: Get "http://10.133.0.31:15021/healthz/ready": dial tcp 10.133.0.31:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-wx4r4 Startup probe failed: Get "http://10.133.0.31:15021/healthz/ready": dial tcp 10.133.0.31:15021: connect: connection refused 57m Normal Pulled pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-2f6wc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-2f6wc Container created 57m Normal Started pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-2jrfg Container started 57m Normal Started pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-2f6wc Container started 57m Normal Pulled pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-2jrfg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-2jrfg Container created 57m Warning Unhealthy pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-2jrfg Startup probe failed: Get "http://10.134.0.34:15021/healthz/ready": dial tcp 10.134.0.34:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-2jrfg Startup probe error: Get "http://10.134.0.34:15021/healthz/ready": dial tcp 10.134.0.34:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-2f6wc Startup probe error: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-2f6wc Startup probe failed: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-1e5e75-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-1e5e75-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Normal Started pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-cml7r Container started 57m Normal Pulled pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-cml7r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-cml7r Container created 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-1e5e75-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning Unhealthy pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-cml7r Startup probe failed: Get "http://10.134.0.33:15021/healthz/ready": dial tcp 10.134.0.33:15021: connect: connection refused 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-e8ba06-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-e8ba06-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-1e5e75-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning ProbeError pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-cml7r Startup probe error: Get "http://10.134.0.33:15021/healthz/ready": dial tcp 10.134.0.33:15021: connect: connection refused... 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-982ae3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-982ae3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Normal Pulled pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-dz2cl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Pulled pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-tl6qf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-tl6qf Container started 56m Warning ProbeError pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-tl6qf Readiness probe error: Get "http://10.132.0.68:15021/healthz/ready": read tcp 10.132.0.2:56690->10.132.0.68:15021: read: connection reset by peer... 56m Warning Unhealthy pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-tl6qf Readiness probe failed: Get "http://10.132.0.68:15021/healthz/ready": read tcp 10.132.0.2:56690->10.132.0.68:15021: read: connection reset by peer 56m Normal Created pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-tl6qf Container created 56m Normal Created pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-dz2cl Container created 56m Normal Started pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-dz2cl Container started 56m Warning ProbeError pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-dz2cl Startup probe error: Get "http://10.133.0.32:15021/healthz/ready": dial tcp 10.133.0.32:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-dz2cl Startup probe failed: Get "http://10.133.0.32:15021/healthz/ready": dial tcp 10.133.0.32:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-tl6qf Startup probe error: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-tl6qf Startup probe failed: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused 56m Normal Created pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-cg79s Container created 56m Normal Pulled pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-cg79s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-cg79s Container started 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-a6f4a5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-a6f4a5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-101b9e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-982ae3-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-101b9e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-982ae3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-da9d07-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-da9d07-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning ProbeError pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-cg79s Startup probe error: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-cg79s Startup probe failed: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-e8ba06-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-e8ba06-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-412743-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-101b9e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-101b9e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-412743-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-da9d07-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-da9d07-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-412743-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-412743-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Normal ScalingReplicaSet deployment/e2e-worker-w6-a6f4a5-openshift-default Scaled up replica set e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75 from 1 to 2 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-a6f4a5-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulCreate replicaset/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75 Created pod: e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-7lvcw 54m Normal AddedInterface pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-7lvcw Add eth0 [10.134.0.56/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-7lvcw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-7lvcw Container created 54m Normal Started pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-7lvcw Container started 54m Warning ProbeError pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-7lvcw Startup probe error: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-7lvcw Startup probe failed: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused 53m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6886b96f9 Deleted pod: maas-default-gateway-openshift-default-6886b96f9-qkkjs 53m Normal Killing pod/maas-default-gateway-openshift-default-6886b96f9-qkkjs Stopping container istio-proxy 53m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: All metrics below target 53m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-6886b96f9 from 5 to 3 53m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6886b96f9 Deleted pod: maas-default-gateway-openshift-default-6886b96f9-ztfwh 53m Normal Killing pod/maas-default-gateway-openshift-default-6886b96f9-ztfwh Stopping container istio-proxy 52m Normal ScalingReplicaSet deployment/e2e-worker-w6-a6f4a5-openshift-default Scaled up replica set e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75 from 2 to 3 52m Normal SuccessfulCreate replicaset/e2e-worker-w2-e8ba06-openshift-default-654dcd546f Created pod: e2e-worker-w2-e8ba06-openshift-default-654dcd546f-xf6mc 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-e8ba06-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 52m Normal Pulled pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-xf6mc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-xf6mc Add eth0 [10.132.0.77/23] from ovn-kubernetes 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-a6f4a5-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 52m Normal SuccessfulCreate replicaset/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75 Created pod: e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-9vg69 52m Normal ScalingReplicaSet deployment/e2e-worker-w2-e8ba06-openshift-default Scaled up replica set e2e-worker-w2-e8ba06-openshift-default-654dcd546f from 1 to 2 52m Normal Created pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-xf6mc Container created 52m Normal AddedInterface pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-9vg69 Add eth0 [10.133.0.49/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-xf6mc Container started 52m Normal Started pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-9vg69 Container started 52m Normal Created pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-9vg69 Container created 52m Normal Pulled pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-9vg69 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Warning Unhealthy pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-9vg69 Startup probe failed: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-9vg69 Startup probe error: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-xf6mc Startup probe error: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-xf6mc Startup probe failed: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused 52m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 2; reason: All metrics below target 52m Normal Killing pod/maas-default-gateway-openshift-default-6886b96f9-mk8f6 Stopping container istio-proxy 52m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-6886b96f9 from 3 to 2 52m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6886b96f9 Deleted pod: maas-default-gateway-openshift-default-6886b96f9-mk8f6 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-1e5e75-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 51m Normal ScalingReplicaSet deployment/e2e-worker-w3-1e5e75-openshift-default Scaled up replica set e2e-worker-w3-1e5e75-openshift-default-879c67bc5 from 1 to 2 51m Normal SuccessfulCreate replicaset/e2e-worker-w3-1e5e75-openshift-default-879c67bc5 Created pod: e2e-worker-w3-1e5e75-openshift-default-879c67bc5-spb2t 51m Normal ScalingReplicaSet deployment/e2e-worker-w4-da9d07-openshift-default Scaled up replica set e2e-worker-w4-da9d07-openshift-default-798d9948bc from 1 to 3 51m Normal AddedInterface pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-s5lkw Add eth0 [10.133.0.50/23] from ovn-kubernetes 51m Normal SuccessfulCreate replicaset/e2e-worker-w4-da9d07-openshift-default-798d9948bc Created pod: e2e-worker-w4-da9d07-openshift-default-798d9948bc-j4tb5 51m Normal SuccessfulCreate replicaset/e2e-worker-w4-da9d07-openshift-default-798d9948bc Created pod: e2e-worker-w4-da9d07-openshift-default-798d9948bc-s5lkw 51m Normal AddedInterface pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-j4tb5 Add eth0 [10.134.0.80/23] from ovn-kubernetes 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-da9d07-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 51m Normal AddedInterface pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-spb2t Add eth0 [10.132.0.78/23] from ovn-kubernetes 51m Normal Started pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-spb2t Container started 51m Normal Pulled pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-spb2t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-spb2t Container created 51m Normal Pulled pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-j4tb5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-j4tb5 Container created 51m Normal Started pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-j4tb5 Container started 51m Normal Pulled pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-s5lkw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-s5lkw Container created 51m Normal Started pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-s5lkw Container started 51m Warning ProbeError pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-j4tb5 Startup probe error: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-j4tb5 Startup probe failed: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-spb2t Startup probe error: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-spb2t Startup probe failed: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-s5lkw Startup probe error: Get "http://10.133.0.50:15021/healthz/ready": dial tcp 10.133.0.50:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-s5lkw Startup probe failed: Get "http://10.133.0.50:15021/healthz/ready": dial tcp 10.133.0.50:15021: connect: connection refused 51m Normal ScalingReplicaSet deployment/e2e-worker-w5-412743-openshift-default Scaled up replica set e2e-worker-w5-412743-openshift-default-7bddc8b5df from 1 to 2 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-412743-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/e2e-worker-w5-412743-openshift-default-7bddc8b5df Created pod: e2e-worker-w5-412743-openshift-default-7bddc8b5df-thfvg 51m Normal Pulled pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-thfvg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-thfvg Container started 51m Normal AddedInterface pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-thfvg Add eth0 [10.134.0.82/23] from ovn-kubernetes 51m Normal Created pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-thfvg Container created 51m Warning Unhealthy pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-thfvg Startup probe failed: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-thfvg Startup probe error: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused... 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-101b9e-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5 Created pod: e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-ngfr5 51m Normal ScalingReplicaSet deployment/e2e-worker-w0-101b9e-openshift-default Scaled up replica set e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5 from 1 to 2 51m Normal Started pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-ngfr5 Container started 51m Normal Created pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-ngfr5 Container created 51m Normal Pulled pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-ngfr5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-ngfr5 Add eth0 [10.132.0.80/23] from ovn-kubernetes 51m Warning Unhealthy pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-ngfr5 Startup probe failed: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-ngfr5 Startup probe error: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused... 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-a6f4a5-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 50m Normal SuccessfulCreate replicaset/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75 Created pod: e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-z5rd6 50m Normal ScalingReplicaSet deployment/e2e-worker-w6-a6f4a5-openshift-default Scaled up replica set e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75 from 3 to 4 50m Normal Started pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-z5rd6 Container started 50m Normal Created pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-z5rd6 Container created 50m Normal Pulled pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-z5rd6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-z5rd6 Add eth0 [10.132.0.82/23] from ovn-kubernetes 50m Warning Unhealthy pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-z5rd6 Startup probe failed: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-z5rd6 Startup probe error: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-xf6mc Readiness probe failed: Get "http://10.132.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-xf6mc Readiness probe error: Get "http://10.132.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Normal SuccessfulCreate replicaset/e2e-worker-w2-e8ba06-openshift-default-654dcd546f Created pod: e2e-worker-w2-e8ba06-openshift-default-654dcd546f-4r6d7 48m Normal Started pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-4r6d7 Container started 48m Normal AddedInterface pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-4r6d7 Add eth0 [10.133.0.55/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-4r6d7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-4r6d7 Container created 48m Warning ProbeError pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-4r6d7 Startup probe error: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-4r6d7 Startup probe failed: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused 48m Normal AddedInterface pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-56xs9 Add eth0 [10.134.0.93/23] from ovn-kubernetes 48m Normal AddedInterface pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-8prk5 Add eth0 [10.133.0.56/23] from ovn-kubernetes 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-a6f4a5-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 48m Normal ScalingReplicaSet deployment/e2e-worker-w6-a6f4a5-openshift-default Scaled up replica set e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75 from 4 to 6 48m Normal SuccessfulCreate replicaset/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75 Created pod: e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-56xs9 48m Normal SuccessfulCreate replicaset/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75 Created pod: e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-8prk5 48m Normal Pulled pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-56xs9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-56xs9 Container created 48m Normal Started pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-56xs9 Container started 48m Normal Started pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-8prk5 Container started 48m Normal Created pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-8prk5 Container created 48m Normal Pulled pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-8prk5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Warning ProbeError pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-8prk5 Startup probe error: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-8prk5 Startup probe failed: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-56xs9 Startup probe failed: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-56xs9 Startup probe error: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused... 48m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6886b96f9 Deleted pod: maas-default-gateway-openshift-default-6886b96f9-c2j7f 48m Normal Killing pod/maas-default-gateway-openshift-default-6886b96f9-c2j7f Stopping container istio-proxy 48m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-6886b96f9 from 2 to 1 48m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 1; reason: All metrics below target 48m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-c2j7f Readiness probe failed: HTTP probe failed with statuscode: 503 48m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-c2j7f Readiness probe error: HTTP probe failed with statuscode: 503... 48m Normal ScalingReplicaSet deployment/e2e-worker-w3-1e5e75-openshift-default Scaled up replica set e2e-worker-w3-1e5e75-openshift-default-879c67bc5 from 2 to 4 48m Normal SuccessfulCreate replicaset/e2e-worker-w3-1e5e75-openshift-default-879c67bc5 Created pod: e2e-worker-w3-1e5e75-openshift-default-879c67bc5-65wmw 48m Normal SuccessfulCreate replicaset/e2e-worker-w3-1e5e75-openshift-default-879c67bc5 Created pod: e2e-worker-w3-1e5e75-openshift-default-879c67bc5-clnbw 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-1e5e75-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 48m Normal Pulled pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-clnbw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-65wmw Add eth0 [10.134.0.96/23] from ovn-kubernetes 48m Normal Started pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-clnbw Container started 48m Normal Created pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-clnbw Container created 48m Normal AddedInterface pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-clnbw Add eth0 [10.132.0.85/23] from ovn-kubernetes 48m Normal Started pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-65wmw Container started 48m Normal Created pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-65wmw Container created 48m Normal Pulled pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-65wmw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Warning Unhealthy pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-clnbw Startup probe failed: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-65wmw Startup probe failed: Get "http://10.134.0.96:15021/healthz/ready": dial tcp 10.134.0.96:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-clnbw Startup probe error: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-65wmw Startup probe error: Get "http://10.134.0.96:15021/healthz/ready": dial tcp 10.134.0.96:15021: connect: connection refused... 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-101b9e-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 48m Normal SuccessfulCreate replicaset/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5 Created pod: e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sf5lc 48m Normal ScalingReplicaSet deployment/e2e-worker-w0-101b9e-openshift-default Scaled up replica set e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5 from 2 to 3 48m Normal AddedInterface pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-5wzlj Add eth0 [10.132.0.86/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sf5lc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sf5lc Container started 48m Normal SuccessfulCreate replicaset/e2e-worker-w5-412743-openshift-default-7bddc8b5df Created pod: e2e-worker-w5-412743-openshift-default-7bddc8b5df-5wzlj 48m Normal SuccessfulCreate replicaset/e2e-worker-w5-412743-openshift-default-7bddc8b5df Created pod: e2e-worker-w5-412743-openshift-default-7bddc8b5df-s66zc 48m Normal ScalingReplicaSet deployment/e2e-worker-w5-412743-openshift-default Scaled up replica set e2e-worker-w5-412743-openshift-default-7bddc8b5df from 2 to 4 48m Normal Created pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sf5lc Container created 48m Normal AddedInterface pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sf5lc Add eth0 [10.134.0.99/23] from ovn-kubernetes 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-412743-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 48m Normal AddedInterface pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-s66zc Add eth0 [10.133.0.57/23] from ovn-kubernetes 48m Normal Created pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-s66zc Container created 48m Normal Started pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-s66zc Container started 48m Normal Started pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-5wzlj Container started 48m Normal Pulled pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-5wzlj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-5wzlj Container created 48m Normal Pulled pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-s66zc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Warning Unhealthy pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-5wzlj Startup probe failed: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-s66zc Startup probe error: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-s66zc Startup probe failed: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-5wzlj Startup probe error: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sf5lc Startup probe failed: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sf5lc Startup probe error: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sf5lc Startup probe error: Get "http://10.134.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sf5lc Startup probe failed: Get "http://10.134.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Normal SuccessfulCreate replicaset/e2e-worker-w2-e8ba06-openshift-default-654dcd546f Created pod: e2e-worker-w2-e8ba06-openshift-default-654dcd546f-rgq28 47m Normal ScalingReplicaSet deployment/e2e-worker-w2-e8ba06-openshift-default Scaled up replica set e2e-worker-w2-e8ba06-openshift-default-654dcd546f from 3 to 4 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-e8ba06-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 47m Normal Created pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-rgq28 Container created 47m Normal Started pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-rgq28 Container started 47m Normal AddedInterface pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-rgq28 Add eth0 [10.132.0.87/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-rgq28 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Warning Unhealthy pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-rgq28 Startup probe failed: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-rgq28 Startup probe error: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused... 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-982ae3-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 47m Normal SuccessfulCreate replicaset/e2e-worker-w1-982ae3-openshift-default-64d9fb649f Created pod: e2e-worker-w1-982ae3-openshift-default-64d9fb649f-l64nw 47m Normal SuccessfulCreate replicaset/e2e-worker-w1-982ae3-openshift-default-64d9fb649f Created pod: e2e-worker-w1-982ae3-openshift-default-64d9fb649f-tlc4t 47m Normal ScalingReplicaSet deployment/e2e-worker-w1-982ae3-openshift-default Scaled up replica set e2e-worker-w1-982ae3-openshift-default-64d9fb649f from 1 to 3 47m Normal Created pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-tlc4t Container created 47m Normal Pulled pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-tlc4t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Started pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-tlc4t Container started 47m Normal AddedInterface pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-tlc4t Add eth0 [10.132.0.88/23] from ovn-kubernetes 47m Normal AddedInterface pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-l64nw Add eth0 [10.133.0.58/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-l64nw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-l64nw Container created 47m Normal Started pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-l64nw Container started 47m Warning Unhealthy pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-tlc4t Startup probe failed: Get "http://10.132.0.88:15021/healthz/ready": dial tcp 10.132.0.88:15021: connect: connection refused 47m Warning Unhealthy pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-l64nw Startup probe failed: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-l64nw Startup probe error: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused... 47m Warning ProbeError pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-tlc4t Startup probe error: Get "http://10.132.0.88:15021/healthz/ready": dial tcp 10.132.0.88:15021: connect: connection refused... 47m Normal AddedLabel gateway/e2e-migrate-26fb7237-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-26fb7237-gw 47m Normal SuccessfulCreate replicaset/e2e-migrate-26fb7237-gw-openshift-default-85889546d5 Created pod: e2e-migrate-26fb7237-gw-openshift-default-85889546d5-9fm2n 47m Normal ScalingReplicaSet deployment/e2e-migrate-26fb7237-gw-openshift-default Scaled up replica set e2e-migrate-26fb7237-gw-openshift-default-85889546d5 from 0 to 1 47m Normal AddedInterface pod/e2e-migrate-26fb7237-gw-openshift-default-85889546d5-9fm2n Add eth0 [10.132.0.91/23] from ovn-kubernetes 47m Normal SuccessfulCreate replicaset/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5 Created pod: e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-qm9sf 47m Normal SuccessfulCreate replicaset/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5 Created pod: e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sxdqn 47m Normal AddedLabel gateway/e2e-ait-e7b593fa Added label istio.io/rev=openshift-gateway to gateway e2e-ait-e7b593fa 47m Normal ScalingReplicaSet deployment/e2e-ait-e7b593fa-openshift-default Scaled up replica set e2e-ait-e7b593fa-openshift-default-655485f877 from 0 to 1 47m Normal SuccessfulCreate replicaset/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5 Created pod: e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-tskrl 47m Normal SuccessfulCreate replicaset/e2e-ait-e7b593fa-openshift-default-655485f877 Created pod: e2e-ait-e7b593fa-openshift-default-655485f877-v8fd7 47m Normal ScalingReplicaSet deployment/e2e-worker-w0-101b9e-openshift-default Scaled up replica set e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5 from 3 to 6 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-101b9e-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 47m Normal AddedInterface pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-tskrl Add eth0 [10.132.0.92/23] from ovn-kubernetes 47m Normal Created pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-qm9sf Container created 47m Normal AddedInterface pod/e2e-ait-e7b593fa-openshift-default-655485f877-v8fd7 Add eth0 [10.132.0.93/23] from ovn-kubernetes 47m Normal Started pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sxdqn Container started 47m Normal Started pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-tskrl Container started 47m Normal Created pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-tskrl Container created 47m Normal AddedInterface pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-qm9sf Add eth0 [10.133.0.60/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-tskrl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sxdqn Add eth0 [10.134.0.118/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sxdqn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Pulled pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-qm9sf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sxdqn Container created 47m Normal Started pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-qm9sf Container started 47m Normal ScalingReplicaSet deployment/e2e-ait-49bb6b1a-openshift-default Scaled up replica set e2e-ait-49bb6b1a-openshift-default-8bcd84ddc from 0 to 1 47m Normal SuccessfulCreate replicaset/e2e-ait-49bb6b1a-openshift-default-8bcd84ddc Created pod: e2e-ait-49bb6b1a-openshift-default-8bcd84ddc-9tqjs 47m Normal AddedLabel gateway/e2e-ait-49bb6b1a Added label istio.io/rev=openshift-gateway to gateway e2e-ait-49bb6b1a 47m Normal AddedInterface pod/e2e-ait-49bb6b1a-openshift-default-8bcd84ddc-9tqjs Add eth0 [10.132.0.94/23] from ovn-kubernetes 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-26fb7237 Scaled up replica set payload-processing-e2e-migrate-26fb7237-75b5fbc687 from 0 to 1 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-26fb7237 Scaled up replica set payload-pre-processing-e2e-migrate-26fb7237-7878f946bc from 0 to 1 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-26fb7237-7878f946bc Error creating: pods "payload-pre-processing-e2e-migrate-26fb7237-7878f946bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-26fb7237: serviceaccount "payload-processing-e2e-migrate-26fb7237" not found 47m Warning ProbeError pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-tskrl Startup probe error: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sxdqn Startup probe failed: Get "http://10.134.0.118:15021/healthz/ready": dial tcp 10.134.0.118:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sxdqn Startup probe error: Get "http://10.134.0.118:15021/healthz/ready": dial tcp 10.134.0.118:15021: connect: connection refused... 47m Warning FailedCreate replicaset/payload-processing-e2e-migrate-26fb7237-75b5fbc687 Error creating: pods "payload-processing-e2e-migrate-26fb7237-75b5fbc687-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-26fb7237: serviceaccount "payload-processing-e2e-migrate-26fb7237" not found 47m Warning Unhealthy pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-tskrl Startup probe failed: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-26fb7237-75b5fbc687 Created pod: payload-processing-e2e-migrate-26fb7237-75b5fbc687-xdh58 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-26fb7237-7878f946bc Created pod: payload-pre-processing-e2e-migrate-26fb7237-7878f946bc-j9xgk 47m Warning Unhealthy pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-qm9sf Startup probe failed: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused 47m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-26fb7237-7878f946bc-j9xgk Add eth0 [10.134.0.119/23] from ovn-kubernetes 47m Normal Started pod/payload-pre-processing-e2e-migrate-26fb7237-7878f946bc-j9xgk Container started 47m Warning ProbeError pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-qm9sf Startup probe error: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused... 47m Normal Started pod/payload-processing-e2e-migrate-26fb7237-75b5fbc687-xdh58 Container started 47m Normal Created pod/payload-processing-e2e-migrate-26fb7237-75b5fbc687-xdh58 Container created 47m Normal AddedInterface pod/payload-processing-e2e-migrate-26fb7237-75b5fbc687-xdh58 Add eth0 [10.134.0.120/23] from ovn-kubernetes 47m Normal Created pod/payload-pre-processing-e2e-migrate-26fb7237-7878f946bc-j9xgk Container created 47m Normal Pulled pod/payload-pre-processing-e2e-migrate-26fb7237-7878f946bc-j9xgk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Pulled pod/payload-processing-e2e-migrate-26fb7237-75b5fbc687-xdh58 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-e7b593fa-5c65dc776c Created pod: payload-pre-processing-e2e-ait-e7b593fa-5c65dc776c-m8ptm 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-e7b593fa Scaled up replica set payload-processing-e2e-ait-e7b593fa-6cffcfbb4c from 0 to 1 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-e7b593fa Scaled up replica set payload-pre-processing-e2e-ait-e7b593fa-5c65dc776c from 0 to 1 47m Warning FailedCreate replicaset/payload-processing-e2e-ait-e7b593fa-6cffcfbb4c Error creating: pods "payload-processing-e2e-ait-e7b593fa-6cffcfbb4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-e7b593fa: serviceaccount "payload-processing-e2e-ait-e7b593fa" not found 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-e7b593fa-5c65dc776c Error creating: pods "payload-pre-processing-e2e-ait-e7b593fa-5c65dc776c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-e7b593fa: serviceaccount "payload-processing-e2e-ait-e7b593fa" not found 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-e7b593fa-6cffcfbb4c Created pod: payload-processing-e2e-ait-e7b593fa-6cffcfbb4c-fr9cc 47m Normal AddedInterface pod/payload-pre-processing-e2e-ait-e7b593fa-5c65dc776c-m8ptm Add eth0 [10.133.0.61/23] from ovn-kubernetes 47m Normal Pulled pod/payload-processing-e2e-ait-e7b593fa-6cffcfbb4c-fr9cc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/payload-processing-e2e-ait-e7b593fa-6cffcfbb4c-fr9cc Add eth0 [10.133.0.62/23] from ovn-kubernetes 47m Normal Created pod/payload-processing-e2e-ait-e7b593fa-6cffcfbb4c-fr9cc Container created 47m Normal Started pod/payload-processing-e2e-ait-e7b593fa-6cffcfbb4c-fr9cc Container started 46m Normal Created pod/payload-pre-processing-e2e-ait-e7b593fa-5c65dc776c-m8ptm Container created 46m Normal Pulled pod/payload-pre-processing-e2e-ait-e7b593fa-5c65dc776c-m8ptm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Started pod/payload-pre-processing-e2e-ait-e7b593fa-5c65dc776c-m8ptm Container started 46m Normal SuccessfulCreate replicaset/e2e-worker-w2-e8ba06-openshift-default-654dcd546f Created pod: e2e-worker-w2-e8ba06-openshift-default-654dcd546f-vkvcx 46m Normal SuccessfulCreate replicaset/e2e-worker-w2-e8ba06-openshift-default-654dcd546f Created pod: e2e-worker-w2-e8ba06-openshift-default-654dcd546f-qwvpf 46m Normal SuccessfulCreate replicaset/e2e-worker-w2-e8ba06-openshift-default-654dcd546f Created pod: e2e-worker-w2-e8ba06-openshift-default-654dcd546f-97rfb 46m Normal SuccessfulCreate replicaset/e2e-worker-w2-e8ba06-openshift-default-654dcd546f Created pod: e2e-worker-w2-e8ba06-openshift-default-654dcd546f-5dxxk 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-e8ba06-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 46m Normal ScalingReplicaSet deployment/e2e-worker-w2-e8ba06-openshift-default Scaled up replica set e2e-worker-w2-e8ba06-openshift-default-654dcd546f from 4 to 8 46m Normal Created pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-5dxxk Container created 46m Normal Started pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-97rfb Container started 46m Normal Started pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-qwvpf Container started 46m Normal Created pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-qwvpf Container created 46m Normal Pulled pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-qwvpf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-5dxxk Add eth0 [10.133.0.63/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-5dxxk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-qwvpf Add eth0 [10.133.0.64/23] from ovn-kubernetes 46m Normal Started pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-5dxxk Container started 46m Normal Started pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-vkvcx Container started 46m Normal Created pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-vkvcx Container created 46m Normal Pulled pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-vkvcx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-vkvcx Add eth0 [10.134.0.122/23] from ovn-kubernetes 46m Normal AddedInterface pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-97rfb Add eth0 [10.132.0.95/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-97rfb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-97rfb Container created 46m Warning FailedCreate replicaset/payload-processing-e2e-ait-49bb6b1a-775dfc4dd5 Error creating: pods "payload-processing-e2e-ait-49bb6b1a-775dfc4dd5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-49bb6b1a: serviceaccount "payload-processing-e2e-ait-49bb6b1a" not found 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-49bb6b1a Scaled up replica set payload-processing-e2e-ait-49bb6b1a-775dfc4dd5 from 0 to 1 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-49bb6b1a-6d969799c5 Error creating: pods "payload-pre-processing-e2e-ait-49bb6b1a-6d969799c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-49bb6b1a: serviceaccount "payload-processing-e2e-ait-49bb6b1a" not found 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-49bb6b1a Scaled up replica set payload-pre-processing-e2e-ait-49bb6b1a-6d969799c5 from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-49bb6b1a-6d969799c5 Created pod: payload-pre-processing-e2e-ait-49bb6b1a-6d969799c5-6lkcs 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-49bb6b1a-775dfc4dd5 Created pod: payload-processing-e2e-ait-49bb6b1a-775dfc4dd5-7jxf7 46m Normal Pulled pod/payload-pre-processing-e2e-ait-49bb6b1a-6d969799c5-6lkcs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Pulled pod/payload-processing-e2e-ait-49bb6b1a-775dfc4dd5-7jxf7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/payload-pre-processing-e2e-ait-49bb6b1a-6d969799c5-6lkcs Container created 46m Normal Started pod/payload-pre-processing-e2e-ait-49bb6b1a-6d969799c5-6lkcs Container started 46m Normal AddedInterface pod/payload-processing-e2e-ait-49bb6b1a-775dfc4dd5-7jxf7 Add eth0 [10.134.0.125/23] from ovn-kubernetes 46m Normal AddedInterface pod/payload-pre-processing-e2e-ait-49bb6b1a-6d969799c5-6lkcs Add eth0 [10.134.0.124/23] from ovn-kubernetes 46m Normal Started pod/payload-processing-e2e-ait-49bb6b1a-775dfc4dd5-7jxf7 Container started 46m Normal Created pod/payload-processing-e2e-ait-49bb6b1a-775dfc4dd5-7jxf7 Container created 46m Warning ProbeError pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-5dxxk Startup probe error: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-5dxxk Startup probe failed: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-qwvpf Startup probe failed: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-qwvpf Startup probe error: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-vkvcx Startup probe error: Get "http://10.134.0.122:15021/healthz/ready": dial tcp 10.134.0.122:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-vkvcx Startup probe failed: Get "http://10.134.0.122:15021/healthz/ready": dial tcp 10.134.0.122:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-97rfb Startup probe failed: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-97rfb Startup probe error: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused... 46m Normal ScalingReplicaSet deployment/e2e-worker-w4-da9d07-openshift-default Scaled up replica set e2e-worker-w4-da9d07-openshift-default-798d9948bc from 3 to 5 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-1e5e75-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 46m Normal SuccessfulCreate replicaset/e2e-worker-w4-da9d07-openshift-default-798d9948bc Created pod: e2e-worker-w4-da9d07-openshift-default-798d9948bc-dq5s7 46m Normal ScalingReplicaSet deployment/e2e-worker-w3-1e5e75-openshift-default Scaled up replica set e2e-worker-w3-1e5e75-openshift-default-879c67bc5 from 4 to 5 46m Normal SuccessfulCreate replicaset/e2e-worker-w3-1e5e75-openshift-default-879c67bc5 Created pod: e2e-worker-w3-1e5e75-openshift-default-879c67bc5-zvppf 46m Normal SuccessfulCreate replicaset/e2e-worker-w4-da9d07-openshift-default-798d9948bc Created pod: e2e-worker-w4-da9d07-openshift-default-798d9948bc-p7bzb 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-da9d07-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 46m Normal SuccessfulCreate replicaset/e2e-worker-w1-982ae3-openshift-default-64d9fb649f Created pod: e2e-worker-w1-982ae3-openshift-default-64d9fb649f-4hsbr 46m Normal AddedInterface pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-4hsbr Add eth0 [10.134.0.127/23] from ovn-kubernetes 46m Normal SuccessfulCreate replicaset/e2e-worker-w1-982ae3-openshift-default-64d9fb649f Created pod: e2e-worker-w1-982ae3-openshift-default-64d9fb649f-f6p8m 46m Normal SuccessfulCreate replicaset/e2e-worker-w1-982ae3-openshift-default-64d9fb649f Created pod: e2e-worker-w1-982ae3-openshift-default-64d9fb649f-8d2dh 46m Normal Created pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-zvppf Container created 46m Normal Started pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-f6p8m Container started 46m Normal Pulled pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-zvppf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-dq5s7 Add eth0 [10.133.0.65/23] from ovn-kubernetes 46m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 46m Normal Pulled pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-dq5s7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 46m Normal Started pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-dq5s7 Container started 46m Normal AddedInterface pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-p7bzb Add eth0 [10.132.0.96/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-p7bzb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-p7bzb Container created 46m Normal Started pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-p7bzb Container started 46m Normal Pulled pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-f6p8m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-zvppf Container started 46m Normal AddedInterface pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-f6p8m Add eth0 [10.133.0.66/23] from ovn-kubernetes 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-e7b593fa-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-e7b593fa-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal AddedInterface pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-zvppf Add eth0 [10.134.0.126/23] from ovn-kubernetes 46m Normal ScalingReplicaSet deployment/e2e-worker-w1-982ae3-openshift-default Scaled up replica set e2e-worker-w1-982ae3-openshift-default-64d9fb649f from 3 to 6 46m Normal Created pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-f6p8m Container created 46m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-xjq9h 46m Normal AddedInterface pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-8d2dh Add eth0 [10.132.0.97/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-8d2dh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-8d2dh Container created 46m Normal Created pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-dq5s7 Container created 46m Normal Started pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-8d2dh Container started 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-982ae3-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 46m Normal Started pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-4hsbr Container started 46m Normal Created pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-4hsbr Container created 46m Normal Pulled pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-4hsbr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-xjq9h Add eth0 [10.132.0.98/23] from ovn-kubernetes 46m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-xjq9h Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 46m Warning Unhealthy pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-4hsbr Startup probe failed: Get "http://10.134.0.127:15021/healthz/ready": dial tcp 10.134.0.127:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-zvppf Startup probe error: Get "http://10.134.0.126:15021/healthz/ready": dial tcp 10.134.0.126:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-zvppf Startup probe failed: Get "http://10.134.0.126:15021/healthz/ready": dial tcp 10.134.0.126:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-8d2dh Startup probe failed: Get "http://10.132.0.97:15021/healthz/ready": dial tcp 10.132.0.97:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-8d2dh Startup probe error: Get "http://10.132.0.97:15021/healthz/ready": dial tcp 10.132.0.97:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-4hsbr Startup probe error: Get "http://10.134.0.127:15021/healthz/ready": dial tcp 10.134.0.127:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-p7bzb Startup probe failed: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-p7bzb Startup probe error: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-dq5s7 Startup probe error: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-f6p8m Startup probe failed: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-f6p8m Startup probe error: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-dq5s7 Startup probe failed: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused 46m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6886b96f9 from 1 to 2 46m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 46m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-qbn7v 46m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-xjq9h Container created 46m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-qbn7v Container started 46m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-xjq9h Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 8.201s (8.201s including waiting). Image size: 185476477 bytes. 46m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-qbn7v Container created 46m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-qbn7v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-qbn7v Add eth0 [10.134.0.128/23] from ovn-kubernetes 46m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-xjq9h Container started 46m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-xjq9h Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-xjq9h Readiness probe error: HTTP probe failed with statuscode: 503... 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-qbn7v Startup probe failed: Get "http://10.134.0.128:15021/healthz/ready": dial tcp 10.134.0.128:15021: connect: connection refused 46m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-qbn7v Startup probe error: Get "http://10.134.0.128:15021/healthz/ready": dial tcp 10.134.0.128:15021: connect: connection refused... 46m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6886b96f9 from 2 to 3 46m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-wspmm 46m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 46m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-wspmm Add eth0 [10.133.0.68/23] from ovn-kubernetes 46m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-wspmm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-wspmm Container created 46m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-wspmm Container started 46m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-wspmm Startup probe error: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused... 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-wspmm Startup probe failed: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused 46m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-wspmm Startup probe error: Get "http://10.133.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-qbn7v Readiness probe error: Get "http://10.134.0.128:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-qbn7v Readiness probe failed: Get "http://10.134.0.128:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-wspmm Startup probe failed: Get "http://10.133.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Normal Created pod/e2e-migrate-26fb7237-gw-openshift-default-85889546d5-9fm2n Container created 45m Normal Started pod/e2e-migrate-26fb7237-gw-openshift-default-85889546d5-9fm2n Container started 45m Normal Pulled pod/e2e-migrate-26fb7237-gw-openshift-default-85889546d5-9fm2n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal ScalingReplicaSet deployment/e2e-worker-w4-da9d07-openshift-default Scaled up replica set e2e-worker-w4-da9d07-openshift-default-798d9948bc from 5 to 6 45m Normal SuccessfulCreate replicaset/e2e-worker-w4-da9d07-openshift-default-798d9948bc Created pod: e2e-worker-w4-da9d07-openshift-default-798d9948bc-hx9qr 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-da9d07-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 45m Normal Created pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-hx9qr Container created 45m Normal Pulled pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-hx9qr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-hx9qr Add eth0 [10.134.0.129/23] from ovn-kubernetes 45m Normal Started pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-hx9qr Container started 45m Warning Unhealthy pod/e2e-migrate-26fb7237-gw-openshift-default-85889546d5-9fm2n Startup probe failed: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused 45m Warning ProbeError pod/e2e-migrate-26fb7237-gw-openshift-default-85889546d5-9fm2n Startup probe error: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused... 45m Warning ProbeError pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-hx9qr Startup probe error: Get "http://10.134.0.129:15021/healthz/ready": dial tcp 10.134.0.129:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-hx9qr Startup probe failed: Get "http://10.134.0.129:15021/healthz/ready": dial tcp 10.134.0.129:15021: connect: connection refused 45m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-lkfxl 45m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 45m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-h4vr2 45m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6886b96f9 from 3 to 5 45m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-lkfxl Container created 45m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-lkfxl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-lkfxl Container started 45m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-h4vr2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-h4vr2 Container started 45m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-h4vr2 Container created 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-h4vr2 Add eth0 [10.133.0.71/23] from ovn-kubernetes 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-lkfxl Add eth0 [10.134.0.130/23] from ovn-kubernetes 45m Normal SuccessfulCreate replicaset/e2e-worker-w1-982ae3-openshift-default-64d9fb649f Created pod: e2e-worker-w1-982ae3-openshift-default-64d9fb649f-vbgx5 45m Normal SuccessfulCreate replicaset/e2e-worker-w1-982ae3-openshift-default-64d9fb649f Created pod: e2e-worker-w1-982ae3-openshift-default-64d9fb649f-bcc9n 45m Normal SuccessfulCreate replicaset/e2e-worker-w1-982ae3-openshift-default-64d9fb649f Created pod: e2e-worker-w1-982ae3-openshift-default-64d9fb649f-vb78p 45m Normal SuccessfulCreate replicaset/e2e-worker-w1-982ae3-openshift-default-64d9fb649f Created pod: e2e-worker-w1-982ae3-openshift-default-64d9fb649f-l56vr 45m Normal ScalingReplicaSet deployment/e2e-worker-w1-982ae3-openshift-default Scaled up replica set e2e-worker-w1-982ae3-openshift-default-64d9fb649f from 6 to 10 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-982ae3-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 45m Normal Created pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-l56vr Container created 45m Normal Created pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-vbgx5 Container created 45m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-h4vr2 Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused... 45m Normal AddedInterface pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-l56vr Add eth0 [10.132.0.99/23] from ovn-kubernetes 45m Normal AddedInterface pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-vbgx5 Add eth0 [10.134.0.132/23] from ovn-kubernetes 45m Normal Started pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-bcc9n Container started 45m Normal Created pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-bcc9n Container created 45m Normal Pulled pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-bcc9n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-bcc9n Add eth0 [10.134.0.131/23] from ovn-kubernetes 45m Normal Pulled pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-vbgx5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-vb78p Container created 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-h4vr2 Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused 45m Normal Started pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-vbgx5 Container started 45m Normal Pulled pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-l56vr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-vb78p Container started 45m Normal Pulled pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-vb78p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-l56vr Container started 45m Normal AddedInterface pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-vb78p Add eth0 [10.133.0.72/23] from ovn-kubernetes 45m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-lkfxl Startup probe error: Get "http://10.134.0.130:15021/healthz/ready": dial tcp 10.134.0.130:15021: connect: connection refused... 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-lkfxl Startup probe failed: Get "http://10.134.0.130:15021/healthz/ready": dial tcp 10.134.0.130:15021: connect: connection refused 45m Warning Unhealthy pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-vb78p Startup probe failed: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused 45m Warning ProbeError pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-vb78p Startup probe error: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused... 45m Warning ProbeError pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-bcc9n Startup probe error: Get "http://10.134.0.131:15021/healthz/ready": dial tcp 10.134.0.131:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-vbgx5 Startup probe failed: Get "http://10.134.0.132:15021/healthz/ready": dial tcp 10.134.0.132:15021: connect: connection refused 45m Warning ProbeError pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-l56vr Startup probe error: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused... 45m Warning ProbeError pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-vbgx5 Startup probe error: Get "http://10.134.0.132:15021/healthz/ready": dial tcp 10.134.0.132:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-l56vr Startup probe failed: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused 45m Warning Unhealthy pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-bcc9n Startup probe failed: Get "http://10.134.0.131:15021/healthz/ready": dial tcp 10.134.0.131:15021: connect: connection refused 45m Warning Unhealthy pod/e2e-ait-e7b593fa-openshift-default-655485f877-v8fd7 Readiness probe failed: Get "http://10.132.0.93:15021/healthz/ready": read tcp 10.132.0.2:39120->10.132.0.93:15021: read: connection reset by peer 45m Warning ProbeError pod/e2e-ait-e7b593fa-openshift-default-655485f877-v8fd7 Readiness probe error: Get "http://10.132.0.93:15021/healthz/ready": read tcp 10.132.0.2:39120->10.132.0.93:15021: read: connection reset by peer... 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-49bb6b1a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-49bb6b1a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal Started pod/e2e-ait-e7b593fa-openshift-default-655485f877-v8fd7 Container started 45m Normal Started pod/e2e-ait-49bb6b1a-openshift-default-8bcd84ddc-9tqjs Container started 45m Normal Created pod/e2e-ait-e7b593fa-openshift-default-655485f877-v8fd7 Container created 45m Normal Pulled pod/e2e-ait-e7b593fa-openshift-default-655485f877-v8fd7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-ait-49bb6b1a-openshift-default-8bcd84ddc-9tqjs Container created 45m Normal Pulled pod/e2e-ait-49bb6b1a-openshift-default-8bcd84ddc-9tqjs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Warning Unhealthy pod/e2e-ait-49bb6b1a-openshift-default-8bcd84ddc-9tqjs Startup probe failed: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused 45m Warning ProbeError pod/e2e-ait-49bb6b1a-openshift-default-8bcd84ddc-9tqjs Startup probe error: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused... 45m Warning ProbeError pod/e2e-ait-e7b593fa-openshift-default-655485f877-v8fd7 Startup probe error: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-ait-e7b593fa-openshift-default-655485f877-v8fd7 Startup probe failed: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-26fb7237-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-26fb7237-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Normal Killing pod/payload-processing-e2e-migrate-26fb7237-75b5fbc687-xdh58 Stopping container payload-processing 45m Normal Killing pod/payload-pre-processing-e2e-migrate-26fb7237-7878f946bc-j9xgk Stopping container payload-pre-processing 44m Normal Killing pod/payload-pre-processing-e2e-ait-e7b593fa-5c65dc776c-m8ptm Stopping container payload-pre-processing 44m Normal Killing pod/payload-processing-e2e-ait-e7b593fa-6cffcfbb4c-fr9cc Stopping container payload-processing 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-e7b593fa-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-e7b593fa-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-49bb6b1a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-49bb6b1a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal Killing pod/payload-processing-e2e-ait-49bb6b1a-775dfc4dd5-7jxf7 Stopping container payload-processing 44m Normal Killing pod/payload-pre-processing-e2e-ait-49bb6b1a-6d969799c5-6lkcs Stopping container payload-pre-processing 44m Normal Killing pod/e2e-migrate-26fb7237-gw-openshift-default-85889546d5-9fm2n Stopping container istio-proxy 44m Normal ScalingReplicaSet deployment/e2e-ait-dd862feb-openshift-default Scaled up replica set e2e-ait-dd862feb-openshift-default-67b7c76895 from 0 to 1 44m Normal AddedLabel gateway/e2e-ait-dd862feb Added label istio.io/rev=openshift-gateway to gateway e2e-ait-dd862feb 44m Normal SuccessfulCreate replicaset/e2e-ait-dd862feb-openshift-default-67b7c76895 Created pod: e2e-ait-dd862feb-openshift-default-67b7c76895-zrjbk 44m Normal AddedInterface pod/e2e-ait-dd862feb-openshift-default-67b7c76895-zrjbk Add eth0 [10.134.0.137/23] from ovn-kubernetes 44m Normal Killing pod/e2e-ait-e7b593fa-openshift-default-655485f877-v8fd7 Stopping container istio-proxy 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-dd862feb-6c5d597495 Created pod: payload-pre-processing-e2e-ait-dd862feb-6c5d597495-9ftp6 44m Normal ScalingReplicaSet deployment/e2e-derive-74c6ba99-openshift-default Scaled up replica set e2e-derive-74c6ba99-openshift-default-65cd5b8f85 from 0 to 1 44m Normal AddedLabel gateway/e2e-derive-74c6ba99 Added label istio.io/rev=openshift-gateway to gateway e2e-derive-74c6ba99 44m Warning FailedCreate replicaset/payload-processing-e2e-ait-dd862feb-9c885f569 Error creating: pods "payload-processing-e2e-ait-dd862feb-9c885f569-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-dd862feb: serviceaccount "payload-processing-e2e-ait-dd862feb" not found 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-dd862feb-6c5d597495 Error creating: pods "payload-pre-processing-e2e-ait-dd862feb-6c5d597495-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-dd862feb: serviceaccount "payload-processing-e2e-ait-dd862feb" not found 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-dd862feb Scaled up replica set payload-pre-processing-e2e-ait-dd862feb-6c5d597495 from 0 to 1 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-dd862feb Scaled up replica set payload-processing-e2e-ait-dd862feb-9c885f569 from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-dd862feb-9c885f569 Created pod: payload-processing-e2e-ait-dd862feb-9c885f569-n957f 44m Normal SuccessfulCreate replicaset/e2e-derive-74c6ba99-openshift-default-65cd5b8f85 Created pod: e2e-derive-74c6ba99-openshift-default-65cd5b8f85-8hddn 44m Normal Started pod/payload-processing-e2e-ait-dd862feb-9c885f569-n957f Container started 44m Normal AddedInterface pod/payload-processing-e2e-ait-dd862feb-9c885f569-n957f Add eth0 [10.134.0.139/23] from ovn-kubernetes 44m Normal Pulled pod/payload-processing-e2e-ait-dd862feb-9c885f569-n957f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-pre-processing-e2e-ait-dd862feb-6c5d597495-9ftp6 Add eth0 [10.134.0.138/23] from ovn-kubernetes 44m Normal Created pod/payload-processing-e2e-ait-dd862feb-9c885f569-n957f Container created 44m Normal Started pod/payload-pre-processing-e2e-ait-dd862feb-6c5d597495-9ftp6 Container started 44m Normal Created pod/payload-pre-processing-e2e-ait-dd862feb-6c5d597495-9ftp6 Container created 44m Normal Pulled pod/payload-pre-processing-e2e-ait-dd862feb-6c5d597495-9ftp6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-8hddn Add eth0 [10.133.0.73/23] from ovn-kubernetes 44m Normal Killing pod/e2e-ait-49bb6b1a-openshift-default-8bcd84ddc-9tqjs Stopping container istio-proxy 44m Warning ProbeError pod/e2e-ait-49bb6b1a-openshift-default-8bcd84ddc-9tqjs Readiness probe error: HTTP probe failed with statuscode: 503... 44m Warning Unhealthy pod/e2e-ait-49bb6b1a-openshift-default-8bcd84ddc-9tqjs Readiness probe failed: HTTP probe failed with statuscode: 503 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-74c6ba99 Scaled up replica set payload-processing-e2e-derive-74c6ba99-cd559cfbf from 0 to 1 44m Warning FailedCreate replicaset/payload-processing-e2e-derive-74c6ba99-cd559cfbf Error creating: pods "payload-processing-e2e-derive-74c6ba99-cd559cfbf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-74c6ba99: serviceaccount "payload-processing-e2e-derive-74c6ba99" not found 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-74c6ba99-cd559cfbf Created pod: payload-processing-e2e-derive-74c6ba99-cd559cfbf-v6nns 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-74c6ba99 Scaled up replica set payload-pre-processing-e2e-derive-74c6ba99-794886fd59 from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-74c6ba99-794886fd59 Created pod: payload-pre-processing-e2e-derive-74c6ba99-794886fd59-cqrgc 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-74c6ba99-794886fd59 Error creating: pods "payload-pre-processing-e2e-derive-74c6ba99-794886fd59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-74c6ba99: serviceaccount "payload-processing-e2e-derive-74c6ba99" not found 44m Normal AddedInterface pod/payload-processing-e2e-derive-74c6ba99-cd559cfbf-v6nns Add eth0 [10.133.0.75/23] from ovn-kubernetes 44m Normal Pulled pod/payload-pre-processing-e2e-derive-74c6ba99-794886fd59-cqrgc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-processing-e2e-derive-74c6ba99-cd559cfbf-v6nns Container created 44m Normal Pulled pod/payload-processing-e2e-derive-74c6ba99-cd559cfbf-v6nns Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Started pod/payload-pre-processing-e2e-derive-74c6ba99-794886fd59-cqrgc Container started 44m Normal Started pod/payload-processing-e2e-derive-74c6ba99-cd559cfbf-v6nns Container started 44m Normal Created pod/payload-pre-processing-e2e-derive-74c6ba99-794886fd59-cqrgc Container created 44m Normal AddedInterface pod/payload-pre-processing-e2e-derive-74c6ba99-794886fd59-cqrgc Add eth0 [10.133.0.74/23] from ovn-kubernetes 44m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-mjxmq 44m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-m47hc 44m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-thjff 44m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-cdzdr 44m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-r57tc 44m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-thjff Container created 44m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-thjff Add eth0 [10.132.0.101/23] from ovn-kubernetes 44m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-thjff Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-thjff Container started 44m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-cdzdr Container created 44m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-mjxmq Container started 44m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-mjxmq Container created 44m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-cdzdr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-cdzdr Add eth0 [10.133.0.76/23] from ovn-kubernetes 44m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-r57tc Add eth0 [10.134.0.141/23] from ovn-kubernetes 44m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-m47hc Container started 44m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-r57tc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-mjxmq Add eth0 [10.134.0.142/23] from ovn-kubernetes 44m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-m47hc Add eth0 [10.132.0.100/23] from ovn-kubernetes 44m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-mjxmq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-r57tc Container started 44m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-r57tc Container created 44m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-m47hc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-m47hc Container created 44m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-cdzdr Container started 43m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-m47hc Startup probe failed: Get "http://10.132.0.100:15021/healthz/ready": dial tcp 10.132.0.100:15021: connect: connection refused 43m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-mjxmq Startup probe failed: Get "http://10.134.0.142:15021/healthz/ready": dial tcp 10.134.0.142:15021: connect: connection refused 43m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-thjff Startup probe error: Get "http://10.132.0.101:15021/healthz/ready": dial tcp 10.132.0.101:15021: connect: connection refused... 43m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-m47hc Startup probe error: Get "http://10.132.0.100:15021/healthz/ready": dial tcp 10.132.0.100:15021: connect: connection refused... 43m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-mjxmq Startup probe error: Get "http://10.134.0.142:15021/healthz/ready": dial tcp 10.134.0.142:15021: connect: connection refused... 43m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-thjff Startup probe failed: Get "http://10.132.0.101:15021/healthz/ready": dial tcp 10.132.0.101:15021: connect: connection refused 43m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-cdzdr Startup probe failed: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused 43m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-r57tc Startup probe error: Get "http://10.134.0.141:15021/healthz/ready": dial tcp 10.134.0.141:15021: connect: connection refused... 43m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-r57tc Startup probe failed: Get "http://10.134.0.141:15021/healthz/ready": dial tcp 10.134.0.141:15021: connect: connection refused 43m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-cdzdr Startup probe error: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused... 43m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-4b86e4-openshift-default Scaled up replica set e2e-shared-a-w4-4b86e4-openshift-default-79c864d79c from 0 to 1 43m Normal AddedInterface pod/e2e-shared-a-w4-4b86e4-openshift-default-79c864d79c-hz7fp Add eth0 [10.133.0.77/23] from ovn-kubernetes 43m Normal AddedLabel gateway/e2e-shared-a-w4-4b86e4 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-4b86e4 43m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-4b86e4-openshift-default-79c864d79c Created pod: e2e-shared-a-w4-4b86e4-openshift-default-79c864d79c-hz7fp 43m Normal AddedLabel gateway/e2e-mt-f0651cac Added label istio.io/rev=openshift-gateway to gateway e2e-mt-f0651cac 43m Normal SuccessfulCreate replicaset/e2e-mt-f0651cac-openshift-default-7fc6f9555c Created pod: e2e-mt-f0651cac-openshift-default-7fc6f9555c-pxtlv 43m Normal ScalingReplicaSet deployment/e2e-mt-f0651cac-openshift-default Scaled up replica set e2e-mt-f0651cac-openshift-default-7fc6f9555c from 0 to 1 43m Normal AddedInterface pod/e2e-mt-f0651cac-openshift-default-7fc6f9555c-pxtlv Add eth0 [10.133.0.78/23] from ovn-kubernetes 43m Normal ScalingReplicaSet deployment/e2e-worker-w6-a6f4a5-openshift-default Scaled down replica set e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75 from 6 to 4 43m Normal Killing pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-z5rd6 Stopping container istio-proxy 43m Normal SuccessfulDelete replicaset/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75 Deleted pod: e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-z5rd6 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-a6f4a5-openshift-default New size: 4; reason: All metrics below target 43m Normal SuccessfulDelete replicaset/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75 Deleted pod: e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-9vg69 43m Normal Killing pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-9vg69 Stopping container istio-proxy 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w4-4b86e4 Scaled up replica set payload-processing-e2e-shared-a-w4-4b86e4-5f8755b487 from 0 to 1 43m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w4-4b86e4-5f8755b487 Error creating: pods "payload-processing-e2e-shared-a-w4-4b86e4-5f8755b487-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-4b86e4: serviceaccount "payload-processing-e2e-shared-a-w4-4b86e4" not found 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w4-4b86e4-5f8755b487 Created pod: payload-processing-e2e-shared-a-w4-4b86e4-5f8755b487-kp5qc 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w4-4b86e4 Scaled up replica set payload-pre-processing-e2e-shared-a-w4-4b86e4-6c67bcf5f from 0 to 1 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w4-4b86e4-6c67bcf5f Error creating: pods "payload-pre-processing-e2e-shared-a-w4-4b86e4-6c67bcf5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-4b86e4: serviceaccount "payload-processing-e2e-shared-a-w4-4b86e4" not found 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w4-4b86e4-6c67bcf5f Created pod: payload-pre-processing-e2e-shared-a-w4-4b86e4-6c67bcf5f-8tb44 43m Normal Pulled pod/payload-processing-e2e-shared-a-w4-4b86e4-5f8755b487-kp5qc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-processing-e2e-shared-a-w4-4b86e4-5f8755b487-kp5qc Container started 43m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w4-4b86e4-6c67bcf5f-8tb44 Add eth0 [10.134.0.143/23] from ovn-kubernetes 43m Normal AddedInterface pod/payload-processing-e2e-shared-a-w4-4b86e4-5f8755b487-kp5qc Add eth0 [10.134.0.144/23] from ovn-kubernetes 43m Normal Started pod/payload-pre-processing-e2e-shared-a-w4-4b86e4-6c67bcf5f-8tb44 Container started 43m Normal Created pod/payload-processing-e2e-shared-a-w4-4b86e4-5f8755b487-kp5qc Container created 43m Normal Created pod/payload-pre-processing-e2e-shared-a-w4-4b86e4-6c67bcf5f-8tb44 Container created 43m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w4-4b86e4-6c67bcf5f-8tb44 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Warning ProbeError pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-9vg69 Readiness probe error: HTTP probe failed with statuscode: 503... 43m Warning Unhealthy pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-9vg69 Readiness probe failed: HTTP probe failed with statuscode: 503 43m Warning ProbeError pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-z5rd6 Readiness probe error: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-z5rd6 Readiness probe failed: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-f0651cac Scaled up replica set payload-processing-e2e-mt-f0651cac-59bc6987b9 from 0 to 1 43m Warning FailedCreate replicaset/payload-processing-e2e-mt-f0651cac-59bc6987b9 Error creating: pods "payload-processing-e2e-mt-f0651cac-59bc6987b9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-f0651cac: serviceaccount "payload-processing-e2e-mt-f0651cac" not found 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-f0651cac Scaled up replica set payload-pre-processing-e2e-mt-f0651cac-6dbbdb5867 from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-f0651cac-59bc6987b9 Created pod: payload-processing-e2e-mt-f0651cac-59bc6987b9-mh675 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-f0651cac-6dbbdb5867 Error creating: pods "payload-pre-processing-e2e-mt-f0651cac-6dbbdb5867-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-f0651cac: serviceaccount "payload-processing-e2e-mt-f0651cac" not found 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-f0651cac-6dbbdb5867 Created pod: payload-pre-processing-e2e-mt-f0651cac-6dbbdb5867-fdt2g 43m Normal Started pod/payload-processing-e2e-mt-f0651cac-59bc6987b9-mh675 Container started 43m Normal Started pod/payload-pre-processing-e2e-mt-f0651cac-6dbbdb5867-fdt2g Container started 43m Normal Created pod/payload-pre-processing-e2e-mt-f0651cac-6dbbdb5867-fdt2g Container created 43m Normal AddedInterface pod/payload-processing-e2e-mt-f0651cac-59bc6987b9-mh675 Add eth0 [10.133.0.80/23] from ovn-kubernetes 43m Normal AddedInterface pod/payload-pre-processing-e2e-mt-f0651cac-6dbbdb5867-fdt2g Add eth0 [10.133.0.79/23] from ovn-kubernetes 43m Normal Pulled pod/payload-processing-e2e-mt-f0651cac-59bc6987b9-mh675 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-processing-e2e-mt-f0651cac-59bc6987b9-mh675 Container created 43m Normal Pulled pod/payload-pre-processing-e2e-mt-f0651cac-6dbbdb5867-fdt2g Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-412743-openshift-default New size: 3; reason: All metrics below target 43m Normal Killing pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-5wzlj Stopping container istio-proxy 43m Normal SuccessfulDelete replicaset/e2e-worker-w5-412743-openshift-default-7bddc8b5df Deleted pod: e2e-worker-w5-412743-openshift-default-7bddc8b5df-5wzlj 43m Normal ScalingReplicaSet deployment/e2e-worker-w5-412743-openshift-default Scaled down replica set e2e-worker-w5-412743-openshift-default-7bddc8b5df from 4 to 3 43m Normal Started pod/e2e-ait-dd862feb-openshift-default-67b7c76895-zrjbk Container started 43m Normal Created pod/e2e-ait-dd862feb-openshift-default-67b7c76895-zrjbk Container created 43m Normal Pulled pod/e2e-ait-dd862feb-openshift-default-67b7c76895-zrjbk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Warning ProbeError pod/e2e-ait-dd862feb-openshift-default-67b7c76895-zrjbk Startup probe error: Get "http://10.134.0.137:15021/healthz/ready": dial tcp 10.134.0.137:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-ait-dd862feb-openshift-default-67b7c76895-zrjbk Startup probe failed: Get "http://10.134.0.137:15021/healthz/ready": dial tcp 10.134.0.137:15021: connect: connection refused 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-a6f4a5-openshift-default New size: 3; reason: All metrics below target 42m Normal SuccessfulDelete replicaset/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75 Deleted pod: e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-56xs9 42m Normal ScalingReplicaSet deployment/e2e-worker-w6-a6f4a5-openshift-default Scaled down replica set e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75 from 4 to 3 42m Normal Killing pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-56xs9 Stopping container istio-proxy 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-dd862feb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-dd862feb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-dd862feb-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal AddedLabel gateway/e2e-shared-a-w2-d58cd3 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-d58cd3 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-dd862feb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal AddedInterface pod/e2e-shared-a-w2-d58cd3-openshift-default-d9c46776f-vklx7 Add eth0 [10.134.0.146/23] from ovn-kubernetes 42m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-d58cd3-openshift-default Scaled up replica set e2e-shared-a-w2-d58cd3-openshift-default-d9c46776f from 0 to 1 42m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-d58cd3-openshift-default-d9c46776f Created pod: e2e-shared-a-w2-d58cd3-openshift-default-d9c46776f-vklx7 42m Normal Killing pod/payload-processing-e2e-ait-dd862feb-9c885f569-n957f Stopping container payload-processing 42m Normal Killing pod/payload-pre-processing-e2e-ait-dd862feb-6c5d597495-9ftp6 Stopping container payload-pre-processing 42m Normal AddedLabel gateway/e2e-mt-7d5f4de7 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-7d5f4de7 42m Normal SuccessfulCreate replicaset/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf Created pod: e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-f9ghj 42m Normal SuccessfulCreate replicaset/e2e-derive-74c6ba99-openshift-default-65cd5b8f85 Created pod: e2e-derive-74c6ba99-openshift-default-65cd5b8f85-m746l 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-derive-74c6ba99-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 42m Normal AddedInterface pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-f9ghj Add eth0 [10.133.0.81/23] from ovn-kubernetes 42m Normal ScalingReplicaSet deployment/e2e-mt-7d5f4de7-openshift-default Scaled up replica set e2e-mt-7d5f4de7-openshift-default-55c88dfdbf from 0 to 1 42m Normal ScalingReplicaSet deployment/e2e-derive-74c6ba99-openshift-default Scaled up replica set e2e-derive-74c6ba99-openshift-default-65cd5b8f85 from 1 to 3 42m Normal SuccessfulCreate replicaset/e2e-derive-74c6ba99-openshift-default-65cd5b8f85 Created pod: e2e-derive-74c6ba99-openshift-default-65cd5b8f85-5ttl9 42m Normal Pulled pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-8hddn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-8hddn Container created 42m Normal Started pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-8hddn Container started 42m Normal Started pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-5ttl9 Container started 42m Normal Pulled pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-5ttl9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-m746l Add eth0 [10.132.0.106/23] from ovn-kubernetes 42m Normal Pulled pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-m746l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-m746l Container created 42m Normal Started pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-m746l Container started 42m Normal AddedInterface pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-5ttl9 Add eth0 [10.134.0.148/23] from ovn-kubernetes 42m Normal Created pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-5ttl9 Container created 42m Warning ProbeError pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-m746l Startup probe error: Get "http://10.132.0.106:15021/healthz/ready": dial tcp 10.132.0.106:15021: connect: connection refused... 42m Warning ProbeError pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-5ttl9 Startup probe error: Get "http://10.134.0.148:15021/healthz/ready": dial tcp 10.134.0.148:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-5ttl9 Startup probe failed: Get "http://10.134.0.148:15021/healthz/ready": dial tcp 10.134.0.148:15021: connect: connection refused 42m Warning Unhealthy pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-m746l Startup probe failed: Get "http://10.132.0.106:15021/healthz/ready": dial tcp 10.132.0.106:15021: connect: connection refused 42m Warning ProbeError pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-8hddn Startup probe error: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-8hddn Startup probe failed: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-d58cd3 Scaled up replica set payload-processing-e2e-shared-a-w2-d58cd3-766b65cfd7 from 0 to 1 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-d58cd3 Scaled up replica set payload-pre-processing-e2e-shared-a-w2-d58cd3-6ff958d595 from 0 to 1 42m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-d58cd3-766b65cfd7 Error creating: pods "payload-processing-e2e-shared-a-w2-d58cd3-766b65cfd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-d58cd3: serviceaccount "payload-processing-e2e-shared-a-w2-d58cd3" not found 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-d58cd3-766b65cfd7 Created pod: payload-processing-e2e-shared-a-w2-d58cd3-766b65cfd7-hjv58 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-d58cd3-6ff958d595 Created pod: payload-pre-processing-e2e-shared-a-w2-d58cd3-6ff958d595-t9nfb 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-d58cd3-6ff958d595 Error creating: pods "payload-pre-processing-e2e-shared-a-w2-d58cd3-6ff958d595-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-d58cd3: serviceaccount "payload-processing-e2e-shared-a-w2-d58cd3" not found 42m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-d58cd3-6ff958d595-t9nfb Container started 42m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-d58cd3-6ff958d595-t9nfb Container created 42m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-d58cd3-6ff958d595-t9nfb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-d58cd3-6ff958d595-t9nfb Add eth0 [10.134.0.149/23] from ovn-kubernetes 42m Normal Killing pod/payload-processing-e2e-derive-74c6ba99-cd559cfbf-v6nns Stopping container payload-processing 42m Normal Killing pod/payload-pre-processing-e2e-derive-74c6ba99-794886fd59-cqrgc Stopping container payload-pre-processing 42m Normal Started pod/payload-processing-e2e-shared-a-w2-d58cd3-766b65cfd7-hjv58 Container started 42m Normal Created pod/payload-processing-e2e-shared-a-w2-d58cd3-766b65cfd7-hjv58 Container created 42m Normal Pulled pod/payload-processing-e2e-shared-a-w2-d58cd3-766b65cfd7-hjv58 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-d58cd3-766b65cfd7-hjv58 Add eth0 [10.134.0.150/23] from ovn-kubernetes 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-7d5f4de7-ffd6698f9 Created pod: payload-pre-processing-e2e-mt-7d5f4de7-ffd6698f9-nwczt 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-7d5f4de7-ffd6698f9 Error creating: pods "payload-pre-processing-e2e-mt-7d5f4de7-ffd6698f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-7d5f4de7: serviceaccount "payload-processing-e2e-mt-7d5f4de7" not found 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-7d5f4de7 Scaled up replica set payload-processing-e2e-mt-7d5f4de7-cf6c94fdc from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-7d5f4de7-cf6c94fdc Created pod: payload-processing-e2e-mt-7d5f4de7-cf6c94fdc-c7jqk 42m Warning FailedCreate replicaset/payload-processing-e2e-mt-7d5f4de7-cf6c94fdc Error creating: pods "payload-processing-e2e-mt-7d5f4de7-cf6c94fdc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-7d5f4de7: serviceaccount "payload-processing-e2e-mt-7d5f4de7" not found 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-7d5f4de7 Scaled up replica set payload-pre-processing-e2e-mt-7d5f4de7-ffd6698f9 from 0 to 1 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-74c6ba99-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-74c6ba99-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Normal Pulled pod/payload-pre-processing-e2e-mt-7d5f4de7-ffd6698f9-nwczt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-processing-e2e-mt-7d5f4de7-cf6c94fdc-c7jqk Add eth0 [10.133.0.83/23] from ovn-kubernetes 42m Normal Started pod/payload-processing-e2e-mt-7d5f4de7-cf6c94fdc-c7jqk Container started 42m Normal AddedInterface pod/payload-pre-processing-e2e-mt-7d5f4de7-ffd6698f9-nwczt Add eth0 [10.133.0.82/23] from ovn-kubernetes 42m Normal Created pod/payload-processing-e2e-mt-7d5f4de7-cf6c94fdc-c7jqk Container created 42m Normal Pulled pod/payload-processing-e2e-mt-7d5f4de7-cf6c94fdc-c7jqk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Started pod/payload-pre-processing-e2e-mt-7d5f4de7-ffd6698f9-nwczt Container started 42m Normal Created pod/payload-pre-processing-e2e-mt-7d5f4de7-ffd6698f9-nwczt Container created 41m Warning Unhealthy pod/e2e-mt-f0651cac-openshift-default-7fc6f9555c-pxtlv Readiness probe failed: Get "http://10.133.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Warning ProbeError pod/e2e-mt-f0651cac-openshift-default-7fc6f9555c-pxtlv Readiness probe error: Get "http://10.133.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Normal Pulled pod/e2e-shared-a-w4-4b86e4-openshift-default-79c864d79c-hz7fp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-shared-a-w4-4b86e4-openshift-default-79c864d79c-hz7fp Container created 41m Normal Started pod/e2e-shared-a-w4-4b86e4-openshift-default-79c864d79c-hz7fp Container started 41m Normal Pulled pod/e2e-mt-f0651cac-openshift-default-7fc6f9555c-pxtlv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-mt-f0651cac-openshift-default-7fc6f9555c-pxtlv Container created 41m Normal Started pod/e2e-mt-f0651cac-openshift-default-7fc6f9555c-pxtlv Container started 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-7d5f4de7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-74c6ba99-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-7d5f4de7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-74c6ba99-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning ProbeError pod/e2e-shared-a-w4-4b86e4-openshift-default-79c864d79c-hz7fp Startup probe error: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-shared-a-w4-4b86e4-openshift-default-79c864d79c-hz7fp Startup probe failed: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused 41m Warning ProbeError pod/e2e-mt-f0651cac-openshift-default-7fc6f9555c-pxtlv Startup probe error: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-mt-f0651cac-openshift-default-7fc6f9555c-pxtlv Startup probe failed: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-4b86e4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-4b86e4-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-f0651cac-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-f0651cac-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal Killing pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-97rfb Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w2-e8ba06-openshift-default-654dcd546f Deleted pod: e2e-worker-w2-e8ba06-openshift-default-654dcd546f-rgq28 41m Normal SuccessfulDelete replicaset/e2e-worker-w2-e8ba06-openshift-default-654dcd546f Deleted pod: e2e-worker-w2-e8ba06-openshift-default-654dcd546f-4r6d7 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-e8ba06-openshift-default New size: 5; reason: All metrics below target 41m Normal Killing pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-rgq28 Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-4r6d7 Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w2-e8ba06-openshift-default-654dcd546f Deleted pod: e2e-worker-w2-e8ba06-openshift-default-654dcd546f-97rfb 41m Normal ScalingReplicaSet deployment/e2e-worker-w2-e8ba06-openshift-default Scaled down replica set e2e-worker-w2-e8ba06-openshift-default-654dcd546f from 8 to 5 41m Normal Killing pod/e2e-ait-dd862feb-openshift-default-67b7c76895-zrjbk Stopping container istio-proxy 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-1e5e75-openshift-default New size: 3; reason: All metrics below target 41m Normal Killing pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sxdqn Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-tskrl Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5 Deleted pod: e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-tskrl 41m Normal Killing pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-65wmw Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-zvppf Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w3-1e5e75-openshift-default-879c67bc5 Deleted pod: e2e-worker-w3-1e5e75-openshift-default-879c67bc5-zvppf 41m Normal SuccessfulDelete replicaset/e2e-worker-w3-1e5e75-openshift-default-879c67bc5 Deleted pod: e2e-worker-w3-1e5e75-openshift-default-879c67bc5-65wmw 41m Normal SuccessfulDelete replicaset/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5 Deleted pod: e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sxdqn 41m Normal ScalingReplicaSet deployment/e2e-worker-w3-1e5e75-openshift-default Scaled down replica set e2e-worker-w3-1e5e75-openshift-default-879c67bc5 from 5 to 3 41m Normal ScalingReplicaSet deployment/e2e-worker-w0-101b9e-openshift-default Scaled down replica set e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5 from 6 to 4 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-101b9e-openshift-default New size: 4; reason: All metrics below target 41m Normal AddedLabel gateway/e2e-shared-a-w0-f32ec2 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-f32ec2 41m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-f32ec2-openshift-default-577fcdbf77 Created pod: e2e-shared-a-w0-f32ec2-openshift-default-577fcdbf77-zd2vj 41m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-f32ec2-openshift-default Scaled up replica set e2e-shared-a-w0-f32ec2-openshift-default-577fcdbf77 from 0 to 1 41m Normal AddedInterface pod/e2e-shared-a-w0-f32ec2-openshift-default-577fcdbf77-zd2vj Add eth0 [10.133.0.84/23] from ovn-kubernetes 41m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-0da668-openshift-default-5467f58b7 Created pod: e2e-shared-b-w4-0da668-openshift-default-5467f58b7-tzrb7 41m Normal Killing pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-5ttl9 Stopping container istio-proxy 41m Normal Killing pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-m746l Stopping container istio-proxy 41m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-0da668-openshift-default Scaled up replica set e2e-shared-b-w4-0da668-openshift-default-5467f58b7 from 0 to 1 41m Normal AddedLabel gateway/e2e-shared-b-w4-0da668 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-0da668 41m Normal Killing pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-8hddn Stopping container istio-proxy 41m Warning ProbeError pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-8hddn Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-5ttl9 Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning ProbeError pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-m746l Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-m746l Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning ProbeError pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-5ttl9 Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-derive-74c6ba99-openshift-default-65cd5b8f85-8hddn Readiness probe failed: HTTP probe failed with statuscode: 503 41m Normal AddedInterface pod/e2e-shared-b-w4-0da668-openshift-default-5467f58b7-tzrb7 Add eth0 [10.134.0.151/23] from ovn-kubernetes 41m Normal AddedLabel gateway/e2e-shared-a-w6-e030e0 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w6-e030e0 41m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-e030e0-openshift-default Scaled up replica set e2e-shared-a-w6-e030e0-openshift-default-5c976d9f4f from 0 to 1 41m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-e030e0-openshift-default-5c976d9f4f Created pod: e2e-shared-a-w6-e030e0-openshift-default-5c976d9f4f-lt8jf 41m Normal AddedInterface pod/e2e-shared-a-w6-e030e0-openshift-default-5c976d9f4f-lt8jf Add eth0 [10.134.0.152/23] from ovn-kubernetes 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w0-f32ec2-85b9c79d79 Error creating: pods "payload-pre-processing-e2e-shared-a-w0-f32ec2-85b9c79d79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-f32ec2: serviceaccount "payload-processing-e2e-shared-a-w0-f32ec2" not found 41m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w0-f32ec2-79c9797c5f Error creating: pods "payload-processing-e2e-shared-a-w0-f32ec2-79c9797c5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-f32ec2: serviceaccount "payload-processing-e2e-shared-a-w0-f32ec2" not found 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w0-f32ec2 Scaled up replica set payload-processing-e2e-shared-a-w0-f32ec2-79c9797c5f from 0 to 1 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w0-f32ec2 Scaled up replica set payload-pre-processing-e2e-shared-a-w0-f32ec2-85b9c79d79 from 0 to 1 41m Normal ScalingReplicaSet deployment/e2e-worker-w2-e8ba06-openshift-default Scaled down replica set e2e-worker-w2-e8ba06-openshift-default-654dcd546f from 5 to 4 41m Normal Created pod/payload-pre-processing-e2e-shared-a-w0-f32ec2-85b9c79d79-kzhp6 Container created 41m Normal SuccessfulDelete replicaset/e2e-worker-w2-e8ba06-openshift-default-654dcd546f Deleted pod: e2e-worker-w2-e8ba06-openshift-default-654dcd546f-5dxxk 41m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w0-f32ec2-85b9c79d79-kzhp6 Add eth0 [10.132.0.107/23] from ovn-kubernetes 41m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w0-f32ec2-85b9c79d79-kzhp6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-e8ba06-openshift-default New size: 4; reason: All metrics below target 41m Normal AddedInterface pod/payload-processing-e2e-shared-a-w0-f32ec2-79c9797c5f-mllx6 Add eth0 [10.132.0.108/23] from ovn-kubernetes 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w0-f32ec2-85b9c79d79 Created pod: payload-pre-processing-e2e-shared-a-w0-f32ec2-85b9c79d79-kzhp6 41m Normal Started pod/payload-pre-processing-e2e-shared-a-w0-f32ec2-85b9c79d79-kzhp6 Container started 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w0-f32ec2-79c9797c5f Created pod: payload-processing-e2e-shared-a-w0-f32ec2-79c9797c5f-mllx6 41m Normal Killing pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-5dxxk Stopping container istio-proxy 41m Normal Started pod/payload-processing-e2e-shared-a-w0-f32ec2-79c9797c5f-mllx6 Container started 41m Normal Created pod/payload-processing-e2e-shared-a-w0-f32ec2-79c9797c5f-mllx6 Container created 41m Normal Pulled pod/payload-processing-e2e-shared-a-w0-f32ec2-79c9797c5f-mllx6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-4b86e4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-4b86e4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w4-0da668-78b845796f Error creating: pods "payload-processing-e2e-shared-b-w4-0da668-78b845796f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-0da668: serviceaccount "payload-processing-e2e-shared-b-w4-0da668" not found 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-f0651cac-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-f0651cac-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w4-0da668-78b845796f Created pod: payload-processing-e2e-shared-b-w4-0da668-78b845796f-5srmm 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w4-0da668-dff997c59 Created pod: payload-pre-processing-e2e-shared-b-w4-0da668-dff997c59-8nrck 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w4-0da668 Scaled up replica set payload-pre-processing-e2e-shared-b-w4-0da668-dff997c59 from 0 to 1 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w4-0da668 Scaled up replica set payload-processing-e2e-shared-b-w4-0da668-78b845796f from 0 to 1 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w4-0da668-dff997c59 Error creating: pods "payload-pre-processing-e2e-shared-b-w4-0da668-dff997c59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-0da668: serviceaccount "payload-processing-e2e-shared-b-w4-0da668" not found 41m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w4-0da668-dff997c59-8nrck Add eth0 [10.134.0.153/23] from ovn-kubernetes 41m Normal AddedInterface pod/payload-processing-e2e-shared-b-w4-0da668-78b845796f-5srmm Add eth0 [10.134.0.154/23] from ovn-kubernetes 41m Normal Started pod/payload-pre-processing-e2e-shared-b-w4-0da668-dff997c59-8nrck Container started 41m Normal Created pod/payload-processing-e2e-shared-b-w4-0da668-78b845796f-5srmm Container created 41m Normal Started pod/payload-processing-e2e-shared-b-w4-0da668-78b845796f-5srmm Container started 41m Normal Created pod/payload-pre-processing-e2e-shared-b-w4-0da668-dff997c59-8nrck Container created 41m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w4-0da668-dff997c59-8nrck Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Pulled pod/payload-processing-e2e-shared-b-w4-0da668-78b845796f-5srmm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 3 41m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: cpu resource utilization (percentage of request) above target 41m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-btpxb 41m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-btpxb Add eth0 [10.133.0.85/23] from ovn-kubernetes 41m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w6-e030e0-57f56bb655 Error creating: pods "payload-processing-e2e-shared-a-w6-e030e0-57f56bb655-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-e030e0: serviceaccount "payload-processing-e2e-shared-a-w6-e030e0" not found 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w6-e030e0 Scaled up replica set payload-pre-processing-e2e-shared-a-w6-e030e0-5686f5c779 from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w6-e030e0-5686f5c779 Created pod: payload-pre-processing-e2e-shared-a-w6-e030e0-5686f5c779-sx2sx 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w6-e030e0 Scaled up replica set payload-processing-e2e-shared-a-w6-e030e0-57f56bb655 from 0 to 1 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w6-e030e0-5686f5c779 Error creating: pods "payload-pre-processing-e2e-shared-a-w6-e030e0-5686f5c779-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-e030e0: serviceaccount "payload-processing-e2e-shared-a-w6-e030e0" not found 41m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-btpxb Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 41m Normal Created pod/payload-pre-processing-e2e-shared-a-w6-e030e0-5686f5c779-sx2sx Container created 41m Normal Started pod/payload-pre-processing-e2e-shared-a-w6-e030e0-5686f5c779-sx2sx Container started 41m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w6-e030e0-5686f5c779-sx2sx Add eth0 [10.134.0.156/23] from ovn-kubernetes 41m Normal AddedInterface pod/payload-processing-e2e-shared-a-w6-e030e0-57f56bb655-wfdmp Add eth0 [10.134.0.157/23] from ovn-kubernetes 41m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w6-e030e0-5686f5c779-sx2sx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-processing-e2e-shared-a-w6-e030e0-57f56bb655-wfdmp Container created 41m Normal Pulled pod/payload-processing-e2e-shared-a-w6-e030e0-57f56bb655-wfdmp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w6-e030e0-57f56bb655 Created pod: payload-processing-e2e-shared-a-w6-e030e0-57f56bb655-wfdmp 41m Normal Started pod/payload-processing-e2e-shared-a-w6-e030e0-57f56bb655-wfdmp Container started 41m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-btpxb Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.352s (4.352s including waiting). Image size: 185476477 bytes. 41m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-btpxb Container created 41m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-btpxb Container started 41m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-btpxb Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-btpxb Readiness probe error: HTTP probe failed with statuscode: 503... 40m Normal Killing pod/payload-pre-processing-e2e-mt-f0651cac-6dbbdb5867-fdt2g Stopping container payload-pre-processing 40m Normal Killing pod/payload-processing-e2e-mt-f0651cac-59bc6987b9-mh675 Stopping container payload-processing 40m Normal SuccessfulDelete replicaset/e2e-worker-w4-da9d07-openshift-default-798d9948bc Deleted pod: e2e-worker-w4-da9d07-openshift-default-798d9948bc-dq5s7 40m Normal SuccessfulDelete replicaset/e2e-worker-w4-da9d07-openshift-default-798d9948bc Deleted pod: e2e-worker-w4-da9d07-openshift-default-798d9948bc-hx9qr 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-da9d07-openshift-default New size: 4; reason: All metrics below target 40m Normal ScalingReplicaSet deployment/e2e-worker-w4-da9d07-openshift-default Scaled down replica set e2e-worker-w4-da9d07-openshift-default-798d9948bc from 6 to 4 40m Normal Killing pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-dq5s7 Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-hx9qr Stopping container istio-proxy 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-982ae3-openshift-default New size: 6; reason: All metrics below target 40m Normal Killing pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-bcc9n Stopping container istio-proxy 40m Normal ScalingReplicaSet deployment/e2e-worker-w1-982ae3-openshift-default Scaled down replica set e2e-worker-w1-982ae3-openshift-default-64d9fb649f from 10 to 6 40m Normal SuccessfulDelete replicaset/e2e-worker-w1-982ae3-openshift-default-64d9fb649f Deleted pod: e2e-worker-w1-982ae3-openshift-default-64d9fb649f-4hsbr 40m Normal Killing pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-4hsbr Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-2jrfg Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w1-982ae3-openshift-default-64d9fb649f Deleted pod: e2e-worker-w1-982ae3-openshift-default-64d9fb649f-2jrfg 40m Normal SuccessfulDelete replicaset/e2e-worker-w1-982ae3-openshift-default-64d9fb649f Deleted pod: e2e-worker-w1-982ae3-openshift-default-64d9fb649f-vbgx5 40m Normal SuccessfulDelete replicaset/e2e-worker-w1-982ae3-openshift-default-64d9fb649f Deleted pod: e2e-worker-w1-982ae3-openshift-default-64d9fb649f-bcc9n 40m Normal Killing pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-vbgx5 Stopping container istio-proxy 40m Normal AddedLabel gateway/e2e-api-a-5b4db8 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-5b4db8 40m Normal SuccessfulCreate replicaset/e2e-api-a-5b4db8-openshift-default-5cc6998bbc Created pod: e2e-api-a-5b4db8-openshift-default-5cc6998bbc-tf2dv 40m Normal Pulled pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-f9ghj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-f9ghj Container created 40m Normal Started pod/e2e-shared-a-w2-d58cd3-openshift-default-d9c46776f-vklx7 Container started 40m Normal Pulled pod/e2e-shared-a-w2-d58cd3-openshift-default-d9c46776f-vklx7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal ScalingReplicaSet deployment/e2e-api-a-5b4db8-openshift-default Scaled up replica set e2e-api-a-5b4db8-openshift-default-5cc6998bbc from 0 to 1 40m Normal Started pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-f9ghj Container started 40m Normal Created pod/e2e-shared-a-w2-d58cd3-openshift-default-d9c46776f-vklx7 Container created 40m Normal AddedInterface pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-tf2dv Add eth0 [10.134.0.158/23] from ovn-kubernetes 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-d58cd3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-d58cd3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning ProbeError pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-f9ghj Startup probe error: Get "http://10.133.0.81:15021/healthz/ready": dial tcp 10.133.0.81:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-shared-a-w2-d58cd3-openshift-default-d9c46776f-vklx7 Startup probe failed: Get "http://10.134.0.146:15021/healthz/ready": dial tcp 10.134.0.146:15021: connect: connection refused 40m Warning ProbeError pod/e2e-shared-a-w2-d58cd3-openshift-default-d9c46776f-vklx7 Startup probe error: Get "http://10.134.0.146:15021/healthz/ready": dial tcp 10.134.0.146:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-f9ghj Startup probe failed: Get "http://10.133.0.81:15021/healthz/ready": dial tcp 10.133.0.81:15021: connect: connection refused 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-5b4db8 Scaled up replica set payload-processing-e2e-api-a-5b4db8-78dcbd7945 from 0 to 1 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-5b4db8-5c87d84775 Error creating: pods "payload-pre-processing-e2e-api-a-5b4db8-5c87d84775-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-5b4db8: serviceaccount "payload-processing-e2e-api-a-5b4db8" not found 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-5b4db8 Scaled up replica set payload-pre-processing-e2e-api-a-5b4db8-5c87d84775 from 0 to 1 40m Warning FailedCreate replicaset/payload-processing-e2e-api-a-5b4db8-78dcbd7945 Error creating: pods "payload-processing-e2e-api-a-5b4db8-78dcbd7945-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-5b4db8: serviceaccount "payload-processing-e2e-api-a-5b4db8" not found 40m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-5b4db8-5c87d84775-wgg4h Add eth0 [10.134.0.159/23] from ovn-kubernetes 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-5b4db8-78dcbd7945 Created pod: payload-processing-e2e-api-a-5b4db8-78dcbd7945-mhhfh 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-5b4db8-5c87d84775 Created pod: payload-pre-processing-e2e-api-a-5b4db8-5c87d84775-wgg4h 40m Normal AddedInterface pod/payload-processing-e2e-api-a-5b4db8-78dcbd7945-mhhfh Add eth0 [10.134.0.160/23] from ovn-kubernetes 40m Normal Pulled pod/payload-processing-e2e-api-a-5b4db8-78dcbd7945-mhhfh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Started pod/payload-processing-e2e-api-a-5b4db8-78dcbd7945-mhhfh Container started 40m Normal Created pod/payload-pre-processing-e2e-api-a-5b4db8-5c87d84775-wgg4h Container created 40m Normal Pulled pod/payload-pre-processing-e2e-api-a-5b4db8-5c87d84775-wgg4h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Started pod/payload-pre-processing-e2e-api-a-5b4db8-5c87d84775-wgg4h Container started 40m Normal Created pod/payload-processing-e2e-api-a-5b4db8-78dcbd7945-mhhfh Container created 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-5b4db8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-5b4db8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal Killing pod/e2e-mt-f0651cac-openshift-default-7fc6f9555c-pxtlv Stopping container istio-proxy 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-d58cd3-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-d58cd3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal AddedLabel gateway/e2e-api-a-f891e6 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-f891e6 40m Normal AddedInterface pod/e2e-api-a-f891e6-openshift-default-565fdf54d7-56xb6 Add eth0 [10.134.0.161/23] from ovn-kubernetes 40m Normal SuccessfulCreate replicaset/e2e-api-a-f891e6-openshift-default-565fdf54d7 Created pod: e2e-api-a-f891e6-openshift-default-565fdf54d7-56xb6 40m Normal ScalingReplicaSet deployment/e2e-api-a-f891e6-openshift-default Scaled up replica set e2e-api-a-f891e6-openshift-default-565fdf54d7 from 0 to 1 39m Normal AddedLabel gateway/e2e-shared-b-w2-dcdbd0 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-dcdbd0 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-f891e6 Scaled up replica set payload-processing-e2e-api-a-f891e6-5b8bd8d58f from 0 to 1 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-f891e6 Scaled up replica set payload-pre-processing-e2e-api-a-f891e6-5cf4b56747 from 0 to 1 39m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-dcdbd0-openshift-default Scaled up replica set e2e-shared-b-w2-dcdbd0-openshift-default-86cd956cc9 from 0 to 1 39m Warning FailedCreate replicaset/payload-processing-e2e-api-a-f891e6-5b8bd8d58f Error creating: pods "payload-processing-e2e-api-a-f891e6-5b8bd8d58f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-f891e6: serviceaccount "payload-processing-e2e-api-a-f891e6" not found 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-f891e6-5b8bd8d58f Created pod: payload-processing-e2e-api-a-f891e6-5b8bd8d58f-wpn4q 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-f891e6-5cf4b56747 Created pod: payload-pre-processing-e2e-api-a-f891e6-5cf4b56747-st4dd 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-f891e6-5cf4b56747 Error creating: pods "payload-pre-processing-e2e-api-a-f891e6-5cf4b56747-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-f891e6: serviceaccount "payload-processing-e2e-api-a-f891e6" not found 39m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-dcdbd0-openshift-default-86cd956cc9 Created pod: e2e-shared-b-w2-dcdbd0-openshift-default-86cd956cc9-p8h4z 39m Normal Created pod/payload-processing-e2e-api-a-f891e6-5b8bd8d58f-wpn4q Container created 39m Normal Created pod/payload-pre-processing-e2e-api-a-f891e6-5cf4b56747-st4dd Container created 39m Normal AddedInterface pod/e2e-shared-b-w2-dcdbd0-openshift-default-86cd956cc9-p8h4z Add eth0 [10.134.0.162/23] from ovn-kubernetes 39m Normal Pulled pod/payload-pre-processing-e2e-api-a-f891e6-5cf4b56747-st4dd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Started pod/payload-processing-e2e-api-a-f891e6-5b8bd8d58f-wpn4q Container started 39m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-f891e6-5cf4b56747-st4dd Add eth0 [10.134.0.163/23] from ovn-kubernetes 39m Normal AddedInterface pod/payload-processing-e2e-api-a-f891e6-5b8bd8d58f-wpn4q Add eth0 [10.134.0.164/23] from ovn-kubernetes 39m Normal Pulled pod/payload-processing-e2e-api-a-f891e6-5b8bd8d58f-wpn4q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Started pod/payload-pre-processing-e2e-api-a-f891e6-5cf4b56747-st4dd Container started 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-7d5f4de7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-7d5f4de7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-dcdbd0 Scaled up replica set payload-pre-processing-e2e-shared-b-w2-dcdbd0-7cc8947c99 from 0 to 1 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-dcdbd0-7cc8947c99 Error creating: pods "payload-pre-processing-e2e-shared-b-w2-dcdbd0-7cc8947c99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-dcdbd0: serviceaccount "payload-processing-e2e-shared-b-w2-dcdbd0" not found 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-dcdbd0 Scaled up replica set payload-processing-e2e-shared-b-w2-dcdbd0-bc7d86cfc from 0 to 1 39m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-dcdbd0-bc7d86cfc Error creating: pods "payload-processing-e2e-shared-b-w2-dcdbd0-bc7d86cfc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-dcdbd0: serviceaccount "payload-processing-e2e-shared-b-w2-dcdbd0" not found 39m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-dcdbd0-7cc8947c99-k4hnc Container created 39m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-dcdbd0-bc7d86cfc-rs4qx Add eth0 [10.134.0.175/23] from ovn-kubernetes 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-dcdbd0-bc7d86cfc Created pod: payload-processing-e2e-shared-b-w2-dcdbd0-bc7d86cfc-rs4qx 39m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-dcdbd0-7cc8947c99-k4hnc Add eth0 [10.134.0.174/23] from ovn-kubernetes 39m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-dcdbd0-7cc8947c99-k4hnc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-dcdbd0-7cc8947c99-k4hnc Container started 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-dcdbd0-7cc8947c99 Created pod: payload-pre-processing-e2e-shared-b-w2-dcdbd0-7cc8947c99-k4hnc 39m Normal Started pod/payload-processing-e2e-shared-b-w2-dcdbd0-bc7d86cfc-rs4qx Container started 39m Normal Created pod/payload-processing-e2e-shared-b-w2-dcdbd0-bc7d86cfc-rs4qx Container created 39m Normal Pulled pod/payload-processing-e2e-shared-b-w2-dcdbd0-bc7d86cfc-rs4qx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Started pod/e2e-shared-a-w6-e030e0-openshift-default-5c976d9f4f-lt8jf Container started 39m Normal Created pod/e2e-shared-a-w0-f32ec2-openshift-default-577fcdbf77-zd2vj Container created 39m Normal Pulled pod/e2e-shared-a-w6-e030e0-openshift-default-5c976d9f4f-lt8jf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Started pod/e2e-shared-a-w0-f32ec2-openshift-default-577fcdbf77-zd2vj Container started 39m Normal Pulled pod/e2e-shared-a-w0-f32ec2-openshift-default-577fcdbf77-zd2vj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-shared-a-w6-e030e0-openshift-default-5c976d9f4f-lt8jf Container created 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-e030e0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-e030e0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning ProbeError pod/e2e-shared-a-w0-f32ec2-openshift-default-577fcdbf77-zd2vj Startup probe error: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-shared-a-w0-f32ec2-openshift-default-577fcdbf77-zd2vj Startup probe failed: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused 39m Warning Unhealthy pod/e2e-shared-a-w6-e030e0-openshift-default-5c976d9f4f-lt8jf Startup probe failed: Get "http://10.134.0.152:15021/healthz/ready": dial tcp 10.134.0.152:15021: connect: connection refused 39m Warning ProbeError pod/e2e-shared-a-w6-e030e0-openshift-default-5c976d9f4f-lt8jf Startup probe error: Get "http://10.134.0.152:15021/healthz/ready": dial tcp 10.134.0.152:15021: connect: connection refused... 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-e030e0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-e030e0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal ScalingReplicaSet deployment/e2e-worker-w0-101b9e-openshift-default Scaled down replica set e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5 from 4 to 3 39m Normal Killing pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-qm9sf Stopping container istio-proxy 39m Normal SuccessfulDelete replicaset/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5 Deleted pod: e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-qm9sf 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-101b9e-openshift-default New size: 3; reason: All metrics below target 39m Normal ScalingReplicaSet deployment/e2e-worker-w5-412743-openshift-default Scaled down replica set e2e-worker-w5-412743-openshift-default-7bddc8b5df from 3 to 2 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-412743-openshift-default New size: 2; reason: All metrics below target 39m Normal Killing pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-s66zc Stopping container istio-proxy 39m Normal SuccessfulDelete replicaset/e2e-worker-w5-412743-openshift-default-7bddc8b5df Deleted pod: e2e-worker-w5-412743-openshift-default-7bddc8b5df-s66zc 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-f32ec2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-f32ec2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal AddedLabel gateway/e2e-shared-b-w0-e9c3c1 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-e9c3c1 39m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-e9c3c1-openshift-default-77447d98f7 Created pod: e2e-shared-b-w0-e9c3c1-openshift-default-77447d98f7-8v6vx 39m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-e9c3c1-openshift-default Scaled up replica set e2e-shared-b-w0-e9c3c1-openshift-default-77447d98f7 from 0 to 1 39m Normal AddedInterface pod/e2e-shared-b-w0-e9c3c1-openshift-default-77447d98f7-8v6vx Add eth0 [10.133.0.91/23] from ovn-kubernetes 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-a6f4a5-openshift-default New size: 2; reason: All metrics below target 38m Warning Unhealthy pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-7lvcw Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning ProbeError pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-7lvcw Readiness probe error: HTTP probe failed with statuscode: 503... 38m Normal SuccessfulDelete replicaset/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75 Deleted pod: e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-7lvcw 38m Normal Killing pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-7lvcw Stopping container istio-proxy 38m Normal ScalingReplicaSet deployment/e2e-worker-w6-a6f4a5-openshift-default Scaled down replica set e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75 from 3 to 2 38m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w0-e9c3c1-57dd846c75 Error creating: pods "payload-processing-e2e-shared-b-w0-e9c3c1-57dd846c75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-e9c3c1: serviceaccount "payload-processing-e2e-shared-b-w0-e9c3c1" not found 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w0-e9c3c1 Scaled up replica set payload-processing-e2e-shared-b-w0-e9c3c1-57dd846c75 from 0 to 1 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w0-e9c3c1-57685bbfff Error creating: pods "payload-pre-processing-e2e-shared-b-w0-e9c3c1-57685bbfff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-e9c3c1: serviceaccount "payload-processing-e2e-shared-b-w0-e9c3c1" not found 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w0-e9c3c1-57685bbfff Created pod: payload-pre-processing-e2e-shared-b-w0-e9c3c1-57685bbfff-ddgqw 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w0-e9c3c1 Scaled up replica set payload-pre-processing-e2e-shared-b-w0-e9c3c1-57685bbfff from 0 to 1 38m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w0-e9c3c1-57685bbfff-ddgqw Add eth0 [10.134.0.177/23] from ovn-kubernetes 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w0-e9c3c1-57dd846c75 Created pod: payload-processing-e2e-shared-b-w0-e9c3c1-57dd846c75-n5td5 38m Normal AddedLabel gateway/e2e-api-a-c0f56c Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-c0f56c 38m Normal ScalingReplicaSet deployment/e2e-api-a-c0f56c-openshift-default Scaled up replica set e2e-api-a-c0f56c-openshift-default-665cfcdf75 from 0 to 1 38m Normal SuccessfulCreate replicaset/e2e-api-a-c0f56c-openshift-default-665cfcdf75 Created pod: e2e-api-a-c0f56c-openshift-default-665cfcdf75-fbgz5 38m Normal AddedInterface pod/payload-processing-e2e-shared-b-w0-e9c3c1-57dd846c75-n5td5 Add eth0 [10.133.0.92/23] from ovn-kubernetes 38m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w0-e9c3c1-57685bbfff-ddgqw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Started pod/payload-pre-processing-e2e-shared-b-w0-e9c3c1-57685bbfff-ddgqw Container started 38m Normal Created pod/payload-pre-processing-e2e-shared-b-w0-e9c3c1-57685bbfff-ddgqw Container created 38m Normal Pulled pod/payload-processing-e2e-shared-b-w0-e9c3c1-57dd846c75-n5td5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/e2e-api-a-c0f56c-openshift-default-665cfcdf75-fbgz5 Add eth0 [10.133.0.93/23] from ovn-kubernetes 38m Normal Started pod/payload-processing-e2e-shared-b-w0-e9c3c1-57dd846c75-n5td5 Container started 38m Normal Created pod/payload-processing-e2e-shared-b-w0-e9c3c1-57dd846c75-n5td5 Container created 38m Normal Pulled pod/e2e-shared-b-w4-0da668-openshift-default-5467f58b7-tzrb7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Started pod/e2e-shared-b-w4-0da668-openshift-default-5467f58b7-tzrb7 Container started 38m Normal Created pod/e2e-shared-b-w4-0da668-openshift-default-5467f58b7-tzrb7 Container created 38m Normal AddedLabel gateway/e2e-shared-b-w6-2594a9 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w6-2594a9 38m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-2594a9-openshift-default-8694ddf5f7 Created pod: e2e-shared-b-w6-2594a9-openshift-default-8694ddf5f7-v4jsh 38m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-2594a9-openshift-default Scaled up replica set e2e-shared-b-w6-2594a9-openshift-default-8694ddf5f7 from 0 to 1 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-c0f56c Scaled up replica set payload-processing-e2e-api-a-c0f56c-679fccf4f from 0 to 1 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-c0f56c-6bbdcff4c Error creating: pods "payload-pre-processing-e2e-api-a-c0f56c-6bbdcff4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-c0f56c: serviceaccount "payload-processing-e2e-api-a-c0f56c" not found 38m Normal AddedInterface pod/e2e-shared-b-w6-2594a9-openshift-default-8694ddf5f7-v4jsh Add eth0 [10.134.0.178/23] from ovn-kubernetes 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-c0f56c Scaled up replica set payload-pre-processing-e2e-api-a-c0f56c-6bbdcff4c from 0 to 1 38m Warning FailedCreate replicaset/payload-processing-e2e-api-a-c0f56c-679fccf4f Error creating: pods "payload-processing-e2e-api-a-c0f56c-679fccf4f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-c0f56c: serviceaccount "payload-processing-e2e-api-a-c0f56c" not found 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-c0f56c-679fccf4f Created pod: payload-processing-e2e-api-a-c0f56c-679fccf4f-wqpxd 38m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-c0f56c-6bbdcff4c-dswhj Add eth0 [10.134.0.179/23] from ovn-kubernetes 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-c0f56c-6bbdcff4c Created pod: payload-pre-processing-e2e-api-a-c0f56c-6bbdcff4c-dswhj 38m Normal Started pod/payload-pre-processing-e2e-api-a-c0f56c-6bbdcff4c-dswhj Container started 38m Normal Created pod/payload-pre-processing-e2e-api-a-c0f56c-6bbdcff4c-dswhj Container created 38m Normal Pulled pod/payload-pre-processing-e2e-api-a-c0f56c-6bbdcff4c-dswhj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Started pod/payload-processing-e2e-api-a-c0f56c-679fccf4f-wqpxd Container started 38m Normal Created pod/payload-processing-e2e-api-a-c0f56c-679fccf4f-wqpxd Container created 38m Normal Pulled pod/payload-processing-e2e-api-a-c0f56c-679fccf4f-wqpxd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/payload-processing-e2e-api-a-c0f56c-679fccf4f-wqpxd Add eth0 [10.134.0.180/23] from ovn-kubernetes 38m Warning ProbeError pod/e2e-shared-b-w4-0da668-openshift-default-5467f58b7-tzrb7 Startup probe error: Get "http://10.134.0.151:15021/healthz/ready": dial tcp 10.134.0.151:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-shared-b-w4-0da668-openshift-default-5467f58b7-tzrb7 Startup probe failed: Get "http://10.134.0.151:15021/healthz/ready": dial tcp 10.134.0.151:15021: connect: connection refused 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w6-2594a9-6f4d5b64dc Error creating: pods "payload-pre-processing-e2e-shared-b-w6-2594a9-6f4d5b64dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-2594a9: serviceaccount "payload-processing-e2e-shared-b-w6-2594a9" not found 38m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w6-2594a9-54c85c6687 Error creating: pods "payload-processing-e2e-shared-b-w6-2594a9-54c85c6687-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-2594a9: serviceaccount "payload-processing-e2e-shared-b-w6-2594a9" not found 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w6-2594a9 Scaled up replica set payload-pre-processing-e2e-shared-b-w6-2594a9-6f4d5b64dc from 0 to 1 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w6-2594a9 Scaled up replica set payload-processing-e2e-shared-b-w6-2594a9-54c85c6687 from 0 to 1 38m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w6-2594a9-6f4d5b64dc-s7mzb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w6-2594a9-54c85c6687 Created pod: payload-processing-e2e-shared-b-w6-2594a9-54c85c6687-lnthc 38m Normal Created pod/payload-pre-processing-e2e-shared-b-w6-2594a9-6f4d5b64dc-s7mzb Container created 38m Normal Started pod/payload-pre-processing-e2e-shared-b-w6-2594a9-6f4d5b64dc-s7mzb Container started 38m Normal Pulled pod/payload-processing-e2e-shared-b-w6-2594a9-54c85c6687-lnthc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w6-2594a9-6f4d5b64dc Created pod: payload-pre-processing-e2e-shared-b-w6-2594a9-6f4d5b64dc-s7mzb 38m Normal AddedInterface pod/payload-processing-e2e-shared-b-w6-2594a9-54c85c6687-lnthc Add eth0 [10.133.0.95/23] from ovn-kubernetes 38m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w6-2594a9-6f4d5b64dc-s7mzb Add eth0 [10.133.0.94/23] from ovn-kubernetes 38m Normal Started pod/payload-processing-e2e-shared-b-w6-2594a9-54c85c6687-lnthc Container started 38m Normal Created pod/payload-processing-e2e-shared-b-w6-2594a9-54c85c6687-lnthc Container created 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-0da668-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-0da668-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-0da668-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-0da668-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Normal Created pod/e2e-api-a-f891e6-openshift-default-565fdf54d7-56xb6 Container created 38m Normal Started pod/e2e-api-a-f891e6-openshift-default-565fdf54d7-56xb6 Container started 38m Normal Pulled pod/e2e-api-a-f891e6-openshift-default-565fdf54d7-56xb6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-f891e6-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-f891e6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-c0f56c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-c0f56c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning Unhealthy pod/e2e-api-a-f891e6-openshift-default-565fdf54d7-56xb6 Startup probe failed: Get "http://10.134.0.161:15021/healthz/ready": dial tcp 10.134.0.161:15021: connect: connection refused 38m Warning ProbeError pod/e2e-api-a-f891e6-openshift-default-565fdf54d7-56xb6 Startup probe error: Get "http://10.134.0.161:15021/healthz/ready": dial tcp 10.134.0.161:15021: connect: connection refused... 37m Normal Killing pod/payload-processing-e2e-mt-7d5f4de7-cf6c94fdc-c7jqk Stopping container payload-processing 37m Normal Killing pod/payload-pre-processing-e2e-mt-7d5f4de7-ffd6698f9-nwczt Stopping container payload-pre-processing 37m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6886b96f9 Deleted pod: maas-default-gateway-openshift-default-6886b96f9-r57tc 37m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: All metrics below target 37m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6886b96f9 Deleted pod: maas-default-gateway-openshift-default-6886b96f9-mjxmq 37m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6886b96f9 Deleted pod: maas-default-gateway-openshift-default-6886b96f9-lkfxl 37m Normal Killing pod/maas-default-gateway-openshift-default-6886b96f9-lkfxl Stopping container istio-proxy 37m Normal Killing pod/maas-default-gateway-openshift-default-6886b96f9-mjxmq Stopping container istio-proxy 37m Normal Killing pod/maas-default-gateway-openshift-default-6886b96f9-r57tc Stopping container istio-proxy 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-f891e6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-f891e6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning Unhealthy pod/payload-pre-processing-e2e-mt-7d5f4de7-ffd6698f9-nwczt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Warning Unhealthy pod/payload-processing-e2e-mt-7d5f4de7-cf6c94fdc-c7jqk Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Normal AddedLabel gateway/e2e-api-b-50a19d Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-50a19d 37m Normal ScalingReplicaSet deployment/e2e-api-b-50a19d-openshift-default Scaled up replica set e2e-api-b-50a19d-openshift-default-68c6d7d74c from 0 to 1 37m Normal SuccessfulCreate replicaset/e2e-api-b-50a19d-openshift-default-68c6d7d74c Created pod: e2e-api-b-50a19d-openshift-default-68c6d7d74c-24fwv 37m Normal ScalingReplicaSet deployment/e2e-api-a-5b4db8-openshift-default Scaled up replica set e2e-api-a-5b4db8-openshift-default-5cc6998bbc from 1 to 4 37m Normal Created pod/e2e-shared-b-w2-dcdbd0-openshift-default-86cd956cc9-p8h4z Container created 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-a-5b4db8-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 37m Normal SuccessfulCreate replicaset/e2e-api-a-5b4db8-openshift-default-5cc6998bbc Created pod: e2e-api-a-5b4db8-openshift-default-5cc6998bbc-wqncc 37m Normal SuccessfulCreate replicaset/e2e-api-a-5b4db8-openshift-default-5cc6998bbc Created pod: e2e-api-a-5b4db8-openshift-default-5cc6998bbc-gvj7f 37m Normal SuccessfulCreate replicaset/e2e-api-a-5b4db8-openshift-default-5cc6998bbc Created pod: e2e-api-a-5b4db8-openshift-default-5cc6998bbc-c5pmx 37m Normal Started pod/e2e-shared-b-w2-dcdbd0-openshift-default-86cd956cc9-p8h4z Container started 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-dcdbd0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-dcdbd0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Normal AddedInterface pod/e2e-api-b-50a19d-openshift-default-68c6d7d74c-24fwv Add eth0 [10.134.0.183/23] from ovn-kubernetes 37m Normal Pulled pod/e2e-shared-b-w2-dcdbd0-openshift-default-86cd956cc9-p8h4z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Started pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-gvj7f Container started 37m Normal Pulled pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-gvj7f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Created pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-wqncc Container created 37m Normal Started pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-wqncc Container started 37m Normal Started pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-c5pmx Container started 37m Normal Created pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-c5pmx Container created 37m Normal AddedInterface pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-gvj7f Add eth0 [10.133.0.96/23] from ovn-kubernetes 37m Normal AddedInterface pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-c5pmx Add eth0 [10.134.0.184/23] from ovn-kubernetes 37m Normal AddedInterface pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-wqncc Add eth0 [10.132.0.118/23] from ovn-kubernetes 37m Normal Pulled pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-wqncc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Pulled pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-c5pmx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Created pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-gvj7f Container created 37m Warning ProbeError pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-tf2dv Readiness probe error: Get "http://10.134.0.158:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning Unhealthy pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-tf2dv Readiness probe failed: Get "http://10.134.0.158:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning Unhealthy pod/e2e-shared-b-w2-dcdbd0-openshift-default-86cd956cc9-p8h4z Startup probe failed: Get "http://10.134.0.162:15021/healthz/ready": dial tcp 10.134.0.162:15021: connect: connection refused 37m Warning ProbeError pod/e2e-shared-b-w2-dcdbd0-openshift-default-86cd956cc9-p8h4z Startup probe error: Get "http://10.134.0.162:15021/healthz/ready": dial tcp 10.134.0.162:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-gvj7f Startup probe failed: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused 37m Warning Unhealthy pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-wqncc Startup probe failed: Get "http://10.132.0.118:15021/healthz/ready": dial tcp 10.132.0.118:15021: connect: connection refused 37m Warning ProbeError pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-gvj7f Startup probe error: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused... 37m Warning ProbeError pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-wqncc Startup probe error: Get "http://10.132.0.118:15021/healthz/ready": dial tcp 10.132.0.118:15021: connect: connection refused... 37m Normal Started pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-tf2dv Container started 37m Warning Unhealthy pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-c5pmx Startup probe failed: Get "http://10.134.0.184:15021/healthz/ready": dial tcp 10.134.0.184:15021: connect: connection refused 37m Normal Pulled pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-tf2dv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Created pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-tf2dv Container created 37m Warning ProbeError pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-c5pmx Startup probe error: Get "http://10.134.0.184:15021/healthz/ready": dial tcp 10.134.0.184:15021: connect: connection refused... 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-50a19d-9874b478c Created pod: payload-pre-processing-e2e-api-b-50a19d-9874b478c-mqhwf 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-50a19d Scaled up replica set payload-pre-processing-e2e-api-b-50a19d-9874b478c from 0 to 1 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-50a19d-9874b478c Error creating: pods "payload-pre-processing-e2e-api-b-50a19d-9874b478c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-50a19d: serviceaccount "payload-processing-e2e-api-b-50a19d" not found 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-50a19d-867c4c57b5 Created pod: payload-processing-e2e-api-b-50a19d-867c4c57b5-xknkf 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-50a19d Scaled up replica set payload-processing-e2e-api-b-50a19d-867c4c57b5 from 0 to 1 37m Warning FailedCreate replicaset/payload-processing-e2e-api-b-50a19d-867c4c57b5 Error creating: pods "payload-processing-e2e-api-b-50a19d-867c4c57b5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-50a19d: serviceaccount "payload-processing-e2e-api-b-50a19d" not found 37m Normal Started pod/payload-processing-e2e-api-b-50a19d-867c4c57b5-xknkf Container started 37m Normal Pulled pod/payload-processing-e2e-api-b-50a19d-867c4c57b5-xknkf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Created pod/payload-processing-e2e-api-b-50a19d-867c4c57b5-xknkf Container created 37m Normal AddedInterface pod/payload-processing-e2e-api-b-50a19d-867c4c57b5-xknkf Add eth0 [10.134.0.186/23] from ovn-kubernetes 37m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-50a19d-9874b478c-mqhwf Add eth0 [10.134.0.185/23] from ovn-kubernetes 37m Normal Pulled pod/payload-pre-processing-e2e-api-b-50a19d-9874b478c-mqhwf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Started pod/payload-pre-processing-e2e-api-b-50a19d-9874b478c-mqhwf Container started 37m Normal Created pod/payload-pre-processing-e2e-api-b-50a19d-9874b478c-mqhwf Container created 37m Warning Unhealthy pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-tf2dv Startup probe failed: Get "http://10.134.0.158:15021/healthz/ready": dial tcp 10.134.0.158:15021: connect: connection refused 37m Warning ProbeError pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-tf2dv Startup probe error: Get "http://10.134.0.158:15021/healthz/ready": dial tcp 10.134.0.158:15021: connect: connection refused... 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-5b4db8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-5b4db8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-dcdbd0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-dcdbd0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-e8ba06-openshift-default New size: 3; reason: All metrics below target 36m Normal ScalingReplicaSet deployment/e2e-worker-w2-e8ba06-openshift-default Scaled down replica set e2e-worker-w2-e8ba06-openshift-default-654dcd546f from 4 to 3 36m Normal Killing pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-vkvcx Stopping container istio-proxy 36m Normal SuccessfulDelete replicaset/e2e-worker-w2-e8ba06-openshift-default-654dcd546f Deleted pod: e2e-worker-w2-e8ba06-openshift-default-654dcd546f-vkvcx 36m Normal Killing pod/payload-pre-processing-e2e-api-a-f891e6-5cf4b56747-st4dd Stopping container payload-pre-processing 36m Normal Killing pod/payload-processing-e2e-api-a-f891e6-5b8bd8d58f-wpn4q Stopping container payload-processing 36m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-d58cd3-6ff958d595-t9nfb Stopping container payload-pre-processing 36m Normal Killing pod/payload-processing-e2e-shared-a-w2-d58cd3-766b65cfd7-hjv58 Stopping container payload-processing 36m Normal Started pod/e2e-shared-b-w0-e9c3c1-openshift-default-77447d98f7-8v6vx Container started 36m Normal Created pod/e2e-shared-b-w0-e9c3c1-openshift-default-77447d98f7-8v6vx Container created 36m Normal Pulled pod/e2e-shared-b-w0-e9c3c1-openshift-default-77447d98f7-8v6vx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/e2e-shared-b-w6-2594a9-openshift-default-8694ddf5f7-v4jsh Container started 36m Normal Pulled pod/e2e-shared-b-w6-2594a9-openshift-default-8694ddf5f7-v4jsh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Created pod/e2e-shared-b-w6-2594a9-openshift-default-8694ddf5f7-v4jsh Container created 36m Warning ProbeError pod/e2e-shared-b-w0-e9c3c1-openshift-default-77447d98f7-8v6vx Startup probe error: Get "http://10.133.0.91:15021/healthz/ready": dial tcp 10.133.0.91:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-shared-b-w0-e9c3c1-openshift-default-77447d98f7-8v6vx Startup probe failed: Get "http://10.133.0.91:15021/healthz/ready": dial tcp 10.133.0.91:15021: connect: connection refused 36m Warning ProbeError pod/e2e-shared-b-w6-2594a9-openshift-default-8694ddf5f7-v4jsh Startup probe error: Get "http://10.134.0.178:15021/healthz/ready": dial tcp 10.134.0.178:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-shared-b-w6-2594a9-openshift-default-8694ddf5f7-v4jsh Startup probe failed: Get "http://10.134.0.178:15021/healthz/ready": dial tcp 10.134.0.178:15021: connect: connection refused 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-2594a9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-2594a9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-e8ba06-openshift-default New size: 2; reason: All metrics below target 36m Normal SuccessfulDelete replicaset/e2e-worker-w2-e8ba06-openshift-default-654dcd546f Deleted pod: e2e-worker-w2-e8ba06-openshift-default-654dcd546f-xf6mc 36m Normal ScalingReplicaSet deployment/e2e-worker-w2-e8ba06-openshift-default Scaled down replica set e2e-worker-w2-e8ba06-openshift-default-654dcd546f from 3 to 2 36m Normal Killing pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-xf6mc Stopping container istio-proxy 36m Normal SuccessfulDelete replicaset/e2e-worker-w3-1e5e75-openshift-default-879c67bc5 Deleted pod: e2e-worker-w3-1e5e75-openshift-default-879c67bc5-clnbw 36m Normal ScalingReplicaSet deployment/e2e-worker-w4-da9d07-openshift-default Scaled down replica set e2e-worker-w4-da9d07-openshift-default-798d9948bc from 4 to 3 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-1e5e75-openshift-default New size: 2; reason: All metrics below target 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-982ae3-openshift-default New size: 5; reason: All metrics below target 36m Normal ScalingReplicaSet deployment/e2e-worker-w3-1e5e75-openshift-default Scaled down replica set e2e-worker-w3-1e5e75-openshift-default-879c67bc5 from 3 to 2 36m Normal Killing pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-8d2dh Stopping container istio-proxy 36m Warning ProbeError pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-xf6mc Readiness probe error: HTTP probe failed with statuscode: 503... 36m Normal ScalingReplicaSet deployment/e2e-worker-w1-982ae3-openshift-default Scaled down replica set e2e-worker-w1-982ae3-openshift-default-64d9fb649f from 6 to 5 36m Normal SuccessfulDelete replicaset/e2e-worker-w1-982ae3-openshift-default-64d9fb649f Deleted pod: e2e-worker-w1-982ae3-openshift-default-64d9fb649f-8d2dh 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-da9d07-openshift-default New size: 3; reason: All metrics below target 36m Normal Killing pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-clnbw Stopping container istio-proxy 36m Normal Killing pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-p7bzb Stopping container istio-proxy 36m Normal SuccessfulDelete replicaset/e2e-worker-w4-da9d07-openshift-default-798d9948bc Deleted pod: e2e-worker-w4-da9d07-openshift-default-798d9948bc-p7bzb 36m Warning Unhealthy pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-xf6mc Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning Unhealthy pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-clnbw Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning ProbeError pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-clnbw Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning ProbeError pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-8d2dh Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning Unhealthy pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-p7bzb Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning Unhealthy pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-8d2dh Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning ProbeError pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-p7bzb Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-e9c3c1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-e9c3c1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6886b96f9 Deleted pod: maas-default-gateway-openshift-default-6886b96f9-m47hc 35m Normal Killing pod/maas-default-gateway-openshift-default-6886b96f9-m47hc Stopping container istio-proxy 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-2594a9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-2594a9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Normal SuccessfulDelete replicaset/e2e-worker-w1-982ae3-openshift-default-64d9fb649f Deleted pod: e2e-worker-w1-982ae3-openshift-default-64d9fb649f-f6p8m 35m Normal Killing pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-f6p8m Stopping container istio-proxy 35m Normal Killing pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-l64nw Stopping container istio-proxy 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-982ae3-openshift-default New size: 3; reason: All metrics below target 35m Normal ScalingReplicaSet deployment/e2e-worker-w1-982ae3-openshift-default Scaled down replica set e2e-worker-w1-982ae3-openshift-default-64d9fb649f from 5 to 3 35m Normal SuccessfulDelete replicaset/e2e-worker-w1-982ae3-openshift-default-64d9fb649f Deleted pod: e2e-worker-w1-982ae3-openshift-default-64d9fb649f-l64nw 35m Warning ProbeError pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-l64nw Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-l64nw Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning Unhealthy pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-f6p8m Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-f6p8m Readiness probe error: HTTP probe failed with statuscode: 503... 35m Normal Killing pod/payload-pre-processing-e2e-shared-a-w0-f32ec2-85b9c79d79-kzhp6 Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-shared-a-w0-f32ec2-79c9797c5f-mllx6 Stopping container payload-processing 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-e9c3c1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-e9c3c1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Normal Killing pod/payload-processing-e2e-api-a-c0f56c-679fccf4f-wqpxd Stopping container payload-processing 35m Normal Killing pod/payload-pre-processing-e2e-api-a-c0f56c-6bbdcff4c-dswhj Stopping container payload-pre-processing 35m Normal Killing pod/payload-pre-processing-e2e-shared-a-w6-e030e0-5686f5c779-sx2sx Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-shared-a-w6-e030e0-57f56bb655-wfdmp Stopping container payload-processing 35m Normal Pulled pod/e2e-api-a-c0f56c-openshift-default-665cfcdf75-fbgz5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-api-a-c0f56c-openshift-default-665cfcdf75-fbgz5 Container created 35m Normal Started pod/e2e-api-a-c0f56c-openshift-default-665cfcdf75-fbgz5 Container started 35m Normal Killing pod/e2e-shared-a-w2-d58cd3-openshift-default-d9c46776f-vklx7 Stopping container istio-proxy 35m Warning ProbeError pod/e2e-shared-a-w2-d58cd3-openshift-default-d9c46776f-vklx7 Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-shared-a-w2-d58cd3-openshift-default-d9c46776f-vklx7 Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-api-a-c0f56c-openshift-default-665cfcdf75-fbgz5 Startup probe error: Get "http://10.133.0.93:15021/healthz/ready": dial tcp 10.133.0.93:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-api-a-c0f56c-openshift-default-665cfcdf75-fbgz5 Startup probe failed: Get "http://10.133.0.93:15021/healthz/ready": dial tcp 10.133.0.93:15021: connect: connection refused 35m Normal Killing pod/e2e-api-a-c0f56c-openshift-default-665cfcdf75-fbgz5 Stopping container istio-proxy 35m Normal SuccessfulCreate replicaset/e2e-inf-a-4ab8e5-openshift-default-6c89ffbf45 Created pod: e2e-inf-a-4ab8e5-openshift-default-6c89ffbf45-mtckf 35m Normal ScalingReplicaSet deployment/e2e-inf-a-4ab8e5-openshift-default Scaled up replica set e2e-inf-a-4ab8e5-openshift-default-6c89ffbf45 from 0 to 1 35m Normal AddedLabel gateway/e2e-inf-a-4ab8e5 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-4ab8e5 35m Normal AddedInterface pod/e2e-inf-a-4ab8e5-openshift-default-6c89ffbf45-mtckf Add eth0 [10.134.0.194/23] from ovn-kubernetes 35m Warning Unhealthy pod/e2e-api-b-50a19d-openshift-default-68c6d7d74c-24fwv Readiness probe failed: Get "http://10.134.0.183:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 35m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-dcdbd0-7cc8947c99-k4hnc Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-shared-b-w2-dcdbd0-bc7d86cfc-rs4qx Stopping container payload-processing 35m Warning ProbeError pod/e2e-api-b-50a19d-openshift-default-68c6d7d74c-24fwv Readiness probe error: Get "http://10.134.0.183:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 35m Normal Pulled pod/e2e-api-b-50a19d-openshift-default-68c6d7d74c-24fwv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-api-b-50a19d-openshift-default-68c6d7d74c-24fwv Container created 35m Normal Started pod/e2e-api-b-50a19d-openshift-default-68c6d7d74c-24fwv Container started 35m Normal Killing pod/e2e-api-a-f891e6-openshift-default-565fdf54d7-56xb6 Stopping container istio-proxy 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-4ab8e5 Scaled up replica set payload-processing-e2e-inf-a-4ab8e5-b77879c55 from 0 to 1 35m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-4ab8e5-b77879c55 Error creating: pods "payload-processing-e2e-inf-a-4ab8e5-b77879c55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-4ab8e5: serviceaccount "payload-processing-e2e-inf-a-4ab8e5" not found 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-4ab8e5-b77879c55 Created pod: payload-processing-e2e-inf-a-4ab8e5-b77879c55-qptzw 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-4ab8e5-79cc98db87 Error creating: pods "payload-pre-processing-e2e-inf-a-4ab8e5-79cc98db87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-4ab8e5: serviceaccount "payload-processing-e2e-inf-a-4ab8e5" not found 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-4ab8e5-79cc98db87 Created pod: payload-pre-processing-e2e-inf-a-4ab8e5-79cc98db87-r95fp 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-4ab8e5 Scaled up replica set payload-pre-processing-e2e-inf-a-4ab8e5-79cc98db87 from 0 to 1 35m Normal Pulled pod/payload-pre-processing-e2e-inf-a-4ab8e5-79cc98db87-r95fp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Started pod/payload-pre-processing-e2e-inf-a-4ab8e5-79cc98db87-r95fp Container started 35m Normal Pulled pod/payload-processing-e2e-inf-a-4ab8e5-b77879c55-qptzw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/payload-processing-e2e-inf-a-4ab8e5-b77879c55-qptzw Add eth0 [10.134.0.196/23] from ovn-kubernetes 35m Normal Created pod/payload-pre-processing-e2e-inf-a-4ab8e5-79cc98db87-r95fp Container created 35m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-4ab8e5-79cc98db87-r95fp Add eth0 [10.134.0.195/23] from ovn-kubernetes 35m Normal Created pod/payload-processing-e2e-inf-a-4ab8e5-b77879c55-qptzw Container created 35m Normal Started pod/payload-processing-e2e-inf-a-4ab8e5-b77879c55-qptzw Container started 35m Warning Unhealthy pod/e2e-api-b-50a19d-openshift-default-68c6d7d74c-24fwv Startup probe failed: Get "http://10.134.0.183:15021/healthz/ready": dial tcp 10.134.0.183:15021: connect: connection refused 35m Warning ProbeError pod/e2e-api-b-50a19d-openshift-default-68c6d7d74c-24fwv Startup probe error: Get "http://10.134.0.183:15021/healthz/ready": dial tcp 10.134.0.183:15021: connect: connection refused... 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-7d5f4de7 Scaled up replica set payload-processing-e2e-mt-7d5f4de7-cf6c94fdc from 0 to 1 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-7d5f4de7 Scaled up replica set payload-pre-processing-e2e-mt-7d5f4de7-ffd6698f9 from 0 to 1 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-7d5f4de7-cf6c94fdc Created pod: payload-processing-e2e-mt-7d5f4de7-cf6c94fdc-zvq97 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-7d5f4de7-ffd6698f9 Error creating: pods "payload-pre-processing-e2e-mt-7d5f4de7-ffd6698f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-7d5f4de7: serviceaccount "payload-processing-e2e-mt-7d5f4de7" not found 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-7d5f4de7-ffd6698f9 Created pod: payload-pre-processing-e2e-mt-7d5f4de7-ffd6698f9-2r7sb 34m Warning FailedCreate replicaset/payload-processing-e2e-mt-7d5f4de7-cf6c94fdc Error creating: pods "payload-processing-e2e-mt-7d5f4de7-cf6c94fdc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-7d5f4de7: serviceaccount "payload-processing-e2e-mt-7d5f4de7" not found 34m Normal Started pod/payload-processing-e2e-mt-7d5f4de7-cf6c94fdc-zvq97 Container started 34m Normal AddedInterface pod/payload-processing-e2e-mt-7d5f4de7-cf6c94fdc-zvq97 Add eth0 [10.134.0.199/23] from ovn-kubernetes 34m Normal Pulled pod/payload-processing-e2e-mt-7d5f4de7-cf6c94fdc-zvq97 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Created pod/payload-processing-e2e-mt-7d5f4de7-cf6c94fdc-zvq97 Container created 34m Normal AddedInterface pod/payload-pre-processing-e2e-mt-7d5f4de7-ffd6698f9-2r7sb Add eth0 [10.134.0.198/23] from ovn-kubernetes 34m Normal Pulled pod/payload-pre-processing-e2e-mt-7d5f4de7-ffd6698f9-2r7sb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Created pod/payload-pre-processing-e2e-mt-7d5f4de7-ffd6698f9-2r7sb Container created 34m Normal Started pod/payload-pre-processing-e2e-mt-7d5f4de7-ffd6698f9-2r7sb Container started 34m Normal Killing pod/payload-pre-processing-e2e-worker-w1-982ae3-6b6857b7dc-92pzx Stopping container payload-pre-processing 34m Normal Killing pod/payload-processing-e2e-worker-w1-982ae3-6c4b5dc9b9-mmctb Stopping container payload-processing 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-50a19d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-50a19d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Normal ScalingReplicaSet deployment/e2e-worker-w1-982ae3-openshift-default Scaled up replica set e2e-worker-w1-982ae3-openshift-default-64d9fb649f from 3 to 4 34m Normal SuccessfulCreate replicaset/e2e-worker-w1-982ae3-openshift-default-64d9fb649f Created pod: e2e-worker-w1-982ae3-openshift-default-64d9fb649f-gxxl8 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-982ae3-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 34m Normal Pulled pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-gxxl8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-gxxl8 Add eth0 [10.134.0.200/23] from ovn-kubernetes 34m Normal Started pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-gxxl8 Container started 34m Normal Created pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-gxxl8 Container created 34m Warning ProbeError pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-gxxl8 Startup probe error: Get "http://10.134.0.200:15021/healthz/ready": dial tcp 10.134.0.200:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-gxxl8 Startup probe failed: Get "http://10.134.0.200:15021/healthz/ready": dial tcp 10.134.0.200:15021: connect: connection refused 34m Normal Killing pod/e2e-shared-a-w0-f32ec2-openshift-default-577fcdbf77-zd2vj Stopping container istio-proxy 34m Normal Killing pod/payload-processing-e2e-api-a-5b4db8-78dcbd7945-mhhfh Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-api-a-5b4db8-5c87d84775-wgg4h Stopping container payload-pre-processing 34m Normal Killing pod/payload-processing-e2e-shared-b-w0-e9c3c1-57dd846c75-n5td5 Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-shared-b-w0-e9c3c1-57685bbfff-ddgqw Stopping container payload-pre-processing 33m Normal Killing pod/e2e-shared-a-w6-e030e0-openshift-default-5c976d9f4f-lt8jf Stopping container istio-proxy 33m Warning ProbeError pod/e2e-shared-a-w6-e030e0-openshift-default-5c976d9f4f-lt8jf Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-shared-a-w6-e030e0-openshift-default-5c976d9f4f-lt8jf Readiness probe failed: HTTP probe failed with statuscode: 503 33m Normal Killing pod/payload-pre-processing-e2e-shared-b-w6-2594a9-6f4d5b64dc-s7mzb Stopping container payload-pre-processing 33m Normal Killing pod/payload-processing-e2e-shared-b-w6-2594a9-54c85c6687-lnthc Stopping container payload-processing 33m Normal Killing pod/e2e-shared-b-w2-dcdbd0-openshift-default-86cd956cc9-p8h4z Stopping container istio-proxy 33m Warning ProbeError pod/e2e-shared-b-w2-dcdbd0-openshift-default-86cd956cc9-p8h4z Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-shared-b-w2-dcdbd0-openshift-default-86cd956cc9-p8h4z Readiness probe failed: HTTP probe failed with statuscode: 503 33m Normal Killing pod/payload-processing-e2e-worker-w2-e8ba06-bb9fb667f-x6fzk Stopping container payload-processing 33m Normal Killing pod/payload-pre-processing-e2e-worker-w2-e8ba06-7b548bc5f-m9r6m Stopping container payload-pre-processing 33m Normal SuccessfulDelete replicaset/e2e-worker-w4-da9d07-openshift-default-798d9948bc Deleted pod: e2e-worker-w4-da9d07-openshift-default-798d9948bc-tl6qf 33m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 33m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 33m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-btpxb Stopping container discovery 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-da9d07-openshift-default New size: 2; reason: All metrics below target 33m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-btpxb 33m Normal ScalingReplicaSet deployment/e2e-worker-w4-da9d07-openshift-default Scaled down replica set e2e-worker-w4-da9d07-openshift-default-798d9948bc from 3 to 2 33m Normal Killing pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-tl6qf Stopping container istio-proxy 33m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-btpxb Readiness probe error: Get "http://10.133.0.85:8080/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 33m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-btpxb Readiness probe failed: Get "http://10.133.0.85:8080/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 33m Normal Started pod/e2e-inf-a-4ab8e5-openshift-default-6c89ffbf45-mtckf Container started 33m Normal Pulled pod/e2e-inf-a-4ab8e5-openshift-default-6c89ffbf45-mtckf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-inf-a-4ab8e5-openshift-default-6c89ffbf45-mtckf Container created 33m Normal Killing pod/maas-default-gateway-openshift-default-6886b96f9-wspmm Stopping container istio-proxy 33m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6886b96f9 Deleted pod: maas-default-gateway-openshift-default-6886b96f9-wspmm 33m Warning ProbeError pod/e2e-inf-a-4ab8e5-openshift-default-6c89ffbf45-mtckf Startup probe error: Get "http://10.134.0.194:15021/healthz/ready": dial tcp 10.134.0.194:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-inf-a-4ab8e5-openshift-default-6c89ffbf45-mtckf Startup probe failed: Get "http://10.134.0.194:15021/healthz/ready": dial tcp 10.134.0.194:15021: connect: connection refused 33m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-e8ba06-7b548bc5f-m9r6m Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Warning Unhealthy pod/payload-processing-e2e-worker-w2-e8ba06-bb9fb667f-x6fzk Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-wspmm Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-wspmm Readiness probe error: HTTP probe failed with statuscode: 503... 33m Normal SuccessfulCreate replicaset/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf Created pod: e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-dln97 33m Normal SuccessfulCreate replicaset/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf Created pod: e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-2rtnt 33m Normal SuccessfulCreate replicaset/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf Created pod: e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-6cwlp 33m Normal ScalingReplicaSet deployment/e2e-mt-7d5f4de7-openshift-default Scaled up replica set e2e-mt-7d5f4de7-openshift-default-55c88dfdbf from 1 to 4 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-7d5f4de7-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 33m Normal Pulled pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-6cwlp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-dln97 Add eth0 [10.132.0.123/23] from ovn-kubernetes 33m Normal Started pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-dln97 Container started 33m Normal Created pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-dln97 Container created 33m Normal AddedInterface pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-2rtnt Add eth0 [10.134.0.207/23] from ovn-kubernetes 33m Normal Pulled pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-2rtnt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-2rtnt Container created 33m Normal Started pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-2rtnt Container started 33m Normal Pulled pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-dln97 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Started pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-6cwlp Container started 33m Normal AddedInterface pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-6cwlp Add eth0 [10.133.0.98/23] from ovn-kubernetes 33m Normal Created pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-6cwlp Container created 32m Warning Unhealthy pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-2rtnt Startup probe failed: Get "http://10.134.0.207:15021/healthz/ready": dial tcp 10.134.0.207:15021: connect: connection refused 32m Warning Unhealthy pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-dln97 Startup probe failed: Get "http://10.132.0.123:15021/healthz/ready": dial tcp 10.132.0.123:15021: connect: connection refused 32m Warning ProbeError pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-dln97 Startup probe error: Get "http://10.132.0.123:15021/healthz/ready": dial tcp 10.132.0.123:15021: connect: connection refused... 32m Warning ProbeError pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-6cwlp Startup probe error: Get "http://10.133.0.98:15021/healthz/ready": dial tcp 10.133.0.98:15021: connect: connection refused... 32m Warning ProbeError pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-2rtnt Startup probe error: Get "http://10.134.0.207:15021/healthz/ready": dial tcp 10.134.0.207:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-6cwlp Startup probe failed: Get "http://10.133.0.98:15021/healthz/ready": dial tcp 10.133.0.98:15021: connect: connection refused 32m Normal SuccessfulCreate replicaset/e2e-worker-w2-e8ba06-openshift-default-654dcd546f Created pod: e2e-worker-w2-e8ba06-openshift-default-654dcd546f-d96gv 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-e8ba06-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 32m Normal ScalingReplicaSet deployment/e2e-worker-w2-e8ba06-openshift-default Scaled up replica set e2e-worker-w2-e8ba06-openshift-default-654dcd546f from 2 to 3 32m Normal Started pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-d96gv Container started 32m Normal Pulled pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-d96gv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-d96gv Add eth0 [10.132.0.124/23] from ovn-kubernetes 32m Normal Created pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-d96gv Container created 32m Normal Killing pod/e2e-shared-b-w0-e9c3c1-openshift-default-77447d98f7-8v6vx Stopping container istio-proxy 32m Normal SuccessfulCreate replicaset/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf Created pod: e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-fcctm 32m Normal ScalingReplicaSet deployment/e2e-mt-7d5f4de7-openshift-default Scaled up replica set e2e-mt-7d5f4de7-openshift-default-55c88dfdbf from 4 to 8 32m Normal SuccessfulCreate replicaset/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf Created pod: e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-mwd5g 32m Normal SuccessfulCreate replicaset/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf Created pod: e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-lkp42 32m Normal SuccessfulCreate replicaset/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf Created pod: e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-hpj9w 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-7d5f4de7-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 32m Normal Created pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-fcctm Container created 32m Normal AddedInterface pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-fcctm Add eth0 [10.133.0.99/23] from ovn-kubernetes 32m Normal Created pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-hpj9w Container created 32m Normal Pulled pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-hpj9w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-hpj9w Add eth0 [10.134.0.209/23] from ovn-kubernetes 32m Normal AddedInterface pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-lkp42 Add eth0 [10.132.0.125/23] from ovn-kubernetes 32m Normal Started pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-hpj9w Container started 32m Normal AddedInterface pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-mwd5g Add eth0 [10.134.0.208/23] from ovn-kubernetes 32m Normal Started pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-fcctm Container started 32m Normal Started pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-mwd5g Container started 32m Normal Pulled pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-lkp42 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Pulled pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-mwd5g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Started pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-lkp42 Container started 32m Normal Pulled pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-fcctm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-lkp42 Container created 32m Normal Created pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-mwd5g Container created 32m Warning Unhealthy pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-d96gv Startup probe failed: Get "http://10.132.0.124:15021/healthz/ready": dial tcp 10.132.0.124:15021: connect: connection refused 32m Warning ProbeError pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-d96gv Startup probe error: Get "http://10.132.0.124:15021/healthz/ready": dial tcp 10.132.0.124:15021: connect: connection refused... 32m Warning ProbeError pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-fcctm Startup probe error: Get "http://10.133.0.99:15021/healthz/ready": dial tcp 10.133.0.99:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-fcctm Startup probe failed: Get "http://10.133.0.99:15021/healthz/ready": dial tcp 10.133.0.99:15021: connect: connection refused 32m Warning ProbeError pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-lkp42 Startup probe error: Get "http://10.132.0.125:15021/healthz/ready": dial tcp 10.132.0.125:15021: connect: connection refused... 32m Warning ProbeError pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-mwd5g Startup probe error: Get "http://10.134.0.208:15021/healthz/ready": dial tcp 10.134.0.208:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-mwd5g Startup probe failed: Get "http://10.134.0.208:15021/healthz/ready": dial tcp 10.134.0.208:15021: connect: connection refused 32m Warning ProbeError pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-hpj9w Startup probe error: Get "http://10.134.0.209:15021/healthz/ready": dial tcp 10.134.0.209:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-hpj9w Startup probe failed: Get "http://10.134.0.209:15021/healthz/ready": dial tcp 10.134.0.209:15021: connect: connection refused 32m Warning Unhealthy pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-lkp42 Startup probe failed: Get "http://10.132.0.125:15021/healthz/ready": dial tcp 10.132.0.125:15021: connect: connection refused 32m Normal Killing pod/e2e-shared-b-w6-2594a9-openshift-default-8694ddf5f7-v4jsh Stopping container istio-proxy 32m Normal SuccessfulCreate replicaset/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf (combined from similar events): Created pod: e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-jztgt 32m Normal SuccessfulCreate replicaset/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf Created pod: e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-lzt2n 32m Normal ScalingReplicaSet deployment/e2e-mt-7d5f4de7-openshift-default Scaled up replica set e2e-mt-7d5f4de7-openshift-default-55c88dfdbf from 8 to 10 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-7d5f4de7-openshift-default New size: 10; reason: 32m Normal Created pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-lzt2n Container created 32m Normal AddedInterface pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-jztgt Add eth0 [10.133.0.101/23] from ovn-kubernetes 32m Normal Pulled pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-jztgt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Pulled pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-lzt2n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal SuccessfulCreate replicaset/e2e-inf-b-bc2453-openshift-default-677bb79dc9 Created pod: e2e-inf-b-bc2453-openshift-default-677bb79dc9-8pn9n 32m Normal ScalingReplicaSet deployment/e2e-inf-b-bc2453-openshift-default Scaled up replica set e2e-inf-b-bc2453-openshift-default-677bb79dc9 from 0 to 1 32m Normal AddedInterface pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-lzt2n Add eth0 [10.132.0.126/23] from ovn-kubernetes 32m Normal Started pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-jztgt Container started 32m Normal AddedLabel gateway/e2e-inf-b-bc2453 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-bc2453 32m Normal Started pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-lzt2n Container started 32m Normal Created pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-jztgt Container created 32m Normal AddedInterface pod/e2e-inf-b-bc2453-openshift-default-677bb79dc9-8pn9n Add eth0 [10.133.0.102/23] from ovn-kubernetes 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-4ab8e5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-4ab8e5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Normal ScalingReplicaSet deployment/e2e-api-a-5b4db8-openshift-default Scaled down replica set e2e-api-a-5b4db8-openshift-default-5cc6998bbc from 4 to 3 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-a-5b4db8-openshift-default New size: 3; reason: All metrics below target 32m Normal SuccessfulDelete replicaset/e2e-api-a-5b4db8-openshift-default-5cc6998bbc Deleted pod: e2e-api-a-5b4db8-openshift-default-5cc6998bbc-c5pmx 32m Normal Killing pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-c5pmx Stopping container istio-proxy 32m Warning ProbeError pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-jztgt Startup probe error: Get "http://10.133.0.101:15021/healthz/ready": dial tcp 10.133.0.101:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-jztgt Startup probe failed: Get "http://10.133.0.101:15021/healthz/ready": dial tcp 10.133.0.101:15021: connect: connection refused 32m Warning ProbeError pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-lzt2n Startup probe error: Get "http://10.132.0.126:15021/healthz/ready": dial tcp 10.132.0.126:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-lzt2n Startup probe failed: Get "http://10.132.0.126:15021/healthz/ready": dial tcp 10.132.0.126:15021: connect: connection refused 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-bc2453-7f45fcd947 Error creating: pods "payload-pre-processing-e2e-inf-b-bc2453-7f45fcd947-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-bc2453: serviceaccount "payload-processing-e2e-inf-b-bc2453" not found 32m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-bc2453-7658dcf9c7 Error creating: pods "payload-processing-e2e-inf-b-bc2453-7658dcf9c7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-bc2453: serviceaccount "payload-processing-e2e-inf-b-bc2453" not found 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-bc2453 Scaled up replica set payload-pre-processing-e2e-inf-b-bc2453-7f45fcd947 from 0 to 1 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-bc2453 Scaled up replica set payload-processing-e2e-inf-b-bc2453-7658dcf9c7 from 0 to 1 32m Normal AddedInterface pod/payload-processing-e2e-inf-b-bc2453-7658dcf9c7-smwvn Add eth0 [10.133.0.104/23] from ovn-kubernetes 32m Normal Created pod/payload-pre-processing-e2e-inf-b-bc2453-7f45fcd947-x5bwz Container created 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-bc2453-7f45fcd947 Created pod: payload-pre-processing-e2e-inf-b-bc2453-7f45fcd947-x5bwz 32m Normal Started pod/payload-pre-processing-e2e-inf-b-bc2453-7f45fcd947-x5bwz Container started 32m Normal Pulled pod/payload-pre-processing-e2e-inf-b-bc2453-7f45fcd947-x5bwz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-bc2453-7f45fcd947-x5bwz Add eth0 [10.133.0.103/23] from ovn-kubernetes 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-bc2453-7658dcf9c7 Created pod: payload-processing-e2e-inf-b-bc2453-7658dcf9c7-smwvn 32m Normal Killing pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-gxxl8 Stopping container istio-proxy 32m Normal Killing pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-tlc4t Stopping container istio-proxy 32m Normal Killing pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-vb78p Stopping container istio-proxy 32m Normal Started pod/payload-processing-e2e-inf-b-bc2453-7658dcf9c7-smwvn Container started 32m Normal Pulled pod/payload-processing-e2e-inf-b-bc2453-7658dcf9c7-smwvn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Killing pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-l56vr Stopping container istio-proxy 32m Normal Created pod/payload-processing-e2e-inf-b-bc2453-7658dcf9c7-smwvn Container created 32m Warning ProbeError pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-vb78p Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning Unhealthy pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-vb78p Readiness probe failed: HTTP probe failed with statuscode: 503 32m Warning ProbeError pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-l56vr Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning Unhealthy pod/e2e-worker-w1-982ae3-openshift-default-64d9fb649f-l56vr Readiness probe failed: HTTP probe failed with statuscode: 503 32m Normal Killing pod/payload-processing-e2e-mt-7d5f4de7-cf6c94fdc-zvq97 Stopping container payload-processing 32m Normal Killing pod/payload-pre-processing-e2e-mt-7d5f4de7-ffd6698f9-2r7sb Stopping container payload-pre-processing 31m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-cmr8q 31m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-l9jbt 31m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-2rzpl 31m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-9nrmp 31m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6886b96f9 Created pod: maas-default-gateway-openshift-default-6886b96f9-6m7ds 31m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 31m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-l9jbt Add eth0 [10.134.0.214/23] from ovn-kubernetes 31m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-6m7ds Container started 31m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-2rzpl Add eth0 [10.134.0.215/23] from ovn-kubernetes 31m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-cmr8q Add eth0 [10.133.0.105/23] from ovn-kubernetes 31m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-cmr8q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-cmr8q Container created 31m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-9nrmp Container started 31m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-9nrmp Container created 31m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-9nrmp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-9nrmp Add eth0 [10.132.0.127/23] from ovn-kubernetes 31m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-cmr8q Container started 31m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-2rzpl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-2rzpl Container created 31m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-2rzpl Container started 31m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-6m7ds Add eth0 [10.133.0.106/23] from ovn-kubernetes 31m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-6m7ds Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-6m7ds Container created 31m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-l9jbt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-l9jbt Container created 31m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-l9jbt Container started 31m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-l9jbt Startup probe error: Get "http://10.134.0.214:15021/healthz/ready": dial tcp 10.134.0.214:15021: connect: connection refused... 31m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-l9jbt Startup probe failed: Get "http://10.134.0.214:15021/healthz/ready": dial tcp 10.134.0.214:15021: connect: connection refused 31m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-9nrmp Startup probe error: Get "http://10.132.0.127:15021/healthz/ready": dial tcp 10.132.0.127:15021: connect: connection refused... 31m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-6m7ds Startup probe error: Get "http://10.133.0.106:15021/healthz/ready": dial tcp 10.133.0.106:15021: connect: connection refused... 31m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-2rzpl Startup probe failed: Get "http://10.134.0.215:15021/healthz/ready": dial tcp 10.134.0.215:15021: connect: connection refused 31m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-2rzpl Startup probe error: Get "http://10.134.0.215:15021/healthz/ready": dial tcp 10.134.0.215:15021: connect: connection refused... 31m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-cmr8q Startup probe failed: Get "http://10.133.0.105:15021/healthz/ready": dial tcp 10.133.0.105:15021: connect: connection refused 31m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-6m7ds Startup probe failed: Get "http://10.133.0.106:15021/healthz/ready": dial tcp 10.133.0.106:15021: connect: connection refused 31m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-cmr8q Startup probe error: Get "http://10.133.0.105:15021/healthz/ready": dial tcp 10.133.0.105:15021: connect: connection refused... 31m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-9nrmp Startup probe failed: Get "http://10.132.0.127:15021/healthz/ready": dial tcp 10.132.0.127:15021: connect: connection refused 31m Normal Killing pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-wqncc Stopping container istio-proxy 31m Normal Killing pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-tf2dv Stopping container istio-proxy 31m Normal Killing pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-gvj7f Stopping container istio-proxy 31m Warning Unhealthy pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-tf2dv Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning ProbeError pod/e2e-api-a-5b4db8-openshift-default-5cc6998bbc-tf2dv Readiness probe error: HTTP probe failed with statuscode: 503... 31m Normal Killing pod/payload-pre-processing-e2e-api-b-50a19d-9874b478c-mqhwf Stopping container payload-pre-processing 31m Normal Killing pod/payload-processing-e2e-api-b-50a19d-867c4c57b5-xknkf Stopping container payload-processing 31m Normal Killing pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-d96gv Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-qwvpf Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-cml7r Stopping container istio-proxy 31m Warning Unhealthy pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-qwvpf Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning ProbeError pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-qwvpf Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning ProbeError pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-cml7r Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning Unhealthy pod/e2e-worker-w2-e8ba06-openshift-default-654dcd546f-cml7r Readiness probe failed: HTTP probe failed with statuscode: 503 30m Normal Killing pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-6cwlp Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-dln97 Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-f9ghj Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-fcctm Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-hpj9w Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-jztgt Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-2rtnt Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-lkp42 Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-lzt2n Stopping container istio-proxy 30m Normal Killing pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-mwd5g Stopping container istio-proxy 30m Warning ProbeError pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-f9ghj Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning Unhealthy pod/e2e-mt-7d5f4de7-openshift-default-55c88dfdbf-f9ghj Readiness probe failed: HTTP probe failed with statuscode: 503 30m Normal Killing pod/e2e-api-b-50a19d-openshift-default-68c6d7d74c-24fwv Stopping container istio-proxy 30m Normal Started pod/e2e-inf-b-bc2453-openshift-default-677bb79dc9-8pn9n Container started 30m Normal Created pod/e2e-inf-b-bc2453-openshift-default-677bb79dc9-8pn9n Container created 30m Normal Pulled pod/e2e-inf-b-bc2453-openshift-default-677bb79dc9-8pn9n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Warning Unhealthy pod/e2e-inf-b-bc2453-openshift-default-677bb79dc9-8pn9n Startup probe failed: Get "http://10.133.0.102:15021/healthz/ready": dial tcp 10.133.0.102:15021: connect: connection refused 30m Warning ProbeError pod/e2e-inf-b-bc2453-openshift-default-677bb79dc9-8pn9n Startup probe error: Get "http://10.133.0.102:15021/healthz/ready": dial tcp 10.133.0.102:15021: connect: connection refused... 30m Normal Killing pod/payload-processing-e2e-worker-w3-1e5e75-ff6df9bb9-d888l Stopping container payload-processing 30m Normal Killing pod/payload-pre-processing-e2e-worker-w3-1e5e75-7bf6cbcdcf-6b95p Stopping container payload-pre-processing 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-bc2453-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-bc2453-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Normal Killing pod/payload-pre-processing-e2e-worker-w6-a6f4a5-7666bb86c-glmj2 Stopping container payload-pre-processing 30m Normal Killing pod/payload-processing-e2e-worker-w6-a6f4a5-7b4d7b4dcc-v2hts Stopping container payload-processing 30m Warning Unhealthy pod/payload-processing-e2e-worker-w3-1e5e75-ff6df9bb9-d888l Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w3-1e5e75-7bf6cbcdcf-6b95p Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Normal Killing pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sf5lc Stopping container istio-proxy 30m Normal ScalingReplicaSet deployment/e2e-worker-w0-101b9e-openshift-default Scaled down replica set e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5 from 3 to 2 30m Normal SuccessfulDelete replicaset/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5 Deleted pod: e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sf5lc 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-101b9e-openshift-default New size: 2; reason: All metrics below target 30m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w6-a6f4a5-7666bb86c-glmj2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Warning Unhealthy pod/payload-processing-e2e-worker-w6-a6f4a5-7b4d7b4dcc-v2hts Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Warning ProbeError pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sf5lc Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning Unhealthy pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-sf5lc Readiness probe failed: HTTP probe failed with statuscode: 503 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-bc2453-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-bc2453-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-412743-openshift-default New size: 1; reason: All metrics below target 29m Normal ScalingReplicaSet deployment/e2e-worker-w5-412743-openshift-default Scaled down replica set e2e-worker-w5-412743-openshift-default-7bddc8b5df from 2 to 1 29m Normal SuccessfulDelete replicaset/e2e-worker-w5-412743-openshift-default-7bddc8b5df Deleted pod: e2e-worker-w5-412743-openshift-default-7bddc8b5df-cg79s 29m Normal Killing pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-cg79s Stopping container istio-proxy 29m Normal AddedInterface pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-lzwsk Add eth0 [10.134.0.227/23] from ovn-kubernetes 29m Normal ScalingReplicaSet deployment/e2e-worker-w3-1e5e75-openshift-default Scaled up replica set e2e-worker-w3-1e5e75-openshift-default-879c67bc5 from 2 to 3 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-1e5e75-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 29m Normal SuccessfulCreate replicaset/e2e-worker-w3-1e5e75-openshift-default-879c67bc5 Created pod: e2e-worker-w3-1e5e75-openshift-default-879c67bc5-lzwsk 29m Normal Created pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-lzwsk Container created 29m Normal Pulled pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-lzwsk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Started pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-lzwsk Container started 29m Warning ProbeError pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-lzwsk Startup probe error: Get "http://10.134.0.227:15021/healthz/ready": dial tcp 10.134.0.227:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-lzwsk Startup probe failed: Get "http://10.134.0.227:15021/healthz/ready": dial tcp 10.134.0.227:15021: connect: connection refused 29m Normal Killing pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-spb2t Stopping container istio-proxy 29m Normal Killing pod/payload-pre-processing-e2e-worker-w5-412743-6d5cb5bd7f-pzb7l Stopping container payload-pre-processing 29m Normal Killing pod/payload-processing-e2e-worker-w5-412743-688d66585f-vk5r7 Stopping container payload-processing 29m Normal Killing pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-wx4r4 Stopping container istio-proxy 29m Normal Killing pod/e2e-worker-w3-1e5e75-openshift-default-879c67bc5-lzwsk Stopping container istio-proxy 29m Normal Killing pod/payload-pre-processing-e2e-worker-w0-101b9e-688fd6f5d7-7nrcm Stopping container payload-pre-processing 29m Normal Killing pod/payload-processing-e2e-worker-w0-101b9e-6c44b8947-88qn7 Stopping container payload-processing 28m Normal Killing pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-8prk5 Stopping container istio-proxy 28m Normal Killing pod/e2e-worker-w6-a6f4a5-openshift-default-75d85b9b75-2f6wc Stopping container istio-proxy 28m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w5-412743-6d5cb5bd7f-pzb7l Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Warning Unhealthy pod/payload-processing-e2e-worker-w5-412743-688d66585f-vk5r7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-101b9e-688fd6f5d7-7nrcm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 28m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-xjq9h 28m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 28m Warning Unhealthy pod/payload-processing-e2e-worker-w0-101b9e-6c44b8947-88qn7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-xjq9h Stopping container discovery 28m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-xjq9h Readiness probe failed: Get "http://10.132.0.98:8080/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 28m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-xjq9h Readiness probe error: Get "http://10.132.0.98:8080/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 28m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-101b9e-688fd6f5d7-7nrcm Readiness probe failed: timeout: failed to connect service "10.134.0.44:50052" within 1s: context deadline exceeded 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-101b9e-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 28m Normal SuccessfulCreate replicaset/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5 Created pod: e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-rf2wq 28m Normal SuccessfulCreate replicaset/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5 Created pod: e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-xnr5x 28m Normal ScalingReplicaSet deployment/e2e-worker-w0-101b9e-openshift-default Scaled up replica set e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5 from 2 to 4 28m Normal Created pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-rf2wq Container created 28m Normal SuccessfulDelete replicaset/e2e-worker-w4-da9d07-openshift-default-798d9948bc Deleted pod: e2e-worker-w4-da9d07-openshift-default-798d9948bc-j4tb5 28m Normal AddedInterface pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-rf2wq Add eth0 [10.134.0.234/23] from ovn-kubernetes 28m Normal Pulled pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-rf2wq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Started pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-xnr5x Container started 28m Normal ScalingReplicaSet deployment/e2e-worker-w4-da9d07-openshift-default Scaled down replica set e2e-worker-w4-da9d07-openshift-default-798d9948bc from 2 to 1 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-da9d07-openshift-default New size: 1; reason: All metrics below target 28m Normal Pulled pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-xnr5x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Killing pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-j4tb5 Stopping container istio-proxy 28m Normal Started pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-rf2wq Container started 28m Normal Created pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-xnr5x Container created 28m Normal AddedInterface pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-xnr5x Add eth0 [10.134.0.235/23] from ovn-kubernetes 28m Warning Unhealthy pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-xnr5x Startup probe failed: Get "http://10.134.0.235:15021/healthz/ready": dial tcp 10.134.0.235:15021: connect: connection refused 28m Warning ProbeError pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-xnr5x Startup probe error: Get "http://10.134.0.235:15021/healthz/ready": dial tcp 10.134.0.235:15021: connect: connection refused... 28m Warning ProbeError pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-rf2wq Startup probe error: Get "http://10.134.0.234:15021/healthz/ready": dial tcp 10.134.0.234:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-rf2wq Startup probe failed: Get "http://10.134.0.234:15021/healthz/ready": dial tcp 10.134.0.234:15021: connect: connection refused 28m Normal Killing pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-dz2cl Stopping container istio-proxy 28m Normal Killing pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-ngfr5 Stopping container istio-proxy 28m Warning FailedToRetrieveImagePullSecret pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-xnr5x Unable to retrieve some image pull secrets (e2e-worker-w0-101b9e-openshift-default-dockercfg-8l66m); attempting to pull the image may not succeed. 28m Warning FailedToRetrieveImagePullSecret pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-rf2wq Unable to retrieve some image pull secrets (e2e-worker-w0-101b9e-openshift-default-dockercfg-8l66m); attempting to pull the image may not succeed. 28m Normal Killing pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-xnr5x Stopping container istio-proxy 28m Warning Unhealthy pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-dz2cl Readiness probe failed: HTTP probe failed with statuscode: 503 28m Normal Killing pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-rf2wq Stopping container istio-proxy 28m Warning ProbeError pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-dz2cl Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning Unhealthy pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-ngfr5 Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning ProbeError pod/e2e-worker-w0-101b9e-openshift-default-5bbfdd65d5-ngfr5 Readiness probe error: HTTP probe failed with statuscode: 503... 27m Normal Killing pod/e2e-worker-w5-412743-openshift-default-7bddc8b5df-thfvg Stopping container istio-proxy 26m Normal Killing pod/payload-processing-e2e-inf-a-4ab8e5-b77879c55-qptzw Stopping container payload-processing 26m Normal Killing pod/payload-pre-processing-e2e-inf-a-4ab8e5-79cc98db87-r95fp Stopping container payload-pre-processing 25m Warning Unhealthy pod/payload-processing-e2e-inf-a-4ab8e5-b77879c55-qptzw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-4ab8e5-79cc98db87-r95fp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Normal Killing pod/e2e-inf-a-4ab8e5-openshift-default-6c89ffbf45-mtckf Stopping container istio-proxy 25m Normal Killing pod/payload-processing-e2e-inf-b-bc2453-7658dcf9c7-smwvn Stopping container payload-processing 25m Normal Killing pod/payload-pre-processing-e2e-inf-b-bc2453-7f45fcd947-x5bwz Stopping container payload-pre-processing 25m Normal Killing pod/e2e-inf-b-bc2453-openshift-default-677bb79dc9-8pn9n Stopping container istio-proxy 25m Warning Unhealthy pod/e2e-inf-b-bc2453-openshift-default-677bb79dc9-8pn9n Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning ProbeError pod/e2e-inf-b-bc2453-openshift-default-677bb79dc9-8pn9n Readiness probe error: HTTP probe failed with statuscode: 503... 24m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-bc2453-7f45fcd947-x5bwz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Normal Killing pod/payload-pre-processing-e2e-shared-a-w4-4b86e4-6c67bcf5f-8tb44 Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-shared-a-w4-4b86e4-5f8755b487-kp5qc Stopping container payload-processing 24m Normal Killing pod/e2e-shared-a-w4-4b86e4-openshift-default-79c864d79c-hz7fp Stopping container istio-proxy 24m Normal Killing pod/payload-processing-e2e-shared-b-w4-0da668-78b845796f-5srmm Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-shared-b-w4-0da668-dff997c59-8nrck Stopping container payload-pre-processing 23m Normal Killing pod/e2e-shared-b-w4-0da668-openshift-default-5467f58b7-tzrb7 Stopping container istio-proxy 23m Normal Killing pod/payload-pre-processing-e2e-worker-w4-da9d07-69bb55d779-hg9l2 Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-worker-w4-da9d07-5848fd449-s4wwm Stopping container payload-processing 23m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6886b96f9 Deleted pod: maas-default-gateway-openshift-default-6886b96f9-6m7ds 23m Normal Killing pod/maas-default-gateway-openshift-default-6886b96f9-6m7ds Stopping container istio-proxy 23m Warning Unhealthy pod/payload-processing-e2e-worker-w4-da9d07-5848fd449-s4wwm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w4-da9d07-69bb55d779-hg9l2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Normal Killing pod/maas-default-gateway-openshift-default-6886b96f9-9nrmp Stopping container istio-proxy 23m Normal Killing pod/maas-default-gateway-openshift-default-6886b96f9-2rzpl Stopping container istio-proxy 23m Normal Killing pod/maas-default-gateway-openshift-default-6886b96f9-l9jbt Stopping container istio-proxy 23m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6886b96f9 (combined from similar events): Deleted pod: maas-default-gateway-openshift-default-6886b96f9-thjff 23m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6886b96f9 Deleted pod: maas-default-gateway-openshift-default-6886b96f9-2rzpl 23m Normal Killing pod/maas-default-gateway-openshift-default-6886b96f9-thjff Stopping container istio-proxy 23m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6886b96f9 Deleted pod: maas-default-gateway-openshift-default-6886b96f9-cmr8q 23m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6886b96f9 Deleted pod: maas-default-gateway-openshift-default-6886b96f9-l9jbt 23m Normal Killing pod/maas-default-gateway-openshift-default-6886b96f9-cmr8q Stopping container istio-proxy 23m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-thjff Readiness probe failed: HTTP probe failed with statuscode: 503 23m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-thjff Readiness probe error: HTTP probe failed with statuscode: 503... 23m Normal Killing pod/e2e-worker-w4-da9d07-openshift-default-798d9948bc-s5lkw Stopping container istio-proxy 22m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-qfcgh Container started 22m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-qfcgh Container created 22m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-qfcgh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-qfcgh Add eth0 [10.132.0.130/23] from ovn-kubernetes 22m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-256br Add eth0 [10.134.0.248/23] from ovn-kubernetes 22m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-256br Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-256br Container created 22m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-256br Container started 22m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-qfcgh Startup probe error: Get "http://10.132.0.130:15021/healthz/ready": dial tcp 10.132.0.130:15021: connect: connection refused... 22m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-qfcgh Startup probe failed: Get "http://10.132.0.130:15021/healthz/ready": dial tcp 10.132.0.130:15021: connect: connection refused 22m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-256br Startup probe error: Get "http://10.134.0.248:15021/healthz/ready": dial tcp 10.134.0.248:15021: connect: connection refused... 22m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-256br Startup probe failed: Get "http://10.134.0.248:15021/healthz/ready": dial tcp 10.134.0.248:15021: connect: connection refused 22m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-jd2w5 Container created 22m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-jd2w5 Add eth0 [10.134.0.250/23] from ovn-kubernetes 22m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-q4c82 Container started 22m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-jd2w5 Container started 22m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-q4c82 Add eth0 [10.133.0.108/23] from ovn-kubernetes 22m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-q4c82 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-jd2w5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-q4c82 Container created 22m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-jd2w5 Startup probe error: Get "http://10.134.0.250:15021/healthz/ready": dial tcp 10.134.0.250:15021: connect: connection refused... 22m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-q4c82 Startup probe error: Get "http://10.133.0.108:15021/healthz/ready": dial tcp 10.133.0.108:15021: connect: connection refused... 22m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-q4c82 Startup probe failed: Get "http://10.133.0.108:15021/healthz/ready": dial tcp 10.133.0.108:15021: connect: connection refused 22m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-jd2w5 Startup probe failed: Get "http://10.134.0.250:15021/healthz/ready": dial tcp 10.134.0.250:15021: connect: connection refused 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-zq2lm Add eth0 [10.134.0.252/23] from ovn-kubernetes 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-6886b96f9-vxqbg Add eth0 [10.132.0.132/23] from ovn-kubernetes 21m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default (combined from similar events): New size: 10; reason: cpu resource utilization (percentage of request) above target 21m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default (combined from similar events): Scaled up replica set maas-default-gateway-openshift-default-6886b96f9 from 8 to 10 21m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-zq2lm Container started 21m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-vxqbg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-vxqbg Container created 21m Normal Pulled pod/maas-default-gateway-openshift-default-6886b96f9-zq2lm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Created pod/maas-default-gateway-openshift-default-6886b96f9-zq2lm Container created 21m Normal Started pod/maas-default-gateway-openshift-default-6886b96f9-vxqbg Container started 21m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-zq2lm Startup probe error: Get "http://10.134.0.252:15021/healthz/ready": dial tcp 10.134.0.252:15021: connect: connection refused... 21m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-vxqbg Startup probe error: Get "http://10.132.0.132:15021/healthz/ready": dial tcp 10.132.0.132:15021: connect: connection refused... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-vxqbg Startup probe failed: Get "http://10.132.0.132:15021/healthz/ready": dial tcp 10.132.0.132:15021: connect: connection refused 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-zq2lm Startup probe failed: Get "http://10.134.0.252:15021/healthz/ready": dial tcp 10.134.0.252:15021: connect: connection refused 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-vxqbg Startup probe failed: Get "http://10.132.0.132:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 21m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-zq2lm Startup probe error: Get "http://10.134.0.252:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-zq2lm Startup probe failed: Get "http://10.134.0.252:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 21m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-vxqbg Startup probe error: Get "http://10.132.0.132:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 17m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-256br Readiness probe failed: Get "http://10.134.0.248:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 17m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-256br Readiness probe error: Get "http://10.134.0.248:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 16m Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-zq2lm Readiness probe error: Get "http://10.134.0.252:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-zq2lm Readiness probe failed: Get "http://10.134.0.252:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 15m Normal AddedLabel gateway/e2e-worker-single-main-863fb0 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-863fb0 15m Normal SuccessfulCreate replicaset/e2e-worker-single-main-863fb0-openshift-default-86c56576cc Created pod: e2e-worker-single-main-863fb0-openshift-default-86c56576ccdwkvd 15m Normal ScalingReplicaSet deployment/e2e-worker-single-main-863fb0-openshift-default Scaled up replica set e2e-worker-single-main-863fb0-openshift-default-86c56576cc from 0 to 1 15m Normal AddedInterface pod/e2e-worker-single-main-863fb0-openshift-default-86c56576ccdwkvd Add eth0 [10.134.1.6/23] from ovn-kubernetes 15m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-863fb0-6cdb99d4d7 Error creating: pods "payload-pre-processing-e2e-worker-single-main-863fb0-6cdb99d4d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-863fb0: serviceaccount "payload-processing-e2e-worker-single-main-863fb0" not found 15m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-863fb0 Scaled up replica set payload-processing-e2e-worker-single-main-863fb0-5ff7d48b77 from 0 to 1 15m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-863fb0-5ff7d48b77 Error creating: pods "payload-processing-e2e-worker-single-main-863fb0-5ff7d48b77-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-863fb0: serviceaccount "payload-processing-e2e-worker-single-main-863fb0" not found 15m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-863fb0 Scaled up replica set payload-pre-processing-e2e-worker-single-main-863fb0-6cdb99d4d7 from 0 to 1 15m Normal Pulled pod/payload-processing-e2e-worker-single-main-863fb0-5ff7d48b7sg7fr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 15m Normal Started pod/payload-processing-e2e-worker-single-main-863fb0-5ff7d48b7sg7fr Container started 15m Normal Created pod/payload-processing-e2e-worker-single-main-863fb0-5ff7d48b7sg7fr Container created 15m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-863fb0-6cdb97h84f Add eth0 [10.133.0.113/23] from ovn-kubernetes 15m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-863fb0-5ff7d48b7sg7fr Add eth0 [10.134.1.7/23] from ovn-kubernetes 15m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-863fb0-6cdb99d4d7 Created pod: payload-pre-processing-e2e-worker-single-main-863fb0-6cdb97h84f 15m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-863fb0-5ff7d48b77 Created pod: payload-processing-e2e-worker-single-main-863fb0-5ff7d48b7sg7fr 15m Normal Started pod/payload-pre-processing-e2e-worker-single-main-863fb0-6cdb97h84f Container started 15m Normal Created pod/payload-pre-processing-e2e-worker-single-main-863fb0-6cdb97h84f Container created 15m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-863fb0-6cdb97h84f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 15m Normal Created pod/e2e-worker-single-main-863fb0-openshift-default-86c56576ccdwkvd Container created 15m Normal Started pod/e2e-worker-single-main-863fb0-openshift-default-86c56576ccdwkvd Container started 15m Normal Pulled pod/e2e-worker-single-main-863fb0-openshift-default-86c56576ccdwkvd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 15m Warning Unhealthy pod/e2e-worker-single-main-863fb0-openshift-default-86c56576ccdwkvd Startup probe failed: Get "http://10.134.1.6:15021/healthz/ready": dial tcp 10.134.1.6:15021: connect: connection refused 15m Warning ProbeError pod/e2e-worker-single-main-863fb0-openshift-default-86c56576ccdwkvd Startup probe error: Get "http://10.134.1.6:15021/healthz/ready": dial tcp 10.134.1.6:15021: connect: connection refused... 14m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-863fb0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 14m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-863fb0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 13m Normal Killing pod/payload-processing-e2e-worker-single-main-863fb0-5ff7d48b7sg7fr Stopping container payload-processing 13m Normal Killing pod/payload-pre-processing-e2e-worker-single-main-863fb0-6cdb97h84f Stopping container payload-pre-processing 13m Normal Killing pod/e2e-worker-single-main-863fb0-openshift-default-86c56576ccdwkvd Stopping container istio-proxy 13m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-863fb0-6cdb97h84f Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 13m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-863fb0-5ff7d48b7sg7fr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 11m Normal ScalingReplicaSet deployment/e2e-worker-single-main-abe7bd-openshift-default Scaled up replica set e2e-worker-single-main-abe7bd-openshift-default-7d97f8b7c7 from 0 to 1 11m Normal SuccessfulCreate replicaset/e2e-worker-single-main-abe7bd-openshift-default-7d97f8b7c7 Created pod: e2e-worker-single-main-abe7bd-openshift-default-7d97f8b7c75x2j7 11m Normal AddedLabel gateway/e2e-worker-single-main-abe7bd Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-abe7bd 11m Normal AddedInterface pod/e2e-worker-single-main-abe7bd-openshift-default-7d97f8b7c75x2j7 Add eth0 [10.134.1.9/23] from ovn-kubernetes 10m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-abe7bd-dff668477 Created pod: payload-processing-e2e-worker-single-main-abe7bd-dff668477pzlvc 10m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-abe7bd-545c49f487 Error creating: pods "payload-pre-processing-e2e-worker-single-main-abe7bd-545c49f487-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-abe7bd: serviceaccount "payload-processing-e2e-worker-single-main-abe7bd" not found 10m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-abe7bd-545c49f487 Created pod: payload-pre-processing-e2e-worker-single-main-abe7bd-545c4j4fxw 10m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-abe7bd Scaled up replica set payload-pre-processing-e2e-worker-single-main-abe7bd-545c49f487 from 0 to 1 10m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-abe7bd-dff668477 Error creating: pods "payload-processing-e2e-worker-single-main-abe7bd-dff668477-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-abe7bd: serviceaccount "payload-processing-e2e-worker-single-main-abe7bd" not found 10m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-abe7bd Scaled up replica set payload-processing-e2e-worker-single-main-abe7bd-dff668477 from 0 to 1 10m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-abe7bd-545c4j4fxw Add eth0 [10.133.0.119/23] from ovn-kubernetes 10m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-abe7bd-dff668477pzlvc Add eth0 [10.134.1.10/23] from ovn-kubernetes 10m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-abe7bd-545c4j4fxw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 10m Normal Created pod/payload-pre-processing-e2e-worker-single-main-abe7bd-545c4j4fxw Container created 10m Normal Pulled pod/payload-processing-e2e-worker-single-main-abe7bd-dff668477pzlvc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 10m Normal Started pod/payload-processing-e2e-worker-single-main-abe7bd-dff668477pzlvc Container started 10m Normal Started pod/payload-pre-processing-e2e-worker-single-main-abe7bd-545c4j4fxw Container started 10m Normal Created pod/payload-processing-e2e-worker-single-main-abe7bd-dff668477pzlvc Container created 10m Normal Pulled pod/e2e-worker-single-main-abe7bd-openshift-default-7d97f8b7c75x2j7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 10m Normal Created pod/e2e-worker-single-main-abe7bd-openshift-default-7d97f8b7c75x2j7 Container created 10m Normal Started pod/e2e-worker-single-main-abe7bd-openshift-default-7d97f8b7c75x2j7 Container started 10m Warning Unhealthy pod/e2e-worker-single-main-abe7bd-openshift-default-7d97f8b7c75x2j7 Startup probe failed: Get "http://10.134.1.9:15021/healthz/ready": dial tcp 10.134.1.9:15021: connect: connection refused 10m Warning ProbeError pod/e2e-worker-single-main-abe7bd-openshift-default-7d97f8b7c75x2j7 Startup probe error: Get "http://10.134.1.9:15021/healthz/ready": dial tcp 10.134.1.9:15021: connect: connection refused... 10m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-abe7bd-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 10m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-abe7bd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 9m22s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-abe7bd-545c4j4fxw Stopping container payload-pre-processing 9m22s Normal Killing pod/payload-processing-e2e-worker-single-main-abe7bd-dff668477pzlvc Stopping container payload-processing 9m5s Normal Killing pod/e2e-worker-single-main-abe7bd-openshift-default-7d97f8b7c75x2j7 Stopping container istio-proxy 9m4s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-abe7bd-545c4j4fxw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 9m4s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-abe7bd-dff668477pzlvc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 7m26s Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-qfcgh Readiness probe failed: Get "http://10.132.0.130:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 7m26s Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-qfcgh Readiness probe error: Get "http://10.132.0.130:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 7m26s Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-jd2w5 Readiness probe failed: Get "http://10.134.0.250:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 7m26s Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-jd2w5 Readiness probe error: Get "http://10.134.0.250:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 6m35s Normal SuccessfulCreate replicaset/e2e-ait-576e86c8-openshift-default-98874d9b5 Created pod: e2e-ait-576e86c8-openshift-default-98874d9b5-w9brf 6m35s Normal AddedLabel gateway/e2e-ait-576e86c8 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-576e86c8 6m35s Normal ScalingReplicaSet deployment/e2e-ait-576e86c8-openshift-default Scaled up replica set e2e-ait-576e86c8-openshift-default-98874d9b5 from 0 to 1 6m34s Normal AddedInterface pod/e2e-ait-576e86c8-openshift-default-98874d9b5-w9brf Add eth0 [10.134.1.16/23] from ovn-kubernetes 6m28s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-576e86c8 Scaled up replica set payload-pre-processing-e2e-ait-576e86c8-6876945bcf from 0 to 1 6m28s Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-576e86c8 Scaled up replica set payload-processing-e2e-ait-576e86c8-6f8546b5c9 from 0 to 1 6m27s Normal AddedInterface pod/payload-pre-processing-e2e-ait-576e86c8-6876945bcf-2shmj Add eth0 [10.134.1.17/23] from ovn-kubernetes 6m27s Warning FailedCreate replicaset/payload-processing-e2e-ait-576e86c8-6f8546b5c9 Error creating: pods "payload-processing-e2e-ait-576e86c8-6f8546b5c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-576e86c8: serviceaccount "payload-processing-e2e-ait-576e86c8" not found 6m27s Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-576e86c8-6f8546b5c9 Created pod: payload-processing-e2e-ait-576e86c8-6f8546b5c9-txsdf 6m27s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-576e86c8-6876945bcf Created pod: payload-pre-processing-e2e-ait-576e86c8-6876945bcf-2shmj 6m27s Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-576e86c8-6876945bcf Error creating: pods "payload-pre-processing-e2e-ait-576e86c8-6876945bcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-576e86c8: serviceaccount "payload-processing-e2e-ait-576e86c8" not found 6m26s Normal Pulled pod/payload-pre-processing-e2e-ait-576e86c8-6876945bcf-2shmj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 6m26s Normal AddedInterface pod/payload-processing-e2e-ait-576e86c8-6f8546b5c9-txsdf Add eth0 [10.134.1.18/23] from ovn-kubernetes 6m26s Normal Started pod/payload-pre-processing-e2e-ait-576e86c8-6876945bcf-2shmj Container started 6m26s Normal Created pod/payload-pre-processing-e2e-ait-576e86c8-6876945bcf-2shmj Container created 6m26s Normal Pulled pod/payload-processing-e2e-ait-576e86c8-6f8546b5c9-txsdf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 6m26s Normal Created pod/payload-processing-e2e-ait-576e86c8-6f8546b5c9-txsdf Container created 6m26s Normal Started pod/payload-processing-e2e-ait-576e86c8-6f8546b5c9-txsdf Container started 6m7s Normal Created pod/e2e-ait-576e86c8-openshift-default-98874d9b5-w9brf Container created 6m7s Normal Started pod/e2e-ait-576e86c8-openshift-default-98874d9b5-w9brf Container started 6m7s Normal Pulled pod/e2e-ait-576e86c8-openshift-default-98874d9b5-w9brf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 6m1s Warning Unhealthy pod/e2e-ait-576e86c8-openshift-default-98874d9b5-w9brf Startup probe failed: Get "http://10.134.1.16:15021/healthz/ready": dial tcp 10.134.1.16:15021: connect: connection refused 6m1s Warning ProbeError pod/e2e-ait-576e86c8-openshift-default-98874d9b5-w9brf Startup probe error: Get "http://10.134.1.16:15021/healthz/ready": dial tcp 10.134.1.16:15021: connect: connection refused... 5m43s Normal Killing pod/payload-pre-processing-e2e-ait-576e86c8-6876945bcf-2shmj Stopping container payload-pre-processing 5m43s Normal Killing pod/payload-processing-e2e-ait-576e86c8-6f8546b5c9-txsdf Stopping container payload-processing 5m34s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-576e86c8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 5m34s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-576e86c8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 5m28s Normal Killing pod/e2e-ait-576e86c8-openshift-default-98874d9b5-w9brf Stopping container istio-proxy 5m26s Normal AddedLabel gateway/e2e-authcfg-http-44d22c Added label istio.io/rev=openshift-gateway to gateway e2e-authcfg-http-44d22c 5m26s Normal SuccessfulCreate replicaset/e2e-authcfg-http-44d22c-openshift-default-6b997dc7fc Created pod: e2e-authcfg-http-44d22c-openshift-default-6b997dc7fc-dgpzm 5m26s Normal ScalingReplicaSet deployment/e2e-authcfg-http-44d22c-openshift-default Scaled up replica set e2e-authcfg-http-44d22c-openshift-default-6b997dc7fc from 0 to 1 5m26s Normal AddedInterface pod/e2e-authcfg-http-44d22c-openshift-default-6b997dc7fc-dgpzm Add eth0 [10.134.1.20/23] from ovn-kubernetes 5m19s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-authcfg-http-44d22c Scaled up replica set payload-pre-processing-e2e-authcfg-http-44d22c-649bc5675f from 0 to 1 5m19s Normal ScalingReplicaSet deployment/payload-processing-e2e-authcfg-http-44d22c Scaled up replica set payload-processing-e2e-authcfg-http-44d22c-9c679cbbf from 0 to 1 5m18s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-authcfg-http-44d22c-649bc5675f Created pod: payload-pre-processing-e2e-authcfg-http-44d22c-649bc5675f-zvc4z 5m18s Warning FailedCreate replicaset/payload-processing-e2e-authcfg-http-44d22c-9c679cbbf Error creating: pods "payload-processing-e2e-authcfg-http-44d22c-9c679cbbf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-44d22c: serviceaccount "payload-processing-e2e-authcfg-http-44d22c" not found 5m18s Warning FailedCreate replicaset/payload-pre-processing-e2e-authcfg-http-44d22c-649bc5675f Error creating: pods "payload-pre-processing-e2e-authcfg-http-44d22c-649bc5675f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-44d22c: serviceaccount "payload-processing-e2e-authcfg-http-44d22c" not found 5m18s Normal AddedInterface pod/payload-pre-processing-e2e-authcfg-http-44d22c-649bc5675f-zvc4z Add eth0 [10.134.1.21/23] from ovn-kubernetes 5m18s Normal SuccessfulCreate replicaset/payload-processing-e2e-authcfg-http-44d22c-9c679cbbf Created pod: payload-processing-e2e-authcfg-http-44d22c-9c679cbbf-rmpq6 5m18s Normal Pulled pod/payload-pre-processing-e2e-authcfg-http-44d22c-649bc5675f-zvc4z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m17s Normal Created pod/payload-pre-processing-e2e-authcfg-http-44d22c-649bc5675f-zvc4z Container created 5m17s Normal Started pod/payload-pre-processing-e2e-authcfg-http-44d22c-649bc5675f-zvc4z Container started 5m17s Normal Started pod/payload-processing-e2e-authcfg-http-44d22c-9c679cbbf-rmpq6 Container started 5m17s Normal Created pod/payload-processing-e2e-authcfg-http-44d22c-9c679cbbf-rmpq6 Container created 5m17s Normal Pulled pod/payload-processing-e2e-authcfg-http-44d22c-9c679cbbf-rmpq6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m17s Normal AddedInterface pod/payload-processing-e2e-authcfg-http-44d22c-9c679cbbf-rmpq6 Add eth0 [10.134.1.22/23] from ovn-kubernetes 4m58s Normal Created pod/e2e-authcfg-http-44d22c-openshift-default-6b997dc7fc-dgpzm Container created 4m58s Normal Pulled pod/e2e-authcfg-http-44d22c-openshift-default-6b997dc7fc-dgpzm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 4m57s Normal Started pod/e2e-authcfg-http-44d22c-openshift-default-6b997dc7fc-dgpzm Container started 4m52s Warning Unhealthy pod/e2e-authcfg-http-44d22c-openshift-default-6b997dc7fc-dgpzm Startup probe failed: Get "http://10.134.1.20:15021/healthz/ready": dial tcp 10.134.1.20:15021: connect: connection refused 4m52s Warning ProbeError pod/e2e-authcfg-http-44d22c-openshift-default-6b997dc7fc-dgpzm Startup probe error: Get "http://10.134.1.20:15021/healthz/ready": dial tcp 10.134.1.20:15021: connect: connection refused... 4m26s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-44d22c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m26s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-44d22c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 3m25s Normal Killing pod/payload-pre-processing-e2e-authcfg-http-44d22c-649bc5675f-zvc4z Stopping container payload-pre-processing 3m25s Normal Killing pod/payload-processing-e2e-authcfg-http-44d22c-9c679cbbf-rmpq6 Stopping container payload-processing 3m8s Normal Killing pod/e2e-authcfg-http-44d22c-openshift-default-6b997dc7fc-dgpzm Stopping container istio-proxy 3m6s Warning ProbeError pod/e2e-authcfg-http-44d22c-openshift-default-6b997dc7fc-dgpzm Readiness probe error: HTTP probe failed with statuscode: 503... 3m6s Warning Unhealthy pod/e2e-authcfg-http-44d22c-openshift-default-6b997dc7fc-dgpzm Readiness probe failed: HTTP probe failed with statuscode: 503 2m20s Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-cnm9b Readiness probe failed: Get "http://10.132.0.34:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m20s Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-cnm9b Readiness probe error: Get "http://10.132.0.34:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 2m20s Warning Unhealthy pod/maas-default-gateway-openshift-default-6886b96f9-cdzdr Readiness probe failed: Get "http://10.133.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m20s Warning ProbeError pod/maas-default-gateway-openshift-default-6886b96f9-cdzdr Readiness probe error: Get "http://10.133.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)...