=== events in openshift-ingress (2026-10-30T22:49:10Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 47m Normal Scheduled pod/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x Successfully assigned openshift-ingress/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x to ip-10-0-135-39.us-west-1.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll to ip-10-0-130-104.us-west-1.compute.internal 65m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-v4zqk Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk to ip-10-0-130-104.us-west-1.compute.internal 26m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-v6jrr Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v6jrr to ip-10-0-132-92.us-west-1.compute.internal 37m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-vxnxc Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc to ip-10-0-130-104.us-west-1.compute.internal 26m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-wwv4p Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p to ip-10-0-130-104.us-west-1.compute.internal 50m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-xmvjc Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xmvjc to ip-10-0-132-92.us-west-1.compute.internal 54m Normal Scheduled pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh Successfully assigned openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh to ip-10-0-130-104.us-west-1.compute.internal 54m Normal Scheduled pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-tzj7w Successfully assigned openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-tzj7w to ip-10-0-132-92.us-west-1.compute.internal 8m58s Normal Scheduled pod/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv Successfully assigned openshift-ingress/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv to ip-10-0-135-39.us-west-1.compute.internal 26m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-zbxls Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-zbxls to ip-10-0-132-92.us-west-1.compute.internal 35m Normal Scheduled pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb Successfully assigned openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb to ip-10-0-135-39.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w1-80c366-openshift-default-666b7db455-6ztpg Successfully assigned openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-6ztpg to ip-10-0-132-92.us-west-1.compute.internal 36m Normal Scheduled pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb Successfully assigned openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb to ip-10-0-135-39.us-west-1.compute.internal 64m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-rz8s5 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rz8s5 to ip-10-0-132-92.us-west-1.compute.internal 35m Normal Scheduled pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt Successfully assigned openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt to ip-10-0-130-104.us-west-1.compute.internal 64m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-qxxgq Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-qxxgq to ip-10-0-135-39.us-west-1.compute.internal 50m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-pxw88 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pxw88 to ip-10-0-135-39.us-west-1.compute.internal 37m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-pwfjh Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pwfjh to ip-10-0-132-92.us-west-1.compute.internal 37m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-pptz4 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-pptz4 to ip-10-0-132-92.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w1-80c366-openshift-default-666b7db455-4kk96 Successfully assigned openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-4kk96 to ip-10-0-132-92.us-west-1.compute.internal 57m Normal Scheduled pod/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk Successfully assigned openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk to ip-10-0-130-104.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb Successfully assigned openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb to ip-10-0-130-104.us-west-1.compute.internal 75m Normal Scheduled pod/router-default-54cbfc65d5-sg48h Successfully assigned openshift-ingress/router-default-54cbfc65d5-sg48h to ip-10-0-132-92.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2 Successfully assigned openshift-ingress/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2 to ip-10-0-135-39.us-west-1.compute.internal 76m Warning FailedScheduling pod/router-default-54cbfc65d5-sg48h no nodes available to schedule pods 82m Warning FailedScheduling pod/router-default-54cbfc65d5-sg48h no nodes available to schedule pods 52m Normal Scheduled pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx Successfully assigned openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx to ip-10-0-130-104.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv Successfully assigned openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv to ip-10-0-135-39.us-west-1.compute.internal 43m Normal Scheduled pod/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx Successfully assigned openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx to ip-10-0-130-104.us-west-1.compute.internal 63m Normal Scheduled pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd Successfully assigned openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd to ip-10-0-130-104.us-west-1.compute.internal 54m Normal Scheduled pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s Successfully assigned openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s to ip-10-0-130-104.us-west-1.compute.internal 54m Normal Scheduled pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh Successfully assigned openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh to ip-10-0-135-39.us-west-1.compute.internal 58m Normal Scheduled pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-lg5tj Successfully assigned openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-lg5tj to ip-10-0-132-92.us-west-1.compute.internal 19m Normal Scheduled pod/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f Successfully assigned openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f to ip-10-0-130-104.us-west-1.compute.internal 63m Normal Scheduled pod/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp to ip-10-0-130-104.us-west-1.compute.internal 54m Normal Scheduled pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ldkbp Successfully assigned openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ldkbp to ip-10-0-132-92.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 Successfully assigned openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 to ip-10-0-130-104.us-west-1.compute.internal 54m Normal Scheduled pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8 Successfully assigned openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8 to ip-10-0-130-104.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-vqsb5 Successfully assigned openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-vqsb5 to ip-10-0-132-92.us-west-1.compute.internal 56m Normal Scheduled pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k Successfully assigned openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k to ip-10-0-135-39.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-ait-e585c53f-openshift-default-6b94747c57-5n6cd Successfully assigned openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-5n6cd to ip-10-0-132-92.us-west-1.compute.internal 54m Normal Scheduled pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wn9kx Successfully assigned openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wn9kx to ip-10-0-132-92.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92 Successfully assigned openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92 to ip-10-0-135-39.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-xb49c Successfully assigned openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-xb49c to ip-10-0-132-92.us-west-1.compute.internal 63m Normal Scheduled pod/payload-processing-e2e-worker-w5-9b51e1-65b8fbbc9f-ng2zh Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-9b51e1-65b8fbbc9f-ng2zh to ip-10-0-132-92.us-west-1.compute.internal 45m Normal Scheduled pod/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7 Successfully assigned openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7 to ip-10-0-130-104.us-west-1.compute.internal 54m Normal Scheduled pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4 Successfully assigned openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4 to ip-10-0-135-39.us-west-1.compute.internal 63m Normal Scheduled pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6 Successfully assigned openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6 to ip-10-0-130-104.us-west-1.compute.internal 56m Normal Scheduled pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-rf429 Successfully assigned openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-rf429 to ip-10-0-132-92.us-west-1.compute.internal 54m Normal Scheduled pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr Successfully assigned openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr to ip-10-0-130-104.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9 Successfully assigned openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9 to ip-10-0-130-104.us-west-1.compute.internal 63m Normal Scheduled pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc Successfully assigned openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc to ip-10-0-135-39.us-west-1.compute.internal 57m Normal Scheduled pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt Successfully assigned openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt to ip-10-0-135-39.us-west-1.compute.internal 63m Normal Scheduled pod/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5 to ip-10-0-130-104.us-west-1.compute.internal 35m Normal Scheduled pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk Successfully assigned openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk to ip-10-0-130-104.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6 Successfully assigned openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6 to ip-10-0-135-39.us-west-1.compute.internal 58m Normal Scheduled pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9 Successfully assigned openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9 to ip-10-0-135-39.us-west-1.compute.internal 57m Normal Scheduled pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26 Successfully assigned openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26 to ip-10-0-135-39.us-west-1.compute.internal 57m Normal Scheduled pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-fkzmf Successfully assigned openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-fkzmf to ip-10-0-132-92.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-khbxp Successfully assigned openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-khbxp to ip-10-0-132-92.us-west-1.compute.internal 63m Normal Scheduled pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5 Successfully assigned openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5 to ip-10-0-130-104.us-west-1.compute.internal 57m Normal Scheduled pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz Successfully assigned openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz to ip-10-0-130-104.us-west-1.compute.internal 63m Normal Scheduled pod/payload-processing-e2e-worker-w3-bc009f-66fdc54f65-88jlx Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-bc009f-66fdc54f65-88jlx to ip-10-0-132-92.us-west-1.compute.internal 43m Normal Scheduled pod/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h Successfully assigned openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h to ip-10-0-130-104.us-west-1.compute.internal 14m Normal Scheduled pod/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh Successfully assigned openshift-ingress/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh to ip-10-0-135-39.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r Successfully assigned openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r to ip-10-0-130-104.us-west-1.compute.internal 56m Normal Scheduled pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x86qc Successfully assigned openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x86qc to ip-10-0-132-92.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl Successfully assigned openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl to ip-10-0-130-104.us-west-1.compute.internal 57m Normal Scheduled pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5 Successfully assigned openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5 to ip-10-0-130-104.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj Successfully assigned openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj to ip-10-0-135-39.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x Successfully assigned openshift-ingress/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x to ip-10-0-135-39.us-west-1.compute.internal 63m Normal Scheduled pod/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh to ip-10-0-130-104.us-west-1.compute.internal 55m Normal Scheduled pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r Successfully assigned openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r to ip-10-0-130-104.us-west-1.compute.internal 61m Normal Scheduled pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26 Successfully assigned openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26 to ip-10-0-135-39.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-xzvzb Successfully assigned openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-xzvzb to ip-10-0-132-92.us-west-1.compute.internal 33m Normal Scheduled pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr Successfully assigned openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr to ip-10-0-135-39.us-west-1.compute.internal 44m Normal Scheduled pod/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz Successfully assigned openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz to ip-10-0-130-104.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4 Successfully assigned openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4 to ip-10-0-130-104.us-west-1.compute.internal 33m Normal Scheduled pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp Successfully assigned openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp to ip-10-0-130-104.us-west-1.compute.internal 63m Normal Scheduled pod/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g to ip-10-0-130-104.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw Successfully assigned openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw to ip-10-0-130-104.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-bf95b Successfully assigned openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-bf95b to ip-10-0-132-92.us-west-1.compute.internal 55m Normal Scheduled pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77 Successfully assigned openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77 to ip-10-0-135-39.us-west-1.compute.internal 57m Normal Scheduled pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-fj8mc Successfully assigned openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-fj8mc to ip-10-0-132-92.us-west-1.compute.internal 58m Normal Scheduled pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj Successfully assigned openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj to ip-10-0-130-104.us-west-1.compute.internal 63m Normal Scheduled pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf Successfully assigned openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf to ip-10-0-130-104.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq Successfully assigned openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq to ip-10-0-130-104.us-west-1.compute.internal 42m Normal Scheduled pod/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb Successfully assigned openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb to ip-10-0-130-104.us-west-1.compute.internal 63m Normal Scheduled pod/payload-processing-e2e-worker-w0-dda9f5-75fcc89f49-bh6xw Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-dda9f5-75fcc89f49-bh6xw to ip-10-0-132-92.us-west-1.compute.internal 13m Normal Scheduled pod/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr to ip-10-0-135-39.us-west-1.compute.internal 18m Normal Scheduled pod/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp to ip-10-0-135-39.us-west-1.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p to ip-10-0-135-39.us-west-1.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94 to ip-10-0-130-104.us-west-1.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5 to ip-10-0-135-39.us-west-1.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-shared-b-w0-485d1e-66c964c857-fddzn Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-485d1e-66c964c857-fddzn to ip-10-0-132-92.us-west-1.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t to ip-10-0-135-39.us-west-1.compute.internal 47m Normal Scheduled pod/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp to ip-10-0-135-39.us-west-1.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l to ip-10-0-130-104.us-west-1.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn to ip-10-0-130-104.us-west-1.compute.internal 47m Normal Scheduled pod/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh Successfully assigned openshift-ingress/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh to ip-10-0-135-39.us-west-1.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n Successfully assigned openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n to ip-10-0-130-104.us-west-1.compute.internal 37m Normal Scheduled pod/payload-processing-e2e-mt-6df3ba0e-84d7646477-6qgzr Successfully assigned openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-6qgzr to ip-10-0-132-92.us-west-1.compute.internal 42m Normal Scheduled pod/e2e-api-a-9c8c4d-openshift-default-dfc9cb655-fzv82 Successfully assigned openshift-ingress/e2e-api-a-9c8c4d-openshift-default-dfc9cb655-fzv82 to ip-10-0-132-92.us-west-1.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m Successfully assigned openshift-ingress/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m to ip-10-0-135-39.us-west-1.compute.internal 36m Normal Scheduled pod/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c to ip-10-0-135-39.us-west-1.compute.internal 38m Normal Scheduled pod/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7 to ip-10-0-135-39.us-west-1.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl Successfully assigned openshift-ingress/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl to ip-10-0-135-39.us-west-1.compute.internal 7m27s Normal Scheduled pod/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5 Successfully assigned openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5 to ip-10-0-135-39.us-west-1.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-api-b-307368-7574bb79f5-t8w22 Successfully assigned openshift-ingress/payload-processing-e2e-api-b-307368-7574bb79f5-t8w22 to ip-10-0-132-92.us-west-1.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt Successfully assigned openshift-ingress/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt to ip-10-0-130-104.us-west-1.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd Successfully assigned openshift-ingress/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd to ip-10-0-135-39.us-west-1.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4 to ip-10-0-135-39.us-west-1.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-ait-f05b6443-fbbf556dc-bdtcf Successfully assigned openshift-ingress/payload-processing-e2e-ait-f05b6443-fbbf556dc-bdtcf to ip-10-0-132-92.us-west-1.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v Successfully assigned openshift-ingress/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v to ip-10-0-135-39.us-west-1.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh Successfully assigned openshift-ingress/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh to ip-10-0-135-39.us-west-1.compute.internal 8m51s Normal Scheduled pod/payload-processing-e2e-ait-5ff4cfa5-97fcb7787-4dfjj Successfully assigned openshift-ingress/payload-processing-e2e-ait-5ff4cfa5-97fcb7787-4dfjj to ip-10-0-132-92.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh Successfully assigned openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh to ip-10-0-135-39.us-west-1.compute.internal 63m Normal Scheduled pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k Successfully assigned openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k to ip-10-0-135-39.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm Successfully assigned openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm to ip-10-0-135-39.us-west-1.compute.internal 44m Normal Scheduled pod/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw Successfully assigned openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw to ip-10-0-130-104.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr Successfully assigned openshift-ingress/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr to ip-10-0-135-39.us-west-1.compute.internal 59m Normal Scheduled pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-msfnj Successfully assigned openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-msfnj to ip-10-0-132-92.us-west-1.compute.internal 56m Normal Scheduled pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-4s94l Successfully assigned openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-4s94l to ip-10-0-132-92.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-5b9gk Successfully assigned openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-5b9gk to ip-10-0-132-92.us-west-1.compute.internal 65m Normal Scheduled pod/payload-processing-55fbbbf9dc-8m8pr Successfully assigned openshift-ingress/payload-processing-55fbbbf9dc-8m8pr to ip-10-0-130-104.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp Successfully assigned openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp to ip-10-0-135-39.us-west-1.compute.internal 58m Normal Scheduled pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94 Successfully assigned openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94 to ip-10-0-135-39.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd Successfully assigned openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd to ip-10-0-130-104.us-west-1.compute.internal 55m Normal Scheduled pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-tmlmf Successfully assigned openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-tmlmf to ip-10-0-132-92.us-west-1.compute.internal 59m Normal Scheduled pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv Successfully assigned openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv to ip-10-0-130-104.us-west-1.compute.internal 59m Normal Scheduled pod/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2 Successfully assigned openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2 to ip-10-0-130-104.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr Successfully assigned openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr to ip-10-0-130-104.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk Successfully assigned openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk to ip-10-0-135-39.us-west-1.compute.internal 7m27s Normal Scheduled pod/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf Successfully assigned openshift-ingress/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf to ip-10-0-135-39.us-west-1.compute.internal 69m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-576b8db9c7m9wmf Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-576b8db9c7m9wmf to ip-10-0-132-92.us-west-1.compute.internal 56m Normal Scheduled pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w Successfully assigned openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w to ip-10-0-130-104.us-west-1.compute.internal 57m Normal Scheduled pod/e2e-worker-w1-80c366-openshift-default-666b7db455-l6mk7 Successfully assigned openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-l6mk7 to ip-10-0-132-92.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w1-80c366-openshift-default-666b7db455-dhs4k Successfully assigned openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-dhs4k to ip-10-0-132-92.us-west-1.compute.internal 54m Normal Scheduled pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-hfdvc Successfully assigned openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-hfdvc to ip-10-0-132-92.us-west-1.compute.internal 71m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-mhjgp Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhjgp to ip-10-0-135-39.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p Successfully assigned openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p to ip-10-0-130-104.us-west-1.compute.internal 35m Normal Scheduled pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-npttq Successfully assigned openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-npttq to ip-10-0-132-92.us-west-1.compute.internal 65m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-p8khq Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-p8khq to ip-10-0-135-39.us-west-1.compute.internal 63m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4 to ip-10-0-135-39.us-west-1.compute.internal 50m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-nzzv2 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2 to ip-10-0-130-104.us-west-1.compute.internal 50m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-nvkls Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nvkls to ip-10-0-135-39.us-west-1.compute.internal 35m Normal Scheduled pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-wc8sm Successfully assigned openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-wc8sm to ip-10-0-132-92.us-west-1.compute.internal 56m Normal Scheduled pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n Successfully assigned openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n to ip-10-0-130-104.us-west-1.compute.internal 64m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-mglq4 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4 to ip-10-0-130-104.us-west-1.compute.internal 54m Normal Scheduled pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs Successfully assigned openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs to ip-10-0-135-39.us-west-1.compute.internal 50m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-jj2hd Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jj2hd to ip-10-0-132-92.us-west-1.compute.internal 7m35s Normal Scheduled pod/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk Successfully assigned openshift-ingress/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk to ip-10-0-135-39.us-west-1.compute.internal 63m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf to ip-10-0-130-104.us-west-1.compute.internal 35m Normal Scheduled pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq Successfully assigned openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq to ip-10-0-135-39.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2 Successfully assigned openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2 to ip-10-0-130-104.us-west-1.compute.internal 51m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-4z8hd Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd to ip-10-0-130-104.us-west-1.compute.internal 68m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-clbjj Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-clbjj to ip-10-0-135-39.us-west-1.compute.internal 48m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-zctbt Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zctbt to ip-10-0-132-92.us-west-1.compute.internal 35m Normal Scheduled pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc Successfully assigned openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc to ip-10-0-130-104.us-west-1.compute.internal 65m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-jhtzt Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-jhtzt to ip-10-0-135-39.us-west-1.compute.internal 63m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b to ip-10-0-130-104.us-west-1.compute.internal 50m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-fwnc4 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4 to ip-10-0-130-104.us-west-1.compute.internal 69m Normal Scheduled pod/kube-auth-proxy-6c46d956fb-42ws7 Successfully assigned openshift-ingress/kube-auth-proxy-6c46d956fb-42ws7 to ip-10-0-135-39.us-west-1.compute.internal 71m Normal Scheduled pod/openshift-ai-inference-openshift-ai-inference-fc457bc99-gjm4s Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-fc457bc99-gjm4s to ip-10-0-132-92.us-west-1.compute.internal 45m Normal Scheduled pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc Successfully assigned openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc to ip-10-0-135-39.us-west-1.compute.internal 35m Normal Scheduled pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-4kk54 Successfully assigned openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-4kk54 to ip-10-0-132-92.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf Successfully assigned openshift-ingress/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf to ip-10-0-135-39.us-west-1.compute.internal 69m Normal Scheduled pod/kube-auth-proxy-6c46d956fb-t9g2k Successfully assigned openshift-ingress/kube-auth-proxy-6c46d956fb-t9g2k to ip-10-0-130-104.us-west-1.compute.internal 65m Normal Scheduled pod/payload-pre-processing-7b98bb678c-l4jhz Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-l4jhz to ip-10-0-130-104.us-west-1.compute.internal 63m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx to ip-10-0-130-104.us-west-1.compute.internal 50m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-fskjv Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fskjv to ip-10-0-132-92.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm Successfully assigned openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm to ip-10-0-130-104.us-west-1.compute.internal 26m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-f5gbx Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx to ip-10-0-130-104.us-west-1.compute.internal 37m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-f2rlm Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm to ip-10-0-130-104.us-west-1.compute.internal 50m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-d5bcl Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-d5bcl to ip-10-0-132-92.us-west-1.compute.internal 71m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-24vpd Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd to ip-10-0-130-104.us-west-1.compute.internal 50m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-bmmjn Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bmmjn to ip-10-0-135-39.us-west-1.compute.internal 63m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48 to ip-10-0-130-104.us-west-1.compute.internal 63m Normal Scheduled pod/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6 Successfully assigned openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6 to ip-10-0-135-39.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg Successfully assigned openshift-ingress/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg to ip-10-0-135-39.us-west-1.compute.internal 47m Normal Scheduled pod/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp Successfully assigned openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp to ip-10-0-130-104.us-west-1.compute.internal 38m Normal Scheduled pod/e2e-inf-a-9da607-openshift-default-5d68dbc6c7-jld42 Successfully assigned openshift-ingress/e2e-inf-a-9da607-openshift-default-5d68dbc6c7-jld42 to ip-10-0-132-92.us-west-1.compute.internal 37m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-44qk9 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-44qk9 to ip-10-0-132-92.us-west-1.compute.internal 26m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-46ldm Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm to ip-10-0-130-104.us-west-1.compute.internal 65m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-4k5dm Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-4k5dm to ip-10-0-132-92.us-west-1.compute.internal 26m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-55hwj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj to ip-10-0-130-104.us-west-1.compute.internal 63m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-dda9f5-5cb698d9c9-hj25v Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-dda9f5-5cb698d9c9-hj25v to ip-10-0-132-92.us-west-1.compute.internal 13m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z to ip-10-0-135-39.us-west-1.compute.internal 18m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk to ip-10-0-135-39.us-west-1.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8 to ip-10-0-135-39.us-west-1.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg to ip-10-0-130-104.us-west-1.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm to ip-10-0-135-39.us-west-1.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w0-485d1e-58bccc79c9-jk9hv Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-485d1e-58bccc79c9-jk9hv to ip-10-0-132-92.us-west-1.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n to ip-10-0-135-39.us-west-1.compute.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p to ip-10-0-135-39.us-west-1.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd to ip-10-0-130-104.us-west-1.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4 to ip-10-0-130-104.us-west-1.compute.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2 to ip-10-0-135-39.us-west-1.compute.internal 36m Normal Scheduled pod/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs Successfully assigned openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs to ip-10-0-130-104.us-west-1.compute.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-mcc6n Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-mcc6n to ip-10-0-132-92.us-west-1.compute.internal 41m Normal Scheduled pod/e2e-api-b-307368-openshift-default-748d46b5cc-hzgkz Successfully assigned openshift-ingress/e2e-api-b-307368-openshift-default-748d46b5cc-hzgkz to ip-10-0-132-92.us-west-1.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd to ip-10-0-135-39.us-west-1.compute.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87 to ip-10-0-135-39.us-west-1.compute.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p to ip-10-0-135-39.us-west-1.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd to ip-10-0-135-39.us-west-1.compute.internal 63m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h to ip-10-0-130-104.us-west-1.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-api-b-307368-68678879f5-dtgqs Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-307368-68678879f5-dtgqs to ip-10-0-132-92.us-west-1.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z to ip-10-0-130-104.us-west-1.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n to ip-10-0-135-39.us-west-1.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-api-a-78cf83-796db64fff-7ph2p Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-78cf83-796db64fff-7ph2p to ip-10-0-132-92.us-west-1.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x to ip-10-0-135-39.us-west-1.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm to ip-10-0-135-39.us-west-1.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-ait-be51942b-b8f988b87-bnklt Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-be51942b-b8f988b87-bnklt to ip-10-0-132-92.us-west-1.compute.internal 8m51s Normal Scheduled pod/payload-pre-processing-e2e-ait-5ff4cfa5-7bdc4f6fb7-ktnhr Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-5ff4cfa5-7bdc4f6fb7-ktnhr to ip-10-0-132-92.us-west-1.compute.internal 65m Normal Scheduled pod/maas-default-gateway-openshift-default-74854d8ccf-8xwmx Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-8xwmx to ip-10-0-132-92.us-west-1.compute.internal 82m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-54cbfc65d5 from 0 to 1 82m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 82m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 82m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 82m Normal SuccessfulCreate replicaset/router-default-54cbfc65d5 Created pod: router-default-54cbfc65d5-sg48h 76m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 75m Warning FailedMount pod/router-default-54cbfc65d5-sg48h MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 75m Warning FailedMount pod/router-default-54cbfc65d5-sg48h MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 75m Warning FailedMount pod/router-default-54cbfc65d5-sg48h MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 75m Normal AddedInterface pod/router-default-54cbfc65d5-sg48h Add eth0 [10.132.0.9/23] from ovn-kubernetes 75m Normal Started pod/router-default-54cbfc65d5-sg48h Container started 75m Normal Created pod/router-default-54cbfc65d5-sg48h Container created 75m Normal Pulled pod/router-default-54cbfc65d5-sg48h Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 71m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 71m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-mhjgp 71m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 71m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-mhjgp Add eth0 [10.133.0.20/23] from ovn-kubernetes 71m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-mhjgp Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 71m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-mhjgp Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.808s (3.808s including waiting). Image size: 185476477 bytes. 71m Normal Created pod/istiod-openshift-gateway-7876bfcb74-mhjgp Container created 71m Normal Started pod/istiod-openshift-gateway-7876bfcb74-mhjgp Container started 71m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-mhjgp Readiness probe failed: HTTP probe failed with statuscode: 503 71m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-mhjgp Readiness probe error: HTTP probe failed with statuscode: 503... 71m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-74854d8ccf from 0 to 1 71m Normal EnsuringLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensuring load balancer 71m Normal Pulling pod/maas-default-gateway-openshift-default-74854d8ccf-24vpd Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 71m Normal SuccessfulCreate replicaset/openshift-ai-inference-openshift-ai-inference-fc457bc99 Created pod: openshift-ai-inference-openshift-ai-inference-fc457bc99-gjm4s 71m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-24vpd Add eth0 [10.134.0.22/23] from ovn-kubernetes 71m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-74854d8ccf Created pod: maas-default-gateway-openshift-default-74854d8ccf-24vpd 71m Normal ScalingReplicaSet deployment/openshift-ai-inference-openshift-ai-inference Scaled up replica set openshift-ai-inference-openshift-ai-inference-fc457bc99 from 0 to 1 71m Normal Pulling pod/openshift-ai-inference-openshift-ai-inference-fc457bc99-gjm4s Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 71m Normal AddedInterface pod/openshift-ai-inference-openshift-ai-inference-fc457bc99-gjm4s Add eth0 [10.132.0.31/23] from ovn-kubernetes 71m Normal Created pod/openshift-ai-inference-openshift-ai-inference-fc457bc99-gjm4s Container created 71m Normal Pulled pod/openshift-ai-inference-openshift-ai-inference-fc457bc99-gjm4s Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.587s (3.587s including waiting). Image size: 183555333 bytes. 71m Normal Started pod/openshift-ai-inference-openshift-ai-inference-fc457bc99-gjm4s Container started 71m Normal EnsuredLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensured load balancer 71m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-24vpd Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 8.781s (8.781s including waiting). Image size: 183555333 bytes. 71m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 71m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 71m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 71m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 70m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 70m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 69m Normal SuccessfulCreate replicaset/kube-auth-proxy-6c46d956fb Created pod: kube-auth-proxy-6c46d956fb-42ws7 69m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-6c46d956fb from 0 to 2 69m Normal SuccessfulCreate replicaset/kube-auth-proxy-6c46d956fb Created pod: kube-auth-proxy-6c46d956fb-t9g2k 69m Normal AddedInterface pod/kube-auth-proxy-6c46d956fb-42ws7 Add eth0 [10.133.0.23/23] from ovn-kubernetes 69m Normal Pulling pod/kube-auth-proxy-6c46d956fb-t9g2k Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 69m Normal AddedInterface pod/kube-auth-proxy-6c46d956fb-t9g2k Add eth0 [10.134.0.32/23] from ovn-kubernetes 69m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-576b8db9c7 Created pod: data-science-gateway-data-science-gateway-class-576b8db9c7m9wmf 69m Normal Pulling pod/kube-auth-proxy-6c46d956fb-42ws7 Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 69m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-576b8db9c7 from 0 to 1 69m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-576b8db9c7m9wmf Add eth0 [10.132.0.36/23] from ovn-kubernetes 69m Normal Started pod/data-science-gateway-data-science-gateway-class-576b8db9c7m9wmf Container started 69m Normal Created pod/data-science-gateway-data-science-gateway-class-576b8db9c7m9wmf Container created 69m Normal Pulled pod/data-science-gateway-data-science-gateway-class-576b8db9c7m9wmf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 69m Normal Started pod/kube-auth-proxy-6c46d956fb-42ws7 Container started 69m Normal Pulled pod/kube-auth-proxy-6c46d956fb-42ws7 Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.665s (2.665s including waiting). Image size: 188904677 bytes. 69m Normal Started pod/kube-auth-proxy-6c46d956fb-t9g2k Container started 69m Normal Created pod/kube-auth-proxy-6c46d956fb-t9g2k Container created 69m Normal Pulled pod/kube-auth-proxy-6c46d956fb-t9g2k Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.932s (2.932s including waiting). Image size: 188904677 bytes. 69m Normal Created pod/kube-auth-proxy-6c46d956fb-42ws7 Container created 69m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 69m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 69m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 69m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 68m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-clbjj Container started 68m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 68m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-clbjj 68m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-clbjj Add eth0 [10.133.0.28/23] from ovn-kubernetes 68m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-clbjj Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 68m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-clbjj Container created 68m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-clbjj Readiness probe failed: HTTP probe failed with statuscode: 503 68m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-clbjj Readiness probe error: HTTP probe failed with statuscode: 503... 68m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-mhjgp Stopping container discovery 68m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 68m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-mhjgp 68m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 68m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 68m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-24vpd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 68m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-24vpd Container started 68m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-24vpd Container created 67m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-24vpd Startup probe error: Get "http://10.134.0.22:15021/healthz/ready": dial tcp 10.134.0.22:15021: connect: connection refused... 67m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-24vpd Startup probe failed: Get "http://10.134.0.22:15021/healthz/ready": dial tcp 10.134.0.22:15021: connect: connection refused 67m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 67m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 67m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 67m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-74854d8ccf from 1 to 4 66m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-74854d8ccf Created pod: maas-default-gateway-openshift-default-74854d8ccf-v4zqk 66m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-74854d8ccf Created pod: maas-default-gateway-openshift-default-74854d8ccf-jhtzt 66m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-74854d8ccf Created pod: maas-default-gateway-openshift-default-74854d8ccf-8xwmx 65m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-jhtzt Add eth0 [10.133.0.37/23] from ovn-kubernetes 65m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-8xwmx Container started 65m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-v4zqk Container started 65m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-v4zqk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-8xwmx Container created 65m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-8xwmx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-v4zqk Container created 65m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-v4zqk Add eth0 [10.134.0.39/23] from ovn-kubernetes 65m Normal Pulling pod/maas-default-gateway-openshift-default-74854d8ccf-jhtzt Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 65m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-8xwmx Add eth0 [10.132.0.47/23] from ovn-kubernetes 65m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-jhtzt Container started 65m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-jhtzt Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.999s (3.999s including waiting). Image size: 183555333 bytes. 65m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-jhtzt Container created 65m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-v4zqk Startup probe error: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused... 65m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-v4zqk Startup probe failed: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused 65m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-8xwmx Startup probe failed: Get "http://10.132.0.47:15021/healthz/ready": dial tcp 10.132.0.47:15021: connect: connection refused 65m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-8xwmx Startup probe error: Get "http://10.132.0.47:15021/healthz/ready": dial tcp 10.132.0.47:15021: connect: connection refused... 65m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-v4zqk Startup probe failed: Get "http://10.134.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 65m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-8xwmx Startup probe failed: Get "http://10.132.0.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 65m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-v4zqk Startup probe error: Get "http://10.134.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 65m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-8xwmx Startup probe error: Get "http://10.132.0.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 65m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-jhtzt Startup probe error: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused... 65m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-jhtzt Startup probe failed: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused 65m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-74854d8ccf Created pod: maas-default-gateway-openshift-default-74854d8ccf-p8khq 65m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-74854d8ccf Created pod: maas-default-gateway-openshift-default-74854d8ccf-4k5dm 65m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 65m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-74854d8ccf from 4 to 6 65m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-p8khq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-p8khq Container created 65m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-p8khq Add eth0 [10.133.0.38/23] from ovn-kubernetes 65m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-p8khq Container started 65m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-4k5dm Add eth0 [10.132.0.48/23] from ovn-kubernetes 65m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-4k5dm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-4k5dm Container created 65m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-4k5dm Container started 65m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-4k5dm Startup probe error: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused... 65m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-p8khq Startup probe error: Get "http://10.133.0.38:15021/healthz/ready": dial tcp 10.133.0.38:15021: connect: connection refused... 65m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-p8khq Startup probe failed: Get "http://10.133.0.38:15021/healthz/ready": dial tcp 10.133.0.38:15021: connect: connection refused 65m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-4k5dm Startup probe failed: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused 65m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-55fbbbf9dc from 0 to 1 65m Warning FailedCreate replicaset/payload-processing-55fbbbf9dc Error creating: pods "payload-processing-55fbbbf9dc-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 65m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 65m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-l4jhz 65m Normal SuccessfulCreate replicaset/payload-processing-55fbbbf9dc Created pod: payload-processing-55fbbbf9dc-8m8pr 65m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 65m Normal Pulling pod/payload-processing-55fbbbf9dc-8m8pr Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 65m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-l4jhz Add eth0 [10.134.0.42/23] from ovn-kubernetes 65m Normal Pulling pod/payload-pre-processing-7b98bb678c-l4jhz Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 65m Normal AddedInterface pod/payload-processing-55fbbbf9dc-8m8pr Add eth0 [10.134.0.43/23] from ovn-kubernetes 65m Normal Pulled pod/payload-pre-processing-7b98bb678c-l4jhz Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.033s (3.033s including waiting). Image size: 125353965 bytes. 65m Normal Pulled pod/payload-processing-55fbbbf9dc-8m8pr Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.896s (2.896s including waiting). Image size: 125353965 bytes. 65m Normal Created pod/payload-processing-55fbbbf9dc-8m8pr Container created 65m Normal Started pod/payload-processing-55fbbbf9dc-8m8pr Container started 65m Normal Started pod/payload-pre-processing-7b98bb678c-l4jhz Container started 65m Normal Created pod/payload-pre-processing-7b98bb678c-l4jhz Container created 64m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-74854d8ccf from 6 to 9 64m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-qxxgq Add eth0 [10.133.0.39/23] from ovn-kubernetes 64m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-74854d8ccf Created pod: maas-default-gateway-openshift-default-74854d8ccf-mglq4 64m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-74854d8ccf Created pod: maas-default-gateway-openshift-default-74854d8ccf-qxxgq 64m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 64m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-74854d8ccf Created pod: maas-default-gateway-openshift-default-74854d8ccf-rz8s5 64m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-mglq4 Add eth0 [10.134.0.45/23] from ovn-kubernetes 64m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-rz8s5 Add eth0 [10.132.0.49/23] from ovn-kubernetes 64m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-qxxgq Container started 64m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-mglq4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-mglq4 Container started 64m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-mglq4 Container created 64m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-qxxgq Container created 64m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-rz8s5 Container created 64m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-rz8s5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-rz8s5 Container started 64m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-qxxgq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-rz8s5 Startup probe failed: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused 64m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-rz8s5 Startup probe error: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused... 64m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-qxxgq Startup probe failed: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused 64m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-qxxgq Startup probe error: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused... 64m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-mglq4 Startup probe failed: Get "http://10.134.0.45:15021/healthz/ready": dial tcp 10.134.0.45:15021: connect: connection refused 64m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-mglq4 Startup probe error: Get "http://10.134.0.45:15021/healthz/ready": dial tcp 10.134.0.45:15021: connect: connection refused... 63m Normal AddedLabel gateway/e2e-worker-w1-80c366 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-80c366 63m Normal AddedLabel gateway/e2e-worker-w5-9b51e1 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-9b51e1 63m Normal AddedLabel gateway/e2e-worker-w3-bc009f Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-bc009f 63m Normal AddedLabel gateway/e2e-worker-w0-dda9f5 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-dda9f5 63m Normal AddedLabel gateway/e2e-worker-w2-c79cb5 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-c79cb5 63m Normal AddedLabel gateway/e2e-worker-w6-557708 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-557708 63m Normal AddedLabel gateway/e2e-worker-w4-c3e2d8 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-c3e2d8 63m Normal ScalingReplicaSet deployment/e2e-worker-w6-557708-openshift-default Scaled up replica set e2e-worker-w6-557708-openshift-default-bd7dbddd5 from 0 to 1 63m Normal SuccessfulCreate replicaset/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c Created pod: e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd 63m Normal ScalingReplicaSet deployment/e2e-worker-w4-c3e2d8-openshift-default Scaled up replica set e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c from 0 to 1 63m Normal SuccessfulCreate replicaset/e2e-worker-w6-557708-openshift-default-bd7dbddd5 Created pod: e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf 63m Normal SuccessfulCreate replicaset/e2e-worker-w2-c79cb5-openshift-default-5464c54587 Created pod: e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k 63m Normal ScalingReplicaSet deployment/e2e-worker-w2-c79cb5-openshift-default Scaled up replica set e2e-worker-w2-c79cb5-openshift-default-5464c54587 from 0 to 1 63m Normal ScalingReplicaSet deployment/e2e-worker-w5-9b51e1-openshift-default Scaled up replica set e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf from 0 to 1 63m Normal ScalingReplicaSet deployment/e2e-worker-w1-80c366-openshift-default Scaled up replica set e2e-worker-w1-80c366-openshift-default-666b7db455 from 0 to 1 63m Normal AddedInterface pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k Add eth0 [10.133.0.40/23] from ovn-kubernetes 63m Normal ScalingReplicaSet deployment/e2e-worker-w3-bc009f-openshift-default Scaled up replica set e2e-worker-w3-bc009f-openshift-default-645d5cbf67 from 0 to 1 63m Normal AddedInterface pod/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6 Add eth0 [10.133.0.41/23] from ovn-kubernetes 63m Normal AddedInterface pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf Add eth0 [10.134.0.47/23] from ovn-kubernetes 63m Normal SuccessfulCreate replicaset/e2e-worker-w1-80c366-openshift-default-666b7db455 Created pod: e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6 63m Normal AddedInterface pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd Add eth0 [10.134.0.48/23] from ovn-kubernetes 63m Normal SuccessfulCreate replicaset/e2e-worker-w3-bc009f-openshift-default-645d5cbf67 Created pod: e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6 63m Normal SuccessfulCreate replicaset/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf Created pod: e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5 63m Normal SuccessfulCreate replicaset/e2e-worker-w0-dda9f5-openshift-default-59dc47f975 Created pod: e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc 63m Normal AddedInterface pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5 Add eth0 [10.134.0.49/23] from ovn-kubernetes 63m Normal ScalingReplicaSet deployment/e2e-worker-w0-dda9f5-openshift-default Scaled up replica set e2e-worker-w0-dda9f5-openshift-default-59dc47f975 from 0 to 1 63m Normal AddedInterface pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc Add eth0 [10.133.0.42/23] from ovn-kubernetes 63m Normal AddedInterface pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6 Add eth0 [10.134.0.50/23] from ovn-kubernetes 63m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-c79cb5 Scaled up replica set payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5 from 0 to 1 63m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-c79cb5 Scaled up replica set payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f from 0 to 1 63m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f Created pod: payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx 63m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5 Error creating: pods "payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-c79cb5: serviceaccount "payload-processing-e2e-worker-w2-c79cb5" not found 63m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f Error creating: pods "payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-c79cb5: serviceaccount "payload-processing-e2e-worker-w2-c79cb5" not found 63m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5 Created pod: payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh 63m Normal Created pod/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx Container created 63m Normal Pulled pod/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh Add eth0 [10.134.0.52/23] from ovn-kubernetes 63m Normal Started pod/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx Container started 63m Normal Created pod/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh Container created 63m Normal Started pod/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh Container started 63m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx Add eth0 [10.134.0.51/23] from ovn-kubernetes 63m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-557708-57fcc65d7f Error creating: pods "payload-processing-e2e-worker-w6-557708-57fcc65d7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-557708: serviceaccount "payload-processing-e2e-worker-w6-557708" not found 63m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-557708-7f89484b55 Error creating: pods "payload-pre-processing-e2e-worker-w6-557708-7f89484b55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-557708: serviceaccount "payload-processing-e2e-worker-w6-557708" not found 63m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-557708 Scaled up replica set payload-pre-processing-e2e-worker-w6-557708-7f89484b55 from 0 to 1 63m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-557708 Scaled up replica set payload-processing-e2e-worker-w6-557708-57fcc65d7f from 0 to 1 63m Normal Started pod/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h Container started 63m Normal Created pod/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp Container created 63m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-557708-57fcc65d7f Created pod: payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp 63m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h Add eth0 [10.134.0.53/23] from ovn-kubernetes 63m Normal AddedInterface pod/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp Add eth0 [10.134.0.54/23] from ovn-kubernetes 63m Normal Pulled pod/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-557708-7f89484b55 Created pod: payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h 63m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal Started pod/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp Container started 63m Normal Created pod/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h Container created 63m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-80c366 Scaled up replica set payload-processing-e2e-worker-w1-80c366-58948dbb9c from 0 to 1 63m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-80c366-58948dbb9c Error creating: pods "payload-processing-e2e-worker-w1-80c366-58948dbb9c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-80c366: serviceaccount "payload-processing-e2e-worker-w1-80c366" not found 63m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-80c366 Scaled up replica set payload-pre-processing-e2e-worker-w1-80c366-78579d4fff from 0 to 1 63m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff Error creating: pods "payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-80c366: serviceaccount "payload-processing-e2e-worker-w1-80c366" not found 63m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff Created pod: payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48 63m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-80c366-58948dbb9c Created pod: payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g 63m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48 Add eth0 [10.134.0.55/23] from ovn-kubernetes 63m Normal AddedInterface pod/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g Add eth0 [10.134.0.56/23] from ovn-kubernetes 63m Normal Pulled pod/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal Created pod/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g Container created 63m Normal Started pod/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g Container started 63m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal Created pod/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48 Container created 63m Normal Started pod/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48 Container started 63m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-bc009f Scaled up replica set payload-processing-e2e-worker-w3-bc009f-66fdc54f65 from 0 to 1 63m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-bc009f-66fdc54f65 Error creating: pods "payload-processing-e2e-worker-w3-bc009f-66fdc54f65-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-bc009f: serviceaccount "payload-processing-e2e-worker-w3-bc009f" not found 63m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-bc009f Scaled up replica set payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9 from 0 to 1 63m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9 Created pod: payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b 63m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9 Error creating: pods "payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-bc009f: serviceaccount "payload-processing-e2e-worker-w3-bc009f" not found 63m Normal Created pod/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b Container created 63m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-bc009f-66fdc54f65 Created pod: payload-processing-e2e-worker-w3-bc009f-66fdc54f65-88jlx 63m Normal Started pod/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b Container started 63m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b Add eth0 [10.134.0.57/23] from ovn-kubernetes 63m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/payload-processing-e2e-worker-w3-bc009f-66fdc54f65-88jlx Add eth0 [10.132.0.50/23] from ovn-kubernetes 63m Normal Pulling pod/payload-processing-e2e-worker-w3-bc009f-66fdc54f65-88jlx Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 63m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-9b51e1 Scaled up replica set payload-processing-e2e-worker-w5-9b51e1-65b8fbbc9f from 0 to 1 63m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f Error creating: pods "payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-9b51e1: serviceaccount "payload-processing-e2e-worker-w5-9b51e1" not found 63m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f Created pod: payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4 63m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-9b51e1 Scaled up replica set payload-pre-processing-e2e-worker-w5-9b51e1-846589545f from 0 to 1 63m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-9b51e1-65b8fbbc9f Created pod: payload-processing-e2e-worker-w5-9b51e1-65b8fbbc9f-ng2zh 63m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-9b51e1-65b8fbbc9f Error creating: pods "payload-processing-e2e-worker-w5-9b51e1-65b8fbbc9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-9b51e1: serviceaccount "payload-processing-e2e-worker-w5-9b51e1" not found 63m Normal Pulling pod/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 63m Normal Created pod/payload-processing-e2e-worker-w3-bc009f-66fdc54f65-88jlx Container created 63m Normal Started pod/payload-processing-e2e-worker-w3-bc009f-66fdc54f65-88jlx Container started 63m Normal Pulling pod/payload-processing-e2e-worker-w5-9b51e1-65b8fbbc9f-ng2zh Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 63m Normal AddedInterface pod/payload-processing-e2e-worker-w5-9b51e1-65b8fbbc9f-ng2zh Add eth0 [10.132.0.51/23] from ovn-kubernetes 63m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4 Add eth0 [10.133.0.45/23] from ovn-kubernetes 63m Normal Pulled pod/payload-processing-e2e-worker-w3-bc009f-66fdc54f65-88jlx Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.836s (2.836s including waiting). Image size: 125353965 bytes. 63m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-dda9f5 Scaled up replica set payload-processing-e2e-worker-w0-dda9f5-75fcc89f49 from 0 to 1 63m Normal Pulled pod/payload-processing-e2e-worker-w5-9b51e1-65b8fbbc9f-ng2zh Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 841ms (841ms including waiting). Image size: 125353965 bytes. 63m Normal Started pod/payload-processing-e2e-worker-w5-9b51e1-65b8fbbc9f-ng2zh Container started 63m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-dda9f5 Scaled up replica set payload-pre-processing-e2e-worker-w0-dda9f5-5cb698d9c9 from 0 to 1 63m Normal Created pod/payload-processing-e2e-worker-w5-9b51e1-65b8fbbc9f-ng2zh Container created 63m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-dda9f5-75fcc89f49 Error creating: pods "payload-processing-e2e-worker-w0-dda9f5-75fcc89f49-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-dda9f5: serviceaccount "payload-processing-e2e-worker-w0-dda9f5" not found 63m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-dda9f5-5cb698d9c9 Error creating: pods "payload-pre-processing-e2e-worker-w0-dda9f5-5cb698d9c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-dda9f5: serviceaccount "payload-processing-e2e-worker-w0-dda9f5" not found 63m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-dda9f5-5cb698d9c9-hj25v Add eth0 [10.132.0.52/23] from ovn-kubernetes 63m Normal Created pod/payload-pre-processing-e2e-worker-w0-dda9f5-5cb698d9c9-hj25v Container created 63m Normal Created pod/payload-processing-e2e-worker-w0-dda9f5-75fcc89f49-bh6xw Container created 63m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-dda9f5-5cb698d9c9-hj25v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-dda9f5-5cb698d9c9 Created pod: payload-pre-processing-e2e-worker-w0-dda9f5-5cb698d9c9-hj25v 63m Normal Started pod/payload-pre-processing-e2e-worker-w0-dda9f5-5cb698d9c9-hj25v Container started 63m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-dda9f5-75fcc89f49 Created pod: payload-processing-e2e-worker-w0-dda9f5-75fcc89f49-bh6xw 63m Normal Started pod/payload-processing-e2e-worker-w0-dda9f5-75fcc89f49-bh6xw Container started 63m Normal AddedInterface pod/payload-processing-e2e-worker-w0-dda9f5-75fcc89f49-bh6xw Add eth0 [10.132.0.53/23] from ovn-kubernetes 63m Normal Pulled pod/payload-processing-e2e-worker-w0-dda9f5-75fcc89f49-bh6xw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal Started pod/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4 Container started 63m Normal Created pod/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4 Container created 63m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.846s (2.846s including waiting). Image size: 125353965 bytes. 63m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-c3e2d8 Scaled up replica set payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc from 0 to 1 63m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-c3e2d8 Scaled up replica set payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9 from 0 to 1 63m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc Error creating: pods "payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8: serviceaccount "payload-processing-e2e-worker-w4-c3e2d8" not found 63m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc Created pod: payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf 63m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9 Created pod: payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5 63m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9 Error creating: pods "payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8: serviceaccount "payload-processing-e2e-worker-w4-c3e2d8" not found 63m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal Started pod/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5 Container started 63m Normal Created pod/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf Container created 63m Normal Started pod/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf Container started 63m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf Add eth0 [10.134.0.58/23] from ovn-kubernetes 63m Normal AddedInterface pod/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5 Add eth0 [10.134.0.59/23] from ovn-kubernetes 63m Normal Pulled pod/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 63m Normal Created pod/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5 Container created 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-c3e2d8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Normal Created pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf Container created 63m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-c3e2d8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Normal Started pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf Container started 63m Normal Pulled pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-dda9f5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-dda9f5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 62m Warning ProbeError pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf Startup probe error: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf Startup probe failed: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused 62m Normal Pulled pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Pulled pod/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Created pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k Container created 62m Normal Started pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k Container started 62m Normal Started pod/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6 Container started 62m Normal Created pod/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6 Container created 62m Warning Unhealthy pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k Startup probe failed: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6 Startup probe error: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6 Startup probe failed: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k Startup probe error: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused... 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-c79cb5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-c79cb5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Normal Pulled pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Created pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6 Container created 62m Normal Started pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6 Container started 62m Warning Unhealthy pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6 Startup probe failed: Get "http://10.134.0.50:15021/healthz/ready": dial tcp 10.134.0.50:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6 Startup probe error: Get "http://10.134.0.50:15021/healthz/ready": dial tcp 10.134.0.50:15021: connect: connection refused... 62m Normal Started pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd Container started 62m Normal Created pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd Container created 62m Normal Pulled pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-c79cb5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-c79cb5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 62m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-bc009f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-bc009f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning Unhealthy pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd Startup probe failed: Get "http://10.134.0.48:15021/healthz/ready": dial tcp 10.134.0.48:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd Startup probe error: Get "http://10.134.0.48:15021/healthz/ready": dial tcp 10.134.0.48:15021: connect: connection refused... 62m Normal Created pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5 Container created 62m Normal Pulled pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Started pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5 Container started 62m Normal Pulled pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Started pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc Container started 62m Normal Created pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc Container created 62m Warning ProbeError pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5 Startup probe error: Get "http://10.134.0.49:15021/healthz/ready": dial tcp 10.134.0.49:15021: connect: connection refused... 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-557708-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning Unhealthy pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5 Startup probe failed: Get "http://10.134.0.49:15021/healthz/ready": dial tcp 10.134.0.49:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc Startup probe error: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc Startup probe failed: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused 62m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-557708-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-80c366-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-80c366-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-c3e2d8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 62m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-c3e2d8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 62m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-bc009f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-bc009f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 61m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-9b51e1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-9b51e1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-dda9f5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-dda9f5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-557708-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 61m Normal ScalingReplicaSet deployment/e2e-worker-w6-557708-openshift-default Scaled up replica set e2e-worker-w6-557708-openshift-default-bd7dbddd5 from 1 to 2 61m Normal SuccessfulCreate replicaset/e2e-worker-w6-557708-openshift-default-bd7dbddd5 Created pod: e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26 61m Normal AddedInterface pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26 Add eth0 [10.133.0.50/23] from ovn-kubernetes 61m Normal Started pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26 Container started 61m Normal Created pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26 Container created 61m Normal Pulled pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Warning ProbeError pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26 Startup probe error: Get "http://10.133.0.50:15021/healthz/ready": dial tcp 10.133.0.50:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26 Startup probe failed: Get "http://10.133.0.50:15021/healthz/ready": dial tcp 10.133.0.50:15021: connect: connection refused 59m Normal Pulled pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-msfnj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal ScalingReplicaSet deployment/e2e-worker-w6-557708-openshift-default Scaled up replica set e2e-worker-w6-557708-openshift-default-bd7dbddd5 from 2 to 3 59m Normal AddedInterface pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-msfnj Add eth0 [10.132.0.65/23] from ovn-kubernetes 59m Normal Created pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-msfnj Container created 59m Normal Started pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-msfnj Container started 59m Normal SuccessfulCreate replicaset/e2e-worker-w6-557708-openshift-default-bd7dbddd5 Created pod: e2e-worker-w6-557708-openshift-default-bd7dbddd5-msfnj 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-557708-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 59m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-74854d8ccf Deleted pod: maas-default-gateway-openshift-default-74854d8ccf-8xwmx 59m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-74854d8ccf from 9 to 6 59m Normal Killing pod/maas-default-gateway-openshift-default-74854d8ccf-8xwmx Stopping container istio-proxy 59m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-74854d8ccf Deleted pod: maas-default-gateway-openshift-default-74854d8ccf-p8khq 59m Normal Killing pod/maas-default-gateway-openshift-default-74854d8ccf-p8khq Stopping container istio-proxy 59m Normal Killing pod/maas-default-gateway-openshift-default-74854d8ccf-rz8s5 Stopping container istio-proxy 59m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-74854d8ccf Deleted pod: maas-default-gateway-openshift-default-74854d8ccf-rz8s5 59m Warning Unhealthy pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-msfnj Startup probe failed: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-msfnj Startup probe error: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused... 59m Normal AddedInterface pod/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2 Add eth0 [10.134.0.64/23] from ovn-kubernetes 59m Normal SuccessfulCreate replicaset/e2e-worker-w1-80c366-openshift-default-666b7db455 Created pod: e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-80c366-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 59m Normal ScalingReplicaSet deployment/e2e-worker-w1-80c366-openshift-default Scaled up replica set e2e-worker-w1-80c366-openshift-default-666b7db455 from 1 to 2 59m Normal SuccessfulCreate replicaset/e2e-worker-w0-dda9f5-openshift-default-59dc47f975 Created pod: e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv 59m Normal Started pod/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2 Container started 59m Normal AddedInterface pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv Add eth0 [10.134.0.65/23] from ovn-kubernetes 59m Normal Pulled pod/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2 Container created 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-dda9f5-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 59m Normal ScalingReplicaSet deployment/e2e-worker-w0-dda9f5-openshift-default Scaled up replica set e2e-worker-w0-dda9f5-openshift-default-59dc47f975 from 1 to 2 59m Normal Created pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv Container created 59m Normal Started pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv Container started 59m Normal Pulled pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Warning Unhealthy pod/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2 Startup probe failed: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2 Startup probe error: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused... 59m Warning ProbeError pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv Startup probe error: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv Startup probe failed: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused 58m Normal SuccessfulCreate replicaset/e2e-worker-w3-bc009f-openshift-default-645d5cbf67 Created pod: e2e-worker-w3-bc009f-openshift-default-645d5cbf67-lg5tj 58m Normal ScalingReplicaSet deployment/e2e-worker-w3-bc009f-openshift-default Scaled up replica set e2e-worker-w3-bc009f-openshift-default-645d5cbf67 from 1 to 2 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-bc009f-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 58m Normal Started pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-lg5tj Container started 58m Normal AddedInterface pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-lg5tj Add eth0 [10.132.0.72/23] from ovn-kubernetes 58m Normal Pulled pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-lg5tj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-lg5tj Container created 58m Warning ProbeError pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-lg5tj Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-lg5tj Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused 58m Normal SuccessfulCreate replicaset/e2e-worker-w6-557708-openshift-default-bd7dbddd5 Created pod: e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj 58m Normal ScalingReplicaSet deployment/e2e-worker-w6-557708-openshift-default Scaled up replica set e2e-worker-w6-557708-openshift-default-bd7dbddd5 from 3 to 5 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-557708-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 58m Normal SuccessfulCreate replicaset/e2e-worker-w6-557708-openshift-default-bd7dbddd5 Created pod: e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94 58m Normal AddedInterface pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94 Add eth0 [10.133.0.55/23] from ovn-kubernetes 58m Normal Pulled pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Started pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj Container started 58m Normal Created pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj Container created 58m Normal Pulled pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj Add eth0 [10.134.0.66/23] from ovn-kubernetes 58m Normal Created pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94 Container created 58m Normal Started pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94 Container started 58m Normal ScalingReplicaSet deployment/e2e-worker-w5-9b51e1-openshift-default Scaled up replica set e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf from 1 to 2 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-9b51e1-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 58m Normal SuccessfulCreate replicaset/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf Created pod: e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9 58m Normal AddedInterface pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9 Add eth0 [10.133.0.56/23] from ovn-kubernetes 58m Normal Started pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9 Container started 58m Normal Created pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9 Container created 58m Normal Pulled pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Warning ProbeError pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94 Startup probe error: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94 Startup probe failed: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj Startup probe error: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj Startup probe failed: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94 Startup probe error: Get "http://10.133.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning Unhealthy pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94 Startup probe failed: Get "http://10.133.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning Unhealthy pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9 Startup probe failed: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9 Startup probe error: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused... 57m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-74854d8ccf Deleted pod: maas-default-gateway-openshift-default-74854d8ccf-mglq4 57m Normal Killing pod/maas-default-gateway-openshift-default-74854d8ccf-mglq4 Stopping container istio-proxy 57m Normal ScalingReplicaSet deployment/e2e-worker-w1-80c366-openshift-default Scaled up replica set e2e-worker-w1-80c366-openshift-default-666b7db455 from 2 to 3 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-80c366-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 57m Normal SuccessfulCreate replicaset/e2e-worker-w1-80c366-openshift-default-666b7db455 Created pod: e2e-worker-w1-80c366-openshift-default-666b7db455-l6mk7 57m Normal Created pod/e2e-worker-w1-80c366-openshift-default-666b7db455-l6mk7 Container created 57m Normal AddedInterface pod/e2e-worker-w1-80c366-openshift-default-666b7db455-l6mk7 Add eth0 [10.132.0.83/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w1-80c366-openshift-default-666b7db455-l6mk7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Started pod/e2e-worker-w1-80c366-openshift-default-666b7db455-l6mk7 Container started 57m Normal Killing pod/maas-default-gateway-openshift-default-74854d8ccf-qxxgq Stopping container istio-proxy 57m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-74854d8ccf from 5 to 4 57m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-74854d8ccf Deleted pod: maas-default-gateway-openshift-default-74854d8ccf-qxxgq 57m Warning Unhealthy pod/e2e-worker-w1-80c366-openshift-default-666b7db455-l6mk7 Startup probe failed: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w1-80c366-openshift-default-666b7db455-l6mk7 Startup probe error: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused... 57m Normal ScalingReplicaSet deployment/e2e-worker-w6-557708-openshift-default Scaled up replica set e2e-worker-w6-557708-openshift-default-bd7dbddd5 from 5 to 6 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-557708-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 57m Normal SuccessfulCreate replicaset/e2e-worker-w6-557708-openshift-default-bd7dbddd5 Created pod: e2e-worker-w6-557708-openshift-default-bd7dbddd5-fj8mc 57m Normal ScalingReplicaSet deployment/e2e-worker-w2-c79cb5-openshift-default Scaled up replica set e2e-worker-w2-c79cb5-openshift-default-5464c54587 from 1 to 2 57m Normal AddedInterface pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-fj8mc Add eth0 [10.132.0.86/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-fj8mc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-c79cb5-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 57m Normal SuccessfulCreate replicaset/e2e-worker-w2-c79cb5-openshift-default-5464c54587 Created pod: e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5 57m Normal Created pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-fj8mc Container created 57m Normal ScalingReplicaSet deployment/e2e-worker-w5-9b51e1-openshift-default Scaled up replica set e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf from 2 to 4 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-9b51e1-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 57m Normal Created pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5 Container created 57m Normal SuccessfulCreate replicaset/e2e-worker-w3-bc009f-openshift-default-645d5cbf67 Created pod: e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz 57m Normal SuccessfulCreate replicaset/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf Created pod: e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26 57m Normal SuccessfulCreate replicaset/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf Created pod: e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-fkzmf 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-80c366-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 57m Normal AddedInterface pod/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk Add eth0 [10.134.0.69/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk Container created 57m Normal Started pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-fj8mc Container started 57m Normal Started pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5 Container started 57m Normal ScalingReplicaSet deployment/e2e-worker-w1-80c366-openshift-default Scaled up replica set e2e-worker-w1-80c366-openshift-default-666b7db455 from 3 to 4 57m Normal SuccessfulCreate replicaset/e2e-worker-w3-bc009f-openshift-default-645d5cbf67 Created pod: e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt 57m Normal Started pod/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk Container started 57m Normal ScalingReplicaSet deployment/e2e-worker-w3-bc009f-openshift-default Scaled up replica set e2e-worker-w3-bc009f-openshift-default-645d5cbf67 from 2 to 4 57m Normal Pulled pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-bc009f-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 57m Normal AddedInterface pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5 Add eth0 [10.134.0.68/23] from ovn-kubernetes 57m Normal SuccessfulCreate replicaset/e2e-worker-w1-80c366-openshift-default-666b7db455 Created pod: e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk 57m Normal Created pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt Container created 57m Normal AddedInterface pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-fkzmf Add eth0 [10.132.0.87/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Started pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz Container started 57m Normal Started pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt Container started 57m Normal Created pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz Container created 57m Normal Pulled pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz Add eth0 [10.134.0.70/23] from ovn-kubernetes 57m Normal AddedInterface pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26 Add eth0 [10.133.0.57/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26 Container created 57m Normal Started pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26 Container started 57m Normal AddedInterface pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt Add eth0 [10.133.0.58/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-fkzmf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Started pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-fkzmf Container started 57m Normal Created pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-fkzmf Container created 57m Warning ProbeError pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-fj8mc Startup probe error: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-fj8mc Startup probe failed: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-fkzmf Startup probe error: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk Startup probe error: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk Startup probe failed: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-fkzmf Startup probe failed: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26 Startup probe failed: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26 Startup probe error: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5 Startup probe error: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5 Startup probe failed: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz Startup probe error: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz Startup probe failed: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt Startup probe failed: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt Startup probe error: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused... 57m Normal Killing pod/maas-default-gateway-openshift-default-74854d8ccf-v4zqk Stopping container istio-proxy 57m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-74854d8ccf Deleted pod: maas-default-gateway-openshift-default-74854d8ccf-v4zqk 57m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-74854d8ccf from 4 to 3 57m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: All metrics below target 56m Normal ScalingReplicaSet deployment/e2e-worker-w4-c3e2d8-openshift-default Scaled up replica set e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c from 1 to 2 56m Normal SuccessfulCreate replicaset/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c Created pod: e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-c3e2d8-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 56m Normal Created pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k Container created 56m Normal Started pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k Container started 56m Normal Pulled pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k Add eth0 [10.133.0.60/23] from ovn-kubernetes 56m Normal SuccessfulCreate replicaset/e2e-worker-w0-dda9f5-openshift-default-59dc47f975 Created pod: e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n 56m Normal SuccessfulCreate replicaset/e2e-worker-w0-dda9f5-openshift-default-59dc47f975 Created pod: e2e-worker-w0-dda9f5-openshift-default-59dc47f975-rf429 56m Normal ScalingReplicaSet deployment/e2e-worker-w0-dda9f5-openshift-default Scaled up replica set e2e-worker-w0-dda9f5-openshift-default-59dc47f975 from 2 to 4 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-dda9f5-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 56m Normal AddedInterface pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n Add eth0 [10.134.0.72/23] from ovn-kubernetes 56m Normal Pulled pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-rf429 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n Container started 56m Normal Started pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-rf429 Container started 56m Normal AddedInterface pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-rf429 Add eth0 [10.132.0.90/23] from ovn-kubernetes 56m Normal Created pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-rf429 Container created 56m Normal Pulled pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n Container created 56m Warning Unhealthy pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k Startup probe failed: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k Startup probe error: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-rf429 Startup probe failed: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused 56m Warning Unhealthy pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n Startup probe failed: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused... 56m Warning ProbeError pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-rf429 Startup probe error: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused... 56m Normal SuccessfulCreate replicaset/e2e-worker-w2-c79cb5-openshift-default-5464c54587 Created pod: e2e-worker-w2-c79cb5-openshift-default-5464c54587-4s94l 56m Normal AddedInterface pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-4s94l Add eth0 [10.132.0.91/23] from ovn-kubernetes 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-c79cb5-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 56m Normal ScalingReplicaSet deployment/e2e-worker-w2-c79cb5-openshift-default Scaled up replica set e2e-worker-w2-c79cb5-openshift-default-5464c54587 from 2 to 3 56m Normal Pulled pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-4s94l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-4s94l Container created 56m Normal Started pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-4s94l Container started 56m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-74854d8ccf Deleted pod: maas-default-gateway-openshift-default-74854d8ccf-4k5dm 56m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 2; reason: All metrics below target 56m Normal Killing pod/maas-default-gateway-openshift-default-74854d8ccf-4k5dm Stopping container istio-proxy 56m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-74854d8ccf from 3 to 2 56m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-4k5dm Readiness probe failed: HTTP probe failed with statuscode: 503 56m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-4k5dm Readiness probe error: HTTP probe failed with statuscode: 503... 56m Warning ProbeError pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-4s94l Startup probe error: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-4s94l Startup probe failed: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused 56m Normal SuccessfulCreate replicaset/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf Created pod: e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w 56m Normal ScalingReplicaSet deployment/e2e-worker-w5-9b51e1-openshift-default Scaled up replica set e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf from 4 to 6 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-9b51e1-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 56m Normal Pulled pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x86qc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Pulled pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w Add eth0 [10.134.0.73/23] from ovn-kubernetes 56m Normal Started pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w Container started 56m Normal SuccessfulCreate replicaset/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf Created pod: e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x86qc 56m Normal Started pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x86qc Container started 56m Normal Created pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x86qc Container created 56m Normal AddedInterface pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x86qc Add eth0 [10.132.0.92/23] from ovn-kubernetes 56m Normal Created pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w Container created 56m Warning ProbeError pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x86qc Startup probe error: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused... 56m Warning ProbeError pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w Startup probe error: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x86qc Startup probe failed: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused 56m Warning Unhealthy pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w Startup probe failed: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused 55m Normal ScalingReplicaSet deployment/e2e-worker-w6-557708-openshift-default Scaled up replica set e2e-worker-w6-557708-openshift-default-bd7dbddd5 from 6 to 9 55m Normal SuccessfulCreate replicaset/e2e-worker-w6-557708-openshift-default-bd7dbddd5 Created pod: e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r 55m Normal SuccessfulCreate replicaset/e2e-worker-w6-557708-openshift-default-bd7dbddd5 Created pod: e2e-worker-w6-557708-openshift-default-bd7dbddd5-tmlmf 55m Normal SuccessfulCreate replicaset/e2e-worker-w6-557708-openshift-default-bd7dbddd5 Created pod: e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-557708-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 55m Normal AddedInterface pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77 Add eth0 [10.133.0.63/23] from ovn-kubernetes 55m Normal Pulled pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77 Container created 55m Normal Pulled pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-tmlmf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-tmlmf Add eth0 [10.132.0.93/23] from ovn-kubernetes 55m Normal Pulled pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-tmlmf Container created 55m Normal Started pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-tmlmf Container started 55m Normal Started pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77 Container started 55m Normal Started pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r Container started 55m Normal Created pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r Container created 55m Normal AddedInterface pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r Add eth0 [10.134.0.75/23] from ovn-kubernetes 55m Warning Unhealthy pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-tmlmf Startup probe failed: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused 55m Warning Unhealthy pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r Startup probe failed: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused 55m Warning Unhealthy pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77 Startup probe failed: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77 Startup probe error: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused... 55m Warning ProbeError pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r Startup probe error: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused... 55m Warning ProbeError pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-tmlmf Startup probe error: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused... 54m Normal SuccessfulCreate replicaset/e2e-worker-w3-bc009f-openshift-default-645d5cbf67 Created pod: e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4 54m Normal SuccessfulCreate replicaset/e2e-worker-w3-bc009f-openshift-default-645d5cbf67 Created pod: e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-bc009f-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 54m Normal ScalingReplicaSet deployment/e2e-worker-w3-bc009f-openshift-default Scaled up replica set e2e-worker-w3-bc009f-openshift-default-645d5cbf67 from 4 to 7 54m Normal SuccessfulCreate replicaset/e2e-worker-w3-bc009f-openshift-default-645d5cbf67 Created pod: e2e-worker-w3-bc009f-openshift-default-645d5cbf67-tzj7w 54m Normal Pulled pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-tzj7w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Pulled pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Started pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh Container started 54m Normal Created pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-tzj7w Container created 54m Normal Started pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-tzj7w Container started 54m Normal AddedInterface pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-tzj7w Add eth0 [10.132.0.95/23] from ovn-kubernetes 54m Normal AddedInterface pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4 Add eth0 [10.133.0.65/23] from ovn-kubernetes 54m Normal Created pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4 Container created 54m Normal Started pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4 Container started 54m Normal Created pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh Container created 54m Normal Pulled pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh Add eth0 [10.134.0.76/23] from ovn-kubernetes 54m Warning Unhealthy pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-tzj7w Startup probe failed: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused 54m Warning Unhealthy pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4 Startup probe failed: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused 54m Warning Unhealthy pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh Startup probe failed: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh Startup probe error: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused... 54m Warning ProbeError pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-tzj7w Startup probe error: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused... 54m Warning ProbeError pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4 Startup probe error: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused... 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-c3e2d8-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 54m Normal ScalingReplicaSet deployment/e2e-worker-w4-c3e2d8-openshift-default Scaled up replica set e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c from 2 to 4 54m Normal SuccessfulCreate replicaset/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c Created pod: e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wn9kx 54m Normal SuccessfulCreate replicaset/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c Created pod: e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8 54m Normal AddedInterface pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wn9kx Add eth0 [10.132.0.97/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8 Add eth0 [10.134.0.78/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wn9kx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8 Container created 54m Normal Started pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8 Container started 54m Normal Created pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wn9kx Container created 54m Normal Started pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wn9kx Container started 54m Warning Unhealthy pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wn9kx Startup probe failed: Get "http://10.132.0.97:15021/healthz/ready": dial tcp 10.132.0.97:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wn9kx Startup probe error: Get "http://10.132.0.97:15021/healthz/ready": dial tcp 10.132.0.97:15021: connect: connection refused... 54m Warning ProbeError pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8 Startup probe error: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8 Startup probe failed: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused 54m Normal ScalingReplicaSet deployment/e2e-worker-w0-dda9f5-openshift-default Scaled up replica set e2e-worker-w0-dda9f5-openshift-default-59dc47f975 from 4 to 7 54m Normal AddedInterface pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs Add eth0 [10.133.0.67/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-hfdvc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-dda9f5-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 54m Normal Pulled pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr Add eth0 [10.134.0.79/23] from ovn-kubernetes 54m Normal AddedInterface pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-hfdvc Add eth0 [10.132.0.99/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal SuccessfulCreate replicaset/e2e-worker-w0-dda9f5-openshift-default-59dc47f975 Created pod: e2e-worker-w0-dda9f5-openshift-default-59dc47f975-hfdvc 54m Normal SuccessfulCreate replicaset/e2e-worker-w0-dda9f5-openshift-default-59dc47f975 Created pod: e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs 54m Normal SuccessfulCreate replicaset/e2e-worker-w0-dda9f5-openshift-default-59dc47f975 Created pod: e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr 54m Normal Created pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr Container created 54m Normal Started pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-hfdvc Container started 54m Normal Created pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs Container created 54m Normal Created pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-hfdvc Container created 54m Normal Started pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs Container started 54m Normal Started pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr Container started 54m Warning Unhealthy pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-hfdvc Startup probe failed: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused 54m Warning Unhealthy pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr Startup probe failed: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-hfdvc Startup probe error: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs Startup probe failed: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr Startup probe error: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused... 54m Warning ProbeError pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs Startup probe error: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused... 54m Normal SuccessfulCreate replicaset/e2e-worker-w3-bc009f-openshift-default-645d5cbf67 Created pod: e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-bc009f-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulCreate replicaset/e2e-worker-w3-bc009f-openshift-default-645d5cbf67 Created pod: e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ldkbp 54m Normal SuccessfulCreate replicaset/e2e-worker-w3-bc009f-openshift-default-645d5cbf67 (combined from similar events): Created pod: e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s 54m Normal ScalingReplicaSet deployment/e2e-worker-w3-bc009f-openshift-default Scaled up replica set e2e-worker-w3-bc009f-openshift-default-645d5cbf67 from 7 to 10 54m Normal AddedInterface pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh Add eth0 [10.133.0.69/23] from ovn-kubernetes 54m Normal AddedInterface pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s Add eth0 [10.134.0.80/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ldkbp Add eth0 [10.132.0.101/23] from ovn-kubernetes 54m Normal Started pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh Container started 54m Normal Created pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh Container created 54m Normal Created pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s Container created 54m Normal Pulled pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ldkbp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Pulled pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ldkbp Container created 54m Normal Started pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ldkbp Container started 54m Warning Unhealthy pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26 Readiness probe failed: Get "http://10.133.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning ProbeError pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26 Readiness probe error: Get "http://10.133.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Normal Started pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s Container started 54m Warning Unhealthy pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ldkbp Startup probe failed: Get "http://10.132.0.101:15021/healthz/ready": dial tcp 10.132.0.101:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ldkbp Startup probe error: Get "http://10.132.0.101:15021/healthz/ready": dial tcp 10.132.0.101:15021: connect: connection refused... 54m Warning ProbeError pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh Startup probe error: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh Startup probe failed: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s Startup probe error: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s Startup probe failed: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused 53m Normal ScalingReplicaSet deployment/e2e-worker-w6-557708-openshift-default Scaled up replica set e2e-worker-w6-557708-openshift-default-bd7dbddd5 from 9 to 10 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-557708-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 53m Normal SuccessfulCreate replicaset/e2e-worker-w6-557708-openshift-default-bd7dbddd5 (combined from similar events): Created pod: e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd 53m Normal SuccessfulCreate replicaset/e2e-worker-w2-c79cb5-openshift-default-5464c54587 Created pod: e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm 53m Normal SuccessfulCreate replicaset/e2e-worker-w2-c79cb5-openshift-default-5464c54587 Created pod: e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4 53m Normal Started pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd Container started 53m Normal AddedInterface pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd Add eth0 [10.134.0.82/23] from ovn-kubernetes 53m Normal ScalingReplicaSet deployment/e2e-worker-w2-c79cb5-openshift-default Scaled up replica set e2e-worker-w2-c79cb5-openshift-default-5464c54587 from 3 to 5 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-c79cb5-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 53m Normal Created pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd Container created 53m Normal Pulled pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf Created pod: e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6 53m Normal ScalingReplicaSet deployment/e2e-worker-w5-9b51e1-openshift-default Scaled up replica set e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf from 6 to 10 53m Normal Pulled pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm Add eth0 [10.133.0.70/23] from ovn-kubernetes 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-9b51e1-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 53m Normal Created pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4 Container created 53m Normal AddedInterface pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl Add eth0 [10.134.0.84/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4 Container started 53m Normal AddedInterface pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4 Add eth0 [10.134.0.83/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf (combined from similar events): Created pod: e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r 53m Normal Pulled pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r Add eth0 [10.134.0.85/23] from ovn-kubernetes 53m Normal Created pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm Container created 53m Normal AddedInterface pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6 Add eth0 [10.133.0.71/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-khbxp Add eth0 [10.132.0.106/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf Created pod: e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-khbxp 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf Created pod: e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl 53m Normal Started pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm Container started 53m Normal Created pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r Container created 53m Normal Started pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl Container started 53m Normal Started pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6 Container started 53m Normal Created pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6 Container created 53m Normal Created pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl Container created 53m Normal Pulled pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-khbxp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r Container started 53m Normal Created pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-khbxp Container created 53m Normal Started pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-khbxp Container started 53m Warning Unhealthy pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm Startup probe failed: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm Startup probe error: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6 Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-khbxp Startup probe error: Get "http://10.132.0.106:15021/healthz/ready": dial tcp 10.132.0.106:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-khbxp Startup probe failed: Get "http://10.132.0.106:15021/healthz/ready": dial tcp 10.132.0.106:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd Startup probe failed: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6 Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd Startup probe error: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4 Startup probe failed: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4 Startup probe error: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd Startup probe error: Get "http://10.134.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 53m Warning Unhealthy pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd Startup probe failed: Get "http://10.134.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 53m Warning Unhealthy pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl Startup probe failed: Get "http://10.134.0.84:15021/healthz/ready": dial tcp 10.134.0.84:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl Startup probe error: Get "http://10.134.0.84:15021/healthz/ready": dial tcp 10.134.0.84:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ldkbp Readiness probe failed: Get "http://10.132.0.101:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 53m Warning ProbeError pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ldkbp Readiness probe error: Get "http://10.132.0.101:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 53m Warning Unhealthy pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r Startup probe failed: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r Startup probe error: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4 Startup probe failed: Get "http://10.134.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 53m Warning ProbeError pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4 Startup probe error: Get "http://10.134.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 53m Warning ProbeError pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl Startup probe error: Get "http://10.134.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 53m Warning Unhealthy pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl Startup probe failed: Get "http://10.134.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 53m Warning ProbeError pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r Startup probe error: Get "http://10.134.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 53m Warning Unhealthy pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r Startup probe failed: Get "http://10.134.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 53m Normal SuccessfulCreate replicaset/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c Created pod: e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-vqsb5 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-c3e2d8-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w4-c3e2d8-openshift-default Scaled up replica set e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c from 4 to 6 53m Normal SuccessfulCreate replicaset/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c Created pod: e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv 53m Normal Pulled pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal SuccessfulCreate replicaset/e2e-worker-w1-80c366-openshift-default-666b7db455 Created pod: e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-80c366-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w1-80c366-openshift-default Scaled up replica set e2e-worker-w1-80c366-openshift-default-666b7db455 from 4 to 7 53m Normal AddedInterface pod/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr Add eth0 [10.134.0.87/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-vqsb5 Add eth0 [10.132.0.111/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-vqsb5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-vqsb5 Container created 53m Normal Started pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-vqsb5 Container started 53m Normal Created pod/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk Container created 53m Normal Pulled pod/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr Container created 53m Normal Started pod/e2e-worker-w1-80c366-openshift-default-666b7db455-4kk96 Container started 53m Normal SuccessfulCreate replicaset/e2e-worker-w1-80c366-openshift-default-666b7db455 Created pod: e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr 53m Normal AddedInterface pod/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk Add eth0 [10.133.0.73/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal SuccessfulCreate replicaset/e2e-worker-w0-dda9f5-openshift-default-59dc47f975 (combined from similar events): Created pod: e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p 53m Normal SuccessfulCreate replicaset/e2e-worker-w1-80c366-openshift-default-666b7db455 Created pod: e2e-worker-w1-80c366-openshift-default-666b7db455-4kk96 53m Normal Started pod/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr Container started 53m Normal Created pod/e2e-worker-w1-80c366-openshift-default-666b7db455-4kk96 Container created 53m Normal SuccessfulCreate replicaset/e2e-worker-w0-dda9f5-openshift-default-59dc47f975 Created pod: e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh 53m Normal SuccessfulCreate replicaset/e2e-worker-w0-dda9f5-openshift-default-59dc47f975 Created pod: e2e-worker-w0-dda9f5-openshift-default-59dc47f975-5b9gk 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-dda9f5-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 53m Normal Pulled pod/e2e-worker-w1-80c366-openshift-default-666b7db455-4kk96 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal ScalingReplicaSet deployment/e2e-worker-w0-dda9f5-openshift-default Scaled up replica set e2e-worker-w0-dda9f5-openshift-default-59dc47f975 from 7 to 10 53m Normal Started pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv Container started 53m Normal Started pod/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk Container started 53m Normal Created pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv Container created 53m Normal AddedInterface pod/e2e-worker-w1-80c366-openshift-default-666b7db455-4kk96 Add eth0 [10.132.0.113/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv Add eth0 [10.133.0.72/23] from ovn-kubernetes 53m Normal Created pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-5b9gk Container created 53m Normal AddedInterface pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p Add eth0 [10.134.0.88/23] from ovn-kubernetes 53m Normal Created pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh Container created 53m Normal Pulled pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Pulled pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-5b9gk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh Add eth0 [10.133.0.74/23] from ovn-kubernetes 53m Normal Created pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p Container created 53m Normal Started pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh Container started 53m Normal AddedInterface pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-5b9gk Add eth0 [10.132.0.114/23] from ovn-kubernetes 53m Normal Started pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p Container started 53m Normal Pulled pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-5b9gk Container started 53m Warning ProbeError pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv Startup probe error: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv Startup probe failed: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w1-80c366-openshift-default-666b7db455-4kk96 Startup probe error: Get "http://10.132.0.113:15021/healthz/ready": dial tcp 10.132.0.113:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-vqsb5 Startup probe failed: Get "http://10.132.0.111:15021/healthz/ready": dial tcp 10.132.0.111:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr Startup probe error: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-vqsb5 Startup probe error: Get "http://10.132.0.111:15021/healthz/ready": dial tcp 10.132.0.111:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w1-80c366-openshift-default-666b7db455-4kk96 Startup probe failed: Get "http://10.132.0.113:15021/healthz/ready": dial tcp 10.132.0.113:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk Startup probe failed: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr Startup probe failed: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk Startup probe error: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p Startup probe error: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p Startup probe failed: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-5b9gk Startup probe error: Get "http://10.132.0.114:15021/healthz/ready": dial tcp 10.132.0.114:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-5b9gk Startup probe failed: Get "http://10.132.0.114:15021/healthz/ready": dial tcp 10.132.0.114:15021: connect: connection refused 52m Normal ScalingReplicaSet deployment/e2e-worker-w4-c3e2d8-openshift-default Scaled up replica set e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c from 6 to 10 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-c3e2d8-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c (combined from similar events): Created pod: e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c Created pod: e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c Created pod: e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-xb49c 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c Created pod: e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx 52m Normal SuccessfulCreate replicaset/e2e-worker-w2-c79cb5-openshift-default-5464c54587 Created pod: e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq 52m Normal ScalingReplicaSet deployment/e2e-worker-w2-c79cb5-openshift-default Scaled up replica set e2e-worker-w2-c79cb5-openshift-default-5464c54587 from 5 to 10 52m Normal AddedInterface pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 Add eth0 [10.134.0.95/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92 Add eth0 [10.133.0.75/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 Container created 52m Normal Started pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 Container started 52m Normal AddedInterface pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw Add eth0 [10.134.0.96/23] from ovn-kubernetes 52m Normal AddedInterface pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-xzvzb Add eth0 [10.132.0.124/23] from ovn-kubernetes 52m Normal SuccessfulCreate replicaset/e2e-worker-w2-c79cb5-openshift-default-5464c54587 Created pod: e2e-worker-w2-c79cb5-openshift-default-5464c54587-bf95b 52m Normal SuccessfulCreate replicaset/e2e-worker-w2-c79cb5-openshift-default-5464c54587 Created pod: e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw 52m Normal AddedInterface pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-xb49c Add eth0 [10.132.0.122/23] from ovn-kubernetes 52m Normal AddedInterface pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx Add eth0 [10.134.0.94/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx Container created 52m Normal Pulled pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-xb49c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-xb49c Container created 52m Normal Started pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-xb49c Container started 52m Normal Started pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx Container started 52m Normal SuccessfulCreate replicaset/e2e-worker-w2-c79cb5-openshift-default-5464c54587 Created pod: e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj 52m Normal AddedInterface pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq Add eth0 [10.134.0.97/23] from ovn-kubernetes 52m Normal SuccessfulCreate replicaset/e2e-worker-w2-c79cb5-openshift-default-5464c54587 (combined from similar events): Created pod: e2e-worker-w2-c79cb5-openshift-default-5464c54587-xzvzb 52m Normal Pulled pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-bf95b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-bf95b Add eth0 [10.132.0.123/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-c79cb5-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 52m Normal Started pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-xzvzb Container started 52m Normal Created pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj Container created 52m Normal Started pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq Container started 52m Normal Pulled pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq Container created 52m Normal Pulled pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-bf95b Container created 52m Normal Started pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw Container started 52m Normal Created pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-xzvzb Container created 52m Normal Created pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw Container created 52m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 1; reason: All metrics below target 52m Normal AddedInterface pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj Add eth0 [10.133.0.76/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-bf95b Container started 52m Normal Killing pod/maas-default-gateway-openshift-default-74854d8ccf-jhtzt Stopping container istio-proxy 52m Normal Pulled pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-xzvzb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-74854d8ccf Deleted pod: maas-default-gateway-openshift-default-74854d8ccf-jhtzt 52m Normal Started pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj Container started 52m Normal Started pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92 Container started 52m Normal Created pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92 Container created 52m Warning ProbeError pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-fj8mc Readiness probe error: Get "http://10.132.0.86:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94 Readiness probe failed: Get "http://10.133.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning Unhealthy pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-fj8mc Readiness probe failed: Get "http://10.132.0.86:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94 Readiness probe error: Get "http://10.133.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-tmlmf Readiness probe error: Get "http://10.132.0.93:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r Readiness probe failed: Get "http://10.134.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv Readiness probe error: Get "http://10.134.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-lg5tj Readiness probe error: Get "http://10.132.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77 Readiness probe error: Get "http://10.133.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-lg5tj Readiness probe failed: Get "http://10.132.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning Unhealthy pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77 Readiness probe failed: Get "http://10.133.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning Unhealthy pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv Readiness probe failed: Get "http://10.134.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning Unhealthy pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-tmlmf Readiness probe failed: Get "http://10.132.0.93:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r Readiness probe error: Get "http://10.134.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt Readiness probe failed: Get "http://10.133.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-rf429 Readiness probe error: Get "http://10.132.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n Readiness probe error: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt Readiness probe error: Get "http://10.133.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-rf429 Readiness probe failed: Get "http://10.132.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning Unhealthy pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n Readiness probe failed: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-jhtzt Readiness probe failed: Get "http://10.133.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh Readiness probe error: Get "http://10.134.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr Readiness probe error: Get "http://10.134.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh Readiness probe failed: Get "http://10.134.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-jhtzt Readiness probe error: Get "http://10.133.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr Readiness probe failed: Get "http://10.134.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj Startup probe error: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj Startup probe failed: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92 Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92 Startup probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq Startup probe failed: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw Startup probe failed: Get "http://10.134.0.96:15021/healthz/ready": dial tcp 10.134.0.96:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-xb49c Startup probe failed: Get "http://10.132.0.122:15021/healthz/ready": dial tcp 10.132.0.122:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx Startup probe failed: Get "http://10.134.0.94:15021/healthz/ready": dial tcp 10.134.0.94:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-xzvzb Startup probe failed: Get "http://10.132.0.124:15021/healthz/ready": dial tcp 10.132.0.124:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-bf95b Startup probe failed: Get "http://10.132.0.123:15021/healthz/ready": dial tcp 10.132.0.123:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 Startup probe failed: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw Startup probe error: Get "http://10.134.0.96:15021/healthz/ready": dial tcp 10.134.0.96:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-xzvzb Startup probe error: Get "http://10.132.0.124:15021/healthz/ready": dial tcp 10.132.0.124:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx Startup probe error: Get "http://10.134.0.94:15021/healthz/ready": dial tcp 10.134.0.94:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 Startup probe error: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq Startup probe error: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-bf95b Startup probe error: Get "http://10.132.0.123:15021/healthz/ready": dial tcp 10.132.0.123:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-xb49c Startup probe error: Get "http://10.132.0.122:15021/healthz/ready": dial tcp 10.132.0.122:15021: connect: connection refused... 51m Normal SuccessfulCreate replicaset/e2e-ait-f05b6443-openshift-default-d8cb8644f Created pod: e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x 51m Normal ScalingReplicaSet deployment/e2e-ait-f05b6443-openshift-default Scaled up replica set e2e-ait-f05b6443-openshift-default-d8cb8644f from 0 to 1 51m Normal AddedLabel gateway/e2e-ait-f05b6443 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-f05b6443 51m Normal AddedInterface pod/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x Add eth0 [10.133.0.77/23] from ovn-kubernetes 51m Warning Unhealthy pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wn9kx Readiness probe failed: Get "http://10.132.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wn9kx Readiness probe error: Get "http://10.132.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm Readiness probe failed: Get "http://10.133.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning Unhealthy pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8 Readiness probe failed: Get "http://10.134.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8 Readiness probe error: Get "http://10.134.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning ProbeError pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm Readiness probe error: Get "http://10.133.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6 Readiness probe failed: Get "http://10.133.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6 Readiness probe error: Get "http://10.133.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-vqsb5 Readiness probe failed: Get "http://10.132.0.111:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-vqsb5 Readiness probe error: Get "http://10.132.0.111:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k Readiness probe failed: Get "http://10.133.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k Readiness probe error: Get "http://10.133.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning ProbeError pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd Readiness probe error: Get "http://10.134.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning ProbeError pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh Readiness probe error: Get "http://10.133.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh Readiness probe failed: Get "http://10.133.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning Unhealthy pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd Readiness probe failed: Get "http://10.134.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-f05b6443 Scaled up replica set payload-processing-e2e-ait-f05b6443-fbbf556dc from 0 to 1 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-f05b6443 Scaled up replica set payload-pre-processing-e2e-ait-f05b6443-869ddf56bc from 0 to 1 51m Normal ScalingReplicaSet deployment/e2e-ait-be51942b-openshift-default Scaled up replica set e2e-ait-be51942b-openshift-default-86bcdd8fcc from 0 to 1 51m Warning FailedCreate replicaset/payload-processing-e2e-ait-f05b6443-fbbf556dc Error creating: pods "payload-processing-e2e-ait-f05b6443-fbbf556dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-f05b6443: serviceaccount "payload-processing-e2e-ait-f05b6443" not found 51m Normal AddedLabel gateway/e2e-ait-be51942b Added label istio.io/rev=openshift-gateway to gateway e2e-ait-be51942b 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc Error creating: pods "payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-f05b6443: serviceaccount "payload-processing-e2e-ait-f05b6443" not found 51m Warning ProbeError pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s Readiness probe error: Get "http://10.134.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s Readiness probe failed: Get "http://10.134.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-f05b6443-fbbf556dc Created pod: payload-processing-e2e-ait-f05b6443-fbbf556dc-bdtcf 51m Normal SuccessfulCreate replicaset/e2e-ait-be51942b-openshift-default-86bcdd8fcc Created pod: e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc Created pod: payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x 51m Normal AddedInterface pod/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2 Add eth0 [10.133.0.78/23] from ovn-kubernetes 51m Normal Started pod/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x Container started 51m Normal Pulled pod/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Created pod/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x Container created 51m Normal Created pod/payload-processing-e2e-ait-f05b6443-fbbf556dc-bdtcf Container created 51m Normal AddedInterface pod/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x Add eth0 [10.133.0.79/23] from ovn-kubernetes 51m Normal Pulled pod/payload-processing-e2e-ait-f05b6443-fbbf556dc-bdtcf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/payload-processing-e2e-ait-f05b6443-fbbf556dc-bdtcf Add eth0 [10.132.0.130/23] from ovn-kubernetes 51m Normal Started pod/payload-processing-e2e-ait-f05b6443-fbbf556dc-bdtcf Container started 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-be51942b Scaled up replica set payload-processing-e2e-ait-be51942b-8574ff8f87 from 0 to 1 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-be51942b-b8f988b87 Error creating: pods "payload-pre-processing-e2e-ait-be51942b-b8f988b87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-be51942b: serviceaccount "payload-processing-e2e-ait-be51942b" not found 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-be51942b-b8f988b87 Created pod: payload-pre-processing-e2e-ait-be51942b-b8f988b87-bnklt 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-be51942b Scaled up replica set payload-pre-processing-e2e-ait-be51942b-b8f988b87 from 0 to 1 51m Warning FailedCreate replicaset/payload-processing-e2e-ait-be51942b-8574ff8f87 Error creating: pods "payload-processing-e2e-ait-be51942b-8574ff8f87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-be51942b: serviceaccount "payload-processing-e2e-ait-be51942b" not found 51m Normal AddedInterface pod/payload-pre-processing-e2e-ait-be51942b-b8f988b87-bnklt Add eth0 [10.132.0.132/23] from ovn-kubernetes 51m Normal SuccessfulCreate replicaset/e2e-ait-e585c53f-openshift-default-6b94747c57 Created pod: e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9 51m Normal Created pod/payload-pre-processing-e2e-ait-be51942b-b8f988b87-bnklt Container created 51m Normal ScalingReplicaSet deployment/e2e-ait-e585c53f-openshift-default Scaled up replica set e2e-ait-e585c53f-openshift-default-6b94747c57 from 0 to 1 51m Normal Pulled pod/payload-pre-processing-e2e-ait-be51942b-b8f988b87-bnklt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal AddedLabel gateway/e2e-ait-e585c53f Added label istio.io/rev=openshift-gateway to gateway e2e-ait-e585c53f 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-be51942b-8574ff8f87 Created pod: payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh 51m Normal Started pod/payload-pre-processing-e2e-ait-be51942b-b8f988b87-bnklt Container started 51m Normal Created pod/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh Container created 51m Normal Started pod/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh Container started 51m Normal AddedInterface pod/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9 Add eth0 [10.134.0.99/23] from ovn-kubernetes 51m Normal AddedInterface pod/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh Add eth0 [10.133.0.80/23] from ovn-kubernetes 51m Normal Pulled pod/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Warning FailedCreate replicaset/payload-processing-e2e-ait-e585c53f-68d69b695f Error creating: pods "payload-processing-e2e-ait-e585c53f-68d69b695f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-e585c53f: serviceaccount "payload-processing-e2e-ait-e585c53f" not found 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-e585c53f Scaled up replica set payload-pre-processing-e2e-ait-e585c53f-79c464f4b7 from 0 to 1 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7 Error creating: pods "payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-e585c53f: serviceaccount "payload-processing-e2e-ait-e585c53f" not found 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-e585c53f Scaled up replica set payload-processing-e2e-ait-e585c53f-68d69b695f from 0 to 1 51m Normal SuccessfulCreate replicaset/e2e-worker-w1-80c366-openshift-default-666b7db455 Created pod: e2e-worker-w1-80c366-openshift-default-666b7db455-dhs4k 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-e585c53f-68d69b695f Created pod: payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-80c366-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/e2e-worker-w1-80c366-openshift-default-666b7db455 (combined from similar events): Created pod: e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp 51m Normal SuccessfulCreate replicaset/e2e-worker-w1-80c366-openshift-default-666b7db455 Created pod: e2e-worker-w1-80c366-openshift-default-666b7db455-6ztpg 51m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-4z8hd 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7 Created pod: payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm 51m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 51m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 51m Normal ScalingReplicaSet deployment/e2e-worker-w1-80c366-openshift-default Scaled up replica set e2e-worker-w1-80c366-openshift-default-666b7db455 from 7 to 10 51m Normal Pulled pod/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-4z8hd Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 51m Normal AddedInterface pod/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm Add eth0 [10.133.0.82/23] from ovn-kubernetes 51m Normal Started pod/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp Container started 51m Normal AddedInterface pod/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v Add eth0 [10.133.0.81/23] from ovn-kubernetes 51m Normal Created pod/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp Container created 51m Normal AddedInterface pod/e2e-worker-w1-80c366-openshift-default-666b7db455-dhs4k Add eth0 [10.132.0.134/23] from ovn-kubernetes 51m Normal Pulled pod/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w1-80c366-openshift-default-666b7db455-dhs4k Container created 51m Normal Created pod/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm Container created 51m Normal Pulled pod/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp Add eth0 [10.133.0.83/23] from ovn-kubernetes 51m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-4z8hd Add eth0 [10.134.0.100/23] from ovn-kubernetes 51m Normal AddedInterface pod/e2e-worker-w1-80c366-openshift-default-666b7db455-6ztpg Add eth0 [10.132.0.133/23] from ovn-kubernetes 51m Normal Created pod/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v Container created 51m Normal Pulled pod/e2e-worker-w1-80c366-openshift-default-666b7db455-6ztpg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v Container started 51m Normal Started pod/e2e-worker-w1-80c366-openshift-default-666b7db455-dhs4k Container started 51m Normal Created pod/e2e-worker-w1-80c366-openshift-default-666b7db455-6ztpg Container created 51m Normal Started pod/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm Container started 51m Normal Pulled pod/e2e-worker-w1-80c366-openshift-default-666b7db455-dhs4k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w1-80c366-openshift-default-666b7db455-6ztpg Container started 51m Warning ProbeError pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6 Readiness probe error: Get "http://10.134.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6 Readiness probe failed: Get "http://10.134.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning Unhealthy pod/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp Startup probe failed: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp Startup probe error: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w1-80c366-openshift-default-666b7db455-6ztpg Startup probe failed: Get "http://10.132.0.133:15021/healthz/ready": dial tcp 10.132.0.133:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w1-80c366-openshift-default-666b7db455-6ztpg Startup probe error: Get "http://10.132.0.133:15021/healthz/ready": dial tcp 10.132.0.133:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w1-80c366-openshift-default-666b7db455-dhs4k Startup probe error: Get "http://10.132.0.134:15021/healthz/ready": dial tcp 10.132.0.134:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w1-80c366-openshift-default-666b7db455-dhs4k Startup probe failed: Get "http://10.132.0.134:15021/healthz/ready": dial tcp 10.132.0.134:15021: connect: connection refused 51m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-4z8hd Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 8.875s (8.875s including waiting). Image size: 185476477 bytes. 51m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-4z8hd Container created 51m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-4z8hd Container started 51m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-4z8hd Readiness probe failed: HTTP probe failed with statuscode: 503 51m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-4z8hd Readiness probe error: HTTP probe failed with statuscode: 503... 50m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-74854d8ccf Created pod: maas-default-gateway-openshift-default-74854d8ccf-xmvjc 50m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 50m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-74854d8ccf Created pod: maas-default-gateway-openshift-default-74854d8ccf-d5bcl 50m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-74854d8ccf Created pod: maas-default-gateway-openshift-default-74854d8ccf-bmmjn 50m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-d5bcl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-d5bcl Container created 50m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-d5bcl Container started 50m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-xmvjc Container created 50m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-bmmjn Container started 50m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-bmmjn Add eth0 [10.133.0.85/23] from ovn-kubernetes 50m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-bmmjn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-d5bcl Add eth0 [10.132.0.137/23] from ovn-kubernetes 50m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-xmvjc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-xmvjc Add eth0 [10.132.0.138/23] from ovn-kubernetes 50m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-bmmjn Container created 50m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-xmvjc Container started 50m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-bmmjn Startup probe failed: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused 50m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-bmmjn Startup probe error: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused... 50m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-xmvjc Startup probe error: Get "http://10.132.0.138:15021/healthz/ready": dial tcp 10.132.0.138:15021: connect: connection refused... 50m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-d5bcl Startup probe failed: Get "http://10.132.0.137:15021/healthz/ready": dial tcp 10.132.0.137:15021: connect: connection refused 50m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-d5bcl Startup probe error: Get "http://10.132.0.137:15021/healthz/ready": dial tcp 10.132.0.137:15021: connect: connection refused... 50m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-xmvjc Startup probe failed: Get "http://10.132.0.138:15021/healthz/ready": dial tcp 10.132.0.138:15021: connect: connection refused 50m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-74854d8ccf Created pod: maas-default-gateway-openshift-default-74854d8ccf-nzzv2 50m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-nzzv2 Container created 50m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-nzzv2 Container started 50m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-nzzv2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-nzzv2 Add eth0 [10.134.0.101/23] from ovn-kubernetes 50m Warning ProbeError pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz Readiness probe error: Get "http://10.134.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz Readiness probe failed: Get "http://10.134.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr Readiness probe error: Get "http://10.134.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr Readiness probe failed: Get "http://10.134.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w1-80c366-openshift-default-666b7db455-4kk96 Readiness probe error: Get "http://10.132.0.113:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w1-80c366-openshift-default-666b7db455-4kk96 Readiness probe failed: Get "http://10.132.0.113:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-nzzv2 Startup probe error: Get "http://10.134.0.101:15021/healthz/ready": dial tcp 10.134.0.101:15021: connect: connection refused... 50m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-nzzv2 Startup probe failed: Get "http://10.134.0.101:15021/healthz/ready": dial tcp 10.134.0.101:15021: connect: connection refused 50m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-74854d8ccf Created pod: maas-default-gateway-openshift-default-74854d8ccf-nvkls 50m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-74854d8ccf Created pod: maas-default-gateway-openshift-default-74854d8ccf-fwnc4 50m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-74854d8ccf Created pod: maas-default-gateway-openshift-default-74854d8ccf-fskjv 50m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-nvkls Add eth0 [10.133.0.88/23] from ovn-kubernetes 50m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-74854d8ccf Created pod: maas-default-gateway-openshift-default-74854d8ccf-jj2hd 50m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-pxw88 Add eth0 [10.133.0.89/23] from ovn-kubernetes 50m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-fwnc4 Add eth0 [10.134.0.102/23] from ovn-kubernetes 50m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-74854d8ccf Created pod: maas-default-gateway-openshift-default-74854d8ccf-pxw88 50m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-jj2hd Container started 50m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-nvkls Container started 50m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-fskjv Container created 50m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-fskjv Add eth0 [10.132.0.139/23] from ovn-kubernetes 50m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-pxw88 Container started 50m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-pxw88 Container created 50m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-fwnc4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-nvkls Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-nvkls Container created 50m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-fskjv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-pxw88 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-fwnc4 Container created 50m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-jj2hd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-fskjv Container started 50m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-jj2hd Container created 50m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-fwnc4 Container started 50m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-jj2hd Add eth0 [10.132.0.140/23] from ovn-kubernetes 50m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-pxw88 Startup probe error: Get "http://10.133.0.89:15021/healthz/ready": dial tcp 10.133.0.89:15021: connect: connection refused... 50m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-fwnc4 Startup probe failed: Get "http://10.134.0.102:15021/healthz/ready": dial tcp 10.134.0.102:15021: connect: connection refused 50m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-nvkls Startup probe failed: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused 50m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-fskjv Startup probe error: Get "http://10.132.0.139:15021/healthz/ready": dial tcp 10.132.0.139:15021: connect: connection refused... 50m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-nvkls Startup probe error: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused... 50m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-jj2hd Startup probe failed: Get "http://10.132.0.140:15021/healthz/ready": dial tcp 10.132.0.140:15021: connect: connection refused 50m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-jj2hd Startup probe error: Get "http://10.132.0.140:15021/healthz/ready": dial tcp 10.132.0.140:15021: connect: connection refused... 50m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-fskjv Startup probe failed: Get "http://10.132.0.139:15021/healthz/ready": dial tcp 10.132.0.139:15021: connect: connection refused 50m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-fwnc4 Startup probe error: Get "http://10.134.0.102:15021/healthz/ready": dial tcp 10.134.0.102:15021: connect: connection refused... 50m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-pxw88 Startup probe failed: Get "http://10.133.0.89:15021/healthz/ready": dial tcp 10.133.0.89:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x Readiness probe failed: Get "http://10.133.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x Readiness probe error: Get "http://10.133.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Normal Created pod/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x Container created 49m Normal SuccessfulCreate replicaset/e2e-ait-e585c53f-openshift-default-6b94747c57 Created pod: e2e-ait-e585c53f-openshift-default-6b94747c57-5n6cd 49m Normal Pulled pod/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal ScalingReplicaSet deployment/e2e-ait-e585c53f-openshift-default Scaled up replica set e2e-ait-e585c53f-openshift-default-6b94747c57 from 1 to 2 49m Normal Started pod/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x Container started 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ait-e585c53f-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 49m Normal Created pod/e2e-ait-e585c53f-openshift-default-6b94747c57-5n6cd Container created 49m Normal Pulled pod/e2e-ait-e585c53f-openshift-default-6b94747c57-5n6cd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-ait-e585c53f-openshift-default-6b94747c57-5n6cd Add eth0 [10.132.0.141/23] from ovn-kubernetes 49m Normal Started pod/e2e-ait-e585c53f-openshift-default-6b94747c57-5n6cd Container started 49m Warning Unhealthy pod/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9 Readiness probe failed: Get "http://10.134.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9 Readiness probe error: Get "http://10.134.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Normal Created pod/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2 Container created 49m Normal Started pod/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2 Container started 49m Normal Pulled pod/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9 Container created 49m Warning Unhealthy pod/e2e-ait-e585c53f-openshift-default-6b94747c57-5n6cd Startup probe failed: Get "http://10.132.0.141:15021/healthz/ready": dial tcp 10.132.0.141:15021: connect: connection refused 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-be51942b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Normal Started pod/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9 Container started 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-be51942b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Normal Pulled pod/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Warning ProbeError pod/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x Startup probe error: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x Startup probe failed: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused 49m Warning ProbeError pod/e2e-ait-e585c53f-openshift-default-6b94747c57-5n6cd Startup probe error: Get "http://10.132.0.141:15021/healthz/ready": dial tcp 10.132.0.141:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2 Startup probe failed: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused 49m Warning ProbeError pod/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2 Startup probe error: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused... 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-f05b6443-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-f05b6443-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning ProbeError pod/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9 Startup probe error: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused... 49m Normal Killing pod/payload-pre-processing-e2e-ait-f05b6443-869ddf56bc-5lq7x Stopping container payload-pre-processing 49m Normal Killing pod/payload-processing-e2e-ait-f05b6443-fbbf556dc-bdtcf Stopping container payload-processing 49m Warning Unhealthy pod/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9 Startup probe failed: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-e585c53f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-e585c53f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Normal Killing pod/payload-pre-processing-e2e-ait-be51942b-b8f988b87-bnklt Stopping container payload-pre-processing 49m Normal Killing pod/payload-processing-e2e-ait-be51942b-8574ff8f87-j6qxh Stopping container payload-processing 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-e585c53f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-e585c53f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-be51942b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-be51942b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-f05b6443-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-f05b6443-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Normal Killing pod/e2e-ait-f05b6443-openshift-default-d8cb8644f-kbd5x Stopping container istio-proxy 49m Normal Killing pod/payload-processing-e2e-ait-e585c53f-68d69b695f-kr95v Stopping container payload-processing 49m Normal Killing pod/payload-pre-processing-e2e-ait-e585c53f-79c464f4b7-44khm Stopping container payload-pre-processing 48m Normal Killing pod/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2 Stopping container istio-proxy 48m Normal AddedLabel gateway/e2e-derive-2b8311b7 Added label istio.io/rev=openshift-gateway to gateway e2e-derive-2b8311b7 48m Normal ScalingReplicaSet deployment/e2e-derive-2b8311b7-openshift-default Scaled up replica set e2e-derive-2b8311b7-openshift-default-6cbd6775b9 from 0 to 1 48m Normal SuccessfulCreate replicaset/e2e-derive-2b8311b7-openshift-default-6cbd6775b9 Created pod: e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf 48m Normal AddedInterface pod/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf Add eth0 [10.133.0.90/23] from ovn-kubernetes 48m Normal SuccessfulCreate replicaset/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45 Created pod: e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg 48m Normal AddedLabel gateway/e2e-migrate-87e2c278-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-87e2c278-gw 48m Normal ScalingReplicaSet deployment/e2e-migrate-87e2c278-gw-openshift-default Scaled up replica set e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45 from 0 to 1 48m Warning Unhealthy pod/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2 Readiness probe failed: HTTP probe failed with statuscode: 503 48m Warning ProbeError pod/e2e-ait-be51942b-openshift-default-86bcdd8fcc-kdrn2 Readiness probe error: HTTP probe failed with statuscode: 503... 48m Normal AddedInterface pod/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg Add eth0 [10.133.0.91/23] from ovn-kubernetes 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-2b8311b7-6956d7f8c5 Created pod: payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl 48m Warning FailedCreate replicaset/payload-processing-e2e-derive-2b8311b7-6956d7f8c5 Error creating: pods "payload-processing-e2e-derive-2b8311b7-6956d7f8c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-2b8311b7: serviceaccount "payload-processing-e2e-derive-2b8311b7" not found 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-2b8311b7 Scaled up replica set payload-processing-e2e-derive-2b8311b7-6956d7f8c5 from 0 to 1 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-2b8311b7 Scaled up replica set payload-pre-processing-e2e-derive-2b8311b7-86575775c5 from 0 to 1 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-2b8311b7-86575775c5 Error creating: pods "payload-pre-processing-e2e-derive-2b8311b7-86575775c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-2b8311b7: serviceaccount "payload-processing-e2e-derive-2b8311b7" not found 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-2b8311b7-86575775c5 Created pod: payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd 48m Normal Started pod/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl Container started 48m Normal Pulled pod/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl Add eth0 [10.133.0.92/23] from ovn-kubernetes 48m Normal Pulled pod/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl Container created 48m Normal Started pod/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd Container started 48m Normal AddedInterface pod/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd Add eth0 [10.133.0.93/23] from ovn-kubernetes 48m Normal Created pod/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd Container created 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9 Error creating: pods "payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-87e2c278: serviceaccount "payload-processing-e2e-migrate-87e2c278" not found 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9 Created pod: payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-87e2c278 Scaled up replica set payload-pre-processing-e2e-migrate-87e2c278-795978f9f9 from 0 to 1 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-87e2c278 Scaled up replica set payload-processing-e2e-migrate-87e2c278-6cf557bb69 from 0 to 1 48m Warning FailedCreate replicaset/payload-processing-e2e-migrate-87e2c278-6cf557bb69 Error creating: pods "payload-processing-e2e-migrate-87e2c278-6cf557bb69-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-87e2c278: serviceaccount "payload-processing-e2e-migrate-87e2c278" not found 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-87e2c278-6cf557bb69 Created pod: payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m 48m Warning Unhealthy pod/e2e-ait-e585c53f-openshift-default-6b94747c57-5n6cd Readiness probe failed: HTTP probe failed with statuscode: 503 48m Warning ProbeError pod/e2e-ait-e585c53f-openshift-default-6b94747c57-5n6cd Readiness probe error: HTTP probe failed with statuscode: 503... 48m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd Add eth0 [10.133.0.94/23] from ovn-kubernetes 48m Normal Killing pod/e2e-ait-e585c53f-openshift-default-6b94747c57-5n6cd Stopping container istio-proxy 48m Normal Killing pod/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9 Stopping container istio-proxy 48m Normal Pulled pod/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Started pod/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m Container started 48m Normal AddedInterface pod/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m Add eth0 [10.133.0.95/23] from ovn-kubernetes 48m Normal Created pod/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m Container created 48m Normal Started pod/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd Container started 48m Normal Pulled pod/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd Container created 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-87e2c278-gw-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-87e2c278-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 3 48m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: cpu resource utilization (percentage of request) above target 48m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-zctbt 48m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-zctbt Add eth0 [10.132.0.153/23] from ovn-kubernetes 48m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-zctbt Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 48m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-zctbt Container created 48m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-zctbt Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.908s (3.908s including waiting). Image size: 185476477 bytes. 48m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-zctbt Container started 48m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-zctbt Readiness probe error: HTTP probe failed with statuscode: 503... 48m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-zctbt Readiness probe failed: HTTP probe failed with statuscode: 503 47m Normal SuccessfulCreate replicaset/e2e-mt-e2deec91-openshift-default-67df997f57 Created pod: e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp 47m Normal ScalingReplicaSet deployment/e2e-mt-e2deec91-openshift-default Scaled up replica set e2e-mt-e2deec91-openshift-default-67df997f57 from 0 to 1 47m Normal AddedLabel gateway/e2e-mt-e2deec91 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-e2deec91 47m Normal AddedLabel gateway/e2e-shared-a-w3-326d76 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-326d76 47m Normal AddedInterface pod/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp Add eth0 [10.134.0.108/23] from ovn-kubernetes 47m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-326d76-openshift-default Scaled up replica set e2e-shared-a-w3-326d76-openshift-default-fdd7fd877 from 0 to 1 47m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877 Created pod: e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x 47m Normal AddedInterface pod/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x Add eth0 [10.133.0.98/23] from ovn-kubernetes 47m Warning Unhealthy pod/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg Readiness probe failed: Get "http://10.133.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg Readiness probe error: Get "http://10.133.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-e2deec91 Scaled up replica set payload-pre-processing-e2e-mt-e2deec91-65fc44c98c from 0 to 1 47m Warning FailedCreate replicaset/payload-processing-e2e-mt-e2deec91-589b46bc55 Error creating: pods "payload-processing-e2e-mt-e2deec91-589b46bc55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-e2deec91: serviceaccount "payload-processing-e2e-mt-e2deec91" not found 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-e2deec91-589b46bc55 Created pod: payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c Created pod: payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c Error creating: pods "payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-e2deec91: serviceaccount "payload-processing-e2e-mt-e2deec91" not found 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-e2deec91 Scaled up replica set payload-processing-e2e-mt-e2deec91-589b46bc55 from 0 to 1 47m Normal AddedInterface pod/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2 Add eth0 [10.133.0.99/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh Add eth0 [10.133.0.100/23] from ovn-kubernetes 47m Normal Started pod/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2 Container started 47m Normal Created pod/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2 Container created 47m Normal Pulled pod/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Pulled pod/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg Container created 47m Normal Started pod/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg Container started 47m Normal Started pod/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh Container started 47m Normal Pulled pod/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh Container created 47m Normal Started pod/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf Container started 47m Normal Created pod/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf Container created 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7 Created pod: payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p 47m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c Error creating: pods "payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-326d76: serviceaccount "payload-processing-e2e-shared-a-w3-326d76" not found 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w3-326d76 Scaled up replica set payload-processing-e2e-shared-a-w3-326d76-79f9b8946c from 0 to 1 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7 Error creating: pods "payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-326d76: serviceaccount "payload-processing-e2e-shared-a-w3-326d76" not found 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w3-326d76 Scaled up replica set payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7 from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c Created pod: payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp 47m Normal Created pod/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p Container created 47m Normal Started pod/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp Container started 47m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p Add eth0 [10.133.0.101/23] from ovn-kubernetes 47m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp Add eth0 [10.133.0.102/23] from ovn-kubernetes 47m Normal Pulled pod/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp Container created 47m Normal Started pod/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p Container started 47m Warning ProbeError pod/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg Startup probe error: Get "http://10.133.0.91:15021/healthz/ready": dial tcp 10.133.0.91:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg Startup probe failed: Get "http://10.133.0.91:15021/healthz/ready": dial tcp 10.133.0.91:15021: connect: connection refused 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-2b8311b7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-2b8311b7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning Unhealthy pod/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf Startup probe failed: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused 47m Warning ProbeError pod/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf Startup probe error: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused... 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-87e2c278-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-87e2c278-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal Killing pod/payload-pre-processing-e2e-derive-2b8311b7-86575775c5-vglkd Stopping container payload-pre-processing 46m Normal Killing pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r Stopping container istio-proxy 46m Normal ScalingReplicaSet deployment/e2e-worker-w6-557708-openshift-default Scaled down replica set e2e-worker-w6-557708-openshift-default-bd7dbddd5 from 10 to 7 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-557708-openshift-default New size: 7; reason: All metrics below target 46m Normal SuccessfulDelete replicaset/e2e-worker-w6-557708-openshift-default-bd7dbddd5 Deleted pod: e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r 46m Normal Killing pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/e2e-worker-w6-557708-openshift-default-bd7dbddd5 Deleted pod: e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd 46m Normal Killing pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/e2e-worker-w6-557708-openshift-default-bd7dbddd5 Deleted pod: e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj 46m Normal Killing pod/payload-processing-e2e-derive-2b8311b7-6956d7f8c5-qq4hl Stopping container payload-processing 46m Normal Killing pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl Stopping container istio-proxy 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-dda9f5-openshift-default New size: 5; reason: All metrics below target 46m Normal Killing pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/e2e-worker-w0-dda9f5-openshift-default-59dc47f975 Deleted pod: e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p 46m Normal Killing pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs Stopping container istio-proxy 46m Normal Killing pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w Stopping container istio-proxy 46m Normal ScalingReplicaSet deployment/e2e-worker-w0-dda9f5-openshift-default Scaled down replica set e2e-worker-w0-dda9f5-openshift-default-59dc47f975 from 10 to 5 46m Normal Killing pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6 Stopping container istio-proxy 46m Normal Killing pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr Stopping container istio-proxy 46m Normal Killing pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5 Stopping container istio-proxy 46m Normal Killing pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/e2e-worker-w0-dda9f5-openshift-default-59dc47f975 Deleted pod: e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr 46m Normal Killing pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/e2e-worker-w0-dda9f5-openshift-default-59dc47f975 Deleted pod: e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs 46m Normal SuccessfulDelete replicaset/e2e-worker-w0-dda9f5-openshift-default-59dc47f975 Deleted pod: e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n 46m Normal SuccessfulDelete replicaset/e2e-worker-w0-dda9f5-openshift-default-59dc47f975 Deleted pod: e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv 46m Normal SuccessfulDelete replicaset/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf Deleted pod: e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl 46m Normal SuccessfulDelete replicaset/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf Deleted pod: e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-59cq6 46m Normal SuccessfulDelete replicaset/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf Deleted pod: e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r 46m Normal SuccessfulDelete replicaset/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf Deleted pod: e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w 46m Normal SuccessfulDelete replicaset/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf Deleted pod: e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5 46m Normal Killing pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv Stopping container istio-proxy 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-9b51e1-openshift-default New size: 5; reason: All metrics below target 46m Normal ScalingReplicaSet deployment/e2e-worker-w5-9b51e1-openshift-default Scaled down replica set e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf from 10 to 5 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-2b8311b7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-2b8311b7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal AddedLabel gateway/e2e-shared-a-w0-99f69c Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-99f69c 46m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc Created pod: e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm 46m Warning Unhealthy pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj Readiness probe failed: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused 46m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-99f69c-openshift-default Scaled up replica set e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc from 0 to 1 46m Warning ProbeError pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj Readiness probe error: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning ProbeError pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv Readiness probe error: HTTP probe failed with statuscode: 503... 46m Normal AddedInterface pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm Add eth0 [10.134.0.109/23] from ovn-kubernetes 46m Warning Unhealthy pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning Unhealthy pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning ProbeError pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n Readiness probe error: HTTP probe failed with statuscode: 503... 46m Warning ProbeError pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w Readiness probe error: HTTP probe failed with statuscode: 503... 46m Warning ProbeError pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs Readiness probe error: HTTP probe failed with statuscode: 503... 46m Warning Unhealthy pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-pdmzs Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning Unhealthy pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning ProbeError pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr Readiness probe error: HTTP probe failed with statuscode: 503... 46m Warning ProbeError pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p Readiness probe error: Get "http://10.134.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p Readiness probe failed: Get "http://10.134.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w0-99f69c Scaled up replica set payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc from 0 to 1 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w0-99f69c Scaled up replica set payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f from 0 to 1 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f Error creating: pods "payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c: serviceaccount "payload-processing-e2e-shared-a-w0-99f69c" not found 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc Created pod: payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f Created pod: payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4 46m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc Error creating: pods "payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c: serviceaccount "payload-processing-e2e-shared-a-w0-99f69c" not found 46m Normal Created pod/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4 Container created 46m Normal Started pod/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn Container started 46m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4 Add eth0 [10.134.0.110/23] from ovn-kubernetes 46m Normal AddedInterface pod/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn Add eth0 [10.134.0.111/23] from ovn-kubernetes 46m Normal Pulled pod/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn Container created 46m Normal Killing pod/payload-pre-processing-e2e-migrate-87e2c278-795978f9f9-tnwhd Stopping container payload-pre-processing 46m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Started pod/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4 Container started 46m Normal Killing pod/payload-processing-e2e-migrate-87e2c278-6cf557bb69-j7k9m Stopping container payload-processing 46m Normal AddedLabel gateway/e2e-mt-6df3ba0e Added label istio.io/rev=openshift-gateway to gateway e2e-mt-6df3ba0e 46m Normal SuccessfulCreate replicaset/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf Created pod: e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-99f69c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-99f69c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal ScalingReplicaSet deployment/e2e-mt-6df3ba0e-openshift-default Scaled up replica set e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf from 0 to 1 46m Normal AddedInterface pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb Add eth0 [10.134.0.113/23] from ovn-kubernetes 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-6df3ba0e Scaled up replica set payload-processing-e2e-mt-6df3ba0e-84d7646477 from 0 to 1 46m Warning FailedCreate replicaset/payload-processing-e2e-mt-6df3ba0e-84d7646477 Error creating: pods "payload-processing-e2e-mt-6df3ba0e-84d7646477-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-6df3ba0e: serviceaccount "payload-processing-e2e-mt-6df3ba0e" not found 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-6df3ba0e-84d7646477 Created pod: payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9 Error creating: pods "payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-6df3ba0e: serviceaccount "payload-processing-e2e-mt-6df3ba0e" not found 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-6df3ba0e Scaled up replica set payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9 from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9 Created pod: payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll 46m Normal SuccessfulDelete replicaset/e2e-worker-w6-557708-openshift-default-bd7dbddd5 Deleted pod: e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77 46m Normal Pulled pod/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-557708-openshift-default New size: 6; reason: All metrics below target 46m Normal ScalingReplicaSet deployment/e2e-worker-w6-557708-openshift-default Scaled down replica set e2e-worker-w6-557708-openshift-default-bd7dbddd5 from 7 to 6 46m Normal Killing pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-8pd77 Stopping container istio-proxy 46m Normal Started pod/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll Container started 46m Normal Created pod/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll Container created 46m Normal Pulled pod/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll Add eth0 [10.134.0.115/23] from ovn-kubernetes 46m Normal Started pod/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n Container started 46m Normal AddedInterface pod/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n Add eth0 [10.134.0.116/23] from ovn-kubernetes 46m Normal Created pod/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n Container created 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-557708-openshift-default New size: 5; reason: All metrics below target 46m Normal Killing pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-msfnj Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/e2e-worker-w6-557708-openshift-default-bd7dbddd5 Deleted pod: e2e-worker-w6-557708-openshift-default-bd7dbddd5-msfnj 46m Normal Killing pod/e2e-derive-2b8311b7-openshift-default-6cbd6775b9-vxhwf Stopping container istio-proxy 46m Normal AddedLabel gateway/e2e-shared-a-w2-ea3cb2 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-ea3cb2 46m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f Created pod: e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2 46m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-ea3cb2-openshift-default Scaled up replica set e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f from 0 to 1 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-c3e2d8-openshift-default New size: 5; reason: All metrics below target 46m Normal AddedInterface pod/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2 Add eth0 [10.134.0.117/23] from ovn-kubernetes 46m Normal SuccessfulDelete replicaset/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c Deleted pod: e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 46m Warning Unhealthy pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-msfnj Readiness probe failed: HTTP probe failed with statuscode: 503 46m Normal Killing pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx Stopping container istio-proxy 46m Normal Killing pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wn9kx Stopping container istio-proxy 46m Normal Killing pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd Stopping container istio-proxy 46m Warning ProbeError pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-msfnj Readiness probe error: HTTP probe failed with statuscode: 503... 46m Normal Killing pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 Stopping container istio-proxy 46m Warning ProbeError pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 Readiness probe error: HTTP probe failed with statuscode: 503... 46m Normal Killing pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8 Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c Deleted pod: e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8 46m Normal ScalingReplicaSet deployment/e2e-worker-w4-c3e2d8-openshift-default Scaled down replica set e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c from 10 to 5 46m Normal SuccessfulDelete replicaset/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c Deleted pod: e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd 46m Normal SuccessfulDelete replicaset/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c Deleted pod: e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wn9kx 46m Normal SuccessfulDelete replicaset/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c Deleted pod: e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx 46m Warning ProbeError pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx Readiness probe error: HTTP probe failed with statuscode: 503... 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-6df3ba0e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal Killing pod/e2e-migrate-87e2c278-gw-openshift-default-796ccbfb45-c9hqg Stopping container istio-proxy 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-6df3ba0e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning Unhealthy pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wn9kx Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning ProbeError pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wn9kx Readiness probe error: HTTP probe failed with statuscode: 503... 46m Normal AddedLabel gateway/e2e-shared-a-w6-9d4043 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w6-9d4043 46m Warning Unhealthy pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8 Readiness probe failed: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused 46m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9 Created pod: e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr 46m Normal AddedInterface pod/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr Add eth0 [10.133.0.103/23] from ovn-kubernetes 46m Warning ProbeError pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8 Readiness probe error: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused... 46m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-9d4043-openshift-default Scaled up replica set e2e-shared-a-w6-9d4043-openshift-default-76f596cc9 from 0 to 1 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-ea3cb2 Scaled up replica set payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9 from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9 Created pod: payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-ea3cb2 Scaled up replica set payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99 from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99 Created pod: payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99 Error creating: pods "payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2: serviceaccount "payload-processing-e2e-shared-a-w2-ea3cb2" not found 45m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9 Error creating: pods "payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2: serviceaccount "payload-processing-e2e-shared-a-w2-ea3cb2" not found 45m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd Container started 45m Normal Pulled pod/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Started pod/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l Container started 45m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd Container created 45m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd Add eth0 [10.134.0.118/23] from ovn-kubernetes 45m Normal Created pod/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l Container created 45m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l Add eth0 [10.134.0.119/23] from ovn-kubernetes 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c Created pod: payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w6-9d4043 Scaled up replica set payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c from 0 to 1 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c Error creating: pods "payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043: serviceaccount "payload-processing-e2e-shared-a-w6-9d4043" not found 45m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7 Error creating: pods "payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-9d4043: serviceaccount "payload-processing-e2e-shared-a-w6-9d4043" not found 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w6-9d4043 Scaled up replica set payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7 from 0 to 1 45m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n Add eth0 [10.133.0.104/23] from ovn-kubernetes 45m Normal AddedInterface pod/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t Add eth0 [10.133.0.105/23] from ovn-kubernetes 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7 Created pod: payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t 45m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Killing pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw Stopping container istio-proxy 45m Normal SuccessfulDelete replicaset/e2e-worker-w2-c79cb5-openshift-default-5464c54587 Deleted pod: e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw 45m Normal Killing pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-bf95b Stopping container istio-proxy 45m Warning ProbeError pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw Readiness probe error: HTTP probe failed with statuscode: 503... 45m Normal Killing pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4 Stopping container istio-proxy 45m Normal Created pod/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n Container created 45m Normal Killing pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5 Stopping container istio-proxy 45m Normal Started pod/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t Container started 45m Normal Started pod/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n Container started 45m Normal Killing pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq Stopping container istio-proxy 45m Normal SuccessfulDelete replicaset/e2e-worker-w2-c79cb5-openshift-default-5464c54587 Deleted pod: e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq 45m Normal SuccessfulDelete replicaset/e2e-worker-w2-c79cb5-openshift-default-5464c54587 Deleted pod: e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4 45m Normal SuccessfulDelete replicaset/e2e-worker-w2-c79cb5-openshift-default-5464c54587 Deleted pod: e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5 45m Normal SuccessfulDelete replicaset/e2e-worker-w2-c79cb5-openshift-default-5464c54587 Deleted pod: e2e-worker-w2-c79cb5-openshift-default-5464c54587-bf95b 45m Normal Created pod/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t Container created 45m Normal Pulled pod/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-c79cb5-openshift-default New size: 5; reason: All metrics below target 45m Normal ScalingReplicaSet deployment/e2e-worker-w2-c79cb5-openshift-default Scaled down replica set e2e-worker-w2-c79cb5-openshift-default-5464c54587 from 10 to 5 45m Warning ProbeError pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning ProbeError pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5 Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning Unhealthy pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5 Readiness probe failed: HTTP probe failed with statuscode: 503 45m Normal Started pod/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp Container started 45m Normal Pulled pod/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp Container created 45m Normal Started pod/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x Container started 45m Normal Created pod/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x Container created 45m Normal Pulled pod/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-326d76-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-326d76-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning ProbeError pod/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp Startup probe error: Get "http://10.134.0.108:15021/healthz/ready": dial tcp 10.134.0.108:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp Startup probe failed: Get "http://10.134.0.108:15021/healthz/ready": dial tcp 10.134.0.108:15021: connect: connection refused 45m Warning Unhealthy pod/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x Startup probe failed: Get "http://10.133.0.98:15021/healthz/ready": dial tcp 10.133.0.98:15021: connect: connection refused 45m Warning ProbeError pod/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x Startup probe error: Get "http://10.133.0.98:15021/healthz/ready": dial tcp 10.133.0.98:15021: connect: connection refused... 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-e2deec91-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-e2deec91-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-326d76-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-326d76-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal AddedLabel gateway/e2e-shared-b-w3-20c7c7 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-20c7c7 45m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95 Created pod: e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7 45m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-20c7c7-openshift-default Scaled up replica set e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95 from 0 to 1 45m Normal AddedInterface pod/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7 Add eth0 [10.134.0.122/23] from ovn-kubernetes 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-e2deec91-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-e2deec91-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal ScalingReplicaSet deployment/e2e-worker-w1-80c366-openshift-default Scaled down replica set e2e-worker-w1-80c366-openshift-default-666b7db455 from 10 to 5 45m Normal Killing pod/e2e-worker-w1-80c366-openshift-default-666b7db455-4kk96 Stopping container istio-proxy 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-80c366-openshift-default New size: 5; reason: All metrics below target 45m Normal SuccessfulDelete replicaset/e2e-worker-w1-80c366-openshift-default-666b7db455 Deleted pod: e2e-worker-w1-80c366-openshift-default-666b7db455-4kk96 45m Normal SuccessfulDelete replicaset/e2e-worker-w1-80c366-openshift-default-666b7db455 Deleted pod: e2e-worker-w1-80c366-openshift-default-666b7db455-6ztpg 45m Normal SuccessfulDelete replicaset/e2e-worker-w1-80c366-openshift-default-666b7db455 Deleted pod: e2e-worker-w1-80c366-openshift-default-666b7db455-dhs4k 45m Normal SuccessfulDelete replicaset/e2e-worker-w1-80c366-openshift-default-666b7db455 Deleted pod: e2e-worker-w1-80c366-openshift-default-666b7db455-l6mk7 45m Normal Killing pod/e2e-worker-w1-80c366-openshift-default-666b7db455-l6mk7 Stopping container istio-proxy 45m Normal Killing pod/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr Stopping container istio-proxy 45m Normal Killing pod/e2e-worker-w1-80c366-openshift-default-666b7db455-dhs4k Stopping container istio-proxy 45m Normal Killing pod/e2e-worker-w1-80c366-openshift-default-666b7db455-6ztpg Stopping container istio-proxy 45m Normal SuccessfulDelete replicaset/e2e-worker-w1-80c366-openshift-default-666b7db455 Deleted pod: e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr 45m Normal SuccessfulDelete replicaset/e2e-worker-w3-bc009f-openshift-default-645d5cbf67 Deleted pod: e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh 45m Normal ScalingReplicaSet deployment/e2e-worker-w3-bc009f-openshift-default Scaled down replica set e2e-worker-w3-bc009f-openshift-default-645d5cbf67 from 10 to 5 45m Normal Killing pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s Stopping container istio-proxy 45m Normal Killing pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ldkbp Stopping container istio-proxy 45m Normal Killing pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6 Stopping container istio-proxy 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-bc009f-openshift-default New size: 5; reason: All metrics below target 45m Normal Killing pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz Stopping container istio-proxy 45m Normal Killing pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh Stopping container istio-proxy 45m Normal SuccessfulDelete replicaset/e2e-worker-w3-bc009f-openshift-default-645d5cbf67 Deleted pod: e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz 45m Normal SuccessfulDelete replicaset/e2e-worker-w3-bc009f-openshift-default-645d5cbf67 Deleted pod: e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s 45m Normal SuccessfulDelete replicaset/e2e-worker-w3-bc009f-openshift-default-645d5cbf67 Deleted pod: e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ldkbp 45m Normal SuccessfulDelete replicaset/e2e-worker-w3-bc009f-openshift-default-645d5cbf67 Deleted pod: e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6 45m Warning ProbeError pod/e2e-worker-w1-80c366-openshift-default-666b7db455-l6mk7 Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning Unhealthy pod/e2e-worker-w1-80c366-openshift-default-666b7db455-l6mk7 Readiness probe failed: HTTP probe failed with statuscode: 503 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-99f69c-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 45m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc Created pod: e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc 45m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-99f69c-openshift-default Scaled up replica set e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc from 1 to 2 45m Warning Unhealthy pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz Readiness probe failed: HTTP probe failed with statuscode: 503 45m Normal Pulled pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w3-20c7c7 Scaled up replica set payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf from 0 to 1 45m Warning ProbeError pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz Readiness probe error: HTTP probe failed with statuscode: 503... 45m Normal Started pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc Container started 45m Normal AddedInterface pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc Add eth0 [10.133.0.106/23] from ovn-kubernetes 45m Normal Created pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc Container created 45m Warning Unhealthy pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh Readiness probe failed: HTTP probe failed with statuscode: 503 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf Created pod: payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94 45m Warning Unhealthy pod/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf Error creating: pods "payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7: serviceaccount "payload-processing-e2e-shared-b-w3-20c7c7" not found 45m Warning ProbeError pod/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning ProbeError pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning ProbeError pod/e2e-worker-w1-80c366-openshift-default-666b7db455-4kk96 Readiness probe error: Get "http://10.132.0.113:15021/healthz/ready": dial tcp 10.132.0.113:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-worker-w1-80c366-openshift-default-666b7db455-4kk96 Readiness probe failed: Get "http://10.132.0.113:15021/healthz/ready": dial tcp 10.132.0.113:15021: connect: connection refused 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655 Error creating: pods "payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7: serviceaccount "payload-processing-e2e-shared-b-w3-20c7c7" not found 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655 Created pod: payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w3-20c7c7 Scaled up replica set payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655 from 0 to 1 45m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg Container created 45m Normal AddedInterface pod/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94 Add eth0 [10.134.0.124/23] from ovn-kubernetes 45m Warning Unhealthy pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb Readiness probe failed: Get "http://10.134.0.113:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Normal Pulled pod/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94 Container created 45m Normal Started pod/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94 Container started 45m Warning ProbeError pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb Readiness probe error: Get "http://10.134.0.113:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Normal Started pod/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg Container started 45m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg Add eth0 [10.134.0.123/23] from ovn-kubernetes 44m Normal Started pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb Container started 44m Normal Created pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb Container created 44m Warning Unhealthy pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc Startup probe failed: Get "http://10.133.0.106:15021/healthz/ready": dial tcp 10.133.0.106:15021: connect: connection refused 44m Normal Pulled pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Warning ProbeError pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc Startup probe error: Get "http://10.133.0.106:15021/healthz/ready": dial tcp 10.133.0.106:15021: connect: connection refused... 44m Normal Pulled pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm Container created 44m Normal Started pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm Container started 44m Warning ProbeError pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb Startup probe error: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb Startup probe failed: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused 44m Warning Unhealthy pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm Startup probe failed: Get "http://10.134.0.109:15021/healthz/ready": dial tcp 10.134.0.109:15021: connect: connection refused 44m Warning ProbeError pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm Startup probe error: Get "http://10.134.0.109:15021/healthz/ready": dial tcp 10.134.0.109:15021: connect: connection refused... 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-99f69c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-99f69c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal AddedLabel gateway/e2e-api-a-9f8e92 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-9f8e92 44m Normal SuccessfulCreate replicaset/e2e-api-a-9f8e92-openshift-default-86856d5b7 Created pod: e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw 44m Normal ScalingReplicaSet deployment/e2e-api-a-9f8e92-openshift-default Scaled up replica set e2e-api-a-9f8e92-openshift-default-86856d5b7 from 0 to 1 44m Normal AddedInterface pod/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw Add eth0 [10.134.0.127/23] from ovn-kubernetes 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-9f8e92 Scaled up replica set payload-processing-e2e-api-a-9f8e92-7b988cb4d5 from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47 Created pod: payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z 44m Warning FailedCreate replicaset/payload-processing-e2e-api-a-9f8e92-7b988cb4d5 Error creating: pods "payload-processing-e2e-api-a-9f8e92-7b988cb4d5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-9f8e92: serviceaccount "payload-processing-e2e-api-a-9f8e92" not found 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-9f8e92 Scaled up replica set payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47 from 0 to 1 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47 Error creating: pods "payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-9f8e92: serviceaccount "payload-processing-e2e-api-a-9f8e92" not found 44m Normal Created pod/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z Container created 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-9f8e92-7b988cb4d5 Created pod: payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt 44m Normal Pulled pod/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z Add eth0 [10.134.0.128/23] from ovn-kubernetes 44m Normal AddedInterface pod/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt Add eth0 [10.134.0.129/23] from ovn-kubernetes 44m Normal Pulled pod/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt Container created 44m Normal Started pod/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt Container started 44m Normal Started pod/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z Container started 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-6df3ba0e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-6df3ba0e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-485d1e-openshift-default Scaled up replica set e2e-shared-b-w0-485d1e-openshift-default-7559ff8579 from 0 to 1 44m Normal AddedLabel gateway/e2e-shared-b-w0-485d1e Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-485d1e 44m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579 Created pod: e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz 44m Normal AddedInterface pod/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz Add eth0 [10.134.0.130/23] from ovn-kubernetes 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w0-485d1e Scaled up replica set payload-pre-processing-e2e-shared-b-w0-485d1e-58bccc79c9 from 0 to 1 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w0-485d1e-58bccc79c9 Error creating: pods "payload-pre-processing-e2e-shared-b-w0-485d1e-58bccc79c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-485d1e: serviceaccount "payload-processing-e2e-shared-b-w0-485d1e" not found 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w0-485d1e Scaled up replica set payload-processing-e2e-shared-b-w0-485d1e-66c964c857 from 0 to 1 44m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w0-485d1e-66c964c857 Error creating: pods "payload-processing-e2e-shared-b-w0-485d1e-66c964c857-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-485d1e: serviceaccount "payload-processing-e2e-shared-b-w0-485d1e" not found 44m Warning ProbeError pod/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2 Readiness probe error: Get "http://10.134.0.117:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Warning Unhealthy pod/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2 Readiness probe failed: Get "http://10.134.0.117:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Normal Started pod/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2 Container started 44m Normal Pulled pod/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-processing-e2e-shared-b-w0-485d1e-66c964c857-fddzn Add eth0 [10.132.0.157/23] from ovn-kubernetes 44m Normal Created pod/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2 Container created 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w0-485d1e-66c964c857 Created pod: payload-processing-e2e-shared-b-w0-485d1e-66c964c857-fddzn 44m Normal Started pod/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr Container started 44m Normal Created pod/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr Container created 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w0-485d1e-58bccc79c9 Created pod: payload-pre-processing-e2e-shared-b-w0-485d1e-58bccc79c9-jk9hv 44m Normal Pulled pod/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w0-485d1e-58bccc79c9-jk9hv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-9d4043-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal Started pod/payload-processing-e2e-shared-b-w0-485d1e-66c964c857-fddzn Container started 44m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w0-485d1e-58bccc79c9-jk9hv Add eth0 [10.132.0.156/23] from ovn-kubernetes 44m Normal Created pod/payload-processing-e2e-shared-b-w0-485d1e-66c964c857-fddzn Container created 44m Normal Pulled pod/payload-processing-e2e-shared-b-w0-485d1e-66c964c857-fddzn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-pre-processing-e2e-shared-b-w0-485d1e-58bccc79c9-jk9hv Container created 44m Normal Started pod/payload-pre-processing-e2e-shared-b-w0-485d1e-58bccc79c9-jk9hv Container started 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-9d4043-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-9f8e92-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-9f8e92-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning Unhealthy pod/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr Startup probe failed: Get "http://10.133.0.103:15021/healthz/ready": dial tcp 10.133.0.103:15021: connect: connection refused 44m Warning ProbeError pod/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr Startup probe error: Get "http://10.133.0.103:15021/healthz/ready": dial tcp 10.133.0.103:15021: connect: connection refused... 44m Warning ProbeError pod/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2 Startup probe error: Get "http://10.134.0.117:15021/healthz/ready": dial tcp 10.134.0.117:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2 Startup probe failed: Get "http://10.134.0.117:15021/healthz/ready": dial tcp 10.134.0.117:15021: connect: connection refused 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-ea3cb2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-ea3cb2-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-485d1e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-485d1e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal Killing pod/payload-pre-processing-e2e-mt-e2deec91-65fc44c98c-hf5c2 Stopping container payload-pre-processing 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-9d4043-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal Killing pod/payload-processing-e2e-mt-e2deec91-589b46bc55-6ttgh Stopping container payload-processing 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-9d4043-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-ea3cb2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-ea3cb2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal AddedLabel gateway/e2e-shared-b-w2-6cd200 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-6cd200 43m Normal AddedInterface pod/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h Add eth0 [10.134.0.133/23] from ovn-kubernetes 43m Normal AddedLabel gateway/e2e-shared-b-w6-1db768 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w6-1db768 43m Normal SuccessfulDelete replicaset/e2e-worker-w6-557708-openshift-default-bd7dbddd5 Deleted pod: e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94 43m Normal Killing pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94 Stopping container istio-proxy 43m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-6cd200-openshift-default Scaled up replica set e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7 from 0 to 1 43m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7 Created pod: e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h 43m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-1db768-openshift-default Scaled up replica set e2e-shared-b-w6-1db768-openshift-default-85c476bb45 from 0 to 1 43m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-1db768-openshift-default-85c476bb45 Created pod: e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx 43m Normal AddedInterface pod/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx Add eth0 [10.134.0.134/23] from ovn-kubernetes 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-c79cb5-openshift-default New size: 4; reason: All metrics below target 43m Normal SuccessfulDelete replicaset/e2e-worker-w2-c79cb5-openshift-default-5464c54587 Deleted pod: e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj 43m Normal Killing pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zx4nj Stopping container istio-proxy 43m Normal ScalingReplicaSet deployment/e2e-worker-w2-c79cb5-openshift-default Scaled down replica set e2e-worker-w2-c79cb5-openshift-default-5464c54587 from 5 to 4 43m Normal SuccessfulDelete replicaset/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf Deleted pod: e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-khbxp 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-9b51e1-openshift-default New size: 4; reason: All metrics below target 43m Normal Killing pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-khbxp Stopping container istio-proxy 43m Warning ProbeError pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94 Readiness probe error: HTTP probe failed with statuscode: 503... 43m Warning Unhealthy pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-qhv94 Readiness probe failed: HTTP probe failed with statuscode: 503 43m Normal ScalingReplicaSet deployment/e2e-worker-w5-9b51e1-openshift-default Scaled down replica set e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf from 5 to 4 43m Warning Unhealthy pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-khbxp Readiness probe failed: HTTP probe failed with statuscode: 503 43m Warning ProbeError pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-khbxp Readiness probe error: HTTP probe failed with statuscode: 503... 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w6-1db768 Scaled up replica set payload-processing-e2e-shared-b-w6-1db768-5d657c68cf from 0 to 1 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w6-1db768 Scaled up replica set payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47 from 0 to 1 43m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf Error creating: pods "payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-1db768: serviceaccount "payload-processing-e2e-shared-b-w6-1db768" not found 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47 Created pod: payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47 Error creating: pods "payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-1db768: serviceaccount "payload-processing-e2e-shared-b-w6-1db768" not found 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf Created pod: payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p 43m Normal Started pod/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8 Container started 43m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p Container created 43m Normal AddedInterface pod/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p Add eth0 [10.133.0.108/23] from ovn-kubernetes 43m Normal Started pod/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p Container started 43m Normal Created pod/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8 Container created 43m Normal Pulled pod/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8 Add eth0 [10.133.0.107/23] from ovn-kubernetes 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-6cd200 Scaled up replica set payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc from 0 to 1 43m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc Error creating: pods "payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200: serviceaccount "payload-processing-e2e-shared-b-w2-6cd200" not found 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95 Created pod: payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc Created pod: payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95 Error creating: pods "payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-6cd200: serviceaccount "payload-processing-e2e-shared-b-w2-6cd200" not found 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-6cd200 Scaled up replica set payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95 from 0 to 1 43m Normal Started pod/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5 Container started 43m Normal Created pod/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5 Container created 43m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm Add eth0 [10.133.0.109/23] from ovn-kubernetes 43m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm Container created 43m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm Container started 43m Normal Pulled pod/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5 Add eth0 [10.133.0.110/23] from ovn-kubernetes 42m Normal Created pod/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7 Container created 42m Normal Started pod/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7 Container started 42m Normal Pulled pod/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-20c7c7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-20c7c7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal Killing pod/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp Stopping container istio-proxy 42m Warning ProbeError pod/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7 Startup probe error: Get "http://10.134.0.122:15021/healthz/ready": dial tcp 10.134.0.122:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7 Startup probe failed: Get "http://10.134.0.122:15021/healthz/ready": dial tcp 10.134.0.122:15021: connect: connection refused 42m Normal AddedLabel gateway/e2e-api-a-9c8c4d Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-9c8c4d 42m Normal ScalingReplicaSet deployment/e2e-api-a-9c8c4d-openshift-default Scaled up replica set e2e-api-a-9c8c4d-openshift-default-dfc9cb655 from 0 to 1 42m Normal SuccessfulCreate replicaset/e2e-api-a-9c8c4d-openshift-default-dfc9cb655 Created pod: e2e-api-a-9c8c4d-openshift-default-dfc9cb655-fzv82 42m Normal AddedInterface pod/e2e-api-a-9c8c4d-openshift-default-dfc9cb655-fzv82 Add eth0 [10.132.0.159/23] from ovn-kubernetes 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-9c8c4d Scaled up replica set payload-processing-e2e-api-a-9c8c4d-57f5c98589 from 0 to 1 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679 Error creating: pods "payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-9c8c4d: serviceaccount "payload-processing-e2e-api-a-9c8c4d" not found 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-9c8c4d Scaled up replica set payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679 from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-9c8c4d-57f5c98589 Created pod: payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679 Created pod: payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n 42m Warning FailedCreate replicaset/payload-processing-e2e-api-a-9c8c4d-57f5c98589 Error creating: pods "payload-processing-e2e-api-a-9c8c4d-57f5c98589-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-9c8c4d: serviceaccount "payload-processing-e2e-api-a-9c8c4d" not found 42m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n Add eth0 [10.133.0.111/23] from ovn-kubernetes 42m Normal Started pod/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n Container started 42m Normal Pulled pod/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Created pod/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd Container created 42m Normal Started pod/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd Container started 42m Normal AddedInterface pod/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd Add eth0 [10.133.0.112/23] from ovn-kubernetes 42m Normal Pulled pod/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Created pod/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n Container created 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-20c7c7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-20c7c7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Normal Pulled pod/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw Container created 42m Normal Started pod/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz Container started 42m Normal Created pod/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz Container created 42m Normal Started pod/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw Container started 42m Normal Pulled pod/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal AddedLabel gateway/e2e-api-a-78cf83 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-78cf83 42m Normal SuccessfulCreate replicaset/e2e-api-a-78cf83-openshift-default-c5b4b7cb7 Created pod: e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb 42m Normal ScalingReplicaSet deployment/e2e-api-a-78cf83-openshift-default Scaled up replica set e2e-api-a-78cf83-openshift-default-c5b4b7cb7 from 0 to 1 42m Normal AddedInterface pod/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb Add eth0 [10.134.0.137/23] from ovn-kubernetes 42m Warning Unhealthy pod/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw Startup probe failed: Get "http://10.134.0.127:15021/healthz/ready": dial tcp 10.134.0.127:15021: connect: connection refused 42m Warning ProbeError pod/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz Startup probe error: Get "http://10.134.0.130:15021/healthz/ready": dial tcp 10.134.0.130:15021: connect: connection refused... 42m Warning ProbeError pod/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw Startup probe error: Get "http://10.134.0.127:15021/healthz/ready": dial tcp 10.134.0.127:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz Startup probe failed: Get "http://10.134.0.130:15021/healthz/ready": dial tcp 10.134.0.130:15021: connect: connection refused 42m Normal SuccessfulDelete replicaset/e2e-worker-w6-557708-openshift-default-bd7dbddd5 Deleted pod: e2e-worker-w6-557708-openshift-default-bd7dbddd5-tmlmf 42m Normal Killing pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-tmlmf Stopping container istio-proxy 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-78cf83-5cb6f94fcc Created pod: payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-78cf83-796db64fff Created pod: payload-pre-processing-e2e-api-a-78cf83-796db64fff-7ph2p 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-78cf83-796db64fff Error creating: pods "payload-pre-processing-e2e-api-a-78cf83-796db64fff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-78cf83: serviceaccount "payload-processing-e2e-api-a-78cf83" not found 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-78cf83 Scaled up replica set payload-processing-e2e-api-a-78cf83-5cb6f94fcc from 0 to 1 42m Warning FailedCreate replicaset/payload-processing-e2e-api-a-78cf83-5cb6f94fcc Error creating: pods "payload-processing-e2e-api-a-78cf83-5cb6f94fcc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-78cf83: serviceaccount "payload-processing-e2e-api-a-78cf83" not found 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-78cf83 Scaled up replica set payload-pre-processing-e2e-api-a-78cf83-796db64fff from 0 to 1 42m Normal Created pod/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4 Container created 42m Normal AddedInterface pod/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4 Add eth0 [10.133.0.113/23] from ovn-kubernetes 42m Normal Started pod/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4 Container started 42m Normal Started pod/payload-pre-processing-e2e-api-a-78cf83-796db64fff-7ph2p Container started 42m Normal Created pod/payload-pre-processing-e2e-api-a-78cf83-796db64fff-7ph2p Container created 42m Normal Pulled pod/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Pulled pod/payload-pre-processing-e2e-api-a-78cf83-796db64fff-7ph2p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-78cf83-796db64fff-7ph2p Add eth0 [10.132.0.160/23] from ovn-kubernetes 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-9b51e1-openshift-default New size: 3; reason: All metrics below target 42m Normal Killing pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-fkzmf Stopping container istio-proxy 42m Normal ScalingReplicaSet deployment/e2e-worker-w5-9b51e1-openshift-default Scaled down replica set e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf from 4 to 3 42m Normal SuccessfulDelete replicaset/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf Deleted pod: e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-fkzmf 42m Warning Unhealthy pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-tmlmf Readiness probe failed: HTTP probe failed with statuscode: 503 42m Warning ProbeError pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-fkzmf Readiness probe error: HTTP probe failed with statuscode: 503... 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-dda9f5-openshift-default New size: 3; reason: All metrics below target 42m Normal Killing pod/maas-default-gateway-openshift-default-74854d8ccf-xmvjc Stopping container istio-proxy 42m Normal Killing pod/maas-default-gateway-openshift-default-74854d8ccf-jj2hd Stopping container istio-proxy 42m Normal Killing pod/maas-default-gateway-openshift-default-74854d8ccf-d5bcl Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-74854d8ccf Deleted pod: maas-default-gateway-openshift-default-74854d8ccf-fskjv 42m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-74854d8ccf Deleted pod: maas-default-gateway-openshift-default-74854d8ccf-d5bcl 42m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-74854d8ccf Deleted pod: maas-default-gateway-openshift-default-74854d8ccf-xmvjc 42m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-74854d8ccf Deleted pod: maas-default-gateway-openshift-default-74854d8ccf-jj2hd 42m Normal SuccessfulDelete replicaset/e2e-worker-w0-dda9f5-openshift-default-59dc47f975 Deleted pod: e2e-worker-w0-dda9f5-openshift-default-59dc47f975-hfdvc 42m Normal Killing pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-5b9gk Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w0-dda9f5-openshift-default-59dc47f975 Deleted pod: e2e-worker-w0-dda9f5-openshift-default-59dc47f975-5b9gk 42m Normal ScalingReplicaSet deployment/e2e-worker-w0-dda9f5-openshift-default Scaled down replica set e2e-worker-w0-dda9f5-openshift-default-59dc47f975 from 5 to 3 42m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: All metrics below target 42m Normal Killing pod/maas-default-gateway-openshift-default-74854d8ccf-fskjv Stopping container istio-proxy 42m Normal Killing pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-hfdvc Stopping container istio-proxy 42m Warning ProbeError pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-tmlmf Readiness probe error: HTTP probe failed with statuscode: 503... 42m Warning Unhealthy pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-fkzmf Readiness probe failed: HTTP probe failed with statuscode: 503 42m Warning Unhealthy pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-hfdvc Readiness probe failed: HTTP probe failed with statuscode: 503 42m Warning ProbeError pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-hfdvc Readiness probe error: HTTP probe failed with statuscode: 503... 42m Warning Unhealthy pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-5b9gk Readiness probe failed: HTTP probe failed with statuscode: 503 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-9f8e92-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-9f8e92-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning ProbeError pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-5b9gk Readiness probe error: HTTP probe failed with statuscode: 503... 41m Normal AddedLabel gateway/e2e-api-b-307368 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-307368 41m Normal SuccessfulCreate replicaset/e2e-api-b-307368-openshift-default-748d46b5cc Created pod: e2e-api-b-307368-openshift-default-748d46b5cc-hzgkz 41m Normal ScalingReplicaSet deployment/e2e-api-b-307368-openshift-default Scaled up replica set e2e-api-b-307368-openshift-default-748d46b5cc from 0 to 1 41m Normal AddedInterface pod/e2e-api-b-307368-openshift-default-748d46b5cc-hzgkz Add eth0 [10.132.0.162/23] from ovn-kubernetes 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-307368-68678879f5 Created pod: payload-pre-processing-e2e-api-b-307368-68678879f5-dtgqs 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-307368 Scaled up replica set payload-processing-e2e-api-b-307368-7574bb79f5 from 0 to 1 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-307368-68678879f5 Error creating: pods "payload-pre-processing-e2e-api-b-307368-68678879f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-307368: serviceaccount "payload-processing-e2e-api-b-307368" not found 41m Warning FailedCreate replicaset/payload-processing-e2e-api-b-307368-7574bb79f5 Error creating: pods "payload-processing-e2e-api-b-307368-7574bb79f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-307368: serviceaccount "payload-processing-e2e-api-b-307368" not found 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-307368 Scaled up replica set payload-pre-processing-e2e-api-b-307368-68678879f5 from 0 to 1 41m Normal Created pod/payload-pre-processing-e2e-api-b-307368-68678879f5-dtgqs Container created 41m Normal Created pod/payload-processing-e2e-api-b-307368-7574bb79f5-t8w22 Container created 41m Normal AddedInterface pod/payload-processing-e2e-api-b-307368-7574bb79f5-t8w22 Add eth0 [10.132.0.164/23] from ovn-kubernetes 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-307368-7574bb79f5 Created pod: payload-processing-e2e-api-b-307368-7574bb79f5-t8w22 41m Normal Started pod/payload-processing-e2e-api-b-307368-7574bb79f5-t8w22 Container started 41m Normal Pulled pod/payload-pre-processing-e2e-api-b-307368-68678879f5-dtgqs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Pulled pod/payload-processing-e2e-api-b-307368-7574bb79f5-t8w22 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Started pod/payload-pre-processing-e2e-api-b-307368-68678879f5-dtgqs Container started 41m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-307368-68678879f5-dtgqs Add eth0 [10.132.0.163/23] from ovn-kubernetes 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-485d1e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-485d1e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal Created pod/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h Container created 41m Normal Pulled pod/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Started pod/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h Container started 41m Normal Pulled pod/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Started pod/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx Container started 41m Normal Created pod/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx Container created 41m Normal Killing pod/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4 Stopping container payload-pre-processing 41m Normal Killing pod/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn Stopping container payload-processing 41m Normal ScalingReplicaSet deployment/e2e-worker-w2-c79cb5-openshift-default Scaled down replica set e2e-worker-w2-c79cb5-openshift-default-5464c54587 from 4 to 3 41m Warning Unhealthy pod/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h Startup probe failed: Get "http://10.134.0.133:15021/healthz/ready": dial tcp 10.134.0.133:15021: connect: connection refused 41m Normal Killing pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92 Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w2-c79cb5-openshift-default-5464c54587 Deleted pod: e2e-worker-w2-c79cb5-openshift-default-5464c54587-xzvzb 41m Warning ProbeError pod/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h Startup probe error: Get "http://10.134.0.133:15021/healthz/ready": dial tcp 10.134.0.133:15021: connect: connection refused... 41m Normal SuccessfulDelete replicaset/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c Deleted pod: e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wpf92 41m Normal Killing pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-xzvzb Stopping container istio-proxy 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-c79cb5-openshift-default New size: 3; reason: All metrics below target 41m Normal ScalingReplicaSet deployment/e2e-worker-w4-c3e2d8-openshift-default Scaled down replica set e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c from 5 to 3 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-c3e2d8-openshift-default New size: 3; reason: All metrics below target 41m Normal SuccessfulDelete replicaset/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c Deleted pod: e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k 41m Warning ProbeError pod/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx Startup probe error: Get "http://10.134.0.134:15021/healthz/ready": dial tcp 10.134.0.134:15021: connect: connection refused... 41m Warning ProbeError pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-wct4k Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning Unhealthy pod/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx Startup probe failed: Get "http://10.134.0.134:15021/healthz/ready": dial tcp 10.134.0.134:15021: connect: connection refused 40m Normal Killing pod/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n Stopping container payload-processing 40m Normal Killing pod/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll Stopping container payload-pre-processing 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-6cd200-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-6cd200-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-1db768-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-1db768-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-6cd200-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-6cd200-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-zctbt 40m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-zctbt Stopping container discovery 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-1db768-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-1db768-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-80c366-openshift-default New size: 4; reason: All metrics below target 40m Normal Killing pod/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk Stopping container istio-proxy 40m Normal ScalingReplicaSet deployment/e2e-worker-w1-80c366-openshift-default Scaled down replica set e2e-worker-w1-80c366-openshift-default-666b7db455 from 5 to 4 40m Normal SuccessfulDelete replicaset/e2e-worker-w1-80c366-openshift-default-666b7db455 Deleted pod: e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk 40m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-74854d8ccf Deleted pod: maas-default-gateway-openshift-default-74854d8ccf-nzzv2 40m Warning Unhealthy pod/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 40m Warning Unhealthy pod/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 40m Normal Killing pod/maas-default-gateway-openshift-default-74854d8ccf-nzzv2 Stopping container istio-proxy 40m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: All metrics below target 40m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-74854d8ccf from 6 to 5 40m Warning Unhealthy pod/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning ProbeError pod/e2e-worker-w1-80c366-openshift-default-666b7db455-n62rk Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning ProbeError pod/e2e-api-a-9c8c4d-openshift-default-dfc9cb655-fzv82 Readiness probe error: Get "http://10.132.0.159:15021/healthz/ready": read tcp 10.132.0.2:36490->10.132.0.159:15021: read: connection reset by peer... 40m Normal Created pod/e2e-api-a-9c8c4d-openshift-default-dfc9cb655-fzv82 Container created 40m Normal Pulled pod/e2e-api-a-9c8c4d-openshift-default-dfc9cb655-fzv82 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Warning Unhealthy pod/e2e-api-a-9c8c4d-openshift-default-dfc9cb655-fzv82 Readiness probe failed: Get "http://10.132.0.159:15021/healthz/ready": read tcp 10.132.0.2:36490->10.132.0.159:15021: read: connection reset by peer 40m Normal Started pod/e2e-api-a-9c8c4d-openshift-default-dfc9cb655-fzv82 Container started 40m Warning Unhealthy pod/e2e-api-a-9c8c4d-openshift-default-dfc9cb655-fzv82 Startup probe failed: Get "http://10.132.0.159:15021/healthz/ready": dial tcp 10.132.0.159:15021: connect: connection refused 40m Warning ProbeError pod/e2e-api-a-9c8c4d-openshift-default-dfc9cb655-fzv82 Startup probe error: Get "http://10.132.0.159:15021/healthz/ready": dial tcp 10.132.0.159:15021: connect: connection refused... 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-80c366-openshift-default New size: 3; reason: All metrics below target 40m Normal SuccessfulDelete replicaset/e2e-worker-w1-80c366-openshift-default-666b7db455 Deleted pod: e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp 40m Normal ScalingReplicaSet deployment/e2e-worker-w1-80c366-openshift-default Scaled down replica set e2e-worker-w1-80c366-openshift-default-666b7db455 from 4 to 3 40m Normal Killing pod/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4 Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w3-bc009f-openshift-default-645d5cbf67 Deleted pod: e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4 40m Normal ScalingReplicaSet deployment/e2e-worker-w3-bc009f-openshift-default Scaled down replica set e2e-worker-w3-bc009f-openshift-default-645d5cbf67 from 5 to 3 40m Normal SuccessfulDelete replicaset/e2e-worker-w3-bc009f-openshift-default-645d5cbf67 Deleted pod: e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-bc009f-openshift-default New size: 3; reason: All metrics below target 40m Warning Unhealthy pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4 Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning ProbeError pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-hmzt4 Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning ProbeError pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-q57qh Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning Unhealthy pod/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp Readiness probe failed: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w1-80c366-openshift-default-666b7db455-v68zp Readiness probe error: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused... 40m Normal Killing pod/payload-processing-e2e-shared-a-w6-9d4043-5869bbdf7-dw74t Stopping container payload-processing 40m Normal Killing pod/payload-pre-processing-e2e-shared-a-w6-9d4043-6dffc4b68c-ttn8n Stopping container payload-pre-processing 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-9c8c4d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-9c8c4d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal Killing pod/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l Stopping container payload-processing 39m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd Stopping container payload-pre-processing 39m Normal Killing pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc Stopping container istio-proxy 39m Normal Killing pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm Stopping container istio-proxy 39m Normal Pulled pod/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Warning Unhealthy pod/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb Readiness probe failed: Get "http://10.134.0.137:15021/healthz/ready": read tcp 10.134.0.2:35510->10.134.0.137:15021: read: connection reset by peer 39m Normal Started pod/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb Container started 39m Warning ProbeError pod/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb Readiness probe error: Get "http://10.134.0.137:15021/healthz/ready": read tcp 10.134.0.2:35510->10.134.0.137:15021: read: connection reset by peer... 39m Normal Created pod/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb Container created 39m Warning ProbeError pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-vh2tc Readiness probe failed: HTTP probe failed with statuscode: 503 39m Normal Killing pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-fj8mc Stopping container istio-proxy 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-557708-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 39m Normal ScalingReplicaSet deployment/e2e-worker-w6-557708-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w6-557708-openshift-default-bd7dbddd5 from 3 to 2 39m Normal SuccessfulDelete replicaset/e2e-worker-w6-557708-openshift-default-bd7dbddd5 Deleted pod: e2e-worker-w6-557708-openshift-default-bd7dbddd5-fj8mc 39m Warning ProbeError pod/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb Startup probe error: Get "http://10.134.0.137:15021/healthz/ready": dial tcp 10.134.0.137:15021: connect: connection refused... 39m Warning ProbeError pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-fj8mc Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-fj8mc Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning Unhealthy pod/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb Startup probe failed: Get "http://10.134.0.137:15021/healthz/ready": dial tcp 10.134.0.137:15021: connect: connection refused 39m Normal SuccessfulDelete replicaset/e2e-worker-w2-c79cb5-openshift-default-5464c54587 Deleted pod: e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-c79cb5-openshift-default New size: 2; reason: All metrics below target 39m Normal Killing pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm Stopping container istio-proxy 39m Normal ScalingReplicaSet deployment/e2e-worker-w2-c79cb5-openshift-default Scaled down replica set e2e-worker-w2-c79cb5-openshift-default-5464c54587 from 3 to 2 39m Warning Unhealthy pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning ProbeError pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-75fwm Readiness probe error: HTTP probe failed with statuscode: 503... 39m Normal Killing pod/payload-pre-processing-e2e-shared-b-w0-485d1e-58bccc79c9-jk9hv Stopping container payload-pre-processing 39m Normal Killing pod/payload-processing-e2e-shared-b-w0-485d1e-66c964c857-fddzn Stopping container payload-processing 39m Normal Killing pod/payload-pre-processing-e2e-api-a-9c8c4d-b8f6bf679-s5m9n Stopping container payload-pre-processing 39m Normal Killing pod/payload-processing-e2e-api-a-9c8c4d-57f5c98589-vlmwd Stopping container payload-processing 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-78cf83-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-78cf83-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal Pulled pod/e2e-api-b-307368-openshift-default-748d46b5cc-hzgkz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-api-b-307368-openshift-default-748d46b5cc-hzgkz Container created 39m Normal Started pod/e2e-api-b-307368-openshift-default-748d46b5cc-hzgkz Container started 39m Warning Unhealthy pod/e2e-api-b-307368-openshift-default-748d46b5cc-hzgkz Startup probe failed: Get "http://10.132.0.162:15021/healthz/ready": dial tcp 10.132.0.162:15021: connect: connection refused 39m Warning ProbeError pod/e2e-api-b-307368-openshift-default-748d46b5cc-hzgkz Startup probe error: Get "http://10.132.0.162:15021/healthz/ready": dial tcp 10.132.0.162:15021: connect: connection refused... 39m Normal ScalingReplicaSet deployment/e2e-worker-w3-bc009f-openshift-default Scaled down replica set e2e-worker-w3-bc009f-openshift-default-645d5cbf67 from 3 to 2 39m Normal SuccessfulDelete replicaset/e2e-worker-w3-bc009f-openshift-default-645d5cbf67 Deleted pod: e2e-worker-w3-bc009f-openshift-default-645d5cbf67-tzj7w 39m Normal Killing pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-tzj7w Stopping container istio-proxy 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-bc009f-openshift-default New size: 2; reason: All metrics below target 39m Warning ProbeError pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-tzj7w Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-tzj7w Readiness probe failed: HTTP probe failed with statuscode: 503 39m Normal Killing pod/payload-processing-e2e-api-a-78cf83-5cb6f94fcc-m7rm4 Stopping container payload-processing 39m Normal Killing pod/payload-pre-processing-e2e-api-a-78cf83-796db64fff-7ph2p Stopping container payload-pre-processing 38m Normal Killing pod/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2 Stopping container istio-proxy 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-307368-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-307368-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Normal Killing pod/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb Stopping container istio-proxy 38m Normal Killing pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26 Stopping container istio-proxy 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-9b51e1-openshift-default New size: 2; reason: All metrics below target 38m Normal SuccessfulDelete replicaset/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf Deleted pod: e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-dqw26 38m Normal ScalingReplicaSet deployment/e2e-worker-w5-9b51e1-openshift-default Scaled down replica set e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf from 3 to 2 38m Normal Killing pod/e2e-shared-a-w6-9d4043-openshift-default-76f596cc9-qqqtr Stopping container istio-proxy 38m Normal AddedLabel gateway/e2e-inf-a-9da607 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-9da607 38m Normal ScalingReplicaSet deployment/e2e-inf-a-9da607-openshift-default Scaled up replica set e2e-inf-a-9da607-openshift-default-5d68dbc6c7 from 0 to 1 38m Normal SuccessfulCreate replicaset/e2e-inf-a-9da607-openshift-default-5d68dbc6c7 Created pod: e2e-inf-a-9da607-openshift-default-5d68dbc6c7-jld42 38m Normal AddedInterface pod/e2e-inf-a-9da607-openshift-default-5d68dbc6c7-jld42 Add eth0 [10.132.0.180/23] from ovn-kubernetes 38m Normal Killing pod/payload-processing-e2e-shared-b-w2-6cd200-77d6899dcc-rrps5 Stopping container payload-processing 38m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-6cd200-5fcfb47f95-x2tjm Stopping container payload-pre-processing 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-9da607-555bb58887 Created pod: payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-9da607 Scaled up replica set payload-pre-processing-e2e-inf-a-9da607-74c77fc695 from 0 to 1 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-9da607 Scaled up replica set payload-processing-e2e-inf-a-9da607-555bb58887 from 0 to 1 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-9da607-74c77fc695 Created pod: payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p 38m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-9da607-555bb58887 Error creating: pods "payload-processing-e2e-inf-a-9da607-555bb58887-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-9da607: serviceaccount "payload-processing-e2e-inf-a-9da607" not found 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-9da607-74c77fc695 Error creating: pods "payload-pre-processing-e2e-inf-a-9da607-74c77fc695-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-9da607: serviceaccount "payload-processing-e2e-inf-a-9da607" not found 38m Normal Created pod/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7 Container created 38m Normal AddedInterface pod/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7 Add eth0 [10.133.0.115/23] from ovn-kubernetes 38m Normal Killing pod/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z Stopping container payload-pre-processing 38m Normal Killing pod/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt Stopping container payload-processing 38m Normal Started pod/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7 Container started 38m Normal Pulled pod/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p Add eth0 [10.133.0.114/23] from ovn-kubernetes 38m Normal Pulled pod/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Created pod/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p Container created 38m Normal Started pod/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p Container started 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-307368-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-307368-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Normal Killing pod/payload-pre-processing-e2e-shared-b-w6-1db768-56c9dfcb47-8w2m8 Stopping container payload-pre-processing 38m Normal Killing pod/payload-processing-e2e-shared-b-w6-1db768-5d657c68cf-mlq2p Stopping container payload-processing 38m Normal Killing pod/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h Stopping container istio-proxy 38m Warning Unhealthy pod/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning ProbeError pod/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h Readiness probe error: HTTP probe failed with statuscode: 503... 38m Normal Killing pod/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz Stopping container istio-proxy 38m Warning Unhealthy pod/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning ProbeError pod/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz Readiness probe error: HTTP probe failed with statuscode: 503... 37m Normal Killing pod/e2e-api-a-9c8c4d-openshift-default-dfc9cb655-fzv82 Stopping container istio-proxy 37m Warning ProbeError pod/e2e-api-a-9c8c4d-openshift-default-dfc9cb655-fzv82 Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/e2e-api-a-9c8c4d-openshift-default-dfc9cb655-fzv82 Readiness probe failed: HTTP probe failed with statuscode: 503 37m Normal Killing pod/payload-pre-processing-e2e-worker-w0-dda9f5-5cb698d9c9-hj25v Stopping container payload-pre-processing 37m Normal Killing pod/payload-processing-e2e-worker-w0-dda9f5-75fcc89f49-bh6xw Stopping container payload-processing 37m Normal ScalingReplicaSet deployment/e2e-worker-w1-80c366-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w1-80c366-openshift-default-666b7db455 from 3 to 2 37m Normal SuccessfulDelete replicaset/e2e-worker-w1-80c366-openshift-default-666b7db455 Deleted pod: e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-80c366-openshift-default New size: 2; reason: All metrics below target 37m Normal Killing pod/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2 Stopping container istio-proxy 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-c3e2d8-openshift-default New size: 2; reason: All metrics below target 37m Normal SuccessfulDelete replicaset/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c Deleted pod: e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-vqsb5 37m Normal Killing pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-vqsb5 Stopping container istio-proxy 37m Normal ScalingReplicaSet deployment/e2e-worker-w4-c3e2d8-openshift-default Scaled down replica set e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c from 3 to 2 37m Warning ProbeError pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-vqsb5 Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-vqsb5 Readiness probe failed: HTTP probe failed with statuscode: 503 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-6df3ba0e Scaled up replica set payload-processing-e2e-mt-6df3ba0e-84d7646477 from 0 to 1 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9 Error creating: pods "payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-6df3ba0e: serviceaccount "payload-processing-e2e-mt-6df3ba0e" not found 37m Warning FailedCreate replicaset/payload-processing-e2e-mt-6df3ba0e-84d7646477 Error creating: pods "payload-processing-e2e-mt-6df3ba0e-84d7646477-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-6df3ba0e: serviceaccount "payload-processing-e2e-mt-6df3ba0e" not found 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-6df3ba0e Scaled up replica set payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9 from 0 to 1 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-6df3ba0e-84d7646477 Created pod: payload-processing-e2e-mt-6df3ba0e-84d7646477-6qgzr 37m Normal AddedInterface pod/payload-processing-e2e-mt-6df3ba0e-84d7646477-6qgzr Add eth0 [10.132.0.184/23] from ovn-kubernetes 37m Normal Pulled pod/payload-processing-e2e-mt-6df3ba0e-84d7646477-6qgzr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9 Created pod: payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-mcc6n 37m Normal AddedInterface pod/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-mcc6n Add eth0 [10.132.0.183/23] from ovn-kubernetes 37m Normal Created pod/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-mcc6n Container created 37m Normal Started pod/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-mcc6n Container started 37m Normal Pulled pod/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-mcc6n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Created pod/payload-processing-e2e-mt-6df3ba0e-84d7646477-6qgzr Container created 37m Normal Started pod/payload-processing-e2e-mt-6df3ba0e-84d7646477-6qgzr Container started 37m Normal Killing pod/payload-processing-e2e-worker-w5-9b51e1-65b8fbbc9f-ng2zh Stopping container payload-processing 37m Normal Killing pod/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4 Stopping container payload-pre-processing 37m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-vxnxc Add eth0 [10.134.0.140/23] from ovn-kubernetes 37m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-74854d8ccf from 5 to 10 37m Normal SuccessfulDelete replicaset/e2e-worker-w0-dda9f5-openshift-default-59dc47f975 Deleted pod: e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-dda9f5-openshift-default New size: 2; reason: All metrics below target 37m Normal Killing pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh Stopping container istio-proxy 37m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-f2rlm Add eth0 [10.134.0.139/23] from ovn-kubernetes 37m Normal ScalingReplicaSet deployment/e2e-worker-w0-dda9f5-openshift-default Scaled down replica set e2e-worker-w0-dda9f5-openshift-default-59dc47f975 from 3 to 2 37m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-44qk9 Add eth0 [10.132.0.187/23] from ovn-kubernetes 37m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-pwfjh Add eth0 [10.132.0.188/23] from ovn-kubernetes 37m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-44qk9 Container started 37m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-vxnxc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-44qk9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-pwfjh Container started 37m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-f2rlm Container started 37m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-44qk9 Container created 37m Warning ProbeError pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh Readiness probe error: HTTP probe failed with statuscode: 503... 37m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-pptz4 Container started 37m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-pptz4 Container created 37m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-pptz4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-pptz4 Add eth0 [10.132.0.189/23] from ovn-kubernetes 37m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-pwfjh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-vxnxc Container created 37m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-f2rlm Container created 37m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-vxnxc Container started 37m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-f2rlm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Warning Unhealthy pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-2kmqh Readiness probe failed: HTTP probe failed with statuscode: 503 37m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-pwfjh Container created 37m Normal Killing pod/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx Stopping container istio-proxy 37m Warning Unhealthy pod/payload-processing-e2e-worker-w0-dda9f5-75fcc89f49-bh6xw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-dda9f5-5cb698d9c9-hj25v Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-f2rlm Startup probe error: Get "http://10.134.0.139:15021/healthz/ready": dial tcp 10.134.0.139:15021: connect: connection refused... 37m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-pptz4 Startup probe error: Get "http://10.132.0.189:15021/healthz/ready": dial tcp 10.132.0.189:15021: connect: connection refused... 37m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-pwfjh Startup probe error: Get "http://10.132.0.188:15021/healthz/ready": dial tcp 10.132.0.188:15021: connect: connection refused... 37m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-pwfjh Startup probe failed: Get "http://10.132.0.188:15021/healthz/ready": dial tcp 10.132.0.188:15021: connect: connection refused 37m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-vxnxc Startup probe failed: Get "http://10.134.0.140:15021/healthz/ready": dial tcp 10.134.0.140:15021: connect: connection refused 37m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-vxnxc Startup probe error: Get "http://10.134.0.140:15021/healthz/ready": dial tcp 10.134.0.140:15021: connect: connection refused... 37m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-44qk9 Startup probe failed: Get "http://10.132.0.187:15021/healthz/ready": dial tcp 10.132.0.187:15021: connect: connection refused 37m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-f2rlm Startup probe failed: Get "http://10.134.0.139:15021/healthz/ready": dial tcp 10.134.0.139:15021: connect: connection refused 37m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-44qk9 Startup probe error: Get "http://10.132.0.187:15021/healthz/ready": dial tcp 10.132.0.187:15021: connect: connection refused... 37m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-pptz4 Startup probe failed: Get "http://10.132.0.189:15021/healthz/ready": dial tcp 10.132.0.189:15021: connect: connection refused 37m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-pwfjh Startup probe failed: Get "http://10.132.0.188:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-pptz4 Startup probe error: Get "http://10.132.0.189:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-pptz4 Startup probe failed: Get "http://10.132.0.189:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-44qk9 Startup probe error: Get "http://10.132.0.187:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-pwfjh Startup probe error: Get "http://10.132.0.188:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-44qk9 Startup probe failed: Get "http://10.132.0.187:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w5-9b51e1-846589545f-zmds4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Warning Unhealthy pod/payload-processing-e2e-worker-w5-9b51e1-65b8fbbc9f-ng2zh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Normal Created pod/e2e-inf-a-9da607-openshift-default-5d68dbc6c7-jld42 Container created 36m Normal Pulled pod/e2e-inf-a-9da607-openshift-default-5d68dbc6c7-jld42 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/e2e-inf-a-9da607-openshift-default-5d68dbc6c7-jld42 Container started 36m Warning Unhealthy pod/e2e-inf-a-9da607-openshift-default-5d68dbc6c7-jld42 Startup probe failed: Get "http://10.132.0.180:15021/healthz/ready": dial tcp 10.132.0.180:15021: connect: connection refused 36m Warning ProbeError pod/e2e-inf-a-9da607-openshift-default-5d68dbc6c7-jld42 Startup probe error: Get "http://10.132.0.180:15021/healthz/ready": dial tcp 10.132.0.180:15021: connect: connection refused... 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-9da607-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-9da607-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal AddedLabel gateway/e2e-inf-b-954628 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-954628 36m Normal Killing pod/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw Stopping container istio-proxy 36m Normal SuccessfulCreate replicaset/e2e-inf-b-954628-openshift-default-789d9b4ccc Created pod: e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs 36m Normal ScalingReplicaSet deployment/e2e-inf-b-954628-openshift-default Scaled up replica set e2e-inf-b-954628-openshift-default-789d9b4ccc from 0 to 1 36m Normal AddedInterface pod/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs Add eth0 [10.134.0.142/23] from ovn-kubernetes 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-9da607-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-9da607-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-954628-648fb887cf Error creating: pods "payload-processing-e2e-inf-b-954628-648fb887cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-954628: serviceaccount "payload-processing-e2e-inf-b-954628" not found 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-954628 Scaled up replica set payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c from 0 to 1 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-954628 Scaled up replica set payload-processing-e2e-inf-b-954628-648fb887cf from 0 to 1 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c Error creating: pods "payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-954628: serviceaccount "payload-processing-e2e-inf-b-954628" not found 36m Normal Started pod/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87 Container started 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-954628-648fb887cf Created pod: payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c Created pod: payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87 36m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87 Add eth0 [10.133.0.116/23] from ovn-kubernetes 36m Normal Created pod/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87 Container created 36m Normal AddedInterface pod/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c Add eth0 [10.133.0.117/23] from ovn-kubernetes 36m Normal Pulled pod/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Started pod/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c Container started 36m Normal Killing pod/payload-pre-processing-e2e-api-b-307368-68678879f5-dtgqs Stopping container payload-pre-processing 36m Normal Pulled pod/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Killing pod/payload-processing-e2e-api-b-307368-7574bb79f5-t8w22 Stopping container payload-processing 36m Normal Created pod/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c Container created 36m Normal ScalingReplicaSet deployment/e2e-mt-6df3ba0e-openshift-default Scaled up replica set e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf from 1 to 2 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-6df3ba0e-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 36m Normal SuccessfulCreate replicaset/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf Created pod: e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb 36m Normal Pulled pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Killing pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-z4jhc Stopping container istio-proxy 36m Normal Started pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb Container started 36m Normal Created pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb Container created 36m Normal AddedInterface pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb Add eth0 [10.133.0.118/23] from ovn-kubernetes 36m Normal Killing pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-rf429 Stopping container istio-proxy 36m Warning Unhealthy pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb Startup probe failed: Get "http://10.133.0.118:15021/healthz/ready": dial tcp 10.133.0.118:15021: connect: connection refused 36m Warning ProbeError pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb Startup probe error: Get "http://10.133.0.118:15021/healthz/ready": dial tcp 10.133.0.118:15021: connect: connection refused... 36m Normal SuccessfulCreate replicaset/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf Created pod: e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-4kk54 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-6df3ba0e-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 36m Normal ScalingReplicaSet deployment/e2e-mt-6df3ba0e-openshift-default Scaled up replica set e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf from 2 to 4 36m Normal SuccessfulCreate replicaset/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf Created pod: e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc 35m Normal Started pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc Container started 35m Normal AddedInterface pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc Add eth0 [10.134.0.143/23] from ovn-kubernetes 35m Normal Created pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc Container created 35m Normal Started pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-4kk54 Container started 35m Normal Created pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-4kk54 Container created 35m Normal Pulled pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-4kk54 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-4kk54 Add eth0 [10.132.0.198/23] from ovn-kubernetes 35m Normal Pulled pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Killing pod/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-mcc6n Stopping container payload-pre-processing 35m Warning Unhealthy pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc Startup probe failed: Get "http://10.134.0.143:15021/healthz/ready": dial tcp 10.134.0.143:15021: connect: connection refused 35m Warning Unhealthy pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-4kk54 Startup probe failed: Get "http://10.132.0.198:15021/healthz/ready": dial tcp 10.132.0.198:15021: connect: connection refused 35m Normal Killing pod/payload-processing-e2e-mt-6df3ba0e-84d7646477-6qgzr Stopping container payload-processing 35m Warning ProbeError pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc Startup probe error: Get "http://10.134.0.143:15021/healthz/ready": dial tcp 10.134.0.143:15021: connect: connection refused... 35m Warning ProbeError pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-4kk54 Startup probe error: Get "http://10.132.0.198:15021/healthz/ready": dial tcp 10.132.0.198:15021: connect: connection refused... 35m Normal ScalingReplicaSet deployment/e2e-mt-6df3ba0e-openshift-default Scaled up replica set e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf from 4 to 8 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-6df3ba0e-openshift-default New size: 8; reason: All metrics below target 35m Normal SuccessfulCreate replicaset/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf Created pod: e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb 35m Normal SuccessfulCreate replicaset/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf Created pod: e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-npttq 35m Normal SuccessfulCreate replicaset/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf Created pod: e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt 35m Normal Created pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt Container created 35m Normal Started pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt Container started 35m Normal Started pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-wc8sm Container started 35m Normal AddedInterface pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-npttq Add eth0 [10.132.0.199/23] from ovn-kubernetes 35m Normal Created pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-wc8sm Container created 35m Normal SuccessfulCreate replicaset/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf Created pod: e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-wc8sm 35m Normal Pulled pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-wc8sm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-wc8sm Add eth0 [10.132.0.200/23] from ovn-kubernetes 35m Normal Started pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-npttq Container started 35m Normal Pulled pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt Add eth0 [10.134.0.144/23] from ovn-kubernetes 35m Normal Created pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-npttq Container created 35m Normal Created pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb Container created 35m Normal Pulled pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-npttq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Pulled pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Started pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb Container started 35m Normal AddedInterface pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb Add eth0 [10.133.0.119/23] from ovn-kubernetes 35m Warning ProbeError pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt Startup probe error: Get "http://10.134.0.144:15021/healthz/ready": dial tcp 10.134.0.144:15021: connect: connection refused... 35m Warning ProbeError pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb Startup probe error: Get "http://10.133.0.119:15021/healthz/ready": dial tcp 10.133.0.119:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt Startup probe failed: Get "http://10.134.0.144:15021/healthz/ready": dial tcp 10.134.0.144:15021: connect: connection refused 35m Warning Unhealthy pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb Startup probe failed: Get "http://10.133.0.119:15021/healthz/ready": dial tcp 10.133.0.119:15021: connect: connection refused 35m Warning Unhealthy pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-npttq Startup probe failed: Get "http://10.132.0.199:15021/healthz/ready": dial tcp 10.132.0.199:15021: connect: connection refused 35m Warning ProbeError pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-npttq Startup probe error: Get "http://10.132.0.199:15021/healthz/ready": dial tcp 10.132.0.199:15021: connect: connection refused... 35m Warning ProbeError pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-wc8sm Startup probe error: Get "http://10.132.0.200:15021/healthz/ready": dial tcp 10.132.0.200:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-wc8sm Startup probe failed: Get "http://10.132.0.200:15021/healthz/ready": dial tcp 10.132.0.200:15021: connect: connection refused 35m Normal Killing pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x86qc Stopping container istio-proxy 35m Normal Killing pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-8x7w9 Stopping container istio-proxy 35m Normal AddedInterface pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq Add eth0 [10.133.0.120/23] from ovn-kubernetes 35m Normal ScalingReplicaSet deployment/e2e-mt-6df3ba0e-openshift-default Scaled up replica set e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf from 8 to 10 35m Normal SuccessfulCreate replicaset/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf Created pod: e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq 35m Normal Pulled pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk Add eth0 [10.134.0.145/23] from ovn-kubernetes 35m Normal Pulled pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-6df3ba0e-openshift-default New size: 10; reason: 35m Normal SuccessfulCreate replicaset/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf Created pod: e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk 35m Normal Started pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq Container started 35m Normal Started pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk Container started 35m Normal Created pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk Container created 35m Normal Created pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq Container created 35m Normal Killing pod/e2e-api-b-307368-openshift-default-748d46b5cc-hzgkz Stopping container istio-proxy 35m Warning Unhealthy pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk Startup probe failed: Get "http://10.134.0.145:15021/healthz/ready": dial tcp 10.134.0.145:15021: connect: connection refused 35m Warning ProbeError pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk Startup probe error: Get "http://10.134.0.145:15021/healthz/ready": dial tcp 10.134.0.145:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq Startup probe failed: Get "http://10.133.0.120:15021/healthz/ready": dial tcp 10.133.0.120:15021: connect: connection refused 35m Warning ProbeError pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq Startup probe error: Get "http://10.133.0.120:15021/healthz/ready": dial tcp 10.133.0.120:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-api-b-307368-openshift-default-748d46b5cc-hzgkz Readiness probe failed: Get "http://10.132.0.162:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 35m Warning ProbeError pod/e2e-api-b-307368-openshift-default-748d46b5cc-hzgkz Readiness probe error: Get "http://10.132.0.162:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 35m Normal Killing pod/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5 Stopping container payload-processing 35m Normal Killing pod/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp Stopping container payload-processing 35m Normal Killing pod/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h Stopping container payload-pre-processing 34m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning Unhealthy pod/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Normal Pulled pod/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Created pod/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs Container created 34m Normal Started pod/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs Container started 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-954628-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-954628-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Normal Killing pod/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx Stopping container payload-pre-processing 34m Warning ProbeError pod/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs Startup probe error: Get "http://10.134.0.142:15021/healthz/ready": dial tcp 10.134.0.142:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs Startup probe failed: Get "http://10.134.0.142:15021/healthz/ready": dial tcp 10.134.0.142:15021: connect: connection refused 34m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning Unhealthy pod/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-954628-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-954628-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Normal Killing pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-6xb26 Stopping container istio-proxy 34m Normal Killing pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf Stopping container istio-proxy 33m Normal Killing pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-xb49c Stopping container istio-proxy 33m Normal Killing pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-d8zwv Stopping container istio-proxy 33m Warning ProbeError pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-xb49c Readiness probe error: Get "http://10.132.0.122:15021/healthz/ready": dial tcp 10.132.0.122:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-xb49c Readiness probe failed: Get "http://10.132.0.122:15021/healthz/ready": dial tcp 10.132.0.122:15021: connect: connection refused 33m Normal Killing pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt Stopping container istio-proxy 33m Normal Killing pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb Stopping container istio-proxy 33m Normal Killing pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk Stopping container istio-proxy 33m Normal Killing pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb Stopping container istio-proxy 33m Normal Killing pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq Stopping container istio-proxy 33m Normal Killing pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-wc8sm Stopping container istio-proxy 33m Normal Killing pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-npttq Stopping container istio-proxy 33m Normal Killing pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-4kk54 Stopping container istio-proxy 33m Normal Killing pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb Stopping container istio-proxy 33m Normal Killing pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc Stopping container istio-proxy 33m Warning Unhealthy pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-npttq Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-npttq Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning ProbeError pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning ProbeError pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fmmkb Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-78rwq Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-8wmfb Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning Unhealthy pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-4kk54 Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-4kk54 Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-wc8sm Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-wc8sm Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb Readiness probe error: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb Readiness probe failed: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-c79cb5-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 33m Normal ScalingReplicaSet deployment/e2e-worker-w2-c79cb5-openshift-default (combined from similar events): Scaled up replica set e2e-worker-w2-c79cb5-openshift-default-5464c54587 from 2 to 4 33m Normal Pulled pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp Container created 33m Normal Started pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp Container started 33m Normal Started pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr Container started 33m Normal SuccessfulCreate replicaset/e2e-worker-w2-c79cb5-openshift-default-5464c54587 Created pod: e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp 33m Normal AddedInterface pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp Add eth0 [10.134.0.153/23] from ovn-kubernetes 33m Normal SuccessfulCreate replicaset/e2e-worker-w2-c79cb5-openshift-default-5464c54587 Created pod: e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr 33m Normal Created pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr Container created 33m Normal Pulled pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr Add eth0 [10.133.0.124/23] from ovn-kubernetes 33m Warning Unhealthy pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr Startup probe failed: Get "http://10.133.0.124:15021/healthz/ready": dial tcp 10.133.0.124:15021: connect: connection refused 33m Warning ProbeError pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr Startup probe error: Get "http://10.133.0.124:15021/healthz/ready": dial tcp 10.133.0.124:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp Startup probe failed: Get "http://10.134.0.153:15021/healthz/ready": dial tcp 10.134.0.153:15021: connect: connection refused 33m Warning ProbeError pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp Startup probe error: Get "http://10.134.0.153:15021/healthz/ready": dial tcp 10.134.0.153:15021: connect: connection refused... 33m Normal Killing pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-lg5tj Stopping container istio-proxy 33m Normal SuccessfulDelete replicaset/e2e-worker-w3-bc009f-openshift-default-645d5cbf67 Deleted pod: e2e-worker-w3-bc009f-openshift-default-645d5cbf67-lg5tj 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-bc009f-openshift-default New size: 1; reason: All metrics below target 33m Normal ScalingReplicaSet deployment/e2e-worker-w3-bc009f-openshift-default Scaled down replica set e2e-worker-w3-bc009f-openshift-default-645d5cbf67 from 2 to 1 33m Normal Killing pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-tppkr Stopping container istio-proxy 33m Normal Killing pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp Stopping container istio-proxy 33m Normal Killing pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k Stopping container istio-proxy 33m Normal Killing pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-4s94l Stopping container istio-proxy 33m Warning Unhealthy pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-4s94l Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-4s94l Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning ProbeError pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-8xc5k Readiness probe failed: HTTP probe failed with statuscode: 503 32m Normal Killing pod/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g Stopping container payload-processing 32m Normal Killing pod/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48 Stopping container payload-pre-processing 32m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Warning Unhealthy pod/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Normal Killing pod/e2e-worker-w1-80c366-openshift-default-666b7db455-9xsw6 Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk Stopping container istio-proxy 30m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-4z8hd Stopping container discovery 30m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-4z8hd 30m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 30m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 30m Normal Killing pod/payload-processing-e2e-inf-a-9da607-555bb58887-sv9b7 Stopping container payload-processing 30m Normal Killing pod/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p Stopping container payload-pre-processing 30m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-9da607-74c77fc695-kfq9p Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Normal Killing pod/e2e-inf-a-9da607-openshift-default-5d68dbc6c7-jld42 Stopping container istio-proxy 29m Warning ProbeError pod/e2e-inf-a-9da607-openshift-default-5d68dbc6c7-jld42 Readiness probe error: HTTP probe failed with statuscode: 503... 29m Warning Unhealthy pod/e2e-inf-a-9da607-openshift-default-5d68dbc6c7-jld42 Readiness probe failed: HTTP probe failed with statuscode: 503 29m Normal Killing pod/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87 Stopping container payload-pre-processing 29m Normal Killing pod/payload-processing-e2e-inf-b-954628-648fb887cf-h5q6c Stopping container payload-processing 29m Normal Killing pod/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs Stopping container istio-proxy 29m Warning Unhealthy pod/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs Readiness probe failed: HTTP probe failed with statuscode: 503 29m Warning ProbeError pod/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs Readiness probe error: HTTP probe failed with statuscode: 503... 29m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-954628-6dc9b97b8c-k6r87 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Normal Killing pod/payload-pre-processing-e2e-shared-a-w3-326d76-5c8655cdb7-j9q6p Stopping container payload-pre-processing 29m Normal Killing pod/payload-processing-e2e-shared-a-w3-326d76-79f9b8946c-8g6bp Stopping container payload-processing 28m Normal Killing pod/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x Stopping container istio-proxy 28m Warning Unhealthy pod/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning ProbeError pod/e2e-shared-a-w3-326d76-openshift-default-fdd7fd877-2688x Readiness probe error: HTTP probe failed with statuscode: 503... 28m Normal Killing pod/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94 Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg Stopping container payload-pre-processing 28m Warning Unhealthy pod/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94 Readiness probe failed: timeout: failed to connect service "10.134.0.124:50052" within 1s: context deadline exceeded 28m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg Readiness probe failed: timeout: failed to connect service "10.134.0.123:50052" within 1s: context deadline exceeded 28m Normal Killing pod/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7 Stopping container istio-proxy 27m Normal Killing pod/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b Stopping container payload-pre-processing 27m Normal Killing pod/payload-processing-e2e-worker-w3-bc009f-66fdc54f65-88jlx Stopping container payload-processing 27m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-74854d8ccf Deleted pod: maas-default-gateway-openshift-default-74854d8ccf-pptz4 27m Normal Killing pod/maas-default-gateway-openshift-default-74854d8ccf-24vpd Stopping container istio-proxy 27m Normal Killing pod/maas-default-gateway-openshift-default-74854d8ccf-pptz4 Stopping container istio-proxy 27m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-74854d8ccf Deleted pod: maas-default-gateway-openshift-default-74854d8ccf-fwnc4 27m Normal Killing pod/maas-default-gateway-openshift-default-74854d8ccf-44qk9 Stopping container istio-proxy 27m Normal Killing pod/maas-default-gateway-openshift-default-74854d8ccf-fwnc4 Stopping container istio-proxy 27m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: All metrics below target 27m Normal Killing pod/maas-default-gateway-openshift-default-74854d8ccf-f2rlm Stopping container istio-proxy 27m Normal Killing pod/maas-default-gateway-openshift-default-74854d8ccf-vxnxc Stopping container istio-proxy 27m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-24vpd Readiness probe error: HTTP probe failed with statuscode: 503... 27m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-24vpd Readiness probe failed: HTTP probe failed with statuscode: 503 27m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Warning Unhealthy pod/payload-processing-e2e-worker-w3-bc009f-66fdc54f65-88jlx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal Killing pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-4l5tt Stopping container istio-proxy 26m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-wwv4p Add eth0 [10.134.0.160/23] from ovn-kubernetes 26m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-zbxls Add eth0 [10.132.0.219/23] from ovn-kubernetes 26m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-zbxls Container created 26m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-wwv4p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-wwv4p Container started 26m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-46ldm Container created 26m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-46ldm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-46ldm Add eth0 [10.134.0.162/23] from ovn-kubernetes 26m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-46ldm Container started 26m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-55hwj Add eth0 [10.134.0.161/23] from ovn-kubernetes 26m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-55hwj Container started 26m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-55hwj Container created 26m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-wwv4p Container created 26m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-zbxls Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-55hwj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-zbxls Container started 26m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-zbxls Startup probe failed: Get "http://10.132.0.219:15021/healthz/ready": dial tcp 10.132.0.219:15021: connect: connection refused 26m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-55hwj Startup probe error: Get "http://10.134.0.161:15021/healthz/ready": dial tcp 10.134.0.161:15021: connect: connection refused... 26m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-zbxls Startup probe error: Get "http://10.132.0.219:15021/healthz/ready": dial tcp 10.132.0.219:15021: connect: connection refused... 26m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-55hwj Startup probe failed: Get "http://10.134.0.161:15021/healthz/ready": dial tcp 10.134.0.161:15021: connect: connection refused 26m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-wwv4p Startup probe failed: Get "http://10.134.0.160:15021/healthz/ready": dial tcp 10.134.0.160:15021: connect: connection refused 26m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-wwv4p Startup probe error: Get "http://10.134.0.160:15021/healthz/ready": dial tcp 10.134.0.160:15021: connect: connection refused... 26m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-46ldm Startup probe failed: Get "http://10.134.0.162:15021/healthz/ready": dial tcp 10.134.0.162:15021: connect: connection refused 26m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-46ldm Startup probe error: Get "http://10.134.0.162:15021/healthz/ready": dial tcp 10.134.0.162:15021: connect: connection refused... 26m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-zbxls Startup probe failed: Get "http://10.132.0.219:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 26m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-zbxls Startup probe error: Get "http://10.132.0.219:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 26m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-wwv4p Startup probe failed: Get "http://10.134.0.160:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 26m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-46ldm Startup probe error: Get "http://10.134.0.162:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 26m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-46ldm Startup probe failed: Get "http://10.134.0.162:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 26m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-wwv4p Startup probe error: Get "http://10.134.0.160:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 26m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-55hwj Startup probe error: Get "http://10.134.0.161:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 26m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-55hwj Startup probe failed: Get "http://10.134.0.161:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 26m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default (combined from similar events): Scaled up replica set maas-default-gateway-openshift-default-74854d8ccf from 8 to 10 26m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default (combined from similar events): New size: 10; reason: All metrics below target 26m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-f5gbx Add eth0 [10.134.0.163/23] from ovn-kubernetes 26m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-f5gbx Container started 26m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-f5gbx Container created 26m Normal Created pod/maas-default-gateway-openshift-default-74854d8ccf-v6jrr Container created 26m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-f5gbx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/maas-default-gateway-openshift-default-74854d8ccf-v6jrr Add eth0 [10.132.0.220/23] from ovn-kubernetes 26m Normal Started pod/maas-default-gateway-openshift-default-74854d8ccf-v6jrr Container started 26m Normal Pulled pod/maas-default-gateway-openshift-default-74854d8ccf-v6jrr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-v6jrr Startup probe error: Get "http://10.132.0.220:15021/healthz/ready": dial tcp 10.132.0.220:15021: connect: connection refused... 26m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-v6jrr Startup probe failed: Get "http://10.132.0.220:15021/healthz/ready": dial tcp 10.132.0.220:15021: connect: connection refused 26m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-f5gbx Startup probe error: Get "http://10.134.0.163:15021/healthz/ready": dial tcp 10.134.0.163:15021: connect: connection refused... 26m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-f5gbx Startup probe failed: Get "http://10.134.0.163:15021/healthz/ready": dial tcp 10.134.0.163:15021: connect: connection refused 22m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-55hwj Readiness probe error: Get "http://10.134.0.161:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 22m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-46ldm Readiness probe error: Get "http://10.134.0.162:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 22m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-55hwj Readiness probe failed: Get "http://10.134.0.161:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 22m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-wwv4p Readiness probe error: Get "http://10.134.0.160:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 22m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-46ldm Readiness probe failed: Get "http://10.134.0.162:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 22m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-wwv4p Readiness probe failed: Get "http://10.134.0.160:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 19m Normal ScalingReplicaSet deployment/e2e-worker-single-main-5b7354-openshift-default Scaled up replica set e2e-worker-single-main-5b7354-openshift-default-dc46bff89 from 0 to 1 19m Normal AddedLabel gateway/e2e-worker-single-main-5b7354 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-5b7354 19m Normal SuccessfulCreate replicaset/e2e-worker-single-main-5b7354-openshift-default-dc46bff89 Created pod: e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f 19m Normal AddedInterface pod/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f Add eth0 [10.134.0.179/23] from ovn-kubernetes 18m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-5b7354 Scaled up replica set payload-processing-e2e-worker-single-main-5b7354-6bdff759 from 0 to 1 18m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-5b7354-6bdff759 Error creating: pods "payload-processing-e2e-worker-single-main-5b7354-6bdff759-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-5b7354: serviceaccount "payload-processing-e2e-worker-single-main-5b7354" not found 18m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fd567bc Created pod: payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk 18m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fd567bc Error creating: pods "payload-pre-processing-e2e-worker-single-main-5b7354-6c4fd567bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-5b7354: serviceaccount "payload-processing-e2e-worker-single-main-5b7354" not found 18m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-5b7354 Scaled up replica set payload-pre-processing-e2e-worker-single-main-5b7354-6c4fd567bc from 0 to 1 18m Normal Started pod/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp Container started 18m Normal Pulled pod/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 18m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk Add eth0 [10.133.0.128/23] from ovn-kubernetes 18m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 18m Normal Created pod/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp Container created 18m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-5b7354-6bdff759 Created pod: payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp 18m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp Add eth0 [10.133.0.129/23] from ovn-kubernetes 18m Normal Started pod/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk Container started 18m Normal Created pod/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk Container created 18m Normal Created pod/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f Container created 18m Normal Started pod/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f Container started 18m Normal Pulled pod/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Warning Unhealthy pod/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f Startup probe failed: Get "http://10.134.0.179:15021/healthz/ready": dial tcp 10.134.0.179:15021: connect: connection refused 18m Warning ProbeError pod/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f Startup probe error: Get "http://10.134.0.179:15021/healthz/ready": dial tcp 10.134.0.179:15021: connect: connection refused... 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-5b7354-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-5b7354-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 17m Normal Killing pod/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp Stopping container payload-processing 17m Normal Killing pod/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk Stopping container payload-pre-processing 17m Warning FailedToRetrieveImagePullSecret pod/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f Unable to retrieve some image pull secrets (e2e-worker-single-main-5b7354-openshift-default-dockercfg-8ddvp); attempting to pull the image may not succeed. 17m Normal Killing pod/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f Stopping container istio-proxy 17m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-5b7354-6bdff759-bp8xp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-5b7354-6c4fdnmlwk Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Normal SuccessfulCreate replicaset/e2e-worker-single-main-aafbca-openshift-default-675587d5bf Created pod: e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh 14m Normal ScalingReplicaSet deployment/e2e-worker-single-main-aafbca-openshift-default Scaled up replica set e2e-worker-single-main-aafbca-openshift-default-675587d5bf from 0 to 1 14m Normal AddedLabel gateway/e2e-worker-single-main-aafbca Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-aafbca 14m Normal AddedInterface pod/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh Add eth0 [10.133.0.131/23] from ovn-kubernetes 13m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87 Error creating: pods "payload-processing-e2e-worker-single-main-aafbca-c88b9ff87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-aafbca: serviceaccount "payload-processing-e2e-worker-single-main-aafbca" not found 13m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75df6c9 Created pod: payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z 13m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75df6c9 Error creating: pods "payload-pre-processing-e2e-worker-single-main-aafbca-5bd75df6c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-aafbca: serviceaccount "payload-processing-e2e-worker-single-main-aafbca" not found 13m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-aafbca Scaled up replica set payload-processing-e2e-worker-single-main-aafbca-c88b9ff87 from 0 to 1 13m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87 Created pod: payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr 13m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-aafbca Scaled up replica set payload-pre-processing-e2e-worker-single-main-aafbca-5bd75df6c9 from 0 to 1 13m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z Add eth0 [10.133.0.132/23] from ovn-kubernetes 13m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr Add eth0 [10.133.0.133/23] from ovn-kubernetes 13m Normal Created pod/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr Container created 13m Normal Started pod/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr Container started 13m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 13m Normal Pulled pod/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 13m Normal Created pod/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z Container created 13m Normal Started pod/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z Container started 13m Normal Started pod/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh Container started 13m Normal Pulled pod/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Normal Created pod/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh Container created 13m Warning ProbeError pod/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh Startup probe error: Get "http://10.133.0.131:15021/healthz/ready": dial tcp 10.133.0.131:15021: connect: connection refused... 13m Warning Unhealthy pod/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh Startup probe failed: Get "http://10.133.0.131:15021/healthz/ready": dial tcp 10.133.0.131:15021: connect: connection refused 13m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-aafbca-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 13m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-aafbca-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 13m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-aafbca-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 13m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-aafbca-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 12m Normal Killing pod/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr Stopping container payload-processing 12m Normal Killing pod/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z Stopping container payload-pre-processing 12m Normal Killing pod/e2e-worker-single-main-aafbca-openshift-default-675587d5bfgvpdh Stopping container istio-proxy 12m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-aafbca-c88b9ff87dz6rr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 12m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-aafbca-5bd75hrb5z Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-nvkls Readiness probe failed: Get "http://10.133.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 11m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-nvkls Readiness probe error: Get "http://10.133.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-74854d8ccf-pxw88 Readiness probe failed: Get "http://10.133.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 11m Warning ProbeError pod/maas-default-gateway-openshift-default-74854d8ccf-pxw88 Readiness probe error: Get "http://10.133.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 8m58s Normal AddedLabel gateway/e2e-ait-5ff4cfa5 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-5ff4cfa5 8m58s Normal ScalingReplicaSet deployment/e2e-ait-5ff4cfa5-openshift-default Scaled up replica set e2e-ait-5ff4cfa5-openshift-default-7688c44f49 from 0 to 1 8m58s Normal SuccessfulCreate replicaset/e2e-ait-5ff4cfa5-openshift-default-7688c44f49 Created pod: e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv 8m58s Normal AddedInterface pod/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv Add eth0 [10.133.0.137/23] from ovn-kubernetes 8m52s Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-5ff4cfa5 Scaled up replica set payload-processing-e2e-ait-5ff4cfa5-97fcb7787 from 0 to 1 8m51s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-5ff4cfa5 Scaled up replica set payload-pre-processing-e2e-ait-5ff4cfa5-7bdc4f6fb7 from 0 to 1 8m51s Warning FailedCreate replicaset/payload-processing-e2e-ait-5ff4cfa5-97fcb7787 Error creating: pods "payload-processing-e2e-ait-5ff4cfa5-97fcb7787-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-5ff4cfa5: serviceaccount "payload-processing-e2e-ait-5ff4cfa5" not found 8m51s Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-5ff4cfa5-7bdc4f6fb7 Error creating: pods "payload-pre-processing-e2e-ait-5ff4cfa5-7bdc4f6fb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-5ff4cfa5: serviceaccount "payload-processing-e2e-ait-5ff4cfa5" not found 8m51s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-5ff4cfa5-7bdc4f6fb7 Created pod: payload-pre-processing-e2e-ait-5ff4cfa5-7bdc4f6fb7-ktnhr 8m51s Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-5ff4cfa5-97fcb7787 Created pod: payload-processing-e2e-ait-5ff4cfa5-97fcb7787-4dfjj 8m50s Normal Pulled pod/payload-pre-processing-e2e-ait-5ff4cfa5-7bdc4f6fb7-ktnhr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m50s Normal AddedInterface pod/payload-processing-e2e-ait-5ff4cfa5-97fcb7787-4dfjj Add eth0 [10.132.0.225/23] from ovn-kubernetes 8m50s Normal Pulled pod/payload-processing-e2e-ait-5ff4cfa5-97fcb7787-4dfjj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m50s Normal AddedInterface pod/payload-pre-processing-e2e-ait-5ff4cfa5-7bdc4f6fb7-ktnhr Add eth0 [10.132.0.224/23] from ovn-kubernetes 8m50s Normal Started pod/payload-pre-processing-e2e-ait-5ff4cfa5-7bdc4f6fb7-ktnhr Container started 8m50s Normal Created pod/payload-processing-e2e-ait-5ff4cfa5-97fcb7787-4dfjj Container created 8m50s Normal Created pod/payload-pre-processing-e2e-ait-5ff4cfa5-7bdc4f6fb7-ktnhr Container created 8m50s Normal Started pod/payload-processing-e2e-ait-5ff4cfa5-97fcb7787-4dfjj Container started 8m32s Normal Pulled pod/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 8m32s Normal Created pod/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv Container created 8m32s Normal Started pod/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv Container started 8m28s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-5ff4cfa5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 8m28s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-5ff4cfa5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 8m27s Warning ProbeError pod/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv Startup probe error: Get "http://10.133.0.137:15021/healthz/ready": dial tcp 10.133.0.137:15021: connect: connection refused... 8m27s Warning Unhealthy pod/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv Startup probe failed: Get "http://10.133.0.137:15021/healthz/ready": dial tcp 10.133.0.137:15021: connect: connection refused 7m58s Normal Killing pod/payload-pre-processing-e2e-ait-5ff4cfa5-7bdc4f6fb7-ktnhr Stopping container payload-pre-processing 7m58s Normal Killing pod/payload-processing-e2e-ait-5ff4cfa5-97fcb7787-4dfjj Stopping container payload-processing 7m58s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-5ff4cfa5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 7m58s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-5ff4cfa5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 7m47s Normal Killing pod/e2e-ait-5ff4cfa5-openshift-default-7688c44f49-rjrsv Stopping container istio-proxy 7m36s Normal AddedLabel gateway/e2e-authcfg-http-f0b1ab Added label istio.io/rev=openshift-gateway to gateway e2e-authcfg-http-f0b1ab 7m35s Normal AddedInterface pod/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk Add eth0 [10.133.0.139/23] from ovn-kubernetes 7m35s Normal SuccessfulCreate replicaset/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c Created pod: e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk 7m35s Normal ScalingReplicaSet deployment/e2e-authcfg-http-f0b1ab-openshift-default Scaled up replica set e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c from 0 to 1 7m28s Normal ScalingReplicaSet deployment/payload-processing-e2e-authcfg-http-f0b1ab Scaled up replica set payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f from 0 to 1 7m27s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-authcfg-http-f0b1ab Scaled up replica set payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599 from 0 to 1 7m27s Warning FailedCreate replicaset/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f Error creating: pods "payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab: serviceaccount "payload-processing-e2e-authcfg-http-f0b1ab" not found 7m27s Normal SuccessfulCreate replicaset/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f Created pod: payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5 7m27s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599 Created pod: payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf 7m27s Warning FailedCreate replicaset/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599 Error creating: pods "payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-f0b1ab: serviceaccount "payload-processing-e2e-authcfg-http-f0b1ab" not found 7m26s Normal Started pod/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5 Container started 7m26s Normal AddedInterface pod/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5 Add eth0 [10.133.0.141/23] from ovn-kubernetes 7m26s Normal Pulled pod/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 7m26s Normal Created pod/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5 Container created 7m26s Normal Started pod/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf Container started 7m26s Normal Created pod/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf Container created 7m26s Normal Pulled pod/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 7m26s Normal AddedInterface pod/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf Add eth0 [10.133.0.140/23] from ovn-kubernetes 7m7s Normal Pulled pod/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 7m6s Normal Started pod/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk Container started 7m6s Normal Created pod/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk Container created 7m1s Warning Unhealthy pod/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk Startup probe failed: Get "http://10.133.0.139:15021/healthz/ready": dial tcp 10.133.0.139:15021: connect: connection refused 7m1s Warning ProbeError pod/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk Startup probe error: Get "http://10.133.0.139:15021/healthz/ready": dial tcp 10.133.0.139:15021: connect: connection refused... 6m50s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-f0b1ab-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 6m50s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-f0b1ab-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 6m35s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-f0b1ab-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 6m35s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-f0b1ab-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 5m35s Normal Killing pod/payload-processing-e2e-authcfg-http-f0b1ab-6f65b7cd9f-f9lf5 Stopping container payload-processing 5m35s Normal Killing pod/payload-pre-processing-e2e-authcfg-http-f0b1ab-7ffd489599-9fvpf Stopping container payload-pre-processing 5m14s Normal Killing pod/e2e-authcfg-http-f0b1ab-openshift-default-5d7f999f9c-jlvdk Stopping container istio-proxy