=== events in openshift-ingress (2026-11-04T03:41:49Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 47m Normal Scheduled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-gph88 Successfully assigned openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-gph88 to ip-10-0-134-125.us-west-1.compute.internal 47m Normal Scheduled pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r Successfully assigned openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r to ip-10-0-128-201.us-west-1.compute.internal 47m Normal Scheduled pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-vh6fd Successfully assigned openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-vh6fd to ip-10-0-134-125.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p Successfully assigned openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p to ip-10-0-128-201.us-west-1.compute.internal 50m Normal Scheduled pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw Successfully assigned openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw to ip-10-0-128-201.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-5mccl Successfully assigned openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-5mccl to ip-10-0-133-209.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-7pg2l Successfully assigned openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-7pg2l to ip-10-0-134-125.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh Successfully assigned openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh to ip-10-0-128-201.us-west-1.compute.internal 50m Normal Scheduled pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-9qgws Successfully assigned openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-9qgws to ip-10-0-133-209.us-west-1.compute.internal 57m Normal Scheduled pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-bjmn9 Successfully assigned openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-bjmn9 to ip-10-0-134-125.us-west-1.compute.internal 43m Normal Scheduled pod/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9 Successfully assigned openshift-ingress/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9 to ip-10-0-128-201.us-west-1.compute.internal 31m Normal Scheduled pod/e2e-mt-4cbcf738-openshift-default-68fb497c9c-2ndqr Successfully assigned openshift-ingress/e2e-mt-4cbcf738-openshift-default-68fb497c9c-2ndqr to ip-10-0-134-125.us-west-1.compute.internal 57m Normal Scheduled pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tzgt6 Successfully assigned openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tzgt6 to ip-10-0-133-209.us-west-1.compute.internal 64m Normal Scheduled pod/openshift-ai-inference-openshift-ai-inference-77d8c79567-w62d2 Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-77d8c79567-w62d2 to ip-10-0-133-209.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-migrate-2c1c4f09-gw-openshift-default-648b465cc-mr59v Successfully assigned openshift-ingress/e2e-migrate-2c1c4f09-gw-openshift-default-648b465cc-mr59v to ip-10-0-133-209.us-west-1.compute.internal 42m Normal Scheduled pod/e2e-mt-db4d0787-openshift-default-7bd494bcfc-zqfcj Successfully assigned openshift-ingress/e2e-mt-db4d0787-openshift-default-7bd494bcfc-zqfcj to ip-10-0-133-209.us-west-1.compute.internal 64m Normal Scheduled pod/maas-default-gateway-openshift-default-76597d6855-zhmt8 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-76597d6855-zhmt8 to ip-10-0-133-209.us-west-1.compute.internal 57m Normal Scheduled pod/maas-default-gateway-openshift-default-76597d6855-z29xk Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-76597d6855-z29xk to ip-10-0-128-201.us-west-1.compute.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-76597d6855-xrlzk Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xrlzk to ip-10-0-128-201.us-west-1.compute.internal 43m Normal Scheduled pod/maas-default-gateway-openshift-default-76597d6855-xr2tj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xr2tj to ip-10-0-133-209.us-west-1.compute.internal 22m Normal Scheduled pod/maas-default-gateway-openshift-default-76597d6855-xjqqw Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-76597d6855-xjqqw to ip-10-0-134-125.us-west-1.compute.internal 58m Normal Scheduled pod/payload-pre-processing-7b98bb678c-r82l2 Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-r82l2 to ip-10-0-133-209.us-west-1.compute.internal 31m Normal Scheduled pod/maas-default-gateway-openshift-default-76597d6855-vxzxx Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-76597d6855-vxzxx to ip-10-0-134-125.us-west-1.compute.internal 30m Normal Scheduled pod/e2e-inf-b-095081-openshift-default-59dfcb6dcf-vf8xr Successfully assigned openshift-ingress/e2e-inf-b-095081-openshift-default-59dfcb6dcf-vf8xr to ip-10-0-133-209.us-west-1.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-76597d6855-v59qw Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-76597d6855-v59qw to ip-10-0-133-209.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-ait-87191e54-openshift-default-7c59fcc9cc-6s8gq Successfully assigned openshift-ingress/e2e-ait-87191e54-openshift-default-7c59fcc9cc-6s8gq to ip-10-0-134-125.us-west-1.compute.internal 40m Normal Scheduled pod/e2e-shared-a-w1-551e2b-openshift-default-5cb6c84ff7-bxgnt Successfully assigned openshift-ingress/e2e-shared-a-w1-551e2b-openshift-default-5cb6c84ff7-bxgnt to ip-10-0-133-209.us-west-1.compute.internal 59m Normal Scheduled pod/maas-default-gateway-openshift-default-76597d6855-ts4v7 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-76597d6855-ts4v7 to ip-10-0-128-201.us-west-1.compute.internal 40m Normal Scheduled pod/e2e-shared-a-w2-57f116-openshift-default-75f4fc499-znglh Successfully assigned openshift-ingress/e2e-shared-a-w2-57f116-openshift-default-75f4fc499-znglh to ip-10-0-134-125.us-west-1.compute.internal 69m Normal Scheduled pod/router-default-76598c4567-mhntk Successfully assigned openshift-ingress/router-default-76598c4567-mhntk to ip-10-0-133-209.us-west-1.compute.internal 69m Warning FailedScheduling pod/router-default-76598c4567-mhntk no nodes available to schedule pods 75m Warning FailedScheduling pod/router-default-76598c4567-mhntk no nodes available to schedule pods 53m Normal Scheduled pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-fc8wx Successfully assigned openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-fc8wx to ip-10-0-133-209.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6 Successfully assigned openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6 to ip-10-0-128-201.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-pnfhp Successfully assigned openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-pnfhp to ip-10-0-134-125.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs Successfully assigned openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs to ip-10-0-128-201.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-w2plx Successfully assigned openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-w2plx to ip-10-0-134-125.us-west-1.compute.internal 50m Normal Scheduled pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-xg2tv Successfully assigned openshift-ingress/e2e-worker-w1-e516ff-openshift-default-6f4b957485-xg2tv to ip-10-0-134-125.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-7mhsd Successfully assigned openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-7mhsd to ip-10-0-133-209.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-mmngc Successfully assigned openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-mmngc to ip-10-0-134-125.us-west-1.compute.internal 6m11s Normal Scheduled pod/e2e-ait-dffbe0c3-openshift-default-857cd74dd7-5sn8z Successfully assigned openshift-ingress/e2e-ait-dffbe0c3-openshift-default-857cd74dd7-5sn8z to ip-10-0-133-209.us-west-1.compute.internal 56m Normal Scheduled pod/payload-processing-e2e-worker-w6-ebb7fd-b8fd547bb-7jkvh Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-ebb7fd-b8fd547bb-7jkvh to ip-10-0-134-125.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-bjdq9 Successfully assigned openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-bjdq9 to ip-10-0-133-209.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-fzwkp Successfully assigned openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-fzwkp to ip-10-0-134-125.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-g52rd Successfully assigned openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-g52rd to ip-10-0-134-125.us-west-1.compute.internal 23m Normal Scheduled pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-g5hw6 Successfully assigned openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-g5hw6 to ip-10-0-134-125.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-j4n6p Successfully assigned openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-j4n6p to ip-10-0-134-125.us-west-1.compute.internal 63m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-6545b4848996xtj Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-6545b4848996xtj to ip-10-0-133-209.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-m97sr Successfully assigned openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-m97sr to ip-10-0-134-125.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj Successfully assigned openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj to ip-10-0-128-201.us-west-1.compute.internal 56m Normal Scheduled pod/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw to ip-10-0-128-201.us-west-1.compute.internal 57m Normal Scheduled pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-v2tpq Successfully assigned openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-v2tpq to ip-10-0-133-209.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-ck68h Successfully assigned openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-ck68h to ip-10-0-133-209.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r Successfully assigned openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r to ip-10-0-128-201.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-8bhc5 Successfully assigned openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-8bhc5 to ip-10-0-133-209.us-west-1.compute.internal 45m Normal Scheduled pod/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l Successfully assigned openshift-ingress/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l to ip-10-0-128-201.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n Successfully assigned openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n to ip-10-0-128-201.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl Successfully assigned openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl to ip-10-0-128-201.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-975ss Successfully assigned openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-975ss to ip-10-0-134-125.us-west-1.compute.internal 56m Normal Scheduled pod/payload-processing-e2e-worker-w4-203c35-688854f56b-jk6wm Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-203c35-688854f56b-jk6wm to ip-10-0-133-209.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn Successfully assigned openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn to ip-10-0-128-201.us-west-1.compute.internal 54m Normal Scheduled pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-6hhrb Successfully assigned openshift-ingress/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-6hhrb to ip-10-0-134-125.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-d5qtm Successfully assigned openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-d5qtm to ip-10-0-133-209.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s Successfully assigned openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s to ip-10-0-128-201.us-west-1.compute.internal 57m Normal Scheduled pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-rwgvq Successfully assigned openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-rwgvq to ip-10-0-134-125.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-v5wsm Successfully assigned openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-v5wsm to ip-10-0-133-209.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-vbc52 Successfully assigned openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-vbc52 to ip-10-0-134-125.us-west-1.compute.internal 57m Normal Scheduled pod/payload-processing-e2e-worker-w3-cdcf67-76579b7df4-vqbkj Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-cdcf67-76579b7df4-vqbkj to ip-10-0-134-125.us-west-1.compute.internal 39m Normal Scheduled pod/e2e-api-a-569fb7-openshift-default-6dfd58f675-h2ttj Successfully assigned openshift-ingress/e2e-api-a-569fb7-openshift-default-6dfd58f675-h2ttj to ip-10-0-133-209.us-west-1.compute.internal 11m Normal Scheduled pod/e2e-worker-single-main-2f48c1-openshift-default-6799d44489xnlnj Successfully assigned openshift-ingress/e2e-worker-single-main-2f48c1-openshift-default-6799d44489xnlnj to ip-10-0-134-125.us-west-1.compute.internal 55m Normal Scheduled pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-w5ldf Successfully assigned openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-w5ldf to ip-10-0-133-209.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2b66r Successfully assigned openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2b66r to ip-10-0-134-125.us-west-1.compute.internal 47m Normal Scheduled pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27 Successfully assigned openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27 to ip-10-0-128-201.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt Successfully assigned openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt to ip-10-0-128-201.us-west-1.compute.internal 57m Normal Scheduled pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-825cw Successfully assigned openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-825cw to ip-10-0-133-209.us-west-1.compute.internal 50m Normal Scheduled pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-czkcm Successfully assigned openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-czkcm to ip-10-0-133-209.us-west-1.compute.internal 57m Normal Scheduled pod/payload-processing-e2e-worker-w2-a0bcbc-5bf4697664-l2vk9 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-a0bcbc-5bf4697664-l2vk9 to ip-10-0-133-209.us-west-1.compute.internal 15m Normal Scheduled pod/e2e-worker-single-main-07ce81-openshift-default-5446f4ff6fwvjql Successfully assigned openshift-ingress/e2e-worker-single-main-07ce81-openshift-default-5446f4ff6fwvjql to ip-10-0-134-125.us-west-1.compute.internal 47m Normal Scheduled pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-ljwqc Successfully assigned openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-ljwqc to ip-10-0-134-125.us-west-1.compute.internal 47m Normal Scheduled pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm Successfully assigned openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm to ip-10-0-128-201.us-west-1.compute.internal 47m Normal Scheduled pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-rsw2s Successfully assigned openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-rsw2s to ip-10-0-133-209.us-west-1.compute.internal 22m Normal Scheduled pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-ssz9p Successfully assigned openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-ssz9p to ip-10-0-133-209.us-west-1.compute.internal 47m Normal Scheduled pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-w6c48 Successfully assigned openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-w6c48 to ip-10-0-134-125.us-west-1.compute.internal 38m Normal Scheduled pod/e2e-api-a-a021b3-openshift-default-6476c6cc55-6zjzw Successfully assigned openshift-ingress/e2e-api-a-a021b3-openshift-default-6476c6cc55-6zjzw to ip-10-0-133-209.us-west-1.compute.internal 50m Normal Scheduled pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-z59vn Successfully assigned openshift-ingress/e2e-worker-w4-203c35-openshift-default-56497cdcc-z59vn to ip-10-0-134-125.us-west-1.compute.internal 56m Normal Scheduled pod/payload-processing-e2e-worker-w1-e516ff-57b6c4bc94-5nc6m Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-e516ff-57b6c4bc94-5nc6m to ip-10-0-133-209.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-52pkm Successfully assigned openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-52pkm to ip-10-0-133-209.us-west-1.compute.internal 31m Normal Scheduled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp Successfully assigned openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp to ip-10-0-128-201.us-west-1.compute.internal 47m Normal Scheduled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6n8hr Successfully assigned openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-6n8hr to ip-10-0-133-209.us-west-1.compute.internal 31m Normal Scheduled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-7zmxp Successfully assigned openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-7zmxp to ip-10-0-133-209.us-west-1.compute.internal 49m Normal Scheduled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-8kz5l Successfully assigned openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-8kz5l to ip-10-0-133-209.us-west-1.compute.internal 47m Normal Scheduled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg Successfully assigned openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg to ip-10-0-128-201.us-west-1.compute.internal 40m Normal Scheduled pod/e2e-shared-b-w5-4b2bf4-openshift-default-59785c9d99-b52gb Successfully assigned openshift-ingress/e2e-shared-b-w5-4b2bf4-openshift-default-59785c9d99-b52gb to ip-10-0-134-125.us-west-1.compute.internal 57m Normal Scheduled pod/payload-processing-e2e-worker-w0-fe4e0c-7dcb6cb4dd-vdb4p Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-fe4e0c-7dcb6cb4dd-vdb4p to ip-10-0-134-125.us-west-1.compute.internal 11m Normal Scheduled pod/payload-processing-e2e-worker-single-main-2f48c1-5c55d7989qtlvn Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-2f48c1-5c55d7989qtlvn to ip-10-0-134-125.us-west-1.compute.internal 15m Normal Scheduled pod/payload-processing-e2e-worker-single-main-07ce81-767654f68x8k9j Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-07ce81-767654f68x8k9j to ip-10-0-134-125.us-west-1.compute.internal 40m Normal Scheduled pod/payload-processing-e2e-shared-b-w5-4b2bf4-69448f59dd-fqq6g Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w5-4b2bf4-69448f59dd-fqq6g to ip-10-0-133-209.us-west-1.compute.internal 40m Normal Scheduled pod/payload-processing-e2e-shared-b-w4-35df3e-7969f6bd6b-ptzqg Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-35df3e-7969f6bd6b-ptzqg to ip-10-0-134-125.us-west-1.compute.internal 37m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-00c846-6865789c84-c24hp Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-00c846-6865789c84-c24hp to ip-10-0-134-125.us-west-1.compute.internal 37m Normal Scheduled pod/payload-processing-e2e-shared-b-w1-d2b8ee-56b894f96d-dbtft Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w1-d2b8ee-56b894f96d-dbtft to ip-10-0-134-125.us-west-1.compute.internal 38m Normal Scheduled pod/e2e-api-a-cdb11d-openshift-default-5ff9547799-2x65m Successfully assigned openshift-ingress/e2e-api-a-cdb11d-openshift-default-5ff9547799-2x65m to ip-10-0-133-209.us-west-1.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-shared-a-w5-297806-6b77744954-k4jwf Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w5-297806-6b77744954-k4jwf to ip-10-0-133-209.us-west-1.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8 to ip-10-0-128-201.us-west-1.compute.internal 40m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-57f116-5b5dd4bd44-fzgp7 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-57f116-5b5dd4bd44-fzgp7 to ip-10-0-134-125.us-west-1.compute.internal 40m Normal Scheduled pod/payload-processing-e2e-shared-a-w1-551e2b-79ddfccbf8-6rkbp Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w1-551e2b-79ddfccbf8-6rkbp to ip-10-0-134-125.us-west-1.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-mt-db4d0787-5558ff6d68-hdw6t Successfully assigned openshift-ingress/payload-processing-e2e-mt-db4d0787-5558ff6d68-hdw6t to ip-10-0-133-209.us-west-1.compute.internal 26m Normal Scheduled pod/payload-processing-e2e-mt-4cbcf738-97f795756-gk52w Successfully assigned openshift-ingress/payload-processing-e2e-mt-4cbcf738-97f795756-gk52w to ip-10-0-134-125.us-west-1.compute.internal 31m Normal Scheduled pod/payload-processing-e2e-mt-4cbcf738-97f795756-8vnkw Successfully assigned openshift-ingress/payload-processing-e2e-mt-4cbcf738-97f795756-8vnkw to ip-10-0-133-209.us-west-1.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-migrate-2c1c4f09-79984b5b5b-8bb4n Successfully assigned openshift-ingress/payload-processing-e2e-migrate-2c1c4f09-79984b5b5b-8bb4n to ip-10-0-133-209.us-west-1.compute.internal 30m Normal Scheduled pod/payload-processing-e2e-inf-b-095081-6c6bf65b7b-rdq44 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-095081-6c6bf65b7b-rdq44 to ip-10-0-134-125.us-west-1.compute.internal 33m Normal Scheduled pod/payload-processing-e2e-inf-a-db5ef5-657546bd74-dh9pg Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-db5ef5-657546bd74-dh9pg to ip-10-0-133-209.us-west-1.compute.internal 35m Normal Scheduled pod/payload-processing-e2e-inf-a-9f4199-55cf88d648-fr4zd Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-9f4199-55cf88d648-fr4zd to ip-10-0-134-125.us-west-1.compute.internal 37m Normal Scheduled pod/payload-processing-e2e-inf-a-675c80-698585c7d-b5hn7 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-675c80-698585c7d-b5hn7 to ip-10-0-133-209.us-west-1.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-inf-a-22e72e-85b6d5f746-nw6tv Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-22e72e-85b6d5f746-nw6tv to ip-10-0-134-125.us-west-1.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-derive-5fde04ee-5fdc7f866d-lk25n Successfully assigned openshift-ingress/payload-processing-e2e-derive-5fde04ee-5fdc7f866d-lk25n to ip-10-0-134-125.us-west-1.compute.internal 4m59s Normal Scheduled pod/payload-processing-e2e-authcfg-http-cc2474-684cc5b57d-fzbbc Successfully assigned openshift-ingress/payload-processing-e2e-authcfg-http-cc2474-684cc5b57d-fzbbc to ip-10-0-133-209.us-west-1.compute.internal 36m Normal Scheduled pod/payload-processing-e2e-api-b-9d2b96-7d8dd46f5b-7rbgg Successfully assigned openshift-ingress/payload-processing-e2e-api-b-9d2b96-7d8dd46f5b-7rbgg to ip-10-0-134-125.us-west-1.compute.internal 38m Normal Scheduled pod/payload-processing-e2e-api-a-cdb11d-87b64cbd4-bvfpd Successfully assigned openshift-ingress/payload-processing-e2e-api-a-cdb11d-87b64cbd4-bvfpd to ip-10-0-134-125.us-west-1.compute.internal 38m Normal Scheduled pod/payload-processing-e2e-api-a-a021b3-6556664496-wmwrg Successfully assigned openshift-ingress/payload-processing-e2e-api-a-a021b3-6556664496-wmwrg to ip-10-0-134-125.us-west-1.compute.internal 36m Normal Scheduled pod/e2e-api-b-9d2b96-openshift-default-cc56c4dff-6w9vk Successfully assigned openshift-ingress/e2e-api-b-9d2b96-openshift-default-cc56c4dff-6w9vk to ip-10-0-133-209.us-west-1.compute.internal 39m Normal Scheduled pod/payload-processing-e2e-api-a-569fb7-69bcd6f796-49qwt Successfully assigned openshift-ingress/payload-processing-e2e-api-a-569fb7-69bcd6f796-49qwt to ip-10-0-133-209.us-west-1.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-ait-f73e00ed-866fdf7694-m4mwj Successfully assigned openshift-ingress/payload-processing-e2e-ait-f73e00ed-866fdf7694-m4mwj to ip-10-0-134-125.us-west-1.compute.internal 6m3s Normal Scheduled pod/payload-processing-e2e-ait-dffbe0c3-7f8d8457b-mmn44 Successfully assigned openshift-ingress/payload-processing-e2e-ait-dffbe0c3-7f8d8457b-mmn44 to ip-10-0-134-125.us-west-1.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-ait-87191e54-77b46c874-mvmn9 Successfully assigned openshift-ingress/payload-processing-e2e-ait-87191e54-77b46c874-mvmn9 to ip-10-0-134-125.us-west-1.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-ait-4d5a0200-5cdf8ddbcd-wdl8z Successfully assigned openshift-ingress/payload-processing-e2e-ait-4d5a0200-5cdf8ddbcd-wdl8z to ip-10-0-134-125.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-f9flw Successfully assigned openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-f9flw to ip-10-0-133-209.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-hht95 Successfully assigned openshift-ingress/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-hht95 to ip-10-0-133-209.us-west-1.compute.internal 50m Normal Scheduled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5 Successfully assigned openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5 to ip-10-0-128-201.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k Successfully assigned openshift-ingress/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k to ip-10-0-128-201.us-west-1.compute.internal 31m Normal Scheduled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56 Successfully assigned openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56 to ip-10-0-128-201.us-west-1.compute.internal 57m Normal Scheduled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-msl8m Successfully assigned openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-msl8m to ip-10-0-134-125.us-west-1.compute.internal 31m Normal Scheduled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-q7gfm Successfully assigned openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-q7gfm to ip-10-0-134-125.us-west-1.compute.internal 58m Normal Scheduled pod/payload-processing-7bb9489676-bqh9n Successfully assigned openshift-ingress/payload-processing-7bb9489676-bqh9n to ip-10-0-133-209.us-west-1.compute.internal 47m Normal Scheduled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-vxwkm Successfully assigned openshift-ingress/e2e-worker-w5-3b7476-openshift-default-59975457bf-vxwkm to ip-10-0-134-125.us-west-1.compute.internal 5m8s Normal Scheduled pod/e2e-authcfg-http-cc2474-openshift-default-7f6458856f-qw52z Successfully assigned openshift-ingress/e2e-authcfg-http-cc2474-openshift-default-7f6458856f-qw52z to ip-10-0-134-125.us-west-1.compute.internal 40m Normal Scheduled pod/e2e-shared-b-w4-35df3e-openshift-default-6c585b76f-vl6wz Successfully assigned openshift-ingress/e2e-shared-b-w4-35df3e-openshift-default-6c585b76f-vl6wz to ip-10-0-134-125.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-47dj7 Successfully assigned openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-47dj7 to ip-10-0-134-125.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-4qkqm Successfully assigned openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-4qkqm to ip-10-0-133-209.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp Successfully assigned openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp to ip-10-0-128-201.us-west-1.compute.internal 50m Normal Scheduled pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql Successfully assigned openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql to ip-10-0-128-201.us-west-1.compute.internal 30m Normal Scheduled pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj Successfully assigned openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj to ip-10-0-128-201.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-db5nw Successfully assigned openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-db5nw to ip-10-0-133-209.us-west-1.compute.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n to ip-10-0-128-201.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-frgsz Successfully assigned openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-frgsz to ip-10-0-133-209.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-mngkl Successfully assigned openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-mngkl to ip-10-0-134-125.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw Successfully assigned openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw to ip-10-0-128-201.us-west-1.compute.internal 30m Normal Scheduled pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-r2z9z Successfully assigned openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-r2z9z to ip-10-0-133-209.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-w6tt7 Successfully assigned openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-w6tt7 to ip-10-0-134-125.us-west-1.compute.internal 37m Normal Scheduled pod/e2e-shared-b-w2-00c846-openshift-default-785fd9fcdc-kf4s5 Successfully assigned openshift-ingress/e2e-shared-b-w2-00c846-openshift-default-785fd9fcdc-kf4s5 to ip-10-0-133-209.us-west-1.compute.internal 57m Normal Scheduled pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-x449v Successfully assigned openshift-ingress/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-x449v to ip-10-0-133-209.us-west-1.compute.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-3b7476-56c5fdb86c-8786b Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-3b7476-56c5fdb86c-8786b to ip-10-0-133-209.us-west-1.compute.internal 43m Normal Scheduled pod/e2e-derive-5fde04ee-openshift-default-7d7db879c-bxx4z Successfully assigned openshift-ingress/e2e-derive-5fde04ee-openshift-default-7d7db879c-bxx4z to ip-10-0-133-209.us-west-1.compute.internal 64m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-wm9hl Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-wm9hl to ip-10-0-128-201.us-west-1.compute.internal 58m Normal Scheduled pod/maas-default-gateway-openshift-default-76597d6855-tlkdl Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tlkdl to ip-10-0-134-125.us-west-1.compute.internal 43m Normal Scheduled pod/maas-default-gateway-openshift-default-76597d6855-rlhz6 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-76597d6855-rlhz6 to ip-10-0-128-201.us-west-1.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-ait-4d5a0200-5959b9699-k2dpp Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-4d5a0200-5959b9699-k2dpp to ip-10-0-134-125.us-west-1.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-ait-87191e54-6ffd5f6b7-5qrkx Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-87191e54-6ffd5f6b7-5qrkx to ip-10-0-134-125.us-west-1.compute.internal 6m3s Normal Scheduled pod/payload-pre-processing-e2e-ait-dffbe0c3-546cf94567-9wrrk Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-dffbe0c3-546cf94567-9wrrk to ip-10-0-134-125.us-west-1.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv to ip-10-0-128-201.us-west-1.compute.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-203c35-5bb8b769bc-q9wmw Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-203c35-5bb8b769bc-q9wmw to ip-10-0-133-209.us-west-1.compute.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-api-a-569fb7-8594f58547-sfqmz Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-569fb7-8594f58547-sfqmz to ip-10-0-133-209.us-west-1.compute.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-api-a-a021b3-775855fc55-hd92k Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-a021b3-775855fc55-hd92k to ip-10-0-134-125.us-west-1.compute.internal 35m Normal Scheduled pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-vfrcc Successfully assigned openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-vfrcc to ip-10-0-133-209.us-west-1.compute.internal 42m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-2g6x2 Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2g6x2 to ip-10-0-134-125.us-west-1.compute.internal 45m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-t6h4m Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-t6h4m to ip-10-0-133-209.us-west-1.compute.internal 61m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-zch9j Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zch9j to ip-10-0-128-201.us-west-1.compute.internal 35m Normal Scheduled pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-qpj8z Successfully assigned openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-qpj8z to ip-10-0-134-125.us-west-1.compute.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-cdcf67-dc57c4455-ljnxm Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-cdcf67-dc57c4455-ljnxm to ip-10-0-133-209.us-west-1.compute.internal 34m Normal Scheduled pod/e2e-inf-a-22e72e-openshift-default-64656d9fb9-26n5c Successfully assigned openshift-ingress/e2e-inf-a-22e72e-openshift-default-64656d9fb9-26n5c to ip-10-0-134-125.us-west-1.compute.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-api-a-cdb11d-585c6fc6dc-dtbnf Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-cdb11d-585c6fc6dc-dtbnf to ip-10-0-134-125.us-west-1.compute.internal 63m Normal Scheduled pod/kube-auth-proxy-54497bf445-q5p8t Successfully assigned openshift-ingress/kube-auth-proxy-54497bf445-q5p8t to ip-10-0-128-201.us-west-1.compute.internal 37m Normal Scheduled pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9 Successfully assigned openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9 to ip-10-0-128-201.us-west-1.compute.internal 63m Normal Scheduled pod/kube-auth-proxy-54497bf445-qvs4m Successfully assigned openshift-ingress/kube-auth-proxy-54497bf445-qvs4m to ip-10-0-134-125.us-west-1.compute.internal 35m Normal Scheduled pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-5zskv Successfully assigned openshift-ingress/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-5zskv to ip-10-0-133-209.us-west-1.compute.internal 57m Normal Scheduled pod/maas-default-gateway-openshift-default-76597d6855-4ndqx Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4ndqx to ip-10-0-134-125.us-west-1.compute.internal 42m Normal Scheduled pod/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67 Successfully assigned openshift-ingress/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67 to ip-10-0-128-201.us-west-1.compute.internal 31m Normal Scheduled pod/maas-default-gateway-openshift-default-76597d6855-6cfhw Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-76597d6855-6cfhw to ip-10-0-128-201.us-west-1.compute.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-a0bcbc-69fdc5f5c9-d4d6x Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-a0bcbc-69fdc5f5c9-d4d6x to ip-10-0-134-125.us-west-1.compute.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-76597d6855-74hkj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-76597d6855-74hkj to ip-10-0-128-201.us-west-1.compute.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-76597d6855-7c95j Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-76597d6855-7c95j to ip-10-0-134-125.us-west-1.compute.internal 22m Normal Scheduled pod/maas-default-gateway-openshift-default-76597d6855-bdgtb Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-76597d6855-bdgtb to ip-10-0-133-209.us-west-1.compute.internal 44m Normal Scheduled pod/maas-default-gateway-openshift-default-76597d6855-brntt Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-76597d6855-brntt to ip-10-0-134-125.us-west-1.compute.internal 59m Normal Scheduled pod/maas-default-gateway-openshift-default-76597d6855-dhvd2 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-76597d6855-dhvd2 to ip-10-0-134-125.us-west-1.compute.internal 37m Normal Scheduled pod/e2e-inf-a-675c80-openshift-default-554bc6cd5c-fsdlt Successfully assigned openshift-ingress/e2e-inf-a-675c80-openshift-default-554bc6cd5c-fsdlt to ip-10-0-133-209.us-west-1.compute.internal 42m Normal Scheduled pod/e2e-shared-a-w4-072ece-openshift-default-78b784fd8c-k2vfq Successfully assigned openshift-ingress/e2e-shared-a-w4-072ece-openshift-default-78b784fd8c-k2vfq to ip-10-0-134-125.us-west-1.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-76597d6855-dqn2b Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-76597d6855-dqn2b to ip-10-0-134-125.us-west-1.compute.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-e516ff-b7d8666c5-8jsr9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-e516ff-b7d8666c5-8jsr9 to ip-10-0-133-209.us-west-1.compute.internal 43m Normal Scheduled pod/maas-default-gateway-openshift-default-76597d6855-dtlt4 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-76597d6855-dtlt4 to ip-10-0-134-125.us-west-1.compute.internal 20m Normal Scheduled pod/maas-default-gateway-openshift-default-76597d6855-hk6mk Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-76597d6855-hk6mk to ip-10-0-128-201.us-west-1.compute.internal 20m Normal Scheduled pod/maas-default-gateway-openshift-default-76597d6855-j8cqc Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-76597d6855-j8cqc to ip-10-0-133-209.us-west-1.compute.internal 43m Normal Scheduled pod/maas-default-gateway-openshift-default-76597d6855-jjxxp Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-76597d6855-jjxxp to ip-10-0-134-125.us-west-1.compute.internal 44m Normal Scheduled pod/maas-default-gateway-openshift-default-76597d6855-jwwww Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-76597d6855-jwwww to ip-10-0-133-209.us-west-1.compute.internal 22m Normal Scheduled pod/maas-default-gateway-openshift-default-76597d6855-mtx78 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mtx78 to ip-10-0-128-201.us-west-1.compute.internal 58m Normal Scheduled pod/maas-default-gateway-openshift-default-76597d6855-qx6f2 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qx6f2 to ip-10-0-133-209.us-west-1.compute.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-fe4e0c-8599b44f79-9xn4d Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-fe4e0c-8599b44f79-9xn4d to ip-10-0-133-209.us-west-1.compute.internal 11m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-2f48c1-7847czcwdv Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-2f48c1-7847czcwdv to ip-10-0-134-125.us-west-1.compute.internal 15m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-07ce81-9ddd44mgvn Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-07ce81-9ddd44mgvn to ip-10-0-134-125.us-west-1.compute.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w5-4b2bf4-6cd5fc4b6c-wwqfq Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w5-4b2bf4-6cd5fc4b6c-wwqfq to ip-10-0-133-209.us-west-1.compute.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w4-35df3e-7d67b7f575-zbnm5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-35df3e-7d67b7f575-zbnm5 to ip-10-0-134-125.us-west-1.compute.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-00c846-5d7b447995-p8qhc Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-00c846-5d7b447995-p8qhc to ip-10-0-134-125.us-west-1.compute.internal 35m Normal Scheduled pod/e2e-inf-a-9f4199-openshift-default-586448895-hl8h4 Successfully assigned openshift-ingress/e2e-inf-a-9f4199-openshift-default-586448895-hl8h4 to ip-10-0-133-209.us-west-1.compute.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29 to ip-10-0-128-201.us-west-1.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w5-297806-bdfd869bc-8j6sw Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w5-297806-bdfd869bc-8j6sw to ip-10-0-133-209.us-west-1.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk to ip-10-0-128-201.us-west-1.compute.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-57f116-599cbf5bbc-2fp4d Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-57f116-599cbf5bbc-2fp4d to ip-10-0-134-125.us-west-1.compute.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w1-551e2b-869c56d547-549jg Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w1-551e2b-869c56d547-549jg to ip-10-0-133-209.us-west-1.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5 to ip-10-0-128-201.us-west-1.compute.internal 31m Normal Scheduled pod/payload-pre-processing-e2e-mt-4cbcf738-67bd6b6f59-mfssb Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-4cbcf738-67bd6b6f59-mfssb to ip-10-0-134-125.us-west-1.compute.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-mt-4cbcf738-67bd6b6f59-59zp6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-4cbcf738-67bd6b6f59-59zp6 to ip-10-0-134-125.us-west-1.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-migrate-2c1c4f09-8689cb4c5f-htpk4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-2c1c4f09-8689cb4c5f-htpk4 to ip-10-0-133-209.us-west-1.compute.internal 30m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-095081-bd58cc787-xz5kc Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-095081-bd58cc787-xz5kc to ip-10-0-134-125.us-west-1.compute.internal 33m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-db5ef5-5b75965f5-j72vc Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-db5ef5-5b75965f5-j72vc to ip-10-0-133-209.us-west-1.compute.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-9f4199-84b49988cf-thdfx Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-9f4199-84b49988cf-thdfx to ip-10-0-134-125.us-west-1.compute.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf to ip-10-0-128-201.us-west-1.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-22e72e-7ffdf7c747-vbt9t Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-22e72e-7ffdf7c747-vbt9t to ip-10-0-134-125.us-west-1.compute.internal 33m Normal Scheduled pod/e2e-inf-a-db5ef5-openshift-default-596b67fbf7-qrgxq Successfully assigned openshift-ingress/e2e-inf-a-db5ef5-openshift-default-596b67fbf7-qrgxq to ip-10-0-133-209.us-west-1.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-derive-5fde04ee-59df47b5d7-gnjpp Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-5fde04ee-59df47b5d7-gnjpp to ip-10-0-134-125.us-west-1.compute.internal 4m59s Normal Scheduled pod/payload-pre-processing-e2e-authcfg-http-cc2474-7df96f57b7-dp897 Successfully assigned openshift-ingress/payload-pre-processing-e2e-authcfg-http-cc2474-7df96f57b7-dp897 to ip-10-0-134-125.us-west-1.compute.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-api-b-9d2b96-7b7d78cbff-gtqd9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-9d2b96-7b7d78cbff-gtqd9 to ip-10-0-133-209.us-west-1.compute.internal 76m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 76m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 76m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 75m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-76598c4567 from 0 to 1 75m Warning FailedCreate replicaset/router-default-76598c4567 Error creating: pods "router-default-76598c4567-" is forbidden: error fetching namespace "openshift-ingress": unable to find annotation openshift.io/sa.scc.uid-range 75m Normal SuccessfulCreate replicaset/router-default-76598c4567 Created pod: router-default-76598c4567-mhntk 69m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 69m Warning FailedMount pod/router-default-76598c4567-mhntk MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 69m Warning FailedMount pod/router-default-76598c4567-mhntk MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 68m Warning FailedMount pod/router-default-76598c4567-mhntk MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 68m Warning FailedMount pod/router-default-76598c4567-mhntk MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 68m Normal Pulled pod/router-default-76598c4567-mhntk Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 68m Normal AddedInterface pod/router-default-76598c4567-mhntk Add eth0 [10.133.0.18/23] from ovn-kubernetes 68m Normal Created pod/router-default-76598c4567-mhntk Container created 68m Warning ProbeError pod/router-default-76598c4567-mhntk Startup probe error: HTTP probe failed with statuscode: 500... 68m Normal Started pod/router-default-76598c4567-mhntk Container started 68m Warning Unhealthy pod/router-default-76598c4567-mhntk Startup probe failed: HTTP probe failed with statuscode: 500 64m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 64m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 64m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-wm9hl Add eth0 [10.132.0.24/23] from ovn-kubernetes 64m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-wm9hl 64m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-wm9hl Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 64m Normal Started pod/istiod-openshift-gateway-7876bfcb74-wm9hl Container started 64m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-wm9hl Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.537s (4.537s including waiting). Image size: 185476477 bytes. 64m Normal Created pod/istiod-openshift-gateway-7876bfcb74-wm9hl Container created 64m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-wm9hl Readiness probe failed: HTTP probe failed with statuscode: 503 64m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-wm9hl Readiness probe error: HTTP probe failed with statuscode: 503... 64m Normal SuccessfulCreate replicaset/openshift-ai-inference-openshift-ai-inference-77d8c79567 Created pod: openshift-ai-inference-openshift-ai-inference-77d8c79567-w62d2 64m Normal ScalingReplicaSet deployment/openshift-ai-inference-openshift-ai-inference Scaled up replica set openshift-ai-inference-openshift-ai-inference-77d8c79567 from 0 to 1 64m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-76597d6855 from 0 to 1 64m Normal EnsuringLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensuring load balancer 64m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-76597d6855 Created pod: maas-default-gateway-openshift-default-76597d6855-zhmt8 64m Normal AddedInterface pod/maas-default-gateway-openshift-default-76597d6855-zhmt8 Add eth0 [10.133.0.32/23] from ovn-kubernetes 64m Normal Pulling pod/maas-default-gateway-openshift-default-76597d6855-zhmt8 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 64m Normal AddedInterface pod/openshift-ai-inference-openshift-ai-inference-77d8c79567-w62d2 Add eth0 [10.133.0.33/23] from ovn-kubernetes 64m Normal Pulling pod/openshift-ai-inference-openshift-ai-inference-77d8c79567-w62d2 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 64m Normal EnsuredLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensured load balancer 64m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-zhmt8 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 9.677s (9.677s including waiting). Image size: 183555333 bytes. 64m Normal Pulled pod/openshift-ai-inference-openshift-ai-inference-77d8c79567-w62d2 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 9.42s (9.42s including waiting). Image size: 183555333 bytes. 64m Normal Started pod/openshift-ai-inference-openshift-ai-inference-77d8c79567-w62d2 Container started 64m Normal Created pod/openshift-ai-inference-openshift-ai-inference-77d8c79567-w62d2 Container created 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 63m Normal SuccessfulCreate replicaset/kube-auth-proxy-54497bf445 Created pod: kube-auth-proxy-54497bf445-q5p8t 63m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-6545b48489 Created pod: data-science-gateway-data-science-gateway-class-6545b4848996xtj 63m Normal AddedInterface pod/kube-auth-proxy-54497bf445-qvs4m Add eth0 [10.134.0.21/23] from ovn-kubernetes 63m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-6545b48489 from 0 to 1 63m Normal Pulling pod/kube-auth-proxy-54497bf445-q5p8t Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 63m Normal AddedInterface pod/kube-auth-proxy-54497bf445-q5p8t Add eth0 [10.132.0.26/23] from ovn-kubernetes 63m Normal Pulling pod/kube-auth-proxy-54497bf445-qvs4m Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 63m Normal SuccessfulCreate replicaset/kube-auth-proxy-54497bf445 Created pod: kube-auth-proxy-54497bf445-qvs4m 63m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-54497bf445 from 0 to 2 63m Normal Started pod/data-science-gateway-data-science-gateway-class-6545b4848996xtj Container started 63m Normal Created pod/data-science-gateway-data-science-gateway-class-6545b4848996xtj Container created 63m Normal Pulled pod/data-science-gateway-data-science-gateway-class-6545b4848996xtj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-6545b4848996xtj Add eth0 [10.133.0.40/23] from ovn-kubernetes 63m Normal Created pod/kube-auth-proxy-54497bf445-qvs4m Container created 63m Normal Pulled pod/kube-auth-proxy-54497bf445-qvs4m Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 3.141s (3.141s including waiting). Image size: 188904677 bytes. 63m Normal Pulled pod/kube-auth-proxy-54497bf445-q5p8t Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 4.149s (4.149s including waiting). Image size: 188904677 bytes. 63m Normal Started pod/kube-auth-proxy-54497bf445-qvs4m Container started 63m Normal Created pod/kube-auth-proxy-54497bf445-q5p8t Container created 63m Normal Started pod/kube-auth-proxy-54497bf445-q5p8t Container started 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 62m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 62m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 61m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-zch9j 61m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 61m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-zch9j Add eth0 [10.132.0.31/23] from ovn-kubernetes 61m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-zch9j Container created 61m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-zch9j Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 61m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-zch9j Container started 61m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-zch9j Readiness probe failed: HTTP probe failed with statuscode: 503 61m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-zch9j Readiness probe error: HTTP probe failed with statuscode: 503... 61m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-wm9hl Readiness probe error: Get "http://10.132.0.24:8080/ready": dial tcp 10.132.0.24:8080: connect: connection refused... 61m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-wm9hl Readiness probe failed: Get "http://10.132.0.24:8080/ready": dial tcp 10.132.0.24:8080: connect: connection refused 61m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-wm9hl 61m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 61m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-wm9hl Stopping container discovery 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-zhmt8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Started pod/maas-default-gateway-openshift-default-76597d6855-zhmt8 Container started 60m Normal Created pod/maas-default-gateway-openshift-default-76597d6855-zhmt8 Container created 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-zhmt8 Startup probe error: Get "http://10.133.0.32:15021/healthz/ready": dial tcp 10.133.0.32:15021: connect: connection refused... 60m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-zhmt8 Startup probe failed: Get "http://10.133.0.32:15021/healthz/ready": dial tcp 10.133.0.32:15021: connect: connection refused 60m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-76597d6855 Created pod: maas-default-gateway-openshift-default-76597d6855-ts4v7 59m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-76597d6855 from 1 to 3 59m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 59m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-76597d6855 Created pod: maas-default-gateway-openshift-default-76597d6855-dhvd2 59m Normal AddedInterface pod/maas-default-gateway-openshift-default-76597d6855-dhvd2 Add eth0 [10.134.0.29/23] from ovn-kubernetes 59m Normal AddedInterface pod/maas-default-gateway-openshift-default-76597d6855-ts4v7 Add eth0 [10.132.0.37/23] from ovn-kubernetes 59m Normal Pulling pod/maas-default-gateway-openshift-default-76597d6855-dhvd2 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 59m Normal Pulling pod/maas-default-gateway-openshift-default-76597d6855-ts4v7 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 59m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-ts4v7 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.193s (4.193s including waiting). Image size: 183555333 bytes. 59m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-dhvd2 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.214s (4.214s including waiting). Image size: 183555333 bytes. 58m Normal Created pod/maas-default-gateway-openshift-default-76597d6855-ts4v7 Container created 58m Normal Started pod/maas-default-gateway-openshift-default-76597d6855-ts4v7 Container started 58m Normal Started pod/maas-default-gateway-openshift-default-76597d6855-dhvd2 Container started 58m Normal Created pod/maas-default-gateway-openshift-default-76597d6855-dhvd2 Container created 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-ts4v7 Startup probe failed: Get "http://10.132.0.37:15021/healthz/ready": dial tcp 10.132.0.37:15021: connect: connection refused 58m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-ts4v7 Startup probe error: Get "http://10.132.0.37:15021/healthz/ready": dial tcp 10.132.0.37:15021: connect: connection refused... 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-dhvd2 Startup probe failed: Get "http://10.134.0.29:15021/healthz/ready": dial tcp 10.134.0.29:15021: connect: connection refused 58m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-dhvd2 Startup probe error: Get "http://10.134.0.29:15021/healthz/ready": dial tcp 10.134.0.29:15021: connect: connection refused... 58m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-ts4v7 Startup probe error: Get "http://10.132.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-ts4v7 Startup probe failed: Get "http://10.132.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-tlkdl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-76597d6855 Created pod: maas-default-gateway-openshift-default-76597d6855-qx6f2 58m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 58m Normal Created pod/maas-default-gateway-openshift-default-76597d6855-qx6f2 Container created 58m Normal AddedInterface pod/maas-default-gateway-openshift-default-76597d6855-qx6f2 Add eth0 [10.133.0.55/23] from ovn-kubernetes 58m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-qx6f2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-dhvd2 Startup probe error: Get "http://10.134.0.29:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Normal AddedInterface pod/maas-default-gateway-openshift-default-76597d6855-tlkdl Add eth0 [10.134.0.31/23] from ovn-kubernetes 58m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-76597d6855 Created pod: maas-default-gateway-openshift-default-76597d6855-tlkdl 58m Normal Started pod/maas-default-gateway-openshift-default-76597d6855-qx6f2 Container started 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-dhvd2 Startup probe failed: Get "http://10.134.0.29:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-76597d6855 from 3 to 5 58m Normal Created pod/maas-default-gateway-openshift-default-76597d6855-tlkdl Container created 58m Normal Started pod/maas-default-gateway-openshift-default-76597d6855-tlkdl Container started 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-tlkdl Startup probe failed: Get "http://10.134.0.31:15021/healthz/ready": dial tcp 10.134.0.31:15021: connect: connection refused 58m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-tlkdl Startup probe error: Get "http://10.134.0.31:15021/healthz/ready": dial tcp 10.134.0.31:15021: connect: connection refused... 58m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-qx6f2 Startup probe error: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused... 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-qx6f2 Startup probe failed: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused 58m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 58m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-7bb9489676 from 0 to 1 58m Normal SuccessfulCreate replicaset/payload-processing-7bb9489676 Created pod: payload-processing-7bb9489676-bqh9n 58m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 58m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-r82l2 58m Normal Pulling pod/payload-pre-processing-7b98bb678c-r82l2 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 58m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-r82l2 Add eth0 [10.133.0.57/23] from ovn-kubernetes 58m Warning FailedCreate replicaset/payload-processing-7bb9489676 Error creating: pods "payload-processing-7bb9489676-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 58m Normal AddedInterface pod/payload-processing-7bb9489676-bqh9n Add eth0 [10.133.0.58/23] from ovn-kubernetes 58m Normal Pulling pod/payload-processing-7bb9489676-bqh9n Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 58m Normal Created pod/payload-processing-7bb9489676-bqh9n Container created 58m Normal Pulled pod/payload-processing-7bb9489676-bqh9n Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.081s (3.081s including waiting). Image size: 125353963 bytes. 58m Normal Started pod/payload-pre-processing-7b98bb678c-r82l2 Container started 58m Normal Created pod/payload-pre-processing-7b98bb678c-r82l2 Container created 58m Normal Pulled pod/payload-pre-processing-7b98bb678c-r82l2 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.332s (3.332s including waiting). Image size: 125353963 bytes. 58m Normal Started pod/payload-processing-7bb9489676-bqh9n Container started 57m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-76597d6855 from 5 to 7 57m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-76597d6855 Created pod: maas-default-gateway-openshift-default-76597d6855-4ndqx 57m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-76597d6855 Created pod: maas-default-gateway-openshift-default-76597d6855-z29xk 57m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 57m Normal Started pod/maas-default-gateway-openshift-default-76597d6855-4ndqx Container started 57m Normal AddedInterface pod/maas-default-gateway-openshift-default-76597d6855-4ndqx Add eth0 [10.134.0.33/23] from ovn-kubernetes 57m Normal AddedInterface pod/maas-default-gateway-openshift-default-76597d6855-z29xk Add eth0 [10.132.0.38/23] from ovn-kubernetes 57m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-z29xk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-4ndqx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/maas-default-gateway-openshift-default-76597d6855-z29xk Container created 57m Normal Started pod/maas-default-gateway-openshift-default-76597d6855-z29xk Container started 57m Normal Created pod/maas-default-gateway-openshift-default-76597d6855-4ndqx Container created 57m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-4ndqx Startup probe error: Get "http://10.134.0.33:15021/healthz/ready": dial tcp 10.134.0.33:15021: connect: connection refused... 57m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-z29xk Startup probe failed: Get "http://10.132.0.38:15021/healthz/ready": dial tcp 10.132.0.38:15021: connect: connection refused 57m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-z29xk Startup probe error: Get "http://10.132.0.38:15021/healthz/ready": dial tcp 10.132.0.38:15021: connect: connection refused... 57m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-4ndqx Startup probe failed: Get "http://10.134.0.33:15021/healthz/ready": dial tcp 10.134.0.33:15021: connect: connection refused 57m Normal AddedLabel gateway/e2e-worker-w3-cdcf67 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-cdcf67 57m Normal ScalingReplicaSet deployment/e2e-worker-w2-a0bcbc-openshift-default Scaled up replica set e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c from 0 to 1 57m Normal AddedLabel gateway/e2e-worker-w4-203c35 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-203c35 57m Normal AddedLabel gateway/e2e-worker-w6-ebb7fd Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-ebb7fd 57m Normal AddedLabel gateway/e2e-worker-w1-e516ff Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-e516ff 57m Normal AddedLabel gateway/e2e-worker-w5-3b7476 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-3b7476 57m Normal AddedLabel gateway/e2e-worker-w0-fe4e0c Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-fe4e0c 57m Normal AddedLabel gateway/e2e-worker-w2-a0bcbc Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-a0bcbc 57m Normal SuccessfulCreate replicaset/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c Created pod: e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-v2tpq 57m Normal ScalingReplicaSet deployment/e2e-worker-w1-e516ff-openshift-default Scaled up replica set e2e-worker-w1-e516ff-openshift-default-6f4b957485 from 0 to 1 57m Normal AddedInterface pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-v2tpq Add eth0 [10.133.0.60/23] from ovn-kubernetes 57m Normal SuccessfulCreate replicaset/e2e-worker-w1-e516ff-openshift-default-6f4b957485 Created pod: e2e-worker-w1-e516ff-openshift-default-6f4b957485-bjmn9 57m Normal SuccessfulCreate replicaset/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c Created pod: e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tzgt6 57m Normal ScalingReplicaSet deployment/e2e-worker-w0-fe4e0c-openshift-default Scaled up replica set e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c from 0 to 1 57m Normal AddedInterface pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-rwgvq Add eth0 [10.134.0.34/23] from ovn-kubernetes 57m Normal SuccessfulCreate replicaset/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 Created pod: e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-rwgvq 57m Normal ScalingReplicaSet deployment/e2e-worker-w3-cdcf67-openshift-default Scaled up replica set e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 from 0 to 1 57m Normal SuccessfulCreate replicaset/e2e-worker-w6-ebb7fd-openshift-default-544f5768f Created pod: e2e-worker-w6-ebb7fd-openshift-default-544f5768f-x449v 57m Normal ScalingReplicaSet deployment/e2e-worker-w5-3b7476-openshift-default Scaled up replica set e2e-worker-w5-3b7476-openshift-default-59975457bf from 0 to 1 57m Normal AddedInterface pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tzgt6 Add eth0 [10.133.0.61/23] from ovn-kubernetes 57m Normal ScalingReplicaSet deployment/e2e-worker-w6-ebb7fd-openshift-default Scaled up replica set e2e-worker-w6-ebb7fd-openshift-default-544f5768f from 0 to 1 57m Normal ScalingReplicaSet deployment/e2e-worker-w4-203c35-openshift-default Scaled up replica set e2e-worker-w4-203c35-openshift-default-56497cdcc from 0 to 1 57m Normal SuccessfulCreate replicaset/e2e-worker-w4-203c35-openshift-default-56497cdcc Created pod: e2e-worker-w4-203c35-openshift-default-56497cdcc-825cw 57m Normal AddedInterface pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-825cw Add eth0 [10.133.0.62/23] from ovn-kubernetes 57m Normal AddedInterface pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-bjmn9 Add eth0 [10.134.0.35/23] from ovn-kubernetes 57m Normal SuccessfulCreate replicaset/e2e-worker-w5-3b7476-openshift-default-59975457bf Created pod: e2e-worker-w5-3b7476-openshift-default-59975457bf-msl8m 57m Normal AddedInterface pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-msl8m Add eth0 [10.134.0.36/23] from ovn-kubernetes 57m Normal AddedInterface pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-x449v Add eth0 [10.133.0.63/23] from ovn-kubernetes 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-a0bcbc Scaled up replica set payload-pre-processing-e2e-worker-w2-a0bcbc-69fdc5f5c9 from 0 to 1 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-a0bcbc-69fdc5f5c9 Created pod: payload-pre-processing-e2e-worker-w2-a0bcbc-69fdc5f5c9-d4d6x 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-a0bcbc-69fdc5f5c9 Error creating: pods "payload-pre-processing-e2e-worker-w2-a0bcbc-69fdc5f5c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-a0bcbc: serviceaccount "payload-processing-e2e-worker-w2-a0bcbc" not found 57m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-a0bcbc-5bf4697664 Error creating: pods "payload-processing-e2e-worker-w2-a0bcbc-5bf4697664-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-a0bcbc: serviceaccount "payload-processing-e2e-worker-w2-a0bcbc" not found 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-a0bcbc-5bf4697664 Created pod: payload-processing-e2e-worker-w2-a0bcbc-5bf4697664-l2vk9 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-a0bcbc Scaled up replica set payload-processing-e2e-worker-w2-a0bcbc-5bf4697664 from 0 to 1 57m Normal Pulled pod/payload-processing-e2e-worker-w2-a0bcbc-5bf4697664-l2vk9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-a0bcbc-69fdc5f5c9-d4d6x Add eth0 [10.134.0.37/23] from ovn-kubernetes 57m Normal Started pod/payload-processing-e2e-worker-w2-a0bcbc-5bf4697664-l2vk9 Container started 57m Normal Created pod/payload-processing-e2e-worker-w2-a0bcbc-5bf4697664-l2vk9 Container created 57m Normal AddedInterface pod/payload-processing-e2e-worker-w2-a0bcbc-5bf4697664-l2vk9 Add eth0 [10.133.0.64/23] from ovn-kubernetes 57m Normal Pulling pod/payload-pre-processing-e2e-worker-w2-a0bcbc-69fdc5f5c9-d4d6x Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 57m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-a0bcbc-69fdc5f5c9-d4d6x Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.735s (3.735s including waiting). Image size: 125353963 bytes. 57m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-cdcf67-76579b7df4 Error creating: pods "payload-processing-e2e-worker-w3-cdcf67-76579b7df4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-cdcf67: serviceaccount "payload-processing-e2e-worker-w3-cdcf67" not found 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-cdcf67-dc57c4455 Error creating: pods "payload-pre-processing-e2e-worker-w3-cdcf67-dc57c4455-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-cdcf67: serviceaccount "payload-processing-e2e-worker-w3-cdcf67" not found 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-cdcf67-dc57c4455 Created pod: payload-pre-processing-e2e-worker-w3-cdcf67-dc57c4455-ljnxm 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-cdcf67 Scaled up replica set payload-pre-processing-e2e-worker-w3-cdcf67-dc57c4455 from 0 to 1 57m Normal Started pod/payload-pre-processing-e2e-worker-w2-a0bcbc-69fdc5f5c9-d4d6x Container started 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-cdcf67 Scaled up replica set payload-processing-e2e-worker-w3-cdcf67-76579b7df4 from 0 to 1 57m Normal Created pod/payload-pre-processing-e2e-worker-w2-a0bcbc-69fdc5f5c9-d4d6x Container created 57m Normal Started pod/payload-processing-e2e-worker-w3-cdcf67-76579b7df4-vqbkj Container started 57m Normal AddedInterface pod/payload-processing-e2e-worker-w3-cdcf67-76579b7df4-vqbkj Add eth0 [10.134.0.39/23] from ovn-kubernetes 57m Normal Created pod/payload-pre-processing-e2e-worker-w3-cdcf67-dc57c4455-ljnxm Container created 57m Normal Started pod/payload-pre-processing-e2e-worker-w3-cdcf67-dc57c4455-ljnxm Container started 57m Normal Created pod/payload-processing-e2e-worker-w3-cdcf67-76579b7df4-vqbkj Container created 57m Normal Pulled pod/payload-processing-e2e-worker-w3-cdcf67-76579b7df4-vqbkj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-cdcf67-76579b7df4 Created pod: payload-processing-e2e-worker-w3-cdcf67-76579b7df4-vqbkj 57m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-cdcf67-dc57c4455-ljnxm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-cdcf67-dc57c4455-ljnxm Add eth0 [10.133.0.65/23] from ovn-kubernetes 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-3b7476-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-3b7476-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-fe4e0c-8599b44f79 Error creating: pods "payload-pre-processing-e2e-worker-w0-fe4e0c-8599b44f79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-fe4e0c: serviceaccount "payload-processing-e2e-worker-w0-fe4e0c" not found 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-fe4e0c-8599b44f79 Created pod: payload-pre-processing-e2e-worker-w0-fe4e0c-8599b44f79-9xn4d 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-fe4e0c Scaled up replica set payload-pre-processing-e2e-worker-w0-fe4e0c-8599b44f79 from 0 to 1 57m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-fe4e0c-7dcb6cb4dd Error creating: pods "payload-processing-e2e-worker-w0-fe4e0c-7dcb6cb4dd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-fe4e0c: serviceaccount "payload-processing-e2e-worker-w0-fe4e0c" not found 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-fe4e0c-7dcb6cb4dd Created pod: payload-processing-e2e-worker-w0-fe4e0c-7dcb6cb4dd-vdb4p 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-fe4e0c Scaled up replica set payload-processing-e2e-worker-w0-fe4e0c-7dcb6cb4dd from 0 to 1 57m Normal Created pod/payload-pre-processing-e2e-worker-w0-fe4e0c-8599b44f79-9xn4d Container created 57m Normal Pulled pod/payload-processing-e2e-worker-w0-fe4e0c-7dcb6cb4dd-vdb4p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Created pod/payload-processing-e2e-worker-w0-fe4e0c-7dcb6cb4dd-vdb4p Container created 57m Normal Started pod/payload-processing-e2e-worker-w0-fe4e0c-7dcb6cb4dd-vdb4p Container started 57m Normal Started pod/payload-pre-processing-e2e-worker-w0-fe4e0c-8599b44f79-9xn4d Container started 57m Normal AddedInterface pod/payload-processing-e2e-worker-w0-fe4e0c-7dcb6cb4dd-vdb4p Add eth0 [10.134.0.40/23] from ovn-kubernetes 57m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-fe4e0c-8599b44f79-9xn4d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-fe4e0c-8599b44f79-9xn4d Add eth0 [10.133.0.66/23] from ovn-kubernetes 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-3b7476 Scaled up replica set payload-processing-e2e-worker-w5-3b7476-846b85d7cd from 0 to 1 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-3b7476 Scaled up replica set payload-pre-processing-e2e-worker-w5-3b7476-56c5fdb86c from 0 to 1 56m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-3b7476-56c5fdb86c-8786b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-3b7476-56c5fdb86c-8786b Add eth0 [10.133.0.67/23] from ovn-kubernetes 56m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-3b7476-846b85d7cd Error creating: pods "payload-processing-e2e-worker-w5-3b7476-846b85d7cd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-3b7476: serviceaccount "payload-processing-e2e-worker-w5-3b7476" not found 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-3b7476-56c5fdb86c Error creating: pods "payload-pre-processing-e2e-worker-w5-3b7476-56c5fdb86c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-3b7476: serviceaccount "payload-processing-e2e-worker-w5-3b7476" not found 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-3b7476-56c5fdb86c Created pod: payload-pre-processing-e2e-worker-w5-3b7476-56c5fdb86c-8786b 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-3b7476-846b85d7cd Created pod: payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw 56m Normal AddedInterface pod/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw Add eth0 [10.132.0.40/23] from ovn-kubernetes 56m Normal Pulling pod/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 56m Normal Created pod/payload-pre-processing-e2e-worker-w5-3b7476-56c5fdb86c-8786b Container created 56m Normal Started pod/payload-pre-processing-e2e-worker-w5-3b7476-56c5fdb86c-8786b Container started 56m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-e516ff-57b6c4bc94 Error creating: pods "payload-processing-e2e-worker-w1-e516ff-57b6c4bc94-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-e516ff: serviceaccount "payload-processing-e2e-worker-w1-e516ff" not found 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-e516ff-b7d8666c5 Error creating: pods "payload-pre-processing-e2e-worker-w1-e516ff-b7d8666c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-e516ff: serviceaccount "payload-processing-e2e-worker-w1-e516ff" not found 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-e516ff Scaled up replica set payload-processing-e2e-worker-w1-e516ff-57b6c4bc94 from 0 to 1 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-e516ff-57b6c4bc94 Created pod: payload-processing-e2e-worker-w1-e516ff-57b6c4bc94-5nc6m 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-e516ff-b7d8666c5 Created pod: payload-pre-processing-e2e-worker-w1-e516ff-b7d8666c5-8jsr9 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-e516ff Scaled up replica set payload-pre-processing-e2e-worker-w1-e516ff-b7d8666c5 from 0 to 1 56m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-e516ff-b7d8666c5-8jsr9 Add eth0 [10.133.0.68/23] from ovn-kubernetes 56m Normal Pulled pod/payload-processing-e2e-worker-w1-e516ff-57b6c4bc94-5nc6m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Started pod/payload-processing-e2e-worker-w1-e516ff-57b6c4bc94-5nc6m Container started 56m Normal Created pod/payload-processing-e2e-worker-w1-e516ff-57b6c4bc94-5nc6m Container created 56m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-e516ff-b7d8666c5-8jsr9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Created pod/payload-pre-processing-e2e-worker-w1-e516ff-b7d8666c5-8jsr9 Container created 56m Normal AddedInterface pod/payload-processing-e2e-worker-w1-e516ff-57b6c4bc94-5nc6m Add eth0 [10.133.0.69/23] from ovn-kubernetes 56m Normal Started pod/payload-pre-processing-e2e-worker-w1-e516ff-b7d8666c5-8jsr9 Container started 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-203c35 Scaled up replica set payload-processing-e2e-worker-w4-203c35-688854f56b from 0 to 1 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-203c35 Scaled up replica set payload-pre-processing-e2e-worker-w4-203c35-5bb8b769bc from 0 to 1 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-203c35-5bb8b769bc Error creating: pods "payload-pre-processing-e2e-worker-w4-203c35-5bb8b769bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-203c35: serviceaccount "payload-processing-e2e-worker-w4-203c35" not found 56m Normal Started pod/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw Container started 56m Normal Created pod/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw Container created 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-203c35-688854f56b Created pod: payload-processing-e2e-worker-w4-203c35-688854f56b-jk6wm 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-203c35-5bb8b769bc Created pod: payload-pre-processing-e2e-worker-w4-203c35-5bb8b769bc-q9wmw 56m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-203c35-5bb8b769bc-q9wmw Add eth0 [10.133.0.70/23] from ovn-kubernetes 56m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-203c35-688854f56b Error creating: pods "payload-processing-e2e-worker-w4-203c35-688854f56b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-203c35: serviceaccount "payload-processing-e2e-worker-w4-203c35" not found 56m Normal Pulled pod/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.792s (3.792s including waiting). Image size: 125353963 bytes. 56m Normal Started pod/payload-processing-e2e-worker-w4-203c35-688854f56b-jk6wm Container started 56m Normal Created pod/payload-pre-processing-e2e-worker-w4-203c35-5bb8b769bc-q9wmw Container created 56m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-203c35-5bb8b769bc-q9wmw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Created pod/payload-processing-e2e-worker-w4-203c35-688854f56b-jk6wm Container created 56m Normal Pulled pod/payload-processing-e2e-worker-w4-203c35-688854f56b-jk6wm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/payload-processing-e2e-worker-w4-203c35-688854f56b-jk6wm Add eth0 [10.133.0.71/23] from ovn-kubernetes 56m Normal Started pod/payload-pre-processing-e2e-worker-w4-203c35-5bb8b769bc-q9wmw Container started 56m Normal Pulled pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-rwgvq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-rwgvq Container created 56m Normal Started pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-rwgvq Container started 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-ebb7fd Scaled up replica set payload-processing-e2e-worker-w6-ebb7fd-b8fd547bb from 0 to 1 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-ebb7fd Scaled up replica set payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859 from 0 to 1 56m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-ebb7fd-b8fd547bb Error creating: pods "payload-processing-e2e-worker-w6-ebb7fd-b8fd547bb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-ebb7fd: serviceaccount "payload-processing-e2e-worker-w6-ebb7fd" not found 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-ebb7fd-b8fd547bb Created pod: payload-processing-e2e-worker-w6-ebb7fd-b8fd547bb-7jkvh 56m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n Add eth0 [10.132.0.42/23] from ovn-kubernetes 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859 Created pod: payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859 Error creating: pods "payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-ebb7fd: serviceaccount "payload-processing-e2e-worker-w6-ebb7fd" not found 56m Normal AddedInterface pod/payload-processing-e2e-worker-w6-ebb7fd-b8fd547bb-7jkvh Add eth0 [10.134.0.43/23] from ovn-kubernetes 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-ebb7fd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Normal Pulled pod/payload-processing-e2e-worker-w6-ebb7fd-b8fd547bb-7jkvh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Started pod/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n Container started 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-ebb7fd-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Normal Created pod/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n Container created 56m Normal Created pod/payload-processing-e2e-worker-w6-ebb7fd-b8fd547bb-7jkvh Container created 56m Normal Started pod/payload-processing-e2e-worker-w6-ebb7fd-b8fd547bb-7jkvh Container started 56m Warning Unhealthy pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-rwgvq Startup probe failed: Get "http://10.134.0.34:15021/healthz/ready": dial tcp 10.134.0.34:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-rwgvq Startup probe error: Get "http://10.134.0.34:15021/healthz/ready": dial tcp 10.134.0.34:15021: connect: connection refused... 56m Normal Started pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tzgt6 Container started 56m Normal Pulled pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tzgt6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tzgt6 Container created 56m Warning ProbeError pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tzgt6 Startup probe error: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tzgt6 Startup probe failed: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused 56m Normal Pulled pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-v2tpq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-v2tpq Container started 56m Normal Created pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-v2tpq Container created 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-cdcf67-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-cdcf67-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-fe4e0c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-fe4e0c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning ProbeError pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-msl8m Readiness probe error: Get "http://10.134.0.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Warning Unhealthy pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-msl8m Readiness probe failed: Get "http://10.134.0.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning Unhealthy pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-v2tpq Startup probe failed: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused 56m Normal Pulled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-msl8m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-msl8m Container created 56m Warning ProbeError pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-v2tpq Startup probe error: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused... 56m Normal Started pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-msl8m Container started 56m Warning Unhealthy pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-msl8m Startup probe failed: Get "http://10.134.0.36:15021/healthz/ready": dial tcp 10.134.0.36:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-msl8m Startup probe error: Get "http://10.134.0.36:15021/healthz/ready": dial tcp 10.134.0.36:15021: connect: connection refused... 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-cdcf67-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-cdcf67-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Normal Pulled pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-bjmn9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-a0bcbc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-a0bcbc-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-fe4e0c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Normal Created pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-bjmn9 Container created 56m Normal Started pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-bjmn9 Container started 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-fe4e0c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-e516ff-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-e516ff-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning ProbeError pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-bjmn9 Startup probe error: Get "http://10.134.0.35:15021/healthz/ready": dial tcp 10.134.0.35:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-bjmn9 Startup probe failed: Get "http://10.134.0.35:15021/healthz/ready": dial tcp 10.134.0.35:15021: connect: connection refused 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-a0bcbc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-a0bcbc-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-3b7476-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-3b7476-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Normal Pulled pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-825cw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-825cw Container started 56m Normal Started pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-x449v Container started 56m Normal Created pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-x449v Container created 56m Normal Pulled pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-x449v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-825cw Container created 55m Warning Unhealthy pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-x449v Startup probe failed: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused 55m Warning Unhealthy pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-825cw Startup probe failed: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-825cw Startup probe error: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused... 55m Warning ProbeError pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-x449v Startup probe error: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused... 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-e516ff-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-e516ff-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-ebb7fd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-ebb7fd-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Normal SuccessfulCreate replicaset/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 Created pod: e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-w5ldf 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-cdcf67-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 55m Normal ScalingReplicaSet deployment/e2e-worker-w3-cdcf67-openshift-default Scaled up replica set e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 from 1 to 2 55m Normal Created pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-w5ldf Container created 55m Normal Pulled pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-w5ldf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Started pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-w5ldf Container started 55m Normal AddedInterface pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-w5ldf Add eth0 [10.133.0.77/23] from ovn-kubernetes 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-203c35-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-203c35-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning Unhealthy pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-w5ldf Startup probe failed: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-w5ldf Startup probe error: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused... 54m Normal Pulled pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-6hhrb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-6hhrb Add eth0 [10.134.0.48/23] from ovn-kubernetes 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-fe4e0c-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulCreate replicaset/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c Created pod: e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-6hhrb 54m Normal ScalingReplicaSet deployment/e2e-worker-w0-fe4e0c-openshift-default Scaled up replica set e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c from 1 to 2 54m Normal Created pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-6hhrb Container created 54m Normal Started pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-6hhrb Container started 53m Warning ProbeError pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-6hhrb Startup probe error: Get "http://10.134.0.48:15021/healthz/ready": dial tcp 10.134.0.48:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-6hhrb Startup probe failed: Get "http://10.134.0.48:15021/healthz/ready": dial tcp 10.134.0.48:15021: connect: connection refused 53m Normal SuccessfulCreate replicaset/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c Created pod: e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-m97sr 53m Normal Started pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-m97sr Container started 53m Normal AddedInterface pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-m97sr Add eth0 [10.134.0.49/23] from ovn-kubernetes 53m Normal Created pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-m97sr Container created 53m Normal Pulled pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-m97sr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Warning ProbeError pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-m97sr Startup probe error: Get "http://10.134.0.49:15021/healthz/ready": dial tcp 10.134.0.49:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-m97sr Startup probe failed: Get "http://10.134.0.49:15021/healthz/ready": dial tcp 10.134.0.49:15021: connect: connection refused 53m Normal ScalingReplicaSet deployment/e2e-worker-w3-cdcf67-openshift-default Scaled up replica set e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 from 2 to 3 53m Normal SuccessfulCreate replicaset/e2e-worker-w1-e516ff-openshift-default-6f4b957485 Created pod: e2e-worker-w1-e516ff-openshift-default-6f4b957485-fc8wx 53m Normal SuccessfulCreate replicaset/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 Created pod: e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e516ff-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 53m Normal AddedInterface pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl Add eth0 [10.132.0.52/23] from ovn-kubernetes 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-cdcf67-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w1-e516ff-openshift-default Scaled up replica set e2e-worker-w1-e516ff-openshift-default-6f4b957485 from 1 to 2 53m Normal Created pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-fc8wx Container created 53m Normal Started pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-fc8wx Container started 53m Normal Pulled pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl Container created 53m Normal Started pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl Container started 53m Normal Pulled pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-fc8wx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-fc8wx Add eth0 [10.133.0.84/23] from ovn-kubernetes 52m Warning Unhealthy pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl Startup probe failed: Get "http://10.132.0.52:15021/healthz/ready": dial tcp 10.132.0.52:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl Startup probe error: Get "http://10.132.0.52:15021/healthz/ready": dial tcp 10.132.0.52:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-fc8wx Startup probe error: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-fc8wx Startup probe failed: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused 52m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-76597d6855 from 7 to 5 52m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-76597d6855 Deleted pod: maas-default-gateway-openshift-default-76597d6855-tlkdl 52m Normal Killing pod/maas-default-gateway-openshift-default-76597d6855-4ndqx Stopping container istio-proxy 52m Normal Killing pod/maas-default-gateway-openshift-default-76597d6855-tlkdl Stopping container istio-proxy 52m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: All metrics below target 52m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-76597d6855 Deleted pod: maas-default-gateway-openshift-default-76597d6855-4ndqx 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-ebb7fd-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 52m Normal SuccessfulCreate replicaset/e2e-worker-w6-ebb7fd-openshift-default-544f5768f Created pod: e2e-worker-w6-ebb7fd-openshift-default-544f5768f-w6tt7 52m Normal AddedInterface pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-w6tt7 Add eth0 [10.134.0.50/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-w6tt7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-w6tt7 Container created 52m Normal Started pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-w6tt7 Container started 52m Normal ScalingReplicaSet deployment/e2e-worker-w6-ebb7fd-openshift-default Scaled up replica set e2e-worker-w6-ebb7fd-openshift-default-544f5768f from 1 to 2 52m Warning Unhealthy pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-w6tt7 Startup probe failed: Get "http://10.134.0.50:15021/healthz/ready": dial tcp 10.134.0.50:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-w6tt7 Startup probe error: Get "http://10.134.0.50:15021/healthz/ready": dial tcp 10.134.0.50:15021: connect: connection refused... 52m Normal AddedInterface pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6 Add eth0 [10.132.0.53/23] from ovn-kubernetes 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-203c35-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 52m Normal SuccessfulCreate replicaset/e2e-worker-w1-e516ff-openshift-default-6f4b957485 Created pod: e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-203c35-openshift-default-56497cdcc Created pod: e2e-worker-w4-203c35-openshift-default-56497cdcc-2b66r 52m Normal ScalingReplicaSet deployment/e2e-worker-w1-e516ff-openshift-default Scaled up replica set e2e-worker-w1-e516ff-openshift-default-6f4b957485 from 2 to 3 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e516ff-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 52m Normal Pulled pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Started pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2b66r Container started 52m Normal Created pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6 Container created 52m Normal Created pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2b66r Container created 52m Normal Started pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6 Container started 52m Normal AddedInterface pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2b66r Add eth0 [10.134.0.51/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2b66r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Warning ProbeError pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6 Startup probe error: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6 Startup probe failed: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2b66r Startup probe error: Get "http://10.134.0.51:15021/healthz/ready": dial tcp 10.134.0.51:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2b66r Startup probe failed: Get "http://10.134.0.51:15021/healthz/ready": dial tcp 10.134.0.51:15021: connect: connection refused 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-3b7476-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 52m Normal Pulled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-52pkm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-52pkm Container created 52m Normal AddedInterface pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-52pkm Add eth0 [10.133.0.97/23] from ovn-kubernetes 52m Normal SuccessfulCreate replicaset/e2e-worker-w5-3b7476-openshift-default-59975457bf Created pod: e2e-worker-w5-3b7476-openshift-default-59975457bf-52pkm 52m Normal ScalingReplicaSet deployment/e2e-worker-w5-3b7476-openshift-default Scaled up replica set e2e-worker-w5-3b7476-openshift-default-59975457bf from 1 to 2 52m Normal Started pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-52pkm Container started 51m Warning ProbeError pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-52pkm Startup probe error: Get "http://10.133.0.97:15021/healthz/ready": dial tcp 10.133.0.97:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-52pkm Startup probe failed: Get "http://10.133.0.97:15021/healthz/ready": dial tcp 10.133.0.97:15021: connect: connection refused 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-a0bcbc-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 51m Normal ScalingReplicaSet deployment/e2e-worker-w2-a0bcbc-openshift-default Scaled up replica set e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c from 2 to 3 51m Normal ScalingReplicaSet deployment/e2e-worker-w3-cdcf67-openshift-default Scaled up replica set e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 from 3 to 5 51m Normal SuccessfulCreate replicaset/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 Created pod: e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-vbc52 51m Normal SuccessfulCreate replicaset/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 Created pod: e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-v5wsm 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-cdcf67-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c Created pod: e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj 51m Normal Pulled pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-v5wsm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj Container created 51m Normal Created pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-vbc52 Container created 51m Normal Pulled pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-vbc52 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-v5wsm Add eth0 [10.133.0.102/23] from ovn-kubernetes 51m Normal AddedInterface pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-vbc52 Add eth0 [10.134.0.53/23] from ovn-kubernetes 51m Normal Created pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-v5wsm Container created 51m Normal AddedInterface pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj Add eth0 [10.132.0.54/23] from ovn-kubernetes 51m Normal Started pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj Container started 51m Normal Started pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-v5wsm Container started 51m Normal Started pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-vbc52 Container started 51m Normal Pulled pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Warning Unhealthy pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-vbc52 Startup probe failed: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj Startup probe failed: Get "http://10.132.0.54:15021/healthz/ready": dial tcp 10.132.0.54:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj Startup probe error: Get "http://10.132.0.54:15021/healthz/ready": dial tcp 10.132.0.54:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-vbc52 Startup probe error: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-v5wsm Startup probe failed: Get "http://10.133.0.102:15021/healthz/ready": dial tcp 10.133.0.102:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-v5wsm Startup probe error: Get "http://10.133.0.102:15021/healthz/ready": dial tcp 10.133.0.102:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj Startup probe error: Get "http://10.132.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj Startup probe failed: Get "http://10.132.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-a0bcbc-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c Created pod: e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-j4n6p 51m Normal SuccessfulCreate replicaset/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c Created pod: e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-bjdq9 51m Normal ScalingReplicaSet deployment/e2e-worker-w2-a0bcbc-openshift-default Scaled up replica set e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c from 3 to 5 51m Normal Created pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n Container created 51m Normal Started pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-j4n6p Container started 51m Normal AddedInterface pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n Add eth0 [10.132.0.55/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal SuccessfulCreate replicaset/e2e-worker-w4-203c35-openshift-default-56497cdcc Created pod: e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt 51m Normal Started pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n Container started 51m Normal AddedInterface pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p Add eth0 [10.132.0.56/23] from ovn-kubernetes 51m Normal AddedInterface pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-bjdq9 Add eth0 [10.133.0.104/23] from ovn-kubernetes 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-203c35-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 51m Normal ScalingReplicaSet deployment/e2e-worker-w4-203c35-openshift-default Scaled up replica set e2e-worker-w4-203c35-openshift-default-56497cdcc from 2 to 3 51m Normal Pulled pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-bjdq9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-bjdq9 Container created 51m Normal Started pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-bjdq9 Container started 51m Normal SuccessfulCreate replicaset/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c Created pod: e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p 51m Normal Created pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-j4n6p Container created 51m Normal Pulled pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-j4n6p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-j4n6p Add eth0 [10.134.0.54/23] from ovn-kubernetes 51m Normal ScalingReplicaSet deployment/e2e-worker-w0-fe4e0c-openshift-default Scaled up replica set e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c from 2 to 3 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e516ff-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/e2e-worker-w1-e516ff-openshift-default-6f4b957485 Created pod: e2e-worker-w1-e516ff-openshift-default-6f4b957485-7pg2l 51m Normal ScalingReplicaSet deployment/e2e-worker-w1-e516ff-openshift-default Scaled up replica set e2e-worker-w1-e516ff-openshift-default-6f4b957485 from 3 to 4 51m Normal SuccessfulCreate replicaset/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 Created pod: e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-cdcf67-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 51m Normal ScalingReplicaSet deployment/e2e-worker-w3-cdcf67-openshift-default Scaled up replica set e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 from 5 to 6 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-fe4e0c-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 51m Normal Created pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p Container created 51m Normal Started pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt Container started 51m Normal Pulled pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Pulled pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt Container created 51m Normal Started pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-7pg2l Container started 51m Normal AddedInterface pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt Add eth0 [10.132.0.57/23] from ovn-kubernetes 51m Normal AddedInterface pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-7pg2l Add eth0 [10.134.0.55/23] from ovn-kubernetes 51m Normal Created pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-7pg2l Container created 51m Normal Pulled pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-7pg2l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p Container started 51m Warning ProbeError pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p Startup probe error: Get "http://10.132.0.56:15021/healthz/ready": dial tcp 10.132.0.56:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p Startup probe failed: Get "http://10.132.0.56:15021/healthz/ready": dial tcp 10.132.0.56:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-bjdq9 Startup probe failed: Get "http://10.133.0.104:15021/healthz/ready": dial tcp 10.133.0.104:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-bjdq9 Startup probe error: Get "http://10.133.0.104:15021/healthz/ready": dial tcp 10.133.0.104:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n Startup probe error: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-j4n6p Startup probe error: Get "http://10.134.0.54:15021/healthz/ready": dial tcp 10.134.0.54:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-j4n6p Startup probe failed: Get "http://10.134.0.54:15021/healthz/ready": dial tcp 10.134.0.54:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n Startup probe failed: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt Startup probe error: Get "http://10.132.0.57:15021/healthz/ready": dial tcp 10.132.0.57:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt Startup probe failed: Get "http://10.132.0.57:15021/healthz/ready": dial tcp 10.132.0.57:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-7pg2l Startup probe error: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-7pg2l Startup probe failed: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-fe4e0c-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c Created pod: e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-ck68h 51m Normal ScalingReplicaSet deployment/e2e-worker-w0-fe4e0c-openshift-default Scaled up replica set e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c from 3 to 4 51m Normal AddedInterface pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-ck68h Add eth0 [10.133.0.106/23] from ovn-kubernetes 51m Normal Killing pod/maas-default-gateway-openshift-default-76597d6855-z29xk Stopping container istio-proxy 51m Normal Started pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-ck68h Container started 51m Normal Pulled pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-ck68h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-76597d6855 from 5 to 4 51m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-76597d6855 Deleted pod: maas-default-gateway-openshift-default-76597d6855-z29xk 51m Normal Created pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-ck68h Container created 51m Warning Unhealthy pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-ck68h Startup probe failed: Get "http://10.133.0.106:15021/healthz/ready": dial tcp 10.133.0.106:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-ck68h Startup probe error: Get "http://10.133.0.106:15021/healthz/ready": dial tcp 10.133.0.106:15021: connect: connection refused... 51m Normal SuccessfulCreate replicaset/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 Created pod: e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-hht95 51m Normal ScalingReplicaSet deployment/e2e-worker-w3-cdcf67-openshift-default Scaled up replica set e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 from 6 to 8 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-cdcf67-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 Created pod: e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-975ss 51m Normal AddedInterface pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-975ss Add eth0 [10.134.0.56/23] from ovn-kubernetes 51m Normal AddedInterface pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-hht95 Add eth0 [10.133.0.107/23] from ovn-kubernetes 51m Normal Created pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-hht95 Container created 51m Normal Created pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-975ss Container created 51m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-76597d6855 Deleted pod: maas-default-gateway-openshift-default-76597d6855-qx6f2 51m Normal Started pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-hht95 Container started 51m Normal Pulled pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-hht95 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Pulled pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-975ss Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Killing pod/maas-default-gateway-openshift-default-76597d6855-qx6f2 Stopping container istio-proxy 51m Normal Started pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-975ss Container started 50m Warning Unhealthy pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-hht95 Startup probe failed: Get "http://10.133.0.107:15021/healthz/ready": dial tcp 10.133.0.107:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-hht95 Startup probe error: Get "http://10.133.0.107:15021/healthz/ready": dial tcp 10.133.0.107:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-975ss Startup probe error: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-975ss Startup probe failed: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-3b7476-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 50m Normal ScalingReplicaSet deployment/e2e-worker-w5-3b7476-openshift-default Scaled up replica set e2e-worker-w5-3b7476-openshift-default-59975457bf from 2 to 3 50m Normal SuccessfulCreate replicaset/e2e-worker-w5-3b7476-openshift-default-59975457bf Created pod: e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5 50m Normal Created pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5 Container created 50m Normal Started pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5 Container started 50m Normal Pulled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5 Add eth0 [10.132.0.58/23] from ovn-kubernetes 50m Warning ProbeError pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5 Startup probe error: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5 Startup probe failed: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused 50m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 2; reason: All metrics below target 50m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-76597d6855 Deleted pod: maas-default-gateway-openshift-default-76597d6855-dhvd2 50m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-76597d6855 from 3 to 2 50m Normal Killing pod/maas-default-gateway-openshift-default-76597d6855-dhvd2 Stopping container istio-proxy 50m Normal SuccessfulCreate replicaset/e2e-worker-w6-ebb7fd-openshift-default-544f5768f Created pod: e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql 50m Normal ScalingReplicaSet deployment/e2e-worker-w6-ebb7fd-openshift-default Scaled up replica set e2e-worker-w6-ebb7fd-openshift-default-544f5768f from 2 to 3 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-ebb7fd-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 50m Normal Started pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql Container started 50m Normal Created pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql Container created 50m Normal Pulled pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql Add eth0 [10.132.0.60/23] from ovn-kubernetes 50m Warning ProbeError pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql Startup probe error: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql Startup probe failed: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused 50m Normal SuccessfulCreate replicaset/e2e-worker-w4-203c35-openshift-default-56497cdcc Created pod: e2e-worker-w4-203c35-openshift-default-56497cdcc-czkcm 50m Normal SuccessfulCreate replicaset/e2e-worker-w1-e516ff-openshift-default-6f4b957485 Created pod: e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e516ff-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 50m Normal SuccessfulCreate replicaset/e2e-worker-w1-e516ff-openshift-default-6f4b957485 Created pod: e2e-worker-w1-e516ff-openshift-default-6f4b957485-9qgws 50m Normal SuccessfulCreate replicaset/e2e-worker-w4-203c35-openshift-default-56497cdcc Created pod: e2e-worker-w4-203c35-openshift-default-56497cdcc-z59vn 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-203c35-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 50m Normal ScalingReplicaSet deployment/e2e-worker-w4-203c35-openshift-default Scaled up replica set e2e-worker-w4-203c35-openshift-default-56497cdcc from 3 to 5 50m Normal ScalingReplicaSet deployment/e2e-worker-w1-e516ff-openshift-default Scaled up replica set e2e-worker-w1-e516ff-openshift-default-6f4b957485 from 4 to 7 50m Normal SuccessfulCreate replicaset/e2e-worker-w1-e516ff-openshift-default-6f4b957485 Created pod: e2e-worker-w1-e516ff-openshift-default-6f4b957485-xg2tv 50m Normal Created pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw Container created 50m Normal Started pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw Container started 50m Normal Started pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-z59vn Container started 50m Normal Created pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-z59vn Container created 50m Normal Started pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-xg2tv Container started 50m Normal AddedInterface pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-czkcm Add eth0 [10.133.0.112/23] from ovn-kubernetes 50m Normal Started pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-9qgws Container started 50m Normal Created pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-xg2tv Container created 50m Normal Pulled pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-xg2tv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-xg2tv Add eth0 [10.134.0.60/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-z59vn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Pulled pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-czkcm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-czkcm Container created 50m Normal Started pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-czkcm Container started 50m Normal Created pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-9qgws Container created 50m Normal Pulled pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-9qgws Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-9qgws Add eth0 [10.133.0.111/23] from ovn-kubernetes 50m Normal AddedInterface pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-z59vn Add eth0 [10.134.0.61/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw Add eth0 [10.132.0.61/23] from ovn-kubernetes 49m Warning ProbeError pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-9qgws Startup probe error: Get "http://10.133.0.111:15021/healthz/ready": dial tcp 10.133.0.111:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-xg2tv Startup probe failed: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-xg2tv Startup probe error: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-9qgws Startup probe failed: Get "http://10.133.0.111:15021/healthz/ready": dial tcp 10.133.0.111:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw Startup probe failed: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw Startup probe error: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-czkcm Startup probe error: Get "http://10.133.0.112:15021/healthz/ready": dial tcp 10.133.0.112:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-czkcm Startup probe failed: Get "http://10.133.0.112:15021/healthz/ready": dial tcp 10.133.0.112:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-z59vn Startup probe error: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-z59vn Startup probe failed: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused 49m Normal SuccessfulCreate replicaset/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 Created pod: e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn 49m Normal SuccessfulCreate replicaset/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 (combined from similar events): Created pod: e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-d5qtm 49m Normal ScalingReplicaSet deployment/e2e-worker-w3-cdcf67-openshift-default Scaled up replica set e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 from 8 to 10 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-cdcf67-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 49m Normal Created pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-d5qtm Container created 49m Normal Created pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn Container created 49m Normal AddedInterface pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-mmngc Add eth0 [10.134.0.63/23] from ovn-kubernetes 49m Normal AddedInterface pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn Add eth0 [10.132.0.62/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn Container started 49m Normal AddedInterface pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-8bhc5 Add eth0 [10.133.0.114/23] from ovn-kubernetes 49m Normal AddedInterface pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-d5qtm Add eth0 [10.133.0.113/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-d5qtm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c Created pod: e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-pnfhp 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c Created pod: e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c Created pod: e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-mmngc 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c Created pod: e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-8bhc5 49m Normal Started pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-d5qtm Container started 49m Normal Pulled pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-8bhc5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-pnfhp Add eth0 [10.134.0.62/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-pnfhp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-fe4e0c-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 49m Normal ScalingReplicaSet deployment/e2e-worker-w0-fe4e0c-openshift-default Scaled up replica set e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c from 4 to 8 49m Normal AddedInterface pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh Add eth0 [10.132.0.63/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-mmngc Container started 49m Normal Created pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh Container created 49m Normal Pulled pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-mmngc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh Container started 49m Normal Started pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-8bhc5 Container started 49m Normal Created pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-8bhc5 Container created 49m Normal Started pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-pnfhp Container started 49m Normal Created pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-mmngc Container created 49m Normal Created pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-pnfhp Container created 49m Warning ProbeError pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-mmngc Startup probe error: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-d5qtm Startup probe failed: Get "http://10.133.0.113:15021/healthz/ready": dial tcp 10.133.0.113:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-8bhc5 Startup probe failed: Get "http://10.133.0.114:15021/healthz/ready": dial tcp 10.133.0.114:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh Startup probe error: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-mmngc Startup probe failed: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh Startup probe failed: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-d5qtm Startup probe error: Get "http://10.133.0.113:15021/healthz/ready": dial tcp 10.133.0.113:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-pnfhp Startup probe failed: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-pnfhp Startup probe error: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn Startup probe failed: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-8bhc5 Startup probe error: Get "http://10.133.0.114:15021/healthz/ready": dial tcp 10.133.0.114:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn Startup probe error: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused... 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-a0bcbc-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 49m Normal ScalingReplicaSet deployment/e2e-worker-w2-a0bcbc-openshift-default Scaled up replica set e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c from 5 to 8 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c Created pod: e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-g52rd 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c Created pod: e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-7mhsd 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c Created pod: e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r 49m Normal AddedInterface pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r Add eth0 [10.132.0.64/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-7mhsd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-g52rd Add eth0 [10.134.0.64/23] from ovn-kubernetes 49m Normal Created pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-g52rd Container created 49m Normal Started pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-7mhsd Container started 49m Normal Started pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-g52rd Container started 49m Normal Pulled pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Pulled pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-g52rd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r Container created 49m Normal Created pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-7mhsd Container created 49m Normal AddedInterface pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-7mhsd Add eth0 [10.133.0.116/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r Container started 49m Warning ProbeError pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r Startup probe error: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-7mhsd Startup probe failed: Get "http://10.133.0.116:15021/healthz/ready": dial tcp 10.133.0.116:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-7mhsd Startup probe error: Get "http://10.133.0.116:15021/healthz/ready": dial tcp 10.133.0.116:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-g52rd Startup probe failed: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-g52rd Startup probe error: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r Startup probe failed: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-3b7476-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 49m Normal ScalingReplicaSet deployment/e2e-worker-w5-3b7476-openshift-default Scaled up replica set e2e-worker-w5-3b7476-openshift-default-59975457bf from 3 to 4 49m Normal AddedInterface pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-8kz5l Add eth0 [10.133.0.118/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/e2e-worker-w5-3b7476-openshift-default-59975457bf Created pod: e2e-worker-w5-3b7476-openshift-default-59975457bf-8kz5l 49m Normal Pulled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-8kz5l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-8kz5l Container created 49m Normal Started pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-8kz5l Container started 49m Warning ProbeError pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-8kz5l Startup probe error: Get "http://10.133.0.118:15021/healthz/ready": dial tcp 10.133.0.118:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-8kz5l Startup probe failed: Get "http://10.133.0.118:15021/healthz/ready": dial tcp 10.133.0.118:15021: connect: connection refused 48m Normal ScalingReplicaSet deployment/e2e-worker-w2-a0bcbc-openshift-default Scaled up replica set e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c from 8 to 10 48m Normal Created pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k Container created 48m Normal AddedInterface pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k Add eth0 [10.132.0.65/23] from ovn-kubernetes 48m Normal SuccessfulCreate replicaset/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c Created pod: e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k 48m Normal SuccessfulCreate replicaset/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c (combined from similar events): Created pod: e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-fzwkp 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-a0bcbc-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 48m Normal Started pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k Container started 48m Normal Pulled pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-fzwkp Add eth0 [10.134.0.66/23] from ovn-kubernetes 48m Normal Started pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-fzwkp Container started 48m Normal Pulled pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-fzwkp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-fzwkp Container created 48m Warning Unhealthy pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k Startup probe failed: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k Startup probe error: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-fzwkp Startup probe error: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-fzwkp Startup probe failed: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k Startup probe failed: Get "http://10.132.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-fzwkp Startup probe failed: Get "http://10.134.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-fzwkp Startup probe error: Get "http://10.134.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k Startup probe error: Get "http://10.132.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Normal SuccessfulCreate replicaset/e2e-worker-w6-ebb7fd-openshift-default-544f5768f Created pod: e2e-worker-w6-ebb7fd-openshift-default-544f5768f-mngkl 48m Normal Created pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw Container created 48m Normal AddedInterface pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-4qkqm Add eth0 [10.133.0.122/23] from ovn-kubernetes 48m Normal SuccessfulCreate replicaset/e2e-worker-w6-ebb7fd-openshift-default-544f5768f Created pod: e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw 48m Normal SuccessfulCreate replicaset/e2e-worker-w6-ebb7fd-openshift-default-544f5768f Created pod: e2e-worker-w6-ebb7fd-openshift-default-544f5768f-4qkqm 48m Normal AddedInterface pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-mngkl Add eth0 [10.134.0.67/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-mngkl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-mngkl Container created 48m Normal Pulled pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-4qkqm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-4qkqm Container created 48m Normal Started pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-4qkqm Container started 48m Normal Started pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-mngkl Container started 48m Normal AddedInterface pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw Add eth0 [10.132.0.67/23] from ovn-kubernetes 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-ebb7fd-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 48m Normal ScalingReplicaSet deployment/e2e-worker-w6-ebb7fd-openshift-default Scaled up replica set e2e-worker-w6-ebb7fd-openshift-default-544f5768f from 3 to 6 48m Normal Pulled pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw Container started 48m Warning ProbeError pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj Readiness probe error: Get "http://10.132.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-9qgws Readiness probe error: Get "http://10.133.0.111:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt Readiness probe failed: Get "http://10.132.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj Readiness probe failed: Get "http://10.132.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt Readiness probe error: Get "http://10.132.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-9qgws Readiness probe failed: Get "http://10.133.0.111:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw Readiness probe error: Get "http://10.132.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw Readiness probe failed: Get "http://10.132.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-czkcm Readiness probe failed: Get "http://10.133.0.112:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-czkcm Readiness probe error: Get "http://10.133.0.112:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw Startup probe failed: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-4qkqm Startup probe error: Get "http://10.133.0.122:15021/healthz/ready": dial tcp 10.133.0.122:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-4qkqm Startup probe failed: Get "http://10.133.0.122:15021/healthz/ready": dial tcp 10.133.0.122:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-fzwkp Readiness probe error: Get "http://10.134.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw Startup probe error: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-fzwkp Readiness probe failed: Get "http://10.134.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-mngkl Startup probe error: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-mngkl Startup probe failed: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused 48m Normal ScalingReplicaSet deployment/e2e-worker-w1-e516ff-openshift-default Scaled up replica set e2e-worker-w1-e516ff-openshift-default-6f4b957485 from 7 to 10 48m Normal SuccessfulCreate replicaset/e2e-worker-w4-203c35-openshift-default-56497cdcc Created pod: e2e-worker-w4-203c35-openshift-default-56497cdcc-w6c48 48m Normal SuccessfulCreate replicaset/e2e-worker-w4-203c35-openshift-default-56497cdcc Created pod: e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27 48m Normal SuccessfulCreate replicaset/e2e-worker-w1-e516ff-openshift-default-6f4b957485 Created pod: e2e-worker-w1-e516ff-openshift-default-6f4b957485-5mccl 48m Normal SuccessfulCreate replicaset/e2e-worker-w1-e516ff-openshift-default-6f4b957485 Created pod: e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs 48m Normal SuccessfulCreate replicaset/e2e-worker-w1-e516ff-openshift-default-6f4b957485 (combined from similar events): Created pod: e2e-worker-w1-e516ff-openshift-default-6f4b957485-w2plx 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e516ff-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 48m Normal ScalingReplicaSet deployment/e2e-worker-w4-203c35-openshift-default Scaled up replica set e2e-worker-w4-203c35-openshift-default-56497cdcc from 5 to 7 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-203c35-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 47m Normal Created pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-5mccl Container created 47m Normal AddedInterface pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-w2plx Add eth0 [10.134.0.69/23] from ovn-kubernetes 47m Normal Started pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27 Container started 47m Normal Created pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27 Container created 47m Normal AddedInterface pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-w6c48 Add eth0 [10.134.0.70/23] from ovn-kubernetes 47m Normal AddedInterface pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs Add eth0 [10.132.0.69/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-w6c48 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Pulled pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27 Add eth0 [10.132.0.70/23] from ovn-kubernetes 47m Normal Created pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-w6c48 Container created 47m Normal Started pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-5mccl Container started 47m Normal Pulled pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-5mccl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Started pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-w2plx Container started 47m Normal Created pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-w2plx Container created 47m Normal Pulled pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-w2plx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Pulled pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-5mccl Add eth0 [10.133.0.126/23] from ovn-kubernetes 47m Normal Started pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-w6c48 Container started 47m Normal Created pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs Container created 47m Normal Started pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs Container started 47m Warning Unhealthy pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27 Startup probe failed: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-w2plx Startup probe error: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-w2plx Startup probe failed: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-5mccl Startup probe error: Get "http://10.133.0.126:15021/healthz/ready": dial tcp 10.133.0.126:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-5mccl Startup probe failed: Get "http://10.133.0.126:15021/healthz/ready": dial tcp 10.133.0.126:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27 Startup probe error: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs Startup probe failed: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs Startup probe error: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-w6c48 Startup probe failed: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-w6c48 Startup probe error: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused... 47m Normal SuccessfulCreate replicaset/e2e-worker-w4-203c35-openshift-default-56497cdcc Created pod: e2e-worker-w4-203c35-openshift-default-56497cdcc-ljwqc 47m Normal SuccessfulCreate replicaset/e2e-worker-w4-203c35-openshift-default-56497cdcc (combined from similar events): Created pod: e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-203c35-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 47m Normal ScalingReplicaSet deployment/e2e-worker-w4-203c35-openshift-default Scaled up replica set e2e-worker-w4-203c35-openshift-default-56497cdcc from 7 to 10 47m Normal SuccessfulCreate replicaset/e2e-worker-w4-203c35-openshift-default-56497cdcc Created pod: e2e-worker-w4-203c35-openshift-default-56497cdcc-rsw2s 47m Normal Pulled pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm Add eth0 [10.132.0.71/23] from ovn-kubernetes 47m Normal Started pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm Container started 47m Normal Created pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm Container created 47m Normal Pulled pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-ljwqc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-ljwqc Container created 47m Normal Started pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-ljwqc Container started 47m Normal AddedInterface pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-ljwqc Add eth0 [10.134.0.71/23] from ovn-kubernetes 47m Normal AddedInterface pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-rsw2s Add eth0 [10.133.0.127/23] from ovn-kubernetes 47m Normal Created pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-rsw2s Container created 47m Normal Started pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-rsw2s Container started 47m Normal Pulled pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-rsw2s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Warning Unhealthy pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm Startup probe failed: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm Startup probe error: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused... 47m Warning ProbeError pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-ljwqc Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-rsw2s Startup probe failed: Get "http://10.133.0.127:15021/healthz/ready": dial tcp 10.133.0.127:15021: connect: connection refused 47m Warning Unhealthy pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-ljwqc Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-rsw2s Startup probe error: Get "http://10.133.0.127:15021/healthz/ready": dial tcp 10.133.0.127:15021: connect: connection refused... 47m Normal AddedInterface pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-vh6fd Add eth0 [10.134.0.72/23] from ovn-kubernetes 47m Normal ScalingReplicaSet deployment/e2e-worker-w0-fe4e0c-openshift-default Scaled up replica set e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c from 8 to 10 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-fe4e0c-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 47m Normal AddedInterface pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r Add eth0 [10.132.0.72/23] from ovn-kubernetes 47m Normal SuccessfulCreate replicaset/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c Created pod: e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r 47m Normal SuccessfulCreate replicaset/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c (combined from similar events): Created pod: e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-vh6fd 47m Normal Pulled pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal SuccessfulCreate replicaset/e2e-worker-w5-3b7476-openshift-default-59975457bf Created pod: e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg 47m Normal Started pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r Container started 47m Normal Started pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-vh6fd Container started 47m Normal ScalingReplicaSet deployment/e2e-worker-w5-3b7476-openshift-default Scaled up replica set e2e-worker-w5-3b7476-openshift-default-59975457bf from 4 to 8 47m Normal SuccessfulCreate replicaset/e2e-worker-w5-3b7476-openshift-default-59975457bf Created pod: e2e-worker-w5-3b7476-openshift-default-59975457bf-gph88 47m Normal SuccessfulCreate replicaset/e2e-worker-w5-3b7476-openshift-default-59975457bf Created pod: e2e-worker-w5-3b7476-openshift-default-59975457bf-6n8hr 47m Normal Created pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-vh6fd Container created 47m Normal Pulled pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-vh6fd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-3b7476-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 47m Normal SuccessfulCreate replicaset/e2e-worker-w5-3b7476-openshift-default-59975457bf Created pod: e2e-worker-w5-3b7476-openshift-default-59975457bf-vxwkm 47m Normal Created pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r Container created 47m Normal AddedInterface pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg Add eth0 [10.132.0.73/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6n8hr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-gph88 Add eth0 [10.134.0.73/23] from ovn-kubernetes 47m Normal AddedInterface pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6n8hr Add eth0 [10.133.0.134/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Pulled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-gph88 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg Container created 47m Normal Started pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6n8hr Container started 47m Normal Created pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6n8hr Container created 47m Normal AddedInterface pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-vxwkm Add eth0 [10.134.0.74/23] from ovn-kubernetes 47m Normal Started pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg Container started 47m Normal Created pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-vxwkm Container created 47m Normal Created pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-gph88 Container created 47m Normal Started pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-vxwkm Container started 47m Normal Pulled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-vxwkm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Started pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-gph88 Container started 47m Warning Unhealthy pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused... 47m Warning ProbeError pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6n8hr Startup probe error: Get "http://10.133.0.134:15021/healthz/ready": dial tcp 10.133.0.134:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg Startup probe failed: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused 47m Warning Unhealthy pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6n8hr Startup probe failed: Get "http://10.133.0.134:15021/healthz/ready": dial tcp 10.133.0.134:15021: connect: connection refused 47m Warning Unhealthy pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-vh6fd Startup probe failed: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg Startup probe error: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused... 47m Warning ProbeError pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-vh6fd Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-vxwkm Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused 47m Warning Unhealthy pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-gph88 Startup probe failed: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-gph88 Startup probe error: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused... 47m Warning ProbeError pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-vxwkm Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused... 46m Normal SuccessfulCreate replicaset/e2e-worker-w6-ebb7fd-openshift-default-544f5768f Created pod: e2e-worker-w6-ebb7fd-openshift-default-544f5768f-db5nw 46m Normal ScalingReplicaSet deployment/e2e-worker-w6-ebb7fd-openshift-default Scaled up replica set e2e-worker-w6-ebb7fd-openshift-default-544f5768f from 6 to 10 46m Normal SuccessfulCreate replicaset/e2e-worker-w6-ebb7fd-openshift-default-544f5768f Created pod: e2e-worker-w6-ebb7fd-openshift-default-544f5768f-47dj7 46m Normal SuccessfulCreate replicaset/e2e-worker-w6-ebb7fd-openshift-default-544f5768f Created pod: e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp 46m Normal SuccessfulCreate replicaset/e2e-worker-w6-ebb7fd-openshift-default-544f5768f (combined from similar events): Created pod: e2e-worker-w6-ebb7fd-openshift-default-544f5768f-frgsz 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-ebb7fd-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 46m Normal Started pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp Container started 46m Normal Created pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-frgsz Container created 46m Normal Created pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp Container created 46m Normal Started pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-frgsz Container started 46m Normal Pulled pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Pulled pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-db5nw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp Add eth0 [10.132.0.74/23] from ovn-kubernetes 46m Normal Started pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-47dj7 Container started 46m Normal Pulled pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-frgsz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-frgsz Add eth0 [10.133.0.141/23] from ovn-kubernetes 46m Normal Created pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-47dj7 Container created 46m Normal Pulled pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-47dj7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-db5nw Container started 46m Normal Created pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-db5nw Container created 46m Normal AddedInterface pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-47dj7 Add eth0 [10.134.0.75/23] from ovn-kubernetes 46m Normal AddedInterface pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-db5nw Add eth0 [10.133.0.140/23] from ovn-kubernetes 46m Warning ProbeError pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp Startup probe error: Get "http://10.132.0.74:15021/healthz/ready": dial tcp 10.132.0.74:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-frgsz Startup probe error: Get "http://10.133.0.141:15021/healthz/ready": dial tcp 10.133.0.141:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-47dj7 Startup probe failed: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-db5nw Startup probe error: Get "http://10.133.0.140:15021/healthz/ready": dial tcp 10.133.0.140:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-db5nw Startup probe failed: Get "http://10.133.0.140:15021/healthz/ready": dial tcp 10.133.0.140:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp Startup probe failed: Get "http://10.132.0.74:15021/healthz/ready": dial tcp 10.132.0.74:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-47dj7 Startup probe error: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-frgsz Startup probe failed: Get "http://10.133.0.141:15021/healthz/ready": dial tcp 10.133.0.141:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-rwgvq Readiness probe failed: Get "http://10.134.0.34:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning Unhealthy pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-v2tpq Readiness probe failed: Get "http://10.133.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-vxwkm Readiness probe error: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning ProbeError pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-rwgvq Readiness probe error: Get "http://10.134.0.34:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning ProbeError pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-v2tpq Readiness probe error: Get "http://10.133.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-vxwkm Readiness probe failed: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-w5ldf Readiness probe error: Get "http://10.133.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-w5ldf Readiness probe failed: Get "http://10.133.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning Unhealthy pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl Readiness probe failed: Get "http://10.132.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl Readiness probe error: Get "http://10.132.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning ProbeError pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-m97sr Readiness probe error: Get "http://10.134.0.49:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning ProbeError pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-hht95 Readiness probe error: Get "http://10.133.0.107:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-hht95 Readiness probe failed: Get "http://10.133.0.107:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning Unhealthy pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-m97sr Readiness probe failed: Get "http://10.134.0.49:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-825cw Readiness probe error: Get "http://10.133.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-825cw Readiness probe failed: Get "http://10.133.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-975ss Readiness probe error: Get "http://10.134.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning ProbeError pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-j4n6p Readiness probe error: Get "http://10.134.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-j4n6p Readiness probe failed: Get "http://10.134.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning Unhealthy pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-975ss Readiness probe failed: Get "http://10.134.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-fc8wx Readiness probe error: Get "http://10.133.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-fc8wx Readiness probe failed: Get "http://10.133.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-7pg2l Readiness probe error: Get "http://10.134.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning ProbeError pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-xg2tv Readiness probe error: Get "http://10.134.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-7pg2l Readiness probe failed: Get "http://10.134.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning Unhealthy pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-xg2tv Readiness probe failed: Get "http://10.134.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning Unhealthy pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2b66r Readiness probe failed: Get "http://10.134.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2b66r Readiness probe error: Get "http://10.134.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning ProbeError pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-z59vn Readiness probe error: Get "http://10.134.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-z59vn Readiness probe failed: Get "http://10.134.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-3b7476-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 46m Normal SuccessfulCreate replicaset/e2e-worker-w5-3b7476-openshift-default-59975457bf (combined from similar events): Created pod: e2e-worker-w5-3b7476-openshift-default-59975457bf-f9flw 46m Normal ScalingReplicaSet deployment/e2e-worker-w5-3b7476-openshift-default Scaled up replica set e2e-worker-w5-3b7476-openshift-default-59975457bf from 8 to 10 46m Normal SuccessfulCreate replicaset/e2e-worker-w5-3b7476-openshift-default-59975457bf Created pod: e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s 46m Normal Started pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s Container started 46m Normal AddedInterface pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s Add eth0 [10.132.0.75/23] from ovn-kubernetes 46m Normal Started pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-f9flw Container started 46m Normal Pulled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-f9flw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Pulled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s Container created 46m Normal AddedInterface pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-f9flw Add eth0 [10.133.0.149/23] from ovn-kubernetes 46m Normal Created pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-f9flw Container created 46m Warning Unhealthy pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s Startup probe failed: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-f9flw Startup probe failed: Get "http://10.133.0.149:15021/healthz/ready": dial tcp 10.133.0.149:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-f9flw Startup probe error: Get "http://10.133.0.149:15021/healthz/ready": dial tcp 10.133.0.149:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s Startup probe error: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused... 46m Normal ScalingReplicaSet deployment/e2e-migrate-2c1c4f09-gw-openshift-default Scaled up replica set e2e-migrate-2c1c4f09-gw-openshift-default-648b465cc from 0 to 1 46m Normal SuccessfulCreate replicaset/e2e-migrate-2c1c4f09-gw-openshift-default-648b465cc Created pod: e2e-migrate-2c1c4f09-gw-openshift-default-648b465cc-mr59v 46m Normal AddedLabel gateway/e2e-migrate-2c1c4f09-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-2c1c4f09-gw 46m Normal AddedInterface pod/e2e-migrate-2c1c4f09-gw-openshift-default-648b465cc-mr59v Add eth0 [10.133.0.156/23] from ovn-kubernetes 46m Normal Killing pod/maas-default-gateway-openshift-default-76597d6855-ts4v7 Stopping container istio-proxy 46m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-76597d6855 from 2 to 1 46m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 1; reason: All metrics below target 46m Normal AddedLabel gateway/e2e-ait-87191e54 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-87191e54 46m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-76597d6855 Deleted pod: maas-default-gateway-openshift-default-76597d6855-ts4v7 46m Normal ScalingReplicaSet deployment/e2e-ait-87191e54-openshift-default Scaled up replica set e2e-ait-87191e54-openshift-default-7c59fcc9cc from 0 to 1 46m Normal SuccessfulCreate replicaset/e2e-ait-87191e54-openshift-default-7c59fcc9cc Created pod: e2e-ait-87191e54-openshift-default-7c59fcc9cc-6s8gq 46m Normal AddedInterface pod/e2e-ait-87191e54-openshift-default-7c59fcc9cc-6s8gq Add eth0 [10.134.0.76/23] from ovn-kubernetes 46m Warning ProbeError pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-w6c48 Readiness probe error: Get "http://10.134.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm Readiness probe failed: Get "http://10.132.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s Readiness probe error: Get "http://10.132.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s Readiness probe failed: Get "http://10.132.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm Readiness probe error: Get "http://10.132.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-w6c48 Readiness probe failed: Get "http://10.134.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-ljwqc Readiness probe error: Get "http://10.134.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-ljwqc Readiness probe failed: Get "http://10.134.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-rsw2s Readiness probe error: Get "http://10.133.0.127:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r Readiness probe failed: Get "http://10.132.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r Readiness probe error: Get "http://10.132.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-rsw2s Readiness probe failed: Get "http://10.133.0.127:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-2c1c4f09-8689cb4c5f Created pod: payload-pre-processing-e2e-migrate-2c1c4f09-8689cb4c5f-htpk4 46m Warning ProbeError pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6n8hr Readiness probe error: Get "http://10.133.0.134:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-2c1c4f09 Scaled up replica set payload-processing-e2e-migrate-2c1c4f09-79984b5b5b from 0 to 1 46m Warning Unhealthy pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6n8hr Readiness probe failed: Get "http://10.133.0.134:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning FailedCreate replicaset/payload-processing-e2e-migrate-2c1c4f09-79984b5b5b Error creating: pods "payload-processing-e2e-migrate-2c1c4f09-79984b5b5b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-2c1c4f09: serviceaccount "payload-processing-e2e-migrate-2c1c4f09" not found 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-2c1c4f09 Scaled up replica set payload-pre-processing-e2e-migrate-2c1c4f09-8689cb4c5f from 0 to 1 46m Warning Unhealthy pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-mngkl Readiness probe failed: Get "http://10.134.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-2c1c4f09-8689cb4c5f Error creating: pods "payload-pre-processing-e2e-migrate-2c1c4f09-8689cb4c5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-2c1c4f09: serviceaccount "payload-processing-e2e-migrate-2c1c4f09" not found 46m Warning ProbeError pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg Readiness probe error: Get "http://10.132.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg Readiness probe failed: Get "http://10.132.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-mngkl Readiness probe error: Get "http://10.134.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-2c1c4f09-79984b5b5b Created pod: payload-processing-e2e-migrate-2c1c4f09-79984b5b5b-8bb4n 46m Warning Unhealthy pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-vh6fd Readiness probe failed: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-vh6fd Readiness probe error: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Normal AddedInterface pod/payload-processing-e2e-migrate-2c1c4f09-79984b5b5b-8bb4n Add eth0 [10.133.0.160/23] from ovn-kubernetes 45m Normal Created pod/payload-pre-processing-e2e-migrate-2c1c4f09-8689cb4c5f-htpk4 Container created 45m Normal Started pod/payload-processing-e2e-migrate-2c1c4f09-79984b5b5b-8bb4n Container started 45m Normal Created pod/payload-processing-e2e-migrate-2c1c4f09-79984b5b5b-8bb4n Container created 45m Normal Pulled pod/payload-processing-e2e-migrate-2c1c4f09-79984b5b5b-8bb4n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Started pod/payload-pre-processing-e2e-migrate-2c1c4f09-8689cb4c5f-htpk4 Container started 45m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-2c1c4f09-8689cb4c5f-htpk4 Add eth0 [10.133.0.159/23] from ovn-kubernetes 45m Normal Pulled pod/payload-pre-processing-e2e-migrate-2c1c4f09-8689cb4c5f-htpk4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 45m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-t6h4m Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 45m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-t6h4m Add eth0 [10.133.0.162/23] from ovn-kubernetes 45m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 45m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-t6h4m 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-87191e54 Scaled up replica set payload-pre-processing-e2e-ait-87191e54-6ffd5f6b7 from 0 to 1 45m Warning FailedCreate replicaset/payload-processing-e2e-ait-87191e54-77b46c874 Error creating: pods "payload-processing-e2e-ait-87191e54-77b46c874-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-87191e54: serviceaccount "payload-processing-e2e-ait-87191e54" not found 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-87191e54-77b46c874 Created pod: payload-processing-e2e-ait-87191e54-77b46c874-mvmn9 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-87191e54 Scaled up replica set payload-processing-e2e-ait-87191e54-77b46c874 from 0 to 1 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-87191e54-6ffd5f6b7 Error creating: pods "payload-pre-processing-e2e-ait-87191e54-6ffd5f6b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-87191e54: serviceaccount "payload-processing-e2e-ait-87191e54" not found 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-87191e54-6ffd5f6b7 Created pod: payload-pre-processing-e2e-ait-87191e54-6ffd5f6b7-5qrkx 45m Normal Pulled pod/payload-pre-processing-e2e-ait-87191e54-6ffd5f6b7-5qrkx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Started pod/payload-pre-processing-e2e-ait-87191e54-6ffd5f6b7-5qrkx Container started 45m Normal AddedInterface pod/payload-processing-e2e-ait-87191e54-77b46c874-mvmn9 Add eth0 [10.134.0.78/23] from ovn-kubernetes 45m Normal Created pod/payload-pre-processing-e2e-ait-87191e54-6ffd5f6b7-5qrkx Container created 45m Normal Pulled pod/payload-processing-e2e-ait-87191e54-77b46c874-mvmn9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/payload-pre-processing-e2e-ait-87191e54-6ffd5f6b7-5qrkx Add eth0 [10.134.0.77/23] from ovn-kubernetes 45m Normal Created pod/payload-processing-e2e-ait-87191e54-77b46c874-mvmn9 Container created 45m Normal Started pod/payload-processing-e2e-ait-87191e54-77b46c874-mvmn9 Container started 45m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-t6h4m Container started 45m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-t6h4m Container created 45m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-t6h4m Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.492s (4.492s including waiting). Image size: 185476477 bytes. 45m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-t6h4m Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-t6h4m Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-2c1c4f09-gw-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-2c1c4f09-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal ScalingReplicaSet deployment/e2e-ait-f73e00ed-openshift-default Scaled up replica set e2e-ait-f73e00ed-openshift-default-65c6475fb5 from 0 to 1 45m Normal SuccessfulCreate replicaset/e2e-ait-f73e00ed-openshift-default-65c6475fb5 Created pod: e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l 45m Normal AddedLabel gateway/e2e-ait-f73e00ed Added label istio.io/rev=openshift-gateway to gateway e2e-ait-f73e00ed 45m Normal AddedInterface pod/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l Add eth0 [10.132.0.76/23] from ovn-kubernetes 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-f73e00ed Scaled up replica set payload-processing-e2e-ait-f73e00ed-866fdf7694 from 0 to 1 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-f73e00ed Scaled up replica set payload-pre-processing-e2e-ait-f73e00ed-9cc558679 from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-f73e00ed-866fdf7694 Created pod: payload-processing-e2e-ait-f73e00ed-866fdf7694-m4mwj 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-f73e00ed-9cc558679 Error creating: pods "payload-pre-processing-e2e-ait-f73e00ed-9cc558679-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-f73e00ed: serviceaccount "payload-processing-e2e-ait-f73e00ed" not found 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-f73e00ed-9cc558679 Created pod: payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv 45m Warning FailedCreate replicaset/payload-processing-e2e-ait-f73e00ed-866fdf7694 Error creating: pods "payload-processing-e2e-ait-f73e00ed-866fdf7694-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-f73e00ed: serviceaccount "payload-processing-e2e-ait-f73e00ed" not found 45m Normal AddedInterface pod/payload-processing-e2e-ait-f73e00ed-866fdf7694-m4mwj Add eth0 [10.134.0.79/23] from ovn-kubernetes 45m Normal Started pod/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv Container started 45m Normal Created pod/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv Container created 45m Normal Pulled pod/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv Add eth0 [10.132.0.77/23] from ovn-kubernetes 45m Normal Started pod/payload-processing-e2e-ait-f73e00ed-866fdf7694-m4mwj Container started 45m Normal Created pod/payload-processing-e2e-ait-f73e00ed-866fdf7694-m4mwj Container created 45m Normal Pulled pod/payload-processing-e2e-ait-f73e00ed-866fdf7694-m4mwj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-76597d6855 Created pod: maas-default-gateway-openshift-default-76597d6855-74hkj 45m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-76597d6855 Created pod: maas-default-gateway-openshift-default-76597d6855-xrlzk 45m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-76597d6855 Created pod: maas-default-gateway-openshift-default-76597d6855-7c95j 45m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 45m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-xrlzk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-7c95j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/maas-default-gateway-openshift-default-76597d6855-74hkj Container started 45m Normal Started pod/maas-default-gateway-openshift-default-76597d6855-xrlzk Container started 45m Normal Created pod/maas-default-gateway-openshift-default-76597d6855-74hkj Container created 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-76597d6855-74hkj Add eth0 [10.132.0.80/23] from ovn-kubernetes 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-76597d6855-xrlzk Add eth0 [10.132.0.79/23] from ovn-kubernetes 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-76597d6855-7c95j Add eth0 [10.134.0.81/23] from ovn-kubernetes 45m Normal Created pod/maas-default-gateway-openshift-default-76597d6855-xrlzk Container created 45m Normal Started pod/maas-default-gateway-openshift-default-76597d6855-7c95j Container started 45m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-74hkj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/maas-default-gateway-openshift-default-76597d6855-7c95j Container created 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-xrlzk Startup probe failed: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused 45m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-xrlzk Startup probe error: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused... 45m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-7c95j Startup probe error: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused... 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-74hkj Startup probe failed: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused 45m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-74hkj Startup probe error: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused... 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-7c95j Startup probe failed: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused 44m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-76597d6855 Created pod: maas-default-gateway-openshift-default-76597d6855-brntt 44m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-76597d6855 Created pod: maas-default-gateway-openshift-default-76597d6855-jwwww 44m Normal AddedInterface pod/maas-default-gateway-openshift-default-76597d6855-brntt Add eth0 [10.134.0.82/23] from ovn-kubernetes 44m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-brntt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/maas-default-gateway-openshift-default-76597d6855-brntt Container created 44m Normal Started pod/maas-default-gateway-openshift-default-76597d6855-brntt Container started 44m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-jwwww Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/maas-default-gateway-openshift-default-76597d6855-jwwww Add eth0 [10.133.0.164/23] from ovn-kubernetes 44m Normal Started pod/maas-default-gateway-openshift-default-76597d6855-jwwww Container started 44m Normal Created pod/maas-default-gateway-openshift-default-76597d6855-jwwww Container created 44m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-jwwww Startup probe failed: Get "http://10.133.0.164:15021/healthz/ready": dial tcp 10.133.0.164:15021: connect: connection refused 44m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-brntt Startup probe failed: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused 44m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-jwwww Startup probe error: Get "http://10.133.0.164:15021/healthz/ready": dial tcp 10.133.0.164:15021: connect: connection refused... 44m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-brntt Startup probe error: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused... 44m Normal Created pod/e2e-ait-87191e54-openshift-default-7c59fcc9cc-6s8gq Container created 44m Normal Pulled pod/e2e-ait-87191e54-openshift-default-7c59fcc9cc-6s8gq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Started pod/e2e-ait-87191e54-openshift-default-7c59fcc9cc-6s8gq Container started 44m Normal Started pod/e2e-migrate-2c1c4f09-gw-openshift-default-648b465cc-mr59v Container started 44m Normal Pulled pod/e2e-migrate-2c1c4f09-gw-openshift-default-648b465cc-mr59v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/e2e-migrate-2c1c4f09-gw-openshift-default-648b465cc-mr59v Container created 44m Warning ProbeError pod/e2e-ait-87191e54-openshift-default-7c59fcc9cc-6s8gq Startup probe error: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-ait-87191e54-openshift-default-7c59fcc9cc-6s8gq Startup probe failed: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused 44m Warning ProbeError pod/e2e-migrate-2c1c4f09-gw-openshift-default-648b465cc-mr59v Startup probe error: Get "http://10.133.0.156:15021/healthz/ready": dial tcp 10.133.0.156:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-migrate-2c1c4f09-gw-openshift-default-648b465cc-mr59v Startup probe failed: Get "http://10.133.0.156:15021/healthz/ready": dial tcp 10.133.0.156:15021: connect: connection refused 44m Normal Killing pod/payload-pre-processing-e2e-migrate-2c1c4f09-8689cb4c5f-htpk4 Stopping container payload-pre-processing 44m Normal Killing pod/payload-processing-e2e-migrate-2c1c4f09-79984b5b5b-8bb4n Stopping container payload-processing 43m Normal Killing pod/payload-processing-e2e-ait-87191e54-77b46c874-mvmn9 Stopping container payload-processing 43m Normal Killing pod/payload-pre-processing-e2e-ait-87191e54-6ffd5f6b7-5qrkx Stopping container payload-pre-processing 43m Normal Pulled pod/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l Container started 43m Normal Created pod/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l Container created 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-87191e54-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-87191e54-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning Unhealthy pod/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l Startup probe failed: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused 43m Warning ProbeError pod/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l Startup probe error: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused... 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-2c1c4f09-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-2c1c4f09-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal Killing pod/e2e-migrate-2c1c4f09-gw-openshift-default-648b465cc-mr59v Stopping container istio-proxy 43m Normal ScalingReplicaSet deployment/e2e-ait-4d5a0200-openshift-default Scaled up replica set e2e-ait-4d5a0200-openshift-default-8d965c49c from 0 to 1 43m Normal AddedLabel gateway/e2e-ait-4d5a0200 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-4d5a0200 43m Normal SuccessfulCreate replicaset/e2e-ait-4d5a0200-openshift-default-8d965c49c Created pod: e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9 43m Normal AddedInterface pod/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9 Add eth0 [10.132.0.82/23] from ovn-kubernetes 43m Normal Killing pod/payload-processing-e2e-ait-f73e00ed-866fdf7694-m4mwj Stopping container payload-processing 43m Normal Killing pod/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv Stopping container payload-pre-processing 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-f73e00ed-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-f73e00ed-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-76597d6855 Created pod: maas-default-gateway-openshift-default-76597d6855-jjxxp 43m Normal Killing pod/e2e-ait-87191e54-openshift-default-7c59fcc9cc-6s8gq Stopping container istio-proxy 43m Normal AddedInterface pod/maas-default-gateway-openshift-default-76597d6855-jjxxp Add eth0 [10.134.0.83/23] from ovn-kubernetes 43m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-jjxxp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/maas-default-gateway-openshift-default-76597d6855-jjxxp Container started 43m Normal Created pod/maas-default-gateway-openshift-default-76597d6855-jjxxp Container created 43m Warning ProbeError pod/e2e-ait-87191e54-openshift-default-7c59fcc9cc-6s8gq Readiness probe error: HTTP probe failed with statuscode: 503... 43m Warning Unhealthy pod/e2e-ait-87191e54-openshift-default-7c59fcc9cc-6s8gq Readiness probe failed: HTTP probe failed with statuscode: 503 43m Normal AddedLabel gateway/e2e-derive-5fde04ee Added label istio.io/rev=openshift-gateway to gateway e2e-derive-5fde04ee 43m Normal AddedInterface pod/e2e-derive-5fde04ee-openshift-default-7d7db879c-bxx4z Add eth0 [10.133.0.169/23] from ovn-kubernetes 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-4d5a0200-5cdf8ddbcd Created pod: payload-processing-e2e-ait-4d5a0200-5cdf8ddbcd-wdl8z 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-4d5a0200 Scaled up replica set payload-processing-e2e-ait-4d5a0200-5cdf8ddbcd from 0 to 1 43m Warning FailedCreate replicaset/payload-processing-e2e-ait-4d5a0200-5cdf8ddbcd Error creating: pods "payload-processing-e2e-ait-4d5a0200-5cdf8ddbcd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-4d5a0200: serviceaccount "payload-processing-e2e-ait-4d5a0200" not found 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-4d5a0200 Scaled up replica set payload-pre-processing-e2e-ait-4d5a0200-5959b9699 from 0 to 1 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-4d5a0200-5959b9699 Error creating: pods "payload-pre-processing-e2e-ait-4d5a0200-5959b9699-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-4d5a0200: serviceaccount "payload-processing-e2e-ait-4d5a0200" not found 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-4d5a0200-5959b9699 Created pod: payload-pre-processing-e2e-ait-4d5a0200-5959b9699-k2dpp 43m Normal SuccessfulCreate replicaset/e2e-derive-5fde04ee-openshift-default-7d7db879c Created pod: e2e-derive-5fde04ee-openshift-default-7d7db879c-bxx4z 43m Normal ScalingReplicaSet deployment/e2e-derive-5fde04ee-openshift-default Scaled up replica set e2e-derive-5fde04ee-openshift-default-7d7db879c from 0 to 1 43m Normal AddedInterface pod/payload-processing-e2e-ait-4d5a0200-5cdf8ddbcd-wdl8z Add eth0 [10.134.0.85/23] from ovn-kubernetes 43m Normal Pulled pod/payload-pre-processing-e2e-ait-4d5a0200-5959b9699-k2dpp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-pre-processing-e2e-ait-4d5a0200-5959b9699-k2dpp Add eth0 [10.134.0.84/23] from ovn-kubernetes 43m Warning Unhealthy pod/payload-pre-processing-e2e-ait-f73e00ed-9cc558679-mgjkv Readiness probe failed: timeout: failed to connect service "10.132.0.77:50052" within 1s: context deadline exceeded 43m Normal Created pod/payload-pre-processing-e2e-ait-4d5a0200-5959b9699-k2dpp Container created 43m Normal Pulled pod/payload-processing-e2e-ait-4d5a0200-5cdf8ddbcd-wdl8z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-processing-e2e-ait-4d5a0200-5cdf8ddbcd-wdl8z Container started 43m Normal Created pod/payload-processing-e2e-ait-4d5a0200-5cdf8ddbcd-wdl8z Container created 43m Normal Started pod/payload-pre-processing-e2e-ait-4d5a0200-5959b9699-k2dpp Container started 43m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-jjxxp Startup probe error: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused... 43m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-jjxxp Startup probe failed: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-5fde04ee Scaled up replica set payload-processing-e2e-derive-5fde04ee-5fdc7f866d from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-5fde04ee-5fdc7f866d Created pod: payload-processing-e2e-derive-5fde04ee-5fdc7f866d-lk25n 43m Warning FailedCreate replicaset/payload-processing-e2e-derive-5fde04ee-5fdc7f866d Error creating: pods "payload-processing-e2e-derive-5fde04ee-5fdc7f866d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-5fde04ee: serviceaccount "payload-processing-e2e-derive-5fde04ee" not found 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-5fde04ee Scaled up replica set payload-pre-processing-e2e-derive-5fde04ee-59df47b5d7 from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-5fde04ee-59df47b5d7 Created pod: payload-pre-processing-e2e-derive-5fde04ee-59df47b5d7-gnjpp 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-5fde04ee-59df47b5d7 Error creating: pods "payload-pre-processing-e2e-derive-5fde04ee-59df47b5d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-5fde04ee: serviceaccount "payload-processing-e2e-derive-5fde04ee" not found 43m Normal AddedInterface pod/payload-processing-e2e-derive-5fde04ee-5fdc7f866d-lk25n Add eth0 [10.134.0.87/23] from ovn-kubernetes 43m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-jjxxp Startup probe failed: Get "http://10.134.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Normal AddedInterface pod/payload-pre-processing-e2e-derive-5fde04ee-59df47b5d7-gnjpp Add eth0 [10.134.0.86/23] from ovn-kubernetes 43m Normal Pulled pod/payload-pre-processing-e2e-derive-5fde04ee-59df47b5d7-gnjpp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-pre-processing-e2e-derive-5fde04ee-59df47b5d7-gnjpp Container created 43m Normal Started pod/payload-pre-processing-e2e-derive-5fde04ee-59df47b5d7-gnjpp Container started 43m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-jjxxp Startup probe error: Get "http://10.134.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Normal Pulled pod/payload-processing-e2e-derive-5fde04ee-5fdc7f866d-lk25n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-processing-e2e-derive-5fde04ee-5fdc7f866d-lk25n Container started 43m Normal Created pod/payload-processing-e2e-derive-5fde04ee-5fdc7f866d-lk25n Container created 43m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default (combined from similar events): New size: 10; reason: cpu resource utilization (percentage of request) above target 43m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-76597d6855 Created pod: maas-default-gateway-openshift-default-76597d6855-xr2tj 43m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-76597d6855 Created pod: maas-default-gateway-openshift-default-76597d6855-dtlt4 43m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default (combined from similar events): Scaled up replica set maas-default-gateway-openshift-default-76597d6855 from 7 to 10 43m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-76597d6855 Created pod: maas-default-gateway-openshift-default-76597d6855-rlhz6 43m Normal Created pod/maas-default-gateway-openshift-default-76597d6855-xr2tj Container created 43m Normal AddedInterface pod/maas-default-gateway-openshift-default-76597d6855-dtlt4 Add eth0 [10.134.0.88/23] from ovn-kubernetes 43m Normal Started pod/maas-default-gateway-openshift-default-76597d6855-rlhz6 Container started 43m Normal AddedInterface pod/maas-default-gateway-openshift-default-76597d6855-rlhz6 Add eth0 [10.132.0.83/23] from ovn-kubernetes 43m Normal Created pod/maas-default-gateway-openshift-default-76597d6855-rlhz6 Container created 43m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-rlhz6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-dtlt4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/maas-default-gateway-openshift-default-76597d6855-xr2tj Container started 43m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-xr2tj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/maas-default-gateway-openshift-default-76597d6855-xr2tj Add eth0 [10.133.0.170/23] from ovn-kubernetes 43m Normal Created pod/maas-default-gateway-openshift-default-76597d6855-dtlt4 Container created 43m Normal Started pod/maas-default-gateway-openshift-default-76597d6855-dtlt4 Container started 43m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-xr2tj Startup probe error: Get "http://10.133.0.170:15021/healthz/ready": dial tcp 10.133.0.170:15021: connect: connection refused... 43m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-xr2tj Startup probe failed: Get "http://10.133.0.170:15021/healthz/ready": dial tcp 10.133.0.170:15021: connect: connection refused 43m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-rlhz6 Startup probe error: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused... 43m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-dtlt4 Startup probe error: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused... 43m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-dtlt4 Startup probe failed: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused 43m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-rlhz6 Startup probe failed: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused 42m Normal Killing pod/e2e-ait-f73e00ed-openshift-default-65c6475fb5-nd46l Stopping container istio-proxy 42m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-072ece-openshift-default-78b784fd8c Created pod: e2e-shared-a-w4-072ece-openshift-default-78b784fd8c-k2vfq 42m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-072ece-openshift-default Scaled up replica set e2e-shared-a-w4-072ece-openshift-default-78b784fd8c from 0 to 1 42m Normal AddedLabel gateway/e2e-shared-a-w4-072ece Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-072ece 42m Normal AddedInterface pod/e2e-shared-a-w4-072ece-openshift-default-78b784fd8c-k2vfq Add eth0 [10.134.0.89/23] from ovn-kubernetes 42m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: cpu resource utilization (percentage of request) above target 42m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 3 42m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-2g6x2 42m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-2g6x2 Add eth0 [10.134.0.90/23] from ovn-kubernetes 42m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-2g6x2 Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w4-072ece Scaled up replica set payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w4-072ece-8f4847696 Created pod: payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff Created pod: payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk 42m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w4-072ece-8f4847696 Error creating: pods "payload-processing-e2e-shared-a-w4-072ece-8f4847696-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-072ece: serviceaccount "payload-processing-e2e-shared-a-w4-072ece" not found 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w4-072ece Scaled up replica set payload-processing-e2e-shared-a-w4-072ece-8f4847696 from 0 to 1 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff Error creating: pods "payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-072ece: serviceaccount "payload-processing-e2e-shared-a-w4-072ece" not found 42m Normal Created pod/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8 Container created 42m Normal Created pod/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk Container created 42m Normal Pulled pod/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Started pod/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8 Container started 42m Normal AddedInterface pod/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8 Add eth0 [10.132.0.85/23] from ovn-kubernetes 42m Normal Started pod/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk Container started 42m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk Add eth0 [10.132.0.84/23] from ovn-kubernetes 42m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-2g6x2 Container created 42m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-2g6x2 Container started 42m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-2g6x2 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 9.504s (9.504s including waiting). Image size: 185476477 bytes. 42m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-2g6x2 Readiness probe failed: HTTP probe failed with statuscode: 503 42m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-2g6x2 Readiness probe error: HTTP probe failed with statuscode: 503... 42m Normal AddedLabel gateway/e2e-shared-a-w5-297806 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w5-297806 42m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-297806-openshift-default-657d54b457 Created pod: e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67 42m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-297806-openshift-default Scaled up replica set e2e-shared-a-w5-297806-openshift-default-657d54b457 from 0 to 1 42m Normal AddedInterface pod/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67 Add eth0 [10.132.0.87/23] from ovn-kubernetes 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w5-297806 Scaled up replica set payload-processing-e2e-shared-a-w5-297806-6b77744954 from 0 to 1 42m Normal AddedLabel gateway/e2e-mt-db4d0787 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-db4d0787 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w5-297806 Scaled up replica set payload-pre-processing-e2e-shared-a-w5-297806-bdfd869bc from 0 to 1 42m Normal ScalingReplicaSet deployment/e2e-mt-db4d0787-openshift-default Scaled up replica set e2e-mt-db4d0787-openshift-default-7bd494bcfc from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w5-297806-bdfd869bc Created pod: payload-pre-processing-e2e-shared-a-w5-297806-bdfd869bc-8j6sw 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w5-297806-6b77744954 Created pod: payload-processing-e2e-shared-a-w5-297806-6b77744954-k4jwf 42m Normal SuccessfulCreate replicaset/e2e-mt-db4d0787-openshift-default-7bd494bcfc Created pod: e2e-mt-db4d0787-openshift-default-7bd494bcfc-zqfcj 42m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w5-297806-6b77744954 Error creating: pods "payload-processing-e2e-shared-a-w5-297806-6b77744954-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-297806: serviceaccount "payload-processing-e2e-shared-a-w5-297806" not found 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w5-297806-bdfd869bc Error creating: pods "payload-pre-processing-e2e-shared-a-w5-297806-bdfd869bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-297806: serviceaccount "payload-processing-e2e-shared-a-w5-297806" not found 42m Normal Pulled pod/payload-processing-e2e-shared-a-w5-297806-6b77744954-k4jwf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-processing-e2e-shared-a-w5-297806-6b77744954-k4jwf Add eth0 [10.133.0.174/23] from ovn-kubernetes 42m Normal Created pod/payload-processing-e2e-shared-a-w5-297806-6b77744954-k4jwf Container created 42m Normal AddedInterface pod/e2e-mt-db4d0787-openshift-default-7bd494bcfc-zqfcj Add eth0 [10.133.0.172/23] from ovn-kubernetes 42m Normal Started pod/payload-pre-processing-e2e-shared-a-w5-297806-bdfd869bc-8j6sw Container started 42m Normal Created pod/payload-pre-processing-e2e-shared-a-w5-297806-bdfd869bc-8j6sw Container created 42m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w5-297806-bdfd869bc-8j6sw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w5-297806-bdfd869bc-8j6sw Add eth0 [10.133.0.173/23] from ovn-kubernetes 42m Normal Started pod/payload-processing-e2e-shared-a-w5-297806-6b77744954-k4jwf Container started 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-db4d0787 Scaled up replica set payload-pre-processing-e2e-mt-db4d0787-57486cbfdc from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-db4d0787-5558ff6d68 Created pod: payload-processing-e2e-mt-db4d0787-5558ff6d68-hdw6t 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-db4d0787 Scaled up replica set payload-processing-e2e-mt-db4d0787-5558ff6d68 from 0 to 1 42m Warning FailedCreate replicaset/payload-processing-e2e-mt-db4d0787-5558ff6d68 Error creating: pods "payload-processing-e2e-mt-db4d0787-5558ff6d68-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-db4d0787: serviceaccount "payload-processing-e2e-mt-db4d0787" not found 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc Error creating: pods "payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-db4d0787: serviceaccount "payload-processing-e2e-mt-db4d0787" not found 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc Created pod: payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5 42m Normal Pulled pod/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Pulled pod/payload-processing-e2e-mt-db4d0787-5558ff6d68-hdw6t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Started pod/payload-processing-e2e-mt-db4d0787-5558ff6d68-hdw6t Container started 42m Normal AddedInterface pod/payload-processing-e2e-mt-db4d0787-5558ff6d68-hdw6t Add eth0 [10.133.0.175/23] from ovn-kubernetes 42m Normal Started pod/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5 Container started 42m Normal AddedInterface pod/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5 Add eth0 [10.132.0.89/23] from ovn-kubernetes 42m Normal Created pod/payload-processing-e2e-mt-db4d0787-5558ff6d68-hdw6t Container created 42m Normal Created pod/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5 Container created 42m Normal SuccessfulDelete replicaset/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c Deleted pod: e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-pnfhp 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-fe4e0c-openshift-default New size: 8; reason: All metrics below target 42m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-zhmt8 Readiness probe failed: Get "http://10.133.0.32:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-zhmt8 Readiness probe error: Get "http://10.133.0.32:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Normal SuccessfulDelete replicaset/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c Deleted pod: e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-vh6fd 42m Normal Killing pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-pnfhp Stopping container istio-proxy 42m Normal Killing pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-vh6fd Stopping container istio-proxy 42m Normal ScalingReplicaSet deployment/e2e-worker-w0-fe4e0c-openshift-default Scaled down replica set e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c from 10 to 8 42m Warning Unhealthy pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-pnfhp Readiness probe failed: HTTP probe failed with statuscode: 503 42m Warning ProbeError pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-pnfhp Readiness probe error: HTTP probe failed with statuscode: 503... 42m Warning Unhealthy pod/e2e-derive-5fde04ee-openshift-default-7d7db879c-bxx4z Readiness probe failed: Get "http://10.133.0.169:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Warning ProbeError pod/e2e-derive-5fde04ee-openshift-default-7d7db879c-bxx4z Readiness probe error: Get "http://10.133.0.169:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Normal Pulled pod/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9 Container created 42m Normal Started pod/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9 Container started 42m Normal Created pod/e2e-derive-5fde04ee-openshift-default-7d7db879c-bxx4z Container created 42m Normal Started pod/e2e-derive-5fde04ee-openshift-default-7d7db879c-bxx4z Container started 42m Normal Pulled pod/e2e-derive-5fde04ee-openshift-default-7d7db879c-bxx4z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Warning ProbeError pod/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9 Startup probe error: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9 Startup probe failed: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused 41m Warning ProbeError pod/e2e-derive-5fde04ee-openshift-default-7d7db879c-bxx4z Startup probe error: Get "http://10.133.0.169:15021/healthz/ready": dial tcp 10.133.0.169:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-derive-5fde04ee-openshift-default-7d7db879c-bxx4z Startup probe failed: Get "http://10.133.0.169:15021/healthz/ready": dial tcp 10.133.0.169:15021: connect: connection refused 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-5fde04ee-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-5fde04ee-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-4d5a0200-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-4d5a0200-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-5fde04ee-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-5fde04ee-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-4d5a0200-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-4d5a0200-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal Killing pod/payload-processing-e2e-ait-4d5a0200-5cdf8ddbcd-wdl8z Stopping container payload-processing 41m Normal Killing pod/payload-pre-processing-e2e-ait-4d5a0200-5959b9699-k2dpp Stopping container payload-pre-processing 41m Normal Pulled pod/e2e-shared-a-w4-072ece-openshift-default-78b784fd8c-k2vfq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Started pod/e2e-shared-a-w4-072ece-openshift-default-78b784fd8c-k2vfq Container started 41m Normal Created pod/e2e-shared-a-w4-072ece-openshift-default-78b784fd8c-k2vfq Container created 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-072ece-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-072ece-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning Unhealthy pod/e2e-shared-a-w4-072ece-openshift-default-78b784fd8c-k2vfq Startup probe failed: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused 41m Warning ProbeError pod/e2e-shared-a-w4-072ece-openshift-default-78b784fd8c-k2vfq Startup probe error: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused... 41m Normal SuccessfulDelete replicaset/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 Deleted pod: e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-d5qtm 41m Normal Killing pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-d5qtm Stopping container istio-proxy 41m Normal ScalingReplicaSet deployment/e2e-worker-w3-cdcf67-openshift-default Scaled down replica set e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 from 10 to 8 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-cdcf67-openshift-default New size: 8; reason: All metrics below target 41m Normal Killing pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-v5wsm Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 Deleted pod: e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-v5wsm 41m Normal Killing pod/payload-processing-e2e-derive-5fde04ee-5fdc7f866d-lk25n Stopping container payload-processing 41m Normal Killing pod/payload-pre-processing-e2e-derive-5fde04ee-59df47b5d7-gnjpp Stopping container payload-pre-processing 41m Warning Unhealthy pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-v5wsm Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning ProbeError pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-v5wsm Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-d5qtm Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning ProbeError pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-d5qtm Readiness probe error: HTTP probe failed with statuscode: 503... 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-cdcf67-openshift-default New size: 5; reason: All metrics below target 40m Normal SuccessfulDelete replicaset/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c Deleted pod: e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-j4n6p 40m Normal SuccessfulDelete replicaset/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c Deleted pod: e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-m97sr 40m Normal Killing pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-fzwkp Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-j4n6p Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c Deleted pod: e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-fzwkp 40m Normal ScalingReplicaSet deployment/e2e-worker-w3-cdcf67-openshift-default Scaled down replica set e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 from 8 to 5 40m Normal ScalingReplicaSet deployment/e2e-worker-w2-a0bcbc-openshift-default Scaled down replica set e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c from 10 to 6 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-a0bcbc-openshift-default New size: 6; reason: All metrics below target 40m Normal SuccessfulDelete replicaset/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c Deleted pod: e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-g52rd 40m Normal Killing pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-g52rd Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-m97sr Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 Deleted pod: e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl 40m Normal Killing pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl Stopping container istio-proxy 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-072ece-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal Killing pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-vbc52 Stopping container istio-proxy 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-072ece-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal SuccessfulDelete replicaset/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 Deleted pod: e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn 40m Normal SuccessfulDelete replicaset/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 Deleted pod: e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-vbc52 40m Warning ProbeError pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-5znpl Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning ProbeError pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-m97sr Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-m97sr Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning Unhealthy pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-vbc52 Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning ProbeError pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-j4n6p Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-j4n6p Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning ProbeError pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-vbc52 Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-g52rd Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning Unhealthy pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning ProbeError pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-g52rd Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning ProbeError pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-cxlvn Readiness probe error: HTTP probe failed with statuscode: 503... 40m Normal AddedLabel gateway/e2e-shared-b-w4-35df3e Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-35df3e 40m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-35df3e-openshift-default-6c585b76f Created pod: e2e-shared-b-w4-35df3e-openshift-default-6c585b76f-vl6wz 40m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-35df3e-openshift-default Scaled up replica set e2e-shared-b-w4-35df3e-openshift-default-6c585b76f from 0 to 1 40m Normal AddedInterface pod/e2e-shared-b-w4-35df3e-openshift-default-6c585b76f-vl6wz Add eth0 [10.134.0.96/23] from ovn-kubernetes 40m Normal Killing pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-ljwqc Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w4-203c35-openshift-default-56497cdcc Deleted pod: e2e-worker-w4-203c35-openshift-default-56497cdcc-2b66r 40m Normal Killing pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-frgsz Stopping container istio-proxy 40m Normal ScalingReplicaSet deployment/e2e-worker-w4-203c35-openshift-default Scaled down replica set e2e-worker-w4-203c35-openshift-default-56497cdcc from 10 to 6 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-203c35-openshift-default New size: 6; reason: All metrics below target 40m Normal ScalingReplicaSet deployment/e2e-worker-w6-ebb7fd-openshift-default Scaled down replica set e2e-worker-w6-ebb7fd-openshift-default-544f5768f from 10 to 6 40m Normal Killing pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-x449v Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-z59vn Stopping container istio-proxy 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-ebb7fd-openshift-default New size: 6; reason: All metrics below target 40m Normal Killing pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-w6c48 Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2b66r Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w4-203c35-openshift-default-56497cdcc Deleted pod: e2e-worker-w4-203c35-openshift-default-56497cdcc-w6c48 40m Normal SuccessfulDelete replicaset/e2e-worker-w6-ebb7fd-openshift-default-544f5768f Deleted pod: e2e-worker-w6-ebb7fd-openshift-default-544f5768f-4qkqm 40m Normal SuccessfulDelete replicaset/e2e-worker-w6-ebb7fd-openshift-default-544f5768f Deleted pod: e2e-worker-w6-ebb7fd-openshift-default-544f5768f-x449v 40m Normal SuccessfulDelete replicaset/e2e-worker-w6-ebb7fd-openshift-default-544f5768f Deleted pod: e2e-worker-w6-ebb7fd-openshift-default-544f5768f-db5nw 40m Normal SuccessfulDelete replicaset/e2e-worker-w6-ebb7fd-openshift-default-544f5768f Deleted pod: e2e-worker-w6-ebb7fd-openshift-default-544f5768f-frgsz 40m Normal Killing pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-4qkqm Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w4-203c35-openshift-default-56497cdcc Deleted pod: e2e-worker-w4-203c35-openshift-default-56497cdcc-ljwqc 40m Normal SuccessfulDelete replicaset/e2e-worker-w4-203c35-openshift-default-56497cdcc Deleted pod: e2e-worker-w4-203c35-openshift-default-56497cdcc-z59vn 40m Warning Unhealthy pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-x449v Readiness probe failed: HTTP probe failed with statuscode: 503 40m Normal Killing pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-db5nw Stopping container istio-proxy 40m Warning ProbeError pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-x449v Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning ProbeError pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2b66r Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2b66r Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning ProbeError pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-z59vn Readiness probe error: HTTP probe failed with statuscode: 503... 40m Normal Killing pod/e2e-ait-4d5a0200-openshift-default-8d965c49c-p54t9 Stopping container istio-proxy 40m Warning Unhealthy pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-z59vn Readiness probe failed: HTTP probe failed with statuscode: 503 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w4-35df3e Scaled up replica set payload-pre-processing-e2e-shared-b-w4-35df3e-7d67b7f575 from 0 to 1 40m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w4-35df3e-7969f6bd6b Error creating: pods "payload-processing-e2e-shared-b-w4-35df3e-7969f6bd6b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-35df3e: serviceaccount "payload-processing-e2e-shared-b-w4-35df3e" not found 40m Normal AddedLabel gateway/e2e-shared-a-w2-57f116 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-57f116 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w4-35df3e-7d67b7f575 Created pod: payload-pre-processing-e2e-shared-b-w4-35df3e-7d67b7f575-zbnm5 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w4-35df3e Scaled up replica set payload-processing-e2e-shared-b-w4-35df3e-7969f6bd6b from 0 to 1 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w4-35df3e-7d67b7f575 Error creating: pods "payload-pre-processing-e2e-shared-b-w4-35df3e-7d67b7f575-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-35df3e: serviceaccount "payload-processing-e2e-shared-b-w4-35df3e" not found 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w4-35df3e-7969f6bd6b Created pod: payload-processing-e2e-shared-b-w4-35df3e-7969f6bd6b-ptzqg 40m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-57f116-openshift-default Scaled up replica set e2e-shared-a-w2-57f116-openshift-default-75f4fc499 from 0 to 1 40m Normal Pulled pod/payload-processing-e2e-shared-b-w4-35df3e-7969f6bd6b-ptzqg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Warning Unhealthy pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-frgsz Readiness probe failed: Get "http://10.133.0.141:15021/healthz/ready": dial tcp 10.133.0.141:15021: connect: connection refused 40m Warning Unhealthy pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-db5nw Readiness probe failed: Get "http://10.133.0.140:15021/healthz/ready": dial tcp 10.133.0.140:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-frgsz Readiness probe error: Get "http://10.133.0.141:15021/healthz/ready": dial tcp 10.133.0.141:15021: connect: connection refused... 40m Normal AddedInterface pod/payload-processing-e2e-shared-b-w4-35df3e-7969f6bd6b-ptzqg Add eth0 [10.134.0.98/23] from ovn-kubernetes 40m Warning ProbeError pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-db5nw Readiness probe error: Get "http://10.133.0.140:15021/healthz/ready": dial tcp 10.133.0.140:15021: connect: connection refused... 40m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-57f116-openshift-default-75f4fc499 Created pod: e2e-shared-a-w2-57f116-openshift-default-75f4fc499-znglh 40m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w4-35df3e-7d67b7f575-zbnm5 Add eth0 [10.134.0.97/23] from ovn-kubernetes 40m Normal Created pod/payload-processing-e2e-shared-b-w4-35df3e-7969f6bd6b-ptzqg Container created 40m Normal Started pod/payload-processing-e2e-shared-b-w4-35df3e-7969f6bd6b-ptzqg Container started 40m Normal AddedInterface pod/e2e-shared-a-w2-57f116-openshift-default-75f4fc499-znglh Add eth0 [10.134.0.99/23] from ovn-kubernetes 40m Normal Started pod/payload-pre-processing-e2e-shared-b-w4-35df3e-7d67b7f575-zbnm5 Container started 40m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w4-35df3e-7d67b7f575-zbnm5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Created pod/payload-pre-processing-e2e-shared-b-w4-35df3e-7d67b7f575-zbnm5 Container created 40m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-551e2b-openshift-default Scaled up replica set e2e-shared-a-w1-551e2b-openshift-default-5cb6c84ff7 from 0 to 1 40m Normal AddedLabel gateway/e2e-shared-a-w1-551e2b Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w1-551e2b 40m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-551e2b-openshift-default-5cb6c84ff7 Created pod: e2e-shared-a-w1-551e2b-openshift-default-5cb6c84ff7-bxgnt 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-35df3e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-35df3e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal AddedInterface pod/e2e-shared-a-w1-551e2b-openshift-default-5cb6c84ff7-bxgnt Add eth0 [10.133.0.184/23] from ovn-kubernetes 40m Warning ProbeError pod/e2e-mt-db4d0787-openshift-default-7bd494bcfc-zqfcj Readiness probe error: Get "http://10.133.0.172:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Normal AddedLabel gateway/e2e-shared-b-w5-4b2bf4 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w5-4b2bf4 40m Warning Unhealthy pod/e2e-mt-db4d0787-openshift-default-7bd494bcfc-zqfcj Readiness probe failed: Get "http://10.133.0.172:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Normal Killing pod/e2e-derive-5fde04ee-openshift-default-7d7db879c-bxx4z Stopping container istio-proxy 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-57f116 Scaled up replica set payload-processing-e2e-shared-a-w2-57f116-5b5dd4bd44 from 0 to 1 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-57f116-5b5dd4bd44 Created pod: payload-processing-e2e-shared-a-w2-57f116-5b5dd4bd44-fzgp7 40m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-4b2bf4-openshift-default-59785c9d99 Created pod: e2e-shared-b-w5-4b2bf4-openshift-default-59785c9d99-b52gb 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-57f116-599cbf5bbc Error creating: pods "payload-pre-processing-e2e-shared-a-w2-57f116-599cbf5bbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-57f116: serviceaccount "payload-processing-e2e-shared-a-w2-57f116" not found 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-57f116-599cbf5bbc Created pod: payload-pre-processing-e2e-shared-a-w2-57f116-599cbf5bbc-2fp4d 40m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-57f116-5b5dd4bd44 Error creating: pods "payload-processing-e2e-shared-a-w2-57f116-5b5dd4bd44-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-57f116: serviceaccount "payload-processing-e2e-shared-a-w2-57f116" not found 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-57f116 Scaled up replica set payload-pre-processing-e2e-shared-a-w2-57f116-599cbf5bbc from 0 to 1 40m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-4b2bf4-openshift-default Scaled up replica set e2e-shared-b-w5-4b2bf4-openshift-default-59785c9d99 from 0 to 1 40m Normal Started pod/payload-processing-e2e-shared-a-w2-57f116-5b5dd4bd44-fzgp7 Container started 40m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-57f116-599cbf5bbc-2fp4d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-57f116-599cbf5bbc-2fp4d Container created 40m Normal Pulled pod/payload-processing-e2e-shared-a-w2-57f116-5b5dd4bd44-fzgp7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-57f116-599cbf5bbc-2fp4d Add eth0 [10.134.0.101/23] from ovn-kubernetes 40m Normal AddedInterface pod/e2e-shared-b-w5-4b2bf4-openshift-default-59785c9d99-b52gb Add eth0 [10.134.0.100/23] from ovn-kubernetes 40m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-57f116-599cbf5bbc-2fp4d Container started 40m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-57f116-5b5dd4bd44-fzgp7 Add eth0 [10.134.0.102/23] from ovn-kubernetes 40m Normal Created pod/payload-processing-e2e-shared-a-w2-57f116-5b5dd4bd44-fzgp7 Container created 40m Normal Created pod/e2e-mt-db4d0787-openshift-default-7bd494bcfc-zqfcj Container created 40m Normal Started pod/e2e-mt-db4d0787-openshift-default-7bd494bcfc-zqfcj Container started 40m Normal Pulled pod/e2e-mt-db4d0787-openshift-default-7bd494bcfc-zqfcj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Warning Unhealthy pod/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67 Readiness probe failed: Get "http://10.132.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Warning ProbeError pod/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67 Readiness probe error: Get "http://10.132.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Normal Started pod/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67 Container started 40m Normal Created pod/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67 Container created 40m Normal Pulled pod/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w1-551e2b Scaled up replica set payload-pre-processing-e2e-shared-a-w1-551e2b-869c56d547 from 0 to 1 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w1-551e2b Scaled up replica set payload-processing-e2e-shared-a-w1-551e2b-79ddfccbf8 from 0 to 1 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w1-551e2b-869c56d547 Created pod: payload-pre-processing-e2e-shared-a-w1-551e2b-869c56d547-549jg 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w1-551e2b-869c56d547 Error creating: pods "payload-pre-processing-e2e-shared-a-w1-551e2b-869c56d547-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-551e2b: serviceaccount "payload-processing-e2e-shared-a-w1-551e2b" not found 40m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w1-551e2b-79ddfccbf8 Error creating: pods "payload-processing-e2e-shared-a-w1-551e2b-79ddfccbf8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-551e2b: serviceaccount "payload-processing-e2e-shared-a-w1-551e2b" not found 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w1-551e2b-79ddfccbf8 Created pod: payload-processing-e2e-shared-a-w1-551e2b-79ddfccbf8-6rkbp 40m Warning Unhealthy pod/e2e-mt-db4d0787-openshift-default-7bd494bcfc-zqfcj Startup probe failed: Get "http://10.133.0.172:15021/healthz/ready": dial tcp 10.133.0.172:15021: connect: connection refused 40m Warning ProbeError pod/e2e-mt-db4d0787-openshift-default-7bd494bcfc-zqfcj Startup probe error: Get "http://10.133.0.172:15021/healthz/ready": dial tcp 10.133.0.172:15021: connect: connection refused... 40m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w1-551e2b-869c56d547-549jg Add eth0 [10.133.0.185/23] from ovn-kubernetes 40m Normal Created pod/payload-pre-processing-e2e-shared-a-w1-551e2b-869c56d547-549jg Container created 40m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w1-551e2b-869c56d547-549jg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Pulled pod/payload-processing-e2e-shared-a-w1-551e2b-79ddfccbf8-6rkbp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Created pod/payload-processing-e2e-shared-a-w1-551e2b-79ddfccbf8-6rkbp Container created 40m Normal Started pod/payload-processing-e2e-shared-a-w1-551e2b-79ddfccbf8-6rkbp Container started 40m Normal AddedInterface pod/payload-processing-e2e-shared-a-w1-551e2b-79ddfccbf8-6rkbp Add eth0 [10.134.0.103/23] from ovn-kubernetes 40m Normal Started pod/payload-pre-processing-e2e-shared-a-w1-551e2b-869c56d547-549jg Container started 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-db4d0787-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-db4d0787-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning Unhealthy pod/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67 Startup probe failed: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused 40m Normal ScalingReplicaSet deployment/e2e-worker-w5-3b7476-openshift-default Scaled down replica set e2e-worker-w5-3b7476-openshift-default-59975457bf from 10 to 9 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-3b7476-openshift-default New size: 9; reason: All metrics below target 40m Warning ProbeError pod/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67 Startup probe error: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused... 40m Normal SuccessfulDelete replicaset/e2e-worker-w5-3b7476-openshift-default-59975457bf Deleted pod: e2e-worker-w5-3b7476-openshift-default-59975457bf-f9flw 40m Normal Killing pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-f9flw Stopping container istio-proxy 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w5-4b2bf4 Scaled up replica set payload-pre-processing-e2e-shared-b-w5-4b2bf4-6cd5fc4b6c from 0 to 1 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w5-4b2bf4-6cd5fc4b6c Error creating: pods "payload-pre-processing-e2e-shared-b-w5-4b2bf4-6cd5fc4b6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-4b2bf4: serviceaccount "payload-processing-e2e-shared-b-w5-4b2bf4" not found 40m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w5-4b2bf4-69448f59dd Error creating: pods "payload-processing-e2e-shared-b-w5-4b2bf4-69448f59dd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-4b2bf4: serviceaccount "payload-processing-e2e-shared-b-w5-4b2bf4" not found 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w5-4b2bf4-6cd5fc4b6c Created pod: payload-pre-processing-e2e-shared-b-w5-4b2bf4-6cd5fc4b6c-wwqfq 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w5-4b2bf4 Scaled up replica set payload-processing-e2e-shared-b-w5-4b2bf4-69448f59dd from 0 to 1 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-297806-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-297806-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal Created pod/payload-processing-e2e-shared-b-w5-4b2bf4-69448f59dd-fqq6g Container created 40m Normal AddedInterface pod/payload-processing-e2e-shared-b-w5-4b2bf4-69448f59dd-fqq6g Add eth0 [10.133.0.188/23] from ovn-kubernetes 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w5-4b2bf4-69448f59dd Created pod: payload-processing-e2e-shared-b-w5-4b2bf4-69448f59dd-fqq6g 40m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w5-4b2bf4-6cd5fc4b6c-wwqfq Add eth0 [10.133.0.187/23] from ovn-kubernetes 40m Normal Started pod/payload-processing-e2e-shared-b-w5-4b2bf4-69448f59dd-fqq6g Container started 40m Normal Pulled pod/payload-processing-e2e-shared-b-w5-4b2bf4-69448f59dd-fqq6g Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Started pod/payload-pre-processing-e2e-shared-b-w5-4b2bf4-6cd5fc4b6c-wwqfq Container started 40m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w5-4b2bf4-6cd5fc4b6c-wwqfq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Created pod/payload-pre-processing-e2e-shared-b-w5-4b2bf4-6cd5fc4b6c-wwqfq Container created 40m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-brntt Readiness probe failed: Get "http://10.134.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-xrlzk Readiness probe failed: Get "http://10.132.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-xrlzk Readiness probe error: Get "http://10.132.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-brntt Readiness probe error: Get "http://10.134.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 39m Normal Killing pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-ck68h Stopping container istio-proxy 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-db4d0787-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Normal SuccessfulDelete replicaset/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c Deleted pod: e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-fe4e0c-openshift-default New size: 6; reason: All metrics below target 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-db4d0787-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Normal SuccessfulDelete replicaset/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c Deleted pod: e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-ck68h 39m Normal Killing pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tv82r Stopping container istio-proxy 39m Normal ScalingReplicaSet deployment/e2e-worker-w0-fe4e0c-openshift-default Scaled down replica set e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c from 8 to 6 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e516ff-openshift-default New size: 7; reason: All metrics below target 39m Normal ScalingReplicaSet deployment/e2e-worker-w1-e516ff-openshift-default Scaled down replica set e2e-worker-w1-e516ff-openshift-default-6f4b957485 from 10 to 7 39m Normal Killing pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-7pg2l Stopping container istio-proxy 39m Normal Killing pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-bjmn9 Stopping container istio-proxy 39m Normal Killing pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-w2plx Stopping container istio-proxy 39m Normal SuccessfulDelete replicaset/e2e-worker-w1-e516ff-openshift-default-6f4b957485 Deleted pod: e2e-worker-w1-e516ff-openshift-default-6f4b957485-7pg2l 39m Normal SuccessfulDelete replicaset/e2e-worker-w1-e516ff-openshift-default-6f4b957485 Deleted pod: e2e-worker-w1-e516ff-openshift-default-6f4b957485-bjmn9 39m Normal SuccessfulDelete replicaset/e2e-worker-w1-e516ff-openshift-default-6f4b957485 Deleted pod: e2e-worker-w1-e516ff-openshift-default-6f4b957485-w2plx 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-3b7476-openshift-default New size: 8; reason: All metrics below target 39m Normal ScalingReplicaSet deployment/e2e-worker-w5-3b7476-openshift-default Scaled down replica set e2e-worker-w5-3b7476-openshift-default-59975457bf from 9 to 8 39m Normal Killing pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s Stopping container istio-proxy 39m Warning Unhealthy pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-ck68h Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning ProbeError pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-ck68h Readiness probe error: HTTP probe failed with statuscode: 503... 39m Normal SuccessfulDelete replicaset/e2e-worker-w5-3b7476-openshift-default-59975457bf Deleted pod: e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s 39m Normal Killing pod/payload-pre-processing-e2e-mt-db4d0787-57486cbfdc-88bb5 Stopping container payload-pre-processing 39m Warning Unhealthy pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-7pg2l Readiness probe failed: HTTP probe failed with statuscode: 503 39m Normal Killing pod/payload-processing-e2e-mt-db4d0787-5558ff6d68-hdw6t Stopping container payload-processing 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-4b2bf4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-4b2bf4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning ProbeError pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-7pg2l Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-297806-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-297806-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning ProbeError pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-w2plx Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-w2plx Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning Unhealthy pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s Readiness probe failed: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused 39m Warning ProbeError pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fh84s Readiness probe error: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused... 39m Normal AddedLabel gateway/e2e-api-a-569fb7 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-569fb7 39m Normal SuccessfulCreate replicaset/e2e-api-a-569fb7-openshift-default-6dfd58f675 Created pod: e2e-api-a-569fb7-openshift-default-6dfd58f675-h2ttj 39m Normal ScalingReplicaSet deployment/e2e-api-a-569fb7-openshift-default Scaled up replica set e2e-api-a-569fb7-openshift-default-6dfd58f675 from 0 to 1 39m Normal AddedInterface pod/e2e-api-a-569fb7-openshift-default-6dfd58f675-h2ttj Add eth0 [10.133.0.189/23] from ovn-kubernetes 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-569fb7-8594f58547 Created pod: payload-pre-processing-e2e-api-a-569fb7-8594f58547-sfqmz 39m Warning FailedCreate replicaset/payload-processing-e2e-api-a-569fb7-69bcd6f796 Error creating: pods "payload-processing-e2e-api-a-569fb7-69bcd6f796-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-569fb7: serviceaccount "payload-processing-e2e-api-a-569fb7" not found 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-569fb7 Scaled up replica set payload-pre-processing-e2e-api-a-569fb7-8594f58547 from 0 to 1 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-569fb7 Scaled up replica set payload-processing-e2e-api-a-569fb7-69bcd6f796 from 0 to 1 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-569fb7-8594f58547 Error creating: pods "payload-pre-processing-e2e-api-a-569fb7-8594f58547-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-569fb7: serviceaccount "payload-processing-e2e-api-a-569fb7" not found 39m Normal AddedInterface pod/payload-processing-e2e-api-a-569fb7-69bcd6f796-49qwt Add eth0 [10.133.0.191/23] from ovn-kubernetes 39m Normal Pulled pod/payload-pre-processing-e2e-api-a-569fb7-8594f58547-sfqmz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-569fb7-69bcd6f796 Created pod: payload-processing-e2e-api-a-569fb7-69bcd6f796-49qwt 39m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-569fb7-8594f58547-sfqmz Add eth0 [10.133.0.190/23] from ovn-kubernetes 39m Normal Pulled pod/payload-processing-e2e-api-a-569fb7-69bcd6f796-49qwt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Started pod/payload-pre-processing-e2e-api-a-569fb7-8594f58547-sfqmz Container started 39m Normal Created pod/payload-pre-processing-e2e-api-a-569fb7-8594f58547-sfqmz Container created 39m Normal Started pod/payload-processing-e2e-api-a-569fb7-69bcd6f796-49qwt Container started 39m Normal Created pod/payload-processing-e2e-api-a-569fb7-69bcd6f796-49qwt Container created 38m Warning ProbeError pod/e2e-shared-b-w4-35df3e-openshift-default-6c585b76f-vl6wz Readiness probe error: Get "http://10.134.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning Unhealthy pod/e2e-shared-b-w4-35df3e-openshift-default-6c585b76f-vl6wz Readiness probe failed: Get "http://10.134.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Normal Killing pod/e2e-mt-db4d0787-openshift-default-7bd494bcfc-zqfcj Stopping container istio-proxy 38m Normal AddedLabel gateway/e2e-api-a-a021b3 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-a021b3 38m Normal ScalingReplicaSet deployment/e2e-api-a-a021b3-openshift-default Scaled up replica set e2e-api-a-a021b3-openshift-default-6476c6cc55 from 0 to 1 38m Normal SuccessfulCreate replicaset/e2e-api-a-a021b3-openshift-default-6476c6cc55 Created pod: e2e-api-a-a021b3-openshift-default-6476c6cc55-6zjzw 38m Normal Created pod/e2e-shared-b-w4-35df3e-openshift-default-6c585b76f-vl6wz Container created 38m Normal Pulled pod/e2e-shared-b-w4-35df3e-openshift-default-6c585b76f-vl6wz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Started pod/e2e-shared-b-w4-35df3e-openshift-default-6c585b76f-vl6wz Container started 38m Normal AddedInterface pod/e2e-api-a-a021b3-openshift-default-6476c6cc55-6zjzw Add eth0 [10.133.0.192/23] from ovn-kubernetes 38m Warning ProbeError pod/e2e-shared-b-w4-35df3e-openshift-default-6c585b76f-vl6wz Startup probe error: Get "http://10.134.0.96:15021/healthz/ready": dial tcp 10.134.0.96:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-shared-b-w4-35df3e-openshift-default-6c585b76f-vl6wz Startup probe failed: Get "http://10.134.0.96:15021/healthz/ready": dial tcp 10.134.0.96:15021: connect: connection refused 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-a021b3-775855fc55 Created pod: payload-pre-processing-e2e-api-a-a021b3-775855fc55-hd92k 38m Warning FailedCreate replicaset/payload-processing-e2e-api-a-a021b3-6556664496 Error creating: pods "payload-processing-e2e-api-a-a021b3-6556664496-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-a021b3: serviceaccount "payload-processing-e2e-api-a-a021b3" not found 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-a021b3-6556664496 Created pod: payload-processing-e2e-api-a-a021b3-6556664496-wmwrg 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-a021b3 Scaled up replica set payload-processing-e2e-api-a-a021b3-6556664496 from 0 to 1 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-a021b3 Scaled up replica set payload-pre-processing-e2e-api-a-a021b3-775855fc55 from 0 to 1 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-a021b3-775855fc55 Error creating: pods "payload-pre-processing-e2e-api-a-a021b3-775855fc55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-a021b3: serviceaccount "payload-processing-e2e-api-a-a021b3" not found 38m Normal Pulled pod/payload-processing-e2e-api-a-a021b3-6556664496-wmwrg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Started pod/payload-pre-processing-e2e-api-a-a021b3-775855fc55-hd92k Container started 38m Normal Created pod/payload-pre-processing-e2e-api-a-a021b3-775855fc55-hd92k Container created 38m Normal AddedInterface pod/payload-processing-e2e-api-a-a021b3-6556664496-wmwrg Add eth0 [10.134.0.107/23] from ovn-kubernetes 38m Normal Pulled pod/payload-pre-processing-e2e-api-a-a021b3-775855fc55-hd92k Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Started pod/payload-processing-e2e-api-a-a021b3-6556664496-wmwrg Container started 38m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-a021b3-775855fc55-hd92k Add eth0 [10.134.0.106/23] from ovn-kubernetes 38m Normal Created pod/payload-processing-e2e-api-a-a021b3-6556664496-wmwrg Container created 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-35df3e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-35df3e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Normal Pulled pod/e2e-shared-b-w5-4b2bf4-openshift-default-59785c9d99-b52gb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/e2e-shared-b-w5-4b2bf4-openshift-default-59785c9d99-b52gb Container created 38m Normal Started pod/e2e-shared-b-w5-4b2bf4-openshift-default-59785c9d99-b52gb Container started 38m Normal Started pod/e2e-shared-a-w1-551e2b-openshift-default-5cb6c84ff7-bxgnt Container started 38m Normal Created pod/e2e-shared-a-w1-551e2b-openshift-default-5cb6c84ff7-bxgnt Container created 38m Normal Pulled pod/e2e-shared-a-w1-551e2b-openshift-default-5cb6c84ff7-bxgnt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal ScalingReplicaSet deployment/e2e-worker-w2-a0bcbc-openshift-default Scaled down replica set e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c from 6 to 5 38m Normal Killing pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-7mhsd Stopping container istio-proxy 38m Normal SuccessfulDelete replicaset/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c Deleted pod: e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-7mhsd 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-a0bcbc-openshift-default New size: 5; reason: All metrics below target 38m Normal ScalingReplicaSet deployment/e2e-worker-w4-203c35-openshift-default Scaled down replica set e2e-worker-w4-203c35-openshift-default-56497cdcc from 6 to 5 38m Normal SuccessfulDelete replicaset/e2e-worker-w6-ebb7fd-openshift-default-544f5768f Deleted pod: e2e-worker-w6-ebb7fd-openshift-default-544f5768f-47dj7 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-ebb7fd-openshift-default New size: 5; reason: All metrics below target 38m Normal ScalingReplicaSet deployment/e2e-worker-w6-ebb7fd-openshift-default Scaled down replica set e2e-worker-w6-ebb7fd-openshift-default-544f5768f from 6 to 5 38m Normal Killing pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27 Stopping container istio-proxy 38m Normal AddedLabel gateway/e2e-api-a-cdb11d Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-cdb11d 38m Normal Killing pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-47dj7 Stopping container istio-proxy 38m Normal SuccessfulDelete replicaset/e2e-worker-w4-203c35-openshift-default-56497cdcc Deleted pod: e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-203c35-openshift-default New size: 5; reason: All metrics below target 38m Normal ScalingReplicaSet deployment/e2e-api-a-cdb11d-openshift-default Scaled up replica set e2e-api-a-cdb11d-openshift-default-5ff9547799 from 0 to 1 38m Normal AddedInterface pod/e2e-api-a-cdb11d-openshift-default-5ff9547799-2x65m Add eth0 [10.133.0.194/23] from ovn-kubernetes 38m Normal SuccessfulCreate replicaset/e2e-api-a-cdb11d-openshift-default-5ff9547799 Created pod: e2e-api-a-cdb11d-openshift-default-5ff9547799-2x65m 38m Warning Unhealthy pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-7mhsd Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning ProbeError pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-7mhsd Readiness probe error: HTTP probe failed with statuscode: 503... 38m Warning Unhealthy pod/e2e-shared-b-w5-4b2bf4-openshift-default-59785c9d99-b52gb Startup probe failed: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused 38m Warning ProbeError pod/e2e-shared-b-w5-4b2bf4-openshift-default-59785c9d99-b52gb Startup probe error: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused... 38m Warning ProbeError pod/e2e-shared-a-w1-551e2b-openshift-default-5cb6c84ff7-bxgnt Startup probe error: Get "http://10.133.0.184:15021/healthz/ready": dial tcp 10.133.0.184:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-shared-a-w1-551e2b-openshift-default-5cb6c84ff7-bxgnt Startup probe failed: Get "http://10.133.0.184:15021/healthz/ready": dial tcp 10.133.0.184:15021: connect: connection refused 38m Warning ProbeError pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27 Readiness probe error: HTTP probe failed with statuscode: 503... 38m Warning Unhealthy pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-2tp27 Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning ProbeError pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-47dj7 Readiness probe error: HTTP probe failed with statuscode: 503... 38m Warning Unhealthy pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-47dj7 Readiness probe failed: HTTP probe failed with statuscode: 503 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-cdb11d Scaled up replica set payload-pre-processing-e2e-api-a-cdb11d-585c6fc6dc from 0 to 1 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-cdb11d Scaled up replica set payload-processing-e2e-api-a-cdb11d-87b64cbd4 from 0 to 1 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-cdb11d-585c6fc6dc Error creating: pods "payload-pre-processing-e2e-api-a-cdb11d-585c6fc6dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-cdb11d: serviceaccount "payload-processing-e2e-api-a-cdb11d" not found 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-cdb11d-87b64cbd4 Created pod: payload-processing-e2e-api-a-cdb11d-87b64cbd4-bvfpd 38m Warning FailedCreate replicaset/payload-processing-e2e-api-a-cdb11d-87b64cbd4 Error creating: pods "payload-processing-e2e-api-a-cdb11d-87b64cbd4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-cdb11d: serviceaccount "payload-processing-e2e-api-a-cdb11d" not found 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-cdb11d-585c6fc6dc Created pod: payload-pre-processing-e2e-api-a-cdb11d-585c6fc6dc-dtbnf 38m Normal Started pod/payload-processing-e2e-api-a-cdb11d-87b64cbd4-bvfpd Container started 38m Normal Created pod/payload-processing-e2e-api-a-cdb11d-87b64cbd4-bvfpd Container created 38m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-cdb11d-585c6fc6dc-dtbnf Add eth0 [10.134.0.109/23] from ovn-kubernetes 38m Normal Pulled pod/payload-pre-processing-e2e-api-a-cdb11d-585c6fc6dc-dtbnf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Created pod/payload-pre-processing-e2e-api-a-cdb11d-585c6fc6dc-dtbnf Container created 38m Normal Started pod/payload-pre-processing-e2e-api-a-cdb11d-585c6fc6dc-dtbnf Container started 38m Normal Pulled pod/payload-processing-e2e-api-a-cdb11d-87b64cbd4-bvfpd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/payload-processing-e2e-api-a-cdb11d-87b64cbd4-bvfpd Add eth0 [10.134.0.108/23] from ovn-kubernetes 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-4b2bf4-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-4b2bf4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Normal AddedLabel gateway/e2e-shared-b-w2-00c846 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-00c846 37m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-00c846-openshift-default Scaled up replica set e2e-shared-b-w2-00c846-openshift-default-785fd9fcdc from 0 to 1 37m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-00c846-openshift-default-785fd9fcdc Created pod: e2e-shared-b-w2-00c846-openshift-default-785fd9fcdc-kf4s5 37m Normal AddedInterface pod/e2e-shared-b-w2-00c846-openshift-default-785fd9fcdc-kf4s5 Add eth0 [10.133.0.195/23] from ovn-kubernetes 37m Warning Unhealthy pod/e2e-shared-a-w2-57f116-openshift-default-75f4fc499-znglh Readiness probe failed: Get "http://10.134.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning ProbeError pod/e2e-shared-a-w2-57f116-openshift-default-75f4fc499-znglh Readiness probe error: Get "http://10.134.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Normal AddedLabel gateway/e2e-shared-b-w1-d2b8ee Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w1-d2b8ee 37m Normal Pulled pod/e2e-shared-a-w2-57f116-openshift-default-75f4fc499-znglh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479 Created pod: e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9 37m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-d2b8ee-openshift-default Scaled up replica set e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479 from 0 to 1 37m Normal Started pod/e2e-shared-a-w2-57f116-openshift-default-75f4fc499-znglh Container started 37m Normal Created pod/e2e-shared-a-w2-57f116-openshift-default-75f4fc499-znglh Container created 37m Normal AddedInterface pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9 Add eth0 [10.132.0.96/23] from ovn-kubernetes 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-551e2b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-551e2b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-00c846 Scaled up replica set payload-processing-e2e-shared-b-w2-00c846-6865789c84 from 0 to 1 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-00c846-5d7b447995 Created pod: payload-pre-processing-e2e-shared-b-w2-00c846-5d7b447995-p8qhc 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-00c846 Scaled up replica set payload-pre-processing-e2e-shared-b-w2-00c846-5d7b447995 from 0 to 1 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-00c846-5d7b447995 Error creating: pods "payload-pre-processing-e2e-shared-b-w2-00c846-5d7b447995-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-00c846: serviceaccount "payload-processing-e2e-shared-b-w2-00c846" not found 37m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-00c846-6865789c84 Error creating: pods "payload-processing-e2e-shared-b-w2-00c846-6865789c84-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-00c846: serviceaccount "payload-processing-e2e-shared-b-w2-00c846" not found 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-00c846-6865789c84 Created pod: payload-processing-e2e-shared-b-w2-00c846-6865789c84-c24hp 37m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-00c846-6865789c84-c24hp Add eth0 [10.134.0.111/23] from ovn-kubernetes 37m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-00c846-5d7b447995-p8qhc Container started 37m Normal Created pod/payload-processing-e2e-shared-b-w2-00c846-6865789c84-c24hp Container created 37m Normal Pulled pod/payload-processing-e2e-shared-b-w2-00c846-6865789c84-c24hp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-00c846-5d7b447995-p8qhc Container created 37m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-00c846-5d7b447995-p8qhc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-00c846-5d7b447995-p8qhc Add eth0 [10.134.0.110/23] from ovn-kubernetes 37m Normal Started pod/payload-processing-e2e-shared-b-w2-00c846-6865789c84-c24hp Container started 37m Warning ProbeError pod/e2e-shared-a-w2-57f116-openshift-default-75f4fc499-znglh Startup probe error: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-shared-a-w2-57f116-openshift-default-75f4fc499-znglh Startup probe failed: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-57f116-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-57f116-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w1-d2b8ee-56b894f96d Error creating: pods "payload-processing-e2e-shared-b-w1-d2b8ee-56b894f96d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-d2b8ee: serviceaccount "payload-processing-e2e-shared-b-w1-d2b8ee" not found 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w1-d2b8ee Scaled up replica set payload-processing-e2e-shared-b-w1-d2b8ee-56b894f96d from 0 to 1 37m Normal ScalingReplicaSet deployment/e2e-inf-a-675c80-openshift-default Scaled up replica set e2e-inf-a-675c80-openshift-default-554bc6cd5c from 0 to 1 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w1-d2b8ee Scaled up replica set payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc from 0 to 1 37m Normal SuccessfulCreate replicaset/e2e-inf-a-675c80-openshift-default-554bc6cd5c Created pod: e2e-inf-a-675c80-openshift-default-554bc6cd5c-fsdlt 37m Normal AddedLabel gateway/e2e-inf-a-675c80 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-675c80 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc Created pod: payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w1-d2b8ee-56b894f96d Created pod: payload-processing-e2e-shared-b-w1-d2b8ee-56b894f96d-dbtft 37m Normal AddedInterface pod/e2e-inf-a-675c80-openshift-default-554bc6cd5c-fsdlt Add eth0 [10.133.0.196/23] from ovn-kubernetes 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc Error creating: pods "payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-d2b8ee: serviceaccount "payload-processing-e2e-shared-b-w1-d2b8ee" not found 37m Normal AddedInterface pod/payload-processing-e2e-shared-b-w1-d2b8ee-56b894f96d-dbtft Add eth0 [10.134.0.112/23] from ovn-kubernetes 37m Normal Created pod/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29 Container created 37m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29 Add eth0 [10.132.0.97/23] from ovn-kubernetes 37m Normal Started pod/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29 Container started 37m Normal Started pod/payload-processing-e2e-shared-b-w1-d2b8ee-56b894f96d-dbtft Container started 37m Normal Created pod/payload-processing-e2e-shared-b-w1-d2b8ee-56b894f96d-dbtft Container created 37m Normal Pulled pod/payload-processing-e2e-shared-b-w1-d2b8ee-56b894f96d-dbtft Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-675c80-698585c7d Error creating: pods "payload-processing-e2e-inf-a-675c80-698585c7d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-675c80: serviceaccount "payload-processing-e2e-inf-a-675c80" not found 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-675c80-84886957f9 Error creating: pods "payload-pre-processing-e2e-inf-a-675c80-84886957f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-675c80: serviceaccount "payload-processing-e2e-inf-a-675c80" not found 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-675c80 Scaled up replica set payload-pre-processing-e2e-inf-a-675c80-84886957f9 from 0 to 1 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-675c80 Scaled up replica set payload-processing-e2e-inf-a-675c80-698585c7d from 0 to 1 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-675c80-698585c7d Created pod: payload-processing-e2e-inf-a-675c80-698585c7d-b5hn7 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-675c80-84886957f9 Created pod: payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf 37m Normal Pulled pod/payload-processing-e2e-inf-a-675c80-698585c7d-b5hn7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Pulled pod/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/payload-processing-e2e-inf-a-675c80-698585c7d-b5hn7 Add eth0 [10.133.0.197/23] from ovn-kubernetes 37m Normal Created pod/payload-processing-e2e-inf-a-675c80-698585c7d-b5hn7 Container created 37m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf Add eth0 [10.132.0.98/23] from ovn-kubernetes 37m Normal Started pod/payload-processing-e2e-inf-a-675c80-698585c7d-b5hn7 Container started 37m Normal Started pod/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf Container started 37m Normal Created pod/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf Container created 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-57f116-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-57f116-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Normal Pulled pod/e2e-api-a-569fb7-openshift-default-6dfd58f675-h2ttj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Started pod/e2e-api-a-569fb7-openshift-default-6dfd58f675-h2ttj Container started 37m Normal Created pod/e2e-api-a-569fb7-openshift-default-6dfd58f675-h2ttj Container created 37m Warning Unhealthy pod/e2e-api-a-569fb7-openshift-default-6dfd58f675-h2ttj Startup probe failed: Get "http://10.133.0.189:15021/healthz/ready": dial tcp 10.133.0.189:15021: connect: connection refused 37m Warning ProbeError pod/e2e-api-a-569fb7-openshift-default-6dfd58f675-h2ttj Startup probe error: Get "http://10.133.0.189:15021/healthz/ready": dial tcp 10.133.0.189:15021: connect: connection refused... 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-cdcf67-openshift-default New size: 4; reason: All metrics below target 37m Normal SuccessfulDelete replicaset/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 Deleted pod: e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-hht95 37m Normal Killing pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-hht95 Stopping container istio-proxy 37m Warning Unhealthy pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-hht95 Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-hht95 Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-675c80-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-675c80-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-569fb7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-569fb7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal Created pod/e2e-api-a-a021b3-openshift-default-6476c6cc55-6zjzw Container created 36m Normal Pulled pod/e2e-api-a-a021b3-openshift-default-6476c6cc55-6zjzw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/e2e-api-a-a021b3-openshift-default-6476c6cc55-6zjzw Container started 36m Normal AddedLabel gateway/e2e-api-b-9d2b96 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-9d2b96 36m Normal ScalingReplicaSet deployment/e2e-api-b-9d2b96-openshift-default Scaled up replica set e2e-api-b-9d2b96-openshift-default-cc56c4dff from 0 to 1 36m Normal SuccessfulCreate replicaset/e2e-api-b-9d2b96-openshift-default-cc56c4dff Created pod: e2e-api-b-9d2b96-openshift-default-cc56c4dff-6w9vk 36m Normal AddedInterface pod/e2e-api-b-9d2b96-openshift-default-cc56c4dff-6w9vk Add eth0 [10.133.0.198/23] from ovn-kubernetes 36m Warning ProbeError pod/e2e-api-a-a021b3-openshift-default-6476c6cc55-6zjzw Startup probe error: Get "http://10.133.0.192:15021/healthz/ready": dial tcp 10.133.0.192:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-api-a-a021b3-openshift-default-6476c6cc55-6zjzw Startup probe failed: Get "http://10.133.0.192:15021/healthz/ready": dial tcp 10.133.0.192:15021: connect: connection refused 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-9d2b96 Scaled up replica set payload-processing-e2e-api-b-9d2b96-7d8dd46f5b from 0 to 1 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-9d2b96 Scaled up replica set payload-pre-processing-e2e-api-b-9d2b96-7b7d78cbff from 0 to 1 36m Warning FailedCreate replicaset/payload-processing-e2e-api-b-9d2b96-7d8dd46f5b Error creating: pods "payload-processing-e2e-api-b-9d2b96-7d8dd46f5b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-9d2b96: serviceaccount "payload-processing-e2e-api-b-9d2b96" not found 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-9d2b96-7b7d78cbff Error creating: pods "payload-pre-processing-e2e-api-b-9d2b96-7b7d78cbff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-9d2b96: serviceaccount "payload-processing-e2e-api-b-9d2b96" not found 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-a021b3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-9d2b96-7b7d78cbff Created pod: payload-pre-processing-e2e-api-b-9d2b96-7b7d78cbff-gtqd9 36m Normal AddedInterface pod/payload-processing-e2e-api-b-9d2b96-7d8dd46f5b-7rbgg Add eth0 [10.134.0.114/23] from ovn-kubernetes 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-9d2b96-7d8dd46f5b Created pod: payload-processing-e2e-api-b-9d2b96-7d8dd46f5b-7rbgg 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-a021b3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal Pulled pod/payload-processing-e2e-api-b-9d2b96-7d8dd46f5b-7rbgg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Created pod/payload-processing-e2e-api-b-9d2b96-7d8dd46f5b-7rbgg Container created 36m Normal Started pod/payload-processing-e2e-api-b-9d2b96-7d8dd46f5b-7rbgg Container started 36m Normal Pulled pod/payload-pre-processing-e2e-api-b-9d2b96-7b7d78cbff-gtqd9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-9d2b96-7b7d78cbff-gtqd9 Add eth0 [10.133.0.199/23] from ovn-kubernetes 36m Normal Created pod/payload-pre-processing-e2e-api-b-9d2b96-7b7d78cbff-gtqd9 Container created 36m Normal Started pod/payload-pre-processing-e2e-api-b-9d2b96-7b7d78cbff-gtqd9 Container started 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-a021b3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-a021b3-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning ProbeError pod/e2e-api-a-cdb11d-openshift-default-5ff9547799-2x65m Readiness probe error: Get "http://10.133.0.194:15021/healthz/ready": read tcp 10.133.0.2:49028->10.133.0.194:15021: read: connection reset by peer... 35m Warning Unhealthy pod/e2e-api-a-cdb11d-openshift-default-5ff9547799-2x65m Readiness probe failed: Get "http://10.133.0.194:15021/healthz/ready": read tcp 10.133.0.2:49028->10.133.0.194:15021: read: connection reset by peer 35m Normal Started pod/e2e-api-a-cdb11d-openshift-default-5ff9547799-2x65m Container started 35m Normal Created pod/e2e-api-a-cdb11d-openshift-default-5ff9547799-2x65m Container created 35m Normal Pulled pod/e2e-api-a-cdb11d-openshift-default-5ff9547799-2x65m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Warning ProbeError pod/e2e-api-a-cdb11d-openshift-default-5ff9547799-2x65m Startup probe error: Get "http://10.133.0.194:15021/healthz/ready": dial tcp 10.133.0.194:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-api-a-cdb11d-openshift-default-5ff9547799-2x65m Startup probe failed: Get "http://10.133.0.194:15021/healthz/ready": dial tcp 10.133.0.194:15021: connect: connection refused 35m Normal Killing pod/payload-pre-processing-e2e-api-a-a021b3-775855fc55-hd92k Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-api-a-a021b3-6556664496-wmwrg Stopping container payload-processing 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-cdb11d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-cdb11d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Normal Killing pod/e2e-api-a-a021b3-openshift-default-6476c6cc55-6zjzw Stopping container istio-proxy 35m Normal SuccessfulCreate replicaset/e2e-inf-a-9f4199-openshift-default-586448895 Created pod: e2e-inf-a-9f4199-openshift-default-586448895-hl8h4 35m Normal ScalingReplicaSet deployment/e2e-inf-a-9f4199-openshift-default Scaled up replica set e2e-inf-a-9f4199-openshift-default-586448895 from 0 to 1 35m Normal AddedLabel gateway/e2e-inf-a-9f4199 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-9f4199 35m Warning ProbeError pod/e2e-api-a-a021b3-openshift-default-6476c6cc55-6zjzw Readiness probe error: HTTP probe failed with statuscode: 503... 35m Normal AddedInterface pod/e2e-inf-a-9f4199-openshift-default-586448895-hl8h4 Add eth0 [10.133.0.200/23] from ovn-kubernetes 35m Warning Unhealthy pod/e2e-api-a-a021b3-openshift-default-6476c6cc55-6zjzw Readiness probe failed: HTTP probe failed with statuscode: 503 35m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-d2b8ee-openshift-default Scaled up replica set e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479 from 1 to 4 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-d2b8ee-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 35m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479 Created pod: e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-5zskv 35m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479 Created pod: e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-qpj8z 35m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479 Created pod: e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-vfrcc 35m Normal Pulled pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-qpj8z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-5zskv Add eth0 [10.133.0.202/23] from ovn-kubernetes 35m Normal Started pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-qpj8z Container started 35m Normal AddedInterface pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-vfrcc Add eth0 [10.133.0.201/23] from ovn-kubernetes 35m Normal Pulled pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-5zskv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-vfrcc Container created 35m Normal Started pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-vfrcc Container started 35m Normal Created pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-qpj8z Container created 35m Normal Started pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-5zskv Container started 35m Normal AddedInterface pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-qpj8z Add eth0 [10.134.0.116/23] from ovn-kubernetes 35m Normal Pulled pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-vfrcc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-5zskv Container created 35m Warning ProbeError pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9 Readiness probe error: Get "http://10.132.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 35m Warning Unhealthy pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9 Readiness probe failed: Get "http://10.132.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 35m Normal Created pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9 Container created 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-9f4199 Scaled up replica set payload-pre-processing-e2e-inf-a-9f4199-84b49988cf from 0 to 1 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-cdb11d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Normal Pulled pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-9f4199-55cf88d648 Error creating: pods "payload-processing-e2e-inf-a-9f4199-55cf88d648-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-9f4199: serviceaccount "payload-processing-e2e-inf-a-9f4199" not found 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-9f4199 Scaled up replica set payload-processing-e2e-inf-a-9f4199-55cf88d648 from 0 to 1 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-cdb11d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Normal Started pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9 Container started 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-9f4199-84b49988cf Error creating: pods "payload-pre-processing-e2e-inf-a-9f4199-84b49988cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-9f4199: serviceaccount "payload-processing-e2e-inf-a-9f4199" not found 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-9f4199-84b49988cf Created pod: payload-pre-processing-e2e-inf-a-9f4199-84b49988cf-thdfx 35m Warning ProbeError pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-qpj8z Startup probe error: Get "http://10.134.0.116:15021/healthz/ready": dial tcp 10.134.0.116:15021: connect: connection refused... 35m Normal Pulled pod/payload-pre-processing-e2e-inf-a-9f4199-84b49988cf-thdfx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-9f4199-55cf88d648 Created pod: payload-processing-e2e-inf-a-9f4199-55cf88d648-fr4zd 35m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-9f4199-84b49988cf-thdfx Add eth0 [10.134.0.117/23] from ovn-kubernetes 35m Warning Unhealthy pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-qpj8z Startup probe failed: Get "http://10.134.0.116:15021/healthz/ready": dial tcp 10.134.0.116:15021: connect: connection refused 35m Normal AddedInterface pod/payload-processing-e2e-inf-a-9f4199-55cf88d648-fr4zd Add eth0 [10.134.0.118/23] from ovn-kubernetes 35m Normal Pulled pod/payload-processing-e2e-inf-a-9f4199-55cf88d648-fr4zd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Created pod/payload-processing-e2e-inf-a-9f4199-55cf88d648-fr4zd Container created 35m Warning ProbeError pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-5zskv Startup probe error: Get "http://10.133.0.202:15021/healthz/ready": dial tcp 10.133.0.202:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-5zskv Startup probe failed: Get "http://10.133.0.202:15021/healthz/ready": dial tcp 10.133.0.202:15021: connect: connection refused 35m Normal Started pod/payload-processing-e2e-inf-a-9f4199-55cf88d648-fr4zd Container started 35m Normal Started pod/payload-pre-processing-e2e-inf-a-9f4199-84b49988cf-thdfx Container started 35m Warning Unhealthy pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-vfrcc Startup probe failed: Get "http://10.133.0.201:15021/healthz/ready": dial tcp 10.133.0.201:15021: connect: connection refused 35m Warning ProbeError pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-vfrcc Startup probe error: Get "http://10.133.0.201:15021/healthz/ready": dial tcp 10.133.0.201:15021: connect: connection refused... 35m Normal Created pod/payload-pre-processing-e2e-inf-a-9f4199-84b49988cf-thdfx Container created 35m Normal Started pod/e2e-shared-b-w2-00c846-openshift-default-785fd9fcdc-kf4s5 Container started 35m Normal Created pod/e2e-shared-b-w2-00c846-openshift-default-785fd9fcdc-kf4s5 Container created 35m Normal Pulled pod/e2e-shared-b-w2-00c846-openshift-default-785fd9fcdc-kf4s5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-00c846-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-00c846-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning Unhealthy pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9 Startup probe failed: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused 35m Warning ProbeError pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9 Startup probe error: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused... 35m Warning ProbeError pod/e2e-shared-b-w2-00c846-openshift-default-785fd9fcdc-kf4s5 Startup probe error: Get "http://10.133.0.195:15021/healthz/ready": dial tcp 10.133.0.195:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-shared-b-w2-00c846-openshift-default-785fd9fcdc-kf4s5 Startup probe failed: Get "http://10.133.0.195:15021/healthz/ready": dial tcp 10.133.0.195:15021: connect: connection refused 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-d2b8ee-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-d2b8ee-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e516ff-openshift-default New size: 5; reason: All metrics below target 35m Normal Killing pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-975ss Stopping container istio-proxy 35m Normal ScalingReplicaSet deployment/e2e-worker-w0-fe4e0c-openshift-default Scaled down replica set e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c from 6 to 5 35m Normal Killing pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-fc8wx Stopping container istio-proxy 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-fe4e0c-openshift-default New size: 5; reason: All metrics below target 35m Normal Killing pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj Stopping container istio-proxy 35m Normal Killing pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh Stopping container istio-proxy 35m Normal ScalingReplicaSet deployment/e2e-worker-w1-e516ff-openshift-default Scaled down replica set e2e-worker-w1-e516ff-openshift-default-6f4b957485 from 7 to 5 35m Normal SuccessfulDelete replicaset/e2e-worker-w1-e516ff-openshift-default-6f4b957485 Deleted pod: e2e-worker-w1-e516ff-openshift-default-6f4b957485-fc8wx 35m Normal SuccessfulDelete replicaset/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 Deleted pod: e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-975ss 35m Normal SuccessfulDelete replicaset/e2e-worker-w1-e516ff-openshift-default-6f4b957485 Deleted pod: e2e-worker-w1-e516ff-openshift-default-6f4b957485-5mccl 35m Normal Killing pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-5mccl Stopping container istio-proxy 35m Normal SuccessfulDelete replicaset/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c Deleted pod: e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k 35m Normal Killing pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k Stopping container istio-proxy 35m Normal SuccessfulDelete replicaset/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c Deleted pod: e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh 35m Normal SuccessfulDelete replicaset/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c Deleted pod: e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-a0bcbc-openshift-default New size: 3; reason: All metrics below target 35m Normal ScalingReplicaSet deployment/e2e-worker-w2-a0bcbc-openshift-default Scaled down replica set e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c from 5 to 3 35m Normal ScalingReplicaSet deployment/e2e-worker-w4-203c35-openshift-default Scaled down replica set e2e-worker-w4-203c35-openshift-default-56497cdcc from 5 to 3 35m Warning ProbeError pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-975ss Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning ProbeError pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-fc8wx Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-fc8wx Readiness probe failed: HTTP probe failed with statuscode: 503 35m Normal Killing pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-8kz5l Stopping container istio-proxy 35m Normal Killing pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-gph88 Stopping container istio-proxy 35m Normal ScalingReplicaSet deployment/e2e-worker-w5-3b7476-openshift-default Scaled down replica set e2e-worker-w5-3b7476-openshift-default-59975457bf from 8 to 4 35m Normal Killing pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-52pkm Stopping container istio-proxy 35m Normal SuccessfulDelete replicaset/e2e-worker-w5-3b7476-openshift-default-59975457bf Deleted pod: e2e-worker-w5-3b7476-openshift-default-59975457bf-52pkm 35m Normal SuccessfulDelete replicaset/e2e-worker-w4-203c35-openshift-default-56497cdcc Deleted pod: e2e-worker-w4-203c35-openshift-default-56497cdcc-czkcm 35m Warning Unhealthy pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-975ss Readiness probe failed: HTTP probe failed with statuscode: 503 35m Normal Killing pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-czkcm Stopping container istio-proxy 35m Normal Killing pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-vxwkm Stopping container istio-proxy 35m Normal Killing pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-rsw2s Stopping container istio-proxy 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-203c35-openshift-default New size: 3; reason: All metrics below target 35m Normal SuccessfulDelete replicaset/e2e-worker-w5-3b7476-openshift-default-59975457bf Deleted pod: e2e-worker-w5-3b7476-openshift-default-59975457bf-gph88 35m Normal SuccessfulDelete replicaset/e2e-worker-w5-3b7476-openshift-default-59975457bf Deleted pod: e2e-worker-w5-3b7476-openshift-default-59975457bf-vxwkm 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-3b7476-openshift-default New size: 4; reason: All metrics below target 35m Normal SuccessfulDelete replicaset/e2e-worker-w5-3b7476-openshift-default-59975457bf Deleted pod: e2e-worker-w5-3b7476-openshift-default-59975457bf-8kz5l 35m Normal SuccessfulDelete replicaset/e2e-worker-w4-203c35-openshift-default-56497cdcc Deleted pod: e2e-worker-w4-203c35-openshift-default-56497cdcc-rsw2s 35m Normal ScalingReplicaSet deployment/e2e-worker-w6-ebb7fd-openshift-default Scaled down replica set e2e-worker-w6-ebb7fd-openshift-default-544f5768f from 5 to 3 35m Normal SuccessfulDelete replicaset/e2e-worker-w6-ebb7fd-openshift-default-544f5768f Deleted pod: e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-ebb7fd-openshift-default New size: 3; reason: All metrics below target 35m Warning ProbeError pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-qzdbh Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning Unhealthy pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj Readiness probe failed: HTTP probe failed with statuscode: 503 35m Normal Killing pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp Stopping container istio-proxy 35m Normal SuccessfulDelete replicaset/e2e-worker-w6-ebb7fd-openshift-default-544f5768f Deleted pod: e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp 35m Warning ProbeError pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-rq5dj Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-52pkm Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-52pkm Readiness probe error: HTTP probe failed with statuscode: 503... 35m Normal Killing pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw Stopping container istio-proxy 35m Warning ProbeError pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-czkcm Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-czkcm Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-8kz5l Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-8kz5l Readiness probe failed: HTTP probe failed with statuscode: 503 35m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 35m Warning Unhealthy pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-5mccl Readiness probe failed: HTTP probe failed with statuscode: 503 35m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-2g6x2 35m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 35m Warning Unhealthy pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k Readiness probe failed: HTTP probe failed with statuscode: 503 35m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-2g6x2 Stopping container discovery 35m Warning ProbeError pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-5mccl Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning ProbeError pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-jdf6k Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning ProbeError pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-7mjxp Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-pmndw Readiness probe failed: HTTP probe failed with statuscode: 503 35m Normal Killing pod/payload-pre-processing-e2e-api-a-cdb11d-585c6fc6dc-dtbnf Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-api-a-cdb11d-87b64cbd4-bvfpd Stopping container payload-processing 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-d2b8ee-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-d2b8ee-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Normal Killing pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs Stopping container istio-proxy 34m Normal Killing pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-8bhc5 Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/e2e-worker-w1-e516ff-openshift-default-6f4b957485 Deleted pod: e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs 34m Normal SuccessfulDelete replicaset/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c Deleted pod: e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-8bhc5 34m Normal Killing pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-mmngc Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c Deleted pod: e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-mmngc 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e516ff-openshift-default New size: 4; reason: All metrics below target 34m Normal ScalingReplicaSet deployment/e2e-worker-w1-e516ff-openshift-default Scaled down replica set e2e-worker-w1-e516ff-openshift-default-6f4b957485 from 5 to 4 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-fe4e0c-openshift-default New size: 3; reason: All metrics below target 34m Normal ScalingReplicaSet deployment/e2e-worker-w0-fe4e0c-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c from 5 to 3 34m Warning Unhealthy pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-mmngc Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning ProbeError pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-mmngc Readiness probe error: HTTP probe failed with statuscode: 503... 34m Warning ProbeError pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-8bhc5 Readiness probe error: HTTP probe failed with statuscode: 503... 34m Warning Unhealthy pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-8bhc5 Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning ProbeError pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs Readiness probe error: HTTP probe failed with statuscode: 503... 34m Warning Unhealthy pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-t4mqs Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-00c846-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-00c846-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Normal AddedLabel gateway/e2e-inf-a-22e72e Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-22e72e 34m Normal SuccessfulCreate replicaset/e2e-inf-a-22e72e-openshift-default-64656d9fb9 Created pod: e2e-inf-a-22e72e-openshift-default-64656d9fb9-26n5c 34m Normal ScalingReplicaSet deployment/e2e-inf-a-22e72e-openshift-default Scaled up replica set e2e-inf-a-22e72e-openshift-default-64656d9fb9 from 0 to 1 34m Normal AddedInterface pod/e2e-inf-a-22e72e-openshift-default-64656d9fb9-26n5c Add eth0 [10.134.0.120/23] from ovn-kubernetes 34m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-57f116-599cbf5bbc-2fp4d Stopping container payload-pre-processing 34m Normal Killing pod/payload-processing-e2e-shared-a-w2-57f116-5b5dd4bd44-fzgp7 Stopping container payload-processing 34m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-22e72e-85b6d5f746 Error creating: pods "payload-processing-e2e-inf-a-22e72e-85b6d5f746-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-22e72e: serviceaccount "payload-processing-e2e-inf-a-22e72e" not found 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-22e72e Scaled up replica set payload-pre-processing-e2e-inf-a-22e72e-7ffdf7c747 from 0 to 1 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-22e72e-7ffdf7c747 Created pod: payload-pre-processing-e2e-inf-a-22e72e-7ffdf7c747-vbt9t 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-22e72e-7ffdf7c747 Error creating: pods "payload-pre-processing-e2e-inf-a-22e72e-7ffdf7c747-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-22e72e: serviceaccount "payload-processing-e2e-inf-a-22e72e" not found 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-22e72e-85b6d5f746 Created pod: payload-processing-e2e-inf-a-22e72e-85b6d5f746-nw6tv 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-22e72e Scaled up replica set payload-processing-e2e-inf-a-22e72e-85b6d5f746 from 0 to 1 34m Normal Started pod/payload-pre-processing-e2e-inf-a-22e72e-7ffdf7c747-vbt9t Container started 34m Normal Pulled pod/payload-processing-e2e-inf-a-22e72e-85b6d5f746-nw6tv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Started pod/payload-processing-e2e-inf-a-22e72e-85b6d5f746-nw6tv Container started 34m Normal Created pod/payload-processing-e2e-inf-a-22e72e-85b6d5f746-nw6tv Container created 34m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-22e72e-7ffdf7c747-vbt9t Add eth0 [10.134.0.121/23] from ovn-kubernetes 34m Normal Pulled pod/payload-pre-processing-e2e-inf-a-22e72e-7ffdf7c747-vbt9t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Created pod/payload-pre-processing-e2e-inf-a-22e72e-7ffdf7c747-vbt9t Container created 34m Normal AddedInterface pod/payload-processing-e2e-inf-a-22e72e-85b6d5f746-nw6tv Add eth0 [10.134.0.122/23] from ovn-kubernetes 34m Normal Killing pod/payload-processing-e2e-shared-a-w1-551e2b-79ddfccbf8-6rkbp Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-shared-a-w1-551e2b-869c56d547-549jg Stopping container payload-pre-processing 34m Normal Killing pod/payload-processing-e2e-inf-a-675c80-698585c7d-b5hn7 Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-inf-a-675c80-84886957f9-m72mf Stopping container payload-pre-processing 34m Warning Unhealthy pod/e2e-inf-a-675c80-openshift-default-554bc6cd5c-fsdlt Readiness probe failed: Get "http://10.133.0.196:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 34m Warning ProbeError pod/e2e-inf-a-675c80-openshift-default-554bc6cd5c-fsdlt Readiness probe error: Get "http://10.133.0.196:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 34m Normal Started pod/e2e-inf-a-675c80-openshift-default-554bc6cd5c-fsdlt Container started 34m Normal Pulled pod/e2e-inf-a-675c80-openshift-default-554bc6cd5c-fsdlt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Created pod/e2e-inf-a-675c80-openshift-default-554bc6cd5c-fsdlt Container created 34m Warning ProbeError pod/e2e-inf-a-675c80-openshift-default-554bc6cd5c-fsdlt Startup probe error: Get "http://10.133.0.196:15021/healthz/ready": dial tcp 10.133.0.196:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-inf-a-675c80-openshift-default-554bc6cd5c-fsdlt Startup probe failed: Get "http://10.133.0.196:15021/healthz/ready": dial tcp 10.133.0.196:15021: connect: connection refused 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-675c80-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-675c80-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Normal Killing pod/e2e-inf-a-675c80-openshift-default-554bc6cd5c-fsdlt Stopping container istio-proxy 33m Normal Killing pod/payload-pre-processing-e2e-shared-a-w5-297806-bdfd869bc-8j6sw Stopping container payload-pre-processing 33m Normal Killing pod/payload-processing-e2e-shared-a-w5-297806-6b77744954-k4jwf Stopping container payload-processing 33m Normal Pulled pod/e2e-api-b-9d2b96-openshift-default-cc56c4dff-6w9vk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Started pod/e2e-api-b-9d2b96-openshift-default-cc56c4dff-6w9vk Container started 33m Normal Created pod/e2e-api-b-9d2b96-openshift-default-cc56c4dff-6w9vk Container created 33m Warning Unhealthy pod/e2e-api-b-9d2b96-openshift-default-cc56c4dff-6w9vk Startup probe failed: Get "http://10.133.0.198:15021/healthz/ready": dial tcp 10.133.0.198:15021: connect: connection refused 33m Warning ProbeError pod/e2e-api-b-9d2b96-openshift-default-cc56c4dff-6w9vk Startup probe error: Get "http://10.133.0.198:15021/healthz/ready": dial tcp 10.133.0.198:15021: connect: connection refused... 33m Normal Killing pod/payload-processing-e2e-api-a-569fb7-69bcd6f796-49qwt Stopping container payload-processing 33m Normal Killing pod/payload-pre-processing-e2e-api-a-569fb7-8594f58547-sfqmz Stopping container payload-pre-processing 33m Normal Killing pod/e2e-api-a-cdb11d-openshift-default-5ff9547799-2x65m Stopping container istio-proxy 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-9d2b96-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-9d2b96-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Normal Killing pod/e2e-shared-a-w5-297806-openshift-default-657d54b457-mdh67 Stopping container istio-proxy 33m Normal AddedLabel gateway/e2e-inf-a-db5ef5 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-db5ef5 33m Normal SuccessfulCreate replicaset/e2e-inf-a-db5ef5-openshift-default-596b67fbf7 Created pod: e2e-inf-a-db5ef5-openshift-default-596b67fbf7-qrgxq 33m Normal ScalingReplicaSet deployment/e2e-inf-a-db5ef5-openshift-default Scaled up replica set e2e-inf-a-db5ef5-openshift-default-596b67fbf7 from 0 to 1 33m Normal AddedInterface pod/e2e-inf-a-db5ef5-openshift-default-596b67fbf7-qrgxq Add eth0 [10.133.0.204/23] from ovn-kubernetes 33m Warning Unhealthy pod/e2e-api-a-cdb11d-openshift-default-5ff9547799-2x65m Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-api-a-cdb11d-openshift-default-5ff9547799-2x65m Readiness probe error: HTTP probe failed with statuscode: 503... 33m Normal Killing pod/e2e-shared-a-w1-551e2b-openshift-default-5cb6c84ff7-bxgnt Stopping container istio-proxy 33m Warning Unhealthy pod/e2e-shared-a-w1-551e2b-openshift-default-5cb6c84ff7-bxgnt Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-shared-a-w1-551e2b-openshift-default-5cb6c84ff7-bxgnt Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-9d2b96-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-9d2b96-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-db5ef5 Scaled up replica set payload-processing-e2e-inf-a-db5ef5-657546bd74 from 0 to 1 33m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-db5ef5-657546bd74 Error creating: pods "payload-processing-e2e-inf-a-db5ef5-657546bd74-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-db5ef5: serviceaccount "payload-processing-e2e-inf-a-db5ef5" not found 33m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-db5ef5-5b75965f5 Error creating: pods "payload-pre-processing-e2e-inf-a-db5ef5-5b75965f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-db5ef5: serviceaccount "payload-processing-e2e-inf-a-db5ef5" not found 33m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-db5ef5 Scaled up replica set payload-pre-processing-e2e-inf-a-db5ef5-5b75965f5 from 0 to 1 33m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-db5ef5-657546bd74 Created pod: payload-processing-e2e-inf-a-db5ef5-657546bd74-dh9pg 33m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-db5ef5-5b75965f5 Created pod: payload-pre-processing-e2e-inf-a-db5ef5-5b75965f5-j72vc 33m Normal Killing pod/payload-pre-processing-e2e-shared-b-w1-d2b8ee-7c8c5b46dc-2vf29 Stopping container payload-pre-processing 33m Normal Pulled pod/payload-processing-e2e-inf-a-db5ef5-657546bd74-dh9pg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal Created pod/payload-processing-e2e-inf-a-db5ef5-657546bd74-dh9pg Container created 33m Normal Killing pod/payload-processing-e2e-shared-b-w1-d2b8ee-56b894f96d-dbtft Stopping container payload-processing 33m Normal AddedInterface pod/payload-processing-e2e-inf-a-db5ef5-657546bd74-dh9pg Add eth0 [10.133.0.206/23] from ovn-kubernetes 33m Normal Created pod/payload-pre-processing-e2e-inf-a-db5ef5-5b75965f5-j72vc Container created 33m Normal Pulled pod/payload-pre-processing-e2e-inf-a-db5ef5-5b75965f5-j72vc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-db5ef5-5b75965f5-j72vc Add eth0 [10.133.0.205/23] from ovn-kubernetes 32m Normal Started pod/payload-pre-processing-e2e-inf-a-db5ef5-5b75965f5-j72vc Container started 32m Normal Started pod/payload-processing-e2e-inf-a-db5ef5-657546bd74-dh9pg Container started 32m Normal Killing pod/payload-pre-processing-e2e-shared-b-w5-4b2bf4-6cd5fc4b6c-wwqfq Stopping container payload-pre-processing 32m Normal Killing pod/payload-processing-e2e-shared-b-w5-4b2bf4-69448f59dd-fqq6g Stopping container payload-processing 32m Normal Created pod/e2e-inf-a-9f4199-openshift-default-586448895-hl8h4 Container created 32m Normal Pulled pod/e2e-inf-a-9f4199-openshift-default-586448895-hl8h4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Started pod/e2e-inf-a-9f4199-openshift-default-586448895-hl8h4 Container started 32m Warning Unhealthy pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9 Readiness probe failed: HTTP probe failed with statuscode: 503 32m Normal Killing pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-qpj8z Stopping container istio-proxy 32m Normal Killing pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-vfrcc Stopping container istio-proxy 32m Normal Killing pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9 Stopping container istio-proxy 32m Warning ProbeError pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-hc7g9 Readiness probe error: HTTP probe failed with statuscode: 503... 32m Normal Killing pod/e2e-shared-b-w1-d2b8ee-openshift-default-6f9c87f479-5zskv Stopping container istio-proxy 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-9f4199-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Normal Killing pod/e2e-shared-b-w5-4b2bf4-openshift-default-59785c9d99-b52gb Stopping container istio-proxy 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-9f4199-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning Unhealthy pod/e2e-inf-a-9f4199-openshift-default-586448895-hl8h4 Startup probe failed: Get "http://10.133.0.200:15021/healthz/ready": dial tcp 10.133.0.200:15021: connect: connection refused 32m Warning ProbeError pod/e2e-inf-a-9f4199-openshift-default-586448895-hl8h4 Startup probe error: Get "http://10.133.0.200:15021/healthz/ready": dial tcp 10.133.0.200:15021: connect: connection refused... 32m Normal Killing pod/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw Stopping container payload-processing 32m Normal Killing pod/payload-pre-processing-e2e-worker-w5-3b7476-56c5fdb86c-8786b Stopping container payload-pre-processing 32m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: All metrics below target 32m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-76597d6855 Deleted pod: maas-default-gateway-openshift-default-76597d6855-dtlt4 32m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-76597d6855 from 10 to 8 32m Normal Killing pod/maas-default-gateway-openshift-default-76597d6855-jjxxp Stopping container istio-proxy 32m Normal Killing pod/maas-default-gateway-openshift-default-76597d6855-dtlt4 Stopping container istio-proxy 32m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-76597d6855 Deleted pod: maas-default-gateway-openshift-default-76597d6855-jjxxp 32m Normal Killing pod/payload-processing-e2e-inf-a-9f4199-55cf88d648-fr4zd Stopping container payload-processing 32m Normal Killing pod/payload-pre-processing-e2e-inf-a-9f4199-84b49988cf-thdfx Stopping container payload-pre-processing 32m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-dtlt4 Readiness probe failed: Get "http://10.134.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 32m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-rlhz6 Readiness probe error: Get "http://10.132.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 32m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-rlhz6 Readiness probe failed: Get "http://10.132.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 32m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-dtlt4 Readiness probe error: Get "http://10.134.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 32m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-jjxxp Readiness probe failed: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused 32m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-jjxxp Readiness probe error: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused... 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-9f4199-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-9f4199-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning Unhealthy pod/payload-processing-e2e-worker-w5-3b7476-846b85d7cd-qjcgw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Normal Killing pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw Stopping container istio-proxy 32m Normal SuccessfulDelete replicaset/e2e-worker-w1-e516ff-openshift-default-6f4b957485 Deleted pod: e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw 32m Normal SuccessfulDelete replicaset/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 Deleted pod: e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n 32m Normal Killing pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n Stopping container istio-proxy 32m Normal ScalingReplicaSet deployment/e2e-worker-w1-e516ff-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w1-e516ff-openshift-default-6f4b957485 from 4 to 3 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e516ff-openshift-default New size: 3; reason: All metrics below target 32m Normal ScalingReplicaSet deployment/e2e-worker-w3-cdcf67-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7 from 3 to 2 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-cdcf67-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 32m Warning ProbeError pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning Unhealthy pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-58g7n Readiness probe failed: HTTP probe failed with statuscode: 503 32m Normal Created pod/e2e-inf-a-22e72e-openshift-default-64656d9fb9-26n5c Container created 32m Normal Started pod/e2e-inf-a-22e72e-openshift-default-64656d9fb9-26n5c Container started 32m Normal Pulled pod/e2e-inf-a-22e72e-openshift-default-64656d9fb9-26n5c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal ScalingReplicaSet deployment/e2e-worker-w6-ebb7fd-openshift-default Scaled down replica set e2e-worker-w6-ebb7fd-openshift-default-544f5768f from 3 to 2 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-ebb7fd-openshift-default New size: 2; reason: All metrics below target 32m Normal Killing pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-mngkl Stopping container istio-proxy 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-3b7476-openshift-default New size: 3; reason: All metrics below target 32m Warning Unhealthy pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw Readiness probe failed: HTTP probe failed with statuscode: 503 32m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w5-3b7476-56c5fdb86c-8786b Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Normal SuccessfulDelete replicaset/e2e-worker-w5-3b7476-openshift-default-59975457bf Deleted pod: e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg 32m Normal Killing pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg Stopping container istio-proxy 32m Warning ProbeError pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-2xfsw Readiness probe error: HTTP probe failed with statuscode: 503... 32m Normal SuccessfulDelete replicaset/e2e-worker-w6-ebb7fd-openshift-default-544f5768f Deleted pod: e2e-worker-w6-ebb7fd-openshift-default-544f5768f-mngkl 32m Warning Unhealthy pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-mngkl Readiness probe failed: HTTP probe failed with statuscode: 503 32m Warning ProbeError pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning Unhealthy pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-czqbg Readiness probe failed: HTTP probe failed with statuscode: 503 32m Warning ProbeError pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-mngkl Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning ProbeError pod/e2e-inf-a-22e72e-openshift-default-64656d9fb9-26n5c Startup probe error: Get "http://10.134.0.120:15021/healthz/ready": dial tcp 10.134.0.120:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-inf-a-22e72e-openshift-default-64656d9fb9-26n5c Startup probe failed: Get "http://10.134.0.120:15021/healthz/ready": dial tcp 10.134.0.120:15021: connect: connection refused 31m Normal Killing pod/e2e-api-a-569fb7-openshift-default-6dfd58f675-h2ttj Stopping container istio-proxy 31m Warning Unhealthy pod/e2e-api-a-569fb7-openshift-default-6dfd58f675-h2ttj Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning ProbeError pod/e2e-api-a-569fb7-openshift-default-6dfd58f675-h2ttj Readiness probe error: HTTP probe failed with statuscode: 503... 31m Normal Killing pod/e2e-shared-a-w2-57f116-openshift-default-75f4fc499-znglh Stopping container istio-proxy 31m Normal SuccessfulCreate replicaset/e2e-worker-w5-3b7476-openshift-default-59975457bf Created pod: e2e-worker-w5-3b7476-openshift-default-59975457bf-7zmxp 31m Warning Unhealthy pod/e2e-shared-a-w2-57f116-openshift-default-75f4fc499-znglh Readiness probe failed: HTTP probe failed with statuscode: 503 31m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-76597d6855 Created pod: maas-default-gateway-openshift-default-76597d6855-vxzxx 31m Normal SuccessfulCreate replicaset/e2e-worker-w5-3b7476-openshift-default-59975457bf Created pod: e2e-worker-w5-3b7476-openshift-default-59975457bf-q7gfm 31m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-76597d6855 Created pod: maas-default-gateway-openshift-default-76597d6855-6cfhw 31m Normal SuccessfulCreate replicaset/e2e-worker-w5-3b7476-openshift-default-59975457bf Created pod: e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56 31m Warning ProbeError pod/e2e-shared-a-w2-57f116-openshift-default-75f4fc499-znglh Readiness probe error: HTTP probe failed with statuscode: 503... 31m Normal Started pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-q7gfm Container started 31m Normal Started pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-7zmxp Container started 31m Normal Started pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56 Container started 31m Normal AddedInterface pod/maas-default-gateway-openshift-default-76597d6855-6cfhw Add eth0 [10.132.0.104/23] from ovn-kubernetes 31m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-vxzxx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-6cfhw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Created pod/maas-default-gateway-openshift-default-76597d6855-vxzxx Container created 31m Normal Started pod/maas-default-gateway-openshift-default-76597d6855-vxzxx Container started 31m Normal Created pod/maas-default-gateway-openshift-default-76597d6855-6cfhw Container created 31m Normal Started pod/maas-default-gateway-openshift-default-76597d6855-6cfhw Container started 31m Normal AddedInterface pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-q7gfm Add eth0 [10.134.0.126/23] from ovn-kubernetes 31m Normal AddedInterface pod/maas-default-gateway-openshift-default-76597d6855-vxzxx Add eth0 [10.134.0.125/23] from ovn-kubernetes 31m Normal Created pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-7zmxp Container created 31m Normal Pulled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-7zmxp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Pulled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-q7gfm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-7zmxp Add eth0 [10.133.0.209/23] from ovn-kubernetes 31m Normal Created pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-q7gfm Container created 31m Normal AddedInterface pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56 Add eth0 [10.132.0.103/23] from ovn-kubernetes 31m Normal Pulled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Created pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56 Container created 31m Normal Killing pod/payload-processing-e2e-api-b-9d2b96-7d8dd46f5b-7rbgg Stopping container payload-processing 31m Normal Killing pod/payload-pre-processing-e2e-api-b-9d2b96-7b7d78cbff-gtqd9 Stopping container payload-pre-processing 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-22e72e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-22e72e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-00c846-5d7b447995-p8qhc Stopping container payload-pre-processing 31m Warning ProbeError pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-7zmxp Startup probe error: Get "http://10.133.0.209:15021/healthz/ready": dial tcp 10.133.0.209:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-7zmxp Startup probe failed: Get "http://10.133.0.209:15021/healthz/ready": dial tcp 10.133.0.209:15021: connect: connection refused 31m Normal Killing pod/payload-processing-e2e-shared-b-w2-00c846-6865789c84-c24hp Stopping container payload-processing 31m Warning Unhealthy pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-q7gfm Startup probe failed: Get "http://10.134.0.126:15021/healthz/ready": dial tcp 10.134.0.126:15021: connect: connection refused 31m Warning Unhealthy pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56 Startup probe failed: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused 31m Warning ProbeError pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-q7gfm Startup probe error: Get "http://10.134.0.126:15021/healthz/ready": dial tcp 10.134.0.126:15021: connect: connection refused... 31m Warning ProbeError pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56 Startup probe error: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused... 31m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-vxzxx Startup probe error: Get "http://10.134.0.125:15021/healthz/ready": dial tcp 10.134.0.125:15021: connect: connection refused... 31m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-vxzxx Startup probe failed: Get "http://10.134.0.125:15021/healthz/ready": dial tcp 10.134.0.125:15021: connect: connection refused 31m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-6cfhw Startup probe failed: Get "http://10.132.0.104:15021/healthz/ready": dial tcp 10.132.0.104:15021: connect: connection refused 31m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-6cfhw Startup probe error: Get "http://10.132.0.104:15021/healthz/ready": dial tcp 10.132.0.104:15021: connect: connection refused... 31m Normal Killing pod/payload-processing-e2e-inf-a-22e72e-85b6d5f746-nw6tv Stopping container payload-processing 31m Normal Killing pod/payload-pre-processing-e2e-inf-a-22e72e-7ffdf7c747-vbt9t Stopping container payload-pre-processing 31m Normal SuccessfulCreate replicaset/e2e-worker-w5-3b7476-openshift-default-59975457bf Created pod: e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp 31m Normal ScalingReplicaSet deployment/e2e-worker-w5-3b7476-openshift-default (combined from similar events): Scaled up replica set e2e-worker-w5-3b7476-openshift-default-59975457bf from 6 to 7 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-3b7476-openshift-default (combined from similar events): New size: 7; reason: 31m Normal Started pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp Container started 31m Normal AddedInterface pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp Add eth0 [10.132.0.105/23] from ovn-kubernetes 31m Normal Created pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp Container created 31m Normal Pulled pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Killing pod/e2e-shared-b-w2-00c846-openshift-default-785fd9fcdc-kf4s5 Stopping container istio-proxy 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-22e72e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-22e72e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Normal SuccessfulCreate replicaset/e2e-mt-4cbcf738-openshift-default-68fb497c9c Created pod: e2e-mt-4cbcf738-openshift-default-68fb497c9c-2ndqr 31m Normal ScalingReplicaSet deployment/e2e-mt-4cbcf738-openshift-default Scaled up replica set e2e-mt-4cbcf738-openshift-default-68fb497c9c from 0 to 1 31m Normal AddedLabel gateway/e2e-mt-4cbcf738 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-4cbcf738 31m Normal AddedInterface pod/e2e-mt-4cbcf738-openshift-default-68fb497c9c-2ndqr Add eth0 [10.134.0.127/23] from ovn-kubernetes 31m Warning Unhealthy pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp Startup probe failed: Get "http://10.132.0.105:15021/healthz/ready": dial tcp 10.132.0.105:15021: connect: connection refused 31m Warning ProbeError pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp Startup probe error: Get "http://10.132.0.105:15021/healthz/ready": dial tcp 10.132.0.105:15021: connect: connection refused... 31m Warning ProbeError pod/e2e-shared-b-w2-00c846-openshift-default-785fd9fcdc-kf4s5 Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning Unhealthy pod/e2e-shared-b-w2-00c846-openshift-default-785fd9fcdc-kf4s5 Readiness probe failed: HTTP probe failed with statuscode: 503 31m Normal Killing pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6n8hr Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-msl8m Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-fr9x5 Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-6frkp Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-q7gfm Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-7zmxp Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-mls56 Stopping container istio-proxy 31m Warning ProbeError pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-msl8m Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning Unhealthy pod/e2e-worker-w5-3b7476-openshift-default-59975457bf-msl8m Readiness probe failed: HTTP probe failed with statuscode: 503 31m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-4cbcf738 Scaled up replica set payload-pre-processing-e2e-mt-4cbcf738-67bd6b6f59 from 0 to 1 31m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-4cbcf738 Scaled up replica set payload-processing-e2e-mt-4cbcf738-97f795756 from 0 to 1 31m Warning FailedCreate replicaset/payload-processing-e2e-mt-4cbcf738-97f795756 Error creating: pods "payload-processing-e2e-mt-4cbcf738-97f795756-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-4cbcf738: serviceaccount "payload-processing-e2e-mt-4cbcf738" not found 31m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-4cbcf738-67bd6b6f59 Error creating: pods "payload-pre-processing-e2e-mt-4cbcf738-67bd6b6f59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-4cbcf738: serviceaccount "payload-processing-e2e-mt-4cbcf738" not found 31m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-4cbcf738-97f795756 Created pod: payload-processing-e2e-mt-4cbcf738-97f795756-8vnkw 31m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-4cbcf738-67bd6b6f59 Created pod: payload-pre-processing-e2e-mt-4cbcf738-67bd6b6f59-mfssb 31m Normal Created pod/payload-processing-e2e-mt-4cbcf738-97f795756-8vnkw Container created 31m Normal Pulled pod/payload-processing-e2e-mt-4cbcf738-97f795756-8vnkw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal Pulled pod/payload-pre-processing-e2e-mt-4cbcf738-67bd6b6f59-mfssb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal Started pod/payload-pre-processing-e2e-mt-4cbcf738-67bd6b6f59-mfssb Container started 31m Normal AddedInterface pod/payload-processing-e2e-mt-4cbcf738-97f795756-8vnkw Add eth0 [10.133.0.210/23] from ovn-kubernetes 31m Normal Started pod/payload-processing-e2e-mt-4cbcf738-97f795756-8vnkw Container started 31m Normal AddedInterface pod/payload-pre-processing-e2e-mt-4cbcf738-67bd6b6f59-mfssb Add eth0 [10.134.0.128/23] from ovn-kubernetes 31m Normal Created pod/payload-pre-processing-e2e-mt-4cbcf738-67bd6b6f59-mfssb Container created 31m Normal Pulled pod/e2e-inf-a-db5ef5-openshift-default-596b67fbf7-qrgxq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Started pod/e2e-inf-a-db5ef5-openshift-default-596b67fbf7-qrgxq Container started 31m Normal Created pod/e2e-inf-a-db5ef5-openshift-default-596b67fbf7-qrgxq Container created 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-db5ef5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-db5ef5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning Unhealthy pod/e2e-inf-a-db5ef5-openshift-default-596b67fbf7-qrgxq Startup probe failed: Get "http://10.133.0.204:15021/healthz/ready": dial tcp 10.133.0.204:15021: connect: connection refused 31m Warning ProbeError pod/e2e-inf-a-db5ef5-openshift-default-596b67fbf7-qrgxq Startup probe error: Get "http://10.133.0.204:15021/healthz/ready": dial tcp 10.133.0.204:15021: connect: connection refused... 31m Normal Killing pod/e2e-inf-a-9f4199-openshift-default-586448895-hl8h4 Stopping container istio-proxy 30m Normal Killing pod/payload-processing-e2e-worker-w0-fe4e0c-7dcb6cb4dd-vdb4p Stopping container payload-processing 30m Normal Killing pod/payload-pre-processing-e2e-worker-w0-fe4e0c-8599b44f79-9xn4d Stopping container payload-pre-processing 30m Normal Killing pod/payload-processing-e2e-worker-w1-e516ff-57b6c4bc94-5nc6m Stopping container payload-processing 30m Normal Killing pod/payload-pre-processing-e2e-worker-w1-e516ff-b7d8666c5-8jsr9 Stopping container payload-pre-processing 30m Normal Killing pod/e2e-api-b-9d2b96-openshift-default-cc56c4dff-6w9vk Stopping container istio-proxy 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-db5ef5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-db5ef5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Normal AddedLabel gateway/e2e-inf-b-095081 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-095081 30m Normal ScalingReplicaSet deployment/e2e-inf-b-095081-openshift-default Scaled up replica set e2e-inf-b-095081-openshift-default-59dfcb6dcf from 0 to 1 30m Normal Killing pod/e2e-inf-a-22e72e-openshift-default-64656d9fb9-26n5c Stopping container istio-proxy 30m Normal SuccessfulCreate replicaset/e2e-inf-b-095081-openshift-default-59dfcb6dcf Created pod: e2e-inf-b-095081-openshift-default-59dfcb6dcf-vf8xr 30m Normal AddedInterface pod/e2e-inf-b-095081-openshift-default-59dfcb6dcf-vf8xr Add eth0 [10.133.0.217/23] from ovn-kubernetes 30m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-095081-6c6bf65b7b Created pod: payload-processing-e2e-inf-b-095081-6c6bf65b7b-rdq44 30m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-095081 Scaled up replica set payload-pre-processing-e2e-inf-b-095081-bd58cc787 from 0 to 1 30m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-095081-bd58cc787 Created pod: payload-pre-processing-e2e-inf-b-095081-bd58cc787-xz5kc 30m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-095081 Scaled up replica set payload-processing-e2e-inf-b-095081-6c6bf65b7b from 0 to 1 30m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-095081-6c6bf65b7b Error creating: pods "payload-processing-e2e-inf-b-095081-6c6bf65b7b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-095081: serviceaccount "payload-processing-e2e-inf-b-095081" not found 30m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-095081-bd58cc787 Error creating: pods "payload-pre-processing-e2e-inf-b-095081-bd58cc787-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-095081: serviceaccount "payload-processing-e2e-inf-b-095081" not found 30m Normal AddedInterface pod/payload-processing-e2e-inf-b-095081-6c6bf65b7b-rdq44 Add eth0 [10.134.0.130/23] from ovn-kubernetes 30m Normal Created pod/payload-pre-processing-e2e-inf-b-095081-bd58cc787-xz5kc Container created 30m Normal Started pod/payload-pre-processing-e2e-inf-b-095081-bd58cc787-xz5kc Container started 30m Normal Pulled pod/payload-pre-processing-e2e-inf-b-095081-bd58cc787-xz5kc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-095081-bd58cc787-xz5kc Add eth0 [10.134.0.129/23] from ovn-kubernetes 30m Normal Started pod/payload-processing-e2e-inf-b-095081-6c6bf65b7b-rdq44 Container started 30m Normal Created pod/payload-processing-e2e-inf-b-095081-6c6bf65b7b-rdq44 Container created 30m Normal Pulled pod/payload-processing-e2e-inf-b-095081-6c6bf65b7b-rdq44 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal Killing pod/payload-pre-processing-e2e-worker-w6-ebb7fd-6fd846f859-fvf9n Stopping container payload-pre-processing 30m Normal Killing pod/payload-processing-e2e-worker-w6-ebb7fd-b8fd547bb-7jkvh Stopping container payload-processing 30m Normal Killing pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-tzgt6 Stopping container istio-proxy 30m Normal Killing pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-6hhrb Stopping container istio-proxy 30m Normal Killing pod/e2e-worker-w0-fe4e0c-openshift-default-c7f4fc84c-rk79p Stopping container istio-proxy 30m Normal Killing pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-xg2tv Stopping container istio-proxy 30m Normal Killing pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-lwdq6 Stopping container istio-proxy 30m Normal Killing pod/e2e-worker-w1-e516ff-openshift-default-6f4b957485-9qgws Stopping container istio-proxy 30m Normal Killing pod/payload-pre-processing-e2e-worker-w3-cdcf67-dc57c4455-ljnxm Stopping container payload-pre-processing 30m Normal Killing pod/payload-processing-e2e-worker-w3-cdcf67-76579b7df4-vqbkj Stopping container payload-processing 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-095081-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-095081-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-ebb7fd-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 30m Normal SuccessfulCreate replicaset/e2e-worker-w6-ebb7fd-openshift-default-544f5768f Created pod: e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj 30m Normal SuccessfulCreate replicaset/e2e-worker-w6-ebb7fd-openshift-default-544f5768f Created pod: e2e-worker-w6-ebb7fd-openshift-default-544f5768f-r2z9z 30m Normal ScalingReplicaSet deployment/e2e-worker-w6-ebb7fd-openshift-default (combined from similar events): Scaled up replica set e2e-worker-w6-ebb7fd-openshift-default-544f5768f from 2 to 4 30m Normal Created pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj Container created 30m Normal Started pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-r2z9z Container started 30m Normal Created pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-r2z9z Container created 30m Normal Started pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj Container started 30m Normal Pulled pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Pulled pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-r2z9z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj Add eth0 [10.132.0.107/23] from ovn-kubernetes 30m Normal AddedInterface pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-r2z9z Add eth0 [10.133.0.221/23] from ovn-kubernetes 29m Warning ProbeError pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-r2z9z Startup probe error: Get "http://10.133.0.221:15021/healthz/ready": dial tcp 10.133.0.221:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-r2z9z Startup probe failed: Get "http://10.133.0.221:15021/healthz/ready": dial tcp 10.133.0.221:15021: connect: connection refused 29m Warning Unhealthy pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj Startup probe failed: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused 29m Warning ProbeError pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj Startup probe error: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused... 29m Normal Killing pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9rrql Stopping container istio-proxy 29m Normal Killing pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-w6tt7 Stopping container istio-proxy 29m Warning FailedToRetrieveImagePullSecret pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-r2z9z Unable to retrieve some image pull secrets (e2e-worker-w6-ebb7fd-openshift-default-dockercfg-k9smx); attempting to pull the image may not succeed. 29m Warning FailedToRetrieveImagePullSecret pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj Unable to retrieve some image pull secrets (e2e-worker-w6-ebb7fd-openshift-default-dockercfg-k9smx); attempting to pull the image may not succeed. 29m Normal Killing pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-9znmj Stopping container istio-proxy 29m Normal Killing pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-r2z9z Stopping container istio-proxy 29m Warning Unhealthy pod/payload-processing-e2e-worker-w3-cdcf67-76579b7df4-vqbkj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w3-cdcf67-dc57c4455-ljnxm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Warning ProbeError pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-w6tt7 Readiness probe error: HTTP probe failed with statuscode: 503... 29m Warning Unhealthy pod/e2e-worker-w6-ebb7fd-openshift-default-544f5768f-w6tt7 Readiness probe failed: HTTP probe failed with statuscode: 503 29m Normal Pulled pod/e2e-mt-4cbcf738-openshift-default-68fb497c9c-2ndqr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Created pod/e2e-mt-4cbcf738-openshift-default-68fb497c9c-2ndqr Container created 29m Normal Started pod/e2e-mt-4cbcf738-openshift-default-68fb497c9c-2ndqr Container started 29m Normal Killing pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-w5ldf Stopping container istio-proxy 29m Normal Killing pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-rwgvq Stopping container istio-proxy 29m Warning Unhealthy pod/e2e-mt-4cbcf738-openshift-default-68fb497c9c-2ndqr Startup probe failed: Get "http://10.134.0.127:15021/healthz/ready": dial tcp 10.134.0.127:15021: connect: connection refused 29m Warning ProbeError pod/e2e-mt-4cbcf738-openshift-default-68fb497c9c-2ndqr Startup probe error: Get "http://10.134.0.127:15021/healthz/ready": dial tcp 10.134.0.127:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-rwgvq Readiness probe failed: Get "http://10.134.0.34:15021/healthz/ready": dial tcp 10.134.0.34:15021: connect: connection refused 29m Warning ProbeError pod/e2e-worker-w3-cdcf67-openshift-default-6ff8f5cbc7-rwgvq Readiness probe error: Get "http://10.134.0.34:15021/healthz/ready": dial tcp 10.134.0.34:15021: connect: connection refused... 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-4cbcf738-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-4cbcf738-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-4cbcf738-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-4cbcf738-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Normal Started pod/e2e-inf-b-095081-openshift-default-59dfcb6dcf-vf8xr Container started 28m Normal Created pod/e2e-inf-b-095081-openshift-default-59dfcb6dcf-vf8xr Container created 28m Normal Pulled pod/e2e-inf-b-095081-openshift-default-59dfcb6dcf-vf8xr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Warning ProbeError pod/e2e-inf-b-095081-openshift-default-59dfcb6dcf-vf8xr Startup probe error: Get "http://10.133.0.217:15021/healthz/ready": dial tcp 10.133.0.217:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-inf-b-095081-openshift-default-59dfcb6dcf-vf8xr Startup probe failed: Get "http://10.133.0.217:15021/healthz/ready": dial tcp 10.133.0.217:15021: connect: connection refused 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-095081-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-095081-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Normal Killing pod/payload-processing-e2e-mt-4cbcf738-97f795756-8vnkw Stopping container payload-processing 27m Normal Killing pod/payload-pre-processing-e2e-mt-4cbcf738-67bd6b6f59-mfssb Stopping container payload-pre-processing 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-4cbcf738 Scaled up replica set payload-processing-e2e-mt-4cbcf738-97f795756 from 0 to 1 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-4cbcf738 Scaled up replica set payload-pre-processing-e2e-mt-4cbcf738-67bd6b6f59 from 0 to 1 26m Warning FailedCreate replicaset/payload-processing-e2e-mt-4cbcf738-97f795756 Error creating: pods "payload-processing-e2e-mt-4cbcf738-97f795756-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-4cbcf738: serviceaccount "payload-processing-e2e-mt-4cbcf738" not found 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-4cbcf738-67bd6b6f59 Error creating: pods "payload-pre-processing-e2e-mt-4cbcf738-67bd6b6f59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-4cbcf738: serviceaccount "payload-processing-e2e-mt-4cbcf738" not found 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-4cbcf738-97f795756 Created pod: payload-processing-e2e-mt-4cbcf738-97f795756-gk52w 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-4cbcf738-67bd6b6f59 Created pod: payload-pre-processing-e2e-mt-4cbcf738-67bd6b6f59-59zp6 26m Normal AddedInterface pod/payload-processing-e2e-mt-4cbcf738-97f795756-gk52w Add eth0 [10.134.0.133/23] from ovn-kubernetes 26m Normal Created pod/payload-processing-e2e-mt-4cbcf738-97f795756-gk52w Container created 26m Normal Started pod/payload-processing-e2e-mt-4cbcf738-97f795756-gk52w Container started 26m Normal Started pod/payload-pre-processing-e2e-mt-4cbcf738-67bd6b6f59-59zp6 Container started 26m Normal Created pod/payload-pre-processing-e2e-mt-4cbcf738-67bd6b6f59-59zp6 Container created 26m Normal Pulled pod/payload-pre-processing-e2e-mt-4cbcf738-67bd6b6f59-59zp6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/payload-pre-processing-e2e-mt-4cbcf738-67bd6b6f59-59zp6 Add eth0 [10.134.0.134/23] from ovn-kubernetes 26m Normal Pulled pod/payload-processing-e2e-mt-4cbcf738-97f795756-gk52w Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-a0bcbc-openshift-default New size: 2; reason: All metrics below target 25m Normal SuccessfulDelete replicaset/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c Deleted pod: e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-v2tpq 25m Normal Killing pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-v2tpq Stopping container istio-proxy 25m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: All metrics below target 25m Normal Killing pod/maas-default-gateway-openshift-default-76597d6855-rlhz6 Stopping container istio-proxy 25m Normal SuccessfulDelete replicaset/e2e-worker-w4-203c35-openshift-default-56497cdcc Deleted pod: e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt 25m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-76597d6855 Deleted pod: maas-default-gateway-openshift-default-76597d6855-6cfhw 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-203c35-openshift-default New size: 2; reason: All metrics below target 25m Normal Killing pod/maas-default-gateway-openshift-default-76597d6855-xrlzk Stopping container istio-proxy 25m Normal Killing pod/maas-default-gateway-openshift-default-76597d6855-6cfhw Stopping container istio-proxy 25m Normal Killing pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-4wmjt Stopping container istio-proxy 25m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-76597d6855 from 10 to 7 25m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-76597d6855 Deleted pod: maas-default-gateway-openshift-default-76597d6855-rlhz6 25m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-76597d6855 Deleted pod: maas-default-gateway-openshift-default-76597d6855-xrlzk 25m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-xrlzk Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-xrlzk Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-rlhz6 Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-rlhz6 Readiness probe failed: HTTP probe failed with statuscode: 503 25m Normal Killing pod/payload-processing-e2e-inf-a-db5ef5-657546bd74-dh9pg Stopping container payload-processing 25m Normal Killing pod/payload-pre-processing-e2e-inf-a-db5ef5-5b75965f5-j72vc Stopping container payload-pre-processing 24m Normal Killing pod/payload-pre-processing-e2e-mt-4cbcf738-67bd6b6f59-59zp6 Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-mt-4cbcf738-97f795756-gk52w Stopping container payload-processing 24m Normal ScalingReplicaSet deployment/e2e-worker-w4-203c35-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w4-203c35-openshift-default-56497cdcc from 2 to 1 24m Normal Killing pod/maas-default-gateway-openshift-default-76597d6855-zhmt8 Stopping container istio-proxy 24m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-76597d6855 Deleted pod: maas-default-gateway-openshift-default-76597d6855-xr2tj 24m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-76597d6855 Deleted pod: maas-default-gateway-openshift-default-76597d6855-zhmt8 24m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-76597d6855 Deleted pod: maas-default-gateway-openshift-default-76597d6855-vxzxx 24m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: All metrics below target 24m Normal Killing pod/maas-default-gateway-openshift-default-76597d6855-xr2tj Stopping container istio-proxy 24m Normal SuccessfulDelete replicaset/e2e-worker-w4-203c35-openshift-default-56497cdcc Deleted pod: e2e-worker-w4-203c35-openshift-default-56497cdcc-825cw 24m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-76597d6855 from 7 to 4 24m Normal Killing pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-825cw Stopping container istio-proxy 24m Normal Killing pod/maas-default-gateway-openshift-default-76597d6855-vxzxx Stopping container istio-proxy 24m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-db5ef5-5b75965f5-j72vc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-xr2tj Readiness probe error: HTTP probe failed with statuscode: 503... 24m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-xr2tj Readiness probe failed: HTTP probe failed with statuscode: 503 24m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 24m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-t6h4m Stopping container discovery 24m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-t6h4m 24m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 24m Normal Killing pod/e2e-inf-a-db5ef5-openshift-default-596b67fbf7-qrgxq Stopping container istio-proxy 24m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-zhmt8 Readiness probe failed: HTTP probe failed with statuscode: 503 24m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-zhmt8 Readiness probe error: HTTP probe failed with statuscode: 503... 24m Warning ProbeError pod/e2e-inf-a-db5ef5-openshift-default-596b67fbf7-qrgxq Readiness probe error: HTTP probe failed with statuscode: 503... 24m Warning Unhealthy pod/e2e-inf-a-db5ef5-openshift-default-596b67fbf7-qrgxq Readiness probe failed: HTTP probe failed with statuscode: 503 24m Normal Killing pod/payload-pre-processing-e2e-inf-b-095081-bd58cc787-xz5kc Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-inf-b-095081-6c6bf65b7b-rdq44 Stopping container payload-processing 24m Normal Killing pod/e2e-mt-4cbcf738-openshift-default-68fb497c9c-2ndqr Stopping container istio-proxy 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-a0bcbc-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 24m Normal ScalingReplicaSet deployment/e2e-worker-w2-a0bcbc-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c from 2 to 1 24m Normal SuccessfulDelete replicaset/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c Deleted pod: e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r 24m Normal Killing pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-w478r Stopping container istio-proxy 24m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: All metrics below target 24m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-76597d6855 from 4 to 3 24m Normal Killing pod/maas-default-gateway-openshift-default-76597d6855-brntt Stopping container istio-proxy 24m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-brntt Readiness probe error: HTTP probe failed with statuscode: 503... 24m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-brntt Readiness probe failed: HTTP probe failed with statuscode: 503 24m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-095081-bd58cc787-xz5kc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Normal Killing pod/e2e-inf-b-095081-openshift-default-59dfcb6dcf-vf8xr Stopping container istio-proxy 24m Normal Killing pod/payload-pre-processing-e2e-worker-w2-a0bcbc-69fdc5f5c9-d4d6x Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-worker-w2-a0bcbc-5bf4697664-l2vk9 Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-shared-a-w4-072ece-5db56894ff-cd5bk Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-shared-a-w4-072ece-8f4847696-zgkn8 Stopping container payload-processing 23m Normal ScalingReplicaSet deployment/e2e-worker-w2-a0bcbc-openshift-default Scaled up replica set e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c from 1 to 2 23m Normal SuccessfulCreate replicaset/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c Created pod: e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-g5hw6 23m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-a0bcbc-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 23m Normal Pulled pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-g5hw6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-g5hw6 Add eth0 [10.134.0.136/23] from ovn-kubernetes 23m Normal Created pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-g5hw6 Container created 23m Normal Started pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-g5hw6 Container started 23m Warning Unhealthy pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-g5hw6 Startup probe failed: Get "http://10.134.0.136:15021/healthz/ready": dial tcp 10.134.0.136:15021: connect: connection refused 23m Normal Killing pod/e2e-shared-a-w4-072ece-openshift-default-78b784fd8c-k2vfq Stopping container istio-proxy 23m Warning ProbeError pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-g5hw6 Startup probe error: Get "http://10.134.0.136:15021/healthz/ready": dial tcp 10.134.0.136:15021: connect: connection refused... 23m Normal Killing pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-bjdq9 Stopping container istio-proxy 23m Normal Killing pod/e2e-worker-w2-a0bcbc-openshift-default-65dc9d9b6c-g5hw6 Stopping container istio-proxy 23m Normal Killing pod/payload-processing-e2e-shared-b-w4-35df3e-7969f6bd6b-ptzqg Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-shared-b-w4-35df3e-7d67b7f575-zbnm5 Stopping container payload-pre-processing 23m Normal Killing pod/e2e-shared-b-w4-35df3e-openshift-default-6c585b76f-vl6wz Stopping container istio-proxy 23m Normal Killing pod/payload-pre-processing-e2e-worker-w4-203c35-5bb8b769bc-q9wmw Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-worker-w4-203c35-688854f56b-jk6wm Stopping container payload-processing 23m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w4-203c35-5bb8b769bc-q9wmw Readiness probe failed: timeout: failed to connect service "10.133.0.70:50052" within 1s: context deadline exceeded 22m Normal SuccessfulCreate replicaset/e2e-worker-w4-203c35-openshift-default-56497cdcc Created pod: e2e-worker-w4-203c35-openshift-default-56497cdcc-ssz9p 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-203c35-openshift-default (combined from similar events): New size: 2; reason: cpu resource utilization (percentage of request) above target 22m Normal ScalingReplicaSet deployment/e2e-worker-w4-203c35-openshift-default Scaled up replica set e2e-worker-w4-203c35-openshift-default-56497cdcc from 1 to 2 22m Normal AddedInterface pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-ssz9p Add eth0 [10.133.0.234/23] from ovn-kubernetes 22m Normal Pulled pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-ssz9p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Created pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-ssz9p Container created 22m Normal Started pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-ssz9p Container started 22m Warning ProbeError pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-ssz9p Startup probe error: Get "http://10.133.0.234:15021/healthz/ready": dial tcp 10.133.0.234:15021: connect: connection refused... 22m Warning Unhealthy pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-ssz9p Startup probe failed: Get "http://10.133.0.234:15021/healthz/ready": dial tcp 10.133.0.234:15021: connect: connection refused 22m Normal Killing pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-qqvgm Stopping container istio-proxy 22m Normal Killing pod/e2e-worker-w4-203c35-openshift-default-56497cdcc-ssz9p Stopping container istio-proxy 22m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-76597d6855 from 3 to 6 22m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 22m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-76597d6855 Created pod: maas-default-gateway-openshift-default-76597d6855-bdgtb 22m Normal Started pod/maas-default-gateway-openshift-default-76597d6855-bdgtb Container started 22m Normal Created pod/maas-default-gateway-openshift-default-76597d6855-bdgtb Container created 22m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-xjqqw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/maas-default-gateway-openshift-default-76597d6855-xjqqw Add eth0 [10.134.0.138/23] from ovn-kubernetes 22m Normal Started pod/maas-default-gateway-openshift-default-76597d6855-mtx78 Container started 22m Normal AddedInterface pod/maas-default-gateway-openshift-default-76597d6855-mtx78 Add eth0 [10.132.0.115/23] from ovn-kubernetes 22m Normal Created pod/maas-default-gateway-openshift-default-76597d6855-mtx78 Container created 22m Normal AddedInterface pod/maas-default-gateway-openshift-default-76597d6855-bdgtb Add eth0 [10.133.0.236/23] from ovn-kubernetes 22m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-mtx78 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-bdgtb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Started pod/maas-default-gateway-openshift-default-76597d6855-xjqqw Container started 22m Normal Created pod/maas-default-gateway-openshift-default-76597d6855-xjqqw Container created 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-bdgtb Startup probe failed: Get "http://10.133.0.236:15021/healthz/ready": dial tcp 10.133.0.236:15021: connect: connection refused 21m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-xjqqw Startup probe error: Get "http://10.134.0.138:15021/healthz/ready": dial tcp 10.134.0.138:15021: connect: connection refused... 21m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-bdgtb Startup probe error: Get "http://10.133.0.236:15021/healthz/ready": dial tcp 10.133.0.236:15021: connect: connection refused... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-mtx78 Startup probe failed: Get "http://10.132.0.115:15021/healthz/ready": dial tcp 10.132.0.115:15021: connect: connection refused 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-xjqqw Startup probe failed: Get "http://10.134.0.138:15021/healthz/ready": dial tcp 10.134.0.138:15021: connect: connection refused 21m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-mtx78 Startup probe error: Get "http://10.132.0.115:15021/healthz/ready": dial tcp 10.132.0.115:15021: connect: connection refused... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-mtx78 Startup probe failed: Get "http://10.132.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-xjqqw Startup probe failed: Get "http://10.134.0.138:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 21m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-xjqqw Startup probe error: Get "http://10.134.0.138:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 21m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-bdgtb Startup probe error: Get "http://10.133.0.236:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-bdgtb Startup probe failed: Get "http://10.133.0.236:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 21m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-mtx78 Startup probe error: Get "http://10.132.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 21m Normal Created pod/maas-default-gateway-openshift-default-76597d6855-v59qw Container created 21m Normal Started pod/maas-default-gateway-openshift-default-76597d6855-v59qw Container started 21m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-76597d6855 from 6 to 8 21m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-v59qw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-76597d6855-v59qw Add eth0 [10.133.0.238/23] from ovn-kubernetes 21m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: 21m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-dqn2b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-76597d6855-dqn2b Add eth0 [10.134.0.139/23] from ovn-kubernetes 21m Normal Created pod/maas-default-gateway-openshift-default-76597d6855-dqn2b Container created 21m Normal Started pod/maas-default-gateway-openshift-default-76597d6855-dqn2b Container started 21m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-dqn2b Startup probe error: Get "http://10.134.0.139:15021/healthz/ready": dial tcp 10.134.0.139:15021: connect: connection refused... 21m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-v59qw Startup probe error: Get "http://10.133.0.238:15021/healthz/ready": dial tcp 10.133.0.238:15021: connect: connection refused... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-v59qw Startup probe failed: Get "http://10.133.0.238:15021/healthz/ready": dial tcp 10.133.0.238:15021: connect: connection refused 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-dqn2b Startup probe failed: Get "http://10.134.0.139:15021/healthz/ready": dial tcp 10.134.0.139:15021: connect: connection refused 20m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 20m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-76597d6855 from 8 to 10 20m Normal AddedInterface pod/maas-default-gateway-openshift-default-76597d6855-j8cqc Add eth0 [10.133.0.239/23] from ovn-kubernetes 20m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-hk6mk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal AddedInterface pod/maas-default-gateway-openshift-default-76597d6855-hk6mk Add eth0 [10.132.0.116/23] from ovn-kubernetes 20m Normal Created pod/maas-default-gateway-openshift-default-76597d6855-hk6mk Container created 20m Normal Pulled pod/maas-default-gateway-openshift-default-76597d6855-j8cqc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Created pod/maas-default-gateway-openshift-default-76597d6855-j8cqc Container created 20m Normal Started pod/maas-default-gateway-openshift-default-76597d6855-hk6mk Container started 20m Normal Started pod/maas-default-gateway-openshift-default-76597d6855-j8cqc Container started 20m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-hk6mk Startup probe failed: Get "http://10.132.0.116:15021/healthz/ready": dial tcp 10.132.0.116:15021: connect: connection refused 20m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-hk6mk Startup probe error: Get "http://10.132.0.116:15021/healthz/ready": dial tcp 10.132.0.116:15021: connect: connection refused... 20m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-j8cqc Startup probe error: Get "http://10.133.0.239:15021/healthz/ready": dial tcp 10.133.0.239:15021: connect: connection refused... 20m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-j8cqc Startup probe failed: Get "http://10.133.0.239:15021/healthz/ready": dial tcp 10.133.0.239:15021: connect: connection refused 20m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-hk6mk Startup probe failed: Get "http://10.132.0.116:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 20m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-hk6mk Startup probe error: Get "http://10.132.0.116:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 20m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-j8cqc Startup probe failed: Get "http://10.133.0.239:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 20m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-j8cqc Startup probe error: Get "http://10.133.0.239:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 19m Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-jwwww Readiness probe error: Get "http://10.133.0.164:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 19m Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-jwwww Readiness probe failed: Get "http://10.133.0.164:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 15m Normal AddedLabel gateway/e2e-worker-single-main-07ce81 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-07ce81 15m Normal ScalingReplicaSet deployment/e2e-worker-single-main-07ce81-openshift-default Scaled up replica set e2e-worker-single-main-07ce81-openshift-default-5446f4ff6f from 0 to 1 15m Normal SuccessfulCreate replicaset/e2e-worker-single-main-07ce81-openshift-default-5446f4ff6f Created pod: e2e-worker-single-main-07ce81-openshift-default-5446f4ff6fwvjql 15m Normal AddedInterface pod/e2e-worker-single-main-07ce81-openshift-default-5446f4ff6fwvjql Add eth0 [10.134.0.154/23] from ovn-kubernetes 15m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-07ce81 Scaled up replica set payload-pre-processing-e2e-worker-single-main-07ce81-9ddd447dc from 0 to 1 15m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-07ce81-9ddd447dc Error creating: pods "payload-pre-processing-e2e-worker-single-main-07ce81-9ddd447dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-07ce81: serviceaccount "payload-processing-e2e-worker-single-main-07ce81" not found 15m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-07ce81 Scaled up replica set payload-processing-e2e-worker-single-main-07ce81-767654f68d from 0 to 1 15m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-07ce81-767654f68d Error creating: pods "payload-processing-e2e-worker-single-main-07ce81-767654f68d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-07ce81: serviceaccount "payload-processing-e2e-worker-single-main-07ce81" not found 15m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-07ce81-9ddd44mgvn Add eth0 [10.134.0.155/23] from ovn-kubernetes 15m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-07ce81-767654f68d Created pod: payload-processing-e2e-worker-single-main-07ce81-767654f68x8k9j 15m Normal Created pod/payload-processing-e2e-worker-single-main-07ce81-767654f68x8k9j Container created 15m Normal Pulled pod/payload-processing-e2e-worker-single-main-07ce81-767654f68x8k9j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 15m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-07ce81-767654f68x8k9j Add eth0 [10.134.0.156/23] from ovn-kubernetes 15m Normal Started pod/payload-pre-processing-e2e-worker-single-main-07ce81-9ddd44mgvn Container started 15m Normal Created pod/payload-pre-processing-e2e-worker-single-main-07ce81-9ddd44mgvn Container created 15m Normal Started pod/payload-processing-e2e-worker-single-main-07ce81-767654f68x8k9j Container started 15m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-07ce81-9ddd44mgvn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 15m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-07ce81-9ddd447dc Created pod: payload-pre-processing-e2e-worker-single-main-07ce81-9ddd44mgvn 15m Normal Pulled pod/e2e-worker-single-main-07ce81-openshift-default-5446f4ff6fwvjql Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 15m Normal Created pod/e2e-worker-single-main-07ce81-openshift-default-5446f4ff6fwvjql Container created 15m Normal Started pod/e2e-worker-single-main-07ce81-openshift-default-5446f4ff6fwvjql Container started 15m Warning Unhealthy pod/e2e-worker-single-main-07ce81-openshift-default-5446f4ff6fwvjql Startup probe failed: Get "http://10.134.0.154:15021/healthz/ready": dial tcp 10.134.0.154:15021: connect: connection refused 15m Warning ProbeError pod/e2e-worker-single-main-07ce81-openshift-default-5446f4ff6fwvjql Startup probe error: Get "http://10.134.0.154:15021/healthz/ready": dial tcp 10.134.0.154:15021: connect: connection refused... 15m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-07ce81-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 15m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-07ce81-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 14m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-07ce81-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 14m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-07ce81-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 14m Normal Killing pod/payload-processing-e2e-worker-single-main-07ce81-767654f68x8k9j Stopping container payload-processing 14m Normal Killing pod/payload-pre-processing-e2e-worker-single-main-07ce81-9ddd44mgvn Stopping container payload-pre-processing 13m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-07ce81-767654f68x8k9j Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 13m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-07ce81-9ddd44mgvn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 13m Normal Killing pod/e2e-worker-single-main-07ce81-openshift-default-5446f4ff6fwvjql Stopping container istio-proxy 11m Normal ScalingReplicaSet deployment/e2e-worker-single-main-2f48c1-openshift-default Scaled up replica set e2e-worker-single-main-2f48c1-openshift-default-6799d44489 from 0 to 1 11m Normal SuccessfulCreate replicaset/e2e-worker-single-main-2f48c1-openshift-default-6799d44489 Created pod: e2e-worker-single-main-2f48c1-openshift-default-6799d44489xnlnj 11m Normal AddedLabel gateway/e2e-worker-single-main-2f48c1 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-2f48c1 11m Normal AddedInterface pod/e2e-worker-single-main-2f48c1-openshift-default-6799d44489xnlnj Add eth0 [10.134.0.162/23] from ovn-kubernetes 11m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-2f48c1-7847c47699 Created pod: payload-pre-processing-e2e-worker-single-main-2f48c1-7847czcwdv 11m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-2f48c1 Scaled up replica set payload-processing-e2e-worker-single-main-2f48c1-5c55d79896 from 0 to 1 11m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-2f48c1 Scaled up replica set payload-pre-processing-e2e-worker-single-main-2f48c1-7847c47699 from 0 to 1 11m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-2f48c1-7847c47699 Error creating: pods "payload-pre-processing-e2e-worker-single-main-2f48c1-7847c47699-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-2f48c1: serviceaccount "payload-processing-e2e-worker-single-main-2f48c1" not found 11m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-2f48c1-5c55d79896 Created pod: payload-processing-e2e-worker-single-main-2f48c1-5c55d7989qtlvn 11m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-2f48c1-5c55d79896 Error creating: pods "payload-processing-e2e-worker-single-main-2f48c1-5c55d79896-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-2f48c1: serviceaccount "payload-processing-e2e-worker-single-main-2f48c1" not found 11m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-2f48c1-7847czcwdv Add eth0 [10.134.0.163/23] from ovn-kubernetes 11m Normal Started pod/payload-pre-processing-e2e-worker-single-main-2f48c1-7847czcwdv Container started 11m Normal Created pod/payload-pre-processing-e2e-worker-single-main-2f48c1-7847czcwdv Container created 11m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-2f48c1-7847czcwdv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 11m Normal Created pod/payload-processing-e2e-worker-single-main-2f48c1-5c55d7989qtlvn Container created 11m Normal Started pod/payload-processing-e2e-worker-single-main-2f48c1-5c55d7989qtlvn Container started 11m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-2f48c1-5c55d7989qtlvn Add eth0 [10.134.0.164/23] from ovn-kubernetes 11m Normal Pulled pod/payload-processing-e2e-worker-single-main-2f48c1-5c55d7989qtlvn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 10m Normal Pulled pod/e2e-worker-single-main-2f48c1-openshift-default-6799d44489xnlnj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 10m Normal Created pod/e2e-worker-single-main-2f48c1-openshift-default-6799d44489xnlnj Container created 10m Normal Started pod/e2e-worker-single-main-2f48c1-openshift-default-6799d44489xnlnj Container started 10m Warning Unhealthy pod/e2e-worker-single-main-2f48c1-openshift-default-6799d44489xnlnj Startup probe failed: Get "http://10.134.0.162:15021/healthz/ready": dial tcp 10.134.0.162:15021: connect: connection refused 10m Warning ProbeError pod/e2e-worker-single-main-2f48c1-openshift-default-6799d44489xnlnj Startup probe error: Get "http://10.134.0.162:15021/healthz/ready": dial tcp 10.134.0.162:15021: connect: connection refused... 10m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-2f48c1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 10m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-2f48c1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 9m38s Normal Killing pod/payload-processing-e2e-worker-single-main-2f48c1-5c55d7989qtlvn Stopping container payload-processing 9m37s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-2f48c1-7847czcwdv Stopping container payload-pre-processing 9m31s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-2f48c1-5c55d7989qtlvn Readiness probe failed: timeout: failed to connect service "10.134.0.164:50052" within 1s: context deadline exceeded 9m31s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-2f48c1-7847czcwdv Readiness probe failed: timeout: failed to connect service "10.134.0.163:50052" within 1s: context deadline exceeded 8m29s Normal Killing pod/e2e-worker-single-main-2f48c1-openshift-default-6799d44489xnlnj Stopping container istio-proxy 7m47s Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-74hkj Readiness probe failed: Get "http://10.132.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 7m47s Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-7c95j Readiness probe failed: Get "http://10.134.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 7m47s Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-74hkj Readiness probe error: Get "http://10.132.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 7m47s Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-7c95j Readiness probe error: Get "http://10.134.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 6m35s Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-mtx78 Readiness probe error: Get "http://10.132.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 6m35s Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-mtx78 Readiness probe failed: Get "http://10.132.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 6m12s Normal AddedLabel gateway/e2e-ait-dffbe0c3 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-dffbe0c3 6m12s Normal ScalingReplicaSet deployment/e2e-ait-dffbe0c3-openshift-default Scaled up replica set e2e-ait-dffbe0c3-openshift-default-857cd74dd7 from 0 to 1 6m12s Normal SuccessfulCreate replicaset/e2e-ait-dffbe0c3-openshift-default-857cd74dd7 Created pod: e2e-ait-dffbe0c3-openshift-default-857cd74dd7-5sn8z 6m11s Normal AddedInterface pod/e2e-ait-dffbe0c3-openshift-default-857cd74dd7-5sn8z Add eth0 [10.133.0.247/23] from ovn-kubernetes 6m4s Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-dffbe0c3 Scaled up replica set payload-processing-e2e-ait-dffbe0c3-7f8d8457b from 0 to 1 6m4s Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-dffbe0c3-546cf94567 Error creating: pods "payload-pre-processing-e2e-ait-dffbe0c3-546cf94567-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-dffbe0c3: serviceaccount "payload-processing-e2e-ait-dffbe0c3" not found 6m4s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-dffbe0c3-546cf94567 Created pod: payload-pre-processing-e2e-ait-dffbe0c3-546cf94567-9wrrk 6m4s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-dffbe0c3 Scaled up replica set payload-pre-processing-e2e-ait-dffbe0c3-546cf94567 from 0 to 1 6m4s Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-dffbe0c3-7f8d8457b Created pod: payload-processing-e2e-ait-dffbe0c3-7f8d8457b-mmn44 6m4s Warning FailedCreate replicaset/payload-processing-e2e-ait-dffbe0c3-7f8d8457b Error creating: pods "payload-processing-e2e-ait-dffbe0c3-7f8d8457b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-dffbe0c3: serviceaccount "payload-processing-e2e-ait-dffbe0c3" not found 6m3s Normal Created pod/payload-pre-processing-e2e-ait-dffbe0c3-546cf94567-9wrrk Container created 6m3s Normal Pulled pod/payload-pre-processing-e2e-ait-dffbe0c3-546cf94567-9wrrk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 6m3s Normal AddedInterface pod/payload-processing-e2e-ait-dffbe0c3-7f8d8457b-mmn44 Add eth0 [10.134.0.174/23] from ovn-kubernetes 6m3s Normal AddedInterface pod/payload-pre-processing-e2e-ait-dffbe0c3-546cf94567-9wrrk Add eth0 [10.134.0.173/23] from ovn-kubernetes 6m3s Normal Started pod/payload-pre-processing-e2e-ait-dffbe0c3-546cf94567-9wrrk Container started 6m2s Normal Pulled pod/payload-processing-e2e-ait-dffbe0c3-7f8d8457b-mmn44 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 6m2s Normal Started pod/payload-processing-e2e-ait-dffbe0c3-7f8d8457b-mmn44 Container started 6m2s Normal Created pod/payload-processing-e2e-ait-dffbe0c3-7f8d8457b-mmn44 Container created 5m46s Normal Pulled pod/e2e-ait-dffbe0c3-openshift-default-857cd74dd7-5sn8z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 5m45s Normal Started pod/e2e-ait-dffbe0c3-openshift-default-857cd74dd7-5sn8z Container started 5m45s Normal Created pod/e2e-ait-dffbe0c3-openshift-default-857cd74dd7-5sn8z Container created 5m40s Warning ProbeError pod/e2e-ait-dffbe0c3-openshift-default-857cd74dd7-5sn8z Startup probe error: Get "http://10.133.0.247:15021/healthz/ready": dial tcp 10.133.0.247:15021: connect: connection refused... 5m40s Warning Unhealthy pod/e2e-ait-dffbe0c3-openshift-default-857cd74dd7-5sn8z Startup probe failed: Get "http://10.133.0.247:15021/healthz/ready": dial tcp 10.133.0.247:15021: connect: connection refused 5m19s Normal Killing pod/payload-pre-processing-e2e-ait-dffbe0c3-546cf94567-9wrrk Stopping container payload-pre-processing 5m19s Normal Killing pod/payload-processing-e2e-ait-dffbe0c3-7f8d8457b-mmn44 Stopping container payload-processing 5m11s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-dffbe0c3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 5m11s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-dffbe0c3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 5m10s Normal Killing pod/e2e-ait-dffbe0c3-openshift-default-857cd74dd7-5sn8z Stopping container istio-proxy 5m9s Warning ProbeError pod/e2e-ait-dffbe0c3-openshift-default-857cd74dd7-5sn8z Readiness probe error: HTTP probe failed with statuscode: 503... 5m9s Warning Unhealthy pod/e2e-ait-dffbe0c3-openshift-default-857cd74dd7-5sn8z Readiness probe failed: HTTP probe failed with statuscode: 503 5m8s Normal AddedInterface pod/e2e-authcfg-http-cc2474-openshift-default-7f6458856f-qw52z Add eth0 [10.134.0.176/23] from ovn-kubernetes 5m8s Normal SuccessfulCreate replicaset/e2e-authcfg-http-cc2474-openshift-default-7f6458856f Created pod: e2e-authcfg-http-cc2474-openshift-default-7f6458856f-qw52z 5m8s Normal ScalingReplicaSet deployment/e2e-authcfg-http-cc2474-openshift-default Scaled up replica set e2e-authcfg-http-cc2474-openshift-default-7f6458856f from 0 to 1 5m8s Normal AddedLabel gateway/e2e-authcfg-http-cc2474 Added label istio.io/rev=openshift-gateway to gateway e2e-authcfg-http-cc2474 5m Warning FailedCreate replicaset/payload-processing-e2e-authcfg-http-cc2474-684cc5b57d Error creating: pods "payload-processing-e2e-authcfg-http-cc2474-684cc5b57d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-cc2474: serviceaccount "payload-processing-e2e-authcfg-http-cc2474" not found 5m Normal SuccessfulCreate replicaset/payload-processing-e2e-authcfg-http-cc2474-684cc5b57d Created pod: payload-processing-e2e-authcfg-http-cc2474-684cc5b57d-fzbbc 5m Normal ScalingReplicaSet deployment/payload-processing-e2e-authcfg-http-cc2474 Scaled up replica set payload-processing-e2e-authcfg-http-cc2474-684cc5b57d from 0 to 1 5m Warning FailedCreate replicaset/payload-pre-processing-e2e-authcfg-http-cc2474-7df96f57b7 Error creating: pods "payload-pre-processing-e2e-authcfg-http-cc2474-7df96f57b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-cc2474: serviceaccount "payload-processing-e2e-authcfg-http-cc2474" not found 5m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-authcfg-http-cc2474 Scaled up replica set payload-pre-processing-e2e-authcfg-http-cc2474-7df96f57b7 from 0 to 1 4m59s Normal AddedInterface pod/payload-pre-processing-e2e-authcfg-http-cc2474-7df96f57b7-dp897 Add eth0 [10.134.0.177/23] from ovn-kubernetes 4m59s Normal Pulled pod/payload-pre-processing-e2e-authcfg-http-cc2474-7df96f57b7-dp897 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 4m59s Normal Created pod/payload-processing-e2e-authcfg-http-cc2474-684cc5b57d-fzbbc Container created 4m59s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-authcfg-http-cc2474-7df96f57b7 Created pod: payload-pre-processing-e2e-authcfg-http-cc2474-7df96f57b7-dp897 4m59s Normal AddedInterface pod/payload-processing-e2e-authcfg-http-cc2474-684cc5b57d-fzbbc Add eth0 [10.133.0.249/23] from ovn-kubernetes 4m59s Normal Created pod/payload-pre-processing-e2e-authcfg-http-cc2474-7df96f57b7-dp897 Container created 4m59s Normal Started pod/payload-pre-processing-e2e-authcfg-http-cc2474-7df96f57b7-dp897 Container started 4m59s Normal Pulled pod/payload-processing-e2e-authcfg-http-cc2474-684cc5b57d-fzbbc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 4m59s Normal Started pod/payload-processing-e2e-authcfg-http-cc2474-684cc5b57d-fzbbc Container started 4m38s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-cc2474-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 4m38s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-cc2474-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 4m38s Normal Pulled pod/e2e-authcfg-http-cc2474-openshift-default-7f6458856f-qw52z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 4m38s Normal Created pod/e2e-authcfg-http-cc2474-openshift-default-7f6458856f-qw52z Container created 4m38s Normal Started pod/e2e-authcfg-http-cc2474-openshift-default-7f6458856f-qw52z Container started 4m32s Warning ProbeError pod/e2e-authcfg-http-cc2474-openshift-default-7f6458856f-qw52z Startup probe error: Get "http://10.134.0.176:15021/healthz/ready": dial tcp 10.134.0.176:15021: connect: connection refused... 4m32s Warning Unhealthy pod/e2e-authcfg-http-cc2474-openshift-default-7f6458856f-qw52z Startup probe failed: Get "http://10.134.0.176:15021/healthz/ready": dial tcp 10.134.0.176:15021: connect: connection refused 3m53s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-cc2474-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 3m53s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-cc2474-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 3m11s Normal Killing pod/payload-processing-e2e-authcfg-http-cc2474-684cc5b57d-fzbbc Stopping container payload-processing 3m11s Normal Killing pod/payload-pre-processing-e2e-authcfg-http-cc2474-7df96f57b7-dp897 Stopping container payload-pre-processing 3m4s Normal Killing pod/e2e-authcfg-http-cc2474-openshift-default-7f6458856f-qw52z Stopping container istio-proxy 3m1s Warning ProbeError pod/e2e-authcfg-http-cc2474-openshift-default-7f6458856f-qw52z Readiness probe error: HTTP probe failed with statuscode: 503... 3m1s Warning Unhealthy pod/e2e-authcfg-http-cc2474-openshift-default-7f6458856f-qw52z Readiness probe failed: HTTP probe failed with statuscode: 503 2m52s Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-v59qw Readiness probe failed: Get "http://10.133.0.238:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m52s Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-v59qw Readiness probe error: Get "http://10.133.0.238:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 2m20s Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-bdgtb Readiness probe error: Get "http://10.133.0.236:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 2m20s Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-bdgtb Readiness probe failed: Get "http://10.133.0.236:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m19s Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-j8cqc Readiness probe error: Get "http://10.133.0.239:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 2m19s Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-j8cqc Readiness probe failed: Get "http://10.133.0.239:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m19s Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-hk6mk Readiness probe failed: Get "http://10.132.0.116:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m19s Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-hk6mk Readiness probe error: Get "http://10.132.0.116:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 2m19s Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-xjqqw Readiness probe failed: Get "http://10.134.0.138:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m19s Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-xjqqw Readiness probe error: Get "http://10.134.0.138:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 112s Warning Unhealthy pod/maas-default-gateway-openshift-default-76597d6855-dqn2b Readiness probe failed: Get "http://10.134.0.139:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 112s Warning ProbeError pod/maas-default-gateway-openshift-default-76597d6855-dqn2b Readiness probe error: Get "http://10.134.0.139:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)...