=== events in openshift-ingress (2026-10-30T23:24:13Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 64m Normal Scheduled pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-7m76p Successfully assigned openshift-ingress/e2e-worker-w3-d52c59-openshift-default-5466968fb7-7m76p to ip-10-0-130-77.ec2.internal 62m Normal Scheduled pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-95kvh Successfully assigned openshift-ingress/e2e-worker-w3-d52c59-openshift-default-5466968fb7-95kvh to ip-10-0-142-185.ec2.internal 61m Normal Scheduled pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-rpk4x Successfully assigned openshift-ingress/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-rpk4x to ip-10-0-130-77.ec2.internal 62m Normal Scheduled pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hstmf Successfully assigned openshift-ingress/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hstmf to ip-10-0-135-132.ec2.internal 46m Normal Scheduled pod/e2e-inf-b-c13db0-openshift-default-f894f747-cbtvq Successfully assigned openshift-ingress/e2e-inf-b-c13db0-openshift-default-f894f747-cbtvq to ip-10-0-135-132.ec2.internal 64m Normal Scheduled pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hq9vz Successfully assigned openshift-ingress/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hq9vz to ip-10-0-130-77.ec2.internal 42m Normal Scheduled pod/e2e-inf-b-c13db0-openshift-default-f894f747-t6h8c Successfully assigned openshift-ingress/e2e-inf-b-c13db0-openshift-default-f894f747-t6h8c to ip-10-0-142-185.ec2.internal 45m Normal Scheduled pod/e2e-inf-b-867f12-openshift-default-6584f84965-jlqnx Successfully assigned openshift-ingress/e2e-inf-b-867f12-openshift-default-6584f84965-jlqnx to ip-10-0-135-132.ec2.internal 62m Normal Scheduled pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hnqx6 Successfully assigned openshift-ingress/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hnqx6 to ip-10-0-142-185.ec2.internal 61m Normal Scheduled pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-gwz9t Successfully assigned openshift-ingress/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-gwz9t to ip-10-0-135-132.ec2.internal 41m Normal Scheduled pod/e2e-inf-b-867f12-openshift-default-6584f84965-9rbbv Successfully assigned openshift-ingress/e2e-inf-b-867f12-openshift-default-6584f84965-9rbbv to ip-10-0-142-185.ec2.internal 5m25s Normal Scheduled pod/e2e-ait-1595eccb-openshift-default-84f7ff667-58jtb Successfully assigned openshift-ingress/e2e-ait-1595eccb-openshift-default-84f7ff667-58jtb to ip-10-0-142-185.ec2.internal 46m Normal Scheduled pod/e2e-inf-b-f0bb45-openshift-default-6b64cff887-s9c4q Successfully assigned openshift-ingress/e2e-inf-b-f0bb45-openshift-default-6b64cff887-s9c4q to ip-10-0-135-132.ec2.internal 62m Normal Scheduled pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-9j5tr Successfully assigned openshift-ingress/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-9j5tr to ip-10-0-130-77.ec2.internal 42m Normal Scheduled pod/e2e-inf-b-f0bb45-openshift-default-6b64cff887-w4hj4 Successfully assigned openshift-ingress/e2e-inf-b-f0bb45-openshift-default-6b64cff887-w4hj4 to ip-10-0-142-185.ec2.internal 62m Normal Scheduled pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-97nr4 Successfully assigned openshift-ingress/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-97nr4 to ip-10-0-135-132.ec2.internal 61m Normal Scheduled pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8wzt7 Successfully assigned openshift-ingress/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8wzt7 to ip-10-0-142-185.ec2.internal 49m Normal Scheduled pod/e2e-inf-b-f9d87b-openshift-default-6fb7566b69-2c4wk Successfully assigned openshift-ingress/e2e-inf-b-f9d87b-openshift-default-6fb7566b69-2c4wk to ip-10-0-142-185.ec2.internal 65m Normal Scheduled pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-25w9c Successfully assigned openshift-ingress/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-25w9c to ip-10-0-142-185.ec2.internal 61m Normal Scheduled pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-57vcj Successfully assigned openshift-ingress/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-57vcj to ip-10-0-135-132.ec2.internal 48m Normal Scheduled pod/e2e-inf-b-6013c8-openshift-default-96f465855-llgxc Successfully assigned openshift-ingress/e2e-inf-b-6013c8-openshift-default-96f465855-llgxc to ip-10-0-135-132.ec2.internal 42m Normal Scheduled pod/e2e-inf-b-f9d87b-openshift-default-6fb7566b69-5hv5d Successfully assigned openshift-ingress/e2e-inf-b-f9d87b-openshift-default-6fb7566b69-5hv5d to ip-10-0-135-132.ec2.internal 67m Normal Scheduled pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8dgzg Successfully assigned openshift-ingress/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8dgzg to ip-10-0-142-185.ec2.internal 65m Normal Scheduled pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-72ws5 Successfully assigned openshift-ingress/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-72ws5 to ip-10-0-135-132.ec2.internal 60m Normal Scheduled pod/e2e-ait-1ba9ae08-openshift-default-7758f9c76c-wbv2b Successfully assigned openshift-ingress/e2e-ait-1ba9ae08-openshift-default-7758f9c76c-wbv2b to ip-10-0-135-132.ec2.internal 32m Normal Scheduled pod/e2e-shared-b-w3-e14e00-openshift-default-565cddf77c-7jg4k Successfully assigned openshift-ingress/e2e-shared-b-w3-e14e00-openshift-default-565cddf77c-7jg4k to ip-10-0-142-185.ec2.internal 67m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-09d0d5-75f8f97f55-6hq48 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-09d0d5-75f8f97f55-6hq48 to ip-10-0-142-185.ec2.internal 64m Normal Scheduled pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-582w4 Successfully assigned openshift-ingress/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-582w4 to ip-10-0-135-132.ec2.internal 32m Normal Scheduled pod/e2e-ipp-a-e3a7dd-openshift-default-6c66b84dc9-v6hrm Successfully assigned openshift-ingress/e2e-ipp-a-e3a7dd-openshift-default-6c66b84dc9-v6hrm to ip-10-0-135-132.ec2.internal 60m Normal Scheduled pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-6wn2g Successfully assigned openshift-ingress/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-6wn2g to ip-10-0-130-77.ec2.internal 61m Normal Scheduled pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-8r62w Successfully assigned openshift-ingress/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-8r62w to ip-10-0-142-185.ec2.internal 47m Normal Scheduled pod/e2e-inf-b-54c7f0-openshift-default-79cd69d455-tfzdz Successfully assigned openshift-ingress/e2e-inf-b-54c7f0-openshift-default-79cd69d455-tfzdz to ip-10-0-142-185.ec2.internal 67m Normal Scheduled pod/payload-processing-e2e-worker-w5-54114e-5cf4644f-8htwc Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-54114e-5cf4644f-8htwc to ip-10-0-135-132.ec2.internal 60m Normal Scheduled pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-97d66 Successfully assigned openshift-ingress/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-97d66 to ip-10-0-135-132.ec2.internal 12m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-2cad7e-d85d5hddg4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-2cad7e-d85d5hddg4 to ip-10-0-142-185.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-b-cdfa86-8f44c5dc9-jgtzp Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-cdfa86-8f44c5dc9-jgtzp to ip-10-0-142-185.ec2.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-b-cdfa86-79959f8c88-hcpjf Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-cdfa86-79959f8c88-hcpjf to ip-10-0-142-185.ec2.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-b-bff950-958c95556-hrzxt Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-bff950-958c95556-hrzxt to ip-10-0-142-185.ec2.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-b-bff950-746d89d77f-w4559 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-bff950-746d89d77f-w4559 to ip-10-0-142-185.ec2.internal 67m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-2b1e88-65c766876c-4qz4v Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-2b1e88-65c766876c-4qz4v to ip-10-0-135-132.ec2.internal 43m Normal Scheduled pod/e2e-inf-b-54c7f0-openshift-default-79cd69d455-dznhn Successfully assigned openshift-ingress/e2e-inf-b-54c7f0-openshift-default-79cd69d455-dznhn to ip-10-0-135-132.ec2.internal 29m Normal Scheduled pod/payload-processing-e2e-ipp-b-a60e80-86d69656d4-9r2ls Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-a60e80-86d69656d4-9r2ls to ip-10-0-135-132.ec2.internal 59m Normal Scheduled pod/e2e-ait-913eddcb-openshift-default-546bdfbdc5-ck6mz Successfully assigned openshift-ingress/e2e-ait-913eddcb-openshift-default-546bdfbdc5-ck6mz to ip-10-0-142-185.ec2.internal 64m Normal Scheduled pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-kvtw4 Successfully assigned openshift-ingress/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-kvtw4 to ip-10-0-130-77.ec2.internal 28m Normal Scheduled pod/e2e-ipp-a-f0a807-openshift-default-66f955d779-qv6wr Successfully assigned openshift-ingress/e2e-ipp-a-f0a807-openshift-default-66f955d779-qv6wr to ip-10-0-135-132.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w6-d9f1c2-5bf66857f-zbxhk Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w6-d9f1c2-5bf66857f-zbxhk to ip-10-0-142-185.ec2.internal 61m Normal Scheduled pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-p4vsn Successfully assigned openshift-ingress/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-p4vsn to ip-10-0-130-77.ec2.internal 67m Normal Scheduled pod/payload-processing-e2e-worker-w4-0ad7c5-7b6cc5f5c5-8c2qz Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-0ad7c5-7b6cc5f5c5-8c2qz to ip-10-0-135-132.ec2.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w5-510f80-54b6b8c75-9w995 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w5-510f80-54b6b8c75-9w995 to ip-10-0-142-185.ec2.internal 67m Normal Scheduled pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-tdvdc Successfully assigned openshift-ingress/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-tdvdc to ip-10-0-135-132.ec2.internal 41m Normal Scheduled pod/e2e-inf-b-3acfdb-openshift-default-5b477b885f-xtl6s Successfully assigned openshift-ingress/e2e-inf-b-3acfdb-openshift-default-5b477b885f-xtl6s to ip-10-0-142-185.ec2.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w4-b03b33-cd67bd8f5-dft76 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-b03b33-cd67bd8f5-dft76 to ip-10-0-135-132.ec2.internal 67m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-421583-bfd5dcc6c-7ztzj Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-421583-bfd5dcc6c-7ztzj to ip-10-0-135-132.ec2.internal 63m Normal Scheduled pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-z58t9 Successfully assigned openshift-ingress/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-z58t9 to ip-10-0-142-185.ec2.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w3-e14e00-bdf56dd9f-67qgc Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-e14e00-bdf56dd9f-67qgc to ip-10-0-135-132.ec2.internal 60m Normal Scheduled pod/e2e-ait-ca259734-openshift-default-64cb5dc965-4f6sz Successfully assigned openshift-ingress/e2e-ait-ca259734-openshift-default-64cb5dc965-4f6sz to ip-10-0-135-132.ec2.internal 32m Normal Scheduled pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-p2chp Successfully assigned openshift-ingress/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-p2chp to ip-10-0-142-185.ec2.internal 67m Normal Scheduled pod/payload-processing-e2e-worker-w3-d52c59-f75cff857-zdrn8 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-d52c59-f75cff857-zdrn8 to ip-10-0-135-132.ec2.internal 45m Normal Scheduled pod/e2e-inf-b-3acfdb-openshift-default-5b477b885f-mrqbm Successfully assigned openshift-ingress/e2e-inf-b-3acfdb-openshift-default-5b477b885f-mrqbm to ip-10-0-135-132.ec2.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-3cd77f-799d6dfd99-2zkl9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3cd77f-799d6dfd99-2zkl9 to ip-10-0-142-185.ec2.internal 27m Normal Scheduled pod/e2e-ipp-a-f88979-openshift-default-674df6d98f-tr49g Successfully assigned openshift-ingress/e2e-ipp-a-f88979-openshift-default-674df6d98f-tr49g to ip-10-0-142-185.ec2.internal 67m Normal Scheduled pod/payload-processing-e2e-worker-w2-421583-7f9c569f65-xm89m Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-421583-7f9c569f65-xm89m to ip-10-0-135-132.ec2.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w1-23aab2-ff485dcf-pzd2r Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w1-23aab2-ff485dcf-pzd2r to ip-10-0-135-132.ec2.internal 48m Normal Scheduled pod/e2e-inf-a-d85da4-openshift-default-6b5f89667-d8fjz Successfully assigned openshift-ingress/e2e-inf-a-d85da4-openshift-default-6b5f89667-d8fjz to ip-10-0-135-132.ec2.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w0-6af86e-64944b4ff9-4229b Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-6af86e-64944b4ff9-4229b to ip-10-0-142-185.ec2.internal 54m Normal Scheduled pod/payload-processing-e2e-mt-2949c143-5bdffff8fc-jk5zn Successfully assigned openshift-ingress/payload-processing-e2e-mt-2949c143-5bdffff8fc-jk5zn to ip-10-0-142-185.ec2.internal 31m Normal Scheduled pod/payload-processing-e2e-ipp-b-a60e80-7b8469d759-5hv8m Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-a60e80-7b8469d759-5hv8m to ip-10-0-142-185.ec2.internal 67m Normal Scheduled pod/payload-processing-e2e-worker-w1-2b1e88-6f4479c569-mltjd Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-2b1e88-6f4479c569-mltjd to ip-10-0-135-132.ec2.internal 67m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-d52c59-56d5c877c5-zvrjx Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-d52c59-56d5c877c5-zvrjx to ip-10-0-142-185.ec2.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-b-76d54a-84746b7f6b-br8pt Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-76d54a-84746b7f6b-br8pt to ip-10-0-135-132.ec2.internal 53m Normal Scheduled pod/e2e-api-a-30edd7-openshift-default-69b49fb475-6hzw2 Successfully assigned openshift-ingress/e2e-api-a-30edd7-openshift-default-69b49fb475-6hzw2 to ip-10-0-135-132.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w6-0ff5b0-6f4dc7d5f-pmqnt Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w6-0ff5b0-6f4dc7d5f-pmqnt to ip-10-0-142-185.ec2.internal 50m Normal Scheduled pod/e2e-inf-a-bdd94b-openshift-default-7bbf9bcd4f-q846p Successfully assigned openshift-ingress/e2e-inf-a-bdd94b-openshift-default-7bbf9bcd4f-q846p to ip-10-0-142-185.ec2.internal 65m Normal Scheduled pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-2w4gw Successfully assigned openshift-ingress/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-2w4gw to ip-10-0-142-185.ec2.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w5-6f2eba-dc7c6b979-rb48s Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w5-6f2eba-dc7c6b979-rb48s to ip-10-0-142-185.ec2.internal 25m Normal Scheduled pod/e2e-ipp-a-f94561-openshift-default-66c966685c-zcxmh Successfully assigned openshift-ingress/e2e-ipp-a-f94561-openshift-default-66c966685c-zcxmh to ip-10-0-142-185.ec2.internal 67m Normal Scheduled pod/payload-processing-e2e-worker-w0-09d0d5-85cb95d7ff-n4lzb Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-09d0d5-85cb95d7ff-n4lzb to ip-10-0-142-185.ec2.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w4-c511a3-7c6f5989ff-r6zv2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-c511a3-7c6f5989ff-r6zv2 to ip-10-0-142-185.ec2.internal 8m29s Normal Scheduled pod/payload-processing-e2e-worker-single-main-ef9544-6849c57fcsxzg9 Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-ef9544-6849c57fcsxzg9 to ip-10-0-142-185.ec2.internal 12m Normal Scheduled pod/payload-processing-e2e-worker-single-main-2cad7e-758854df5pq2ws Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-2cad7e-758854df5pq2ws to ip-10-0-142-185.ec2.internal 33m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w3-cdc122-5b697dd967-7qll4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-cdc122-5b697dd967-7qll4 to ip-10-0-135-132.ec2.internal 62m Normal Scheduled pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-5zllw Successfully assigned openshift-ingress/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-5zllw to ip-10-0-135-132.ec2.internal 67m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-0ad7c5-dffdb7775-kt9zp Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-0ad7c5-dffdb7775-kt9zp to ip-10-0-135-132.ec2.internal 58m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-80ecb3-7484c79f79-cndqx Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-80ecb3-7484c79f79-cndqx to ip-10-0-142-185.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-shared-b-w6-d9f1c2-6475ccbbb9-x7xgs Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w6-d9f1c2-6475ccbbb9-x7xgs to ip-10-0-142-185.ec2.internal 25m Normal Scheduled pod/e2e-ipp-b-76d54a-openshift-default-6bcffbc9bc-8d4d2 Successfully assigned openshift-ingress/e2e-ipp-b-76d54a-openshift-default-6bcffbc9bc-8d4d2 to ip-10-0-142-185.ec2.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w1-bed544-6bc95f58bc-xmfrv Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w1-bed544-6bc95f58bc-xmfrv to ip-10-0-142-185.ec2.internal 52m Normal Scheduled pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-2bzhm Successfully assigned openshift-ingress/e2e-api-a-370ec2-openshift-default-6fccc4f697-2bzhm to ip-10-0-130-77.ec2.internal 55m Normal Scheduled pod/payload-processing-e2e-shared-b-w5-510f80-c84b8b845-fpfjm Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w5-510f80-c84b8b845-fpfjm to ip-10-0-142-185.ec2.internal 62m Normal Scheduled pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-zr6tr Successfully assigned openshift-ingress/e2e-worker-w3-d52c59-openshift-default-5466968fb7-zr6tr to ip-10-0-142-185.ec2.internal 58m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w0-5983ea-f76b4bb8c-xvp6k Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5983ea-f76b4bb8c-xvp6k to ip-10-0-142-185.ec2.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-mt-d399fb07-755755d479-wsns4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-d399fb07-755755d479-wsns4 to ip-10-0-135-132.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-mt-2949c143-5f67f57d8c-pw4d8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-2949c143-5f67f57d8c-pw4d8 to ip-10-0-135-132.ec2.internal 58m Normal Scheduled pod/payload-pre-processing-e2e-mt-2949c143-5f67f57d8c-6qcmt Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-2949c143-5f67f57d8c-6qcmt to ip-10-0-142-185.ec2.internal 59m Normal Scheduled pod/payload-pre-processing-e2e-migrate-476e593d-6555654587-lljr8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-476e593d-6555654587-lljr8 to ip-10-0-142-185.ec2.internal 35m Normal Scheduled pod/e2e-inf-a-bc48cc-openshift-default-d9498d65-p62qs Successfully assigned openshift-ingress/e2e-inf-a-bc48cc-openshift-default-d9498d65-p62qs to ip-10-0-142-185.ec2.internal 62m Normal Scheduled pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-7wsq8 Successfully assigned openshift-ingress/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-7wsq8 to ip-10-0-142-185.ec2.internal 53m Normal Scheduled pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-2mvj6 Successfully assigned openshift-ingress/e2e-api-a-370ec2-openshift-default-6fccc4f697-2mvj6 to ip-10-0-135-132.ec2.internal 37m Normal Scheduled pod/payload-processing-e2e-shared-b-w4-b03b33-7c5fcd4d57-g2t4v Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-b03b33-7c5fcd4d57-g2t4v to ip-10-0-135-132.ec2.internal 67m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-54114e-7f6f89654c-sb4mv Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-54114e-7f6f89654c-sb4mv to ip-10-0-135-132.ec2.internal 31m Normal Scheduled pod/e2e-ipp-b-a60e80-openshift-default-c8f96989c-rvvlt Successfully assigned openshift-ingress/e2e-ipp-b-a60e80-openshift-default-c8f96989c-rvvlt to ip-10-0-135-132.ec2.internal 47m Normal Scheduled pod/e2e-inf-a-bc48cc-openshift-default-d9498d65-kbdtw Successfully assigned openshift-ingress/e2e-inf-a-bc48cc-openshift-default-d9498d65-kbdtw to ip-10-0-135-132.ec2.internal 67m Normal Scheduled pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-w2q4t Successfully assigned openshift-ingress/e2e-worker-w3-d52c59-openshift-default-5466968fb7-w2q4t to ip-10-0-142-185.ec2.internal 63m Normal Scheduled pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-c9dkh Successfully assigned openshift-ingress/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-c9dkh to ip-10-0-142-185.ec2.internal 48m Normal Scheduled pod/payload-processing-e2e-inf-b-6013c8-7678cc4db9-mppqg Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-6013c8-7678cc4db9-mppqg to ip-10-0-135-132.ec2.internal 53m Normal Scheduled pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-4cm9t Successfully assigned openshift-ingress/e2e-api-a-370ec2-openshift-default-6fccc4f697-4cm9t to ip-10-0-130-77.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-cdfa86-b49f58999-qjqrs Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-cdfa86-b49f58999-qjqrs to ip-10-0-135-132.ec2.internal 32m Normal Scheduled pod/payload-processing-e2e-shared-b-w3-e14e00-56ff64bff5-ht6tb Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-e14e00-56ff64bff5-ht6tb to ip-10-0-135-132.ec2.internal 62m Normal Scheduled pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-tbj69 Successfully assigned openshift-ingress/e2e-worker-w3-d52c59-openshift-default-5466968fb7-tbj69 to ip-10-0-130-77.ec2.internal 67m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-012ba0-669ff7bfff-8w9bc Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-012ba0-669ff7bfff-8w9bc to ip-10-0-142-185.ec2.internal 22m Normal Scheduled pod/e2e-ipp-b-bff950-openshift-default-5dd888d8fc-46j6j Successfully assigned openshift-ingress/e2e-ipp-b-bff950-openshift-default-5dd888d8fc-46j6j to ip-10-0-135-132.ec2.internal 57m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-3cd77f-6d7cccccf5-gxpwd Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-3cd77f-6d7cccccf5-gxpwd to ip-10-0-142-185.ec2.internal 62m Normal Scheduled pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-ssk97 Successfully assigned openshift-ingress/e2e-worker-w3-d52c59-openshift-default-5466968fb7-ssk97 to ip-10-0-135-132.ec2.internal 34m Normal Scheduled pod/payload-processing-e2e-shared-b-w1-23aab2-67f75c4687-2wq29 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w1-23aab2-67f75c4687-2wq29 to ip-10-0-135-132.ec2.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-cdfa86-5799cbf495-h927d Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-cdfa86-5799cbf495-h927d to ip-10-0-135-132.ec2.internal 55m Normal Scheduled pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-78w6t Successfully assigned openshift-ingress/e2e-api-a-370ec2-openshift-default-6fccc4f697-78w6t to ip-10-0-142-185.ec2.internal 64m Normal Scheduled pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-qtv79 Successfully assigned openshift-ingress/e2e-worker-w3-d52c59-openshift-default-5466968fb7-qtv79 to ip-10-0-142-185.ec2.internal 67m Normal Scheduled pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-kzdls Successfully assigned openshift-ingress/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-kzdls to ip-10-0-135-132.ec2.internal 48m Normal Scheduled pod/e2e-inf-a-999470-openshift-default-6887745cf9-xt2fd Successfully assigned openshift-ingress/e2e-inf-a-999470-openshift-default-6887745cf9-xt2fd to ip-10-0-142-185.ec2.internal 25m Normal Scheduled pod/e2e-ipp-b-cdfa86-openshift-default-77bdfd8797-52tjw Successfully assigned openshift-ingress/e2e-ipp-b-cdfa86-openshift-default-77bdfd8797-52tjw to ip-10-0-135-132.ec2.internal 68m Normal Scheduled pod/payload-processing-55fbbbf9dc-rp6t6 Successfully assigned openshift-ingress/payload-processing-55fbbbf9dc-rp6t6 to ip-10-0-142-185.ec2.internal 57m Normal Scheduled pod/payload-processing-e2e-shared-b-w0-6af86e-8bfcb675f-rgngw Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-6af86e-8bfcb675f-rgngw to ip-10-0-135-132.ec2.internal 64m Normal Scheduled pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-lg99w Successfully assigned openshift-ingress/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-lg99w to ip-10-0-130-77.ec2.internal 53m Normal Scheduled pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-7v8wh Successfully assigned openshift-ingress/e2e-api-a-370ec2-openshift-default-6fccc4f697-7v8wh to ip-10-0-142-185.ec2.internal 63m Normal Scheduled pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-qchs2 Successfully assigned openshift-ingress/e2e-worker-w3-d52c59-openshift-default-5466968fb7-qchs2 to ip-10-0-135-132.ec2.internal 54m Normal Scheduled pod/payload-processing-e2e-shared-a-w6-0ff5b0-7c4b7489d7-xlk52 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w6-0ff5b0-7c4b7489d7-xlk52 to ip-10-0-142-185.ec2.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-bff950-d477c7687-vm8sg Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-bff950-d477c7687-vm8sg to ip-10-0-142-185.ec2.internal 62m Normal Scheduled pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-gt2jk Successfully assigned openshift-ingress/e2e-worker-w3-d52c59-openshift-default-5466968fb7-gt2jk to ip-10-0-130-77.ec2.internal 59m Normal Scheduled pod/e2e-migrate-476e593d-gw-openshift-default-789fdfb6d5-k5k6v Successfully assigned openshift-ingress/e2e-migrate-476e593d-gw-openshift-default-789fdfb6d5-k5k6v to ip-10-0-142-185.ec2.internal 71m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-6c85d847d5kw5cm Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-6c85d847d5kw5cm to ip-10-0-135-132.ec2.internal 5m20s Normal Scheduled pod/payload-processing-e2e-ait-1595eccb-65879f7ddf-dxpjx Successfully assigned openshift-ingress/payload-processing-e2e-ait-1595eccb-65879f7ddf-dxpjx to ip-10-0-142-185.ec2.internal 53m Normal Scheduled pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-9nsmf Successfully assigned openshift-ingress/e2e-api-a-370ec2-openshift-default-6fccc4f697-9nsmf to ip-10-0-130-77.ec2.internal 60m Normal Scheduled pod/payload-processing-e2e-ait-1ba9ae08-5fdd496c5c-4sz29 Successfully assigned openshift-ingress/payload-processing-e2e-ait-1ba9ae08-5fdd496c5c-4sz29 to ip-10-0-142-185.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-b-76d54a-5c57cb5785-lvdp9 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-76d54a-5c57cb5785-lvdp9 to ip-10-0-142-185.ec2.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-a-f94561-b76dcb6f6-jvts2 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-f94561-b76dcb6f6-jvts2 to ip-10-0-142-185.ec2.internal 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-bff950-67c96cc479-ll5pt Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-bff950-67c96cc479-ll5pt to ip-10-0-142-185.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-a-f94561-5ddf9d6659-rntqt Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-f94561-5ddf9d6659-rntqt to ip-10-0-142-185.ec2.internal 58m Normal Scheduled pod/payload-processing-e2e-mt-2949c143-5bdffff8fc-jm8q7 Successfully assigned openshift-ingress/payload-processing-e2e-mt-2949c143-5bdffff8fc-jm8q7 to ip-10-0-142-185.ec2.internal 28m Normal Scheduled pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-bmccc Successfully assigned openshift-ingress/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-bmccc to ip-10-0-130-77.ec2.internal 37m Normal Scheduled pod/e2e-shared-b-w4-b03b33-openshift-default-79f6954695-882qd Successfully assigned openshift-ingress/e2e-shared-b-w4-b03b33-openshift-default-79f6954695-882qd to ip-10-0-142-185.ec2.internal 27m Normal Scheduled pod/payload-processing-e2e-ipp-a-f88979-7dd56f4c95-jnwkh Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-f88979-7dd56f4c95-jnwkh to ip-10-0-142-185.ec2.internal 53m Normal Scheduled pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-j564x Successfully assigned openshift-ingress/e2e-api-a-370ec2-openshift-default-6fccc4f697-j564x to ip-10-0-135-132.ec2.internal 59m Normal Scheduled pod/payload-processing-e2e-ait-913eddcb-d7477fc45-nd68z Successfully assigned openshift-ingress/payload-processing-e2e-ait-913eddcb-d7477fc45-nd68z to ip-10-0-142-185.ec2.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-a60e80-5db87c97f9-7rjpw Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-a60e80-5db87c97f9-7rjpw to ip-10-0-135-132.ec2.internal 37m Normal Scheduled pod/e2e-inf-b-867f12-openshift-default-6584f84965-znd9z Successfully assigned openshift-ingress/e2e-inf-b-867f12-openshift-default-6584f84965-znd9z to ip-10-0-130-77.ec2.internal 60m Normal Scheduled pod/payload-processing-e2e-ait-ca259734-6d97f789c5-dfxt8 Successfully assigned openshift-ingress/payload-processing-e2e-ait-ca259734-6d97f789c5-dfxt8 to ip-10-0-130-77.ec2.internal 31m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-a60e80-5949b5b66c-h6x68 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-a60e80-5949b5b66c-h6x68 to ip-10-0-142-185.ec2.internal 25m Normal Scheduled pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-4qscf Successfully assigned openshift-ingress/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-4qscf to ip-10-0-142-185.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-a-f88979-74fd6f8bd-vfz9j Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-f88979-74fd6f8bd-vfz9j to ip-10-0-135-132.ec2.internal 52m Normal Scheduled pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-mhwhp Successfully assigned openshift-ingress/e2e-api-a-370ec2-openshift-default-6fccc4f697-mhwhp to ip-10-0-142-185.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-a-f0a807-69fbbcbd9d-9r72m Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-f0a807-69fbbcbd9d-9r72m to ip-10-0-142-185.ec2.internal 27m Normal Scheduled pod/payload-processing-e2e-ipp-a-f0a807-6968cf6bcc-xbgf8 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-f0a807-6968cf6bcc-xbgf8 to ip-10-0-142-185.ec2.internal 30m Normal Scheduled pod/payload-processing-e2e-ipp-a-e3a7dd-6b6b8b4cfd-9gg8g Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-e3a7dd-6b6b8b4cfd-9gg8g to ip-10-0-130-77.ec2.internal 55m Normal Scheduled pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-wjvgx Successfully assigned openshift-ingress/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-wjvgx to ip-10-0-135-132.ec2.internal 54m Normal Scheduled pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-zqkns Successfully assigned openshift-ingress/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-zqkns to ip-10-0-142-185.ec2.internal 32m Normal Scheduled pod/e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447-hs292 Successfully assigned openshift-ingress/e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447-hs292 to ip-10-0-142-185.ec2.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-76d54a-fcfc75c9f-lvgx2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-76d54a-fcfc75c9f-lvgx2 to ip-10-0-135-132.ec2.internal 53m Normal Scheduled pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-w7p95 Successfully assigned openshift-ingress/e2e-api-a-370ec2-openshift-default-6fccc4f697-w7p95 to ip-10-0-142-185.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-api-a-30edd7-57c7586d97-zmlbs Successfully assigned openshift-ingress/payload-processing-e2e-api-a-30edd7-57c7586d97-zmlbs to ip-10-0-135-132.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-76d54a-d97b6f8b7-p8h4c Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-76d54a-d97b6f8b7-p8h4c to ip-10-0-142-185.ec2.internal 54m Normal Scheduled pod/payload-processing-e2e-api-a-370ec2-7658c775d7-t69wq Successfully assigned openshift-ingress/payload-processing-e2e-api-a-370ec2-7658c775d7-t69wq to ip-10-0-142-185.ec2.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-f94561-66b74889b7-j6lk5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-f94561-66b74889b7-j6lk5 to ip-10-0-135-132.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-api-a-7a1ee7-7dbbd77679-kbzmg Successfully assigned openshift-ingress/payload-processing-e2e-api-a-7a1ee7-7dbbd77679-kbzmg to ip-10-0-142-185.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-f94561-5c684bc9-8ccfn Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-f94561-5c684bc9-8ccfn to ip-10-0-135-132.ec2.internal 54m Normal Scheduled pod/payload-processing-e2e-api-a-de31dd-7d9cf5d4f-9lr2z Successfully assigned openshift-ingress/payload-processing-e2e-api-a-de31dd-7d9cf5d4f-9lr2z to ip-10-0-135-132.ec2.internal 53m Normal Scheduled pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-xwkss Successfully assigned openshift-ingress/e2e-api-a-370ec2-openshift-default-6fccc4f697-xwkss to ip-10-0-135-132.ec2.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-f88979-844f98df9f-d7zsv Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-f88979-844f98df9f-d7zsv to ip-10-0-142-185.ec2.internal 54m Normal Scheduled pod/payload-processing-e2e-api-a-f443d6-784b88c549-cwkhm Successfully assigned openshift-ingress/payload-processing-e2e-api-a-f443d6-784b88c549-cwkhm to ip-10-0-135-132.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-f88979-669bd98b7f-9lsrp Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-f88979-669bd98b7f-9lsrp to ip-10-0-135-132.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-api-b-3d38f6-5874cbccb9-4r8w9 Successfully assigned openshift-ingress/payload-processing-e2e-api-b-3d38f6-5874cbccb9-4r8w9 to ip-10-0-135-132.ec2.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-f0a807-78f9946d59-tjxws Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-f0a807-78f9946d59-tjxws to ip-10-0-135-132.ec2.internal 52m Normal Scheduled pod/payload-processing-e2e-api-b-4f7439-db569fdbc-lx8l8 Successfully assigned openshift-ingress/payload-processing-e2e-api-b-4f7439-db569fdbc-lx8l8 to ip-10-0-135-132.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-f0a807-6b45d8f8f9-x5x2n Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-f0a807-6b45d8f8f9-x5x2n to ip-10-0-142-185.ec2.internal 51m Normal Scheduled pod/payload-processing-e2e-api-b-62236a-754c5cbbb9-sxrtf Successfully assigned openshift-ingress/payload-processing-e2e-api-b-62236a-754c5cbbb9-sxrtf to ip-10-0-142-185.ec2.internal 30m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-e3a7dd-fc4fb5599-mg28w Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-e3a7dd-fc4fb5599-mg28w to ip-10-0-130-77.ec2.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-e3a7dd-55c66888c9-fbrpc Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-e3a7dd-55c66888c9-fbrpc to ip-10-0-142-185.ec2.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-f9d87b-d46894c99-vcmjs Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-f9d87b-d46894c99-vcmjs to ip-10-0-135-132.ec2.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-f0bb45-64bb8c9b67-2dxq4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-f0bb45-64bb8c9b67-2dxq4 to ip-10-0-142-185.ec2.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-c13db0-bcdbd4c9f-tqwjn Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-c13db0-bcdbd4c9f-tqwjn to ip-10-0-135-132.ec2.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-867f12-f59b9bd47-xkz7q Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-867f12-f59b9bd47-xkz7q to ip-10-0-135-132.ec2.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-6013c8-94b84fcd7-b54fw Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-6013c8-94b84fcd7-b54fw to ip-10-0-135-132.ec2.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-54c7f0-545cb4dc95-xpm4m Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-54c7f0-545cb4dc95-xpm4m to ip-10-0-135-132.ec2.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-3acfdb-66c888cfdc-ktrkf Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-3acfdb-66c888cfdc-ktrkf to ip-10-0-142-185.ec2.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-d85da4-7bdd674455-fjhgp Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-d85da4-7bdd674455-fjhgp to ip-10-0-142-185.ec2.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-bdd94b-74d9b684bc-m4kfl Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-bdd94b-74d9b684bc-m4kfl to ip-10-0-135-132.ec2.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-bc48cc-5554659f79-79xg7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-bc48cc-5554659f79-79xg7 to ip-10-0-135-132.ec2.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-999470-7757847d95-dfvnm Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-999470-7757847d95-dfvnm to ip-10-0-135-132.ec2.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-6e0edc-6cb4f7d887-wn55s Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-6e0edc-6cb4f7d887-wn55s to ip-10-0-135-132.ec2.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-6922d3-69c86dfc75-ktwbq Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-6922d3-69c86dfc75-ktwbq to ip-10-0-142-185.ec2.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-40229e-5fb694f79-5glps Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-40229e-5fb694f79-5glps to ip-10-0-135-132.ec2.internal 53m Normal Scheduled pod/e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447-l89ql Successfully assigned openshift-ingress/e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447-l89ql to ip-10-0-135-132.ec2.internal 57m Normal Scheduled pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-52wf7 Successfully assigned openshift-ingress/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-52wf7 to ip-10-0-135-132.ec2.internal 32m Normal Scheduled pod/payload-processing-e2e-ipp-a-e3a7dd-666fdd8fcc-qvds2 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-e3a7dd-666fdd8fcc-qvds2 to ip-10-0-135-132.ec2.internal 12m Normal Scheduled pod/e2e-worker-single-main-2cad7e-openshift-default-6465775477q6cwj Successfully assigned openshift-ingress/e2e-worker-single-main-2cad7e-openshift-default-6465775477q6cwj to ip-10-0-142-185.ec2.internal 49m Normal Scheduled pod/payload-processing-e2e-inf-b-f9d87b-66796c8fff-5mhk5 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-f9d87b-66796c8fff-5mhk5 to ip-10-0-142-185.ec2.internal 53m Normal Scheduled pod/e2e-api-a-7a1ee7-openshift-default-68c5c46ccc-5gx2f Successfully assigned openshift-ingress/e2e-api-a-7a1ee7-openshift-default-68c5c46ccc-5gx2f to ip-10-0-135-132.ec2.internal 46m Normal Scheduled pod/payload-processing-e2e-inf-b-f0bb45-5f54c4488f-hdmsx Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-f0bb45-5f54c4488f-hdmsx to ip-10-0-142-185.ec2.internal 8m35s Normal Scheduled pod/e2e-worker-single-main-ef9544-openshift-default-68f86596d5hj74l Successfully assigned openshift-ingress/e2e-worker-single-main-ef9544-openshift-default-68f86596d5hj74l to ip-10-0-142-185.ec2.internal 60m Normal Scheduled pod/payload-pre-processing-e2e-derive-36d7ee39-69869b5bc-7jmws Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-36d7ee39-69869b5bc-7jmws to ip-10-0-135-132.ec2.internal 52m Normal Scheduled pod/payload-processing-e2e-api-b-8b3e43-685659576c-mxc2x Successfully assigned openshift-ingress/payload-processing-e2e-api-b-8b3e43-685659576c-mxc2x to ip-10-0-142-185.ec2.internal 4m42s Normal Scheduled pod/payload-pre-processing-e2e-authcfg-http-9cdd18-7f54b6fc9f-9sg44 Successfully assigned openshift-ingress/payload-pre-processing-e2e-authcfg-http-9cdd18-7f54b6fc9f-9sg44 to ip-10-0-142-185.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-api-b-bf9c2f-69f6d78d99-b2xh6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-bf9c2f-69f6d78d99-b2xh6 to ip-10-0-142-185.ec2.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-api-b-8b3e43-57f97b8bcf-x88zv Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-8b3e43-57f97b8bcf-x88zv to ip-10-0-142-185.ec2.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-api-b-62236a-56cfbd57f9-ldg6l Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-62236a-56cfbd57f9-ldg6l to ip-10-0-142-185.ec2.internal 52m Normal Scheduled pod/payload-pre-processing-e2e-api-b-4f7439-55564f8b9f-xttgk Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-4f7439-55564f8b9f-xttgk to ip-10-0-135-132.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-api-b-3d38f6-8fdb5988c-rjl42 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-3d38f6-8fdb5988c-rjl42 to ip-10-0-135-132.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-api-a-f443d6-6bcbc5c45f-72rp5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-f443d6-6bcbc5c45f-72rp5 to ip-10-0-135-132.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-api-a-de31dd-7579cc4f79-rq4dm Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-de31dd-7579cc4f79-rq4dm to ip-10-0-142-185.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-api-a-7a1ee7-55c68b95cf-s428w Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-7a1ee7-55c68b95cf-s428w to ip-10-0-142-185.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-e2e-api-a-370ec2-7957558899-vkjqx Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-370ec2-7957558899-vkjqx to ip-10-0-142-185.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-api-a-30edd7-5699985987-rczc8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-30edd7-5699985987-rczc8 to ip-10-0-135-132.ec2.internal 46m Normal Scheduled pod/payload-processing-e2e-inf-b-c13db0-5dbfdd689-q5zt5 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-c13db0-5dbfdd689-q5zt5 to ip-10-0-142-185.ec2.internal 54m Normal Scheduled pod/e2e-api-a-de31dd-openshift-default-56f5bd455-d8vcq Successfully assigned openshift-ingress/e2e-api-a-de31dd-openshift-default-56f5bd455-d8vcq to ip-10-0-142-185.ec2.internal 45m Normal Scheduled pod/payload-processing-e2e-inf-b-867f12-66d87c857f-vsjkv Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-867f12-66d87c857f-vsjkv to ip-10-0-142-185.ec2.internal 65m Normal Scheduled pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-2t4b8 Successfully assigned openshift-ingress/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-2t4b8 to ip-10-0-142-185.ec2.internal 59m Normal Scheduled pod/payload-processing-e2e-migrate-476e593d-796fb9bdc9-sq29f Successfully assigned openshift-ingress/payload-processing-e2e-migrate-476e593d-796fb9bdc9-sq29f to ip-10-0-142-185.ec2.internal 62m Normal Scheduled pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-m74tl Successfully assigned openshift-ingress/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-m74tl to ip-10-0-130-77.ec2.internal 58m Normal Scheduled pod/e2e-mt-2949c143-openshift-default-cc8cc894f-jnvwn Successfully assigned openshift-ingress/e2e-mt-2949c143-openshift-default-cc8cc894f-jnvwn to ip-10-0-142-185.ec2.internal 65m Normal Scheduled pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-67nxn Successfully assigned openshift-ingress/e2e-worker-w3-d52c59-openshift-default-5466968fb7-67nxn to ip-10-0-135-132.ec2.internal 39m Normal Scheduled pod/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-q7w2l Successfully assigned openshift-ingress/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-q7w2l to ip-10-0-130-77.ec2.internal 60m Normal Scheduled pod/payload-pre-processing-e2e-ait-ca259734-86c66dfcf9-5dzzk Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-ca259734-86c66dfcf9-5dzzk to ip-10-0-130-77.ec2.internal 59m Normal Scheduled pod/payload-pre-processing-e2e-ait-913eddcb-865ff6c5f5-lc242 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-913eddcb-865ff6c5f5-lc242 to ip-10-0-142-185.ec2.internal 61m Normal Scheduled pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-n9dj2 Successfully assigned openshift-ingress/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-n9dj2 to ip-10-0-142-185.ec2.internal 63m Normal Scheduled pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-bhn2k Successfully assigned openshift-ingress/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-bhn2k to ip-10-0-142-185.ec2.internal 34m Normal Scheduled pod/e2e-shared-b-w1-23aab2-openshift-default-86c4fcc59c-qvpxp Successfully assigned openshift-ingress/e2e-shared-b-w1-23aab2-openshift-default-86c4fcc59c-qvpxp to ip-10-0-135-132.ec2.internal 62m Normal Scheduled pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-cnmb7 Successfully assigned openshift-ingress/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-cnmb7 to ip-10-0-135-132.ec2.internal 63m Normal Scheduled pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-d5lpp Successfully assigned openshift-ingress/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-d5lpp to ip-10-0-135-132.ec2.internal 57m Normal Scheduled pod/e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c-ztnmj Successfully assigned openshift-ingress/e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c-ztnmj to ip-10-0-142-185.ec2.internal 62m Normal Scheduled pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-m6s8k Successfully assigned openshift-ingress/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-m6s8k to ip-10-0-142-185.ec2.internal 54m Normal Scheduled pod/e2e-api-a-f443d6-openshift-default-6dc8b74cd7-nkzfv Successfully assigned openshift-ingress/e2e-api-a-f443d6-openshift-default-6dc8b74cd7-nkzfv to ip-10-0-135-132.ec2.internal 60m Normal Scheduled pod/payload-pre-processing-e2e-ait-1ba9ae08-6847f996c5-n5b48 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-1ba9ae08-6847f996c5-n5b48 to ip-10-0-142-185.ec2.internal 5m20s Normal Scheduled pod/payload-pre-processing-e2e-ait-1595eccb-589744bfdc-m69n7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-1595eccb-589744bfdc-m69n7 to ip-10-0-142-185.ec2.internal 55m Normal Scheduled pod/e2e-mt-d399fb07-openshift-default-6d54788969-47qsn Successfully assigned openshift-ingress/e2e-mt-d399fb07-openshift-default-6d54788969-47qsn to ip-10-0-130-77.ec2.internal 41m Normal Scheduled pod/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-mthln Successfully assigned openshift-ingress/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-mthln to ip-10-0-142-185.ec2.internal 61m Normal Scheduled pod/e2e-worker-w2-421583-openshift-default-5cb767b447-xkmrr Successfully assigned openshift-ingress/e2e-worker-w2-421583-openshift-default-5cb767b447-xkmrr to ip-10-0-142-185.ec2.internal 55m Normal Scheduled pod/e2e-mt-d399fb07-openshift-default-6d54788969-k9sr9 Successfully assigned openshift-ingress/e2e-mt-d399fb07-openshift-default-6d54788969-k9sr9 to ip-10-0-142-185.ec2.internal 62m Normal Scheduled pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-sqxvr Successfully assigned openshift-ingress/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-sqxvr to ip-10-0-130-77.ec2.internal 62m Normal Scheduled pod/e2e-worker-w2-421583-openshift-default-5cb767b447-spjhs Successfully assigned openshift-ingress/e2e-worker-w2-421583-openshift-default-5cb767b447-spjhs to ip-10-0-130-77.ec2.internal 47m Normal Scheduled pod/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-86ztv Successfully assigned openshift-ingress/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-86ztv to ip-10-0-135-132.ec2.internal 68m Normal Scheduled pod/payload-pre-processing-7b98bb678c-s6mwr Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-s6mwr to ip-10-0-135-132.ec2.internal 61m Normal Scheduled pod/e2e-worker-w2-421583-openshift-default-5cb767b447-l4c4h Successfully assigned openshift-ingress/e2e-worker-w2-421583-openshift-default-5cb767b447-l4c4h to ip-10-0-135-132.ec2.internal 58m Normal Scheduled pod/e2e-shared-a-w0-5983ea-openshift-default-849df9ccdf-bbqjg Successfully assigned openshift-ingress/e2e-shared-a-w0-5983ea-openshift-default-849df9ccdf-bbqjg to ip-10-0-142-185.ec2.internal 64m Normal Scheduled pod/e2e-worker-w2-421583-openshift-default-5cb767b447-k94dp Successfully assigned openshift-ingress/e2e-worker-w2-421583-openshift-default-5cb767b447-k94dp to ip-10-0-142-185.ec2.internal 63m Normal Scheduled pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-v5lkq Successfully assigned openshift-ingress/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-v5lkq to ip-10-0-135-132.ec2.internal 31m Normal Scheduled pod/e2e-shared-a-w0-5983ea-openshift-default-849df9ccdf-ffzk6 Successfully assigned openshift-ingress/e2e-shared-a-w0-5983ea-openshift-default-849df9ccdf-ffzk6 to ip-10-0-135-132.ec2.internal 56m Normal Scheduled pod/payload-processing-e2e-shared-a-w5-6f2eba-5576597779-g8wwj Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w5-6f2eba-5576597779-g8wwj to ip-10-0-135-132.ec2.internal 53m Normal Scheduled pod/e2e-api-b-3d38f6-openshift-default-79b69d9955-wjvpp Successfully assigned openshift-ingress/e2e-api-b-3d38f6-openshift-default-79b69d9955-wjvpp to ip-10-0-142-185.ec2.internal 67m Normal Scheduled pod/e2e-worker-w2-421583-openshift-default-5cb767b447-k4nm5 Successfully assigned openshift-ingress/e2e-worker-w2-421583-openshift-default-5cb767b447-k4nm5 to ip-10-0-135-132.ec2.internal 39m Normal Scheduled pod/payload-processing-e2e-shared-a-w4-c511a3-5ddf74769-fztjg Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-c511a3-5ddf74769-fztjg to ip-10-0-142-185.ec2.internal 35m Normal Scheduled pod/e2e-shared-a-w1-bed544-openshift-default-8659947965-dmmj5 Successfully assigned openshift-ingress/e2e-shared-a-w1-bed544-openshift-default-8659947965-dmmj5 to ip-10-0-135-132.ec2.internal 58m Normal Scheduled pod/e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949-2xr8n Successfully assigned openshift-ingress/e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949-2xr8n to ip-10-0-142-185.ec2.internal 27m Normal Scheduled pod/e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949-dr69x Successfully assigned openshift-ingress/e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949-dr69x to ip-10-0-135-132.ec2.internal 64m Normal Scheduled pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-svsgv Successfully assigned openshift-ingress/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-svsgv to ip-10-0-135-132.ec2.internal 47m Normal Scheduled pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-x68sw Successfully assigned openshift-ingress/e2e-inf-a-6922d3-openshift-default-689b5f95bc-x68sw to ip-10-0-130-77.ec2.internal 73m Normal Scheduled pod/openshift-ai-inference-openshift-ai-inference-69694c4c9-scp7q Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-69694c4c9-scp7q to ip-10-0-135-132.ec2.internal 60m Normal Scheduled pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-nflrt Successfully assigned openshift-ingress/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-nflrt to ip-10-0-142-185.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-api-b-bf9c2f-59899c7df-58lpq Successfully assigned openshift-ingress/payload-processing-e2e-api-b-bf9c2f-59899c7df-58lpq to ip-10-0-142-185.ec2.internal 32m Normal Scheduled pod/e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c-2xvj7 Successfully assigned openshift-ingress/e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c-2xvj7 to ip-10-0-135-132.ec2.internal 57m Normal Scheduled pod/payload-processing-e2e-mt-d399fb07-56bfdc9c75-7d8qb Successfully assigned openshift-ingress/payload-processing-e2e-mt-d399fb07-56bfdc9c75-7d8qb to ip-10-0-142-185.ec2.internal 63m Normal Scheduled pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-pmswk Successfully assigned openshift-ingress/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-pmswk to ip-10-0-130-77.ec2.internal 8m29s Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-ef9544-69bf6pfp67 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-ef9544-69bf6pfp67 to ip-10-0-142-185.ec2.internal 33m Normal Scheduled pod/payload-processing-e2e-shared-a-w3-cdc122-5584f9d75c-fljm5 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-cdc122-5584f9d75c-fljm5 to ip-10-0-142-185.ec2.internal 60m Normal Scheduled pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-n6fb8 Successfully assigned openshift-ingress/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-n6fb8 to ip-10-0-130-77.ec2.internal 62m Normal Scheduled pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-mb6lc Successfully assigned openshift-ingress/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-mb6lc to ip-10-0-135-132.ec2.internal 49m Normal Scheduled pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-txq8f Successfully assigned openshift-ingress/e2e-inf-a-6922d3-openshift-default-689b5f95bc-txq8f to ip-10-0-130-77.ec2.internal 52m Normal Scheduled pod/e2e-api-b-4f7439-openshift-default-84f666696f-7wlcj Successfully assigned openshift-ingress/e2e-api-b-4f7439-openshift-default-84f666696f-7wlcj to ip-10-0-142-185.ec2.internal 58m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-80ecb3-67699fb85f-7tctg Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-80ecb3-67699fb85f-7tctg to ip-10-0-135-132.ec2.internal 73m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-dlh79 Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-dlh79 to ip-10-0-142-185.ec2.internal 49m Normal Scheduled pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-tjbw5 Successfully assigned openshift-ingress/e2e-inf-a-6922d3-openshift-default-689b5f95bc-tjbw5 to ip-10-0-135-132.ec2.internal 4m43s Normal Scheduled pod/payload-processing-e2e-authcfg-http-9cdd18-664b9ddfc-56b6l Successfully assigned openshift-ingress/payload-processing-e2e-authcfg-http-9cdd18-664b9ddfc-56b6l to ip-10-0-142-185.ec2.internal 60m Normal Scheduled pod/payload-processing-e2e-derive-36d7ee39-5cf5d67547-bbzkn Successfully assigned openshift-ingress/payload-processing-e2e-derive-36d7ee39-5cf5d67547-bbzkn to ip-10-0-142-185.ec2.internal 67m Normal Scheduled pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-qhw2n Successfully assigned openshift-ingress/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-qhw2n to ip-10-0-135-132.ec2.internal 60m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-47qt2 Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-47qt2 to ip-10-0-135-132.ec2.internal 70m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-c926c Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-c926c to ip-10-0-142-185.ec2.internal 47m Normal Scheduled pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-q5795 Successfully assigned openshift-ingress/e2e-inf-a-6922d3-openshift-default-689b5f95bc-q5795 to ip-10-0-142-185.ec2.internal 60m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-wx4cl Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wx4cl to ip-10-0-130-77.ec2.internal 47m Normal Scheduled pod/payload-processing-e2e-inf-b-54c7f0-5448fb4d5c-nrrpn Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-54c7f0-5448fb4d5c-nrrpn to ip-10-0-135-132.ec2.internal 47m Normal Scheduled pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-df6cw Successfully assigned openshift-ingress/e2e-inf-a-6922d3-openshift-default-689b5f95bc-df6cw to ip-10-0-135-132.ec2.internal 71m Normal Scheduled pod/kube-auth-proxy-6f865f994d-9lf4d Successfully assigned openshift-ingress/kube-auth-proxy-6f865f994d-9lf4d to ip-10-0-142-185.ec2.internal 71m Normal Scheduled pod/kube-auth-proxy-6f865f994d-vg2nd Successfully assigned openshift-ingress/kube-auth-proxy-6f865f994d-vg2nd to ip-10-0-130-77.ec2.internal 51m Normal Scheduled pod/e2e-api-b-62236a-openshift-default-899986fcf-8kdtl Successfully assigned openshift-ingress/e2e-api-b-62236a-openshift-default-899986fcf-8kdtl to ip-10-0-135-132.ec2.internal 49m Normal Scheduled pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-dbxb6 Successfully assigned openshift-ingress/e2e-inf-a-6922d3-openshift-default-689b5f95bc-dbxb6 to ip-10-0-142-185.ec2.internal 51m Normal Scheduled pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-bf77z Successfully assigned openshift-ingress/e2e-inf-a-6922d3-openshift-default-689b5f95bc-bf77z to ip-10-0-142-185.ec2.internal 64m Normal Scheduled pod/maas-default-gateway-openshift-default-6db586dd6c-24vbj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-24vbj to ip-10-0-135-132.ec2.internal 32m Normal Scheduled pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-6bvgh Successfully assigned openshift-ingress/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-6bvgh to ip-10-0-130-77.ec2.internal 62m Normal Scheduled pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-vtwp6 Successfully assigned openshift-ingress/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-vtwp6 to ip-10-0-130-77.ec2.internal 39m Normal Scheduled pod/e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5-tz9sg Successfully assigned openshift-ingress/e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5-tz9sg to ip-10-0-135-132.ec2.internal 54m Normal Scheduled pod/e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5-6m98h Successfully assigned openshift-ingress/e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5-6m98h to ip-10-0-142-185.ec2.internal 62m Normal Scheduled pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-z7ggc Successfully assigned openshift-ingress/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-z7ggc to ip-10-0-130-77.ec2.internal 62m Normal Scheduled pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-zb2qv Successfully assigned openshift-ingress/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-zb2qv to ip-10-0-142-185.ec2.internal 57m Normal Scheduled pod/e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c-krxtr Successfully assigned openshift-ingress/e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c-krxtr to ip-10-0-135-132.ec2.internal 64m Normal Scheduled pod/maas-default-gateway-openshift-default-6db586dd6c-vbv8b Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-vbv8b to ip-10-0-135-132.ec2.internal 67m Normal Scheduled pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-dxslt Successfully assigned openshift-ingress/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-dxslt to ip-10-0-142-185.ec2.internal 32m Normal Scheduled pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-bmvsb Successfully assigned openshift-ingress/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-bmvsb to ip-10-0-135-132.ec2.internal 34m Normal Scheduled pod/e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c-9ps8d Successfully assigned openshift-ingress/e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c-9ps8d to ip-10-0-142-185.ec2.internal 58m Normal Scheduled pod/payload-processing-e2e-shared-a-w0-5983ea-65fbff47dc-kjmcc Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-5983ea-65fbff47dc-kjmcc to ip-10-0-142-185.ec2.internal 47m Normal Scheduled pod/payload-processing-e2e-inf-a-40229e-86586c78fc-288bj Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-40229e-86586c78fc-288bj to ip-10-0-142-185.ec2.internal 52m Normal Scheduled pod/e2e-api-b-8b3e43-openshift-default-78bccfc7fc-n448p Successfully assigned openshift-ingress/e2e-api-b-8b3e43-openshift-default-78bccfc7fc-n448p to ip-10-0-142-185.ec2.internal 17m Normal Scheduled pod/maas-default-gateway-openshift-default-6db586dd6c-v8xjj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-v8xjj to ip-10-0-142-185.ec2.internal 51m Normal Scheduled pod/payload-processing-e2e-inf-a-6922d3-59cf97fb57-v5z2m Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-6922d3-59cf97fb57-v5z2m to ip-10-0-135-132.ec2.internal 16m Normal Scheduled pod/maas-default-gateway-openshift-default-6db586dd6c-tbk5w Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-tbk5w to ip-10-0-135-132.ec2.internal 61m Normal Scheduled pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-crp67 Successfully assigned openshift-ingress/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-crp67 to ip-10-0-130-77.ec2.internal 49m Normal Scheduled pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-7bhqj Successfully assigned openshift-ingress/e2e-inf-a-6922d3-openshift-default-689b5f95bc-7bhqj to ip-10-0-130-77.ec2.internal 16m Normal Scheduled pod/maas-default-gateway-openshift-default-6db586dd6c-7qjgv Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7qjgv to ip-10-0-142-185.ec2.internal 45m Normal Scheduled pod/payload-processing-e2e-inf-b-3acfdb-7f8bb6c86f-7bsb5 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-3acfdb-7f8bb6c86f-7bsb5 to ip-10-0-142-185.ec2.internal 64m Normal Scheduled pod/maas-default-gateway-openshift-default-6db586dd6c-8c8cz Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-8c8cz to ip-10-0-142-185.ec2.internal 32m Normal Scheduled pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-wsd2x Successfully assigned openshift-ingress/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-wsd2x to ip-10-0-142-185.ec2.internal 68m Normal Scheduled pod/maas-default-gateway-openshift-default-6db586dd6c-sv4fp Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-sv4fp to ip-10-0-135-132.ec2.internal 63m Normal Scheduled pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-cjkc9 Successfully assigned openshift-ingress/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-cjkc9 to ip-10-0-130-77.ec2.internal 60m Normal Scheduled pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-6d9p2 Successfully assigned openshift-ingress/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-6d9p2 to ip-10-0-135-132.ec2.internal 34m Normal Scheduled pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-x4lb9 Successfully assigned openshift-ingress/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-x4lb9 to ip-10-0-135-132.ec2.internal 61m Normal Scheduled pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-58wkt Successfully assigned openshift-ingress/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-58wkt to ip-10-0-142-185.ec2.internal 35m Normal Scheduled pod/payload-processing-e2e-shared-a-w1-bed544-5b5dc64c47-5dwdv Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w1-bed544-5b5dc64c47-5dwdv to ip-10-0-142-185.ec2.internal 49m Normal Scheduled pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-6t544 Successfully assigned openshift-ingress/e2e-inf-a-6922d3-openshift-default-689b5f95bc-6t544 to ip-10-0-135-132.ec2.internal 53m Normal Scheduled pod/e2e-api-b-bf9c2f-openshift-default-7c859cb8dc-d962b Successfully assigned openshift-ingress/e2e-api-b-bf9c2f-openshift-default-7c859cb8dc-d962b to ip-10-0-135-132.ec2.internal 68m Normal Scheduled pod/maas-default-gateway-openshift-default-6db586dd6c-bjfxc Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-bjfxc to ip-10-0-130-77.ec2.internal 68m Normal Scheduled pod/maas-default-gateway-openshift-default-6db586dd6c-p65ln Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-p65ln to ip-10-0-142-185.ec2.internal 46m Normal Scheduled pod/payload-processing-e2e-inf-a-6e0edc-c7974cff5-rr5v8 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-6e0edc-c7974cff5-rr5v8 to ip-10-0-135-132.ec2.internal 17m Normal Scheduled pod/maas-default-gateway-openshift-default-6db586dd6c-p2flk Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-p2flk to ip-10-0-142-185.ec2.internal 68m Normal Scheduled pod/maas-default-gateway-openshift-default-6db586dd6c-cmklv Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cmklv to ip-10-0-142-185.ec2.internal 16m Normal Scheduled pod/maas-default-gateway-openshift-default-6db586dd6c-d8sgs Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-d8sgs to ip-10-0-130-77.ec2.internal 48m Normal Scheduled pod/payload-processing-e2e-inf-a-d85da4-7d5f448d7c-cwh99 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-d85da4-7d5f448d7c-cwh99 to ip-10-0-142-185.ec2.internal 68m Normal Scheduled pod/maas-default-gateway-openshift-default-6db586dd6c-frdpl Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-frdpl to ip-10-0-130-77.ec2.internal 48m Normal Scheduled pod/e2e-inf-a-40229e-openshift-default-5f54d778f5-zhln9 Successfully assigned openshift-ingress/e2e-inf-a-40229e-openshift-default-5f54d778f5-zhln9 to ip-10-0-135-132.ec2.internal 40m Normal Scheduled pod/e2e-inf-a-40229e-openshift-default-5f54d778f5-n4d9v Successfully assigned openshift-ingress/e2e-inf-a-40229e-openshift-default-5f54d778f5-n4d9v to ip-10-0-142-185.ec2.internal 61m Normal Scheduled pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-47dz7 Successfully assigned openshift-ingress/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-47dz7 to ip-10-0-135-132.ec2.internal 16m Normal Scheduled pod/maas-default-gateway-openshift-default-6db586dd6c-h8mv5 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-h8mv5 to ip-10-0-135-132.ec2.internal 73m Normal Scheduled pod/maas-default-gateway-openshift-default-6db586dd6c-lnh72 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-lnh72 to ip-10-0-135-132.ec2.internal 48m Normal Scheduled pod/payload-processing-e2e-inf-a-999470-99979b95c-2x956 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-999470-99979b95c-2x956 to ip-10-0-135-132.ec2.internal 16m Normal Scheduled pod/maas-default-gateway-openshift-default-6db586dd6c-jp9lg Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jp9lg to ip-10-0-142-185.ec2.internal 50m Normal Scheduled pod/payload-processing-e2e-inf-a-bdd94b-5f4f4579c9-4mq7q Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-bdd94b-5f4f4579c9-4mq7q to ip-10-0-135-132.ec2.internal 4m48s Normal Scheduled pod/e2e-authcfg-http-9cdd18-openshift-default-575d9dcd85-h972c Successfully assigned openshift-ingress/e2e-authcfg-http-9cdd18-openshift-default-575d9dcd85-h972c to ip-10-0-142-185.ec2.internal 17m Normal Scheduled pod/maas-default-gateway-openshift-default-6db586dd6c-hcq62 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-hcq62 to ip-10-0-135-132.ec2.internal 77m Normal Scheduled pod/router-default-fb457cdb7-m99g8 Successfully assigned openshift-ingress/router-default-fb457cdb7-m99g8 to ip-10-0-142-185.ec2.internal 77m Warning FailedScheduling pod/router-default-fb457cdb7-m99g8 no nodes available to schedule pods 83m Warning FailedScheduling pod/router-default-fb457cdb7-m99g8 no nodes available to schedule pods 47m Normal Scheduled pod/payload-processing-e2e-inf-a-bc48cc-7d467bc759-tqr4d Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-bc48cc-7d467bc759-tqr4d to ip-10-0-142-185.ec2.internal 39m Normal Scheduled pod/e2e-shared-a-w4-c511a3-openshift-default-59bc6d8b7-9m4nr Successfully assigned openshift-ingress/e2e-shared-a-w4-c511a3-openshift-default-59bc6d8b7-9m4nr to ip-10-0-142-185.ec2.internal 64m Normal Scheduled pod/maas-default-gateway-openshift-default-6db586dd6c-hv7z4 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-hv7z4 to ip-10-0-130-77.ec2.internal 57m Normal Scheduled pod/e2e-mt-d399fb07-openshift-default-6d54788969-4z9xf Successfully assigned openshift-ingress/e2e-mt-d399fb07-openshift-default-6d54788969-4z9xf to ip-10-0-135-132.ec2.internal 60m Normal Scheduled pod/e2e-derive-36d7ee39-openshift-default-6b7cbf646f-nnh2m Successfully assigned openshift-ingress/e2e-derive-36d7ee39-openshift-default-6b7cbf646f-nnh2m to ip-10-0-142-185.ec2.internal 67m Normal Scheduled pod/payload-processing-e2e-worker-w6-012ba0-5669b88f55-swpg6 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-012ba0-5669b88f55-swpg6 to ip-10-0-142-185.ec2.internal 83m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-fb457cdb7 from 0 to 1 83m Normal SuccessfulCreate replicaset/router-default-fb457cdb7 Created pod: router-default-fb457cdb7-m99g8 82m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 82m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 82m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 77m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 77m Warning FailedMount pod/router-default-fb457cdb7-m99g8 MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 77m Warning FailedMount pod/router-default-fb457cdb7-m99g8 MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 77m Warning FailedMount pod/router-default-fb457cdb7-m99g8 MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 77m Normal Pulled pod/router-default-fb457cdb7-m99g8 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 77m Normal Created pod/router-default-fb457cdb7-m99g8 Container created 77m Normal Started pod/router-default-fb457cdb7-m99g8 Container started 77m Normal AddedInterface pod/router-default-fb457cdb7-m99g8 Add eth0 [10.132.0.10/23] from ovn-kubernetes 73m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 73m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-dlh79 73m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 73m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-dlh79 Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 73m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-dlh79 Add eth0 [10.132.0.26/23] from ovn-kubernetes 73m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-dlh79 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.598s (2.598s including waiting). Image size: 185476477 bytes. 73m Normal Created pod/istiod-openshift-gateway-7876bfcb74-dlh79 Container created 73m Normal Started pod/istiod-openshift-gateway-7876bfcb74-dlh79 Container started 73m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-dlh79 Readiness probe error: HTTP probe failed with statuscode: 503... 73m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-dlh79 Readiness probe failed: HTTP probe failed with statuscode: 503 73m Normal Pulling pod/maas-default-gateway-openshift-default-6db586dd6c-lnh72 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 73m Normal AddedInterface pod/maas-default-gateway-openshift-default-6db586dd6c-lnh72 Add eth0 [10.133.0.24/23] from ovn-kubernetes 73m Normal EnsuringLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensuring load balancer 73m Normal AddedInterface pod/openshift-ai-inference-openshift-ai-inference-69694c4c9-scp7q Add eth0 [10.133.0.26/23] from ovn-kubernetes 73m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6db586dd6c from 0 to 1 73m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6db586dd6c Created pod: maas-default-gateway-openshift-default-6db586dd6c-lnh72 73m Normal Pulling pod/openshift-ai-inference-openshift-ai-inference-69694c4c9-scp7q Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 73m Normal ScalingReplicaSet deployment/openshift-ai-inference-openshift-ai-inference Scaled up replica set openshift-ai-inference-openshift-ai-inference-69694c4c9 from 0 to 1 73m Normal SuccessfulCreate replicaset/openshift-ai-inference-openshift-ai-inference-69694c4c9 Created pod: openshift-ai-inference-openshift-ai-inference-69694c4c9-scp7q 73m Normal EnsuredLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensured load balancer 72m Normal Pulled pod/maas-default-gateway-openshift-default-6db586dd6c-lnh72 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 10.232s (10.232s including waiting). Image size: 183555333 bytes. 72m Normal Started pod/openshift-ai-inference-openshift-ai-inference-69694c4c9-scp7q Container started 72m Normal Created pod/openshift-ai-inference-openshift-ai-inference-69694c4c9-scp7q Container created 72m Normal Pulled pod/openshift-ai-inference-openshift-ai-inference-69694c4c9-scp7q Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 10.722s (10.722s including waiting). Image size: 183555333 bytes. 72m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 72m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 72m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 72m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 71m Normal AddedInterface pod/kube-auth-proxy-6f865f994d-vg2nd Add eth0 [10.134.0.25/23] from ovn-kubernetes 71m Normal Pulling pod/kube-auth-proxy-6f865f994d-9lf4d Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 71m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-6f865f994d from 0 to 2 71m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-6c85d847d5 Created pod: data-science-gateway-data-science-gateway-class-6c85d847d5kw5cm 71m Normal SuccessfulCreate replicaset/kube-auth-proxy-6f865f994d Created pod: kube-auth-proxy-6f865f994d-9lf4d 71m Normal AddedInterface pod/kube-auth-proxy-6f865f994d-9lf4d Add eth0 [10.132.0.28/23] from ovn-kubernetes 71m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-6c85d847d5 from 0 to 1 71m Normal Pulling pod/kube-auth-proxy-6f865f994d-vg2nd Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 71m Normal SuccessfulCreate replicaset/kube-auth-proxy-6f865f994d Created pod: kube-auth-proxy-6f865f994d-vg2nd 71m Normal Created pod/data-science-gateway-data-science-gateway-class-6c85d847d5kw5cm Container created 71m Normal Pulled pod/data-science-gateway-data-science-gateway-class-6c85d847d5kw5cm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 71m Normal Started pod/data-science-gateway-data-science-gateway-class-6c85d847d5kw5cm Container started 71m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-6c85d847d5kw5cm Add eth0 [10.133.0.38/23] from ovn-kubernetes 71m Normal Pulled pod/kube-auth-proxy-6f865f994d-vg2nd Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.341s (2.341s including waiting). Image size: 188904677 bytes. 71m Normal Created pod/kube-auth-proxy-6f865f994d-vg2nd Container created 71m Normal Pulled pod/kube-auth-proxy-6f865f994d-9lf4d Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.311s (2.311s including waiting). Image size: 188904677 bytes. 71m Normal Started pod/kube-auth-proxy-6f865f994d-vg2nd Container started 71m Normal Started pod/kube-auth-proxy-6f865f994d-9lf4d Container started 71m Normal Created pod/kube-auth-proxy-6f865f994d-9lf4d Container created 71m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 71m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 71m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 71m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 71m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 71m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 71m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 71m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 70m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 70m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-c926c 70m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-c926c Container created 70m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-c926c Add eth0 [10.132.0.32/23] from ovn-kubernetes 70m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-c926c Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 70m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-c926c Container started 70m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-dlh79 70m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 70m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-dlh79 Stopping container discovery 70m Normal Pulled pod/maas-default-gateway-openshift-default-6db586dd6c-lnh72 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 70m Normal Started pod/maas-default-gateway-openshift-default-6db586dd6c-lnh72 Container started 70m Normal Created pod/maas-default-gateway-openshift-default-6db586dd6c-lnh72 Container created 70m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 70m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 70m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 70m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-lnh72 Startup probe error: Get "http://10.133.0.24:15021/healthz/ready": dial tcp 10.133.0.24:15021: connect: connection refused... 70m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 70m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-lnh72 Startup probe failed: Get "http://10.133.0.24:15021/healthz/ready": dial tcp 10.133.0.24:15021: connect: connection refused 69m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 69m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 68m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-55fbbbf9dc from 0 to 1 68m Normal AddedInterface pod/payload-processing-55fbbbf9dc-rp6t6 Add eth0 [10.132.0.42/23] from ovn-kubernetes 68m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 68m Warning FailedCreate replicaset/payload-processing-55fbbbf9dc Error creating: pods "payload-processing-55fbbbf9dc-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 68m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-s6mwr 68m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-s6mwr Add eth0 [10.133.0.47/23] from ovn-kubernetes 68m Normal Pulling pod/payload-pre-processing-7b98bb678c-s6mwr Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 68m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 68m Normal SuccessfulCreate replicaset/payload-processing-55fbbbf9dc Created pod: payload-processing-55fbbbf9dc-rp6t6 68m Normal Pulling pod/payload-processing-55fbbbf9dc-rp6t6 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 68m Normal Pulled pod/payload-processing-55fbbbf9dc-rp6t6 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 1.976s (1.976s including waiting). Image size: 125353965 bytes. 68m Normal Started pod/payload-pre-processing-7b98bb678c-s6mwr Container started 68m Normal Created pod/payload-pre-processing-7b98bb678c-s6mwr Container created 68m Normal Pulled pod/payload-pre-processing-7b98bb678c-s6mwr Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.511s (2.511s including waiting). Image size: 125353965 bytes. 68m Normal Started pod/payload-processing-55fbbbf9dc-rp6t6 Container started 68m Normal Created pod/payload-processing-55fbbbf9dc-rp6t6 Container created 68m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6db586dd6c Created pod: maas-default-gateway-openshift-default-6db586dd6c-sv4fp 68m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6db586dd6c from 1 to 4 68m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6db586dd6c Created pod: maas-default-gateway-openshift-default-6db586dd6c-p65ln 68m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6db586dd6c Created pod: maas-default-gateway-openshift-default-6db586dd6c-frdpl 68m Normal Pulling pod/maas-default-gateway-openshift-default-6db586dd6c-frdpl Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 68m Normal Pulling pod/maas-default-gateway-openshift-default-6db586dd6c-p65ln Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 68m Normal AddedInterface pod/maas-default-gateway-openshift-default-6db586dd6c-frdpl Add eth0 [10.134.0.38/23] from ovn-kubernetes 68m Normal AddedInterface pod/maas-default-gateway-openshift-default-6db586dd6c-p65ln Add eth0 [10.132.0.43/23] from ovn-kubernetes 68m Normal AddedInterface pod/maas-default-gateway-openshift-default-6db586dd6c-sv4fp Add eth0 [10.133.0.48/23] from ovn-kubernetes 68m Normal Pulled pod/maas-default-gateway-openshift-default-6db586dd6c-sv4fp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 68m Normal Created pod/maas-default-gateway-openshift-default-6db586dd6c-sv4fp Container created 68m Normal Started pod/maas-default-gateway-openshift-default-6db586dd6c-sv4fp Container started 68m Normal Pulled pod/maas-default-gateway-openshift-default-6db586dd6c-p65ln Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.459s (2.459s including waiting). Image size: 183555333 bytes. 68m Normal Started pod/maas-default-gateway-openshift-default-6db586dd6c-p65ln Container started 68m Normal Created pod/maas-default-gateway-openshift-default-6db586dd6c-frdpl Container created 68m Normal Pulled pod/maas-default-gateway-openshift-default-6db586dd6c-frdpl Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.025s (3.025s including waiting). Image size: 183555333 bytes. 68m Normal Started pod/maas-default-gateway-openshift-default-6db586dd6c-frdpl Container started 68m Normal Created pod/maas-default-gateway-openshift-default-6db586dd6c-p65ln Container created 68m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-sv4fp Startup probe error: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused... 68m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-sv4fp Startup probe failed: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused 68m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-frdpl Startup probe error: Get "http://10.134.0.38:15021/healthz/ready": dial tcp 10.134.0.38:15021: connect: connection refused... 68m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-frdpl Startup probe failed: Get "http://10.134.0.38:15021/healthz/ready": dial tcp 10.134.0.38:15021: connect: connection refused 68m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-p65ln Startup probe failed: Get "http://10.132.0.43:15021/healthz/ready": dial tcp 10.132.0.43:15021: connect: connection refused 68m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-p65ln Startup probe error: Get "http://10.132.0.43:15021/healthz/ready": dial tcp 10.132.0.43:15021: connect: connection refused... 68m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6db586dd6c from 4 to 6 68m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 68m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6db586dd6c Created pod: maas-default-gateway-openshift-default-6db586dd6c-cmklv 68m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6db586dd6c Created pod: maas-default-gateway-openshift-default-6db586dd6c-bjfxc 68m Normal Started pod/maas-default-gateway-openshift-default-6db586dd6c-cmklv Container started 68m Normal AddedInterface pod/maas-default-gateway-openshift-default-6db586dd6c-bjfxc Add eth0 [10.134.0.39/23] from ovn-kubernetes 68m Normal Pulled pod/maas-default-gateway-openshift-default-6db586dd6c-bjfxc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 68m Normal Created pod/maas-default-gateway-openshift-default-6db586dd6c-bjfxc Container created 68m Normal Started pod/maas-default-gateway-openshift-default-6db586dd6c-bjfxc Container started 68m Normal AddedInterface pod/maas-default-gateway-openshift-default-6db586dd6c-cmklv Add eth0 [10.132.0.44/23] from ovn-kubernetes 68m Normal Pulled pod/maas-default-gateway-openshift-default-6db586dd6c-cmklv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 68m Normal Created pod/maas-default-gateway-openshift-default-6db586dd6c-cmklv Container created 68m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-cmklv Startup probe error: Get "http://10.132.0.44:15021/healthz/ready": dial tcp 10.132.0.44:15021: connect: connection refused... 68m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-bjfxc Startup probe error: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused... 68m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-bjfxc Startup probe failed: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused 68m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-cmklv Startup probe failed: Get "http://10.132.0.44:15021/healthz/ready": dial tcp 10.132.0.44:15021: connect: connection refused 67m Normal AddedLabel gateway/e2e-worker-w5-54114e Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-54114e 67m Normal AddedLabel gateway/e2e-worker-w6-012ba0 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-012ba0 67m Normal SuccessfulCreate replicaset/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 Created pod: e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-qhw2n 67m Normal AddedLabel gateway/e2e-worker-w4-0ad7c5 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-0ad7c5 67m Normal ScalingReplicaSet deployment/e2e-worker-w3-d52c59-openshift-default Scaled up replica set e2e-worker-w3-d52c59-openshift-default-5466968fb7 from 0 to 1 67m Normal SuccessfulCreate replicaset/e2e-worker-w5-54114e-openshift-default-7fc467ddd7 Created pod: e2e-worker-w5-54114e-openshift-default-7fc467ddd7-tdvdc 67m Normal SuccessfulCreate replicaset/e2e-worker-w3-d52c59-openshift-default-5466968fb7 Created pod: e2e-worker-w3-d52c59-openshift-default-5466968fb7-w2q4t 67m Normal AddedLabel gateway/e2e-worker-w2-421583 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-421583 67m Normal ScalingReplicaSet deployment/e2e-worker-w5-54114e-openshift-default Scaled up replica set e2e-worker-w5-54114e-openshift-default-7fc467ddd7 from 0 to 1 67m Normal ScalingReplicaSet deployment/e2e-worker-w0-09d0d5-openshift-default Scaled up replica set e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 from 0 to 1 67m Normal AddedLabel gateway/e2e-worker-w1-2b1e88 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-2b1e88 67m Normal AddedLabel gateway/e2e-worker-w3-d52c59 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-d52c59 67m Normal AddedLabel gateway/e2e-worker-w0-09d0d5 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-09d0d5 67m Normal AddedInterface pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-kzdls Add eth0 [10.133.0.52/23] from ovn-kubernetes 67m Normal AddedInterface pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-qhw2n Add eth0 [10.133.0.50/23] from ovn-kubernetes 67m Normal SuccessfulCreate replicaset/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc Created pod: e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-kzdls 67m Normal AddedInterface pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-w2q4t Add eth0 [10.132.0.45/23] from ovn-kubernetes 67m Normal AddedInterface pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8dgzg Add eth0 [10.132.0.46/23] from ovn-kubernetes 67m Normal AddedInterface pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-dxslt Add eth0 [10.132.0.47/23] from ovn-kubernetes 67m Normal SuccessfulCreate replicaset/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c Created pod: e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8dgzg 67m Normal SuccessfulCreate replicaset/e2e-worker-w2-421583-openshift-default-5cb767b447 Created pod: e2e-worker-w2-421583-openshift-default-5cb767b447-k4nm5 67m Normal ScalingReplicaSet deployment/e2e-worker-w2-421583-openshift-default Scaled up replica set e2e-worker-w2-421583-openshift-default-5cb767b447 from 0 to 1 67m Normal ScalingReplicaSet deployment/e2e-worker-w6-012ba0-openshift-default Scaled up replica set e2e-worker-w6-012ba0-openshift-default-846bcc5bfc from 0 to 1 67m Normal AddedInterface pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-tdvdc Add eth0 [10.133.0.51/23] from ovn-kubernetes 67m Normal ScalingReplicaSet deployment/e2e-worker-w1-2b1e88-openshift-default Scaled up replica set e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 from 0 to 1 67m Normal SuccessfulCreate replicaset/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 Created pod: e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-dxslt 67m Normal ScalingReplicaSet deployment/e2e-worker-w4-0ad7c5-openshift-default Scaled up replica set e2e-worker-w4-0ad7c5-openshift-default-788b484d5c from 0 to 1 67m Normal AddedInterface pod/e2e-worker-w2-421583-openshift-default-5cb767b447-k4nm5 Add eth0 [10.133.0.53/23] from ovn-kubernetes 67m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-09d0d5 Scaled up replica set payload-pre-processing-e2e-worker-w0-09d0d5-75f8f97f55 from 0 to 1 67m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-09d0d5-85cb95d7ff Error creating: pods "payload-processing-e2e-worker-w0-09d0d5-85cb95d7ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-09d0d5: serviceaccount "payload-processing-e2e-worker-w0-09d0d5" not found 67m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-09d0d5 Scaled up replica set payload-processing-e2e-worker-w0-09d0d5-85cb95d7ff from 0 to 1 67m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-09d0d5-75f8f97f55 Error creating: pods "payload-pre-processing-e2e-worker-w0-09d0d5-75f8f97f55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-09d0d5: serviceaccount "payload-processing-e2e-worker-w0-09d0d5" not found 67m Normal Created pod/payload-processing-e2e-worker-w0-09d0d5-85cb95d7ff-n4lzb Container created 67m Normal Started pod/payload-pre-processing-e2e-worker-w0-09d0d5-75f8f97f55-6hq48 Container started 67m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-09d0d5-85cb95d7ff Created pod: payload-processing-e2e-worker-w0-09d0d5-85cb95d7ff-n4lzb 67m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-09d0d5-75f8f97f55-6hq48 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Normal Started pod/payload-processing-e2e-worker-w0-09d0d5-85cb95d7ff-n4lzb Container started 67m Normal AddedInterface pod/payload-processing-e2e-worker-w0-09d0d5-85cb95d7ff-n4lzb Add eth0 [10.132.0.49/23] from ovn-kubernetes 67m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-09d0d5-75f8f97f55 Created pod: payload-pre-processing-e2e-worker-w0-09d0d5-75f8f97f55-6hq48 67m Normal Pulled pod/payload-processing-e2e-worker-w0-09d0d5-85cb95d7ff-n4lzb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-09d0d5-75f8f97f55-6hq48 Add eth0 [10.132.0.48/23] from ovn-kubernetes 67m Normal Created pod/payload-pre-processing-e2e-worker-w0-09d0d5-75f8f97f55-6hq48 Container created 67m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-54114e-5cf4644f Error creating: pods "payload-processing-e2e-worker-w5-54114e-5cf4644f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-54114e: serviceaccount "payload-processing-e2e-worker-w5-54114e" not found 67m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-54114e Scaled up replica set payload-processing-e2e-worker-w5-54114e-5cf4644f from 0 to 1 67m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-54114e-5cf4644f Created pod: payload-processing-e2e-worker-w5-54114e-5cf4644f-8htwc 67m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-54114e-7f6f89654c Error creating: pods "payload-pre-processing-e2e-worker-w5-54114e-7f6f89654c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-54114e: serviceaccount "payload-processing-e2e-worker-w5-54114e" not found 67m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-54114e-7f6f89654c Created pod: payload-pre-processing-e2e-worker-w5-54114e-7f6f89654c-sb4mv 67m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-54114e Scaled up replica set payload-pre-processing-e2e-worker-w5-54114e-7f6f89654c from 0 to 1 67m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-d52c59-56d5c877c5 Created pod: payload-pre-processing-e2e-worker-w3-d52c59-56d5c877c5-zvrjx 67m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-d52c59-56d5c877c5-zvrjx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Normal Created pod/payload-processing-e2e-worker-w3-d52c59-f75cff857-zdrn8 Container created 67m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-d52c59 Scaled up replica set payload-pre-processing-e2e-worker-w3-d52c59-56d5c877c5 from 0 to 1 67m Normal Pulled pod/payload-processing-e2e-worker-w3-d52c59-f75cff857-zdrn8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Normal AddedInterface pod/payload-processing-e2e-worker-w3-d52c59-f75cff857-zdrn8 Add eth0 [10.133.0.56/23] from ovn-kubernetes 67m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-d52c59-56d5c877c5 Error creating: pods "payload-pre-processing-e2e-worker-w3-d52c59-56d5c877c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-d52c59: serviceaccount "payload-processing-e2e-worker-w3-d52c59" not found 67m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-d52c59-f75cff857 Error creating: pods "payload-processing-e2e-worker-w3-d52c59-f75cff857-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-d52c59: serviceaccount "payload-processing-e2e-worker-w3-d52c59" not found 67m Normal Created pod/payload-processing-e2e-worker-w5-54114e-5cf4644f-8htwc Container created 67m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-54114e-7f6f89654c-sb4mv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Normal Pulled pod/payload-processing-e2e-worker-w5-54114e-5cf4644f-8htwc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-54114e-7f6f89654c-sb4mv Add eth0 [10.133.0.54/23] from ovn-kubernetes 67m Normal AddedInterface pod/payload-processing-e2e-worker-w5-54114e-5cf4644f-8htwc Add eth0 [10.133.0.55/23] from ovn-kubernetes 67m Normal Started pod/payload-processing-e2e-worker-w5-54114e-5cf4644f-8htwc Container started 67m Normal Started pod/payload-processing-e2e-worker-w3-d52c59-f75cff857-zdrn8 Container started 67m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-d52c59 Scaled up replica set payload-processing-e2e-worker-w3-d52c59-f75cff857 from 0 to 1 67m Normal Started pod/payload-pre-processing-e2e-worker-w5-54114e-7f6f89654c-sb4mv Container started 67m Normal Created pod/payload-pre-processing-e2e-worker-w5-54114e-7f6f89654c-sb4mv Container created 67m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-d52c59-f75cff857 Created pod: payload-processing-e2e-worker-w3-d52c59-f75cff857-zdrn8 67m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-d52c59-56d5c877c5-zvrjx Add eth0 [10.132.0.50/23] from ovn-kubernetes 67m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-0ad7c5-dffdb7775 Error creating: pods "payload-pre-processing-e2e-worker-w4-0ad7c5-dffdb7775-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-0ad7c5: serviceaccount "payload-processing-e2e-worker-w4-0ad7c5" not found 67m Normal Created pod/payload-pre-processing-e2e-worker-w3-d52c59-56d5c877c5-zvrjx Container created 67m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-0ad7c5 Scaled up replica set payload-processing-e2e-worker-w4-0ad7c5-7b6cc5f5c5 from 0 to 1 67m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-0ad7c5-7b6cc5f5c5 Created pod: payload-processing-e2e-worker-w4-0ad7c5-7b6cc5f5c5-8c2qz 67m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-0ad7c5 Scaled up replica set payload-pre-processing-e2e-worker-w4-0ad7c5-dffdb7775 from 0 to 1 67m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-0ad7c5-7b6cc5f5c5 Error creating: pods "payload-processing-e2e-worker-w4-0ad7c5-7b6cc5f5c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-0ad7c5: serviceaccount "payload-processing-e2e-worker-w4-0ad7c5" not found 67m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-0ad7c5-dffdb7775 Created pod: payload-pre-processing-e2e-worker-w4-0ad7c5-dffdb7775-kt9zp 67m Normal Started pod/payload-pre-processing-e2e-worker-w3-d52c59-56d5c877c5-zvrjx Container started 67m Normal Started pod/payload-processing-e2e-worker-w4-0ad7c5-7b6cc5f5c5-8c2qz Container started 67m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-012ba0-5669b88f55 Created pod: payload-processing-e2e-worker-w6-012ba0-5669b88f55-swpg6 67m Normal Created pod/payload-pre-processing-e2e-worker-w4-0ad7c5-dffdb7775-kt9zp Container created 67m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-012ba0-5669b88f55 Error creating: pods "payload-processing-e2e-worker-w6-012ba0-5669b88f55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-012ba0: serviceaccount "payload-processing-e2e-worker-w6-012ba0" not found 67m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-012ba0-669ff7bfff Created pod: payload-pre-processing-e2e-worker-w6-012ba0-669ff7bfff-8w9bc 67m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-0ad7c5-dffdb7775-kt9zp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-0ad7c5-dffdb7775-kt9zp Add eth0 [10.133.0.57/23] from ovn-kubernetes 67m Normal Started pod/payload-pre-processing-e2e-worker-w4-0ad7c5-dffdb7775-kt9zp Container started 67m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-012ba0 Scaled up replica set payload-pre-processing-e2e-worker-w6-012ba0-669ff7bfff from 0 to 1 67m Normal Created pod/payload-processing-e2e-worker-w4-0ad7c5-7b6cc5f5c5-8c2qz Container created 67m Normal Pulled pod/payload-processing-e2e-worker-w4-0ad7c5-7b6cc5f5c5-8c2qz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-012ba0-669ff7bfff Error creating: pods "payload-pre-processing-e2e-worker-w6-012ba0-669ff7bfff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-012ba0: serviceaccount "payload-processing-e2e-worker-w6-012ba0" not found 67m Normal AddedInterface pod/payload-processing-e2e-worker-w4-0ad7c5-7b6cc5f5c5-8c2qz Add eth0 [10.133.0.58/23] from ovn-kubernetes 67m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-012ba0 Scaled up replica set payload-processing-e2e-worker-w6-012ba0-5669b88f55 from 0 to 1 67m Normal Created pod/payload-processing-e2e-worker-w6-012ba0-5669b88f55-swpg6 Container created 67m Normal Started pod/payload-pre-processing-e2e-worker-w6-012ba0-669ff7bfff-8w9bc Container started 67m Normal AddedInterface pod/payload-processing-e2e-worker-w6-012ba0-5669b88f55-swpg6 Add eth0 [10.132.0.52/23] from ovn-kubernetes 67m Normal Started pod/payload-processing-e2e-worker-w6-012ba0-5669b88f55-swpg6 Container started 67m Normal Created pod/payload-pre-processing-e2e-worker-w6-012ba0-669ff7bfff-8w9bc Container created 67m Normal Pulled pod/payload-processing-e2e-worker-w6-012ba0-5669b88f55-swpg6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-012ba0-669ff7bfff-8w9bc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-012ba0-669ff7bfff-8w9bc Add eth0 [10.132.0.53/23] from ovn-kubernetes 67m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-421583-7f9c569f65 Error creating: pods "payload-processing-e2e-worker-w2-421583-7f9c569f65-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-421583: serviceaccount "payload-processing-e2e-worker-w2-421583" not found 67m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-421583 Scaled up replica set payload-pre-processing-e2e-worker-w2-421583-bfd5dcc6c from 0 to 1 67m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-421583-7f9c569f65 Created pod: payload-processing-e2e-worker-w2-421583-7f9c569f65-xm89m 67m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-421583-bfd5dcc6c Error creating: pods "payload-pre-processing-e2e-worker-w2-421583-bfd5dcc6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-421583: serviceaccount "payload-processing-e2e-worker-w2-421583" not found 67m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-421583-bfd5dcc6c Created pod: payload-pre-processing-e2e-worker-w2-421583-bfd5dcc6c-7ztzj 67m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-421583 Scaled up replica set payload-processing-e2e-worker-w2-421583-7f9c569f65 from 0 to 1 67m Normal Created pod/payload-processing-e2e-worker-w2-421583-7f9c569f65-xm89m Container created 67m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-2b1e88-65c766876c Created pod: payload-pre-processing-e2e-worker-w1-2b1e88-65c766876c-4qz4v 67m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-2b1e88 Scaled up replica set payload-pre-processing-e2e-worker-w1-2b1e88-65c766876c from 0 to 1 67m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-421583-bfd5dcc6c-7ztzj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-2b1e88-65c766876c Error creating: pods "payload-pre-processing-e2e-worker-w1-2b1e88-65c766876c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-2b1e88: serviceaccount "payload-processing-e2e-worker-w1-2b1e88" not found 67m Normal Created pod/payload-pre-processing-e2e-worker-w2-421583-bfd5dcc6c-7ztzj Container created 67m Normal Started pod/payload-processing-e2e-worker-w2-421583-7f9c569f65-xm89m Container started 67m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-421583-bfd5dcc6c-7ztzj Add eth0 [10.133.0.59/23] from ovn-kubernetes 67m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-2b1e88-6f4479c569 Error creating: pods "payload-processing-e2e-worker-w1-2b1e88-6f4479c569-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-2b1e88: serviceaccount "payload-processing-e2e-worker-w1-2b1e88" not found 67m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-2b1e88-6f4479c569 Created pod: payload-processing-e2e-worker-w1-2b1e88-6f4479c569-mltjd 67m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-2b1e88 Scaled up replica set payload-processing-e2e-worker-w1-2b1e88-6f4479c569 from 0 to 1 67m Normal Started pod/payload-pre-processing-e2e-worker-w2-421583-bfd5dcc6c-7ztzj Container started 67m Normal AddedInterface pod/payload-processing-e2e-worker-w2-421583-7f9c569f65-xm89m Add eth0 [10.133.0.60/23] from ovn-kubernetes 67m Normal Pulled pod/payload-processing-e2e-worker-w2-421583-7f9c569f65-xm89m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Normal Created pod/payload-processing-e2e-worker-w1-2b1e88-6f4479c569-mltjd Container created 67m Normal Pulled pod/payload-processing-e2e-worker-w1-2b1e88-6f4479c569-mltjd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Normal AddedInterface pod/payload-processing-e2e-worker-w1-2b1e88-6f4479c569-mltjd Add eth0 [10.133.0.61/23] from ovn-kubernetes 67m Normal Started pod/payload-processing-e2e-worker-w1-2b1e88-6f4479c569-mltjd Container started 67m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-2b1e88-65c766876c-4qz4v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 67m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-2b1e88-65c766876c-4qz4v Add eth0 [10.133.0.62/23] from ovn-kubernetes 67m Normal Started pod/payload-pre-processing-e2e-worker-w1-2b1e88-65c766876c-4qz4v Container started 67m Normal Created pod/payload-pre-processing-e2e-worker-w1-2b1e88-65c766876c-4qz4v Container created 67m Normal Pulled pod/e2e-worker-w2-421583-openshift-default-5cb767b447-k4nm5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 67m Normal Started pod/e2e-worker-w2-421583-openshift-default-5cb767b447-k4nm5 Container started 67m Normal Created pod/e2e-worker-w2-421583-openshift-default-5cb767b447-k4nm5 Container created 67m Normal Started pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-dxslt Container started 67m Normal Created pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-dxslt Container created 67m Normal Pulled pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-dxslt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 67m Warning ProbeError pod/e2e-worker-w2-421583-openshift-default-5cb767b447-k4nm5 Startup probe error: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused... 67m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-2b1e88-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 67m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-2b1e88-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 67m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-54114e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 67m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-54114e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 67m Warning Unhealthy pod/e2e-worker-w2-421583-openshift-default-5cb767b447-k4nm5 Startup probe failed: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused 67m Warning Unhealthy pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-tdvdc Readiness probe failed: Get "http://10.133.0.51:15021/healthz/ready": read tcp 10.133.0.2:60028->10.133.0.51:15021: read: connection reset by peer 67m Warning ProbeError pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-tdvdc Readiness probe error: Get "http://10.133.0.51:15021/healthz/ready": read tcp 10.133.0.2:60028->10.133.0.51:15021: read: connection reset by peer... 67m Normal Started pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-tdvdc Container started 67m Normal Created pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-tdvdc Container created 67m Normal Pulled pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-tdvdc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 67m Warning ProbeError pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-dxslt Startup probe error: Get "http://10.132.0.47:15021/healthz/ready": dial tcp 10.132.0.47:15021: connect: connection refused... 67m Warning Unhealthy pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-dxslt Startup probe failed: Get "http://10.132.0.47:15021/healthz/ready": dial tcp 10.132.0.47:15021: connect: connection refused 67m Normal Started pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8dgzg Container started 67m Normal Started pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-w2q4t Container started 67m Normal Pulled pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-w2q4t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 67m Normal Created pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8dgzg Container created 67m Normal Pulled pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8dgzg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 67m Normal Created pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-w2q4t Container created 66m Normal Started pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-kzdls Container started 66m Normal Created pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-kzdls Container created 66m Normal Pulled pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-kzdls Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Warning ProbeError pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-tdvdc Startup probe error: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused... 66m Warning Unhealthy pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-tdvdc Startup probe failed: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused 66m Normal Pulled pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-qhw2n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 66m Normal Started pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-qhw2n Container started 66m Normal Created pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-qhw2n Container created 66m Warning ProbeError pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8dgzg Startup probe error: Get "http://10.132.0.46:15021/healthz/ready": dial tcp 10.132.0.46:15021: connect: connection refused... 66m Warning Unhealthy pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8dgzg Startup probe failed: Get "http://10.132.0.46:15021/healthz/ready": dial tcp 10.132.0.46:15021: connect: connection refused 66m Warning Unhealthy pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-kzdls Startup probe failed: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused 66m Warning ProbeError pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-kzdls Startup probe error: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused... 66m Warning Unhealthy pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-w2q4t Startup probe failed: Get "http://10.132.0.45:15021/healthz/ready": dial tcp 10.132.0.45:15021: connect: connection refused 66m Warning ProbeError pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-w2q4t Startup probe error: Get "http://10.132.0.45:15021/healthz/ready": dial tcp 10.132.0.45:15021: connect: connection refused... 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-09d0d5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-09d0d5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-421583-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning ProbeError pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-qhw2n Startup probe error: Get "http://10.133.0.50:15021/healthz/ready": dial tcp 10.133.0.50:15021: connect: connection refused... 66m Warning Unhealthy pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-qhw2n Startup probe failed: Get "http://10.133.0.50:15021/healthz/ready": dial tcp 10.133.0.50:15021: connect: connection refused 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-0ad7c5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-0ad7c5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-421583-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-d52c59-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-d52c59-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-2b1e88-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-012ba0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-012ba0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-421583-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-2b1e88-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-421583-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-09d0d5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-d52c59-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-09d0d5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-d52c59-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-54114e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-012ba0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 66m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-0ad7c5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-0ad7c5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-54114e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-012ba0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 65m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-09d0d5-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 65m Normal ScalingReplicaSet deployment/e2e-worker-w0-09d0d5-openshift-default Scaled up replica set e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 from 1 to 2 65m Normal Created pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-25w9c Container created 65m Normal ScalingReplicaSet deployment/e2e-worker-w5-54114e-openshift-default Scaled up replica set e2e-worker-w5-54114e-openshift-default-7fc467ddd7 from 1 to 2 65m Normal AddedInterface pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-25w9c Add eth0 [10.132.0.61/23] from ovn-kubernetes 65m Normal SuccessfulCreate replicaset/e2e-worker-w5-54114e-openshift-default-7fc467ddd7 Created pod: e2e-worker-w5-54114e-openshift-default-7fc467ddd7-25w9c 65m Normal Pulled pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-25w9c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal Started pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-25w9c Container started 65m Normal AddedInterface pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-2t4b8 Add eth0 [10.132.0.60/23] from ovn-kubernetes 65m Normal Created pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-2t4b8 Container created 65m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-54114e-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 65m Normal SuccessfulCreate replicaset/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 Created pod: e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-2t4b8 65m Normal Started pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-2t4b8 Container started 65m Normal Pulled pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-2t4b8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Warning ProbeError pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-25w9c Startup probe error: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused... 65m Warning ProbeError pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-2t4b8 Startup probe error: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused... 65m Warning Unhealthy pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-25w9c Startup probe failed: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused 65m Warning Unhealthy pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-2t4b8 Startup probe failed: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused 65m Normal Created pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-67nxn Container created 65m Normal Pulled pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-67nxn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal AddedInterface pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-67nxn Add eth0 [10.133.0.71/23] from ovn-kubernetes 65m Normal ScalingReplicaSet deployment/e2e-worker-w3-d52c59-openshift-default Scaled up replica set e2e-worker-w3-d52c59-openshift-default-5466968fb7 from 1 to 2 65m Normal SuccessfulCreate replicaset/e2e-worker-w3-d52c59-openshift-default-5466968fb7 Created pod: e2e-worker-w3-d52c59-openshift-default-5466968fb7-67nxn 65m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-d52c59-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 65m Normal Started pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-67nxn Container started 65m Warning Unhealthy pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-67nxn Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused 65m Warning ProbeError pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-67nxn Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused... 65m Normal ScalingReplicaSet deployment/e2e-worker-w4-0ad7c5-openshift-default Scaled up replica set e2e-worker-w4-0ad7c5-openshift-default-788b484d5c from 1 to 2 65m Normal SuccessfulCreate replicaset/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c Created pod: e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-72ws5 65m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0ad7c5-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 65m Normal ScalingReplicaSet deployment/e2e-worker-w6-012ba0-openshift-default Scaled up replica set e2e-worker-w6-012ba0-openshift-default-846bcc5bfc from 1 to 2 65m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-012ba0-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 65m Normal SuccessfulCreate replicaset/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc Created pod: e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-2w4gw 65m Normal Created pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-72ws5 Container created 65m Normal Started pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-2w4gw Container started 65m Normal Created pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-2w4gw Container created 65m Normal Started pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-72ws5 Container started 65m Normal Pulled pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-2w4gw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal Pulled pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-72ws5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Normal AddedInterface pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-72ws5 Add eth0 [10.133.0.74/23] from ovn-kubernetes 65m Normal AddedInterface pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-2w4gw Add eth0 [10.132.0.62/23] from ovn-kubernetes 65m Warning ProbeError pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-2w4gw Startup probe error: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused... 65m Warning Unhealthy pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-2w4gw Startup probe failed: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused 64m Warning ProbeError pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-72ws5 Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused... 64m Warning Unhealthy pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-72ws5 Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused 64m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6db586dd6c Created pod: maas-default-gateway-openshift-default-6db586dd6c-hv7z4 64m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 64m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6db586dd6c Created pod: maas-default-gateway-openshift-default-6db586dd6c-24vbj 64m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6db586dd6c from 6 to 9 64m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6db586dd6c Created pod: maas-default-gateway-openshift-default-6db586dd6c-8c8cz 64m Normal AddedInterface pod/maas-default-gateway-openshift-default-6db586dd6c-hv7z4 Add eth0 [10.134.0.49/23] from ovn-kubernetes 64m Normal AddedInterface pod/maas-default-gateway-openshift-default-6db586dd6c-24vbj Add eth0 [10.133.0.85/23] from ovn-kubernetes 64m Normal Created pod/maas-default-gateway-openshift-default-6db586dd6c-8c8cz Container created 64m Normal AddedInterface pod/maas-default-gateway-openshift-default-6db586dd6c-8c8cz Add eth0 [10.132.0.63/23] from ovn-kubernetes 64m Normal Started pod/maas-default-gateway-openshift-default-6db586dd6c-hv7z4 Container started 64m Normal Pulled pod/maas-default-gateway-openshift-default-6db586dd6c-8c8cz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Started pod/maas-default-gateway-openshift-default-6db586dd6c-8c8cz Container started 64m Normal Started pod/maas-default-gateway-openshift-default-6db586dd6c-24vbj Container started 64m Normal Created pod/maas-default-gateway-openshift-default-6db586dd6c-24vbj Container created 64m Normal Pulled pod/maas-default-gateway-openshift-default-6db586dd6c-hv7z4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Created pod/maas-default-gateway-openshift-default-6db586dd6c-hv7z4 Container created 64m Normal Pulled pod/maas-default-gateway-openshift-default-6db586dd6c-24vbj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-hv7z4 Startup probe failed: Get "http://10.134.0.49:15021/healthz/ready": dial tcp 10.134.0.49:15021: connect: connection refused 64m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-hv7z4 Startup probe error: Get "http://10.134.0.49:15021/healthz/ready": dial tcp 10.134.0.49:15021: connect: connection refused... 64m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-8c8cz Startup probe failed: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused 64m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-24vbj Startup probe error: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused... 64m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-24vbj Startup probe failed: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused 64m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-8c8cz Startup probe error: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused... 64m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6db586dd6c from 9 to 10 64m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6db586dd6c (combined from similar events): Created pod: maas-default-gateway-openshift-default-6db586dd6c-vbv8b 64m Normal AddedInterface pod/maas-default-gateway-openshift-default-6db586dd6c-vbv8b Add eth0 [10.133.0.90/23] from ovn-kubernetes 64m Normal Pulled pod/maas-default-gateway-openshift-default-6db586dd6c-vbv8b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Created pod/maas-default-gateway-openshift-default-6db586dd6c-vbv8b Container created 64m Normal Started pod/maas-default-gateway-openshift-default-6db586dd6c-vbv8b Container started 64m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-vbv8b Startup probe failed: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused 64m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-vbv8b Startup probe error: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused... 64m Normal ScalingReplicaSet deployment/e2e-worker-w3-d52c59-openshift-default Scaled up replica set e2e-worker-w3-d52c59-openshift-default-5466968fb7 from 2 to 4 64m Normal AddedInterface pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-7m76p Add eth0 [10.134.0.53/23] from ovn-kubernetes 64m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-012ba0-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 64m Normal ScalingReplicaSet deployment/e2e-worker-w6-012ba0-openshift-default Scaled up replica set e2e-worker-w6-012ba0-openshift-default-846bcc5bfc from 2 to 3 64m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0ad7c5-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 64m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-2b1e88-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 64m Normal ScalingReplicaSet deployment/e2e-worker-w1-2b1e88-openshift-default Scaled up replica set e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 from 1 to 2 64m Normal SuccessfulCreate replicaset/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c Created pod: e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hq9vz 64m Normal ScalingReplicaSet deployment/e2e-worker-w5-54114e-openshift-default Scaled up replica set e2e-worker-w5-54114e-openshift-default-7fc467ddd7 from 2 to 4 64m Normal SuccessfulCreate replicaset/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc Created pod: e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-lg99w 64m Normal ScalingReplicaSet deployment/e2e-worker-w4-0ad7c5-openshift-default Scaled up replica set e2e-worker-w4-0ad7c5-openshift-default-788b484d5c from 2 to 3 64m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-d52c59-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 64m Normal SuccessfulCreate replicaset/e2e-worker-w3-d52c59-openshift-default-5466968fb7 Created pod: e2e-worker-w3-d52c59-openshift-default-5466968fb7-qtv79 64m Normal SuccessfulCreate replicaset/e2e-worker-w3-d52c59-openshift-default-5466968fb7 Created pod: e2e-worker-w3-d52c59-openshift-default-5466968fb7-7m76p 64m Normal SuccessfulCreate replicaset/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 Created pod: e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-svsgv 64m Normal SuccessfulCreate replicaset/e2e-worker-w5-54114e-openshift-default-7fc467ddd7 Created pod: e2e-worker-w5-54114e-openshift-default-7fc467ddd7-582w4 64m Normal SuccessfulCreate replicaset/e2e-worker-w5-54114e-openshift-default-7fc467ddd7 Created pod: e2e-worker-w5-54114e-openshift-default-7fc467ddd7-kvtw4 64m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-54114e-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 64m Normal Created pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-7m76p Container created 64m Normal Created pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-qtv79 Container created 64m Normal ScalingReplicaSet deployment/e2e-worker-w2-421583-openshift-default Scaled up replica set e2e-worker-w2-421583-openshift-default-5cb767b447 from 1 to 2 64m Normal Started pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-7m76p Container started 64m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-421583-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 64m Normal Pulled pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-lg99w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal AddedInterface pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-lg99w Add eth0 [10.134.0.56/23] from ovn-kubernetes 64m Normal Started pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-kvtw4 Container started 64m Normal Created pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-kvtw4 Container created 64m Normal Pulled pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-kvtw4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Started pod/e2e-worker-w2-421583-openshift-default-5cb767b447-k94dp Container started 64m Normal Created pod/e2e-worker-w2-421583-openshift-default-5cb767b447-k94dp Container created 64m Normal Pulled pod/e2e-worker-w2-421583-openshift-default-5cb767b447-k94dp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal AddedInterface pod/e2e-worker-w2-421583-openshift-default-5cb767b447-k94dp Add eth0 [10.132.0.66/23] from ovn-kubernetes 64m Normal AddedInterface pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-kvtw4 Add eth0 [10.134.0.54/23] from ovn-kubernetes 64m Normal AddedInterface pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-qtv79 Add eth0 [10.132.0.65/23] from ovn-kubernetes 64m Normal Pulled pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-qtv79 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Pulled pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-7m76p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal AddedInterface pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-svsgv Add eth0 [10.133.0.95/23] from ovn-kubernetes 64m Normal Started pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-lg99w Container started 64m Normal Created pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-lg99w Container created 64m Normal SuccessfulCreate replicaset/e2e-worker-w2-421583-openshift-default-5cb767b447 Created pod: e2e-worker-w2-421583-openshift-default-5cb767b447-k94dp 64m Normal Started pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-582w4 Container started 64m Normal AddedInterface pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hq9vz Add eth0 [10.134.0.55/23] from ovn-kubernetes 64m Normal Pulled pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hq9vz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Created pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hq9vz Container created 64m Normal Started pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hq9vz Container started 64m Normal Created pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-582w4 Container created 64m Normal Started pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-svsgv Container started 64m Normal Pulled pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-582w4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Pulled pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-svsgv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal Started pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-qtv79 Container started 64m Normal AddedInterface pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-582w4 Add eth0 [10.133.0.94/23] from ovn-kubernetes 64m Normal Created pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-svsgv Container created 63m Warning ProbeError pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-582w4 Startup probe error: Get "http://10.133.0.94:15021/healthz/ready": dial tcp 10.133.0.94:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-kvtw4 Startup probe failed: Get "http://10.134.0.54:15021/healthz/ready": dial tcp 10.134.0.54:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-kvtw4 Startup probe error: Get "http://10.134.0.54:15021/healthz/ready": dial tcp 10.134.0.54:15021: connect: connection refused... 63m Warning ProbeError pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-qtv79 Startup probe error: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-qtv79 Startup probe failed: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused 63m Warning Unhealthy pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hq9vz Startup probe failed: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hq9vz Startup probe error: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-582w4 Startup probe failed: Get "http://10.133.0.94:15021/healthz/ready": dial tcp 10.133.0.94:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w2-421583-openshift-default-5cb767b447-k94dp Startup probe error: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused... 63m Warning ProbeError pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-svsgv Startup probe error: Get "http://10.133.0.95:15021/healthz/ready": dial tcp 10.133.0.95:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-svsgv Startup probe failed: Get "http://10.133.0.95:15021/healthz/ready": dial tcp 10.133.0.95:15021: connect: connection refused 63m Warning Unhealthy pod/e2e-worker-w2-421583-openshift-default-5cb767b447-k94dp Startup probe failed: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused 63m Warning Unhealthy pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-7m76p Startup probe failed: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-7m76p Startup probe error: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-lg99w Startup probe failed: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-lg99w Startup probe error: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused... 63m Normal SuccessfulCreate replicaset/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 Created pod: e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-bhn2k 63m Normal ScalingReplicaSet deployment/e2e-worker-w0-09d0d5-openshift-default Scaled up replica set e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 from 2 to 4 63m Normal SuccessfulCreate replicaset/e2e-worker-w3-d52c59-openshift-default-5466968fb7 Created pod: e2e-worker-w3-d52c59-openshift-default-5466968fb7-qchs2 63m Normal ScalingReplicaSet deployment/e2e-worker-w1-2b1e88-openshift-default Scaled up replica set e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 from 2 to 3 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-09d0d5-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 63m Normal SuccessfulCreate replicaset/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 Created pod: e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-pmswk 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-d52c59-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 63m Normal ScalingReplicaSet deployment/e2e-worker-w3-d52c59-openshift-default Scaled up replica set e2e-worker-w3-d52c59-openshift-default-5466968fb7 from 4 to 5 63m Normal Pulled pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-bhn2k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-bhn2k Add eth0 [10.132.0.67/23] from ovn-kubernetes 63m Normal Pulled pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-pmswk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-pmswk Add eth0 [10.134.0.57/23] from ovn-kubernetes 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-2b1e88-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 63m Normal Created pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-cjkc9 Container created 63m Normal Pulled pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-qchs2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Pulled pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-cjkc9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-cjkc9 Add eth0 [10.134.0.58/23] from ovn-kubernetes 63m Normal Started pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-pmswk Container started 63m Normal Created pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-pmswk Container created 63m Normal Started pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-bhn2k Container started 63m Normal Started pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-qchs2 Container started 63m Normal Created pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-bhn2k Container created 63m Normal AddedInterface pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-qchs2 Add eth0 [10.133.0.96/23] from ovn-kubernetes 63m Normal Started pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-cjkc9 Container started 63m Normal Created pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-qchs2 Container created 63m Normal SuccessfulCreate replicaset/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 Created pod: e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-cjkc9 63m Warning Unhealthy pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-pmswk Startup probe failed: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-bhn2k Startup probe error: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-bhn2k Startup probe failed: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-pmswk Startup probe error: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused... 63m Warning ProbeError pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-qchs2 Startup probe error: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-cjkc9 Startup probe failed: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused 63m Warning Unhealthy pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-qchs2 Startup probe failed: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused 63m Warning ProbeError pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-cjkc9 Startup probe error: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused... 63m Normal ScalingReplicaSet deployment/e2e-worker-w0-09d0d5-openshift-default Scaled up replica set e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 from 4 to 5 63m Normal Pulled pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-d5lpp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal SuccessfulCreate replicaset/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 Created pod: e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-d5lpp 63m Normal AddedInterface pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-d5lpp Add eth0 [10.133.0.98/23] from ovn-kubernetes 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-09d0d5-openshift-default New size: 5; reason: 63m Normal Started pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-d5lpp Container started 63m Normal Created pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-d5lpp Container created 63m Warning ProbeError pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-d5lpp Startup probe error: Get "http://10.133.0.98:15021/healthz/ready": dial tcp 10.133.0.98:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-d5lpp Startup probe failed: Get "http://10.133.0.98:15021/healthz/ready": dial tcp 10.133.0.98:15021: connect: connection refused 63m Normal SuccessfulCreate replicaset/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc Created pod: e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-c9dkh 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-012ba0-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 63m Normal ScalingReplicaSet deployment/e2e-worker-w6-012ba0-openshift-default Scaled up replica set e2e-worker-w6-012ba0-openshift-default-846bcc5bfc from 3 to 5 63m Normal SuccessfulCreate replicaset/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc Created pod: e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-v5lkq 63m Normal Created pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-c9dkh Container created 63m Normal Started pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-c9dkh Container started 63m Normal Pulled pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-c9dkh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Started pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-v5lkq Container started 63m Normal AddedInterface pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-c9dkh Add eth0 [10.132.0.71/23] from ovn-kubernetes 63m Normal AddedInterface pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-v5lkq Add eth0 [10.133.0.100/23] from ovn-kubernetes 63m Normal Pulled pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-v5lkq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Created pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-v5lkq Container created 63m Warning ProbeError pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-c9dkh Startup probe error: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused... 63m Warning ProbeError pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-v5lkq Startup probe error: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused... 63m Warning Unhealthy pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-v5lkq Startup probe failed: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused 63m Warning Unhealthy pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-c9dkh Startup probe failed: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused 63m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-54114e-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 63m Normal ScalingReplicaSet deployment/e2e-worker-w5-54114e-openshift-default Scaled up replica set e2e-worker-w5-54114e-openshift-default-7fc467ddd7 from 4 to 5 63m Normal SuccessfulCreate replicaset/e2e-worker-w5-54114e-openshift-default-7fc467ddd7 Created pod: e2e-worker-w5-54114e-openshift-default-7fc467ddd7-z58t9 63m Normal Created pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-z58t9 Container created 63m Normal Pulled pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-z58t9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-z58t9 Add eth0 [10.132.0.72/23] from ovn-kubernetes 63m Normal Started pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-z58t9 Container started 62m Warning Unhealthy pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-z58t9 Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-z58t9 Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused... 62m Normal SuccessfulCreate replicaset/e2e-worker-w3-d52c59-openshift-default-5466968fb7 Created pod: e2e-worker-w3-d52c59-openshift-default-5466968fb7-ssk97 62m Normal SuccessfulCreate replicaset/e2e-worker-w3-d52c59-openshift-default-5466968fb7 (combined from similar events): Created pod: e2e-worker-w3-d52c59-openshift-default-5466968fb7-95kvh 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-d52c59-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 62m Normal SuccessfulCreate replicaset/e2e-worker-w3-d52c59-openshift-default-5466968fb7 Created pod: e2e-worker-w3-d52c59-openshift-default-5466968fb7-gt2jk 62m Normal ScalingReplicaSet deployment/e2e-worker-w3-d52c59-openshift-default Scaled up replica set e2e-worker-w3-d52c59-openshift-default-5466968fb7 from 5 to 10 62m Normal SuccessfulCreate replicaset/e2e-worker-w3-d52c59-openshift-default-5466968fb7 Created pod: e2e-worker-w3-d52c59-openshift-default-5466968fb7-zr6tr 62m Normal SuccessfulCreate replicaset/e2e-worker-w3-d52c59-openshift-default-5466968fb7 Created pod: e2e-worker-w3-d52c59-openshift-default-5466968fb7-tbj69 62m Normal AddedInterface pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-zr6tr Add eth0 [10.132.0.73/23] from ovn-kubernetes 62m Normal Pulled pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-95kvh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Pulled pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-gt2jk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal ScalingReplicaSet deployment/e2e-worker-w1-2b1e88-openshift-default Scaled up replica set e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 from 3 to 4 62m Normal Created pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-gt2jk Container created 62m Normal Started pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-gt2jk Container started 62m Normal AddedInterface pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-gt2jk Add eth0 [10.134.0.63/23] from ovn-kubernetes 62m Normal Started pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-tbj69 Container started 62m Normal Created pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-tbj69 Container created 62m Normal Pulled pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-tbj69 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal AddedInterface pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-tbj69 Add eth0 [10.134.0.62/23] from ovn-kubernetes 62m Normal Started pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-95kvh Container started 62m Normal AddedInterface pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-mb6lc Add eth0 [10.133.0.103/23] from ovn-kubernetes 62m Normal Pulled pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-mb6lc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Started pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-ssk97 Container started 62m Normal Created pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-95kvh Container created 62m Normal Created pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-ssk97 Container created 62m Normal Pulled pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-zr6tr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Pulled pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-ssk97 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal AddedInterface pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-ssk97 Add eth0 [10.133.0.102/23] from ovn-kubernetes 62m Normal Created pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-zr6tr Container created 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-2b1e88-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 62m Normal Started pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-zr6tr Container started 62m Normal Created pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-mb6lc Container created 62m Normal Started pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-mb6lc Container started 62m Normal SuccessfulCreate replicaset/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 Created pod: e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-mb6lc 62m Normal AddedInterface pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-95kvh Add eth0 [10.132.0.74/23] from ovn-kubernetes 62m Warning ProbeError pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-gt2jk Startup probe error: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-zr6tr Startup probe failed: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-zr6tr Startup probe error: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused... 62m Warning ProbeError pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-ssk97 Startup probe error: Get "http://10.133.0.102:15021/healthz/ready": dial tcp 10.133.0.102:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-ssk97 Startup probe failed: Get "http://10.133.0.102:15021/healthz/ready": dial tcp 10.133.0.102:15021: connect: connection refused 62m Warning Unhealthy pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-tbj69 Startup probe failed: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-tbj69 Startup probe error: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-gt2jk Startup probe failed: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused 62m Warning Unhealthy pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-95kvh Startup probe failed: Get "http://10.132.0.74:15021/healthz/ready": dial tcp 10.132.0.74:15021: connect: connection refused 62m Warning Unhealthy pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-mb6lc Startup probe failed: Get "http://10.133.0.103:15021/healthz/ready": dial tcp 10.133.0.103:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-mb6lc Startup probe error: Get "http://10.133.0.103:15021/healthz/ready": dial tcp 10.133.0.103:15021: connect: connection refused... 62m Warning ProbeError pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-95kvh Startup probe error: Get "http://10.132.0.74:15021/healthz/ready": dial tcp 10.132.0.74:15021: connect: connection refused... 62m Normal SuccessfulCreate replicaset/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c Created pod: e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-9j5tr 62m Normal SuccessfulCreate replicaset/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c Created pod: e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-97nr4 62m Normal ScalingReplicaSet deployment/e2e-worker-w4-0ad7c5-openshift-default Scaled up replica set e2e-worker-w4-0ad7c5-openshift-default-788b484d5c from 3 to 6 62m Normal SuccessfulCreate replicaset/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c Created pod: e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hnqx6 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0ad7c5-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 62m Normal Created pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-m74tl Container created 62m Normal ScalingReplicaSet deployment/e2e-worker-w6-012ba0-openshift-default Scaled up replica set e2e-worker-w6-012ba0-openshift-default-846bcc5bfc from 5 to 9 62m Normal SuccessfulCreate replicaset/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc Created pod: e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-m74tl 62m Normal Started pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hnqx6 Container started 62m Normal Started pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-7wsq8 Container started 62m Normal Created pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-7wsq8 Container created 62m Normal Pulled pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-7wsq8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal AddedInterface pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-7wsq8 Add eth0 [10.132.0.76/23] from ovn-kubernetes 62m Normal Created pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hnqx6 Container created 62m Normal Pulled pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hnqx6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal AddedInterface pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hnqx6 Add eth0 [10.132.0.75/23] from ovn-kubernetes 62m Normal Started pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-5zllw Container started 62m Normal Created pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-5zllw Container created 62m Normal Pulled pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-5zllw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal AddedInterface pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-5zllw Add eth0 [10.133.0.107/23] from ovn-kubernetes 62m Normal Started pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-sqxvr Container started 62m Normal Started pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-9j5tr Container started 62m Normal Created pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-sqxvr Container created 62m Normal Created pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-9j5tr Container created 62m Normal Pulled pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-9j5tr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal AddedInterface pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-9j5tr Add eth0 [10.134.0.64/23] from ovn-kubernetes 62m Normal SuccessfulCreate replicaset/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc Created pod: e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-7wsq8 62m Normal AddedInterface pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-m74tl Add eth0 [10.134.0.66/23] from ovn-kubernetes 62m Normal Pulled pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-m74tl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Pulled pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-sqxvr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Started pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-m74tl Container started 62m Normal Started pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-97nr4 Container started 62m Normal Created pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-97nr4 Container created 62m Normal Pulled pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-97nr4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal SuccessfulCreate replicaset/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc Created pod: e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-5zllw 62m Normal AddedInterface pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-97nr4 Add eth0 [10.133.0.106/23] from ovn-kubernetes 62m Normal AddedInterface pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-sqxvr Add eth0 [10.134.0.65/23] from ovn-kubernetes 62m Normal SuccessfulCreate replicaset/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc Created pod: e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-sqxvr 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-012ba0-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 62m Warning ProbeError pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-m74tl Startup probe error: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-9j5tr Startup probe failed: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused 62m Warning Unhealthy pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-m74tl Startup probe failed: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-9j5tr Startup probe error: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused... 62m Warning ProbeError pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-7wsq8 Startup probe error: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-5zllw Startup probe failed: Get "http://10.133.0.107:15021/healthz/ready": dial tcp 10.133.0.107:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-5zllw Startup probe error: Get "http://10.133.0.107:15021/healthz/ready": dial tcp 10.133.0.107:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-7wsq8 Startup probe failed: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused 62m Warning Unhealthy pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-sqxvr Startup probe failed: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-sqxvr Startup probe error: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused... 62m Warning ProbeError pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hnqx6 Startup probe error: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-97nr4 Startup probe failed: Get "http://10.133.0.106:15021/healthz/ready": dial tcp 10.133.0.106:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-97nr4 Startup probe error: Get "http://10.133.0.106:15021/healthz/ready": dial tcp 10.133.0.106:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hnqx6 Startup probe failed: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused 62m Normal SuccessfulCreate replicaset/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c Created pod: e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hstmf 62m Normal ScalingReplicaSet deployment/e2e-worker-w0-09d0d5-openshift-default Scaled up replica set e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 from 5 to 10 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-09d0d5-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 62m Normal SuccessfulCreate replicaset/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 (combined from similar events): Created pod: e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-z7ggc 62m Normal SuccessfulCreate replicaset/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 Created pod: e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-m6s8k 62m Normal SuccessfulCreate replicaset/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 Created pod: e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-cnmb7 62m Normal ScalingReplicaSet deployment/e2e-worker-w4-0ad7c5-openshift-default Scaled up replica set e2e-worker-w4-0ad7c5-openshift-default-788b484d5c from 6 to 7 62m Normal SuccessfulCreate replicaset/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 Created pod: e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-zb2qv 62m Normal SuccessfulCreate replicaset/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 Created pod: e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-vtwp6 62m Normal AddedInterface pod/e2e-worker-w2-421583-openshift-default-5cb767b447-spjhs Add eth0 [10.134.0.69/23] from ovn-kubernetes 62m Normal AddedInterface pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-z7ggc Add eth0 [10.134.0.68/23] from ovn-kubernetes 62m Normal AddedInterface pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hstmf Add eth0 [10.133.0.112/23] from ovn-kubernetes 62m Normal Pulled pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hstmf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Created pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-vtwp6 Container created 62m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-421583-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 62m Normal ScalingReplicaSet deployment/e2e-worker-w2-421583-openshift-default Scaled up replica set e2e-worker-w2-421583-openshift-default-5cb767b447 from 2 to 3 62m Normal Pulled pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-cnmb7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal AddedInterface pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-zb2qv Add eth0 [10.132.0.78/23] from ovn-kubernetes 62m Normal Pulled pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-zb2qv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Pulled pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-vtwp6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Started pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-cnmb7 Container started 62m Normal Created pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-zb2qv Container created 62m Normal Created pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hstmf Container created 62m Normal AddedInterface pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-cnmb7 Add eth0 [10.133.0.111/23] from ovn-kubernetes 62m Normal Started pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hstmf Container started 62m Normal Started pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-z7ggc Container started 62m Normal Started pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-vtwp6 Container started 62m Normal AddedInterface pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-vtwp6 Add eth0 [10.134.0.67/23] from ovn-kubernetes 62m Normal Created pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-cnmb7 Container created 62m Normal Started pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-zb2qv Container started 62m Normal Created pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-z7ggc Container created 62m Normal SuccessfulCreate replicaset/e2e-worker-w2-421583-openshift-default-5cb767b447 Created pod: e2e-worker-w2-421583-openshift-default-5cb767b447-spjhs 62m Normal Pulled pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-z7ggc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal AddedInterface pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-m6s8k Add eth0 [10.132.0.79/23] from ovn-kubernetes 62m Normal Pulled pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-m6s8k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Started pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-m6s8k Container started 62m Normal Created pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-m6s8k Container created 62m Normal Pulled pod/e2e-worker-w2-421583-openshift-default-5cb767b447-spjhs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Created pod/e2e-worker-w2-421583-openshift-default-5cb767b447-spjhs Container created 62m Normal Started pod/e2e-worker-w2-421583-openshift-default-5cb767b447-spjhs Container started 62m Warning ProbeError pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-z7ggc Startup probe error: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-vtwp6 Startup probe failed: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-m6s8k Startup probe error: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-z7ggc Startup probe failed: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-vtwp6 Startup probe error: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-m6s8k Startup probe failed: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused 62m Warning Unhealthy pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-zb2qv Startup probe failed: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-zb2qv Startup probe error: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-cnmb7 Startup probe failed: Get "http://10.133.0.111:15021/healthz/ready": dial tcp 10.133.0.111:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-cnmb7 Startup probe error: Get "http://10.133.0.111:15021/healthz/ready": dial tcp 10.133.0.111:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-m6s8k Startup probe failed: Get "http://10.132.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 62m Warning Unhealthy pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hstmf Startup probe failed: Get "http://10.133.0.112:15021/healthz/ready": dial tcp 10.133.0.112:15021: connect: connection refused 62m Warning Unhealthy pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-vtwp6 Startup probe failed: Get "http://10.134.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 62m Warning ProbeError pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-m6s8k Startup probe error: Get "http://10.132.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 62m Warning ProbeError pod/e2e-worker-w2-421583-openshift-default-5cb767b447-spjhs Startup probe error: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused... 62m Warning ProbeError pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hstmf Startup probe error: Get "http://10.133.0.112:15021/healthz/ready": dial tcp 10.133.0.112:15021: connect: connection refused... 62m Warning Unhealthy pod/e2e-worker-w2-421583-openshift-default-5cb767b447-spjhs Startup probe failed: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused 62m Warning ProbeError pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-vtwp6 Startup probe error: Get "http://10.134.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 62m Warning ProbeError pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-cnmb7 Startup probe error: Get "http://10.133.0.111:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 62m Warning Unhealthy pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-cnmb7 Startup probe failed: Get "http://10.133.0.111:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 62m Warning ProbeError pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-tdvdc Readiness probe error: Get "http://10.133.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 62m Warning Unhealthy pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-tdvdc Readiness probe failed: Get "http://10.133.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 62m Warning ProbeError pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-zb2qv Startup probe error: Get "http://10.132.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 62m Warning Unhealthy pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-zb2qv Startup probe failed: Get "http://10.132.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 62m Warning ProbeError pod/e2e-worker-w2-421583-openshift-default-5cb767b447-spjhs Startup probe error: Get "http://10.134.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 62m Warning Unhealthy pod/e2e-worker-w2-421583-openshift-default-5cb767b447-spjhs Startup probe failed: Get "http://10.134.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 62m Warning Unhealthy pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hstmf Startup probe failed: Get "http://10.133.0.112:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 62m Warning ProbeError pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hstmf Startup probe error: Get "http://10.133.0.112:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Normal ScalingReplicaSet deployment/e2e-worker-w6-012ba0-openshift-default Scaled up replica set e2e-worker-w6-012ba0-openshift-default-846bcc5bfc from 9 to 10 61m Normal SuccessfulCreate replicaset/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc (combined from similar events): Created pod: e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-n9dj2 61m Normal SuccessfulCreate replicaset/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 Created pod: e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-47dz7 61m Normal SuccessfulCreate replicaset/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 Created pod: e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-58wkt 61m Normal SuccessfulCreate replicaset/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 Created pod: e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-crp67 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-012ba0-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-2b1e88-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 61m Normal ScalingReplicaSet deployment/e2e-worker-w1-2b1e88-openshift-default Scaled up replica set e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 from 4 to 7 61m Normal AddedInterface pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-crp67 Add eth0 [10.134.0.70/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-crp67 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-47dz7 Add eth0 [10.133.0.118/23] from ovn-kubernetes 61m Normal Created pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-crp67 Container created 61m Normal Started pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-crp67 Container started 61m Normal Pulled pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-47dz7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-n9dj2 Add eth0 [10.132.0.80/23] from ovn-kubernetes 61m Normal AddedInterface pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-58wkt Add eth0 [10.132.0.81/23] from ovn-kubernetes 61m Normal Started pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-58wkt Container started 61m Normal Created pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-47dz7 Container created 61m Normal Created pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-58wkt Container created 61m Normal Pulled pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-58wkt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Started pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-n9dj2 Container started 61m Normal Created pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-n9dj2 Container created 61m Normal Started pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-47dz7 Container started 61m Normal Pulled pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-n9dj2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Warning ProbeError pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-crp67 Startup probe error: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-crp67 Startup probe failed: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-58wkt Startup probe error: Get "http://10.132.0.81:15021/healthz/ready": dial tcp 10.132.0.81:15021: connect: connection refused... 61m Warning ProbeError pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-47dz7 Startup probe error: Get "http://10.133.0.118:15021/healthz/ready": dial tcp 10.133.0.118:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-47dz7 Startup probe failed: Get "http://10.133.0.118:15021/healthz/ready": dial tcp 10.133.0.118:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-n9dj2 Startup probe error: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-n9dj2 Startup probe failed: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused 61m Warning Unhealthy pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-58wkt Startup probe failed: Get "http://10.132.0.81:15021/healthz/ready": dial tcp 10.132.0.81:15021: connect: connection refused 61m Normal SuccessfulCreate replicaset/e2e-worker-w5-54114e-openshift-default-7fc467ddd7 Created pod: e2e-worker-w5-54114e-openshift-default-7fc467ddd7-57vcj 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-54114e-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 61m Normal ScalingReplicaSet deployment/e2e-worker-w5-54114e-openshift-default Scaled up replica set e2e-worker-w5-54114e-openshift-default-7fc467ddd7 from 5 to 8 61m Normal SuccessfulCreate replicaset/e2e-worker-w5-54114e-openshift-default-7fc467ddd7 Created pod: e2e-worker-w5-54114e-openshift-default-7fc467ddd7-8r62w 61m Normal SuccessfulCreate replicaset/e2e-worker-w5-54114e-openshift-default-7fc467ddd7 Created pod: e2e-worker-w5-54114e-openshift-default-7fc467ddd7-p4vsn 61m Normal Started pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-gwz9t Container started 61m Normal AddedInterface pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-rpk4x Add eth0 [10.134.0.73/23] from ovn-kubernetes 61m Normal Created pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-8r62w Container created 61m Normal Started pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-8r62w Container started 61m Normal AddedInterface pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-8r62w Add eth0 [10.132.0.82/23] from ovn-kubernetes 61m Normal AddedInterface pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-p4vsn Add eth0 [10.134.0.72/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-p4vsn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Created pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-p4vsn Container created 61m Normal SuccessfulCreate replicaset/e2e-worker-w2-421583-openshift-default-5cb767b447 Created pod: e2e-worker-w2-421583-openshift-default-5cb767b447-l4c4h 61m Normal SuccessfulCreate replicaset/e2e-worker-w2-421583-openshift-default-5cb767b447 Created pod: e2e-worker-w2-421583-openshift-default-5cb767b447-xkmrr 61m Normal Started pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-p4vsn Container started 61m Normal Started pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-57vcj Container started 61m Normal Created pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-57vcj Container created 61m Normal Pulled pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-57vcj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-57vcj Add eth0 [10.133.0.122/23] from ovn-kubernetes 61m Normal ScalingReplicaSet deployment/e2e-worker-w4-0ad7c5-openshift-default Scaled up replica set e2e-worker-w4-0ad7c5-openshift-default-788b484d5c from 7 to 10 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0ad7c5-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 61m Normal AddedInterface pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8wzt7 Add eth0 [10.132.0.83/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8wzt7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-421583-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 61m Normal Created pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8wzt7 Container created 61m Normal Started pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8wzt7 Container started 61m Normal Pulled pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-8r62w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-gwz9t Add eth0 [10.133.0.123/23] from ovn-kubernetes 61m Normal Pulled pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-gwz9t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal ScalingReplicaSet deployment/e2e-worker-w2-421583-openshift-default Scaled up replica set e2e-worker-w2-421583-openshift-default-5cb767b447 from 3 to 5 61m Normal Started pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-rpk4x Container started 61m Normal Created pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-rpk4x Container created 61m Normal Pulled pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-rpk4x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Created pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-gwz9t Container created 61m Normal SuccessfulCreate replicaset/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c (combined from similar events): Created pod: e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-gwz9t 61m Normal SuccessfulCreate replicaset/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c Created pod: e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-rpk4x 61m Normal SuccessfulCreate replicaset/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c Created pod: e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8wzt7 61m Normal Created pod/e2e-worker-w2-421583-openshift-default-5cb767b447-l4c4h Container created 61m Normal AddedInterface pod/e2e-worker-w2-421583-openshift-default-5cb767b447-l4c4h Add eth0 [10.133.0.124/23] from ovn-kubernetes 61m Normal Started pod/e2e-worker-w2-421583-openshift-default-5cb767b447-xkmrr Container started 61m Normal Created pod/e2e-worker-w2-421583-openshift-default-5cb767b447-xkmrr Container created 61m Normal Pulled pod/e2e-worker-w2-421583-openshift-default-5cb767b447-xkmrr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/e2e-worker-w2-421583-openshift-default-5cb767b447-xkmrr Add eth0 [10.132.0.84/23] from ovn-kubernetes 61m Normal Started pod/e2e-worker-w2-421583-openshift-default-5cb767b447-l4c4h Container started 61m Normal Pulled pod/e2e-worker-w2-421583-openshift-default-5cb767b447-l4c4h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Warning Unhealthy pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-rpk4x Startup probe failed: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused 61m Warning Unhealthy pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8wzt7 Startup probe failed: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8wzt7 Startup probe error: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-p4vsn Startup probe failed: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-p4vsn Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused... 61m Warning ProbeError pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-rpk4x Startup probe error: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused... 61m Warning ProbeError pod/e2e-worker-w2-421583-openshift-default-5cb767b447-l4c4h Startup probe error: Get "http://10.133.0.124:15021/healthz/ready": dial tcp 10.133.0.124:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w2-421583-openshift-default-5cb767b447-xkmrr Startup probe failed: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-8r62w Startup probe error: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-gwz9t Startup probe failed: Get "http://10.133.0.123:15021/healthz/ready": dial tcp 10.133.0.123:15021: connect: connection refused 61m Warning Unhealthy pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-57vcj Startup probe failed: Get "http://10.133.0.122:15021/healthz/ready": dial tcp 10.133.0.122:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-57vcj Startup probe error: Get "http://10.133.0.122:15021/healthz/ready": dial tcp 10.133.0.122:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-8r62w Startup probe failed: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w2-421583-openshift-default-5cb767b447-xkmrr Startup probe error: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w2-421583-openshift-default-5cb767b447-l4c4h Startup probe failed: Get "http://10.133.0.124:15021/healthz/ready": dial tcp 10.133.0.124:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-gwz9t Startup probe error: Get "http://10.133.0.123:15021/healthz/ready": dial tcp 10.133.0.123:15021: connect: connection refused... 61m Warning ProbeError pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-2w4gw Readiness probe error: Get "http://10.132.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning ProbeError pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-cjkc9 Readiness probe error: Get "http://10.134.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning Unhealthy pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-2w4gw Readiness probe failed: Get "http://10.132.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning Unhealthy pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-72ws5 Readiness probe failed: Get "http://10.133.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning Unhealthy pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-cjkc9 Readiness probe failed: Get "http://10.134.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning ProbeError pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-72ws5 Readiness probe error: Get "http://10.133.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning ProbeError pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-sqxvr Readiness probe error: Get "http://10.134.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning ProbeError pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hq9vz Readiness probe error: Get "http://10.134.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning Unhealthy pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-c9dkh Readiness probe failed: Get "http://10.132.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning ProbeError pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-rpk4x Readiness probe error: Get "http://10.134.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning ProbeError pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-c9dkh Readiness probe error: Get "http://10.132.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning Unhealthy pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-m74tl Readiness probe failed: Get "http://10.134.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning ProbeError pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-m74tl Readiness probe error: Get "http://10.134.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning ProbeError pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-svsgv Readiness probe error: Get "http://10.133.0.95:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning Unhealthy pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-sqxvr Readiness probe failed: Get "http://10.134.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning Unhealthy pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hq9vz Readiness probe failed: Get "http://10.134.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning Unhealthy pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-v5lkq Readiness probe failed: Get "http://10.133.0.100:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning Unhealthy pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-mb6lc Readiness probe failed: Get "http://10.133.0.103:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning Unhealthy pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-z7ggc Readiness probe failed: Get "http://10.134.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning ProbeError pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-z7ggc Readiness probe error: Get "http://10.134.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning ProbeError pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-9j5tr Readiness probe error: Get "http://10.134.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning Unhealthy pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-9j5tr Readiness probe failed: Get "http://10.134.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning ProbeError pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-v5lkq Readiness probe error: Get "http://10.133.0.100:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning ProbeError pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-mb6lc Readiness probe error: Get "http://10.133.0.103:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning Unhealthy pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-rpk4x Readiness probe failed: Get "http://10.134.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning Unhealthy pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-svsgv Readiness probe failed: Get "http://10.133.0.95:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning Unhealthy pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-7wsq8 Readiness probe failed: Get "http://10.132.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning ProbeError pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-7wsq8 Readiness probe error: Get "http://10.132.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning ProbeError pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-vtwp6 Readiness probe error: Get "http://10.134.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning ProbeError pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-8r62w Readiness probe error: Get "http://10.132.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning Unhealthy pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-57vcj Readiness probe failed: Get "http://10.133.0.122:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning Unhealthy pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-8r62w Readiness probe failed: Get "http://10.132.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning ProbeError pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-lg99w Readiness probe error: Get "http://10.134.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning ProbeError pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-57vcj Readiness probe error: Get "http://10.133.0.122:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning Unhealthy pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hnqx6 Readiness probe failed: Get "http://10.132.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning Unhealthy pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-95kvh Readiness probe failed: Get "http://10.132.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning ProbeError pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hnqx6 Readiness probe error: Get "http://10.132.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning Unhealthy pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-gwz9t Readiness probe failed: Get "http://10.133.0.123:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning ProbeError pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-gwz9t Readiness probe error: Get "http://10.133.0.123:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning Unhealthy pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-lg99w Readiness probe failed: Get "http://10.134.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning ProbeError pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-95kvh Readiness probe error: Get "http://10.132.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning Unhealthy pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-97nr4 Readiness probe failed: Get "http://10.133.0.106:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning ProbeError pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-97nr4 Readiness probe error: Get "http://10.133.0.106:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning Unhealthy pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8wzt7 Readiness probe failed: Get "http://10.132.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning ProbeError pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8wzt7 Readiness probe error: Get "http://10.132.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning Unhealthy pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-5zllw Readiness probe failed: Get "http://10.133.0.107:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning Unhealthy pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-vtwp6 Readiness probe failed: Get "http://10.134.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning ProbeError pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-5zllw Readiness probe error: Get "http://10.133.0.107:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning ProbeError pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-zb2qv Readiness probe error: Get "http://10.132.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning Unhealthy pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-zb2qv Readiness probe failed: Get "http://10.132.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 61m Warning ProbeError pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-w2q4t Readiness probe error: Get "http://10.132.0.45:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 61m Warning Unhealthy pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-w2q4t Readiness probe failed: Get "http://10.132.0.45:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Normal AddedLabel gateway/e2e-derive-36d7ee39 Added label istio.io/rev=openshift-gateway to gateway e2e-derive-36d7ee39 60m Normal AddedLabel gateway/e2e-ait-1ba9ae08 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-1ba9ae08 60m Normal SuccessfulCreate replicaset/e2e-ait-ca259734-openshift-default-64cb5dc965 Created pod: e2e-ait-ca259734-openshift-default-64cb5dc965-4f6sz 60m Normal AddedInterface pod/e2e-ait-ca259734-openshift-default-64cb5dc965-4f6sz Add eth0 [10.133.0.138/23] from ovn-kubernetes 60m Normal AddedLabel gateway/e2e-ait-ca259734 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-ca259734 60m Normal ScalingReplicaSet deployment/e2e-ait-ca259734-openshift-default Scaled up replica set e2e-ait-ca259734-openshift-default-64cb5dc965 from 0 to 1 60m Normal ScalingReplicaSet deployment/e2e-derive-36d7ee39-openshift-default Scaled up replica set e2e-derive-36d7ee39-openshift-default-6b7cbf646f from 0 to 1 60m Normal ScalingReplicaSet deployment/e2e-ait-1ba9ae08-openshift-default Scaled up replica set e2e-ait-1ba9ae08-openshift-default-7758f9c76c from 0 to 1 60m Normal SuccessfulCreate replicaset/e2e-ait-1ba9ae08-openshift-default-7758f9c76c Created pod: e2e-ait-1ba9ae08-openshift-default-7758f9c76c-wbv2b 60m Normal SuccessfulCreate replicaset/e2e-derive-36d7ee39-openshift-default-6b7cbf646f Created pod: e2e-derive-36d7ee39-openshift-default-6b7cbf646f-nnh2m 60m Normal AddedInterface pod/e2e-ait-1ba9ae08-openshift-default-7758f9c76c-wbv2b Add eth0 [10.133.0.139/23] from ovn-kubernetes 60m Normal AddedInterface pod/e2e-derive-36d7ee39-openshift-default-6b7cbf646f-nnh2m Add eth0 [10.132.0.85/23] from ovn-kubernetes 60m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-ca259734 Scaled up replica set payload-pre-processing-e2e-ait-ca259734-86c66dfcf9 from 0 to 1 60m Warning FailedCreate replicaset/payload-processing-e2e-ait-ca259734-6d97f789c5 Error creating: pods "payload-processing-e2e-ait-ca259734-6d97f789c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-ca259734: serviceaccount "payload-processing-e2e-ait-ca259734" not found 60m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-ca259734-6d97f789c5 Created pod: payload-processing-e2e-ait-ca259734-6d97f789c5-dfxt8 60m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-ca259734 Scaled up replica set payload-processing-e2e-ait-ca259734-6d97f789c5 from 0 to 1 60m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-ca259734-86c66dfcf9 Created pod: payload-pre-processing-e2e-ait-ca259734-86c66dfcf9-5dzzk 60m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-ca259734-86c66dfcf9 Error creating: pods "payload-pre-processing-e2e-ait-ca259734-86c66dfcf9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-ca259734: serviceaccount "payload-processing-e2e-ait-ca259734" not found 60m Normal Pulling pod/payload-processing-e2e-ait-ca259734-6d97f789c5-dfxt8 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 60m Normal AddedInterface pod/payload-processing-e2e-ait-ca259734-6d97f789c5-dfxt8 Add eth0 [10.134.0.79/23] from ovn-kubernetes 60m Normal Pulling pod/payload-pre-processing-e2e-ait-ca259734-86c66dfcf9-5dzzk Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 60m Normal AddedInterface pod/payload-pre-processing-e2e-ait-ca259734-86c66dfcf9-5dzzk Add eth0 [10.134.0.80/23] from ovn-kubernetes 60m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-36d7ee39 Scaled up replica set payload-processing-e2e-derive-36d7ee39-5cf5d67547 from 0 to 1 60m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-36d7ee39-69869b5bc Error creating: pods "payload-pre-processing-e2e-derive-36d7ee39-69869b5bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-36d7ee39: serviceaccount "payload-processing-e2e-derive-36d7ee39" not found 60m Warning FailedCreate replicaset/payload-processing-e2e-derive-36d7ee39-5cf5d67547 Error creating: pods "payload-processing-e2e-derive-36d7ee39-5cf5d67547-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-36d7ee39: serviceaccount "payload-processing-e2e-derive-36d7ee39" not found 60m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-36d7ee39 Scaled up replica set payload-pre-processing-e2e-derive-36d7ee39-69869b5bc from 0 to 1 60m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-1ba9ae08 Scaled up replica set payload-processing-e2e-ait-1ba9ae08-5fdd496c5c from 0 to 1 60m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-1ba9ae08-6847f996c5 Created pod: payload-pre-processing-e2e-ait-1ba9ae08-6847f996c5-n5b48 60m Warning FailedCreate replicaset/payload-processing-e2e-ait-1ba9ae08-5fdd496c5c Error creating: pods "payload-processing-e2e-ait-1ba9ae08-5fdd496c5c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-1ba9ae08: serviceaccount "payload-processing-e2e-ait-1ba9ae08" not found 60m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-36d7ee39-69869b5bc Created pod: payload-pre-processing-e2e-derive-36d7ee39-69869b5bc-7jmws 60m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-36d7ee39-5cf5d67547 Created pod: payload-processing-e2e-derive-36d7ee39-5cf5d67547-bbzkn 60m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-1ba9ae08-6847f996c5 Error creating: pods "payload-pre-processing-e2e-ait-1ba9ae08-6847f996c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-1ba9ae08: serviceaccount "payload-processing-e2e-ait-1ba9ae08" not found 60m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-1ba9ae08 Scaled up replica set payload-pre-processing-e2e-ait-1ba9ae08-6847f996c5 from 0 to 1 60m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-1ba9ae08-5fdd496c5c Created pod: payload-processing-e2e-ait-1ba9ae08-5fdd496c5c-4sz29 60m Normal Started pod/payload-pre-processing-e2e-ait-ca259734-86c66dfcf9-5dzzk Container started 60m Normal AddedInterface pod/payload-processing-e2e-derive-36d7ee39-5cf5d67547-bbzkn Add eth0 [10.132.0.86/23] from ovn-kubernetes 60m Normal Pulled pod/payload-processing-e2e-ait-ca259734-6d97f789c5-dfxt8 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.845s (2.845s including waiting). Image size: 125353965 bytes. 60m Normal Created pod/payload-processing-e2e-ait-ca259734-6d97f789c5-dfxt8 Container created 60m Normal Started pod/payload-processing-e2e-ait-ca259734-6d97f789c5-dfxt8 Container started 60m Normal Pulled pod/payload-processing-e2e-derive-36d7ee39-5cf5d67547-bbzkn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 60m Normal Created pod/payload-pre-processing-e2e-ait-1ba9ae08-6847f996c5-n5b48 Container created 60m Normal Pulled pod/payload-pre-processing-e2e-derive-36d7ee39-69869b5bc-7jmws Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 60m Normal Pulled pod/payload-pre-processing-e2e-ait-1ba9ae08-6847f996c5-n5b48 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 60m Normal Pulled pod/payload-pre-processing-e2e-ait-ca259734-86c66dfcf9-5dzzk Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.712s (2.712s including waiting). Image size: 125353965 bytes. 60m Normal Started pod/payload-pre-processing-e2e-derive-36d7ee39-69869b5bc-7jmws Container started 60m Normal Created pod/payload-pre-processing-e2e-ait-ca259734-86c66dfcf9-5dzzk Container created 60m Normal Created pod/payload-pre-processing-e2e-derive-36d7ee39-69869b5bc-7jmws Container created 60m Normal Created pod/payload-processing-e2e-derive-36d7ee39-5cf5d67547-bbzkn Container created 60m Normal AddedInterface pod/payload-pre-processing-e2e-ait-1ba9ae08-6847f996c5-n5b48 Add eth0 [10.132.0.88/23] from ovn-kubernetes 60m Normal AddedInterface pod/payload-pre-processing-e2e-derive-36d7ee39-69869b5bc-7jmws Add eth0 [10.133.0.140/23] from ovn-kubernetes 60m Normal Created pod/payload-processing-e2e-ait-1ba9ae08-5fdd496c5c-4sz29 Container created 60m Normal Started pod/payload-processing-e2e-ait-1ba9ae08-5fdd496c5c-4sz29 Container started 60m Normal Started pod/payload-processing-e2e-derive-36d7ee39-5cf5d67547-bbzkn Container started 60m Normal AddedInterface pod/payload-processing-e2e-ait-1ba9ae08-5fdd496c5c-4sz29 Add eth0 [10.132.0.87/23] from ovn-kubernetes 60m Normal Started pod/payload-pre-processing-e2e-ait-1ba9ae08-6847f996c5-n5b48 Container started 60m Normal Pulled pod/payload-processing-e2e-ait-1ba9ae08-5fdd496c5c-4sz29 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-1ba9ae08-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-1ba9ae08-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning Unhealthy pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8dgzg Readiness probe failed: Get "http://10.132.0.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8dgzg Readiness probe error: Get "http://10.132.0.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning Unhealthy pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-crp67 Readiness probe failed: Get "http://10.134.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-crp67 Readiness probe error: Get "http://10.134.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning ProbeError pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-kzdls Readiness probe error: Get "http://10.133.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning Unhealthy pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-kzdls Readiness probe failed: Get "http://10.133.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning Unhealthy pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-58wkt Readiness probe failed: Get "http://10.132.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-58wkt Readiness probe error: Get "http://10.132.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Normal SuccessfulCreate replicaset/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 Created pod: e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-nflrt 60m Normal SuccessfulCreate replicaset/e2e-worker-w5-54114e-openshift-default-7fc467ddd7 Created pod: e2e-worker-w5-54114e-openshift-default-7fc467ddd7-6wn2g 60m Normal SuccessfulCreate replicaset/e2e-worker-w5-54114e-openshift-default-7fc467ddd7 (combined from similar events): Created pod: e2e-worker-w5-54114e-openshift-default-7fc467ddd7-97d66 60m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-54114e-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 60m Normal ScalingReplicaSet deployment/e2e-worker-w5-54114e-openshift-default Scaled up replica set e2e-worker-w5-54114e-openshift-default-7fc467ddd7 from 8 to 10 60m Normal SuccessfulCreate replicaset/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 Created pod: e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-n6fb8 60m Normal SuccessfulCreate replicaset/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 (combined from similar events): Created pod: e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-6d9p2 60m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-2b1e88-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 60m Normal ScalingReplicaSet deployment/e2e-worker-w1-2b1e88-openshift-default Scaled up replica set e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 from 7 to 10 60m Warning Unhealthy pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-n9dj2 Readiness probe failed: Get "http://10.132.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-n9dj2 Readiness probe error: Get "http://10.132.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning ProbeError pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-cnmb7 Readiness probe error: Get "http://10.133.0.111:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning Unhealthy pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-47dz7 Readiness probe failed: Get "http://10.133.0.118:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-47dz7 Readiness probe error: Get "http://10.133.0.118:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning Unhealthy pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-cnmb7 Readiness probe failed: Get "http://10.133.0.111:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Normal Created pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-n6fb8 Container created 60m Normal Created pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-6wn2g Container created 60m Normal AddedInterface pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-n6fb8 Add eth0 [10.134.0.82/23] from ovn-kubernetes 60m Normal AddedInterface pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-6wn2g Add eth0 [10.134.0.81/23] from ovn-kubernetes 60m Normal Pulled pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-6wn2g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Pulled pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-n6fb8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Started pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-n6fb8 Container started 60m Normal Started pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-6wn2g Container started 60m Warning Unhealthy pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-qhw2n Readiness probe failed: Get "http://10.133.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Warning ProbeError pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-qhw2n Readiness probe error: Get "http://10.133.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Normal Pulled pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-nflrt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Created pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-nflrt Container created 60m Normal Started pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-nflrt Container started 60m Normal AddedInterface pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-nflrt Add eth0 [10.132.0.89/23] from ovn-kubernetes 60m Normal AddedInterface pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-97d66 Add eth0 [10.133.0.142/23] from ovn-kubernetes 60m Normal Started pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-6d9p2 Container started 60m Normal Pulled pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-97d66 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Created pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-97d66 Container created 60m Normal Started pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-97d66 Container started 60m Normal AddedInterface pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-6d9p2 Add eth0 [10.133.0.143/23] from ovn-kubernetes 60m Normal Pulled pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-6d9p2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Created pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-6d9p2 Container created 60m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: cpu resource utilization (percentage of request) above target 60m Warning ProbeError pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-6wn2g Startup probe error: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused... 60m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-47qt2 60m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-wx4cl 60m Warning Unhealthy pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-6wn2g Startup probe failed: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused 60m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 3 60m Warning ProbeError pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-n6fb8 Startup probe error: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-n6fb8 Startup probe failed: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused 60m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-wx4cl Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 60m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-wx4cl Add eth0 [10.134.0.85/23] from ovn-kubernetes 60m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-47qt2 Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 60m Warning Unhealthy pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-nflrt Startup probe failed: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-nflrt Startup probe error: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused... 60m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-47qt2 Add eth0 [10.133.0.144/23] from ovn-kubernetes 60m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-ca259734-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-ca259734-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-36d7ee39-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Warning Unhealthy pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-6d9p2 Startup probe failed: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused 60m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-wx4cl Container created 60m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-wx4cl Container started 60m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-36d7ee39-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Warning ProbeError pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-6d9p2 Startup probe error: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused... 60m Warning ProbeError pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-97d66 Startup probe error: Get "http://10.133.0.142:15021/healthz/ready": dial tcp 10.133.0.142:15021: connect: connection refused... 60m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-wx4cl Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.854s (2.854s including waiting). Image size: 185476477 bytes. 60m Warning Unhealthy pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-97d66 Startup probe failed: Get "http://10.133.0.142:15021/healthz/ready": dial tcp 10.133.0.142:15021: connect: connection refused 60m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-wx4cl Readiness probe error: HTTP probe failed with statuscode: 503... 60m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-wx4cl Readiness probe failed: HTTP probe failed with statuscode: 503 60m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-47qt2 Container created 60m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-47qt2 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.049s (4.049s including waiting). Image size: 185476477 bytes. 60m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-47qt2 Container started 60m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-47qt2 Readiness probe failed: HTTP probe failed with statuscode: 503 60m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-47qt2 Readiness probe error: HTTP probe failed with statuscode: 503... 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-sv4fp Readiness probe failed: Get "http://10.133.0.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-sv4fp Readiness probe error: Get "http://10.133.0.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-vbv8b Readiness probe failed: Get "http://10.133.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-vbv8b Readiness probe error: Get "http://10.133.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Normal Started pod/e2e-derive-36d7ee39-openshift-default-6b7cbf646f-nnh2m Container started 59m Warning Unhealthy pod/e2e-ait-ca259734-openshift-default-64cb5dc965-4f6sz Readiness probe failed: Get "http://10.133.0.138:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Warning ProbeError pod/e2e-ait-ca259734-openshift-default-64cb5dc965-4f6sz Readiness probe error: Get "http://10.133.0.138:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Normal Pulled pod/e2e-derive-36d7ee39-openshift-default-6b7cbf646f-nnh2m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/e2e-derive-36d7ee39-openshift-default-6b7cbf646f-nnh2m Container created 59m Warning ProbeError pod/e2e-ait-1ba9ae08-openshift-default-7758f9c76c-wbv2b Readiness probe error: Get "http://10.133.0.139:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 59m Warning Unhealthy pod/e2e-ait-1ba9ae08-openshift-default-7758f9c76c-wbv2b Readiness probe failed: Get "http://10.133.0.139:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 59m Normal Killing pod/payload-processing-e2e-ait-1ba9ae08-5fdd496c5c-4sz29 Stopping container payload-processing 59m Normal Killing pod/payload-pre-processing-e2e-ait-1ba9ae08-6847f996c5-n5b48 Stopping container payload-pre-processing 59m Normal Pulled pod/e2e-ait-1ba9ae08-openshift-default-7758f9c76c-wbv2b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Started pod/e2e-ait-1ba9ae08-openshift-default-7758f9c76c-wbv2b Container started 59m Normal Created pod/e2e-ait-1ba9ae08-openshift-default-7758f9c76c-wbv2b Container created 59m Normal Pulled pod/e2e-ait-ca259734-openshift-default-64cb5dc965-4f6sz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/e2e-ait-ca259734-openshift-default-64cb5dc965-4f6sz Container created 59m Normal Started pod/e2e-ait-ca259734-openshift-default-64cb5dc965-4f6sz Container started 59m Warning ProbeError pod/e2e-derive-36d7ee39-openshift-default-6b7cbf646f-nnh2m Startup probe error: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-derive-36d7ee39-openshift-default-6b7cbf646f-nnh2m Startup probe failed: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused 59m Normal Killing pod/payload-processing-e2e-ait-ca259734-6d97f789c5-dfxt8 Stopping container payload-processing 59m Normal Killing pod/payload-pre-processing-e2e-ait-ca259734-86c66dfcf9-5dzzk Stopping container payload-pre-processing 59m Normal Killing pod/payload-processing-e2e-derive-36d7ee39-5cf5d67547-bbzkn Stopping container payload-processing 59m Normal Killing pod/payload-pre-processing-e2e-derive-36d7ee39-69869b5bc-7jmws Stopping container payload-pre-processing 59m Warning Unhealthy pod/e2e-ait-1ba9ae08-openshift-default-7758f9c76c-wbv2b Startup probe failed: Get "http://10.133.0.139:15021/healthz/ready": dial tcp 10.133.0.139:15021: connect: connection refused 59m Warning ProbeError pod/e2e-ait-1ba9ae08-openshift-default-7758f9c76c-wbv2b Startup probe error: Get "http://10.133.0.139:15021/healthz/ready": dial tcp 10.133.0.139:15021: connect: connection refused... 59m Warning Unhealthy pod/e2e-ait-ca259734-openshift-default-64cb5dc965-4f6sz Startup probe failed: Get "http://10.133.0.138:15021/healthz/ready": dial tcp 10.133.0.138:15021: connect: connection refused 59m Normal Killing pod/e2e-ait-ca259734-openshift-default-64cb5dc965-4f6sz Stopping container istio-proxy 59m Warning ProbeError pod/e2e-ait-ca259734-openshift-default-64cb5dc965-4f6sz Startup probe error: Get "http://10.133.0.138:15021/healthz/ready": dial tcp 10.133.0.138:15021: connect: connection refused... 59m Normal AddedLabel gateway/e2e-migrate-476e593d-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-476e593d-gw 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-1ba9ae08-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 59m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-1ba9ae08-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 59m Normal ScalingReplicaSet deployment/e2e-migrate-476e593d-gw-openshift-default Scaled up replica set e2e-migrate-476e593d-gw-openshift-default-789fdfb6d5 from 0 to 1 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-36d7ee39-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-36d7ee39-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Normal SuccessfulCreate replicaset/e2e-migrate-476e593d-gw-openshift-default-789fdfb6d5 Created pod: e2e-migrate-476e593d-gw-openshift-default-789fdfb6d5-k5k6v 59m Normal AddedInterface pod/e2e-migrate-476e593d-gw-openshift-default-789fdfb6d5-k5k6v Add eth0 [10.132.0.91/23] from ovn-kubernetes 59m Normal Created pod/e2e-migrate-476e593d-gw-openshift-default-789fdfb6d5-k5k6v Container created 59m Normal Started pod/e2e-migrate-476e593d-gw-openshift-default-789fdfb6d5-k5k6v Container started 59m Warning FailedToRetrieveImagePullSecret pod/e2e-ait-1ba9ae08-openshift-default-7758f9c76c-wbv2b Unable to retrieve some image pull secrets (e2e-ait-1ba9ae08-openshift-default-dockercfg-nnpzt); attempting to pull the image may not succeed. 59m Normal Pulled pod/e2e-migrate-476e593d-gw-openshift-default-789fdfb6d5-k5k6v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal AddedLabel gateway/e2e-ait-913eddcb Added label istio.io/rev=openshift-gateway to gateway e2e-ait-913eddcb 59m Warning Unhealthy pod/e2e-migrate-476e593d-gw-openshift-default-789fdfb6d5-k5k6v Startup probe failed: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused 59m Normal ScalingReplicaSet deployment/e2e-ait-913eddcb-openshift-default Scaled up replica set e2e-ait-913eddcb-openshift-default-546bdfbdc5 from 0 to 1 59m Normal SuccessfulCreate replicaset/e2e-ait-913eddcb-openshift-default-546bdfbdc5 Created pod: e2e-ait-913eddcb-openshift-default-546bdfbdc5-ck6mz 59m Warning ProbeError pod/e2e-migrate-476e593d-gw-openshift-default-789fdfb6d5-k5k6v Startup probe error: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused... 59m Normal Killing pod/e2e-derive-36d7ee39-openshift-default-6b7cbf646f-nnh2m Stopping container istio-proxy 59m Normal Killing pod/e2e-ait-1ba9ae08-openshift-default-7758f9c76c-wbv2b Stopping container istio-proxy 59m Normal Created pod/e2e-ait-913eddcb-openshift-default-546bdfbdc5-ck6mz Container created 59m Normal Started pod/e2e-ait-913eddcb-openshift-default-546bdfbdc5-ck6mz Container started 59m Normal Pulled pod/e2e-ait-913eddcb-openshift-default-546bdfbdc5-ck6mz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/e2e-ait-913eddcb-openshift-default-546bdfbdc5-ck6mz Add eth0 [10.132.0.92/23] from ovn-kubernetes 59m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-476e593d Scaled up replica set payload-processing-e2e-migrate-476e593d-796fb9bdc9 from 0 to 1 59m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-476e593d-6555654587 Error creating: pods "payload-pre-processing-e2e-migrate-476e593d-6555654587-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-476e593d: serviceaccount "payload-processing-e2e-migrate-476e593d" not found 59m Warning FailedCreate replicaset/payload-processing-e2e-migrate-476e593d-796fb9bdc9 Error creating: pods "payload-processing-e2e-migrate-476e593d-796fb9bdc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-476e593d: serviceaccount "payload-processing-e2e-migrate-476e593d" not found 59m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-476e593d Scaled up replica set payload-pre-processing-e2e-migrate-476e593d-6555654587 from 0 to 1 59m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-476e593d-796fb9bdc9 Created pod: payload-processing-e2e-migrate-476e593d-796fb9bdc9-sq29f 59m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-476e593d-6555654587 Created pod: payload-pre-processing-e2e-migrate-476e593d-6555654587-lljr8 59m Normal Pulled pod/payload-pre-processing-e2e-migrate-476e593d-6555654587-lljr8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal Started pod/payload-pre-processing-e2e-migrate-476e593d-6555654587-lljr8 Container started 59m Normal Created pod/payload-pre-processing-e2e-migrate-476e593d-6555654587-lljr8 Container created 59m Normal Pulled pod/payload-processing-e2e-migrate-476e593d-796fb9bdc9-sq29f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/payload-processing-e2e-migrate-476e593d-796fb9bdc9-sq29f Add eth0 [10.132.0.93/23] from ovn-kubernetes 59m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-476e593d-6555654587-lljr8 Add eth0 [10.132.0.94/23] from ovn-kubernetes 59m Normal Created pod/payload-processing-e2e-migrate-476e593d-796fb9bdc9-sq29f Container created 59m Normal Started pod/payload-processing-e2e-migrate-476e593d-796fb9bdc9-sq29f Container started 59m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-913eddcb-d7477fc45 Created pod: payload-processing-e2e-ait-913eddcb-d7477fc45-nd68z 59m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-913eddcb-865ff6c5f5 Created pod: payload-pre-processing-e2e-ait-913eddcb-865ff6c5f5-lc242 59m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-913eddcb-865ff6c5f5 Error creating: pods "payload-pre-processing-e2e-ait-913eddcb-865ff6c5f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-913eddcb: serviceaccount "payload-processing-e2e-ait-913eddcb" not found 59m Warning FailedCreate replicaset/payload-processing-e2e-ait-913eddcb-d7477fc45 Error creating: pods "payload-processing-e2e-ait-913eddcb-d7477fc45-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-913eddcb: serviceaccount "payload-processing-e2e-ait-913eddcb" not found 59m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-913eddcb Scaled up replica set payload-pre-processing-e2e-ait-913eddcb-865ff6c5f5 from 0 to 1 59m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-913eddcb Scaled up replica set payload-processing-e2e-ait-913eddcb-d7477fc45 from 0 to 1 59m Normal Started pod/payload-processing-e2e-ait-913eddcb-d7477fc45-nd68z Container started 59m Normal AddedInterface pod/payload-processing-e2e-ait-913eddcb-d7477fc45-nd68z Add eth0 [10.132.0.96/23] from ovn-kubernetes 59m Normal AddedInterface pod/payload-pre-processing-e2e-ait-913eddcb-865ff6c5f5-lc242 Add eth0 [10.132.0.95/23] from ovn-kubernetes 59m Normal Started pod/payload-pre-processing-e2e-ait-913eddcb-865ff6c5f5-lc242 Container started 59m Normal Created pod/payload-processing-e2e-ait-913eddcb-d7477fc45-nd68z Container created 59m Normal Pulled pod/payload-pre-processing-e2e-ait-913eddcb-865ff6c5f5-lc242 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal Pulled pod/payload-processing-e2e-ait-913eddcb-d7477fc45-nd68z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal Created pod/payload-pre-processing-e2e-ait-913eddcb-865ff6c5f5-lc242 Container created 59m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-476e593d-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-476e593d-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-476e593d-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 59m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-476e593d-gw-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-913eddcb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-913eddcb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Normal Killing pod/payload-pre-processing-e2e-migrate-476e593d-6555654587-lljr8 Stopping container payload-pre-processing 58m Normal Killing pod/payload-processing-e2e-migrate-476e593d-796fb9bdc9-sq29f Stopping container payload-processing 58m Normal SuccessfulCreate replicaset/e2e-mt-2949c143-openshift-default-cc8cc894f Created pod: e2e-mt-2949c143-openshift-default-cc8cc894f-jnvwn 58m Normal ScalingReplicaSet deployment/e2e-mt-2949c143-openshift-default Scaled up replica set e2e-mt-2949c143-openshift-default-cc8cc894f from 0 to 1 58m Normal AddedLabel gateway/e2e-mt-2949c143 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-2949c143 58m Normal AddedInterface pod/e2e-mt-2949c143-openshift-default-cc8cc894f-jnvwn Add eth0 [10.132.0.98/23] from ovn-kubernetes 58m Normal Killing pod/e2e-migrate-476e593d-gw-openshift-default-789fdfb6d5-k5k6v Stopping container istio-proxy 58m Normal Killing pod/payload-processing-e2e-ait-913eddcb-d7477fc45-nd68z Stopping container payload-processing 58m Normal Killing pod/payload-pre-processing-e2e-ait-913eddcb-865ff6c5f5-lc242 Stopping container payload-pre-processing 58m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949 Created pod: e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949-2xr8n 58m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-80ecb3-openshift-default Scaled up replica set e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949 from 0 to 1 58m Normal AddedLabel gateway/e2e-shared-a-w2-80ecb3 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-80ecb3 58m Normal AddedInterface pod/e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949-2xr8n Add eth0 [10.132.0.99/23] from ovn-kubernetes 58m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-2949c143-5bdffff8fc Created pod: payload-processing-e2e-mt-2949c143-5bdffff8fc-jm8q7 58m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-2949c143-5f67f57d8c Error creating: pods "payload-pre-processing-e2e-mt-2949c143-5f67f57d8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-2949c143: serviceaccount "payload-processing-e2e-mt-2949c143" not found 58m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-2949c143 Scaled up replica set payload-pre-processing-e2e-mt-2949c143-5f67f57d8c from 0 to 1 58m Warning FailedCreate replicaset/payload-processing-e2e-mt-2949c143-5bdffff8fc Error creating: pods "payload-processing-e2e-mt-2949c143-5bdffff8fc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-2949c143: serviceaccount "payload-processing-e2e-mt-2949c143" not found 58m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-2949c143-5f67f57d8c Created pod: payload-pre-processing-e2e-mt-2949c143-5f67f57d8c-6qcmt 58m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-2949c143 Scaled up replica set payload-processing-e2e-mt-2949c143-5bdffff8fc from 0 to 1 58m Normal Pulled pod/payload-pre-processing-e2e-mt-2949c143-5f67f57d8c-6qcmt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/payload-processing-e2e-mt-2949c143-5bdffff8fc-jm8q7 Add eth0 [10.132.0.101/23] from ovn-kubernetes 58m Normal Killing pod/e2e-ait-913eddcb-openshift-default-546bdfbdc5-ck6mz Stopping container istio-proxy 58m Normal Pulled pod/payload-processing-e2e-mt-2949c143-5bdffff8fc-jm8q7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal Started pod/payload-pre-processing-e2e-mt-2949c143-5f67f57d8c-6qcmt Container started 58m Normal Created pod/payload-pre-processing-e2e-mt-2949c143-5f67f57d8c-6qcmt Container created 58m Normal AddedInterface pod/payload-pre-processing-e2e-mt-2949c143-5f67f57d8c-6qcmt Add eth0 [10.132.0.100/23] from ovn-kubernetes 58m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-5983ea-openshift-default Scaled up replica set e2e-shared-a-w0-5983ea-openshift-default-849df9ccdf from 0 to 1 58m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-5983ea-openshift-default-849df9ccdf Created pod: e2e-shared-a-w0-5983ea-openshift-default-849df9ccdf-bbqjg 58m Normal Started pod/payload-processing-e2e-mt-2949c143-5bdffff8fc-jm8q7 Container started 58m Normal AddedLabel gateway/e2e-shared-a-w0-5983ea Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-5983ea 58m Normal Created pod/payload-processing-e2e-mt-2949c143-5bdffff8fc-jm8q7 Container created 58m Warning Unhealthy pod/e2e-migrate-476e593d-gw-openshift-default-789fdfb6d5-k5k6v Readiness probe failed: Get "http://10.132.0.91:15021/healthz/ready": read tcp 10.132.0.2:35094->10.132.0.91:15021: read: connection reset by peer 58m Normal AddedInterface pod/e2e-shared-a-w0-5983ea-openshift-default-849df9ccdf-bbqjg Add eth0 [10.132.0.102/23] from ovn-kubernetes 58m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-80ecb3 Scaled up replica set payload-processing-e2e-shared-a-w2-80ecb3-67699fb85f from 0 to 1 58m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-80ecb3-7484c79f79 Error creating: pods "payload-pre-processing-e2e-shared-a-w2-80ecb3-7484c79f79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-80ecb3: serviceaccount "payload-processing-e2e-shared-a-w2-80ecb3" not found 58m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-80ecb3-7484c79f79 Created pod: payload-pre-processing-e2e-shared-a-w2-80ecb3-7484c79f79-cndqx 58m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-80ecb3 Scaled up replica set payload-pre-processing-e2e-shared-a-w2-80ecb3-7484c79f79 from 0 to 1 58m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-80ecb3-67699fb85f Created pod: payload-processing-e2e-shared-a-w2-80ecb3-67699fb85f-7tctg 58m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-80ecb3-67699fb85f Error creating: pods "payload-processing-e2e-shared-a-w2-80ecb3-67699fb85f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-80ecb3: serviceaccount "payload-processing-e2e-shared-a-w2-80ecb3" not found 58m Warning ProbeError pod/e2e-migrate-476e593d-gw-openshift-default-789fdfb6d5-k5k6v Readiness probe error: Get "http://10.132.0.91:15021/healthz/ready": read tcp 10.132.0.2:35094->10.132.0.91:15021: read: connection reset by peer... 58m Warning Unhealthy pod/e2e-ait-913eddcb-openshift-default-546bdfbdc5-ck6mz Readiness probe failed: Get "http://10.132.0.92:15021/healthz/ready": read tcp 10.132.0.2:41168->10.132.0.92:15021: read: connection reset by peer 58m Normal Started pod/payload-processing-e2e-shared-a-w2-80ecb3-67699fb85f-7tctg Container started 58m Normal Pulled pod/payload-processing-e2e-shared-a-w2-80ecb3-67699fb85f-7tctg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-80ecb3-7484c79f79-cndqx Container created 58m Normal Created pod/payload-processing-e2e-shared-a-w2-80ecb3-67699fb85f-7tctg Container created 58m Warning ProbeError pod/e2e-ait-913eddcb-openshift-default-546bdfbdc5-ck6mz Readiness probe error: Get "http://10.132.0.92:15021/healthz/ready": read tcp 10.132.0.2:41168->10.132.0.92:15021: read: connection reset by peer... 58m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-80ecb3-67699fb85f-7tctg Add eth0 [10.133.0.151/23] from ovn-kubernetes 58m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-80ecb3-7484c79f79-cndqx Container started 58m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-80ecb3-7484c79f79-cndqx Add eth0 [10.132.0.103/23] from ovn-kubernetes 58m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-80ecb3-7484c79f79-cndqx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w0-5983ea Scaled up replica set payload-pre-processing-e2e-shared-a-w0-5983ea-f76b4bb8c from 0 to 1 58m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w0-5983ea-65fbff47dc Error creating: pods "payload-processing-e2e-shared-a-w0-5983ea-65fbff47dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-5983ea: serviceaccount "payload-processing-e2e-shared-a-w0-5983ea" not found 58m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w0-5983ea Scaled up replica set payload-processing-e2e-shared-a-w0-5983ea-65fbff47dc from 0 to 1 58m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w0-5983ea-65fbff47dc Created pod: payload-processing-e2e-shared-a-w0-5983ea-65fbff47dc-kjmcc 58m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w0-5983ea-f76b4bb8c Error creating: pods "payload-pre-processing-e2e-shared-a-w0-5983ea-f76b4bb8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-5983ea: serviceaccount "payload-processing-e2e-shared-a-w0-5983ea" not found 58m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w0-5983ea-f76b4bb8c Created pod: payload-pre-processing-e2e-shared-a-w0-5983ea-f76b4bb8c-xvp6k 58m Normal Created pod/payload-pre-processing-e2e-shared-a-w0-5983ea-f76b4bb8c-xvp6k Container created 58m Normal AddedInterface pod/payload-processing-e2e-shared-a-w0-5983ea-65fbff47dc-kjmcc Add eth0 [10.132.0.104/23] from ovn-kubernetes 58m Normal Created pod/payload-processing-e2e-shared-a-w0-5983ea-65fbff47dc-kjmcc Container created 58m Normal Started pod/payload-pre-processing-e2e-shared-a-w0-5983ea-f76b4bb8c-xvp6k Container started 58m Normal Started pod/payload-processing-e2e-shared-a-w0-5983ea-65fbff47dc-kjmcc Container started 58m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w0-5983ea-f76b4bb8c-xvp6k Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w0-5983ea-f76b4bb8c-xvp6k Add eth0 [10.132.0.105/23] from ovn-kubernetes 58m Normal Pulled pod/payload-processing-e2e-shared-a-w0-5983ea-65fbff47dc-kjmcc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-3cd77f-openshift-default Scaled up replica set e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c from 0 to 1 57m Normal AddedLabel gateway/e2e-shared-b-w2-3cd77f Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-3cd77f 57m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c Created pod: e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-52wf7 57m Normal AddedInterface pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-52wf7 Add eth0 [10.133.0.152/23] from ovn-kubernetes 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-3cd77f Scaled up replica set payload-processing-e2e-shared-b-w2-3cd77f-6d7cccccf5 from 0 to 1 57m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-3cd77f-6d7cccccf5 Error creating: pods "payload-processing-e2e-shared-b-w2-3cd77f-6d7cccccf5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-3cd77f: serviceaccount "payload-processing-e2e-shared-b-w2-3cd77f" not found 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-3cd77f Scaled up replica set payload-pre-processing-e2e-shared-b-w2-3cd77f-799d6dfd99 from 0 to 1 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-3cd77f-799d6dfd99 Error creating: pods "payload-pre-processing-e2e-shared-b-w2-3cd77f-799d6dfd99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-3cd77f: serviceaccount "payload-processing-e2e-shared-b-w2-3cd77f" not found 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-3cd77f-6d7cccccf5 Created pod: payload-processing-e2e-shared-b-w2-3cd77f-6d7cccccf5-gxpwd 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-3cd77f-799d6dfd99 Created pod: payload-pre-processing-e2e-shared-b-w2-3cd77f-799d6dfd99-2zkl9 57m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-3cd77f-799d6dfd99-2zkl9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-3cd77f-6d7cccccf5-gxpwd Add eth0 [10.132.0.108/23] from ovn-kubernetes 57m Normal Pulled pod/payload-processing-e2e-shared-b-w2-3cd77f-6d7cccccf5-gxpwd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Created pod/payload-processing-e2e-shared-b-w2-3cd77f-6d7cccccf5-gxpwd Container created 57m Normal Started pod/payload-processing-e2e-shared-b-w2-3cd77f-6d7cccccf5-gxpwd Container started 57m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-3cd77f-799d6dfd99-2zkl9 Add eth0 [10.132.0.107/23] from ovn-kubernetes 57m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-3cd77f-799d6dfd99-2zkl9 Container created 57m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-3cd77f-799d6dfd99-2zkl9 Container started 57m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-6af86e-openshift-default Scaled up replica set e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c from 0 to 1 57m Normal AddedLabel gateway/e2e-shared-b-w0-6af86e Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-6af86e 57m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c Created pod: e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c-ztnmj 57m Normal AddedInterface pod/e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c-ztnmj Add eth0 [10.132.0.109/23] from ovn-kubernetes 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w0-6af86e-64944b4ff9 Created pod: payload-pre-processing-e2e-shared-b-w0-6af86e-64944b4ff9-4229b 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w0-6af86e-8bfcb675f Created pod: payload-processing-e2e-shared-b-w0-6af86e-8bfcb675f-rgngw 57m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w0-6af86e-8bfcb675f Error creating: pods "payload-processing-e2e-shared-b-w0-6af86e-8bfcb675f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-6af86e: serviceaccount "payload-processing-e2e-shared-b-w0-6af86e" not found 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w0-6af86e Scaled up replica set payload-pre-processing-e2e-shared-b-w0-6af86e-64944b4ff9 from 0 to 1 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w0-6af86e Scaled up replica set payload-processing-e2e-shared-b-w0-6af86e-8bfcb675f from 0 to 1 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w0-6af86e-64944b4ff9 Error creating: pods "payload-pre-processing-e2e-shared-b-w0-6af86e-64944b4ff9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-6af86e: serviceaccount "payload-processing-e2e-shared-b-w0-6af86e" not found 57m Normal Created pod/payload-processing-e2e-shared-b-w0-6af86e-8bfcb675f-rgngw Container created 57m Normal Started pod/payload-processing-e2e-shared-b-w0-6af86e-8bfcb675f-rgngw Container started 57m Normal Pulled pod/payload-processing-e2e-shared-b-w0-6af86e-8bfcb675f-rgngw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/payload-processing-e2e-shared-b-w0-6af86e-8bfcb675f-rgngw Add eth0 [10.133.0.153/23] from ovn-kubernetes 57m Normal Started pod/payload-pre-processing-e2e-shared-b-w0-6af86e-64944b4ff9-4229b Container started 57m Warning ProbeError pod/e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949-2xr8n Readiness probe error: Get "http://10.132.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 57m Warning Unhealthy pod/e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949-2xr8n Readiness probe failed: Get "http://10.132.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 57m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w0-6af86e-64944b4ff9-4229b Add eth0 [10.132.0.110/23] from ovn-kubernetes 57m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w0-6af86e-64944b4ff9-4229b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Created pod/payload-pre-processing-e2e-shared-b-w0-6af86e-64944b4ff9-4229b Container created 57m Normal Pulled pod/e2e-shared-a-w0-5983ea-openshift-default-849df9ccdf-bbqjg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Pulled pod/e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949-2xr8n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-shared-a-w0-5983ea-openshift-default-849df9ccdf-bbqjg Container created 57m Normal Pulled pod/e2e-mt-2949c143-openshift-default-cc8cc894f-jnvwn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Started pod/e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949-2xr8n Container started 57m Normal Created pod/e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949-2xr8n Container created 57m Normal Started pod/e2e-shared-a-w0-5983ea-openshift-default-849df9ccdf-bbqjg Container started 57m Normal Started pod/e2e-mt-2949c143-openshift-default-cc8cc894f-jnvwn Container started 57m Normal Created pod/e2e-mt-2949c143-openshift-default-cc8cc894f-jnvwn Container created 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-2949c143-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-2949c143-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning ProbeError pod/e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949-2xr8n Startup probe error: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949-2xr8n Startup probe failed: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-shared-a-w0-5983ea-openshift-default-849df9ccdf-bbqjg Startup probe failed: Get "http://10.132.0.102:15021/healthz/ready": dial tcp 10.132.0.102:15021: connect: connection refused 57m Warning ProbeError pod/e2e-shared-a-w0-5983ea-openshift-default-849df9ccdf-bbqjg Startup probe error: Get "http://10.132.0.102:15021/healthz/ready": dial tcp 10.132.0.102:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-mt-2949c143-openshift-default-cc8cc894f-jnvwn Startup probe error: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-mt-2949c143-openshift-default-cc8cc894f-jnvwn Startup probe failed: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-5983ea-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-5983ea-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-80ecb3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-80ecb3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Normal SuccessfulCreate replicaset/e2e-mt-d399fb07-openshift-default-6d54788969 Created pod: e2e-mt-d399fb07-openshift-default-6d54788969-4z9xf 57m Normal ScalingReplicaSet deployment/e2e-mt-d399fb07-openshift-default Scaled up replica set e2e-mt-d399fb07-openshift-default-6d54788969 from 0 to 1 57m Normal AddedLabel gateway/e2e-mt-d399fb07 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-d399fb07 57m Normal AddedInterface pod/e2e-mt-d399fb07-openshift-default-6d54788969-4z9xf Add eth0 [10.133.0.154/23] from ovn-kubernetes 57m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-6f2eba-openshift-default Scaled up replica set e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c from 0 to 1 57m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c Created pod: e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c-krxtr 57m Normal AddedLabel gateway/e2e-shared-a-w5-6f2eba Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w5-6f2eba 57m Normal AddedInterface pod/e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c-krxtr Add eth0 [10.133.0.155/23] from ovn-kubernetes 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-d399fb07-755755d479 Error creating: pods "payload-pre-processing-e2e-mt-d399fb07-755755d479-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d399fb07: serviceaccount "payload-processing-e2e-mt-d399fb07" not found 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-d399fb07 Scaled up replica set payload-pre-processing-e2e-mt-d399fb07-755755d479 from 0 to 1 57m Warning FailedCreate replicaset/payload-processing-e2e-mt-d399fb07-56bfdc9c75 Error creating: pods "payload-processing-e2e-mt-d399fb07-56bfdc9c75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d399fb07: serviceaccount "payload-processing-e2e-mt-d399fb07" not found 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-d399fb07-755755d479 Created pod: payload-pre-processing-e2e-mt-d399fb07-755755d479-wsns4 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-d399fb07-56bfdc9c75 Created pod: payload-processing-e2e-mt-d399fb07-56bfdc9c75-7d8qb 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-d399fb07 Scaled up replica set payload-processing-e2e-mt-d399fb07-56bfdc9c75 from 0 to 1 57m Normal Created pod/payload-processing-e2e-mt-d399fb07-56bfdc9c75-7d8qb Container created 57m Normal Started pod/payload-processing-e2e-mt-d399fb07-56bfdc9c75-7d8qb Container started 57m Normal Pulled pod/payload-processing-e2e-mt-d399fb07-56bfdc9c75-7d8qb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Pulled pod/payload-pre-processing-e2e-mt-d399fb07-755755d479-wsns4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Created pod/payload-pre-processing-e2e-mt-d399fb07-755755d479-wsns4 Container created 57m Normal AddedInterface pod/payload-pre-processing-e2e-mt-d399fb07-755755d479-wsns4 Add eth0 [10.133.0.156/23] from ovn-kubernetes 57m Normal AddedInterface pod/payload-processing-e2e-mt-d399fb07-56bfdc9c75-7d8qb Add eth0 [10.132.0.111/23] from ovn-kubernetes 57m Normal Started pod/payload-pre-processing-e2e-mt-d399fb07-755755d479-wsns4 Container started 56m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w5-6f2eba-5576597779 Error creating: pods "payload-processing-e2e-shared-a-w5-6f2eba-5576597779-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-6f2eba: serviceaccount "payload-processing-e2e-shared-a-w5-6f2eba" not found 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w5-6f2eba-dc7c6b979 Error creating: pods "payload-pre-processing-e2e-shared-a-w5-6f2eba-dc7c6b979-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-6f2eba: serviceaccount "payload-processing-e2e-shared-a-w5-6f2eba" not found 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w5-6f2eba Scaled up replica set payload-pre-processing-e2e-shared-a-w5-6f2eba-dc7c6b979 from 0 to 1 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w5-6f2eba-dc7c6b979 Created pod: payload-pre-processing-e2e-shared-a-w5-6f2eba-dc7c6b979-rb48s 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w5-6f2eba-5576597779 Created pod: payload-processing-e2e-shared-a-w5-6f2eba-5576597779-g8wwj 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w5-6f2eba Scaled up replica set payload-processing-e2e-shared-a-w5-6f2eba-5576597779 from 0 to 1 56m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w5-6f2eba-dc7c6b979-rb48s Add eth0 [10.132.0.112/23] from ovn-kubernetes 56m Normal Created pod/payload-processing-e2e-shared-a-w5-6f2eba-5576597779-g8wwj Container created 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-2949c143-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Normal AddedInterface pod/payload-processing-e2e-shared-a-w5-6f2eba-5576597779-g8wwj Add eth0 [10.133.0.157/23] from ovn-kubernetes 56m Normal Started pod/payload-pre-processing-e2e-shared-a-w5-6f2eba-dc7c6b979-rb48s Container started 56m Normal Created pod/payload-pre-processing-e2e-shared-a-w5-6f2eba-dc7c6b979-rb48s Container created 56m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w5-6f2eba-dc7c6b979-rb48s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-2949c143-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Normal Pulled pod/payload-processing-e2e-shared-a-w5-6f2eba-5576597779-g8wwj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Started pod/payload-processing-e2e-shared-a-w5-6f2eba-5576597779-g8wwj Container started 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-80ecb3-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-80ecb3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-5983ea-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-5983ea-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning Unhealthy pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-52wf7 Readiness probe failed: Get "http://10.133.0.152:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-52wf7 Readiness probe error: Get "http://10.133.0.152:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Normal Pulled pod/e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c-ztnmj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-52wf7 Container created 56m Normal Pulled pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-52wf7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-52wf7 Container started 56m Normal Created pod/e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c-ztnmj Container created 56m Normal Started pod/e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c-ztnmj Container started 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-6af86e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-6af86e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning Unhealthy pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-52wf7 Startup probe failed: Get "http://10.133.0.152:15021/healthz/ready": dial tcp 10.133.0.152:15021: connect: connection refused 56m Warning ProbeError pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-52wf7 Startup probe error: Get "http://10.133.0.152:15021/healthz/ready": dial tcp 10.133.0.152:15021: connect: connection refused... 56m Warning ProbeError pod/e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c-ztnmj Startup probe error: Get "http://10.132.0.109:15021/healthz/ready": dial tcp 10.132.0.109:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c-ztnmj Startup probe failed: Get "http://10.132.0.109:15021/healthz/ready": dial tcp 10.132.0.109:15021: connect: connection refused 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-3cd77f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-3cd77f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Normal Killing pod/e2e-worker-w2-421583-openshift-default-5cb767b447-l4c4h Stopping container istio-proxy 56m Normal ScalingReplicaSet deployment/e2e-worker-w2-421583-openshift-default Scaled down replica set e2e-worker-w2-421583-openshift-default-5cb767b447 from 5 to 4 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-421583-openshift-default New size: 4; reason: All metrics below target 56m Normal SuccessfulDelete replicaset/e2e-worker-w2-421583-openshift-default-5cb767b447 Deleted pod: e2e-worker-w2-421583-openshift-default-5cb767b447-l4c4h 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-6af86e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf Created pod: e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-wjvgx 55m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-510f80-openshift-default Scaled up replica set e2e-shared-b-w5-510f80-openshift-default-6465ccddcf from 0 to 1 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-6af86e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Normal AddedLabel gateway/e2e-shared-b-w5-510f80 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w5-510f80 55m Normal AddedInterface pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-wjvgx Add eth0 [10.133.0.160/23] from ovn-kubernetes 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-3cd77f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-3cd77f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Normal SuccessfulCreate replicaset/e2e-mt-d399fb07-openshift-default-6d54788969 Created pod: e2e-mt-d399fb07-openshift-default-6d54788969-k9sr9 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w5-510f80-54b6b8c75 Created pod: payload-pre-processing-e2e-shared-b-w5-510f80-54b6b8c75-9w995 55m Normal SuccessfulCreate replicaset/e2e-mt-d399fb07-openshift-default-6d54788969 Created pod: e2e-mt-d399fb07-openshift-default-6d54788969-47qsn 55m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w5-510f80-c84b8b845 Error creating: pods "payload-processing-e2e-shared-b-w5-510f80-c84b8b845-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-510f80: serviceaccount "payload-processing-e2e-shared-b-w5-510f80" not found 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w5-510f80 Scaled up replica set payload-pre-processing-e2e-shared-b-w5-510f80-54b6b8c75 from 0 to 1 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w5-510f80-54b6b8c75 Error creating: pods "payload-pre-processing-e2e-shared-b-w5-510f80-54b6b8c75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-510f80: serviceaccount "payload-processing-e2e-shared-b-w5-510f80" not found 55m Normal ScalingReplicaSet deployment/e2e-mt-d399fb07-openshift-default Scaled up replica set e2e-mt-d399fb07-openshift-default-6d54788969 from 1 to 3 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w5-510f80-c84b8b845 Created pod: payload-processing-e2e-shared-b-w5-510f80-c84b8b845-fpfjm 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-d399fb07-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w5-510f80 Scaled up replica set payload-processing-e2e-shared-b-w5-510f80-c84b8b845 from 0 to 1 55m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w5-510f80-54b6b8c75-9w995 Add eth0 [10.132.0.119/23] from ovn-kubernetes 55m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w5-510f80-54b6b8c75-9w995 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Pulled pod/e2e-mt-d399fb07-openshift-default-6d54788969-47qsn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/e2e-mt-d399fb07-openshift-default-6d54788969-47qsn Add eth0 [10.134.0.94/23] from ovn-kubernetes 55m Normal Started pod/e2e-mt-d399fb07-openshift-default-6d54788969-k9sr9 Container started 55m Normal Created pod/e2e-mt-d399fb07-openshift-default-6d54788969-k9sr9 Container created 55m Normal Pulled pod/e2e-mt-d399fb07-openshift-default-6d54788969-k9sr9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/e2e-mt-d399fb07-openshift-default-6d54788969-47qsn Container created 55m Normal Started pod/e2e-mt-d399fb07-openshift-default-6d54788969-47qsn Container started 55m Normal AddedInterface pod/e2e-mt-d399fb07-openshift-default-6d54788969-k9sr9 Add eth0 [10.132.0.118/23] from ovn-kubernetes 55m Normal Started pod/payload-processing-e2e-shared-b-w5-510f80-c84b8b845-fpfjm Container started 55m Normal Created pod/payload-pre-processing-e2e-shared-b-w5-510f80-54b6b8c75-9w995 Container created 55m Normal Started pod/payload-pre-processing-e2e-shared-b-w5-510f80-54b6b8c75-9w995 Container started 55m Normal Pulled pod/payload-processing-e2e-shared-b-w5-510f80-c84b8b845-fpfjm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Created pod/payload-processing-e2e-shared-b-w5-510f80-c84b8b845-fpfjm Container created 55m Normal AddedInterface pod/payload-processing-e2e-shared-b-w5-510f80-c84b8b845-fpfjm Add eth0 [10.132.0.120/23] from ovn-kubernetes 55m Normal Created pod/e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c-krxtr Container created 55m Normal Pulled pod/e2e-mt-d399fb07-openshift-default-6d54788969-4z9xf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Pulled pod/e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c-krxtr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Started pod/e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c-krxtr Container started 55m Normal Created pod/e2e-mt-d399fb07-openshift-default-6d54788969-4z9xf Container created 55m Normal Started pod/e2e-mt-d399fb07-openshift-default-6d54788969-4z9xf Container started 55m Warning Unhealthy pod/e2e-mt-d399fb07-openshift-default-6d54788969-4z9xf Readiness probe failed: Get "http://10.133.0.154:15021/healthz/ready": dial tcp 10.133.0.154:15021: connect: connection refused 55m Warning ProbeError pod/e2e-mt-d399fb07-openshift-default-6d54788969-4z9xf Readiness probe error: Get "http://10.133.0.154:15021/healthz/ready": dial tcp 10.133.0.154:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-mt-d399fb07-openshift-default-6d54788969-47qsn Startup probe failed: Get "http://10.134.0.94:15021/healthz/ready": dial tcp 10.134.0.94:15021: connect: connection refused 55m Warning ProbeError pod/e2e-mt-d399fb07-openshift-default-6d54788969-47qsn Startup probe error: Get "http://10.134.0.94:15021/healthz/ready": dial tcp 10.134.0.94:15021: connect: connection refused... 55m Warning ProbeError pod/e2e-mt-d399fb07-openshift-default-6d54788969-k9sr9 Startup probe error: Get "http://10.132.0.118:15021/healthz/ready": dial tcp 10.132.0.118:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-mt-d399fb07-openshift-default-6d54788969-k9sr9 Startup probe failed: Get "http://10.132.0.118:15021/healthz/ready": dial tcp 10.132.0.118:15021: connect: connection refused 55m Warning ProbeError pod/e2e-mt-d399fb07-openshift-default-6d54788969-4z9xf Startup probe error: Get "http://10.133.0.154:15021/healthz/ready": dial tcp 10.133.0.154:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-mt-d399fb07-openshift-default-6d54788969-4z9xf Startup probe failed: Get "http://10.133.0.154:15021/healthz/ready": dial tcp 10.133.0.154:15021: connect: connection refused 55m Warning Unhealthy pod/e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c-krxtr Startup probe failed: Get "http://10.133.0.155:15021/healthz/ready": dial tcp 10.133.0.155:15021: connect: connection refused 55m Warning ProbeError pod/e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c-krxtr Startup probe error: Get "http://10.133.0.155:15021/healthz/ready": dial tcp 10.133.0.155:15021: connect: connection refused... 55m Normal Killing pod/payload-processing-e2e-mt-2949c143-5bdffff8fc-jm8q7 Stopping container payload-processing 55m Normal Killing pod/payload-pre-processing-e2e-mt-2949c143-5f67f57d8c-6qcmt Stopping container payload-pre-processing 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-d399fb07-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-d399fb07-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-6f2eba-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-6f2eba-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-510f80-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-510f80-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning Unhealthy pod/payload-pre-processing-e2e-mt-2949c143-5f67f57d8c-6qcmt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 55m Warning Unhealthy pod/payload-processing-e2e-mt-2949c143-5bdffff8fc-jm8q7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-d399fb07-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-d399fb07-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-6f2eba-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-6f2eba-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Normal SuccessfulDelete replicaset/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 Deleted pod: e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-bhn2k 55m Normal Killing pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-zb2qv Stopping container istio-proxy 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-09d0d5-openshift-default New size: 6; reason: All metrics below target 55m Normal ScalingReplicaSet deployment/e2e-worker-w0-09d0d5-openshift-default Scaled down replica set e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 from 10 to 6 55m Normal Killing pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-2t4b8 Stopping container istio-proxy 55m Normal SuccessfulDelete replicaset/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 Deleted pod: e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-zb2qv 55m Normal SuccessfulDelete replicaset/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 Deleted pod: e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-2t4b8 55m Normal SuccessfulDelete replicaset/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 Deleted pod: e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-m6s8k 55m Normal Killing pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-m6s8k Stopping container istio-proxy 55m Normal Killing pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-bhn2k Stopping container istio-proxy 55m Normal SuccessfulDelete replicaset/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc Deleted pod: e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-c9dkh 55m Normal AddedLabel gateway/e2e-api-a-370ec2 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-370ec2 55m Normal Killing pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-c9dkh Stopping container istio-proxy 55m Normal Killing pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-n9dj2 Stopping container istio-proxy 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0ad7c5-openshift-default New size: 7; reason: All metrics below target 55m Normal SuccessfulDelete replicaset/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c Deleted pod: e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-97nr4 55m Normal SuccessfulDelete replicaset/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc Deleted pod: e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-7wsq8 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-54114e-openshift-default New size: 8; reason: All metrics below target 55m Normal ScalingReplicaSet deployment/e2e-worker-w6-012ba0-openshift-default Scaled down replica set e2e-worker-w6-012ba0-openshift-default-846bcc5bfc from 10 to 6 55m Normal Killing pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-97nr4 Stopping container istio-proxy 55m Normal Killing pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-57vcj Stopping container istio-proxy 55m Normal ScalingReplicaSet deployment/e2e-worker-w4-0ad7c5-openshift-default Scaled down replica set e2e-worker-w4-0ad7c5-openshift-default-788b484d5c from 10 to 7 55m Normal SuccessfulDelete replicaset/e2e-worker-w5-54114e-openshift-default-7fc467ddd7 Deleted pod: e2e-worker-w5-54114e-openshift-default-7fc467ddd7-57vcj 55m Normal Killing pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-7wsq8 Stopping container istio-proxy 55m Normal Killing pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-97d66 Stopping container istio-proxy 55m Normal SuccessfulDelete replicaset/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c Deleted pod: e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-gwz9t 55m Normal Killing pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-gwz9t Stopping container istio-proxy 55m Normal SuccessfulDelete replicaset/e2e-worker-w5-54114e-openshift-default-7fc467ddd7 Deleted pod: e2e-worker-w5-54114e-openshift-default-7fc467ddd7-97d66 55m Normal SuccessfulDelete replicaset/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc Deleted pod: e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-n9dj2 55m Normal SuccessfulDelete replicaset/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc Deleted pod: e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-2w4gw 55m Normal SuccessfulDelete replicaset/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c Deleted pod: e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hstmf 55m Normal Killing pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hstmf Stopping container istio-proxy 55m Warning ProbeError pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hstmf Readiness probe error: HTTP probe failed with statuscode: 503... 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-012ba0-openshift-default New size: 6; reason: All metrics below target 55m Normal Killing pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-2w4gw Stopping container istio-proxy 55m Normal ScalingReplicaSet deployment/e2e-worker-w5-54114e-openshift-default Scaled down replica set e2e-worker-w5-54114e-openshift-default-7fc467ddd7 from 10 to 8 55m Normal ScalingReplicaSet deployment/e2e-api-a-370ec2-openshift-default Scaled up replica set e2e-api-a-370ec2-openshift-default-6fccc4f697 from 0 to 1 55m Normal SuccessfulCreate replicaset/e2e-api-a-370ec2-openshift-default-6fccc4f697 Created pod: e2e-api-a-370ec2-openshift-default-6fccc4f697-78w6t 55m Normal AddedInterface pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-78w6t Add eth0 [10.132.0.121/23] from ovn-kubernetes 54m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf Created pod: e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-zqkns 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-370ec2-7658c775d7 Created pod: payload-processing-e2e-api-a-370ec2-7658c775d7-t69wq 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-370ec2-7957558899 Created pod: payload-pre-processing-e2e-api-a-370ec2-7957558899-vkjqx 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-370ec2-7957558899 Error creating: pods "payload-pre-processing-e2e-api-a-370ec2-7957558899-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-370ec2: serviceaccount "payload-processing-e2e-api-a-370ec2" not found 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-370ec2 Scaled up replica set payload-pre-processing-e2e-api-a-370ec2-7957558899 from 0 to 1 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-370ec2 Scaled up replica set payload-processing-e2e-api-a-370ec2-7658c775d7 from 0 to 1 54m Warning FailedCreate replicaset/payload-processing-e2e-api-a-370ec2-7658c775d7 Error creating: pods "payload-processing-e2e-api-a-370ec2-7658c775d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-370ec2: serviceaccount "payload-processing-e2e-api-a-370ec2" not found 54m Normal Started pod/payload-processing-e2e-api-a-370ec2-7658c775d7-t69wq Container started 54m Normal Pulled pod/payload-processing-e2e-api-a-370ec2-7658c775d7-t69wq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Started pod/payload-pre-processing-e2e-api-a-370ec2-7957558899-vkjqx Container started 54m Normal Created pod/payload-pre-processing-e2e-api-a-370ec2-7957558899-vkjqx Container created 54m Normal Pulled pod/payload-pre-processing-e2e-api-a-370ec2-7957558899-vkjqx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Created pod/payload-processing-e2e-api-a-370ec2-7658c775d7-t69wq Container created 54m Normal AddedInterface pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-zqkns Add eth0 [10.132.0.124/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-zqkns Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-zqkns Container created 54m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-370ec2-7957558899-vkjqx Add eth0 [10.132.0.123/23] from ovn-kubernetes 54m Normal Started pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-zqkns Container started 54m Normal AddedInterface pod/payload-processing-e2e-api-a-370ec2-7658c775d7-t69wq Add eth0 [10.132.0.122/23] from ovn-kubernetes 54m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-0ff5b0-openshift-default Scaled up replica set e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5 from 0 to 1 54m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5 Created pod: e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5-6m98h 54m Normal AddedLabel gateway/e2e-shared-a-w6-0ff5b0 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w6-0ff5b0 54m Normal AddedInterface pod/e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5-6m98h Add eth0 [10.132.0.125/23] from ovn-kubernetes 54m Warning ProbeError pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-wjvgx Readiness probe error: Get "http://10.133.0.160:15021/healthz/ready": read tcp 10.133.0.2:37448->10.133.0.160:15021: read: connection reset by peer... 54m Warning Unhealthy pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-wjvgx Readiness probe failed: Get "http://10.133.0.160:15021/healthz/ready": read tcp 10.133.0.2:37448->10.133.0.160:15021: read: connection reset by peer 54m Normal Started pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-wjvgx Container started 54m Normal Created pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-wjvgx Container created 54m Normal Pulled pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-wjvgx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedLabel gateway/e2e-api-a-f443d6 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-f443d6 54m Normal SuccessfulCreate replicaset/e2e-api-a-f443d6-openshift-default-6dc8b74cd7 Created pod: e2e-api-a-f443d6-openshift-default-6dc8b74cd7-nkzfv 54m Normal ScalingReplicaSet deployment/e2e-api-a-f443d6-openshift-default Scaled up replica set e2e-api-a-f443d6-openshift-default-6dc8b74cd7 from 0 to 1 54m Normal AddedInterface pod/e2e-api-a-f443d6-openshift-default-6dc8b74cd7-nkzfv Add eth0 [10.133.0.164/23] from ovn-kubernetes 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w6-0ff5b0 Scaled up replica set payload-processing-e2e-shared-a-w6-0ff5b0-7c4b7489d7 from 0 to 1 54m Warning Unhealthy pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-zqkns Startup probe failed: Get "http://10.132.0.124:15021/healthz/ready": dial tcp 10.132.0.124:15021: connect: connection refused 54m Warning ProbeError pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-zqkns Startup probe error: Get "http://10.132.0.124:15021/healthz/ready": dial tcp 10.132.0.124:15021: connect: connection refused... 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w6-0ff5b0 Scaled up replica set payload-pre-processing-e2e-shared-a-w6-0ff5b0-6f4dc7d5f from 0 to 1 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w6-0ff5b0-6f4dc7d5f Created pod: payload-pre-processing-e2e-shared-a-w6-0ff5b0-6f4dc7d5f-pmqnt 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w6-0ff5b0-6f4dc7d5f Error creating: pods "payload-pre-processing-e2e-shared-a-w6-0ff5b0-6f4dc7d5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-0ff5b0: serviceaccount "payload-processing-e2e-shared-a-w6-0ff5b0" not found 54m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w6-0ff5b0-7c4b7489d7 Error creating: pods "payload-processing-e2e-shared-a-w6-0ff5b0-7c4b7489d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-0ff5b0: serviceaccount "payload-processing-e2e-shared-a-w6-0ff5b0" not found 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w6-0ff5b0-7c4b7489d7 Created pod: payload-processing-e2e-shared-a-w6-0ff5b0-7c4b7489d7-xlk52 54m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w6-0ff5b0-6f4dc7d5f-pmqnt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Created pod/payload-processing-e2e-shared-a-w6-0ff5b0-7c4b7489d7-xlk52 Container created 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-370ec2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w6-0ff5b0-6f4dc7d5f-pmqnt Add eth0 [10.132.0.126/23] from ovn-kubernetes 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-370ec2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Normal Started pod/payload-pre-processing-e2e-shared-a-w6-0ff5b0-6f4dc7d5f-pmqnt Container started 54m Normal Created pod/payload-pre-processing-e2e-shared-a-w6-0ff5b0-6f4dc7d5f-pmqnt Container created 54m Normal Started pod/payload-processing-e2e-shared-a-w6-0ff5b0-7c4b7489d7-xlk52 Container started 54m Normal AddedInterface pod/payload-processing-e2e-shared-a-w6-0ff5b0-7c4b7489d7-xlk52 Add eth0 [10.132.0.127/23] from ovn-kubernetes 54m Normal Pulled pod/payload-processing-e2e-shared-a-w6-0ff5b0-7c4b7489d7-xlk52 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Warning Unhealthy pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-wjvgx Startup probe failed: Get "http://10.133.0.160:15021/healthz/ready": dial tcp 10.133.0.160:15021: connect: connection refused 54m Warning ProbeError pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-wjvgx Startup probe error: Get "http://10.133.0.160:15021/healthz/ready": dial tcp 10.133.0.160:15021: connect: connection refused... 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-f443d6 Scaled up replica set payload-processing-e2e-api-a-f443d6-784b88c549 from 0 to 1 54m Normal AddedInterface pod/payload-processing-e2e-api-a-f443d6-784b88c549-cwkhm Add eth0 [10.133.0.165/23] from ovn-kubernetes 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-f443d6-6bcbc5c45f Created pod: payload-pre-processing-e2e-api-a-f443d6-6bcbc5c45f-72rp5 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-f443d6 Scaled up replica set payload-pre-processing-e2e-api-a-f443d6-6bcbc5c45f from 0 to 1 54m Warning FailedCreate replicaset/payload-processing-e2e-api-a-f443d6-784b88c549 Error creating: pods "payload-processing-e2e-api-a-f443d6-784b88c549-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-f443d6: serviceaccount "payload-processing-e2e-api-a-f443d6" not found 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-f443d6-784b88c549 Created pod: payload-processing-e2e-api-a-f443d6-784b88c549-cwkhm 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-f443d6-6bcbc5c45f Error creating: pods "payload-pre-processing-e2e-api-a-f443d6-6bcbc5c45f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-f443d6: serviceaccount "payload-processing-e2e-api-a-f443d6" not found 54m Normal Created pod/payload-pre-processing-e2e-api-a-f443d6-6bcbc5c45f-72rp5 Container created 54m Normal Started pod/payload-pre-processing-e2e-api-a-f443d6-6bcbc5c45f-72rp5 Container started 54m Normal Pulled pod/payload-pre-processing-e2e-api-a-f443d6-6bcbc5c45f-72rp5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Started pod/payload-processing-e2e-api-a-f443d6-784b88c549-cwkhm Container started 54m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-f443d6-6bcbc5c45f-72rp5 Add eth0 [10.133.0.166/23] from ovn-kubernetes 54m Normal Pulled pod/payload-processing-e2e-api-a-f443d6-784b88c549-cwkhm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Created pod/payload-processing-e2e-api-a-f443d6-784b88c549-cwkhm Container created 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-2949c143 Scaled up replica set payload-processing-e2e-mt-2949c143-5bdffff8fc from 0 to 1 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-2949c143-5f67f57d8c Error creating: pods "payload-pre-processing-e2e-mt-2949c143-5f67f57d8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-2949c143: serviceaccount "payload-processing-e2e-mt-2949c143" not found 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-2949c143-5f67f57d8c Created pod: payload-pre-processing-e2e-mt-2949c143-5f67f57d8c-pw4d8 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-2949c143 Scaled up replica set payload-pre-processing-e2e-mt-2949c143-5f67f57d8c from 0 to 1 54m Normal AddedInterface pod/payload-pre-processing-e2e-mt-2949c143-5f67f57d8c-pw4d8 Add eth0 [10.133.0.167/23] from ovn-kubernetes 54m Warning FailedCreate replicaset/payload-processing-e2e-mt-2949c143-5bdffff8fc Error creating: pods "payload-processing-e2e-mt-2949c143-5bdffff8fc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-2949c143: serviceaccount "payload-processing-e2e-mt-2949c143" not found 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-2949c143-5bdffff8fc Created pod: payload-processing-e2e-mt-2949c143-5bdffff8fc-jk5zn 54m Normal SuccessfulDelete replicaset/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 Deleted pod: e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-mb6lc 54m Normal Killing pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-6d9p2 Stopping container istio-proxy 54m Normal SuccessfulDelete replicaset/e2e-worker-w3-d52c59-openshift-default-5466968fb7 Deleted pod: e2e-worker-w3-d52c59-openshift-default-5466968fb7-95kvh 54m Normal Killing pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-zr6tr Stopping container istio-proxy 54m Normal AddedInterface pod/payload-processing-e2e-mt-2949c143-5bdffff8fc-jk5zn Add eth0 [10.132.0.128/23] from ovn-kubernetes 54m Normal Pulled pod/payload-processing-e2e-mt-2949c143-5bdffff8fc-jk5zn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Created pod/payload-processing-e2e-mt-2949c143-5bdffff8fc-jk5zn Container created 54m Normal SuccessfulDelete replicaset/e2e-worker-w3-d52c59-openshift-default-5466968fb7 Deleted pod: e2e-worker-w3-d52c59-openshift-default-5466968fb7-zr6tr 54m Normal SuccessfulDelete replicaset/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 Deleted pod: e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-6d9p2 54m Normal Killing pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-w2q4t Stopping container istio-proxy 54m Normal Killing pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-mb6lc Stopping container istio-proxy 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-d52c59-openshift-default New size: 7; reason: All metrics below target 54m Normal ScalingReplicaSet deployment/e2e-worker-w3-d52c59-openshift-default Scaled down replica set e2e-worker-w3-d52c59-openshift-default-5466968fb7 from 10 to 7 54m Normal SuccessfulDelete replicaset/e2e-worker-w3-d52c59-openshift-default-5466968fb7 Deleted pod: e2e-worker-w3-d52c59-openshift-default-5466968fb7-w2q4t 54m Normal ScalingReplicaSet deployment/e2e-worker-w1-2b1e88-openshift-default Scaled down replica set e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 from 10 to 8 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-2b1e88-openshift-default New size: 8; reason: All metrics below target 54m Normal Killing pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-95kvh Stopping container istio-proxy 54m Normal Pulled pod/payload-pre-processing-e2e-mt-2949c143-5f67f57d8c-pw4d8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Started pod/payload-pre-processing-e2e-mt-2949c143-5f67f57d8c-pw4d8 Container started 54m Normal Started pod/payload-processing-e2e-mt-2949c143-5bdffff8fc-jk5zn Container started 54m Normal Created pod/payload-pre-processing-e2e-mt-2949c143-5f67f57d8c-pw4d8 Container created 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-510f80-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-510f80-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Normal AddedLabel gateway/e2e-api-a-de31dd Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-de31dd 54m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-8c8cz Readiness probe error: Get "http://10.132.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-8c8cz Readiness probe failed: Get "http://10.132.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Normal ScalingReplicaSet deployment/e2e-api-a-de31dd-openshift-default Scaled up replica set e2e-api-a-de31dd-openshift-default-56f5bd455 from 0 to 1 54m Normal AddedInterface pod/e2e-api-a-de31dd-openshift-default-56f5bd455-d8vcq Add eth0 [10.132.0.130/23] from ovn-kubernetes 54m Normal SuccessfulCreate replicaset/e2e-api-a-de31dd-openshift-default-56f5bd455 Created pod: e2e-api-a-de31dd-openshift-default-56f5bd455-d8vcq 54m Normal Killing pod/payload-processing-e2e-mt-d399fb07-56bfdc9c75-7d8qb Stopping container payload-processing 54m Normal Killing pod/payload-pre-processing-e2e-mt-d399fb07-755755d479-wsns4 Stopping container payload-pre-processing 54m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-de31dd-7579cc4f79 Created pod: payload-pre-processing-e2e-api-a-de31dd-7579cc4f79-rq4dm 54m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-de31dd Scaled up replica set payload-processing-e2e-api-a-de31dd-7d9cf5d4f from 0 to 1 54m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-de31dd-7d9cf5d4f Created pod: payload-processing-e2e-api-a-de31dd-7d9cf5d4f-9lr2z 54m Warning FailedCreate replicaset/payload-processing-e2e-api-a-de31dd-7d9cf5d4f Error creating: pods "payload-processing-e2e-api-a-de31dd-7d9cf5d4f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-de31dd: serviceaccount "payload-processing-e2e-api-a-de31dd" not found 54m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-de31dd Scaled up replica set payload-pre-processing-e2e-api-a-de31dd-7579cc4f79 from 0 to 1 54m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-de31dd-7579cc4f79 Error creating: pods "payload-pre-processing-e2e-api-a-de31dd-7579cc4f79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-de31dd: serviceaccount "payload-processing-e2e-api-a-de31dd" not found 54m Normal Started pod/payload-processing-e2e-api-a-de31dd-7d9cf5d4f-9lr2z Container started 54m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-de31dd-7579cc4f79-rq4dm Add eth0 [10.132.0.131/23] from ovn-kubernetes 54m Normal AddedInterface pod/payload-processing-e2e-api-a-de31dd-7d9cf5d4f-9lr2z Add eth0 [10.133.0.169/23] from ovn-kubernetes 54m Normal Pulled pod/payload-processing-e2e-api-a-de31dd-7d9cf5d4f-9lr2z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal Created pod/payload-processing-e2e-api-a-de31dd-7d9cf5d4f-9lr2z Container created 54m Normal Started pod/payload-pre-processing-e2e-api-a-de31dd-7579cc4f79-rq4dm Container started 54m Normal Created pod/payload-pre-processing-e2e-api-a-de31dd-7579cc4f79-rq4dm Container created 54m Normal Pulled pod/payload-pre-processing-e2e-api-a-de31dd-7579cc4f79-rq4dm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 54m Normal SuccessfulDelete replicaset/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c Deleted pod: e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-9j5tr 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0ad7c5-openshift-default New size: 6; reason: All metrics below target 54m Normal SuccessfulDelete replicaset/e2e-worker-w5-54114e-openshift-default-7fc467ddd7 Deleted pod: e2e-worker-w5-54114e-openshift-default-7fc467ddd7-p4vsn 54m Normal SuccessfulDelete replicaset/e2e-worker-w2-421583-openshift-default-5cb767b447 Deleted pod: e2e-worker-w2-421583-openshift-default-5cb767b447-xkmrr 54m Normal SuccessfulDelete replicaset/e2e-worker-w5-54114e-openshift-default-7fc467ddd7 Deleted pod: e2e-worker-w5-54114e-openshift-default-7fc467ddd7-6wn2g 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-54114e-openshift-default New size: 6; reason: All metrics below target 54m Normal ScalingReplicaSet deployment/e2e-worker-w5-54114e-openshift-default Scaled down replica set e2e-worker-w5-54114e-openshift-default-7fc467ddd7 from 8 to 6 54m Normal Killing pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-6wn2g Stopping container istio-proxy 54m Normal Killing pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-p4vsn Stopping container istio-proxy 54m Normal Killing pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-9j5tr Stopping container istio-proxy 54m Normal Killing pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-crp67 Stopping container istio-proxy 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-421583-openshift-default New size: 3; reason: All metrics below target 54m Normal ScalingReplicaSet deployment/e2e-worker-w2-421583-openshift-default Scaled down replica set e2e-worker-w2-421583-openshift-default-5cb767b447 from 4 to 3 54m Normal Killing pod/e2e-worker-w2-421583-openshift-default-5cb767b447-xkmrr Stopping container istio-proxy 54m Normal SuccessfulDelete replicaset/e2e-worker-w3-d52c59-openshift-default-5466968fb7 Deleted pod: e2e-worker-w3-d52c59-openshift-default-5466968fb7-gt2jk 54m Normal SuccessfulDelete replicaset/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 Deleted pod: e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-crp67 54m Normal ScalingReplicaSet deployment/e2e-worker-w4-0ad7c5-openshift-default Scaled down replica set e2e-worker-w4-0ad7c5-openshift-default-788b484d5c from 7 to 6 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-d52c59-openshift-default New size: 6; reason: All metrics below target 54m Normal ScalingReplicaSet deployment/e2e-worker-w3-d52c59-openshift-default Scaled down replica set e2e-worker-w3-d52c59-openshift-default-5466968fb7 from 7 to 6 54m Normal ScalingReplicaSet deployment/e2e-worker-w1-2b1e88-openshift-default Scaled down replica set e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 from 8 to 7 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-2b1e88-openshift-default New size: 7; reason: All metrics below target 54m Normal Killing pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-gt2jk Stopping container istio-proxy 53m Normal Killing pod/e2e-mt-d399fb07-openshift-default-6d54788969-4z9xf Stopping container istio-proxy 53m Normal Killing pod/e2e-mt-d399fb07-openshift-default-6d54788969-k9sr9 Stopping container istio-proxy 53m Normal Killing pod/e2e-mt-d399fb07-openshift-default-6d54788969-47qsn Stopping container istio-proxy 53m Normal SuccessfulCreate replicaset/e2e-api-a-30edd7-openshift-default-69b49fb475 Created pod: e2e-api-a-30edd7-openshift-default-69b49fb475-6hzw2 53m Normal ScalingReplicaSet deployment/e2e-api-a-30edd7-openshift-default Scaled up replica set e2e-api-a-30edd7-openshift-default-69b49fb475 from 0 to 1 53m Normal AddedLabel gateway/e2e-api-a-30edd7 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-30edd7 53m Normal AddedInterface pod/e2e-api-a-30edd7-openshift-default-69b49fb475-6hzw2 Add eth0 [10.133.0.182/23] from ovn-kubernetes 53m Warning FailedCreate replicaset/payload-processing-e2e-api-a-30edd7-57c7586d97 Error creating: pods "payload-processing-e2e-api-a-30edd7-57c7586d97-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-30edd7: serviceaccount "payload-processing-e2e-api-a-30edd7" not found 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-30edd7 Scaled up replica set payload-pre-processing-e2e-api-a-30edd7-5699985987 from 0 to 1 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-30edd7-57c7586d97 Created pod: payload-processing-e2e-api-a-30edd7-57c7586d97-zmlbs 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-30edd7-5699985987 Error creating: pods "payload-pre-processing-e2e-api-a-30edd7-5699985987-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-30edd7: serviceaccount "payload-processing-e2e-api-a-30edd7" not found 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-30edd7 Scaled up replica set payload-processing-e2e-api-a-30edd7-57c7586d97 from 0 to 1 53m Normal SuccessfulCreate replicaset/e2e-api-b-bf9c2f-openshift-default-7c859cb8dc Created pod: e2e-api-b-bf9c2f-openshift-default-7c859cb8dc-d962b 53m Normal AddedInterface pod/payload-processing-e2e-api-a-30edd7-57c7586d97-zmlbs Add eth0 [10.133.0.183/23] from ovn-kubernetes 53m Normal ScalingReplicaSet deployment/e2e-api-b-bf9c2f-openshift-default Scaled up replica set e2e-api-b-bf9c2f-openshift-default-7c859cb8dc from 0 to 1 53m Normal AddedLabel gateway/e2e-api-b-bf9c2f Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-bf9c2f 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-30edd7-5699985987 Created pod: payload-pre-processing-e2e-api-a-30edd7-5699985987-rczc8 53m Normal Created pod/payload-processing-e2e-api-a-30edd7-57c7586d97-zmlbs Container created 53m Normal Started pod/payload-pre-processing-e2e-api-a-30edd7-5699985987-rczc8 Container started 53m Normal Pulled pod/payload-processing-e2e-api-a-30edd7-57c7586d97-zmlbs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-30edd7-5699985987-rczc8 Add eth0 [10.133.0.184/23] from ovn-kubernetes 53m Normal Started pod/payload-processing-e2e-api-a-30edd7-57c7586d97-zmlbs Container started 53m Normal Created pod/payload-pre-processing-e2e-api-a-30edd7-5699985987-rczc8 Container created 53m Normal AddedInterface pod/e2e-api-b-bf9c2f-openshift-default-7c859cb8dc-d962b Add eth0 [10.133.0.185/23] from ovn-kubernetes 53m Normal Pulled pod/payload-pre-processing-e2e-api-a-30edd7-5699985987-rczc8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-bf9c2f Scaled up replica set payload-pre-processing-e2e-api-b-bf9c2f-69f6d78d99 from 0 to 1 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-bf9c2f-69f6d78d99 Created pod: payload-pre-processing-e2e-api-b-bf9c2f-69f6d78d99-b2xh6 53m Warning FailedCreate replicaset/payload-processing-e2e-api-b-bf9c2f-59899c7df Error creating: pods "payload-processing-e2e-api-b-bf9c2f-59899c7df-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-bf9c2f: serviceaccount "payload-processing-e2e-api-b-bf9c2f" not found 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-bf9c2f-69f6d78d99 Error creating: pods "payload-pre-processing-e2e-api-b-bf9c2f-69f6d78d99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-bf9c2f: serviceaccount "payload-processing-e2e-api-b-bf9c2f" not found 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-bf9c2f Scaled up replica set payload-processing-e2e-api-b-bf9c2f-59899c7df from 0 to 1 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-bf9c2f-59899c7df Created pod: payload-processing-e2e-api-b-bf9c2f-59899c7df-58lpq 53m Normal Created pod/payload-processing-e2e-api-b-bf9c2f-59899c7df-58lpq Container created 53m Normal Pulled pod/payload-pre-processing-e2e-api-b-bf9c2f-69f6d78d99-b2xh6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Created pod/payload-pre-processing-e2e-api-b-bf9c2f-69f6d78d99-b2xh6 Container created 53m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-bf9c2f-69f6d78d99-b2xh6 Add eth0 [10.132.0.138/23] from ovn-kubernetes 53m Normal Started pod/payload-pre-processing-e2e-api-b-bf9c2f-69f6d78d99-b2xh6 Container started 53m Normal Pulled pod/payload-processing-e2e-api-b-bf9c2f-59899c7df-58lpq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/payload-processing-e2e-api-b-bf9c2f-59899c7df-58lpq Add eth0 [10.132.0.139/23] from ovn-kubernetes 53m Normal Started pod/payload-processing-e2e-api-b-bf9c2f-59899c7df-58lpq Container started 53m Normal ScalingReplicaSet deployment/e2e-api-a-370ec2-openshift-default Scaled up replica set e2e-api-a-370ec2-openshift-default-6fccc4f697 from 1 to 4 53m Normal Pulled pod/e2e-api-a-f443d6-openshift-default-6dc8b74cd7-nkzfv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal SuccessfulCreate replicaset/e2e-api-a-370ec2-openshift-default-6fccc4f697 Created pod: e2e-api-a-370ec2-openshift-default-6fccc4f697-xwkss 53m Normal SuccessfulCreate replicaset/e2e-api-a-370ec2-openshift-default-6fccc4f697 Created pod: e2e-api-a-370ec2-openshift-default-6fccc4f697-9nsmf 53m Normal SuccessfulCreate replicaset/e2e-api-a-370ec2-openshift-default-6fccc4f697 Created pod: e2e-api-a-370ec2-openshift-default-6fccc4f697-7v8wh 53m Normal Pulled pod/e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5-6m98h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5-6m98h Container created 53m Normal Started pod/e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5-6m98h Container started 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-a-370ec2-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 53m Normal Started pod/e2e-api-a-f443d6-openshift-default-6dc8b74cd7-nkzfv Container started 53m Normal Created pod/e2e-api-a-f443d6-openshift-default-6dc8b74cd7-nkzfv Container created 53m Normal AddedInterface pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-7v8wh Add eth0 [10.132.0.140/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-9nsmf Add eth0 [10.134.0.99/23] from ovn-kubernetes 53m Normal Created pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-78w6t Container created 53m Normal SuccessfulCreate replicaset/e2e-api-a-7a1ee7-openshift-default-68c5c46ccc Created pod: e2e-api-a-7a1ee7-openshift-default-68c5c46ccc-5gx2f 53m Normal Started pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-78w6t Container started 53m Normal Created pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-xwkss Container created 53m Normal Started pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-9nsmf Container started 53m Normal AddedLabel gateway/e2e-api-a-7a1ee7 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-7a1ee7 53m Normal ScalingReplicaSet deployment/e2e-api-a-7a1ee7-openshift-default Scaled up replica set e2e-api-a-7a1ee7-openshift-default-68c5c46ccc from 0 to 1 53m Normal AddedInterface pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-xwkss Add eth0 [10.133.0.186/23] from ovn-kubernetes 53m Normal Created pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-9nsmf Container created 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-bf9c2f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Normal Pulled pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-7v8wh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Pulled pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-9nsmf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-7v8wh Container created 53m Normal Started pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-7v8wh Container started 53m Normal Pulled pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-78w6t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-xwkss Container started 53m Normal Pulled pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-xwkss Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-bf9c2f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Normal AddedInterface pod/e2e-api-a-7a1ee7-openshift-default-68c5c46ccc-5gx2f Add eth0 [10.133.0.187/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447 Created pod: e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447-l89ql 53m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-d9f1c2-openshift-default Scaled up replica set e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447 from 0 to 1 53m Normal AddedLabel gateway/e2e-shared-b-w6-d9f1c2 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w6-d9f1c2 53m Normal AddedInterface pod/e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447-l89ql Add eth0 [10.133.0.188/23] from ovn-kubernetes 53m Warning Unhealthy pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-9nsmf Startup probe failed: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused 53m Warning ProbeError pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-9nsmf Startup probe error: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused... 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-0ff5b0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-7a1ee7 Scaled up replica set payload-processing-e2e-api-a-7a1ee7-7dbbd77679 from 0 to 1 53m Warning Unhealthy pod/e2e-api-a-f443d6-openshift-default-6dc8b74cd7-nkzfv Startup probe failed: Get "http://10.133.0.164:15021/healthz/ready": dial tcp 10.133.0.164:15021: connect: connection refused 53m Warning ProbeError pod/e2e-api-a-f443d6-openshift-default-6dc8b74cd7-nkzfv Startup probe error: Get "http://10.133.0.164:15021/healthz/ready": dial tcp 10.133.0.164:15021: connect: connection refused... 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-7a1ee7-7dbbd77679 Created pod: payload-processing-e2e-api-a-7a1ee7-7dbbd77679-kbzmg 53m Warning FailedCreate replicaset/payload-processing-e2e-api-a-7a1ee7-7dbbd77679 Error creating: pods "payload-processing-e2e-api-a-7a1ee7-7dbbd77679-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-7a1ee7: serviceaccount "payload-processing-e2e-api-a-7a1ee7" not found 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-7a1ee7-55c68b95cf Error creating: pods "payload-pre-processing-e2e-api-a-7a1ee7-55c68b95cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-7a1ee7: serviceaccount "payload-processing-e2e-api-a-7a1ee7" not found 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-7a1ee7-55c68b95cf Created pod: payload-pre-processing-e2e-api-a-7a1ee7-55c68b95cf-s428w 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-7a1ee7 Scaled up replica set payload-pre-processing-e2e-api-a-7a1ee7-55c68b95cf from 0 to 1 53m Warning Unhealthy pod/e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5-6m98h Startup probe failed: Get "http://10.132.0.125:15021/healthz/ready": dial tcp 10.132.0.125:15021: connect: connection refused 53m Warning ProbeError pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-xwkss Startup probe error: Get "http://10.133.0.186:15021/healthz/ready": dial tcp 10.133.0.186:15021: connect: connection refused... 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-0ff5b0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning Unhealthy pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-xwkss Startup probe failed: Get "http://10.133.0.186:15021/healthz/ready": dial tcp 10.133.0.186:15021: connect: connection refused 53m Warning ProbeError pod/e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5-6m98h Startup probe error: Get "http://10.132.0.125:15021/healthz/ready": dial tcp 10.132.0.125:15021: connect: connection refused... 53m Normal Started pod/payload-processing-e2e-api-a-7a1ee7-7dbbd77679-kbzmg Container started 53m Normal Created pod/payload-pre-processing-e2e-api-a-7a1ee7-55c68b95cf-s428w Container created 53m Warning Unhealthy pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-7v8wh Startup probe failed: Get "http://10.132.0.140:15021/healthz/ready": dial tcp 10.132.0.140:15021: connect: connection refused 53m Warning ProbeError pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-7v8wh Startup probe error: Get "http://10.132.0.140:15021/healthz/ready": dial tcp 10.132.0.140:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-78w6t Startup probe failed: Get "http://10.132.0.121:15021/healthz/ready": dial tcp 10.132.0.121:15021: connect: connection refused 53m Warning ProbeError pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-78w6t Startup probe error: Get "http://10.132.0.121:15021/healthz/ready": dial tcp 10.132.0.121:15021: connect: connection refused... 53m Normal Started pod/payload-pre-processing-e2e-api-a-7a1ee7-55c68b95cf-s428w Container started 53m Normal AddedInterface pod/payload-processing-e2e-api-a-7a1ee7-7dbbd77679-kbzmg Add eth0 [10.132.0.141/23] from ovn-kubernetes 53m Normal Pulled pod/payload-processing-e2e-api-a-7a1ee7-7dbbd77679-kbzmg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Created pod/payload-processing-e2e-api-a-7a1ee7-7dbbd77679-kbzmg Container created 53m Normal Pulled pod/payload-pre-processing-e2e-api-a-7a1ee7-55c68b95cf-s428w Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-7a1ee7-55c68b95cf-s428w Add eth0 [10.132.0.142/23] from ovn-kubernetes 53m Normal ScalingReplicaSet deployment/e2e-api-b-3d38f6-openshift-default Scaled up replica set e2e-api-b-3d38f6-openshift-default-79b69d9955 from 0 to 1 53m Normal AddedLabel gateway/e2e-api-b-3d38f6 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-3d38f6 53m Normal SuccessfulCreate replicaset/e2e-api-b-3d38f6-openshift-default-79b69d9955 Created pod: e2e-api-b-3d38f6-openshift-default-79b69d9955-wjvpp 53m Normal AddedInterface pod/e2e-api-b-3d38f6-openshift-default-79b69d9955-wjvpp Add eth0 [10.132.0.143/23] from ovn-kubernetes 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w6-d9f1c2-5bf66857f Error creating: pods "payload-pre-processing-e2e-shared-b-w6-d9f1c2-5bf66857f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-d9f1c2: serviceaccount "payload-processing-e2e-shared-b-w6-d9f1c2" not found 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w6-d9f1c2-6475ccbbb9 Created pod: payload-processing-e2e-shared-b-w6-d9f1c2-6475ccbbb9-x7xgs 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-f443d6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-f443d6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w6-d9f1c2-5bf66857f Created pod: payload-pre-processing-e2e-shared-b-w6-d9f1c2-5bf66857f-zbxhk 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w6-d9f1c2 Scaled up replica set payload-processing-e2e-shared-b-w6-d9f1c2-6475ccbbb9 from 0 to 1 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w6-d9f1c2 Scaled up replica set payload-pre-processing-e2e-shared-b-w6-d9f1c2-5bf66857f from 0 to 1 53m Normal AddedInterface pod/payload-processing-e2e-shared-b-w6-d9f1c2-6475ccbbb9-x7xgs Add eth0 [10.132.0.144/23] from ovn-kubernetes 53m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w6-d9f1c2-6475ccbbb9 Error creating: pods "payload-processing-e2e-shared-b-w6-d9f1c2-6475ccbbb9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-d9f1c2: serviceaccount "payload-processing-e2e-shared-b-w6-d9f1c2" not found 53m Normal Started pod/payload-processing-e2e-shared-b-w6-d9f1c2-6475ccbbb9-x7xgs Container started 53m Normal Started pod/payload-pre-processing-e2e-shared-b-w6-d9f1c2-5bf66857f-zbxhk Container started 53m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w6-d9f1c2-5bf66857f-zbxhk Add eth0 [10.132.0.145/23] from ovn-kubernetes 53m Normal Created pod/payload-processing-e2e-shared-b-w6-d9f1c2-6475ccbbb9-x7xgs Container created 53m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w6-d9f1c2-5bf66857f-zbxhk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Created pod/payload-pre-processing-e2e-shared-b-w6-d9f1c2-5bf66857f-zbxhk Container created 53m Normal Pulled pod/payload-processing-e2e-shared-b-w6-d9f1c2-6475ccbbb9-x7xgs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-370ec2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-370ec2-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-3d38f6-8fdb5988c Created pod: payload-pre-processing-e2e-api-b-3d38f6-8fdb5988c-rjl42 53m Warning FailedCreate replicaset/payload-processing-e2e-api-b-3d38f6-5874cbccb9 Error creating: pods "payload-processing-e2e-api-b-3d38f6-5874cbccb9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-3d38f6: serviceaccount "payload-processing-e2e-api-b-3d38f6" not found 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-3d38f6 Scaled up replica set payload-processing-e2e-api-b-3d38f6-5874cbccb9 from 0 to 1 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-3d38f6-5874cbccb9 Created pod: payload-processing-e2e-api-b-3d38f6-5874cbccb9-4r8w9 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-3d38f6-8fdb5988c Error creating: pods "payload-pre-processing-e2e-api-b-3d38f6-8fdb5988c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-3d38f6: serviceaccount "payload-processing-e2e-api-b-3d38f6" not found 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-3d38f6 Scaled up replica set payload-pre-processing-e2e-api-b-3d38f6-8fdb5988c from 0 to 1 53m Normal Started pod/payload-processing-e2e-api-b-3d38f6-5874cbccb9-4r8w9 Container started 53m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-3d38f6-8fdb5988c-rjl42 Add eth0 [10.133.0.190/23] from ovn-kubernetes 53m Normal Pulled pod/payload-pre-processing-e2e-api-b-3d38f6-8fdb5988c-rjl42 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Created pod/payload-pre-processing-e2e-api-b-3d38f6-8fdb5988c-rjl42 Container created 53m Normal Started pod/payload-pre-processing-e2e-api-b-3d38f6-8fdb5988c-rjl42 Container started 53m Normal Pulled pod/payload-processing-e2e-api-b-3d38f6-5874cbccb9-4r8w9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/payload-processing-e2e-api-b-3d38f6-5874cbccb9-4r8w9 Add eth0 [10.133.0.189/23] from ovn-kubernetes 53m Normal Created pod/payload-processing-e2e-api-b-3d38f6-5874cbccb9-4r8w9 Container created 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-0ff5b0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-0ff5b0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Normal Killing pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-8r62w Stopping container istio-proxy 53m Normal SuccessfulDelete replicaset/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc Deleted pod: e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-sqxvr 53m Normal Killing pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-z7ggc Stopping container istio-proxy 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0ad7c5-openshift-default New size: 5; reason: All metrics below target 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-2b1e88-openshift-default New size: 5; reason: All metrics below target 53m Normal Killing pod/payload-processing-e2e-mt-2949c143-5bdffff8fc-jk5zn Stopping container payload-processing 53m Normal SuccessfulDelete replicaset/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 Deleted pod: e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-z7ggc 53m Normal ScalingReplicaSet deployment/e2e-worker-w1-2b1e88-openshift-default Scaled down replica set e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 from 7 to 5 53m Normal ScalingReplicaSet deployment/e2e-worker-w3-d52c59-openshift-default Scaled down replica set e2e-worker-w3-d52c59-openshift-default-5466968fb7 from 6 to 5 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-d52c59-openshift-default New size: 5; reason: All metrics below target 53m Normal ScalingReplicaSet deployment/e2e-worker-w6-012ba0-openshift-default Scaled down replica set e2e-worker-w6-012ba0-openshift-default-846bcc5bfc from 6 to 5 53m Normal SuccessfulDelete replicaset/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 Deleted pod: e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-nflrt 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-012ba0-openshift-default New size: 5; reason: All metrics below target 53m Normal SuccessfulDelete replicaset/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c Deleted pod: e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8wzt7 53m Normal Killing pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-sqxvr Stopping container istio-proxy 53m Normal ScalingReplicaSet deployment/e2e-worker-w0-09d0d5-openshift-default Scaled down replica set e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 from 6 to 5 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-09d0d5-openshift-default New size: 5; reason: All metrics below target 53m Normal ScalingReplicaSet deployment/e2e-worker-w5-54114e-openshift-default Scaled down replica set e2e-worker-w5-54114e-openshift-default-7fc467ddd7 from 6 to 5 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-54114e-openshift-default New size: 5; reason: All metrics below target 53m Normal SuccessfulDelete replicaset/e2e-worker-w3-d52c59-openshift-default-5466968fb7 Deleted pod: e2e-worker-w3-d52c59-openshift-default-5466968fb7-ssk97 53m Normal SuccessfulDelete replicaset/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 Deleted pod: e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-58wkt 53m Normal Killing pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-ssk97 Stopping container istio-proxy 53m Normal Killing pod/payload-pre-processing-e2e-mt-2949c143-5f67f57d8c-pw4d8 Stopping container payload-pre-processing 53m Normal Killing pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8wzt7 Stopping container istio-proxy 53m Normal SuccessfulDelete replicaset/e2e-worker-w5-54114e-openshift-default-7fc467ddd7 Deleted pod: e2e-worker-w5-54114e-openshift-default-7fc467ddd7-8r62w 53m Normal ScalingReplicaSet deployment/e2e-worker-w4-0ad7c5-openshift-default Scaled down replica set e2e-worker-w4-0ad7c5-openshift-default-788b484d5c from 6 to 5 53m Normal SuccessfulCreate replicaset/e2e-api-a-370ec2-openshift-default-6fccc4f697 Created pod: e2e-api-a-370ec2-openshift-default-6fccc4f697-4cm9t 53m Normal SuccessfulCreate replicaset/e2e-api-a-370ec2-openshift-default-6fccc4f697 Created pod: e2e-api-a-370ec2-openshift-default-6fccc4f697-2mvj6 53m Normal Killing pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-58wkt Stopping container istio-proxy 53m Normal SuccessfulCreate replicaset/e2e-api-a-370ec2-openshift-default-6fccc4f697 Created pod: e2e-api-a-370ec2-openshift-default-6fccc4f697-w7p95 53m Normal Killing pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-nflrt Stopping container istio-proxy 53m Normal SuccessfulCreate replicaset/e2e-api-a-370ec2-openshift-default-6fccc4f697 Created pod: e2e-api-a-370ec2-openshift-default-6fccc4f697-j564x 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-a-370ec2-openshift-default New size: 8; reason: All metrics below target 53m Normal ScalingReplicaSet deployment/e2e-api-a-370ec2-openshift-default Scaled up replica set e2e-api-a-370ec2-openshift-default-6fccc4f697 from 4 to 8 53m Normal Created pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-w7p95 Container created 53m Normal Created pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-4cm9t Container created 53m Normal Started pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-2mvj6 Container started 53m Normal Created pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-j564x Container created 53m Normal Pulled pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-2mvj6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-2mvj6 Add eth0 [10.133.0.192/23] from ovn-kubernetes 53m Normal Started pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-4cm9t Container started 53m Normal Started pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-w7p95 Container started 53m Normal Pulled pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-j564x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-j564x Add eth0 [10.133.0.193/23] from ovn-kubernetes 53m Normal Started pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-j564x Container started 53m Normal Pulled pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-w7p95 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Pulled pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-4cm9t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-4cm9t Add eth0 [10.134.0.102/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-w7p95 Add eth0 [10.132.0.146/23] from ovn-kubernetes 53m Normal Created pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-2mvj6 Container created 52m Warning Unhealthy pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-4cm9t Startup probe failed: Get "http://10.134.0.102:15021/healthz/ready": dial tcp 10.134.0.102:15021: connect: connection refused 52m Warning ProbeError pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-4cm9t Startup probe error: Get "http://10.134.0.102:15021/healthz/ready": dial tcp 10.134.0.102:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-w7p95 Startup probe failed: Get "http://10.132.0.146:15021/healthz/ready": dial tcp 10.132.0.146:15021: connect: connection refused 52m Warning ProbeError pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-2mvj6 Startup probe error: Get "http://10.133.0.192:15021/healthz/ready": dial tcp 10.133.0.192:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-2mvj6 Startup probe failed: Get "http://10.133.0.192:15021/healthz/ready": dial tcp 10.133.0.192:15021: connect: connection refused 52m Warning ProbeError pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-j564x Startup probe error: Get "http://10.133.0.193:15021/healthz/ready": dial tcp 10.133.0.193:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-j564x Startup probe failed: Get "http://10.133.0.193:15021/healthz/ready": dial tcp 10.133.0.193:15021: connect: connection refused 52m Warning ProbeError pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-w7p95 Startup probe error: Get "http://10.132.0.146:15021/healthz/ready": dial tcp 10.132.0.146:15021: connect: connection refused... 52m Normal AddedInterface pod/e2e-api-b-8b3e43-openshift-default-78bccfc7fc-n448p Add eth0 [10.132.0.147/23] from ovn-kubernetes 52m Normal SuccessfulCreate replicaset/e2e-api-b-8b3e43-openshift-default-78bccfc7fc Created pod: e2e-api-b-8b3e43-openshift-default-78bccfc7fc-n448p 52m Normal ScalingReplicaSet deployment/e2e-api-b-8b3e43-openshift-default Scaled up replica set e2e-api-b-8b3e43-openshift-default-78bccfc7fc from 0 to 1 52m Normal AddedLabel gateway/e2e-api-b-8b3e43 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-8b3e43 52m Normal Pulled pod/e2e-api-a-de31dd-openshift-default-56f5bd455-d8vcq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Warning ProbeError pod/e2e-api-a-de31dd-openshift-default-56f5bd455-d8vcq Readiness probe error: Get "http://10.132.0.130:15021/healthz/ready": read tcp 10.132.0.2:53948->10.132.0.130:15021: read: connection reset by peer... 52m Warning Unhealthy pod/e2e-api-a-de31dd-openshift-default-56f5bd455-d8vcq Readiness probe failed: Get "http://10.132.0.130:15021/healthz/ready": read tcp 10.132.0.2:53948->10.132.0.130:15021: read: connection reset by peer 52m Normal Started pod/e2e-api-a-de31dd-openshift-default-56f5bd455-d8vcq Container started 52m Normal Created pod/e2e-api-a-de31dd-openshift-default-56f5bd455-d8vcq Container created 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-f443d6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-f443d6-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Normal ScalingReplicaSet deployment/e2e-api-a-370ec2-openshift-default Scaled up replica set e2e-api-a-370ec2-openshift-default-6fccc4f697 from 8 to 10 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-a-370ec2-openshift-default New size: 10; reason: All metrics below target 52m Normal SuccessfulCreate replicaset/e2e-api-a-370ec2-openshift-default-6fccc4f697 (combined from similar events): Created pod: e2e-api-a-370ec2-openshift-default-6fccc4f697-mhwhp 52m Normal SuccessfulCreate replicaset/e2e-api-a-370ec2-openshift-default-6fccc4f697 Created pod: e2e-api-a-370ec2-openshift-default-6fccc4f697-2bzhm 52m Normal Started pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-2bzhm Container started 52m Normal Pulled pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-2bzhm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Pulled pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-mhwhp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-mhwhp Container created 52m Normal Started pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-mhwhp Container started 52m Normal Created pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-2bzhm Container created 52m Normal AddedInterface pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-mhwhp Add eth0 [10.132.0.148/23] from ovn-kubernetes 52m Normal AddedInterface pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-2bzhm Add eth0 [10.134.0.104/23] from ovn-kubernetes 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-8b3e43-57f97b8bcf Created pod: payload-pre-processing-e2e-api-b-8b3e43-57f97b8bcf-x88zv 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-8b3e43 Scaled up replica set payload-processing-e2e-api-b-8b3e43-685659576c from 0 to 1 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-8b3e43 Scaled up replica set payload-pre-processing-e2e-api-b-8b3e43-57f97b8bcf from 0 to 1 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-8b3e43-685659576c Created pod: payload-processing-e2e-api-b-8b3e43-685659576c-mxc2x 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-8b3e43-57f97b8bcf Error creating: pods "payload-pre-processing-e2e-api-b-8b3e43-57f97b8bcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-8b3e43: serviceaccount "payload-processing-e2e-api-b-8b3e43" not found 52m Warning FailedCreate replicaset/payload-processing-e2e-api-b-8b3e43-685659576c Error creating: pods "payload-processing-e2e-api-b-8b3e43-685659576c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-8b3e43: serviceaccount "payload-processing-e2e-api-b-8b3e43" not found 52m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-8b3e43-57f97b8bcf-x88zv Add eth0 [10.132.0.149/23] from ovn-kubernetes 52m Normal Pulled pod/payload-pre-processing-e2e-api-b-8b3e43-57f97b8bcf-x88zv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Pulled pod/payload-processing-e2e-api-b-8b3e43-685659576c-mxc2x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/payload-processing-e2e-api-b-8b3e43-685659576c-mxc2x Add eth0 [10.132.0.150/23] from ovn-kubernetes 52m Normal Started pod/payload-processing-e2e-api-b-8b3e43-685659576c-mxc2x Container started 52m Normal Created pod/payload-pre-processing-e2e-api-b-8b3e43-57f97b8bcf-x88zv Container created 52m Warning Unhealthy pod/e2e-api-a-de31dd-openshift-default-56f5bd455-d8vcq Startup probe failed: Get "http://10.132.0.130:15021/healthz/ready": dial tcp 10.132.0.130:15021: connect: connection refused 52m Warning ProbeError pod/e2e-api-a-de31dd-openshift-default-56f5bd455-d8vcq Startup probe error: Get "http://10.132.0.130:15021/healthz/ready": dial tcp 10.132.0.130:15021: connect: connection refused... 52m Normal Started pod/payload-pre-processing-e2e-api-b-8b3e43-57f97b8bcf-x88zv Container started 52m Normal Created pod/payload-processing-e2e-api-b-8b3e43-685659576c-mxc2x Container created 52m Warning Unhealthy pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-2bzhm Startup probe failed: Get "http://10.134.0.104:15021/healthz/ready": dial tcp 10.134.0.104:15021: connect: connection refused 52m Warning ProbeError pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-2bzhm Startup probe error: Get "http://10.134.0.104:15021/healthz/ready": dial tcp 10.134.0.104:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-mhwhp Startup probe failed: Get "http://10.132.0.148:15021/healthz/ready": dial tcp 10.132.0.148:15021: connect: connection refused 52m Warning ProbeError pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-mhwhp Startup probe error: Get "http://10.132.0.148:15021/healthz/ready": dial tcp 10.132.0.148:15021: connect: connection refused... 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-de31dd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-de31dd-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-de31dd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-de31dd-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-8b3e43-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-8b3e43-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Normal Killing pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-5zllw Stopping container istio-proxy 52m Normal ScalingReplicaSet deployment/e2e-worker-w3-d52c59-openshift-default Scaled down replica set e2e-worker-w3-d52c59-openshift-default-5466968fb7 from 5 to 4 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-012ba0-openshift-default New size: 4; reason: All metrics below target 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-d52c59-openshift-default New size: 4; reason: All metrics below target 52m Normal ScalingReplicaSet deployment/e2e-worker-w6-012ba0-openshift-default Scaled down replica set e2e-worker-w6-012ba0-openshift-default-846bcc5bfc from 5 to 4 52m Normal SuccessfulDelete replicaset/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c Deleted pod: e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-rpk4x 52m Normal SuccessfulDelete replicaset/e2e-worker-w3-d52c59-openshift-default-5466968fb7 Deleted pod: e2e-worker-w3-d52c59-openshift-default-5466968fb7-7m76p 52m Normal Killing pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-rpk4x Stopping container istio-proxy 52m Normal Killing pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-7m76p Stopping container istio-proxy 52m Normal SuccessfulDelete replicaset/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc Deleted pod: e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-5zllw 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0ad7c5-openshift-default New size: 4; reason: All metrics below target 52m Normal AddedLabel gateway/e2e-api-b-4f7439 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-4f7439 52m Normal ScalingReplicaSet deployment/e2e-api-b-4f7439-openshift-default Scaled up replica set e2e-api-b-4f7439-openshift-default-84f666696f from 0 to 1 52m Normal SuccessfulCreate replicaset/e2e-api-b-4f7439-openshift-default-84f666696f Created pod: e2e-api-b-4f7439-openshift-default-84f666696f-7wlcj 52m Normal AddedInterface pod/e2e-api-b-4f7439-openshift-default-84f666696f-7wlcj Add eth0 [10.132.0.152/23] from ovn-kubernetes 52m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-4f7439-55564f8b9f Error creating: pods "payload-pre-processing-e2e-api-b-4f7439-55564f8b9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-4f7439: serviceaccount "payload-processing-e2e-api-b-4f7439" not found 52m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-4f7439 Scaled up replica set payload-pre-processing-e2e-api-b-4f7439-55564f8b9f from 0 to 1 52m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-4f7439-db569fdbc Created pod: payload-processing-e2e-api-b-4f7439-db569fdbc-lx8l8 52m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-4f7439 Scaled up replica set payload-processing-e2e-api-b-4f7439-db569fdbc from 0 to 1 52m Warning FailedCreate replicaset/payload-processing-e2e-api-b-4f7439-db569fdbc Error creating: pods "payload-processing-e2e-api-b-4f7439-db569fdbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-4f7439: serviceaccount "payload-processing-e2e-api-b-4f7439" not found 52m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-4f7439-55564f8b9f Created pod: payload-pre-processing-e2e-api-b-4f7439-55564f8b9f-xttgk 52m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-4f7439-55564f8b9f-xttgk Add eth0 [10.133.0.194/23] from ovn-kubernetes 52m Normal Created pod/payload-pre-processing-e2e-api-b-4f7439-55564f8b9f-xttgk Container created 52m Normal Pulled pod/payload-pre-processing-e2e-api-b-4f7439-55564f8b9f-xttgk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Started pod/payload-processing-e2e-api-b-4f7439-db569fdbc-lx8l8 Container started 52m Normal Created pod/payload-processing-e2e-api-b-4f7439-db569fdbc-lx8l8 Container created 52m Normal AddedInterface pod/payload-processing-e2e-api-b-4f7439-db569fdbc-lx8l8 Add eth0 [10.133.0.195/23] from ovn-kubernetes 52m Normal Started pod/payload-pre-processing-e2e-api-b-4f7439-55564f8b9f-xttgk Container started 52m Normal Pulled pod/payload-processing-e2e-api-b-4f7439-db569fdbc-lx8l8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-api-a-30edd7-openshift-default-69b49fb475-6hzw2 Container created 52m Warning ProbeError pod/e2e-api-a-30edd7-openshift-default-69b49fb475-6hzw2 Readiness probe error: Get "http://10.133.0.182:15021/healthz/ready": read tcp 10.133.0.2:48960->10.133.0.182:15021: read: connection reset by peer... 52m Normal Started pod/e2e-api-a-30edd7-openshift-default-69b49fb475-6hzw2 Container started 52m Warning Unhealthy pod/e2e-api-a-30edd7-openshift-default-69b49fb475-6hzw2 Readiness probe failed: Get "http://10.133.0.182:15021/healthz/ready": read tcp 10.133.0.2:48960->10.133.0.182:15021: read: connection reset by peer 52m Normal Pulled pod/e2e-api-a-30edd7-openshift-default-69b49fb475-6hzw2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Warning Unhealthy pod/e2e-api-a-30edd7-openshift-default-69b49fb475-6hzw2 Startup probe failed: Get "http://10.133.0.182:15021/healthz/ready": dial tcp 10.133.0.182:15021: connect: connection refused 51m Warning ProbeError pod/e2e-api-a-30edd7-openshift-default-69b49fb475-6hzw2 Startup probe error: Get "http://10.133.0.182:15021/healthz/ready": dial tcp 10.133.0.182:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-api-b-bf9c2f-openshift-default-7c859cb8dc-d962b Readiness probe failed: Get "http://10.133.0.185:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-api-b-bf9c2f-openshift-default-7c859cb8dc-d962b Readiness probe error: Get "http://10.133.0.185:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Normal Pulled pod/e2e-api-b-bf9c2f-openshift-default-7c859cb8dc-d962b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-api-b-bf9c2f-openshift-default-7c859cb8dc-d962b Container created 51m Normal Started pod/e2e-api-b-bf9c2f-openshift-default-7c859cb8dc-d962b Container started 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-30edd7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-30edd7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Normal Killing pod/payload-pre-processing-e2e-api-a-370ec2-7957558899-vkjqx Stopping container payload-pre-processing 51m Normal Killing pod/payload-processing-e2e-api-a-370ec2-7658c775d7-t69wq Stopping container payload-processing 51m Warning ProbeError pod/e2e-api-b-bf9c2f-openshift-default-7c859cb8dc-d962b Startup probe error: Get "http://10.133.0.185:15021/healthz/ready": dial tcp 10.133.0.185:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-api-b-bf9c2f-openshift-default-7c859cb8dc-d962b Startup probe failed: Get "http://10.133.0.185:15021/healthz/ready": dial tcp 10.133.0.185:15021: connect: connection refused 51m Normal Killing pod/e2e-mt-2949c143-openshift-default-cc8cc894f-jnvwn Stopping container istio-proxy 51m Normal AddedLabel gateway/e2e-api-b-62236a Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-62236a 51m Normal SuccessfulCreate replicaset/e2e-api-b-62236a-openshift-default-899986fcf Created pod: e2e-api-b-62236a-openshift-default-899986fcf-8kdtl 51m Normal ScalingReplicaSet deployment/e2e-api-b-62236a-openshift-default Scaled up replica set e2e-api-b-62236a-openshift-default-899986fcf from 0 to 1 51m Normal AddedInterface pod/e2e-api-b-62236a-openshift-default-899986fcf-8kdtl Add eth0 [10.133.0.197/23] from ovn-kubernetes 51m Normal ScalingReplicaSet deployment/e2e-inf-a-6922d3-openshift-default Scaled up replica set e2e-inf-a-6922d3-openshift-default-689b5f95bc from 0 to 1 51m Normal AddedLabel gateway/e2e-inf-a-6922d3 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-6922d3 51m Normal SuccessfulCreate replicaset/e2e-inf-a-6922d3-openshift-default-689b5f95bc Created pod: e2e-inf-a-6922d3-openshift-default-689b5f95bc-bf77z 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-62236a Scaled up replica set payload-pre-processing-e2e-api-b-62236a-56cfbd57f9 from 0 to 1 51m Normal AddedInterface pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-bf77z Add eth0 [10.132.0.153/23] from ovn-kubernetes 51m Warning FailedCreate replicaset/payload-processing-e2e-api-b-62236a-754c5cbbb9 Error creating: pods "payload-processing-e2e-api-b-62236a-754c5cbbb9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-62236a: serviceaccount "payload-processing-e2e-api-b-62236a" not found 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-62236a-56cfbd57f9 Error creating: pods "payload-pre-processing-e2e-api-b-62236a-56cfbd57f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-62236a: serviceaccount "payload-processing-e2e-api-b-62236a" not found 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-62236a Scaled up replica set payload-processing-e2e-api-b-62236a-754c5cbbb9 from 0 to 1 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-62236a-754c5cbbb9 Created pod: payload-processing-e2e-api-b-62236a-754c5cbbb9-sxrtf 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-62236a-56cfbd57f9 Created pod: payload-pre-processing-e2e-api-b-62236a-56cfbd57f9-ldg6l 51m Normal Created pod/payload-processing-e2e-api-b-62236a-754c5cbbb9-sxrtf Container created 51m Normal Started pod/payload-processing-e2e-api-b-62236a-754c5cbbb9-sxrtf Container started 51m Normal Started pod/payload-pre-processing-e2e-api-b-62236a-56cfbd57f9-ldg6l Container started 51m Normal Created pod/payload-pre-processing-e2e-api-b-62236a-56cfbd57f9-ldg6l Container created 51m Normal AddedInterface pod/payload-processing-e2e-api-b-62236a-754c5cbbb9-sxrtf Add eth0 [10.132.0.155/23] from ovn-kubernetes 51m Normal Pulled pod/payload-pre-processing-e2e-api-b-62236a-56cfbd57f9-ldg6l Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-62236a-56cfbd57f9-ldg6l Add eth0 [10.132.0.154/23] from ovn-kubernetes 51m Normal Pulled pod/payload-processing-e2e-api-b-62236a-754c5cbbb9-sxrtf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-6922d3-59cf97fb57 Created pod: payload-processing-e2e-inf-a-6922d3-59cf97fb57-v5z2m 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-6922d3 Scaled up replica set payload-pre-processing-e2e-inf-a-6922d3-69c86dfc75 from 0 to 1 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-6922d3-69c86dfc75 Error creating: pods "payload-pre-processing-e2e-inf-a-6922d3-69c86dfc75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-6922d3: serviceaccount "payload-processing-e2e-inf-a-6922d3" not found 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-6922d3 Scaled up replica set payload-processing-e2e-inf-a-6922d3-59cf97fb57 from 0 to 1 51m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-6922d3-59cf97fb57 Error creating: pods "payload-processing-e2e-inf-a-6922d3-59cf97fb57-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-6922d3: serviceaccount "payload-processing-e2e-inf-a-6922d3" not found 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-30edd7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-6922d3-69c86dfc75 Created pod: payload-pre-processing-e2e-inf-a-6922d3-69c86dfc75-ktwbq 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-30edd7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Normal Created pod/payload-pre-processing-e2e-inf-a-6922d3-69c86dfc75-ktwbq Container created 51m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-6922d3-69c86dfc75-ktwbq Add eth0 [10.132.0.156/23] from ovn-kubernetes 51m Normal Pulled pod/payload-pre-processing-e2e-inf-a-6922d3-69c86dfc75-ktwbq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Created pod/payload-processing-e2e-inf-a-6922d3-59cf97fb57-v5z2m Container created 51m Normal Started pod/payload-processing-e2e-inf-a-6922d3-59cf97fb57-v5z2m Container started 51m Normal Started pod/payload-pre-processing-e2e-inf-a-6922d3-69c86dfc75-ktwbq Container started 51m Normal AddedInterface pod/payload-processing-e2e-inf-a-6922d3-59cf97fb57-v5z2m Add eth0 [10.133.0.199/23] from ovn-kubernetes 51m Normal Pulled pod/payload-processing-e2e-inf-a-6922d3-59cf97fb57-v5z2m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal ScalingReplicaSet deployment/e2e-worker-w2-421583-openshift-default Scaled down replica set e2e-worker-w2-421583-openshift-default-5cb767b447 from 3 to 2 51m Normal Killing pod/e2e-worker-w2-421583-openshift-default-5cb767b447-k4nm5 Stopping container istio-proxy 51m Warning ProbeError pod/e2e-api-b-3d38f6-openshift-default-79b69d9955-wjvpp Readiness probe error: Get "http://10.132.0.143:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-421583-openshift-default New size: 2; reason: All metrics below target 51m Warning Unhealthy pod/e2e-api-b-3d38f6-openshift-default-79b69d9955-wjvpp Readiness probe failed: Get "http://10.132.0.143:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Normal SuccessfulDelete replicaset/e2e-worker-w2-421583-openshift-default-5cb767b447 Deleted pod: e2e-worker-w2-421583-openshift-default-5cb767b447-k4nm5 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-bf9c2f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Normal Started pod/e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447-l89ql Container started 51m Normal Pulled pod/e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447-l89ql Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-api-b-3d38f6-openshift-default-79b69d9955-wjvpp Container started 51m Normal Created pod/e2e-api-b-3d38f6-openshift-default-79b69d9955-wjvpp Container created 51m Normal Pulled pod/e2e-api-b-3d38f6-openshift-default-79b69d9955-wjvpp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-bf9c2f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Normal Created pod/e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447-l89ql Container created 51m Normal Killing pod/payload-processing-e2e-api-a-f443d6-784b88c549-cwkhm Stopping container payload-processing 51m Normal Killing pod/payload-pre-processing-e2e-api-a-f443d6-6bcbc5c45f-72rp5 Stopping container payload-pre-processing 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-d9f1c2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-d9f1c2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning Unhealthy pod/e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447-l89ql Startup probe failed: Get "http://10.133.0.188:15021/healthz/ready": dial tcp 10.133.0.188:15021: connect: connection refused 51m Warning ProbeError pod/e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447-l89ql Startup probe error: Get "http://10.133.0.188:15021/healthz/ready": dial tcp 10.133.0.188:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-api-b-3d38f6-openshift-default-79b69d9955-wjvpp Startup probe error: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-api-b-3d38f6-openshift-default-79b69d9955-wjvpp Startup probe failed: Get "http://10.132.0.143:15021/healthz/ready": dial tcp 10.132.0.143:15021: connect: connection refused 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-3d38f6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-3d38f6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-d9f1c2-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-d9f1c2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Normal Created pod/e2e-api-b-8b3e43-openshift-default-78bccfc7fc-n448p Container created 51m Normal Pulled pod/e2e-api-b-8b3e43-openshift-default-78bccfc7fc-n448p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-api-b-8b3e43-openshift-default-78bccfc7fc-n448p Container started 51m Warning ProbeError pod/e2e-api-b-8b3e43-openshift-default-78bccfc7fc-n448p Startup probe error: Get "http://10.132.0.147:15021/healthz/ready": dial tcp 10.132.0.147:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-api-b-8b3e43-openshift-default-78bccfc7fc-n448p Startup probe failed: Get "http://10.132.0.147:15021/healthz/ready": dial tcp 10.132.0.147:15021: connect: connection refused 50m Normal Killing pod/payload-processing-e2e-api-a-de31dd-7d9cf5d4f-9lr2z Stopping container payload-processing 50m Normal Killing pod/payload-pre-processing-e2e-api-a-de31dd-7579cc4f79-rq4dm Stopping container payload-pre-processing 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-3d38f6-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-3d38f6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Normal Created pod/e2e-api-b-4f7439-openshift-default-84f666696f-7wlcj Container created 50m Normal Pulled pod/e2e-api-b-4f7439-openshift-default-84f666696f-7wlcj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-api-b-4f7439-openshift-default-84f666696f-7wlcj Container started 50m Normal AddedLabel gateway/e2e-inf-a-bdd94b Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-bdd94b 50m Normal ScalingReplicaSet deployment/e2e-inf-a-bdd94b-openshift-default Scaled up replica set e2e-inf-a-bdd94b-openshift-default-7bbf9bcd4f from 0 to 1 50m Normal SuccessfulCreate replicaset/e2e-inf-a-bdd94b-openshift-default-7bbf9bcd4f Created pod: e2e-inf-a-bdd94b-openshift-default-7bbf9bcd4f-q846p 50m Normal AddedInterface pod/e2e-inf-a-bdd94b-openshift-default-7bbf9bcd4f-q846p Add eth0 [10.132.0.157/23] from ovn-kubernetes 50m Normal Killing pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-2mvj6 Stopping container istio-proxy 50m Normal Killing pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-4cm9t Stopping container istio-proxy 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-4f7439-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-4f7439-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Normal Killing pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-w7p95 Stopping container istio-proxy 50m Normal Killing pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-j564x Stopping container istio-proxy 50m Normal Killing pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-9nsmf Stopping container istio-proxy 50m Normal Killing pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-7v8wh Stopping container istio-proxy 50m Normal Killing pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-mhwhp Stopping container istio-proxy 50m Normal Killing pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-2bzhm Stopping container istio-proxy 50m Normal Killing pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-78w6t Stopping container istio-proxy 50m Warning Unhealthy pod/e2e-api-b-4f7439-openshift-default-84f666696f-7wlcj Startup probe failed: Get "http://10.132.0.152:15021/healthz/ready": dial tcp 10.132.0.152:15021: connect: connection refused 50m Normal Killing pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-xwkss Stopping container istio-proxy 50m Warning ProbeError pod/e2e-api-b-4f7439-openshift-default-84f666696f-7wlcj Startup probe error: Get "http://10.132.0.152:15021/healthz/ready": dial tcp 10.132.0.152:15021: connect: connection refused... 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-bdd94b Scaled up replica set payload-processing-e2e-inf-a-bdd94b-5f4f4579c9 from 0 to 1 50m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-bdd94b-5f4f4579c9 Error creating: pods "payload-processing-e2e-inf-a-bdd94b-5f4f4579c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-bdd94b: serviceaccount "payload-processing-e2e-inf-a-bdd94b" not found 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-bdd94b-74d9b684bc Error creating: pods "payload-pre-processing-e2e-inf-a-bdd94b-74d9b684bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-bdd94b: serviceaccount "payload-processing-e2e-inf-a-bdd94b" not found 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-bdd94b Scaled up replica set payload-pre-processing-e2e-inf-a-bdd94b-74d9b684bc from 0 to 1 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-bdd94b-74d9b684bc Created pod: payload-pre-processing-e2e-inf-a-bdd94b-74d9b684bc-m4kfl 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-bdd94b-5f4f4579c9 Created pod: payload-processing-e2e-inf-a-bdd94b-5f4f4579c9-4mq7q 50m Normal Started pod/payload-pre-processing-e2e-inf-a-bdd94b-74d9b684bc-m4kfl Container started 50m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-bdd94b-74d9b684bc-m4kfl Add eth0 [10.133.0.204/23] from ovn-kubernetes 50m Normal Pulled pod/payload-processing-e2e-inf-a-bdd94b-5f4f4579c9-4mq7q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/payload-processing-e2e-inf-a-bdd94b-5f4f4579c9-4mq7q Add eth0 [10.133.0.205/23] from ovn-kubernetes 50m Normal Pulled pod/payload-pre-processing-e2e-inf-a-bdd94b-74d9b684bc-m4kfl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Created pod/payload-pre-processing-e2e-inf-a-bdd94b-74d9b684bc-m4kfl Container created 50m Normal Started pod/payload-processing-e2e-inf-a-bdd94b-5f4f4579c9-4mq7q Container started 50m Normal Created pod/payload-processing-e2e-inf-a-bdd94b-5f4f4579c9-4mq7q Container created 50m Warning Unhealthy pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-9nsmf Readiness probe failed: HTTP probe failed with statuscode: 503 50m Warning ProbeError pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-9nsmf Readiness probe error: HTTP probe failed with statuscode: 503... 50m Warning ProbeError pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-4cm9t Readiness probe error: Get "http://10.134.0.102:15021/healthz/ready": read tcp 10.134.0.2:34298->10.134.0.102:15021: read: connection reset by peer... 50m Warning Unhealthy pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-2bzhm Readiness probe failed: Get "http://10.134.0.104:15021/healthz/ready": dial tcp 10.134.0.104:15021: connect: connection refused 50m Warning ProbeError pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-2bzhm Readiness probe error: Get "http://10.134.0.104:15021/healthz/ready": dial tcp 10.134.0.104:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-api-a-370ec2-openshift-default-6fccc4f697-4cm9t Readiness probe failed: Get "http://10.134.0.102:15021/healthz/ready": read tcp 10.134.0.2:34298->10.134.0.102:15021: read: connection reset by peer 50m Normal Killing pod/payload-pre-processing-e2e-api-a-30edd7-5699985987-rczc8 Stopping container payload-pre-processing 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-8b3e43-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-8b3e43-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Normal Killing pod/payload-processing-e2e-api-a-30edd7-57c7586d97-zmlbs Stopping container payload-processing 50m Normal Killing pod/payload-pre-processing-e2e-api-b-bf9c2f-69f6d78d99-b2xh6 Stopping container payload-pre-processing 50m Normal Killing pod/payload-processing-e2e-api-b-bf9c2f-59899c7df-58lpq Stopping container payload-processing 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-4f7439-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-4f7439-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Normal Killing pod/e2e-api-a-f443d6-openshift-default-6dc8b74cd7-nkzfv Stopping container istio-proxy 50m Warning Unhealthy pod/e2e-api-a-f443d6-openshift-default-6dc8b74cd7-nkzfv Readiness probe failed: HTTP probe failed with statuscode: 503 50m Warning ProbeError pod/e2e-api-a-f443d6-openshift-default-6dc8b74cd7-nkzfv Readiness probe error: HTTP probe failed with statuscode: 503... 50m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-lnh72 Readiness probe failed: Get "http://10.133.0.24:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-lnh72 Readiness probe error: Get "http://10.133.0.24:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Normal Killing pod/payload-processing-e2e-api-b-3d38f6-5874cbccb9-4r8w9 Stopping container payload-processing 50m Normal Killing pod/payload-pre-processing-e2e-api-b-3d38f6-8fdb5988c-rjl42 Stopping container payload-pre-processing 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-6922d3-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 49m Normal SuccessfulCreate replicaset/e2e-inf-a-6922d3-openshift-default-689b5f95bc Created pod: e2e-inf-a-6922d3-openshift-default-689b5f95bc-tjbw5 49m Normal ScalingReplicaSet deployment/e2e-inf-a-6922d3-openshift-default Scaled up replica set e2e-inf-a-6922d3-openshift-default-689b5f95bc from 1 to 4 49m Normal SuccessfulCreate replicaset/e2e-inf-a-6922d3-openshift-default-689b5f95bc Created pod: e2e-inf-a-6922d3-openshift-default-689b5f95bc-dbxb6 49m Normal SuccessfulCreate replicaset/e2e-inf-a-6922d3-openshift-default-689b5f95bc Created pod: e2e-inf-a-6922d3-openshift-default-689b5f95bc-7bhqj 49m Normal Created pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-dbxb6 Container created 49m Normal Started pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-dbxb6 Container started 49m Normal Started pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-7bhqj Container started 49m Normal Pulled pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-tjbw5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-7bhqj Container created 49m Normal Created pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-tjbw5 Container created 49m Normal Pulled pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-7bhqj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-dbxb6 Add eth0 [10.132.0.158/23] from ovn-kubernetes 49m Normal AddedInterface pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-tjbw5 Add eth0 [10.133.0.208/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-dbxb6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-7bhqj Add eth0 [10.134.0.111/23] from ovn-kubernetes 49m Normal Started pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-tjbw5 Container started 49m Normal ScalingReplicaSet deployment/e2e-inf-b-f9d87b-openshift-default Scaled up replica set e2e-inf-b-f9d87b-openshift-default-6fb7566b69 from 0 to 1 49m Normal SuccessfulCreate replicaset/e2e-inf-b-f9d87b-openshift-default-6fb7566b69 Created pod: e2e-inf-b-f9d87b-openshift-default-6fb7566b69-2c4wk 49m Normal AddedLabel gateway/e2e-inf-b-f9d87b Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-f9d87b 49m Normal AddedInterface pod/e2e-inf-b-f9d87b-openshift-default-6fb7566b69-2c4wk Add eth0 [10.132.0.159/23] from ovn-kubernetes 49m Normal Killing pod/payload-pre-processing-e2e-api-a-7a1ee7-55c68b95cf-s428w Stopping container payload-pre-processing 49m Normal Killing pod/payload-processing-e2e-api-a-7a1ee7-7dbbd77679-kbzmg Stopping container payload-processing 49m Warning ProbeError pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-bf77z Readiness probe error: Get "http://10.132.0.153:15021/healthz/ready": read tcp 10.132.0.2:50754->10.132.0.153:15021: read: connection reset by peer... 49m Warning Unhealthy pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-bf77z Readiness probe failed: Get "http://10.132.0.153:15021/healthz/ready": read tcp 10.132.0.2:50754->10.132.0.153:15021: read: connection reset by peer 49m Normal Created pod/e2e-api-b-62236a-openshift-default-899986fcf-8kdtl Container created 49m Normal Pulled pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-bf77z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-bf77z Container created 49m Normal Started pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-bf77z Container started 49m Normal Started pod/e2e-api-b-62236a-openshift-default-899986fcf-8kdtl Container started 49m Normal Pulled pod/e2e-api-b-62236a-openshift-default-899986fcf-8kdtl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Warning ProbeError pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-7bhqj Startup probe error: Get "http://10.134.0.111:15021/healthz/ready": dial tcp 10.134.0.111:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-7bhqj Startup probe failed: Get "http://10.134.0.111:15021/healthz/ready": dial tcp 10.134.0.111:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-dbxb6 Startup probe failed: Get "http://10.132.0.158:15021/healthz/ready": dial tcp 10.132.0.158:15021: connect: connection refused 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-f9d87b Scaled up replica set payload-processing-e2e-inf-b-f9d87b-66796c8fff from 0 to 1 49m Warning ProbeError pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-tjbw5 Startup probe error: Get "http://10.133.0.208:15021/healthz/ready": dial tcp 10.133.0.208:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-tjbw5 Startup probe failed: Get "http://10.133.0.208:15021/healthz/ready": dial tcp 10.133.0.208:15021: connect: connection refused 49m Warning ProbeError pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-dbxb6 Startup probe error: Get "http://10.132.0.158:15021/healthz/ready": dial tcp 10.132.0.158:15021: connect: connection refused... 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-f9d87b-d46894c99 Created pod: payload-pre-processing-e2e-inf-b-f9d87b-d46894c99-vcmjs 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-f9d87b Scaled up replica set payload-pre-processing-e2e-inf-b-f9d87b-d46894c99 from 0 to 1 49m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-f9d87b-66796c8fff Error creating: pods "payload-processing-e2e-inf-b-f9d87b-66796c8fff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-f9d87b: serviceaccount "payload-processing-e2e-inf-b-f9d87b" not found 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-f9d87b-d46894c99 Error creating: pods "payload-pre-processing-e2e-inf-b-f9d87b-d46894c99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-f9d87b: serviceaccount "payload-processing-e2e-inf-b-f9d87b" not found 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-f9d87b-66796c8fff Created pod: payload-processing-e2e-inf-b-f9d87b-66796c8fff-5mhk5 49m Normal AddedInterface pod/payload-processing-e2e-inf-b-f9d87b-66796c8fff-5mhk5 Add eth0 [10.132.0.160/23] from ovn-kubernetes 49m Normal Pulled pod/payload-processing-e2e-inf-b-f9d87b-66796c8fff-5mhk5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Created pod/payload-processing-e2e-inf-b-f9d87b-66796c8fff-5mhk5 Container created 49m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-f9d87b-d46894c99-vcmjs Add eth0 [10.133.0.209/23] from ovn-kubernetes 49m Normal Pulled pod/payload-pre-processing-e2e-inf-b-f9d87b-d46894c99-vcmjs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Started pod/payload-processing-e2e-inf-b-f9d87b-66796c8fff-5mhk5 Container started 49m Normal Started pod/payload-pre-processing-e2e-inf-b-f9d87b-d46894c99-vcmjs Container started 49m Normal Created pod/payload-pre-processing-e2e-inf-b-f9d87b-d46894c99-vcmjs Container created 49m Warning Unhealthy pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-bf77z Startup probe failed: Get "http://10.132.0.153:15021/healthz/ready": dial tcp 10.132.0.153:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-api-b-62236a-openshift-default-899986fcf-8kdtl Startup probe failed: Get "http://10.133.0.197:15021/healthz/ready": dial tcp 10.133.0.197:15021: connect: connection refused 49m Warning ProbeError pod/e2e-api-b-62236a-openshift-default-899986fcf-8kdtl Startup probe error: Get "http://10.133.0.197:15021/healthz/ready": dial tcp 10.133.0.197:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-bf77z Startup probe error: Get "http://10.132.0.153:15021/healthz/ready": dial tcp 10.132.0.153:15021: connect: connection refused... 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-6922d3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-6922d3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning Unhealthy pod/e2e-api-a-7a1ee7-openshift-default-68c5c46ccc-5gx2f Readiness probe failed: Get "http://10.133.0.187:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-api-a-7a1ee7-openshift-default-68c5c46ccc-5gx2f Readiness probe error: Get "http://10.133.0.187:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Normal Created pod/e2e-api-a-7a1ee7-openshift-default-68c5c46ccc-5gx2f Container created 49m Normal Pulled pod/e2e-api-a-7a1ee7-openshift-default-68c5c46ccc-5gx2f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-api-a-7a1ee7-openshift-default-68c5c46ccc-5gx2f Container started 49m Normal Killing pod/e2e-api-a-de31dd-openshift-default-56f5bd455-d8vcq Stopping container istio-proxy 49m Warning ProbeError pod/e2e-api-a-7a1ee7-openshift-default-68c5c46ccc-5gx2f Startup probe error: Get "http://10.133.0.187:15021/healthz/ready": dial tcp 10.133.0.187:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-api-a-7a1ee7-openshift-default-68c5c46ccc-5gx2f Startup probe failed: Get "http://10.133.0.187:15021/healthz/ready": dial tcp 10.133.0.187:15021: connect: connection refused 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-62236a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-62236a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-7a1ee7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-7a1ee7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-6922d3-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Normal Killing pod/payload-pre-processing-e2e-api-b-8b3e43-57f97b8bcf-x88zv Stopping container payload-pre-processing 49m Normal Killing pod/payload-processing-e2e-api-b-8b3e43-685659576c-mxc2x Stopping container payload-processing 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-6922d3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-62236a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-62236a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-7a1ee7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-7a1ee7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-6922d3-openshift-default New size: 6; reason: All metrics below target 49m Normal SuccessfulCreate replicaset/e2e-inf-a-6922d3-openshift-default-689b5f95bc Created pod: e2e-inf-a-6922d3-openshift-default-689b5f95bc-txq8f 49m Normal ScalingReplicaSet deployment/e2e-inf-a-6922d3-openshift-default Scaled up replica set e2e-inf-a-6922d3-openshift-default-689b5f95bc from 4 to 6 49m Normal SuccessfulCreate replicaset/e2e-inf-a-6922d3-openshift-default-689b5f95bc Created pod: e2e-inf-a-6922d3-openshift-default-689b5f95bc-6t544 49m Normal AddedInterface pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-6t544 Add eth0 [10.133.0.211/23] from ovn-kubernetes 49m Normal Started pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-6t544 Container started 49m Normal Created pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-6t544 Container created 49m Normal Started pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-txq8f Container started 49m Normal Pulled pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-6t544 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-txq8f Container created 49m Normal AddedInterface pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-txq8f Add eth0 [10.134.0.112/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-txq8f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Warning ProbeError pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-txq8f Startup probe error: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-txq8f Startup probe failed: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused 49m Normal Pulled pod/e2e-inf-a-bdd94b-openshift-default-7bbf9bcd4f-q846p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-inf-a-bdd94b-openshift-default-7bbf9bcd4f-q846p Container created 49m Normal Started pod/e2e-inf-a-bdd94b-openshift-default-7bbf9bcd4f-q846p Container started 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-bdd94b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning ProbeError pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-6t544 Startup probe error: Get "http://10.133.0.211:15021/healthz/ready": dial tcp 10.133.0.211:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-6t544 Startup probe failed: Get "http://10.133.0.211:15021/healthz/ready": dial tcp 10.133.0.211:15021: connect: connection refused 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-bdd94b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Normal Killing pod/e2e-api-a-30edd7-openshift-default-69b49fb475-6hzw2 Stopping container istio-proxy 48m Warning ProbeError pod/e2e-inf-a-bdd94b-openshift-default-7bbf9bcd4f-q846p Startup probe error: Get "http://10.132.0.157:15021/healthz/ready": dial tcp 10.132.0.157:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-inf-a-bdd94b-openshift-default-7bbf9bcd4f-q846p Startup probe failed: Get "http://10.132.0.157:15021/healthz/ready": dial tcp 10.132.0.157:15021: connect: connection refused 48m Normal Killing pod/e2e-api-b-bf9c2f-openshift-default-7c859cb8dc-d962b Stopping container istio-proxy 48m Normal AddedLabel gateway/e2e-inf-a-999470 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-999470 48m Normal ScalingReplicaSet deployment/e2e-inf-a-999470-openshift-default Scaled up replica set e2e-inf-a-999470-openshift-default-6887745cf9 from 0 to 1 48m Normal SuccessfulCreate replicaset/e2e-inf-a-999470-openshift-default-6887745cf9 Created pod: e2e-inf-a-999470-openshift-default-6887745cf9-xt2fd 48m Normal AddedInterface pod/e2e-inf-a-999470-openshift-default-6887745cf9-xt2fd Add eth0 [10.132.0.163/23] from ovn-kubernetes 48m Normal ScalingReplicaSet deployment/e2e-inf-b-6013c8-openshift-default Scaled up replica set e2e-inf-b-6013c8-openshift-default-96f465855 from 0 to 1 48m Normal SuccessfulCreate replicaset/e2e-inf-b-6013c8-openshift-default-96f465855 Created pod: e2e-inf-b-6013c8-openshift-default-96f465855-llgxc 48m Normal AddedLabel gateway/e2e-inf-b-6013c8 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-6013c8 48m Normal Killing pod/payload-pre-processing-e2e-api-b-4f7439-55564f8b9f-xttgk Stopping container payload-pre-processing 48m Normal Killing pod/payload-processing-e2e-api-b-4f7439-db569fdbc-lx8l8 Stopping container payload-processing 48m Normal AddedInterface pod/e2e-inf-b-6013c8-openshift-default-96f465855-llgxc Add eth0 [10.133.0.213/23] from ovn-kubernetes 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-999470 Scaled up replica set payload-processing-e2e-inf-a-999470-99979b95c from 0 to 1 48m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-999470-99979b95c Error creating: pods "payload-processing-e2e-inf-a-999470-99979b95c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-999470: serviceaccount "payload-processing-e2e-inf-a-999470" not found 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-999470-99979b95c Created pod: payload-processing-e2e-inf-a-999470-99979b95c-2x956 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-999470-7757847d95 Error creating: pods "payload-pre-processing-e2e-inf-a-999470-7757847d95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-999470: serviceaccount "payload-processing-e2e-inf-a-999470" not found 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-999470-7757847d95 Created pod: payload-pre-processing-e2e-inf-a-999470-7757847d95-dfvnm 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-999470 Scaled up replica set payload-pre-processing-e2e-inf-a-999470-7757847d95 from 0 to 1 48m Normal Started pod/payload-pre-processing-e2e-inf-a-999470-7757847d95-dfvnm Container started 48m Normal Pulled pod/payload-processing-e2e-inf-a-999470-99979b95c-2x956 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-999470-7757847d95-dfvnm Add eth0 [10.133.0.214/23] from ovn-kubernetes 48m Normal Created pod/payload-pre-processing-e2e-inf-a-999470-7757847d95-dfvnm Container created 48m Normal Started pod/payload-processing-e2e-inf-a-999470-99979b95c-2x956 Container started 48m Normal Created pod/payload-processing-e2e-inf-a-999470-99979b95c-2x956 Container created 48m Normal Pulled pod/payload-pre-processing-e2e-inf-a-999470-7757847d95-dfvnm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-processing-e2e-inf-a-999470-99979b95c-2x956 Add eth0 [10.133.0.215/23] from ovn-kubernetes 48m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-6013c8-7678cc4db9 Error creating: pods "payload-processing-e2e-inf-b-6013c8-7678cc4db9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-6013c8: serviceaccount "payload-processing-e2e-inf-b-6013c8" not found 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-6013c8-94b84fcd7 Error creating: pods "payload-pre-processing-e2e-inf-b-6013c8-94b84fcd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-6013c8: serviceaccount "payload-processing-e2e-inf-b-6013c8" not found 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-6013c8 Scaled up replica set payload-pre-processing-e2e-inf-b-6013c8-94b84fcd7 from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-6013c8-94b84fcd7 Created pod: payload-pre-processing-e2e-inf-b-6013c8-94b84fcd7-b54fw 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-6013c8 Scaled up replica set payload-processing-e2e-inf-b-6013c8-7678cc4db9 from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-6013c8-7678cc4db9 Created pod: payload-processing-e2e-inf-b-6013c8-7678cc4db9-mppqg 48m Normal Pulled pod/payload-pre-processing-e2e-inf-b-6013c8-94b84fcd7-b54fw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Pulled pod/payload-processing-e2e-inf-b-6013c8-7678cc4db9-mppqg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/payload-pre-processing-e2e-inf-b-6013c8-94b84fcd7-b54fw Container created 48m Normal AddedInterface pod/payload-processing-e2e-inf-b-6013c8-7678cc4db9-mppqg Add eth0 [10.133.0.217/23] from ovn-kubernetes 48m Normal Created pod/payload-processing-e2e-inf-b-6013c8-7678cc4db9-mppqg Container created 48m Normal Started pod/payload-processing-e2e-inf-b-6013c8-7678cc4db9-mppqg Container started 48m Normal Started pod/payload-pre-processing-e2e-inf-b-6013c8-94b84fcd7-b54fw Container started 48m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-6013c8-94b84fcd7-b54fw Add eth0 [10.133.0.216/23] from ovn-kubernetes 48m Normal Killing pod/e2e-api-a-7a1ee7-openshift-default-68c5c46ccc-5gx2f Stopping container istio-proxy 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-bdd94b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-bdd94b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Normal Killing pod/e2e-api-b-3d38f6-openshift-default-79b69d9955-wjvpp Stopping container istio-proxy 48m Normal ScalingReplicaSet deployment/e2e-inf-a-d85da4-openshift-default Scaled up replica set e2e-inf-a-d85da4-openshift-default-6b5f89667 from 0 to 1 48m Normal SuccessfulCreate replicaset/e2e-inf-a-d85da4-openshift-default-6b5f89667 Created pod: e2e-inf-a-d85da4-openshift-default-6b5f89667-d8fjz 48m Normal AddedLabel gateway/e2e-inf-a-d85da4 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-d85da4 48m Normal AddedInterface pod/e2e-inf-a-d85da4-openshift-default-6b5f89667-d8fjz Add eth0 [10.133.0.219/23] from ovn-kubernetes 48m Warning ProbeError pod/e2e-api-b-3d38f6-openshift-default-79b69d9955-wjvpp Readiness probe error: HTTP probe failed with statuscode: 503... 48m Normal Killing pod/payload-processing-e2e-api-b-62236a-754c5cbbb9-sxrtf Stopping container payload-processing 48m Warning Unhealthy pod/e2e-api-b-3d38f6-openshift-default-79b69d9955-wjvpp Readiness probe failed: HTTP probe failed with statuscode: 503 48m Normal Killing pod/payload-pre-processing-e2e-api-b-62236a-56cfbd57f9-ldg6l Stopping container payload-pre-processing 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-d85da4-7bdd674455 Created pod: payload-pre-processing-e2e-inf-a-d85da4-7bdd674455-fjhgp 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-d85da4-7bdd674455 Error creating: pods "payload-pre-processing-e2e-inf-a-d85da4-7bdd674455-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-d85da4: serviceaccount "payload-processing-e2e-inf-a-d85da4" not found 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-d85da4 Scaled up replica set payload-pre-processing-e2e-inf-a-d85da4-7bdd674455 from 0 to 1 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-d85da4 Scaled up replica set payload-processing-e2e-inf-a-d85da4-7d5f448d7c from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-d85da4-7d5f448d7c Created pod: payload-processing-e2e-inf-a-d85da4-7d5f448d7c-cwh99 48m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-d85da4-7d5f448d7c Error creating: pods "payload-processing-e2e-inf-a-d85da4-7d5f448d7c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-d85da4: serviceaccount "payload-processing-e2e-inf-a-d85da4" not found 48m Normal Started pod/payload-processing-e2e-inf-a-d85da4-7d5f448d7c-cwh99 Container started 48m Normal Pulled pod/payload-processing-e2e-inf-a-d85da4-7d5f448d7c-cwh99 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-processing-e2e-inf-a-d85da4-7d5f448d7c-cwh99 Add eth0 [10.132.0.164/23] from ovn-kubernetes 48m Normal Created pod/payload-pre-processing-e2e-inf-a-d85da4-7bdd674455-fjhgp Container created 48m Normal Started pod/payload-pre-processing-e2e-inf-a-d85da4-7bdd674455-fjhgp Container started 48m Normal Pulled pod/payload-pre-processing-e2e-inf-a-d85da4-7bdd674455-fjhgp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-d85da4-7bdd674455-fjhgp Add eth0 [10.132.0.165/23] from ovn-kubernetes 48m Normal Created pod/payload-processing-e2e-inf-a-d85da4-7d5f448d7c-cwh99 Container created 48m Normal Killing pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-d5lpp Stopping container istio-proxy 48m Normal ScalingReplicaSet deployment/e2e-worker-w0-09d0d5-openshift-default Scaled down replica set e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 from 5 to 4 48m Normal SuccessfulDelete replicaset/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 Deleted pod: e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-d5lpp 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-09d0d5-openshift-default New size: 4; reason: All metrics below target 48m Normal Killing pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-67nxn Stopping container istio-proxy 48m Normal SuccessfulDelete replicaset/e2e-worker-w3-d52c59-openshift-default-5466968fb7 Deleted pod: e2e-worker-w3-d52c59-openshift-default-5466968fb7-67nxn 48m Normal Killing pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hnqx6 Stopping container istio-proxy 48m Normal SuccessfulDelete replicaset/e2e-worker-w5-54114e-openshift-default-7fc467ddd7 Deleted pod: e2e-worker-w5-54114e-openshift-default-7fc467ddd7-25w9c 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-d52c59-openshift-default New size: 3; reason: All metrics below target 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-2b1e88-openshift-default New size: 4; reason: All metrics below target 48m Normal Killing pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-25w9c Stopping container istio-proxy 48m Normal SuccessfulDelete replicaset/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c Deleted pod: e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hnqx6 48m Normal SuccessfulDelete replicaset/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 Deleted pod: e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-svsgv 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-54114e-openshift-default New size: 4; reason: All metrics below target 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0ad7c5-openshift-default New size: 3; reason: All metrics below target 48m Normal Killing pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-svsgv Stopping container istio-proxy 48m Warning Unhealthy pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-25w9c Readiness probe failed: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused 48m Normal Pulled pod/e2e-inf-b-f9d87b-openshift-default-6fb7566b69-2c4wk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-inf-b-f9d87b-openshift-default-6fb7566b69-2c4wk Container started 48m Warning ProbeError pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-25w9c Readiness probe error: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused... 48m Normal Created pod/e2e-inf-b-f9d87b-openshift-default-6fb7566b69-2c4wk Container created 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-f9d87b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-f9d87b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning Unhealthy pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-67nxn Readiness probe failed: Get "http://10.133.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-67nxn Readiness probe error: Get "http://10.133.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-inf-b-f9d87b-openshift-default-6fb7566b69-2c4wk Startup probe failed: Get "http://10.132.0.159:15021/healthz/ready": dial tcp 10.132.0.159:15021: connect: connection refused 48m Warning ProbeError pod/e2e-inf-b-f9d87b-openshift-default-6fb7566b69-2c4wk Startup probe error: Get "http://10.132.0.159:15021/healthz/ready": dial tcp 10.132.0.159:15021: connect: connection refused... 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-09d0d5-openshift-default New size: 3; reason: All metrics below target 48m Normal SuccessfulDelete replicaset/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 Deleted pod: e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-cnmb7 48m Normal Killing pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-cnmb7 Stopping container istio-proxy 48m Normal ScalingReplicaSet deployment/e2e-worker-w0-09d0d5-openshift-default Scaled down replica set e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 from 4 to 3 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-012ba0-openshift-default New size: 3; reason: All metrics below target 48m Normal Killing pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-m74tl Stopping container istio-proxy 48m Normal SuccessfulDelete replicaset/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc Deleted pod: e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-m74tl 48m Normal Killing pod/e2e-api-b-8b3e43-openshift-default-78bccfc7fc-n448p Stopping container istio-proxy 48m Normal ScalingReplicaSet deployment/e2e-inf-a-40229e-openshift-default Scaled up replica set e2e-inf-a-40229e-openshift-default-5f54d778f5 from 0 to 1 48m Normal SuccessfulCreate replicaset/e2e-inf-a-40229e-openshift-default-5f54d778f5 Created pod: e2e-inf-a-40229e-openshift-default-5f54d778f5-zhln9 48m Normal AddedLabel gateway/e2e-inf-a-40229e Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-40229e 48m Warning Unhealthy pod/e2e-api-b-8b3e43-openshift-default-78bccfc7fc-n448p Readiness probe failed: HTTP probe failed with statuscode: 503 48m Warning ProbeError pod/e2e-api-b-8b3e43-openshift-default-78bccfc7fc-n448p Readiness probe error: HTTP probe failed with statuscode: 503... 48m Normal AddedInterface pod/e2e-inf-a-40229e-openshift-default-5f54d778f5-zhln9 Add eth0 [10.133.0.220/23] from ovn-kubernetes 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-d85da4-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-d85da4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-40229e Scaled up replica set payload-pre-processing-e2e-inf-a-40229e-5fb694f79 from 0 to 1 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-40229e-5fb694f79 Error creating: pods "payload-pre-processing-e2e-inf-a-40229e-5fb694f79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-40229e: serviceaccount "payload-processing-e2e-inf-a-40229e" not found 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-40229e Scaled up replica set payload-processing-e2e-inf-a-40229e-86586c78fc from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-40229e-5fb694f79 Created pod: payload-pre-processing-e2e-inf-a-40229e-5fb694f79-5glps 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-40229e-86586c78fc Created pod: payload-processing-e2e-inf-a-40229e-86586c78fc-288bj 47m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-40229e-86586c78fc Error creating: pods "payload-processing-e2e-inf-a-40229e-86586c78fc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-40229e: serviceaccount "payload-processing-e2e-inf-a-40229e" not found 47m Normal Started pod/payload-processing-e2e-inf-a-40229e-86586c78fc-288bj Container started 47m Normal Started pod/payload-pre-processing-e2e-inf-a-40229e-5fb694f79-5glps Container started 47m Normal Created pod/payload-processing-e2e-inf-a-40229e-86586c78fc-288bj Container created 47m Normal Pulled pod/payload-pre-processing-e2e-inf-a-40229e-5fb694f79-5glps Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-40229e-5fb694f79-5glps Add eth0 [10.133.0.221/23] from ovn-kubernetes 47m Normal Pulled pod/payload-processing-e2e-inf-a-40229e-86586c78fc-288bj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-pre-processing-e2e-inf-a-40229e-5fb694f79-5glps Container created 47m Normal AddedInterface pod/payload-processing-e2e-inf-a-40229e-86586c78fc-288bj Add eth0 [10.132.0.167/23] from ovn-kubernetes 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-f9d87b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-f9d87b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning ProbeError pod/e2e-inf-a-999470-openshift-default-6887745cf9-xt2fd Readiness probe error: Get "http://10.132.0.163:15021/healthz/ready": read tcp 10.132.0.2:36230->10.132.0.163:15021: read: connection reset by peer... 47m Warning Unhealthy pod/e2e-inf-a-999470-openshift-default-6887745cf9-xt2fd Readiness probe failed: Get "http://10.132.0.163:15021/healthz/ready": read tcp 10.132.0.2:36230->10.132.0.163:15021: read: connection reset by peer 47m Normal Started pod/e2e-inf-b-6013c8-openshift-default-96f465855-llgxc Container started 47m Normal Created pod/e2e-inf-b-6013c8-openshift-default-96f465855-llgxc Container created 47m Normal Pulled pod/e2e-inf-b-6013c8-openshift-default-96f465855-llgxc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Started pod/e2e-inf-a-999470-openshift-default-6887745cf9-xt2fd Container started 47m Normal Created pod/e2e-inf-a-999470-openshift-default-6887745cf9-xt2fd Container created 47m Normal Pulled pod/e2e-inf-a-999470-openshift-default-6887745cf9-xt2fd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Warning ProbeError pod/e2e-inf-b-6013c8-openshift-default-96f465855-llgxc Startup probe error: Get "http://10.133.0.213:15021/healthz/ready": dial tcp 10.133.0.213:15021: connect: connection refused... 47m Warning ProbeError pod/e2e-inf-a-999470-openshift-default-6887745cf9-xt2fd Startup probe error: Get "http://10.132.0.163:15021/healthz/ready": dial tcp 10.132.0.163:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-inf-b-6013c8-openshift-default-96f465855-llgxc Startup probe failed: Get "http://10.133.0.213:15021/healthz/ready": dial tcp 10.133.0.213:15021: connect: connection refused 47m Warning Unhealthy pod/e2e-inf-a-999470-openshift-default-6887745cf9-xt2fd Startup probe failed: Get "http://10.132.0.163:15021/healthz/ready": dial tcp 10.132.0.163:15021: connect: connection refused 47m Normal SuccessfulCreate replicaset/e2e-inf-a-6922d3-openshift-default-689b5f95bc Created pod: e2e-inf-a-6922d3-openshift-default-689b5f95bc-x68sw 47m Normal Started pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-x68sw Container started 47m Normal AddedInterface pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-x68sw Add eth0 [10.134.0.116/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-q5795 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-q5795 Container created 47m Normal SuccessfulCreate replicaset/e2e-inf-a-6922d3-openshift-default-689b5f95bc Created pod: e2e-inf-a-6922d3-openshift-default-689b5f95bc-df6cw 47m Normal AddedInterface pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-df6cw Add eth0 [10.133.0.222/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-df6cw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal SuccessfulCreate replicaset/e2e-inf-a-6922d3-openshift-default-689b5f95bc Created pod: e2e-inf-a-6922d3-openshift-default-689b5f95bc-q5795 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-6922d3-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 47m Normal Created pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-x68sw Container created 47m Normal AddedInterface pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-q5795 Add eth0 [10.132.0.169/23] from ovn-kubernetes 47m Normal Created pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-df6cw Container created 47m Normal Started pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-q5795 Container started 47m Normal Pulled pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-x68sw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Started pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-df6cw Container started 47m Normal ScalingReplicaSet deployment/e2e-inf-a-6922d3-openshift-default Scaled up replica set e2e-inf-a-6922d3-openshift-default-689b5f95bc from 6 to 9 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-999470-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-999470-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Normal Killing pod/e2e-api-b-4f7439-openshift-default-84f666696f-7wlcj Stopping container istio-proxy 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-6013c8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Normal AddedLabel gateway/e2e-inf-a-bc48cc Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-bc48cc 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-6013c8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning Unhealthy pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-x68sw Startup probe failed: Get "http://10.134.0.116:15021/healthz/ready": dial tcp 10.134.0.116:15021: connect: connection refused 47m Warning ProbeError pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-df6cw Startup probe error: Get "http://10.133.0.222:15021/healthz/ready": dial tcp 10.133.0.222:15021: connect: connection refused... 47m Normal SuccessfulCreate replicaset/e2e-inf-a-bc48cc-openshift-default-d9498d65 Created pod: e2e-inf-a-bc48cc-openshift-default-d9498d65-kbdtw 47m Warning ProbeError pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-x68sw Startup probe error: Get "http://10.134.0.116:15021/healthz/ready": dial tcp 10.134.0.116:15021: connect: connection refused... 47m Normal ScalingReplicaSet deployment/e2e-inf-a-bc48cc-openshift-default Scaled up replica set e2e-inf-a-bc48cc-openshift-default-d9498d65 from 0 to 1 47m Warning ProbeError pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-q5795 Startup probe error: Get "http://10.132.0.169:15021/healthz/ready": dial tcp 10.132.0.169:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-q5795 Startup probe failed: Get "http://10.132.0.169:15021/healthz/ready": dial tcp 10.132.0.169:15021: connect: connection refused 47m Warning Unhealthy pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-df6cw Startup probe failed: Get "http://10.133.0.222:15021/healthz/ready": dial tcp 10.133.0.222:15021: connect: connection refused 47m Normal AddedInterface pod/e2e-inf-a-bc48cc-openshift-default-d9498d65-kbdtw Add eth0 [10.133.0.223/23] from ovn-kubernetes 47m Normal AddedLabel gateway/e2e-inf-b-54c7f0 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-54c7f0 47m Normal ScalingReplicaSet deployment/e2e-inf-b-54c7f0-openshift-default Scaled up replica set e2e-inf-b-54c7f0-openshift-default-79cd69d455 from 0 to 1 47m Normal SuccessfulCreate replicaset/e2e-inf-b-54c7f0-openshift-default-79cd69d455 Created pod: e2e-inf-b-54c7f0-openshift-default-79cd69d455-tfzdz 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-bc48cc Scaled up replica set payload-processing-e2e-inf-a-bc48cc-7d467bc759 from 0 to 1 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-bc48cc Scaled up replica set payload-pre-processing-e2e-inf-a-bc48cc-5554659f79 from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-bc48cc-5554659f79 Created pod: payload-pre-processing-e2e-inf-a-bc48cc-5554659f79-79xg7 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-bc48cc-5554659f79 Error creating: pods "payload-pre-processing-e2e-inf-a-bc48cc-5554659f79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-bc48cc: serviceaccount "payload-processing-e2e-inf-a-bc48cc" not found 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-bc48cc-7d467bc759 Created pod: payload-processing-e2e-inf-a-bc48cc-7d467bc759-tqr4d 47m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-bc48cc-7d467bc759 Error creating: pods "payload-processing-e2e-inf-a-bc48cc-7d467bc759-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-bc48cc: serviceaccount "payload-processing-e2e-inf-a-bc48cc" not found 47m Normal AddedInterface pod/e2e-inf-b-54c7f0-openshift-default-79cd69d455-tfzdz Add eth0 [10.132.0.170/23] from ovn-kubernetes 47m Normal Pulled pod/payload-pre-processing-e2e-inf-a-bc48cc-5554659f79-79xg7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-bc48cc-5554659f79-79xg7 Add eth0 [10.133.0.224/23] from ovn-kubernetes 47m Normal Pulled pod/payload-processing-e2e-inf-a-bc48cc-7d467bc759-tqr4d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Started pod/payload-pre-processing-e2e-inf-a-bc48cc-5554659f79-79xg7 Container started 47m Normal Created pod/payload-processing-e2e-inf-a-bc48cc-7d467bc759-tqr4d Container created 47m Normal AddedInterface pod/payload-processing-e2e-inf-a-bc48cc-7d467bc759-tqr4d Add eth0 [10.132.0.171/23] from ovn-kubernetes 47m Normal Started pod/payload-processing-e2e-inf-a-bc48cc-7d467bc759-tqr4d Container started 47m Normal Created pod/payload-pre-processing-e2e-inf-a-bc48cc-5554659f79-79xg7 Container created 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-54c7f0-545cb4dc95 Error creating: pods "payload-pre-processing-e2e-inf-b-54c7f0-545cb4dc95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-54c7f0: serviceaccount "payload-processing-e2e-inf-b-54c7f0" not found 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-54c7f0-5448fb4d5c Created pod: payload-processing-e2e-inf-b-54c7f0-5448fb4d5c-nrrpn 47m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-54c7f0-5448fb4d5c Error creating: pods "payload-processing-e2e-inf-b-54c7f0-5448fb4d5c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-54c7f0: serviceaccount "payload-processing-e2e-inf-b-54c7f0" not found 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-54c7f0-545cb4dc95 Created pod: payload-pre-processing-e2e-inf-b-54c7f0-545cb4dc95-xpm4m 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-54c7f0 Scaled up replica set payload-pre-processing-e2e-inf-b-54c7f0-545cb4dc95 from 0 to 1 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-54c7f0 Scaled up replica set payload-processing-e2e-inf-b-54c7f0-5448fb4d5c from 0 to 1 47m Normal Created pod/payload-processing-e2e-inf-b-54c7f0-5448fb4d5c-nrrpn Container created 47m Normal Pulled pod/payload-processing-e2e-inf-b-54c7f0-5448fb4d5c-nrrpn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/payload-processing-e2e-inf-b-54c7f0-5448fb4d5c-nrrpn Add eth0 [10.133.0.226/23] from ovn-kubernetes 47m Normal Pulled pod/payload-pre-processing-e2e-inf-b-54c7f0-545cb4dc95-xpm4m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-999470-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Normal Created pod/payload-pre-processing-e2e-inf-b-54c7f0-545cb4dc95-xpm4m Container created 47m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-54c7f0-545cb4dc95-xpm4m Add eth0 [10.133.0.225/23] from ovn-kubernetes 47m Normal Started pod/payload-processing-e2e-inf-b-54c7f0-5448fb4d5c-nrrpn Container started 47m Normal Started pod/payload-pre-processing-e2e-inf-b-54c7f0-545cb4dc95-xpm4m Container started 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-999470-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Normal Killing pod/e2e-api-b-62236a-openshift-default-899986fcf-8kdtl Stopping container istio-proxy 47m Normal AddedLabel gateway/e2e-inf-a-6e0edc Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-6e0edc 47m Normal SuccessfulCreate replicaset/e2e-inf-a-6e0edc-openshift-default-7968b4dddf Created pod: e2e-inf-a-6e0edc-openshift-default-7968b4dddf-86ztv 47m Normal ScalingReplicaSet deployment/e2e-inf-a-6e0edc-openshift-default Scaled up replica set e2e-inf-a-6e0edc-openshift-default-7968b4dddf from 0 to 1 47m Normal AddedInterface pod/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-86ztv Add eth0 [10.133.0.227/23] from ovn-kubernetes 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-6e0edc-c7974cff5 Created pod: payload-processing-e2e-inf-a-6e0edc-c7974cff5-rr5v8 46m Warning ProbeError pod/e2e-api-b-62236a-openshift-default-899986fcf-8kdtl Readiness probe error: Get "http://10.133.0.197:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-6e0edc-6cb4f7d887 Error creating: pods "payload-pre-processing-e2e-inf-a-6e0edc-6cb4f7d887-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-6e0edc: serviceaccount "payload-processing-e2e-inf-a-6e0edc" not found 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-6e0edc Scaled up replica set payload-pre-processing-e2e-inf-a-6e0edc-6cb4f7d887 from 0 to 1 46m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-6e0edc-c7974cff5 Error creating: pods "payload-processing-e2e-inf-a-6e0edc-c7974cff5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-6e0edc: serviceaccount "payload-processing-e2e-inf-a-6e0edc" not found 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-6e0edc-6cb4f7d887 Created pod: payload-pre-processing-e2e-inf-a-6e0edc-6cb4f7d887-wn55s 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-6e0edc Scaled up replica set payload-processing-e2e-inf-a-6e0edc-c7974cff5 from 0 to 1 46m Warning Unhealthy pod/e2e-api-b-62236a-openshift-default-899986fcf-8kdtl Readiness probe failed: Get "http://10.133.0.197:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Normal Created pod/payload-pre-processing-e2e-inf-a-6e0edc-6cb4f7d887-wn55s Container created 46m Normal Started pod/payload-pre-processing-e2e-inf-a-6e0edc-6cb4f7d887-wn55s Container started 46m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-6e0edc-6cb4f7d887-wn55s Add eth0 [10.133.0.228/23] from ovn-kubernetes 46m Normal Pulled pod/payload-pre-processing-e2e-inf-a-6e0edc-6cb4f7d887-wn55s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/payload-processing-e2e-inf-a-6e0edc-c7974cff5-rr5v8 Add eth0 [10.133.0.229/23] from ovn-kubernetes 46m Normal Pulled pod/payload-processing-e2e-inf-a-6e0edc-c7974cff5-rr5v8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/payload-processing-e2e-inf-a-6e0edc-c7974cff5-rr5v8 Container created 46m Normal Started pod/payload-processing-e2e-inf-a-6e0edc-c7974cff5-rr5v8 Container started 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-6013c8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-6013c8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal AddedLabel gateway/e2e-inf-b-f0bb45 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-f0bb45 46m Normal SuccessfulCreate replicaset/e2e-inf-b-f0bb45-openshift-default-6b64cff887 Created pod: e2e-inf-b-f0bb45-openshift-default-6b64cff887-s9c4q 46m Normal ScalingReplicaSet deployment/e2e-inf-b-f0bb45-openshift-default Scaled up replica set e2e-inf-b-f0bb45-openshift-default-6b64cff887 from 0 to 1 46m Normal AddedInterface pod/e2e-inf-b-f0bb45-openshift-default-6b64cff887-s9c4q Add eth0 [10.133.0.230/23] from ovn-kubernetes 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-f0bb45-64bb8c9b67 Created pod: payload-pre-processing-e2e-inf-b-f0bb45-64bb8c9b67-2dxq4 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-f0bb45 Scaled up replica set payload-processing-e2e-inf-b-f0bb45-5f54c4488f from 0 to 1 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-f0bb45-64bb8c9b67 Error creating: pods "payload-pre-processing-e2e-inf-b-f0bb45-64bb8c9b67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-f0bb45: serviceaccount "payload-processing-e2e-inf-b-f0bb45" not found 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-f0bb45 Scaled up replica set payload-pre-processing-e2e-inf-b-f0bb45-64bb8c9b67 from 0 to 1 46m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-f0bb45-5f54c4488f Error creating: pods "payload-processing-e2e-inf-b-f0bb45-5f54c4488f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-f0bb45: serviceaccount "payload-processing-e2e-inf-b-f0bb45" not found 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-f0bb45-5f54c4488f Created pod: payload-processing-e2e-inf-b-f0bb45-5f54c4488f-hdmsx 46m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-f0bb45-64bb8c9b67-2dxq4 Add eth0 [10.132.0.172/23] from ovn-kubernetes 46m Normal Created pod/payload-pre-processing-e2e-inf-b-f0bb45-64bb8c9b67-2dxq4 Container created 46m Normal AddedInterface pod/payload-processing-e2e-inf-b-f0bb45-5f54c4488f-hdmsx Add eth0 [10.132.0.173/23] from ovn-kubernetes 46m Normal Created pod/payload-processing-e2e-inf-b-f0bb45-5f54c4488f-hdmsx Container created 46m Normal Pulled pod/payload-processing-e2e-inf-b-f0bb45-5f54c4488f-hdmsx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Started pod/payload-processing-e2e-inf-b-f0bb45-5f54c4488f-hdmsx Container started 46m Normal Pulled pod/payload-pre-processing-e2e-inf-b-f0bb45-64bb8c9b67-2dxq4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Started pod/payload-pre-processing-e2e-inf-b-f0bb45-64bb8c9b67-2dxq4 Container started 46m Normal Pulled pod/e2e-inf-a-d85da4-openshift-default-6b5f89667-d8fjz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-inf-a-d85da4-openshift-default-6b5f89667-d8fjz Container started 46m Normal Created pod/e2e-inf-a-d85da4-openshift-default-6b5f89667-d8fjz Container created 46m Warning ProbeError pod/e2e-inf-a-d85da4-openshift-default-6b5f89667-d8fjz Startup probe error: Get "http://10.133.0.219:15021/healthz/ready": dial tcp 10.133.0.219:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-inf-a-d85da4-openshift-default-6b5f89667-d8fjz Startup probe failed: Get "http://10.133.0.219:15021/healthz/ready": dial tcp 10.133.0.219:15021: connect: connection refused 46m Normal AddedLabel gateway/e2e-inf-b-c13db0 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-c13db0 46m Normal SuccessfulCreate replicaset/e2e-inf-b-c13db0-openshift-default-f894f747 Created pod: e2e-inf-b-c13db0-openshift-default-f894f747-cbtvq 46m Normal ScalingReplicaSet deployment/e2e-inf-b-c13db0-openshift-default Scaled up replica set e2e-inf-b-c13db0-openshift-default-f894f747 from 0 to 1 46m Normal Killing pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-582w4 Stopping container istio-proxy 46m Normal AddedInterface pod/e2e-inf-b-c13db0-openshift-default-f894f747-cbtvq Add eth0 [10.133.0.231/23] from ovn-kubernetes 46m Normal SuccessfulDelete replicaset/e2e-worker-w5-54114e-openshift-default-7fc467ddd7 Deleted pod: e2e-worker-w5-54114e-openshift-default-7fc467ddd7-582w4 46m Normal Killing pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-n6fb8 Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 Deleted pod: e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-n6fb8 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-c13db0-5dbfdd689 Created pod: payload-processing-e2e-inf-b-c13db0-5dbfdd689-q5zt5 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-c13db0 Scaled up replica set payload-pre-processing-e2e-inf-b-c13db0-bcdbd4c9f from 0 to 1 46m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-c13db0-5dbfdd689 Error creating: pods "payload-processing-e2e-inf-b-c13db0-5dbfdd689-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-c13db0: serviceaccount "payload-processing-e2e-inf-b-c13db0" not found 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-c13db0 Scaled up replica set payload-processing-e2e-inf-b-c13db0-5dbfdd689 from 0 to 1 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-c13db0-bcdbd4c9f Error creating: pods "payload-pre-processing-e2e-inf-b-c13db0-bcdbd4c9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-c13db0: serviceaccount "payload-processing-e2e-inf-b-c13db0" not found 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-c13db0-bcdbd4c9f Created pod: payload-pre-processing-e2e-inf-b-c13db0-bcdbd4c9f-tqwjn 46m Normal Created pod/payload-pre-processing-e2e-inf-b-c13db0-bcdbd4c9f-tqwjn Container created 46m Warning Unhealthy pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-582w4 Readiness probe failed: Get "http://10.133.0.94:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Normal Pulled pod/payload-processing-e2e-inf-b-c13db0-5dbfdd689-q5zt5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Started pod/payload-pre-processing-e2e-inf-b-c13db0-bcdbd4c9f-tqwjn Container started 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-d85da4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-d85da4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal Pulled pod/payload-pre-processing-e2e-inf-b-c13db0-bcdbd4c9f-tqwjn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-c13db0-bcdbd4c9f-tqwjn Add eth0 [10.133.0.232/23] from ovn-kubernetes 46m Normal AddedInterface pod/payload-processing-e2e-inf-b-c13db0-5dbfdd689-q5zt5 Add eth0 [10.132.0.175/23] from ovn-kubernetes 46m Normal Started pod/payload-processing-e2e-inf-b-c13db0-5dbfdd689-q5zt5 Container started 46m Warning ProbeError pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-582w4 Readiness probe error: Get "http://10.133.0.94:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Normal Created pod/payload-processing-e2e-inf-b-c13db0-5dbfdd689-q5zt5 Container created 45m Warning ProbeError pod/e2e-inf-a-40229e-openshift-default-5f54d778f5-zhln9 Readiness probe error: Get "http://10.133.0.220:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning Unhealthy pod/e2e-inf-a-40229e-openshift-default-5f54d778f5-zhln9 Readiness probe failed: Get "http://10.133.0.220:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Normal Created pod/e2e-inf-a-bc48cc-openshift-default-d9498d65-kbdtw Container created 45m Normal Pulled pod/e2e-inf-a-40229e-openshift-default-5f54d778f5-zhln9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/e2e-inf-a-bc48cc-openshift-default-d9498d65-kbdtw Container started 45m Normal Created pod/e2e-inf-a-40229e-openshift-default-5f54d778f5-zhln9 Container created 45m Normal Started pod/e2e-inf-a-40229e-openshift-default-5f54d778f5-zhln9 Container started 45m Normal Pulled pod/e2e-inf-a-bc48cc-openshift-default-d9498d65-kbdtw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-bc48cc-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-bc48cc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning ProbeError pod/e2e-inf-a-40229e-openshift-default-5f54d778f5-zhln9 Startup probe error: Get "http://10.133.0.220:15021/healthz/ready": dial tcp 10.133.0.220:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-inf-a-40229e-openshift-default-5f54d778f5-zhln9 Startup probe failed: Get "http://10.133.0.220:15021/healthz/ready": dial tcp 10.133.0.220:15021: connect: connection refused 45m Warning Unhealthy pod/e2e-inf-a-bc48cc-openshift-default-d9498d65-kbdtw Startup probe failed: Get "http://10.133.0.223:15021/healthz/ready": dial tcp 10.133.0.223:15021: connect: connection refused 45m Warning ProbeError pod/e2e-inf-a-bc48cc-openshift-default-d9498d65-kbdtw Startup probe error: Get "http://10.133.0.223:15021/healthz/ready": dial tcp 10.133.0.223:15021: connect: connection refused... 45m Normal AddedLabel gateway/e2e-inf-b-867f12 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-867f12 45m Normal ScalingReplicaSet deployment/e2e-inf-b-867f12-openshift-default Scaled up replica set e2e-inf-b-867f12-openshift-default-6584f84965 from 0 to 1 45m Normal AddedInterface pod/e2e-inf-b-867f12-openshift-default-6584f84965-jlqnx Add eth0 [10.133.0.233/23] from ovn-kubernetes 45m Normal SuccessfulCreate replicaset/e2e-inf-b-867f12-openshift-default-6584f84965 Created pod: e2e-inf-b-867f12-openshift-default-6584f84965-jlqnx 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-40229e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-40229e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-867f12-f59b9bd47 Created pod: payload-pre-processing-e2e-inf-b-867f12-f59b9bd47-xkz7q 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-867f12-66d87c857f Created pod: payload-processing-e2e-inf-b-867f12-66d87c857f-vsjkv 45m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-867f12-66d87c857f Error creating: pods "payload-processing-e2e-inf-b-867f12-66d87c857f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-867f12: serviceaccount "payload-processing-e2e-inf-b-867f12" not found 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-867f12 Scaled up replica set payload-pre-processing-e2e-inf-b-867f12-f59b9bd47 from 0 to 1 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-867f12-f59b9bd47 Error creating: pods "payload-pre-processing-e2e-inf-b-867f12-f59b9bd47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-867f12: serviceaccount "payload-processing-e2e-inf-b-867f12" not found 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-867f12 Scaled up replica set payload-processing-e2e-inf-b-867f12-66d87c857f from 0 to 1 45m Normal Pulled pod/payload-processing-e2e-inf-b-867f12-66d87c857f-vsjkv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-pre-processing-e2e-inf-b-867f12-f59b9bd47-xkz7q Container created 45m Normal AddedInterface pod/payload-processing-e2e-inf-b-867f12-66d87c857f-vsjkv Add eth0 [10.132.0.176/23] from ovn-kubernetes 45m Normal Pulled pod/payload-pre-processing-e2e-inf-b-867f12-f59b9bd47-xkz7q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Started pod/payload-processing-e2e-inf-b-867f12-66d87c857f-vsjkv Container started 45m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-867f12-f59b9bd47-xkz7q Add eth0 [10.133.0.234/23] from ovn-kubernetes 45m Normal Started pod/payload-pre-processing-e2e-inf-b-867f12-f59b9bd47-xkz7q Container started 45m Normal Created pod/payload-processing-e2e-inf-b-867f12-66d87c857f-vsjkv Container created 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-bc48cc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-bc48cc-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Normal AddedLabel gateway/e2e-inf-b-3acfdb Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-3acfdb 45m Normal SuccessfulCreate replicaset/e2e-inf-b-3acfdb-openshift-default-5b477b885f Created pod: e2e-inf-b-3acfdb-openshift-default-5b477b885f-mrqbm 45m Normal ScalingReplicaSet deployment/e2e-inf-b-3acfdb-openshift-default Scaled up replica set e2e-inf-b-3acfdb-openshift-default-5b477b885f from 0 to 1 45m Normal AddedInterface pod/e2e-inf-b-3acfdb-openshift-default-5b477b885f-mrqbm Add eth0 [10.133.0.235/23] from ovn-kubernetes 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-3acfdb-7f8bb6c86f Created pod: payload-processing-e2e-inf-b-3acfdb-7f8bb6c86f-7bsb5 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-3acfdb Scaled up replica set payload-processing-e2e-inf-b-3acfdb-7f8bb6c86f from 0 to 1 45m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-3acfdb-7f8bb6c86f Error creating: pods "payload-processing-e2e-inf-b-3acfdb-7f8bb6c86f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-3acfdb: serviceaccount "payload-processing-e2e-inf-b-3acfdb" not found 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-3acfdb-66c888cfdc Error creating: pods "payload-pre-processing-e2e-inf-b-3acfdb-66c888cfdc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-3acfdb: serviceaccount "payload-processing-e2e-inf-b-3acfdb" not found 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-3acfdb-66c888cfdc Created pod: payload-pre-processing-e2e-inf-b-3acfdb-66c888cfdc-ktrkf 45m Normal Pulled pod/e2e-inf-b-54c7f0-openshift-default-79cd69d455-tfzdz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-inf-b-54c7f0-openshift-default-79cd69d455-tfzdz Container created 45m Normal Started pod/e2e-inf-b-54c7f0-openshift-default-79cd69d455-tfzdz Container started 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-3acfdb Scaled up replica set payload-pre-processing-e2e-inf-b-3acfdb-66c888cfdc from 0 to 1 45m Normal Started pod/payload-processing-e2e-inf-b-3acfdb-7f8bb6c86f-7bsb5 Container started 45m Normal Started pod/payload-pre-processing-e2e-inf-b-3acfdb-66c888cfdc-ktrkf Container started 45m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-3acfdb-66c888cfdc-ktrkf Add eth0 [10.132.0.177/23] from ovn-kubernetes 45m Normal AddedInterface pod/payload-processing-e2e-inf-b-3acfdb-7f8bb6c86f-7bsb5 Add eth0 [10.132.0.178/23] from ovn-kubernetes 45m Normal Pulled pod/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-86ztv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Pulled pod/payload-pre-processing-e2e-inf-b-3acfdb-66c888cfdc-ktrkf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-pre-processing-e2e-inf-b-3acfdb-66c888cfdc-ktrkf Container created 45m Normal Pulled pod/payload-processing-e2e-inf-b-3acfdb-7f8bb6c86f-7bsb5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-processing-e2e-inf-b-3acfdb-7f8bb6c86f-7bsb5 Container created 45m Normal Pulled pod/e2e-inf-b-f0bb45-openshift-default-6b64cff887-s9c4q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-86ztv Container started 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-6e0edc-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal Created pod/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-86ztv Container created 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-6e0edc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal Created pod/e2e-inf-b-f0bb45-openshift-default-6b64cff887-s9c4q Container created 45m Normal Started pod/e2e-inf-b-f0bb45-openshift-default-6b64cff887-s9c4q Container started 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-54c7f0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-54c7f0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning Unhealthy pod/e2e-inf-b-54c7f0-openshift-default-79cd69d455-tfzdz Startup probe failed: Get "http://10.132.0.170:15021/healthz/ready": dial tcp 10.132.0.170:15021: connect: connection refused 45m Warning ProbeError pod/e2e-inf-b-54c7f0-openshift-default-79cd69d455-tfzdz Startup probe error: Get "http://10.132.0.170:15021/healthz/ready": dial tcp 10.132.0.170:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-86ztv Startup probe failed: Get "http://10.133.0.227:15021/healthz/ready": dial tcp 10.133.0.227:15021: connect: connection refused 44m Warning ProbeError pod/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-86ztv Startup probe error: Get "http://10.133.0.227:15021/healthz/ready": dial tcp 10.133.0.227:15021: connect: connection refused... 44m Warning ProbeError pod/e2e-inf-b-f0bb45-openshift-default-6b64cff887-s9c4q Startup probe error: Get "http://10.133.0.230:15021/healthz/ready": dial tcp 10.133.0.230:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-inf-b-f0bb45-openshift-default-6b64cff887-s9c4q Startup probe failed: Get "http://10.133.0.230:15021/healthz/ready": dial tcp 10.133.0.230:15021: connect: connection refused 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-f0bb45-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-f0bb45-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-54c7f0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-54c7f0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-f0bb45-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-f0bb45-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6db586dd6c Deleted pod: maas-default-gateway-openshift-default-6db586dd6c-lnh72 44m Normal Killing pod/maas-default-gateway-openshift-default-6db586dd6c-lnh72 Stopping container istio-proxy 44m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 9; reason: All metrics below target 44m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-6db586dd6c from 10 to 9 44m Normal Killing pod/maas-default-gateway-openshift-default-6db586dd6c-vbv8b Stopping container istio-proxy 44m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6db586dd6c Deleted pod: maas-default-gateway-openshift-default-6db586dd6c-vbv8b 44m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-6db586dd6c from 9 to 8 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-6e0edc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-6e0edc-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: All metrics below target 44m Normal Started pod/e2e-inf-b-c13db0-openshift-default-f894f747-cbtvq Container started 44m Normal Created pod/e2e-inf-b-c13db0-openshift-default-f894f747-cbtvq Container created 44m Normal Pulled pod/e2e-inf-b-c13db0-openshift-default-f894f747-cbtvq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Warning ProbeError pod/e2e-inf-b-c13db0-openshift-default-f894f747-cbtvq Startup probe error: Get "http://10.133.0.231:15021/healthz/ready": dial tcp 10.133.0.231:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-inf-b-c13db0-openshift-default-f894f747-cbtvq Startup probe failed: Get "http://10.133.0.231:15021/healthz/ready": dial tcp 10.133.0.231:15021: connect: connection refused 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-c13db0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-c13db0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning Unhealthy pod/e2e-inf-b-867f12-openshift-default-6584f84965-jlqnx Readiness probe failed: Get "http://10.133.0.233:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Warning ProbeError pod/e2e-inf-b-867f12-openshift-default-6584f84965-jlqnx Readiness probe error: Get "http://10.133.0.233:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Normal Created pod/e2e-inf-b-867f12-openshift-default-6584f84965-jlqnx Container created 43m Normal Pulled pod/e2e-inf-b-867f12-openshift-default-6584f84965-jlqnx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-inf-b-867f12-openshift-default-6584f84965-jlqnx Container started 43m Warning ProbeError pod/e2e-inf-b-867f12-openshift-default-6584f84965-jlqnx Startup probe error: Get "http://10.133.0.233:15021/healthz/ready": dial tcp 10.133.0.233:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-inf-b-867f12-openshift-default-6584f84965-jlqnx Startup probe failed: Get "http://10.133.0.233:15021/healthz/ready": dial tcp 10.133.0.233:15021: connect: connection refused 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-867f12-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-867f12-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-54c7f0-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 43m Normal ScalingReplicaSet deployment/e2e-inf-b-54c7f0-openshift-default Scaled up replica set e2e-inf-b-54c7f0-openshift-default-79cd69d455 from 1 to 2 43m Normal SuccessfulCreate replicaset/e2e-inf-b-54c7f0-openshift-default-79cd69d455 Created pod: e2e-inf-b-54c7f0-openshift-default-79cd69d455-dznhn 43m Normal AddedInterface pod/e2e-inf-b-54c7f0-openshift-default-79cd69d455-dznhn Add eth0 [10.133.0.237/23] from ovn-kubernetes 43m Normal Created pod/e2e-inf-b-54c7f0-openshift-default-79cd69d455-dznhn Container created 43m Normal Started pod/e2e-inf-b-54c7f0-openshift-default-79cd69d455-dznhn Container started 43m Normal Pulled pod/e2e-inf-b-54c7f0-openshift-default-79cd69d455-dznhn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Warning Unhealthy pod/e2e-inf-b-54c7f0-openshift-default-79cd69d455-dznhn Startup probe failed: Get "http://10.133.0.237:15021/healthz/ready": dial tcp 10.133.0.237:15021: connect: connection refused 43m Warning ProbeError pod/e2e-inf-b-54c7f0-openshift-default-79cd69d455-dznhn Startup probe error: Get "http://10.133.0.237:15021/healthz/ready": dial tcp 10.133.0.237:15021: connect: connection refused... 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-c13db0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-867f12-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-867f12-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-c13db0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal SuccessfulDelete replicaset/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 Deleted pod: e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-vtwp6 43m Normal SuccessfulDelete replicaset/e2e-worker-w3-d52c59-openshift-default-5466968fb7 Deleted pod: e2e-worker-w3-d52c59-openshift-default-5466968fb7-qtv79 43m Normal Killing pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-vtwp6 Stopping container istio-proxy 43m Normal Killing pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-qtv79 Stopping container istio-proxy 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-09d0d5-openshift-default New size: 2; reason: All metrics below target 43m Normal Killing pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-v5lkq Stopping container istio-proxy 43m Normal SuccessfulDelete replicaset/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc Deleted pod: e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-v5lkq 43m Warning ProbeError pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-qtv79 Readiness probe error: HTTP probe failed with statuscode: 503... 43m Warning Unhealthy pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-qtv79 Readiness probe failed: HTTP probe failed with statuscode: 503 43m Warning ProbeError pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-v5lkq Readiness probe error: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-v5lkq Readiness probe failed: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused 42m Warning ProbeError pod/e2e-inf-b-3acfdb-openshift-default-5b477b885f-mrqbm Readiness probe error: Get "http://10.133.0.235:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Warning Unhealthy pod/e2e-inf-b-3acfdb-openshift-default-5b477b885f-mrqbm Readiness probe failed: Get "http://10.133.0.235:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Normal Started pod/e2e-inf-b-3acfdb-openshift-default-5b477b885f-mrqbm Container started 42m Normal Pulled pod/e2e-inf-b-3acfdb-openshift-default-5b477b885f-mrqbm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-inf-b-3acfdb-openshift-default-5b477b885f-mrqbm Container created 42m Normal ScalingReplicaSet deployment/e2e-inf-b-c13db0-openshift-default Scaled up replica set e2e-inf-b-c13db0-openshift-default-f894f747 from 1 to 2 42m Normal SuccessfulCreate replicaset/e2e-inf-b-c13db0-openshift-default-f894f747 Created pod: e2e-inf-b-c13db0-openshift-default-f894f747-t6h8c 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-c13db0-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 42m Normal AddedInterface pod/e2e-inf-b-c13db0-openshift-default-f894f747-t6h8c Add eth0 [10.132.0.183/23] from ovn-kubernetes 42m Normal Pulled pod/e2e-inf-b-c13db0-openshift-default-f894f747-t6h8c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-inf-b-c13db0-openshift-default-f894f747-t6h8c Container created 42m Normal Started pod/e2e-inf-b-c13db0-openshift-default-f894f747-t6h8c Container started 42m Warning ProbeError pod/e2e-inf-b-3acfdb-openshift-default-5b477b885f-mrqbm Startup probe error: Get "http://10.133.0.235:15021/healthz/ready": dial tcp 10.133.0.235:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-inf-b-3acfdb-openshift-default-5b477b885f-mrqbm Startup probe failed: Get "http://10.133.0.235:15021/healthz/ready": dial tcp 10.133.0.235:15021: connect: connection refused 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-3acfdb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-3acfdb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning ProbeError pod/e2e-inf-b-c13db0-openshift-default-f894f747-t6h8c Startup probe error: Get "http://10.132.0.183:15021/healthz/ready": dial tcp 10.132.0.183:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-inf-b-c13db0-openshift-default-f894f747-t6h8c Startup probe failed: Get "http://10.132.0.183:15021/healthz/ready": dial tcp 10.132.0.183:15021: connect: connection refused 42m Normal ScalingReplicaSet deployment/e2e-inf-b-f9d87b-openshift-default Scaled up replica set e2e-inf-b-f9d87b-openshift-default-6fb7566b69 from 1 to 2 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-f9d87b-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 42m Normal SuccessfulCreate replicaset/e2e-inf-b-f9d87b-openshift-default-6fb7566b69 Created pod: e2e-inf-b-f9d87b-openshift-default-6fb7566b69-5hv5d 42m Normal Created pod/e2e-inf-b-f9d87b-openshift-default-6fb7566b69-5hv5d Container created 42m Normal Started pod/e2e-inf-b-f9d87b-openshift-default-6fb7566b69-5hv5d Container started 42m Normal Pulled pod/e2e-inf-b-f9d87b-openshift-default-6fb7566b69-5hv5d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/e2e-inf-b-f9d87b-openshift-default-6fb7566b69-5hv5d Add eth0 [10.133.0.239/23] from ovn-kubernetes 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-f0bb45-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 42m Normal ScalingReplicaSet deployment/e2e-inf-b-f0bb45-openshift-default Scaled up replica set e2e-inf-b-f0bb45-openshift-default-6b64cff887 from 1 to 2 42m Normal SuccessfulCreate replicaset/e2e-inf-b-f0bb45-openshift-default-6b64cff887 Created pod: e2e-inf-b-f0bb45-openshift-default-6b64cff887-w4hj4 42m Normal Pulled pod/e2e-inf-b-f0bb45-openshift-default-6b64cff887-w4hj4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/e2e-inf-b-f0bb45-openshift-default-6b64cff887-w4hj4 Add eth0 [10.132.0.184/23] from ovn-kubernetes 42m Normal Created pod/e2e-inf-b-f0bb45-openshift-default-6b64cff887-w4hj4 Container created 42m Normal Started pod/e2e-inf-b-f0bb45-openshift-default-6b64cff887-w4hj4 Container started 42m Warning Unhealthy pod/e2e-inf-b-f9d87b-openshift-default-6fb7566b69-5hv5d Startup probe failed: Get "http://10.133.0.239:15021/healthz/ready": dial tcp 10.133.0.239:15021: connect: connection refused 42m Warning ProbeError pod/e2e-inf-b-f9d87b-openshift-default-6fb7566b69-5hv5d Startup probe error: Get "http://10.133.0.239:15021/healthz/ready": dial tcp 10.133.0.239:15021: connect: connection refused... 42m Warning ProbeError pod/e2e-inf-b-f0bb45-openshift-default-6b64cff887-w4hj4 Startup probe error: Get "http://10.132.0.184:15021/healthz/ready": dial tcp 10.132.0.184:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-inf-b-f0bb45-openshift-default-6b64cff887-w4hj4 Startup probe failed: Get "http://10.132.0.184:15021/healthz/ready": dial tcp 10.132.0.184:15021: connect: connection refused 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-3acfdb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-3acfdb-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: All metrics below target 41m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6db586dd6c Deleted pod: maas-default-gateway-openshift-default-6db586dd6c-p65ln 41m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6db586dd6c Deleted pod: maas-default-gateway-openshift-default-6db586dd6c-cmklv 41m Normal ScalingReplicaSet deployment/e2e-inf-b-867f12-openshift-default Scaled up replica set e2e-inf-b-867f12-openshift-default-6584f84965 from 1 to 2 41m Normal Killing pod/maas-default-gateway-openshift-default-6db586dd6c-cmklv Stopping container istio-proxy 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-867f12-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 41m Normal SuccessfulCreate replicaset/e2e-inf-b-867f12-openshift-default-6584f84965 Created pod: e2e-inf-b-867f12-openshift-default-6584f84965-9rbbv 41m Normal Killing pod/payload-processing-e2e-inf-a-6922d3-59cf97fb57-v5z2m Stopping container payload-processing 41m Normal Killing pod/maas-default-gateway-openshift-default-6db586dd6c-p65ln Stopping container istio-proxy 41m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-6db586dd6c from 8 to 6 41m Normal Killing pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-dxslt Stopping container istio-proxy 41m Normal AddedInterface pod/e2e-inf-b-867f12-openshift-default-6584f84965-9rbbv Add eth0 [10.132.0.185/23] from ovn-kubernetes 41m Normal Killing pod/payload-pre-processing-e2e-inf-a-6922d3-69c86dfc75-ktwbq Stopping container payload-pre-processing 41m Normal SuccessfulDelete replicaset/e2e-worker-w5-54114e-openshift-default-7fc467ddd7 Deleted pod: e2e-worker-w5-54114e-openshift-default-7fc467ddd7-kvtw4 41m Normal SuccessfulDelete replicaset/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 Deleted pod: e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-dxslt 41m Normal Pulled pod/e2e-inf-b-867f12-openshift-default-6584f84965-9rbbv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-inf-b-867f12-openshift-default-6584f84965-9rbbv Container created 41m Normal Killing pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8dgzg Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c Deleted pod: e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-8dgzg 41m Normal Started pod/e2e-inf-b-867f12-openshift-default-6584f84965-9rbbv Container started 41m Normal Killing pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-kvtw4 Stopping container istio-proxy 41m Warning ProbeError pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-dxslt Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-dxslt Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-cmklv Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-cmklv Readiness probe error: HTTP probe failed with statuscode: 503... 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-3acfdb-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 41m Warning ProbeError pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-kvtw4 Readiness probe error: HTTP probe failed with statuscode: 503... 41m Normal ScalingReplicaSet deployment/e2e-inf-b-3acfdb-openshift-default Scaled up replica set e2e-inf-b-3acfdb-openshift-default-5b477b885f from 1 to 2 41m Warning Unhealthy pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-kvtw4 Readiness probe failed: HTTP probe failed with statuscode: 503 41m Normal SuccessfulCreate replicaset/e2e-inf-b-3acfdb-openshift-default-5b477b885f Created pod: e2e-inf-b-3acfdb-openshift-default-5b477b885f-xtl6s 41m Normal AddedInterface pod/e2e-inf-b-3acfdb-openshift-default-5b477b885f-xtl6s Add eth0 [10.132.0.186/23] from ovn-kubernetes 41m Normal Created pod/e2e-inf-b-3acfdb-openshift-default-5b477b885f-xtl6s Container created 41m Normal Pulled pod/e2e-inf-b-3acfdb-openshift-default-5b477b885f-xtl6s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Started pod/e2e-inf-b-3acfdb-openshift-default-5b477b885f-xtl6s Container started 41m Warning ProbeError pod/e2e-inf-b-867f12-openshift-default-6584f84965-9rbbv Startup probe error: Get "http://10.132.0.185:15021/healthz/ready": dial tcp 10.132.0.185:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-inf-b-867f12-openshift-default-6584f84965-9rbbv Startup probe failed: Get "http://10.132.0.185:15021/healthz/ready": dial tcp 10.132.0.185:15021: connect: connection refused 41m Normal Killing pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-6t544 Stopping container istio-proxy 41m Normal Killing pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-dbxb6 Stopping container istio-proxy 41m Normal Killing pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-x68sw Stopping container istio-proxy 41m Normal Killing pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-df6cw Stopping container istio-proxy 41m Normal Killing pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-bf77z Stopping container istio-proxy 41m Normal Killing pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-7bhqj Stopping container istio-proxy 41m Normal Killing pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-q5795 Stopping container istio-proxy 41m Normal Killing pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-tjbw5 Stopping container istio-proxy 41m Normal Killing pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-txq8f Stopping container istio-proxy 41m Warning Unhealthy pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-7bhqj Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning ProbeError pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-7bhqj Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning ProbeError pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-txq8f Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-txq8f Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning Unhealthy pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-q5795 Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning Unhealthy pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-x68sw Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning Unhealthy pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-tjbw5 Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning ProbeError pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-df6cw Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning ProbeError pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-q5795 Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning ProbeError pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-x68sw Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-dbxb6 Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning ProbeError pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-6t544 Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-6t544 Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning Unhealthy pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-df6cw Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning ProbeError pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-dbxb6 Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning ProbeError pod/e2e-inf-a-6922d3-openshift-default-689b5f95bc-tjbw5 Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning ProbeError pod/e2e-inf-b-3acfdb-openshift-default-5b477b885f-xtl6s Startup probe error: Get "http://10.132.0.186:15021/healthz/ready": dial tcp 10.132.0.186:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-inf-b-3acfdb-openshift-default-5b477b885f-xtl6s Startup probe failed: Get "http://10.132.0.186:15021/healthz/ready": dial tcp 10.132.0.186:15021: connect: connection refused 41m Normal Killing pod/payload-processing-e2e-inf-b-f9d87b-66796c8fff-5mhk5 Stopping container payload-processing 41m Normal Killing pod/payload-pre-processing-e2e-inf-b-f9d87b-d46894c99-vcmjs Stopping container payload-pre-processing 41m Warning Unhealthy pod/payload-processing-e2e-inf-a-6922d3-59cf97fb57-v5z2m Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 41m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-6922d3-69c86dfc75-ktwbq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-6e0edc-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 41m Normal ScalingReplicaSet deployment/e2e-inf-a-6e0edc-openshift-default Scaled up replica set e2e-inf-a-6e0edc-openshift-default-7968b4dddf from 1 to 2 41m Normal SuccessfulCreate replicaset/e2e-inf-a-6e0edc-openshift-default-7968b4dddf Created pod: e2e-inf-a-6e0edc-openshift-default-7968b4dddf-mthln 41m Normal AddedInterface pod/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-mthln Add eth0 [10.132.0.187/23] from ovn-kubernetes 41m Normal Started pod/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-mthln Container started 41m Normal Pulled pod/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-mthln Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-mthln Container created 41m Warning Unhealthy pod/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-mthln Startup probe failed: Get "http://10.132.0.187:15021/healthz/ready": dial tcp 10.132.0.187:15021: connect: connection refused 41m Warning ProbeError pod/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-mthln Startup probe error: Get "http://10.132.0.187:15021/healthz/ready": dial tcp 10.132.0.187:15021: connect: connection refused... 41m Warning Unhealthy pod/payload-processing-e2e-inf-b-f9d87b-66796c8fff-5mhk5 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 41m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-f9d87b-d46894c99-vcmjs Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 40m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 40m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 40m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-c926c Stopping container discovery 40m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-c926c 40m Normal Killing pod/payload-pre-processing-e2e-inf-a-bdd94b-74d9b684bc-m4kfl Stopping container payload-pre-processing 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-421583-openshift-default New size: 1; reason: All metrics below target 40m Normal Killing pod/e2e-worker-w2-421583-openshift-default-5cb767b447-spjhs Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w2-421583-openshift-default-5cb767b447 Deleted pod: e2e-worker-w2-421583-openshift-default-5cb767b447-spjhs 40m Normal Killing pod/payload-processing-e2e-inf-a-bdd94b-5f4f4579c9-4mq7q Stopping container payload-processing 40m Normal ScalingReplicaSet deployment/e2e-worker-w2-421583-openshift-default Scaled down replica set e2e-worker-w2-421583-openshift-default-5cb767b447 from 2 to 1 40m Normal Killing pod/e2e-inf-b-f9d87b-openshift-default-6fb7566b69-5hv5d Stopping container istio-proxy 40m Normal ScalingReplicaSet deployment/e2e-inf-a-40229e-openshift-default Scaled up replica set e2e-inf-a-40229e-openshift-default-5f54d778f5 from 1 to 2 40m Normal Killing pod/e2e-inf-b-f9d87b-openshift-default-6fb7566b69-2c4wk Stopping container istio-proxy 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-40229e-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 40m Normal SuccessfulCreate replicaset/e2e-inf-a-40229e-openshift-default-5f54d778f5 Created pod: e2e-inf-a-40229e-openshift-default-5f54d778f5-n4d9v 40m Normal AddedInterface pod/e2e-inf-a-40229e-openshift-default-5f54d778f5-n4d9v Add eth0 [10.132.0.188/23] from ovn-kubernetes 40m Normal Created pod/e2e-inf-a-40229e-openshift-default-5f54d778f5-n4d9v Container created 40m Normal Pulled pod/e2e-inf-a-40229e-openshift-default-5f54d778f5-n4d9v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Started pod/e2e-inf-a-40229e-openshift-default-5f54d778f5-n4d9v Container started 40m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-bdd94b-74d9b684bc-m4kfl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 40m Warning Unhealthy pod/e2e-inf-a-40229e-openshift-default-5f54d778f5-n4d9v Startup probe failed: Get "http://10.132.0.188:15021/healthz/ready": dial tcp 10.132.0.188:15021: connect: connection refused 40m Warning ProbeError pod/e2e-inf-a-40229e-openshift-default-5f54d778f5-n4d9v Startup probe error: Get "http://10.132.0.188:15021/healthz/ready": dial tcp 10.132.0.188:15021: connect: connection refused... 40m Normal Killing pod/e2e-inf-a-bdd94b-openshift-default-7bbf9bcd4f-q846p Stopping container istio-proxy 40m Normal Killing pod/payload-pre-processing-e2e-inf-b-6013c8-94b84fcd7-b54fw Stopping container payload-pre-processing 40m Normal Killing pod/payload-processing-e2e-inf-b-6013c8-7678cc4db9-mppqg Stopping container payload-processing 39m Normal Killing pod/maas-default-gateway-openshift-default-6db586dd6c-frdpl Stopping container istio-proxy 39m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: All metrics below target 39m Normal Killing pod/maas-default-gateway-openshift-default-6db586dd6c-hv7z4 Stopping container istio-proxy 39m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6db586dd6c Deleted pod: maas-default-gateway-openshift-default-6db586dd6c-frdpl 39m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6db586dd6c Deleted pod: maas-default-gateway-openshift-default-6db586dd6c-hv7z4 39m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-6db586dd6c from 6 to 4 39m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-6013c8-94b84fcd7-b54fw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 39m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5 Created pod: e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5-tz9sg 39m Normal Pulled pod/e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5-tz9sg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-0ff5b0-openshift-default Scaled up replica set e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5 from 1 to 2 39m Normal AddedInterface pod/e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5-tz9sg Add eth0 [10.133.0.248/23] from ovn-kubernetes 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-0ff5b0-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 39m Normal Started pod/e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5-tz9sg Container started 39m Normal Created pod/e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5-tz9sg Container created 39m Normal SuccessfulCreate replicaset/e2e-inf-a-6e0edc-openshift-default-7968b4dddf Created pod: e2e-inf-a-6e0edc-openshift-default-7968b4dddf-q7w2l 39m Normal ScalingReplicaSet deployment/e2e-inf-a-6e0edc-openshift-default Scaled up replica set e2e-inf-a-6e0edc-openshift-default-7968b4dddf from 2 to 3 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-6e0edc-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 39m Normal AddedInterface pod/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-q7w2l Add eth0 [10.134.0.130/23] from ovn-kubernetes 39m Normal Pulled pod/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-q7w2l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-q7w2l Container created 39m Normal Killing pod/e2e-inf-b-6013c8-openshift-default-96f465855-llgxc Stopping container istio-proxy 39m Normal Started pod/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-q7w2l Container started 39m Normal AddedLabel gateway/e2e-shared-a-w4-c511a3 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-c511a3 39m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-c511a3-openshift-default Scaled up replica set e2e-shared-a-w4-c511a3-openshift-default-59bc6d8b7 from 0 to 1 39m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-c511a3-openshift-default-59bc6d8b7 Created pod: e2e-shared-a-w4-c511a3-openshift-default-59bc6d8b7-9m4nr 39m Warning Unhealthy pod/e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5-tz9sg Startup probe failed: Get "http://10.133.0.248:15021/healthz/ready": dial tcp 10.133.0.248:15021: connect: connection refused 39m Normal AddedInterface pod/e2e-shared-a-w4-c511a3-openshift-default-59bc6d8b7-9m4nr Add eth0 [10.132.0.191/23] from ovn-kubernetes 39m Warning ProbeError pod/e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5-tz9sg Startup probe error: Get "http://10.133.0.248:15021/healthz/ready": dial tcp 10.133.0.248:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-q7w2l Startup probe failed: Get "http://10.134.0.130:15021/healthz/ready": dial tcp 10.134.0.130:15021: connect: connection refused 39m Warning ProbeError pod/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-q7w2l Startup probe error: Get "http://10.134.0.130:15021/healthz/ready": dial tcp 10.134.0.130:15021: connect: connection refused... 39m Warning ProbeError pod/e2e-inf-b-6013c8-openshift-default-96f465855-llgxc Readiness probe error: Get "http://10.133.0.213:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 39m Warning Unhealthy pod/e2e-inf-b-6013c8-openshift-default-96f465855-llgxc Readiness probe failed: Get "http://10.133.0.213:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w4-c511a3-7c6f5989ff Created pod: payload-pre-processing-e2e-shared-a-w4-c511a3-7c6f5989ff-r6zv2 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w4-c511a3 Scaled up replica set payload-pre-processing-e2e-shared-a-w4-c511a3-7c6f5989ff from 0 to 1 39m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w4-c511a3-5ddf74769 Error creating: pods "payload-processing-e2e-shared-a-w4-c511a3-5ddf74769-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-c511a3: serviceaccount "payload-processing-e2e-shared-a-w4-c511a3" not found 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w4-c511a3-5ddf74769 Created pod: payload-processing-e2e-shared-a-w4-c511a3-5ddf74769-fztjg 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w4-c511a3-7c6f5989ff Error creating: pods "payload-pre-processing-e2e-shared-a-w4-c511a3-7c6f5989ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-c511a3: serviceaccount "payload-processing-e2e-shared-a-w4-c511a3" not found 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w4-c511a3 Scaled up replica set payload-processing-e2e-shared-a-w4-c511a3-5ddf74769 from 0 to 1 39m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w4-c511a3-7c6f5989ff-r6zv2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w4-c511a3-7c6f5989ff-r6zv2 Add eth0 [10.132.0.201/23] from ovn-kubernetes 39m Normal Pulled pod/payload-processing-e2e-shared-a-w4-c511a3-5ddf74769-fztjg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/payload-processing-e2e-shared-a-w4-c511a3-5ddf74769-fztjg Add eth0 [10.132.0.202/23] from ovn-kubernetes 39m Normal Created pod/payload-pre-processing-e2e-shared-a-w4-c511a3-7c6f5989ff-r6zv2 Container created 39m Normal Started pod/payload-pre-processing-e2e-shared-a-w4-c511a3-7c6f5989ff-r6zv2 Container started 39m Normal Started pod/payload-processing-e2e-shared-a-w4-c511a3-5ddf74769-fztjg Container started 39m Normal Created pod/payload-processing-e2e-shared-a-w4-c511a3-5ddf74769-fztjg Container created 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-d52c59-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 39m Normal SuccessfulDelete replicaset/e2e-worker-w3-d52c59-openshift-default-5466968fb7 Deleted pod: e2e-worker-w3-d52c59-openshift-default-5466968fb7-qchs2 39m Normal Killing pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-qchs2 Stopping container istio-proxy 39m Normal ScalingReplicaSet deployment/e2e-worker-w3-d52c59-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w3-d52c59-openshift-default-5466968fb7 from 2 to 1 38m Warning Unhealthy pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-qchs2 Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning ProbeError pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-qchs2 Readiness probe error: HTTP probe failed with statuscode: 503... 38m Normal Killing pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-qhw2n Stopping container istio-proxy 38m Normal SuccessfulDelete replicaset/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 Deleted pod: e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-qhw2n 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-09d0d5-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 38m Normal ScalingReplicaSet deployment/e2e-worker-w0-09d0d5-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w0-09d0d5-openshift-default-fdf9b9665 from 2 to 1 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-c13db0-openshift-default New size: 1; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/e2e-inf-b-c13db0-openshift-default Scaled down replica set e2e-inf-b-c13db0-openshift-default-f894f747 from 2 to 1 37m Normal Killing pod/e2e-inf-b-c13db0-openshift-default-f894f747-t6h8c Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/e2e-inf-b-c13db0-openshift-default-f894f747 Deleted pod: e2e-inf-b-c13db0-openshift-default-f894f747-t6h8c 37m Normal SuccessfulDelete replicaset/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 Deleted pod: e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-47dz7 37m Normal Killing pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-47dz7 Stopping container istio-proxy 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-2b1e88-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/e2e-worker-w1-2b1e88-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w1-2b1e88-openshift-default-84bb47bf79 from 2 to 1 37m Normal Killing pod/payload-pre-processing-e2e-inf-a-999470-7757847d95-dfvnm Stopping container payload-pre-processing 37m Normal Killing pod/payload-processing-e2e-inf-a-999470-99979b95c-2x956 Stopping container payload-processing 37m Normal Pulled pod/e2e-shared-a-w4-c511a3-openshift-default-59bc6d8b7-9m4nr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Started pod/e2e-shared-a-w4-c511a3-openshift-default-59bc6d8b7-9m4nr Container started 37m Normal Created pod/e2e-shared-a-w4-c511a3-openshift-default-59bc6d8b7-9m4nr Container created 37m Warning ProbeError pod/e2e-shared-a-w4-c511a3-openshift-default-59bc6d8b7-9m4nr Startup probe error: Get "http://10.132.0.191:15021/healthz/ready": dial tcp 10.132.0.191:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-shared-a-w4-c511a3-openshift-default-59bc6d8b7-9m4nr Startup probe failed: Get "http://10.132.0.191:15021/healthz/ready": dial tcp 10.132.0.191:15021: connect: connection refused 37m Normal Killing pod/maas-default-gateway-openshift-default-6db586dd6c-sv4fp Stopping container istio-proxy 37m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: All metrics below target 37m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6db586dd6c Deleted pod: maas-default-gateway-openshift-default-6db586dd6c-sv4fp 37m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-6db586dd6c from 4 to 3 37m Normal AddedLabel gateway/e2e-shared-b-w4-b03b33 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-b03b33 37m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-b03b33-openshift-default Scaled up replica set e2e-shared-b-w4-b03b33-openshift-default-79f6954695 from 0 to 1 37m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-b03b33-openshift-default-79f6954695 Created pod: e2e-shared-b-w4-b03b33-openshift-default-79f6954695-882qd 37m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-sv4fp Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-sv4fp Readiness probe error: HTTP probe failed with statuscode: 503... 37m Normal AddedInterface pod/e2e-shared-b-w4-b03b33-openshift-default-79f6954695-882qd Add eth0 [10.132.0.204/23] from ovn-kubernetes 37m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-999470-7757847d95-dfvnm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w4-b03b33 Scaled up replica set payload-processing-e2e-shared-b-w4-b03b33-7c5fcd4d57 from 0 to 1 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w4-b03b33 Scaled up replica set payload-pre-processing-e2e-shared-b-w4-b03b33-cd67bd8f5 from 0 to 1 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w4-b03b33-cd67bd8f5 Created pod: payload-pre-processing-e2e-shared-b-w4-b03b33-cd67bd8f5-dft76 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w4-b03b33-cd67bd8f5 Error creating: pods "payload-pre-processing-e2e-shared-b-w4-b03b33-cd67bd8f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-b03b33: serviceaccount "payload-processing-e2e-shared-b-w4-b03b33" not found 37m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w4-b03b33-7c5fcd4d57 Error creating: pods "payload-processing-e2e-shared-b-w4-b03b33-7c5fcd4d57-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-b03b33: serviceaccount "payload-processing-e2e-shared-b-w4-b03b33" not found 37m Warning Unhealthy pod/payload-processing-e2e-inf-a-999470-99979b95c-2x956 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w4-b03b33-7c5fcd4d57 Created pod: payload-processing-e2e-shared-b-w4-b03b33-7c5fcd4d57-g2t4v 37m Normal Started pod/payload-pre-processing-e2e-shared-b-w4-b03b33-cd67bd8f5-dft76 Container started 37m Normal Created pod/payload-pre-processing-e2e-shared-b-w4-b03b33-cd67bd8f5-dft76 Container created 37m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w4-b03b33-cd67bd8f5-dft76 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/payload-processing-e2e-shared-b-w4-b03b33-7c5fcd4d57-g2t4v Add eth0 [10.133.1.15/23] from ovn-kubernetes 37m Normal Pulled pod/payload-processing-e2e-shared-b-w4-b03b33-7c5fcd4d57-g2t4v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Created pod/payload-processing-e2e-shared-b-w4-b03b33-7c5fcd4d57-g2t4v Container created 37m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w4-b03b33-cd67bd8f5-dft76 Add eth0 [10.133.1.16/23] from ovn-kubernetes 37m Normal Started pod/payload-processing-e2e-shared-b-w4-b03b33-7c5fcd4d57-g2t4v Container started 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-c511a3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-c511a3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-b03b33-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-b03b33-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Normal SuccessfulCreate replicaset/e2e-inf-b-867f12-openshift-default-6584f84965 Created pod: e2e-inf-b-867f12-openshift-default-6584f84965-znd9z 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-867f12-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 37m Normal ScalingReplicaSet deployment/e2e-inf-b-867f12-openshift-default Scaled up replica set e2e-inf-b-867f12-openshift-default-6584f84965 from 2 to 3 37m Normal AddedInterface pod/e2e-inf-b-867f12-openshift-default-6584f84965-znd9z Add eth0 [10.134.0.131/23] from ovn-kubernetes 37m Normal Created pod/e2e-inf-b-867f12-openshift-default-6584f84965-znd9z Container created 37m Normal Started pod/e2e-inf-b-867f12-openshift-default-6584f84965-znd9z Container started 37m Normal Pulled pod/e2e-inf-b-867f12-openshift-default-6584f84965-znd9z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Killing pod/e2e-inf-a-999470-openshift-default-6887745cf9-xt2fd Stopping container istio-proxy 36m Warning ProbeError pod/e2e-inf-b-867f12-openshift-default-6584f84965-znd9z Startup probe error: Get "http://10.134.0.131:15021/healthz/ready": dial tcp 10.134.0.131:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-inf-b-867f12-openshift-default-6584f84965-znd9z Startup probe failed: Get "http://10.134.0.131:15021/healthz/ready": dial tcp 10.134.0.131:15021: connect: connection refused 36m Normal Killing pod/payload-processing-e2e-inf-b-54c7f0-5448fb4d5c-nrrpn Stopping container payload-processing 36m Normal Killing pod/payload-pre-processing-e2e-inf-b-54c7f0-545cb4dc95-xpm4m Stopping container payload-pre-processing 36m Normal Killing pod/payload-processing-e2e-inf-a-d85da4-7d5f448d7c-cwh99 Stopping container payload-processing 36m Normal Killing pod/payload-pre-processing-e2e-inf-a-d85da4-7bdd674455-fjhgp Stopping container payload-pre-processing 36m Normal Killing pod/payload-processing-e2e-inf-a-40229e-86586c78fc-288bj Stopping container payload-processing 36m Normal Killing pod/payload-pre-processing-e2e-inf-a-40229e-5fb694f79-5glps Stopping container payload-pre-processing 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-54114e-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 36m Normal Killing pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-z58t9 Stopping container istio-proxy 36m Normal SuccessfulDelete replicaset/e2e-worker-w5-54114e-openshift-default-7fc467ddd7 Deleted pod: e2e-worker-w5-54114e-openshift-default-7fc467ddd7-z58t9 36m Normal ScalingReplicaSet deployment/e2e-worker-w5-54114e-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w5-54114e-openshift-default-7fc467ddd7 from 2 to 1 36m Warning Unhealthy pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-z58t9 Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning ProbeError pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-z58t9 Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-54c7f0-545cb4dc95-xpm4m Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Normal Killing pod/e2e-inf-a-d85da4-openshift-default-6b5f89667-d8fjz Stopping container istio-proxy 36m Normal Killing pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-zqkns Stopping container istio-proxy 36m Normal SuccessfulDelete replicaset/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf Deleted pod: e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-zqkns 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-510f80-openshift-default New size: 1; reason: All metrics below target 36m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-510f80-openshift-default Scaled down replica set e2e-shared-b-w5-510f80-openshift-default-6465ccddcf from 2 to 1 36m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-d85da4-7bdd674455-fjhgp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Normal Killing pod/payload-processing-e2e-inf-b-f0bb45-5f54c4488f-hdmsx Stopping container payload-processing 36m Normal Killing pod/payload-pre-processing-e2e-inf-b-f0bb45-64bb8c9b67-2dxq4 Stopping container payload-pre-processing 36m Warning Unhealthy pod/payload-processing-e2e-inf-a-40229e-86586c78fc-288bj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-40229e-5fb694f79-5glps Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Normal Killing pod/e2e-inf-a-40229e-openshift-default-5f54d778f5-n4d9v Stopping container istio-proxy 36m Normal Killing pod/e2e-inf-a-40229e-openshift-default-5f54d778f5-zhln9 Stopping container istio-proxy 36m Normal Killing pod/e2e-inf-b-54c7f0-openshift-default-79cd69d455-tfzdz Stopping container istio-proxy 36m Normal Killing pod/e2e-inf-b-54c7f0-openshift-default-79cd69d455-dznhn Stopping container istio-proxy 36m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-bed544-openshift-default-8659947965 Created pod: e2e-shared-a-w1-bed544-openshift-default-8659947965-dmmj5 36m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-bed544-openshift-default Scaled up replica set e2e-shared-a-w1-bed544-openshift-default-8659947965 from 0 to 1 36m Normal AddedLabel gateway/e2e-shared-a-w1-bed544 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w1-bed544 35m Normal AddedInterface pod/e2e-shared-a-w1-bed544-openshift-default-8659947965-dmmj5 Add eth0 [10.133.1.26/23] from ovn-kubernetes 35m Warning ProbeError pod/e2e-inf-b-54c7f0-openshift-default-79cd69d455-tfzdz Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-inf-b-54c7f0-openshift-default-79cd69d455-tfzdz Readiness probe failed: HTTP probe failed with statuscode: 503 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w1-bed544-6bc95f58bc Created pod: payload-pre-processing-e2e-shared-a-w1-bed544-6bc95f58bc-xmfrv 35m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w1-bed544-5b5dc64c47 Error creating: pods "payload-processing-e2e-shared-a-w1-bed544-5b5dc64c47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-bed544: serviceaccount "payload-processing-e2e-shared-a-w1-bed544" not found 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w1-bed544 Scaled up replica set payload-pre-processing-e2e-shared-a-w1-bed544-6bc95f58bc from 0 to 1 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w1-bed544-5b5dc64c47 Created pod: payload-processing-e2e-shared-a-w1-bed544-5b5dc64c47-5dwdv 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w1-bed544 Scaled up replica set payload-processing-e2e-shared-a-w1-bed544-5b5dc64c47 from 0 to 1 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w1-bed544-6bc95f58bc Error creating: pods "payload-pre-processing-e2e-shared-a-w1-bed544-6bc95f58bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-bed544: serviceaccount "payload-processing-e2e-shared-a-w1-bed544" not found 35m Normal Created pod/payload-processing-e2e-shared-a-w1-bed544-5b5dc64c47-5dwdv Container created 35m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-f0bb45-64bb8c9b67-2dxq4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 35m Normal Created pod/payload-pre-processing-e2e-shared-a-w1-bed544-6bc95f58bc-xmfrv Container created 35m Normal Pulled pod/payload-processing-e2e-shared-a-w1-bed544-5b5dc64c47-5dwdv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w1-bed544-6bc95f58bc-xmfrv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w1-bed544-6bc95f58bc-xmfrv Add eth0 [10.132.0.207/23] from ovn-kubernetes 35m Normal Started pod/payload-processing-e2e-shared-a-w1-bed544-5b5dc64c47-5dwdv Container started 35m Normal Started pod/payload-pre-processing-e2e-shared-a-w1-bed544-6bc95f58bc-xmfrv Container started 35m Normal AddedInterface pod/payload-processing-e2e-shared-a-w1-bed544-5b5dc64c47-5dwdv Add eth0 [10.132.0.206/23] from ovn-kubernetes 35m Warning Unhealthy pod/payload-processing-e2e-inf-b-f0bb45-5f54c4488f-hdmsx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 35m Normal Killing pod/payload-processing-e2e-inf-b-c13db0-5dbfdd689-q5zt5 Stopping container payload-processing 35m Normal Killing pod/payload-pre-processing-e2e-inf-b-c13db0-bcdbd4c9f-tqwjn Stopping container payload-pre-processing 35m Normal Killing pod/payload-pre-processing-e2e-inf-a-6e0edc-6cb4f7d887-wn55s Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-inf-a-6e0edc-c7974cff5-rr5v8 Stopping container payload-processing 35m Normal Killing pod/e2e-inf-b-f0bb45-openshift-default-6b64cff887-s9c4q Stopping container istio-proxy 35m Normal Killing pod/e2e-inf-b-f0bb45-openshift-default-6b64cff887-w4hj4 Stopping container istio-proxy 35m Normal ScalingReplicaSet deployment/e2e-worker-w4-0ad7c5-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w4-0ad7c5-openshift-default-788b484d5c from 2 to 1 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0ad7c5-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 35m Normal SuccessfulDelete replicaset/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c Deleted pod: e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hq9vz 35m Normal Killing pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-lg99w Stopping container istio-proxy 35m Normal Killing pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hq9vz Stopping container istio-proxy 35m Normal SuccessfulDelete replicaset/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc Deleted pod: e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-lg99w 35m Normal ScalingReplicaSet deployment/e2e-worker-w6-012ba0-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w6-012ba0-openshift-default-846bcc5bfc from 2 to 1 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-012ba0-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 35m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-c13db0-bcdbd4c9f-tqwjn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 35m Warning Unhealthy pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hq9vz Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-hq9vz Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-6e0edc-6cb4f7d887-wn55s Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 35m Warning Unhealthy pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-lg99w Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning Unhealthy pod/payload-processing-e2e-inf-a-6e0edc-c7974cff5-rr5v8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 35m Warning ProbeError pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-lg99w Readiness probe error: HTTP probe failed with statuscode: 503... 35m Normal Pulled pod/e2e-shared-b-w4-b03b33-openshift-default-79f6954695-882qd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Started pod/e2e-shared-b-w4-b03b33-openshift-default-79f6954695-882qd Container started 35m Normal Created pod/e2e-shared-b-w4-b03b33-openshift-default-79f6954695-882qd Container created 35m Normal Killing pod/e2e-inf-b-c13db0-openshift-default-f894f747-cbtvq Stopping container istio-proxy 35m Warning Unhealthy pod/e2e-shared-b-w4-b03b33-openshift-default-79f6954695-882qd Startup probe failed: Get "http://10.132.0.204:15021/healthz/ready": dial tcp 10.132.0.204:15021: connect: connection refused 35m Warning ProbeError pod/e2e-shared-b-w4-b03b33-openshift-default-79f6954695-882qd Startup probe error: Get "http://10.132.0.204:15021/healthz/ready": dial tcp 10.132.0.204:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-inf-b-c13db0-openshift-default-f894f747-cbtvq Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-inf-b-c13db0-openshift-default-f894f747-cbtvq Readiness probe error: HTTP probe failed with statuscode: 503... 35m Normal Killing pod/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-q7w2l Stopping container istio-proxy 35m Normal Killing pod/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-mthln Stopping container istio-proxy 35m Normal Killing pod/e2e-inf-a-6e0edc-openshift-default-7968b4dddf-86ztv Stopping container istio-proxy 35m Normal AddedInterface pod/e2e-inf-a-bc48cc-openshift-default-d9498d65-p62qs Add eth0 [10.132.0.211/23] from ovn-kubernetes 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-bc48cc-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 35m Normal SuccessfulCreate replicaset/e2e-inf-a-bc48cc-openshift-default-d9498d65 Created pod: e2e-inf-a-bc48cc-openshift-default-d9498d65-p62qs 35m Normal Pulled pod/e2e-inf-a-bc48cc-openshift-default-d9498d65-p62qs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal ScalingReplicaSet deployment/e2e-inf-a-bc48cc-openshift-default Scaled up replica set e2e-inf-a-bc48cc-openshift-default-d9498d65 from 1 to 2 35m Normal Created pod/e2e-inf-a-bc48cc-openshift-default-d9498d65-p62qs Container created 35m Normal Started pod/e2e-inf-a-bc48cc-openshift-default-d9498d65-p62qs Container started 35m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 2; reason: All metrics below target 35m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-6db586dd6c from 3 to 2 35m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-6db586dd6c Deleted pod: maas-default-gateway-openshift-default-6db586dd6c-8c8cz 35m Normal Killing pod/maas-default-gateway-openshift-default-6db586dd6c-8c8cz Stopping container istio-proxy 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-b03b33-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-b03b33-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning Unhealthy pod/e2e-inf-a-bc48cc-openshift-default-d9498d65-p62qs Startup probe failed: Get "http://10.132.0.211:15021/healthz/ready": dial tcp 10.132.0.211:15021: connect: connection refused 34m Warning ProbeError pod/e2e-inf-a-bc48cc-openshift-default-d9498d65-p62qs Startup probe error: Get "http://10.132.0.211:15021/healthz/ready": dial tcp 10.132.0.211:15021: connect: connection refused... 34m Normal Killing pod/payload-pre-processing-e2e-inf-b-3acfdb-66c888cfdc-ktrkf Stopping container payload-pre-processing 34m Normal Killing pod/payload-processing-e2e-inf-b-3acfdb-7f8bb6c86f-7bsb5 Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-inf-a-bc48cc-5554659f79-79xg7 Stopping container payload-pre-processing 34m Normal Killing pod/payload-processing-e2e-inf-a-bc48cc-7d467bc759-tqr4d Stopping container payload-processing 34m Normal Killing pod/e2e-inf-a-bc48cc-openshift-default-d9498d65-p62qs Stopping container istio-proxy 34m Warning ProbeError pod/e2e-inf-a-bc48cc-openshift-default-d9498d65-p62qs Readiness probe error: HTTP probe failed with statuscode: 503... 34m Warning Unhealthy pod/e2e-inf-a-bc48cc-openshift-default-d9498d65-p62qs Readiness probe failed: HTTP probe failed with statuscode: 503 34m Normal Killing pod/e2e-inf-a-bc48cc-openshift-default-d9498d65-kbdtw Stopping container istio-proxy 34m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-3acfdb-66c888cfdc-ktrkf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning Unhealthy pod/payload-processing-e2e-inf-a-bc48cc-7d467bc759-tqr4d Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c Created pod: e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c-9ps8d 34m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-6f2eba-openshift-default Scaled up replica set e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c from 1 to 2 34m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-bc48cc-5554659f79-79xg7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Normal Killing pod/payload-pre-processing-e2e-inf-b-867f12-f59b9bd47-xkz7q Stopping container payload-pre-processing 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w5-6f2eba-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 34m Normal Killing pod/payload-processing-e2e-inf-b-867f12-66d87c857f-vsjkv Stopping container payload-processing 34m Normal AddedInterface pod/e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c-9ps8d Add eth0 [10.132.0.212/23] from ovn-kubernetes 34m Normal Started pod/e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c-9ps8d Container started 34m Normal Created pod/e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c-9ps8d Container created 34m Normal Pulled pod/e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c-9ps8d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Created pod/e2e-shared-a-w1-bed544-openshift-default-8659947965-dmmj5 Container created 34m Normal Pulled pod/e2e-shared-a-w1-bed544-openshift-default-8659947965-dmmj5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Started pod/e2e-shared-a-w1-bed544-openshift-default-8659947965-dmmj5 Container started 34m Normal AddedLabel gateway/e2e-shared-b-w1-23aab2 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w1-23aab2 34m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-23aab2-openshift-default Scaled up replica set e2e-shared-b-w1-23aab2-openshift-default-86c4fcc59c from 0 to 1 34m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-23aab2-openshift-default-86c4fcc59c Created pod: e2e-shared-b-w1-23aab2-openshift-default-86c4fcc59c-qvpxp 34m Normal AddedInterface pod/e2e-shared-b-w1-23aab2-openshift-default-86c4fcc59c-qvpxp Add eth0 [10.133.1.33/23] from ovn-kubernetes 34m Warning ProbeError pod/e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c-9ps8d Startup probe error: Get "http://10.132.0.212:15021/healthz/ready": dial tcp 10.132.0.212:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c-9ps8d Startup probe failed: Get "http://10.132.0.212:15021/healthz/ready": dial tcp 10.132.0.212:15021: connect: connection refused 34m Warning Unhealthy pod/e2e-shared-a-w1-bed544-openshift-default-8659947965-dmmj5 Startup probe failed: Get "http://10.133.1.26:15021/healthz/ready": dial tcp 10.133.1.26:15021: connect: connection refused 34m Warning ProbeError pod/e2e-shared-a-w1-bed544-openshift-default-8659947965-dmmj5 Startup probe error: Get "http://10.133.1.26:15021/healthz/ready": dial tcp 10.133.1.26:15021: connect: connection refused... 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w1-23aab2-ff485dcf Error creating: pods "payload-pre-processing-e2e-shared-b-w1-23aab2-ff485dcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-23aab2: serviceaccount "payload-processing-e2e-shared-b-w1-23aab2" not found 34m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w1-23aab2-67f75c4687 Error creating: pods "payload-processing-e2e-shared-b-w1-23aab2-67f75c4687-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-23aab2: serviceaccount "payload-processing-e2e-shared-b-w1-23aab2" not found 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w1-23aab2 Scaled up replica set payload-pre-processing-e2e-shared-b-w1-23aab2-ff485dcf from 0 to 1 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w1-23aab2 Scaled up replica set payload-processing-e2e-shared-b-w1-23aab2-67f75c4687 from 0 to 1 34m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w1-23aab2-ff485dcf-pzd2r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Created pod/payload-pre-processing-e2e-shared-b-w1-23aab2-ff485dcf-pzd2r Container created 34m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w1-23aab2-ff485dcf-pzd2r Add eth0 [10.133.1.34/23] from ovn-kubernetes 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w1-23aab2-67f75c4687 Created pod: payload-processing-e2e-shared-b-w1-23aab2-67f75c4687-2wq29 34m Normal Started pod/payload-pre-processing-e2e-shared-b-w1-23aab2-ff485dcf-pzd2r Container started 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w1-23aab2-ff485dcf Created pod: payload-pre-processing-e2e-shared-b-w1-23aab2-ff485dcf-pzd2r 34m Normal AddedInterface pod/payload-processing-e2e-shared-b-w1-23aab2-67f75c4687-2wq29 Add eth0 [10.133.1.35/23] from ovn-kubernetes 34m Normal Started pod/payload-processing-e2e-shared-b-w1-23aab2-67f75c4687-2wq29 Container started 34m Normal Created pod/payload-processing-e2e-shared-b-w1-23aab2-67f75c4687-2wq29 Container created 34m Normal Pulled pod/payload-processing-e2e-shared-b-w1-23aab2-67f75c4687-2wq29 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Killing pod/e2e-inf-b-3acfdb-openshift-default-5b477b885f-xtl6s Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-b-3acfdb-openshift-default-5b477b885f-mrqbm Stopping container istio-proxy 34m Warning ProbeError pod/e2e-inf-b-3acfdb-openshift-default-5b477b885f-xtl6s Readiness probe error: HTTP probe failed with statuscode: 503... 34m Warning Unhealthy pod/e2e-inf-b-3acfdb-openshift-default-5b477b885f-xtl6s Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning Unhealthy pod/payload-processing-e2e-inf-b-867f12-66d87c857f-vsjkv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-867f12-f59b9bd47-xkz7q Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning Unhealthy pod/payload-processing-e2e-inf-b-867f12-66d87c857f-vsjkv Readiness probe failed: timeout: failed to connect service "10.132.0.176:50052" within 1s: context deadline exceeded 34m Normal Killing pod/e2e-inf-b-867f12-openshift-default-6584f84965-jlqnx Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-b-867f12-openshift-default-6584f84965-9rbbv Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-b-867f12-openshift-default-6584f84965-znd9z Stopping container istio-proxy 34m Normal AddedLabel gateway/e2e-shared-a-w3-cdc122 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-cdc122 34m Normal AddedInterface pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-x4lb9 Add eth0 [10.133.1.36/23] from ovn-kubernetes 34m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc Created pod: e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-x4lb9 34m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-cdc122-openshift-default Scaled up replica set e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc from 0 to 1 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-bed544-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-bed544-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-23aab2-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-23aab2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w3-cdc122-5b697dd967 Error creating: pods "payload-pre-processing-e2e-shared-a-w3-cdc122-5b697dd967-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-cdc122: serviceaccount "payload-processing-e2e-shared-a-w3-cdc122" not found 33m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w3-cdc122-5b697dd967 Created pod: payload-pre-processing-e2e-shared-a-w3-cdc122-5b697dd967-7qll4 33m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w3-cdc122-5584f9d75c Error creating: pods "payload-processing-e2e-shared-a-w3-cdc122-5584f9d75c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-cdc122: serviceaccount "payload-processing-e2e-shared-a-w3-cdc122" not found 33m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w3-cdc122 Scaled up replica set payload-processing-e2e-shared-a-w3-cdc122-5584f9d75c from 0 to 1 33m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w3-cdc122 Scaled up replica set payload-pre-processing-e2e-shared-a-w3-cdc122-5b697dd967 from 0 to 1 33m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w3-cdc122-5584f9d75c Created pod: payload-processing-e2e-shared-a-w3-cdc122-5584f9d75c-fljm5 33m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w3-cdc122-5b697dd967-7qll4 Add eth0 [10.133.1.37/23] from ovn-kubernetes 33m Normal Started pod/payload-processing-e2e-shared-a-w3-cdc122-5584f9d75c-fljm5 Container started 33m Normal Created pod/payload-pre-processing-e2e-shared-a-w3-cdc122-5b697dd967-7qll4 Container created 33m Normal Started pod/payload-pre-processing-e2e-shared-a-w3-cdc122-5b697dd967-7qll4 Container started 33m Normal Created pod/payload-processing-e2e-shared-a-w3-cdc122-5584f9d75c-fljm5 Container created 33m Normal AddedInterface pod/payload-processing-e2e-shared-a-w3-cdc122-5584f9d75c-fljm5 Add eth0 [10.132.0.215/23] from ovn-kubernetes 33m Normal Pulled pod/payload-processing-e2e-shared-a-w3-cdc122-5584f9d75c-fljm5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w3-cdc122-5b697dd967-7qll4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal AddedLabel gateway/e2e-ipp-a-e3a7dd Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-e3a7dd 32m Normal SuccessfulCreate replicaset/e2e-ipp-a-e3a7dd-openshift-default-6c66b84dc9 Created pod: e2e-ipp-a-e3a7dd-openshift-default-6c66b84dc9-v6hrm 32m Normal ScalingReplicaSet deployment/e2e-ipp-a-e3a7dd-openshift-default Scaled up replica set e2e-ipp-a-e3a7dd-openshift-default-6c66b84dc9 from 0 to 1 32m Normal AddedInterface pod/e2e-ipp-a-e3a7dd-openshift-default-6c66b84dc9-v6hrm Add eth0 [10.133.1.40/23] from ovn-kubernetes 32m Warning Unhealthy pod/e2e-shared-b-w1-23aab2-openshift-default-86c4fcc59c-qvpxp Readiness probe failed: Get "http://10.133.1.33:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 32m Warning ProbeError pod/e2e-shared-b-w1-23aab2-openshift-default-86c4fcc59c-qvpxp Readiness probe error: Get "http://10.133.1.33:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 32m Normal Pulled pod/e2e-shared-b-w1-23aab2-openshift-default-86c4fcc59c-qvpxp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-shared-b-w1-23aab2-openshift-default-86c4fcc59c-qvpxp Container created 32m Normal Started pod/e2e-shared-b-w1-23aab2-openshift-default-86c4fcc59c-qvpxp Container started 32m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-e3a7dd-666fdd8fcc Error creating: pods "payload-processing-e2e-ipp-a-e3a7dd-666fdd8fcc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e3a7dd: serviceaccount "payload-processing-e2e-ipp-a-e3a7dd" not found 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-e3a7dd-55c66888c9 Error creating: pods "payload-pre-processing-e2e-ipp-a-e3a7dd-55c66888c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e3a7dd: serviceaccount "payload-processing-e2e-ipp-a-e3a7dd" not found 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-e3a7dd-55c66888c9 Created pod: payload-pre-processing-e2e-ipp-a-e3a7dd-55c66888c9-fbrpc 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-e3a7dd Scaled up replica set payload-pre-processing-e2e-ipp-a-e3a7dd-55c66888c9 from 0 to 1 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-e3a7dd Scaled up replica set payload-processing-e2e-ipp-a-e3a7dd-666fdd8fcc from 0 to 1 32m Normal Started pod/payload-pre-processing-e2e-ipp-a-e3a7dd-55c66888c9-fbrpc Container started 32m Normal Created pod/payload-pre-processing-e2e-ipp-a-e3a7dd-55c66888c9-fbrpc Container created 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-e3a7dd-666fdd8fcc Created pod: payload-processing-e2e-ipp-a-e3a7dd-666fdd8fcc-qvds2 32m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-e3a7dd-55c66888c9-fbrpc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-e3a7dd-55c66888c9-fbrpc Add eth0 [10.132.0.217/23] from ovn-kubernetes 32m Normal AddedInterface pod/payload-processing-e2e-ipp-a-e3a7dd-666fdd8fcc-qvds2 Add eth0 [10.133.1.41/23] from ovn-kubernetes 32m Normal Pulled pod/payload-processing-e2e-ipp-a-e3a7dd-666fdd8fcc-qvds2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Created pod/payload-processing-e2e-ipp-a-e3a7dd-666fdd8fcc-qvds2 Container created 32m Normal Started pod/payload-processing-e2e-ipp-a-e3a7dd-666fdd8fcc-qvds2 Container started 32m Warning ProbeError pod/e2e-shared-b-w1-23aab2-openshift-default-86c4fcc59c-qvpxp Startup probe error: Get "http://10.133.1.33:15021/healthz/ready": dial tcp 10.133.1.33:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-shared-b-w1-23aab2-openshift-default-86c4fcc59c-qvpxp Startup probe failed: Get "http://10.133.1.33:15021/healthz/ready": dial tcp 10.133.1.33:15021: connect: connection refused 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-d9f1c2-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 32m Normal AddedInterface pod/e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447-hs292 Add eth0 [10.132.0.219/23] from ovn-kubernetes 32m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-6af86e-openshift-default Scaled up replica set e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c from 1 to 2 32m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447 Created pod: e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447-hs292 32m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c Created pod: e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c-2xvj7 32m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-d9f1c2-openshift-default Scaled up replica set e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447 from 1 to 2 32m Normal Created pod/e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447-hs292 Container created 32m Normal Started pod/e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447-hs292 Container started 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-6af86e-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 32m Normal Pulled pod/e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447-hs292 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Started pod/e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c-2xvj7 Container started 32m Normal Created pod/e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c-2xvj7 Container created 32m Normal AddedInterface pod/e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c-2xvj7 Add eth0 [10.133.1.42/23] from ovn-kubernetes 32m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-e14e00-openshift-default-565cddf77c Created pod: e2e-shared-b-w3-e14e00-openshift-default-565cddf77c-7jg4k 32m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-e14e00-openshift-default Scaled up replica set e2e-shared-b-w3-e14e00-openshift-default-565cddf77c from 0 to 1 32m Normal AddedLabel gateway/e2e-shared-b-w3-e14e00 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-e14e00 32m Normal Pulled pod/e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c-2xvj7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/e2e-shared-b-w3-e14e00-openshift-default-565cddf77c-7jg4k Add eth0 [10.132.0.220/23] from ovn-kubernetes 32m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c Created pod: e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-p2chp 32m Warning ProbeError pod/e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447-hs292 Startup probe error: Get "http://10.132.0.219:15021/healthz/ready": dial tcp 10.132.0.219:15021: connect: connection refused... 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-3cd77f-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 32m Warning Unhealthy pod/e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447-hs292 Startup probe failed: Get "http://10.132.0.219:15021/healthz/ready": dial tcp 10.132.0.219:15021: connect: connection refused 32m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-3cd77f-openshift-default Scaled up replica set e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c from 1 to 2 32m Normal AddedInterface pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-p2chp Add eth0 [10.132.0.221/23] from ovn-kubernetes 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w3-e14e00 Scaled up replica set payload-pre-processing-e2e-shared-b-w3-e14e00-bdf56dd9f from 0 to 1 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w3-e14e00-bdf56dd9f Error creating: pods "payload-pre-processing-e2e-shared-b-w3-e14e00-bdf56dd9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-e14e00: serviceaccount "payload-processing-e2e-shared-b-w3-e14e00" not found 32m Normal Pulled pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-p2chp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-p2chp Container created 32m Normal Started pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-p2chp Container started 32m Warning Unhealthy pod/e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c-2xvj7 Startup probe failed: Get "http://10.133.1.42:15021/healthz/ready": dial tcp 10.133.1.42:15021: connect: connection refused 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w3-e14e00 Scaled up replica set payload-processing-e2e-shared-b-w3-e14e00-56ff64bff5 from 0 to 1 32m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w3-e14e00-56ff64bff5 Error creating: pods "payload-processing-e2e-shared-b-w3-e14e00-56ff64bff5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-e14e00: serviceaccount "payload-processing-e2e-shared-b-w3-e14e00" not found 32m Warning ProbeError pod/e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c-2xvj7 Startup probe error: Get "http://10.133.1.42:15021/healthz/ready": dial tcp 10.133.1.42:15021: connect: connection refused... 32m Normal Pulled pod/payload-processing-e2e-shared-b-w3-e14e00-56ff64bff5-ht6tb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/payload-processing-e2e-shared-b-w3-e14e00-56ff64bff5-ht6tb Add eth0 [10.133.1.44/23] from ovn-kubernetes 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w3-e14e00-56ff64bff5 Created pod: payload-processing-e2e-shared-b-w3-e14e00-56ff64bff5-ht6tb 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w3-e14e00-bdf56dd9f Created pod: payload-pre-processing-e2e-shared-b-w3-e14e00-bdf56dd9f-67qgc 32m Normal Created pod/payload-pre-processing-e2e-shared-b-w3-e14e00-bdf56dd9f-67qgc Container created 32m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w3-e14e00-bdf56dd9f-67qgc Add eth0 [10.133.1.43/23] from ovn-kubernetes 32m Normal Started pod/payload-pre-processing-e2e-shared-b-w3-e14e00-bdf56dd9f-67qgc Container started 32m Normal Started pod/payload-processing-e2e-shared-b-w3-e14e00-56ff64bff5-ht6tb Container started 32m Normal Created pod/payload-processing-e2e-shared-b-w3-e14e00-56ff64bff5-ht6tb Container created 32m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w3-e14e00-bdf56dd9f-67qgc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Warning Unhealthy pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-p2chp Startup probe failed: Get "http://10.132.0.221:15021/healthz/ready": dial tcp 10.132.0.221:15021: connect: connection refused 32m Warning ProbeError pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-p2chp Startup probe error: Get "http://10.132.0.221:15021/healthz/ready": dial tcp 10.132.0.221:15021: connect: connection refused... 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-23aab2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-23aab2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-cdc122-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 32m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc Created pod: e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-bmvsb 32m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-cdc122-openshift-default Scaled up replica set e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc from 1 to 4 32m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc Created pod: e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-6bvgh 32m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc Created pod: e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-wsd2x 31m Normal Pulled pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-wsd2x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Started pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-bmvsb Container started 31m Normal AddedInterface pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-6bvgh Add eth0 [10.134.0.139/23] from ovn-kubernetes 31m Normal Pulled pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-6bvgh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Created pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-6bvgh Container created 31m Normal Started pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-x4lb9 Container started 31m Normal Created pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-x4lb9 Container created 31m Normal Pulled pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-x4lb9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Started pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-6bvgh Container started 31m Normal AddedInterface pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-bmvsb Add eth0 [10.133.1.45/23] from ovn-kubernetes 31m Normal Pulled pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-bmvsb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Started pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-wsd2x Container started 31m Normal Created pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-wsd2x Container created 31m Normal AddedInterface pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-wsd2x Add eth0 [10.132.0.222/23] from ovn-kubernetes 31m Normal Created pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-bmvsb Container created 31m Warning ProbeError pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-bmvsb Startup probe error: Get "http://10.133.1.45:15021/healthz/ready": dial tcp 10.133.1.45:15021: connect: connection refused... 31m Warning ProbeError pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-wsd2x Startup probe error: Get "http://10.132.0.222:15021/healthz/ready": dial tcp 10.132.0.222:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-wsd2x Startup probe failed: Get "http://10.132.0.222:15021/healthz/ready": dial tcp 10.132.0.222:15021: connect: connection refused 31m Warning ProbeError pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-6bvgh Startup probe error: Get "http://10.134.0.139:15021/healthz/ready": dial tcp 10.134.0.139:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-bmvsb Startup probe failed: Get "http://10.133.1.45:15021/healthz/ready": dial tcp 10.133.1.45:15021: connect: connection refused 31m Warning Unhealthy pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-6bvgh Startup probe failed: Get "http://10.134.0.139:15021/healthz/ready": dial tcp 10.134.0.139:15021: connect: connection refused 31m Warning ProbeError pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-x4lb9 Startup probe error: Get "http://10.133.1.36:15021/healthz/ready": dial tcp 10.133.1.36:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-x4lb9 Startup probe failed: Get "http://10.133.1.36:15021/healthz/ready": dial tcp 10.133.1.36:15021: connect: connection refused 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-cdc122-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-cdc122-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Normal Killing pod/payload-processing-e2e-ipp-a-e3a7dd-666fdd8fcc-qvds2 Stopping container payload-processing 31m Normal Killing pod/payload-pre-processing-e2e-ipp-a-e3a7dd-55c66888c9-fbrpc Stopping container payload-pre-processing 31m Normal AddedLabel gateway/e2e-ipp-b-a60e80 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-a60e80 31m Normal ScalingReplicaSet deployment/e2e-ipp-b-a60e80-openshift-default Scaled up replica set e2e-ipp-b-a60e80-openshift-default-c8f96989c from 0 to 1 31m Normal SuccessfulCreate replicaset/e2e-ipp-b-a60e80-openshift-default-c8f96989c Created pod: e2e-ipp-b-a60e80-openshift-default-c8f96989c-rvvlt 31m Normal AddedInterface pod/e2e-ipp-b-a60e80-openshift-default-c8f96989c-rvvlt Add eth0 [10.133.1.46/23] from ovn-kubernetes 31m Normal Started pod/e2e-ipp-a-e3a7dd-openshift-default-6c66b84dc9-v6hrm Container started 31m Normal Pulled pod/e2e-ipp-a-e3a7dd-openshift-default-6c66b84dc9-v6hrm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Created pod/e2e-ipp-a-e3a7dd-openshift-default-6c66b84dc9-v6hrm Container created 31m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-a60e80-5949b5b66c Created pod: payload-pre-processing-e2e-ipp-b-a60e80-5949b5b66c-h6x68 31m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-a60e80-5949b5b66c Error creating: pods "payload-pre-processing-e2e-ipp-b-a60e80-5949b5b66c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-a60e80: serviceaccount "payload-processing-e2e-ipp-b-a60e80" not found 31m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-a60e80-7b8469d759 Error creating: pods "payload-processing-e2e-ipp-b-a60e80-7b8469d759-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-a60e80: serviceaccount "payload-processing-e2e-ipp-b-a60e80" not found 31m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-a60e80-7b8469d759 Created pod: payload-processing-e2e-ipp-b-a60e80-7b8469d759-5hv8m 31m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-a60e80 Scaled up replica set payload-processing-e2e-ipp-b-a60e80-7b8469d759 from 0 to 1 31m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-a60e80 Scaled up replica set payload-pre-processing-e2e-ipp-b-a60e80-5949b5b66c from 0 to 1 31m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-a60e80-5949b5b66c-h6x68 Add eth0 [10.132.0.223/23] from ovn-kubernetes 31m Normal Started pod/payload-pre-processing-e2e-ipp-b-a60e80-5949b5b66c-h6x68 Container started 31m Normal Created pod/payload-pre-processing-e2e-ipp-b-a60e80-5949b5b66c-h6x68 Container created 31m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-a60e80-5949b5b66c-h6x68 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/payload-processing-e2e-ipp-b-a60e80-7b8469d759-5hv8m Add eth0 [10.132.0.224/23] from ovn-kubernetes 31m Normal Started pod/payload-processing-e2e-ipp-b-a60e80-7b8469d759-5hv8m Container started 31m Normal Created pod/payload-processing-e2e-ipp-b-a60e80-7b8469d759-5hv8m Container created 31m Normal Pulled pod/payload-processing-e2e-ipp-b-a60e80-7b8469d759-5hv8m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Warning Unhealthy pod/e2e-ipp-a-e3a7dd-openshift-default-6c66b84dc9-v6hrm Startup probe failed: Get "http://10.133.1.40:15021/healthz/ready": dial tcp 10.133.1.40:15021: connect: connection refused 31m Warning ProbeError pod/e2e-ipp-a-e3a7dd-openshift-default-6c66b84dc9-v6hrm Startup probe error: Get "http://10.133.1.40:15021/healthz/ready": dial tcp 10.133.1.40:15021: connect: connection refused... 31m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-5983ea-openshift-default Scaled up replica set e2e-shared-a-w0-5983ea-openshift-default-849df9ccdf from 1 to 2 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-5983ea-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 31m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-5983ea-openshift-default-849df9ccdf Created pod: e2e-shared-a-w0-5983ea-openshift-default-849df9ccdf-ffzk6 31m Normal Pulled pod/e2e-shared-a-w0-5983ea-openshift-default-849df9ccdf-ffzk6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/e2e-shared-a-w0-5983ea-openshift-default-849df9ccdf-ffzk6 Add eth0 [10.133.1.47/23] from ovn-kubernetes 31m Normal Created pod/e2e-shared-a-w0-5983ea-openshift-default-849df9ccdf-ffzk6 Container created 31m Normal Started pod/e2e-shared-a-w0-5983ea-openshift-default-849df9ccdf-ffzk6 Container started 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-e3a7dd-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-e3a7dd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning ProbeError pod/e2e-shared-a-w0-5983ea-openshift-default-849df9ccdf-ffzk6 Startup probe error: Get "http://10.133.1.47:15021/healthz/ready": dial tcp 10.133.1.47:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-shared-a-w0-5983ea-openshift-default-849df9ccdf-ffzk6 Startup probe failed: Get "http://10.133.1.47:15021/healthz/ready": dial tcp 10.133.1.47:15021: connect: connection refused 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-e3a7dd-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-e3a7dd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-e3a7dd-fc4fb5599 Created pod: payload-pre-processing-e2e-ipp-a-e3a7dd-fc4fb5599-mg28w 30m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-e3a7dd-6b6b8b4cfd Created pod: payload-processing-e2e-ipp-a-e3a7dd-6b6b8b4cfd-9gg8g 30m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-e3a7dd Scaled up replica set payload-pre-processing-e2e-ipp-a-e3a7dd-fc4fb5599 from 0 to 1 30m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-e3a7dd Scaled up replica set payload-processing-e2e-ipp-a-e3a7dd-6b6b8b4cfd from 0 to 1 30m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-e3a7dd-fc4fb5599 Error creating: pods "payload-pre-processing-e2e-ipp-a-e3a7dd-fc4fb5599-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e3a7dd: serviceaccount "payload-processing-e2e-ipp-a-e3a7dd" not found 30m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-e3a7dd-6b6b8b4cfd Error creating: pods "payload-processing-e2e-ipp-a-e3a7dd-6b6b8b4cfd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e3a7dd: serviceaccount "payload-processing-e2e-ipp-a-e3a7dd" not found 30m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-e3a7dd-fc4fb5599-mg28w Add eth0 [10.134.0.141/23] from ovn-kubernetes 30m Normal AddedInterface pod/payload-processing-e2e-ipp-a-e3a7dd-6b6b8b4cfd-9gg8g Add eth0 [10.134.0.142/23] from ovn-kubernetes 30m Normal Pulling pod/payload-processing-e2e-ipp-a-e3a7dd-6b6b8b4cfd-9gg8g Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 30m Normal Pulling pod/payload-pre-processing-e2e-ipp-a-e3a7dd-fc4fb5599-mg28w Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 30m Normal Started pod/payload-pre-processing-e2e-ipp-a-e3a7dd-fc4fb5599-mg28w Container started 30m Normal Created pod/payload-pre-processing-e2e-ipp-a-e3a7dd-fc4fb5599-mg28w Container created 30m Normal Started pod/payload-processing-e2e-ipp-a-e3a7dd-6b6b8b4cfd-9gg8g Container started 30m Normal Created pod/payload-processing-e2e-ipp-a-e3a7dd-6b6b8b4cfd-9gg8g Container created 30m Normal Pulled pod/payload-processing-e2e-ipp-a-e3a7dd-6b6b8b4cfd-9gg8g Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.213s (1.213s including waiting). Image size: 166612641 bytes. 30m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-e3a7dd-fc4fb5599-mg28w Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.391s (1.391s including waiting). Image size: 166612641 bytes. 30m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-47qt2 Stopping container discovery 30m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-47qt2 30m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 30m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 30m Normal Created pod/e2e-shared-b-w3-e14e00-openshift-default-565cddf77c-7jg4k Container created 30m Normal Started pod/e2e-shared-b-w3-e14e00-openshift-default-565cddf77c-7jg4k Container started 30m Normal Pulled pod/e2e-shared-b-w3-e14e00-openshift-default-565cddf77c-7jg4k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Warning ProbeError pod/e2e-shared-b-w3-e14e00-openshift-default-565cddf77c-7jg4k Startup probe error: Get "http://10.132.0.220:15021/healthz/ready": dial tcp 10.132.0.220:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-shared-b-w3-e14e00-openshift-default-565cddf77c-7jg4k Startup probe failed: Get "http://10.132.0.220:15021/healthz/ready": dial tcp 10.132.0.220:15021: connect: connection refused 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-e14e00-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-e14e00-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-e14e00-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-e14e00-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Normal Killing pod/payload-processing-e2e-ipp-b-a60e80-7b8469d759-5hv8m Stopping container payload-processing 29m Normal Killing pod/payload-pre-processing-e2e-ipp-b-a60e80-5949b5b66c-h6x68 Stopping container payload-pre-processing 29m Normal Pulled pod/e2e-ipp-b-a60e80-openshift-default-c8f96989c-rvvlt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Created pod/e2e-ipp-b-a60e80-openshift-default-c8f96989c-rvvlt Container created 29m Normal Started pod/e2e-ipp-b-a60e80-openshift-default-c8f96989c-rvvlt Container started 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-a60e80-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-a60e80-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning ProbeError pod/e2e-ipp-b-a60e80-openshift-default-c8f96989c-rvvlt Startup probe error: Get "http://10.133.1.46:15021/healthz/ready": dial tcp 10.133.1.46:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-ipp-b-a60e80-openshift-default-c8f96989c-rvvlt Startup probe failed: Get "http://10.133.1.46:15021/healthz/ready": dial tcp 10.133.1.46:15021: connect: connection refused 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-a60e80 Scaled up replica set payload-pre-processing-e2e-ipp-b-a60e80-5db87c97f9 from 0 to 1 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-a60e80-5db87c97f9 Error creating: pods "payload-pre-processing-e2e-ipp-b-a60e80-5db87c97f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-a60e80: serviceaccount "payload-processing-e2e-ipp-b-a60e80" not found 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-a60e80-86d69656d4 Created pod: payload-processing-e2e-ipp-b-a60e80-86d69656d4-9r2ls 29m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-a60e80-86d69656d4 Error creating: pods "payload-processing-e2e-ipp-b-a60e80-86d69656d4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-a60e80: serviceaccount "payload-processing-e2e-ipp-b-a60e80" not found 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-a60e80-5db87c97f9 Created pod: payload-pre-processing-e2e-ipp-b-a60e80-5db87c97f9-7rjpw 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-a60e80 Scaled up replica set payload-processing-e2e-ipp-b-a60e80-86d69656d4 from 0 to 1 29m Normal AddedInterface pod/payload-processing-e2e-ipp-b-a60e80-86d69656d4-9r2ls Add eth0 [10.133.1.50/23] from ovn-kubernetes 29m Normal Pulling pod/payload-processing-e2e-ipp-b-a60e80-86d69656d4-9r2ls Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 29m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-a60e80-5db87c97f9-7rjpw Add eth0 [10.133.1.49/23] from ovn-kubernetes 29m Normal Pulling pod/payload-pre-processing-e2e-ipp-b-a60e80-5db87c97f9-7rjpw Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 29m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-a60e80-5db87c97f9-7rjpw Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.032s (2.032s including waiting). Image size: 166612641 bytes. 29m Normal Started pod/payload-pre-processing-e2e-ipp-b-a60e80-5db87c97f9-7rjpw Container started 29m Normal Created pod/payload-processing-e2e-ipp-b-a60e80-86d69656d4-9r2ls Container created 29m Normal Created pod/payload-pre-processing-e2e-ipp-b-a60e80-5db87c97f9-7rjpw Container created 29m Normal Started pod/payload-processing-e2e-ipp-b-a60e80-86d69656d4-9r2ls Container started 29m Normal Pulled pod/payload-processing-e2e-ipp-b-a60e80-86d69656d4-9r2ls Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.022s (2.022s including waiting). Image size: 166612641 bytes. 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-a60e80-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-a60e80-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-3cd77f-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 28m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-3cd77f-openshift-default Scaled up replica set e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c from 2 to 3 28m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c Created pod: e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-bmccc 28m Normal Created pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-bmccc Container created 28m Normal AddedInterface pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-bmccc Add eth0 [10.134.0.144/23] from ovn-kubernetes 28m Normal Pulled pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-bmccc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Started pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-bmccc Container started 28m Warning Unhealthy pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-bmccc Startup probe failed: Get "http://10.134.0.144:15021/healthz/ready": dial tcp 10.134.0.144:15021: connect: connection refused 28m Warning ProbeError pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-bmccc Startup probe error: Get "http://10.134.0.144:15021/healthz/ready": dial tcp 10.134.0.144:15021: connect: connection refused... 28m Normal AddedLabel gateway/e2e-ipp-a-f0a807 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-f0a807 28m Normal SuccessfulCreate replicaset/e2e-ipp-a-f0a807-openshift-default-66f955d779 Created pod: e2e-ipp-a-f0a807-openshift-default-66f955d779-qv6wr 28m Normal ScalingReplicaSet deployment/e2e-ipp-a-f0a807-openshift-default Scaled up replica set e2e-ipp-a-f0a807-openshift-default-66f955d779 from 0 to 1 28m Normal AddedInterface pod/e2e-ipp-a-f0a807-openshift-default-66f955d779-qv6wr Add eth0 [10.133.1.63/23] from ovn-kubernetes 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-f0a807-78f9946d59 Error creating: pods "payload-pre-processing-e2e-ipp-a-f0a807-78f9946d59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f0a807: serviceaccount "payload-processing-e2e-ipp-a-f0a807" not found 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-f0a807 Scaled up replica set payload-pre-processing-e2e-ipp-a-f0a807-78f9946d59 from 0 to 1 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-f0a807 Scaled up replica set payload-processing-e2e-ipp-a-f0a807-6968cf6bcc from 0 to 1 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-f0a807-6968cf6bcc Error creating: pods "payload-processing-e2e-ipp-a-f0a807-6968cf6bcc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f0a807: serviceaccount "payload-processing-e2e-ipp-a-f0a807" not found 27m Normal Started pod/payload-processing-e2e-ipp-a-f0a807-6968cf6bcc-xbgf8 Container started 27m Normal Created pod/payload-processing-e2e-ipp-a-f0a807-6968cf6bcc-xbgf8 Container created 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-f0a807-6968cf6bcc Created pod: payload-processing-e2e-ipp-a-f0a807-6968cf6bcc-xbgf8 27m Normal AddedInterface pod/payload-processing-e2e-ipp-a-f0a807-6968cf6bcc-xbgf8 Add eth0 [10.132.0.226/23] from ovn-kubernetes 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-f0a807-78f9946d59 Created pod: payload-pre-processing-e2e-ipp-a-f0a807-78f9946d59-tjxws 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-f0a807-78f9946d59-tjxws Add eth0 [10.133.1.64/23] from ovn-kubernetes 27m Normal Pulled pod/payload-processing-e2e-ipp-a-f0a807-6968cf6bcc-xbgf8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-f0a807-78f9946d59-tjxws Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal Created pod/payload-pre-processing-e2e-ipp-a-f0a807-78f9946d59-tjxws Container created 27m Normal Started pod/payload-pre-processing-e2e-ipp-a-f0a807-78f9946d59-tjxws Container started 27m Normal Killing pod/payload-processing-e2e-shared-a-w4-c511a3-5ddf74769-fztjg Stopping container payload-processing 27m Normal Killing pod/payload-pre-processing-e2e-shared-a-w4-c511a3-7c6f5989ff-r6zv2 Stopping container payload-pre-processing 27m Normal SuccessfulCreate replicaset/e2e-ipp-a-f88979-openshift-default-674df6d98f Created pod: e2e-ipp-a-f88979-openshift-default-674df6d98f-tr49g 27m Normal ScalingReplicaSet deployment/e2e-ipp-a-f88979-openshift-default Scaled up replica set e2e-ipp-a-f88979-openshift-default-674df6d98f from 0 to 1 27m Normal AddedLabel gateway/e2e-ipp-a-f88979 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-f88979 27m Normal AddedInterface pod/e2e-ipp-a-f88979-openshift-default-674df6d98f-tr49g Add eth0 [10.132.0.227/23] from ovn-kubernetes 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-f88979 Scaled up replica set payload-processing-e2e-ipp-a-f88979-7dd56f4c95 from 0 to 1 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-f88979-7dd56f4c95 Created pod: payload-processing-e2e-ipp-a-f88979-7dd56f4c95-jnwkh 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-f88979-7dd56f4c95 Error creating: pods "payload-processing-e2e-ipp-a-f88979-7dd56f4c95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f88979: serviceaccount "payload-processing-e2e-ipp-a-f88979" not found 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-f88979-844f98df9f Error creating: pods "payload-pre-processing-e2e-ipp-a-f88979-844f98df9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f88979: serviceaccount "payload-processing-e2e-ipp-a-f88979" not found 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-f88979-844f98df9f Created pod: payload-pre-processing-e2e-ipp-a-f88979-844f98df9f-d7zsv 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-f88979 Scaled up replica set payload-pre-processing-e2e-ipp-a-f88979-844f98df9f from 0 to 1 27m Normal Started pod/payload-processing-e2e-ipp-a-f88979-7dd56f4c95-jnwkh Container started 27m Normal Started pod/payload-pre-processing-e2e-ipp-a-f88979-844f98df9f-d7zsv Container started 27m Normal Created pod/payload-pre-processing-e2e-ipp-a-f88979-844f98df9f-d7zsv Container created 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-f88979-844f98df9f-d7zsv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/payload-processing-e2e-ipp-a-f88979-7dd56f4c95-jnwkh Add eth0 [10.132.0.228/23] from ovn-kubernetes 27m Normal Pulled pod/payload-processing-e2e-ipp-a-f88979-7dd56f4c95-jnwkh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal Created pod/payload-processing-e2e-ipp-a-f88979-7dd56f4c95-jnwkh Container created 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-f88979-844f98df9f-d7zsv Add eth0 [10.132.0.229/23] from ovn-kubernetes 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-0ff5b0-openshift-default New size: 1; reason: All metrics below target 27m Normal SuccessfulDelete replicaset/e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5 Deleted pod: e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5-tz9sg 27m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-0ff5b0-openshift-default Scaled down replica set e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5 from 2 to 1 27m Normal Killing pod/e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5-tz9sg Stopping container istio-proxy 27m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w4-c511a3-7c6f5989ff-r6zv2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Warning Unhealthy pod/payload-processing-e2e-shared-a-w4-c511a3-5ddf74769-fztjg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-d9f1c2-openshift-default New size: 1; reason: All metrics below target 27m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-d9f1c2-openshift-default Scaled down replica set e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447 from 2 to 1 27m Normal Killing pod/e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447-hs292 Stopping container istio-proxy 27m Normal SuccessfulDelete replicaset/e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447 Deleted pod: e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447-hs292 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-80ecb3-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 27m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-80ecb3-openshift-default Scaled up replica set e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949 from 1 to 2 27m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949 Created pod: e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949-dr69x 27m Normal Created pod/e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949-dr69x Container created 27m Normal AddedInterface pod/e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949-dr69x Add eth0 [10.133.1.66/23] from ovn-kubernetes 27m Normal Started pod/e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949-dr69x Container started 27m Normal Pulled pod/e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949-dr69x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Warning Unhealthy pod/e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949-dr69x Startup probe failed: Get "http://10.133.1.66:15021/healthz/ready": dial tcp 10.133.1.66:15021: connect: connection refused 27m Warning ProbeError pod/e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949-dr69x Startup probe error: Get "http://10.133.1.66:15021/healthz/ready": dial tcp 10.133.1.66:15021: connect: connection refused... 27m Normal Killing pod/e2e-shared-a-w4-c511a3-openshift-default-59bc6d8b7-9m4nr Stopping container istio-proxy 27m Warning Unhealthy pod/e2e-shared-a-w4-c511a3-openshift-default-59bc6d8b7-9m4nr Readiness probe failed: HTTP probe failed with statuscode: 503 27m Warning ProbeError pod/e2e-shared-a-w4-c511a3-openshift-default-59bc6d8b7-9m4nr Readiness probe error: HTTP probe failed with statuscode: 503... 26m Normal SuccessfulDelete replicaset/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc Deleted pod: e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-x4lb9 26m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-cdc122-openshift-default Scaled down replica set e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc from 4 to 3 26m Normal Killing pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-x4lb9 Stopping container istio-proxy 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-cdc122-openshift-default New size: 3; reason: All metrics below target 26m Normal Killing pod/payload-processing-e2e-shared-b-w4-b03b33-7c5fcd4d57-g2t4v Stopping container payload-processing 26m Normal Killing pod/payload-pre-processing-e2e-shared-b-w4-b03b33-cd67bd8f5-dft76 Stopping container payload-pre-processing 26m Normal Created pod/e2e-ipp-a-f0a807-openshift-default-66f955d779-qv6wr Container created 26m Normal Pulled pod/e2e-ipp-a-f0a807-openshift-default-66f955d779-qv6wr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Started pod/e2e-ipp-a-f0a807-openshift-default-66f955d779-qv6wr Container started 26m Warning Unhealthy pod/e2e-ipp-a-f0a807-openshift-default-66f955d779-qv6wr Startup probe failed: Get "http://10.133.1.63:15021/healthz/ready": dial tcp 10.133.1.63:15021: connect: connection refused 26m Warning ProbeError pod/e2e-ipp-a-f0a807-openshift-default-66f955d779-qv6wr Startup probe error: Get "http://10.133.1.63:15021/healthz/ready": dial tcp 10.133.1.63:15021: connect: connection refused... 26m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w4-b03b33-cd67bd8f5-dft76 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Warning Unhealthy pod/payload-processing-e2e-shared-b-w4-b03b33-7c5fcd4d57-g2t4v Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Normal Killing pod/payload-pre-processing-e2e-ipp-a-f0a807-78f9946d59-tjxws Stopping container payload-pre-processing 26m Normal Killing pod/payload-processing-e2e-ipp-a-f0a807-6968cf6bcc-xbgf8 Stopping container payload-processing 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-f0a807-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-f0a807-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Normal Killing pod/payload-pre-processing-e2e-ipp-a-f88979-844f98df9f-d7zsv Stopping container payload-pre-processing 26m Normal Killing pod/payload-processing-e2e-ipp-a-f88979-7dd56f4c95-jnwkh Stopping container payload-processing 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-f0a807-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-f0a807-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Normal Killing pod/e2e-shared-b-w4-b03b33-openshift-default-79f6954695-882qd Stopping container istio-proxy 25m Normal Killing pod/payload-processing-e2e-worker-w4-0ad7c5-7b6cc5f5c5-8c2qz Stopping container payload-processing 25m Normal Killing pod/payload-pre-processing-e2e-worker-w4-0ad7c5-dffdb7775-kt9zp Stopping container payload-pre-processing 25m Normal Pulled pod/e2e-ipp-a-f88979-openshift-default-674df6d98f-tr49g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Created pod/e2e-ipp-a-f88979-openshift-default-674df6d98f-tr49g Container created 25m Normal Started pod/e2e-ipp-a-f88979-openshift-default-674df6d98f-tr49g Container started 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-f88979-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-f88979-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-f0a807 Scaled up replica set payload-pre-processing-e2e-ipp-a-f0a807-6b45d8f8f9 from 0 to 1 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-f0a807-6b45d8f8f9 Error creating: pods "payload-pre-processing-e2e-ipp-a-f0a807-6b45d8f8f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f0a807: serviceaccount "payload-processing-e2e-ipp-a-f0a807" not found 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-f0a807-6b45d8f8f9 Created pod: payload-pre-processing-e2e-ipp-a-f0a807-6b45d8f8f9-x5x2n 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-f0a807-69fbbcbd9d Created pod: payload-processing-e2e-ipp-a-f0a807-69fbbcbd9d-9r72m 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-f0a807 Scaled up replica set payload-processing-e2e-ipp-a-f0a807-69fbbcbd9d from 0 to 1 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-f0a807-69fbbcbd9d Error creating: pods "payload-processing-e2e-ipp-a-f0a807-69fbbcbd9d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f0a807: serviceaccount "payload-processing-e2e-ipp-a-f0a807" not found 25m Normal AddedInterface pod/payload-processing-e2e-ipp-a-f0a807-69fbbcbd9d-9r72m Add eth0 [10.132.0.235/23] from ovn-kubernetes 25m Normal Pulling pod/payload-processing-e2e-ipp-a-f0a807-69fbbcbd9d-9r72m Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-f0a807-6b45d8f8f9-x5x2n Add eth0 [10.132.0.234/23] from ovn-kubernetes 25m Normal Pulling pod/payload-pre-processing-e2e-ipp-a-f0a807-6b45d8f8f9-x5x2n Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 25m Warning Unhealthy pod/e2e-ipp-a-f88979-openshift-default-674df6d98f-tr49g Startup probe failed: Get "http://10.132.0.227:15021/healthz/ready": dial tcp 10.132.0.227:15021: connect: connection refused 25m Warning ProbeError pod/e2e-ipp-a-f88979-openshift-default-674df6d98f-tr49g Startup probe error: Get "http://10.132.0.227:15021/healthz/ready": dial tcp 10.132.0.227:15021: connect: connection refused... 25m Normal Created pod/payload-processing-e2e-ipp-a-f0a807-69fbbcbd9d-9r72m Container created 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-f0a807-6b45d8f8f9-x5x2n Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.659s (2.659s including waiting). Image size: 166612641 bytes. 25m Normal Pulled pod/payload-processing-e2e-ipp-a-f0a807-69fbbcbd9d-9r72m Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.572s (2.572s including waiting). Image size: 166612641 bytes. 25m Normal Started pod/payload-pre-processing-e2e-ipp-a-f0a807-6b45d8f8f9-x5x2n Container started 25m Normal Started pod/payload-processing-e2e-ipp-a-f0a807-69fbbcbd9d-9r72m Container started 25m Normal Created pod/payload-pre-processing-e2e-ipp-a-f0a807-6b45d8f8f9-x5x2n Container created 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf Created pod: e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-4qscf 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-510f80-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 25m Normal AddedInterface pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-4qscf Add eth0 [10.132.0.237/23] from ovn-kubernetes 25m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-510f80-openshift-default Scaled up replica set e2e-shared-b-w5-510f80-openshift-default-6465ccddcf from 1 to 2 25m Normal Started pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-4qscf Container started 25m Normal Created pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-4qscf Container created 25m Normal Pulled pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-4qscf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Warning ProbeError pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-4qscf Startup probe error: Get "http://10.132.0.237:15021/healthz/ready": dial tcp 10.132.0.237:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-4qscf Startup probe failed: Get "http://10.132.0.237:15021/healthz/ready": dial tcp 10.132.0.237:15021: connect: connection refused 25m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w4-0ad7c5-dffdb7775-kt9zp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Normal AddedLabel gateway/e2e-ipp-b-76d54a Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-76d54a 25m Normal ScalingReplicaSet deployment/e2e-ipp-b-76d54a-openshift-default Scaled up replica set e2e-ipp-b-76d54a-openshift-default-6bcffbc9bc from 0 to 1 25m Normal SuccessfulCreate replicaset/e2e-ipp-b-76d54a-openshift-default-6bcffbc9bc Created pod: e2e-ipp-b-76d54a-openshift-default-6bcffbc9bc-8d4d2 25m Warning Unhealthy pod/payload-processing-e2e-worker-w4-0ad7c5-7b6cc5f5c5-8c2qz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-f88979 Scaled up replica set payload-pre-processing-e2e-ipp-a-f88979-669bd98b7f from 0 to 1 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-f88979-669bd98b7f Created pod: payload-pre-processing-e2e-ipp-a-f88979-669bd98b7f-9lsrp 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-f88979-74fd6f8bd Error creating: pods "payload-processing-e2e-ipp-a-f88979-74fd6f8bd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f88979: serviceaccount "payload-processing-e2e-ipp-a-f88979" not found 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-f88979-74fd6f8bd Created pod: payload-processing-e2e-ipp-a-f88979-74fd6f8bd-vfz9j 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-f88979 Scaled up replica set payload-processing-e2e-ipp-a-f88979-74fd6f8bd from 0 to 1 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-f88979-669bd98b7f Error creating: pods "payload-pre-processing-e2e-ipp-a-f88979-669bd98b7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f88979: serviceaccount "payload-processing-e2e-ipp-a-f88979" not found 25m Normal AddedInterface pod/e2e-ipp-b-76d54a-openshift-default-6bcffbc9bc-8d4d2 Add eth0 [10.132.0.238/23] from ovn-kubernetes 25m Normal Started pod/payload-pre-processing-e2e-ipp-a-f88979-669bd98b7f-9lsrp Container started 25m Normal Started pod/payload-processing-e2e-ipp-a-f88979-74fd6f8bd-vfz9j Container started 25m Normal AddedInterface pod/payload-processing-e2e-ipp-a-f88979-74fd6f8bd-vfz9j Add eth0 [10.133.1.77/23] from ovn-kubernetes 25m Normal Pulled pod/payload-processing-e2e-ipp-a-f88979-74fd6f8bd-vfz9j Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal Created pod/payload-pre-processing-e2e-ipp-a-f88979-669bd98b7f-9lsrp Container created 25m Normal Created pod/payload-processing-e2e-ipp-a-f88979-74fd6f8bd-vfz9j Container created 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-f88979-669bd98b7f-9lsrp Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-f88979-669bd98b7f-9lsrp Add eth0 [10.133.1.76/23] from ovn-kubernetes 25m Normal SuccessfulCreate replicaset/e2e-ipp-a-f94561-openshift-default-66c966685c Created pod: e2e-ipp-a-f94561-openshift-default-66c966685c-zcxmh 25m Normal ScalingReplicaSet deployment/e2e-ipp-a-f94561-openshift-default Scaled up replica set e2e-ipp-a-f94561-openshift-default-66c966685c from 0 to 1 25m Normal AddedLabel gateway/e2e-ipp-a-f94561 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-f94561 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-76d54a-d97b6f8b7 Created pod: payload-pre-processing-e2e-ipp-b-76d54a-d97b6f8b7-p8h4c 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-76d54a-d97b6f8b7 Error creating: pods "payload-pre-processing-e2e-ipp-b-76d54a-d97b6f8b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-76d54a: serviceaccount "payload-processing-e2e-ipp-b-76d54a" not found 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-76d54a Scaled up replica set payload-processing-e2e-ipp-b-76d54a-5c57cb5785 from 0 to 1 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-76d54a-5c57cb5785 Error creating: pods "payload-processing-e2e-ipp-b-76d54a-5c57cb5785-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-76d54a: serviceaccount "payload-processing-e2e-ipp-b-76d54a" not found 25m Normal AddedInterface pod/e2e-ipp-a-f94561-openshift-default-66c966685c-zcxmh Add eth0 [10.132.0.239/23] from ovn-kubernetes 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-76d54a Scaled up replica set payload-pre-processing-e2e-ipp-b-76d54a-d97b6f8b7 from 0 to 1 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-76d54a-5c57cb5785 Created pod: payload-processing-e2e-ipp-b-76d54a-5c57cb5785-lvdp9 25m Normal AddedInterface pod/payload-processing-e2e-ipp-b-76d54a-5c57cb5785-lvdp9 Add eth0 [10.132.0.240/23] from ovn-kubernetes 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-76d54a-d97b6f8b7-p8h4c Add eth0 [10.132.0.241/23] from ovn-kubernetes 25m Normal Created pod/payload-processing-e2e-ipp-b-76d54a-5c57cb5785-lvdp9 Container created 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-76d54a-d97b6f8b7-p8h4c Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal Started pod/payload-pre-processing-e2e-ipp-b-76d54a-d97b6f8b7-p8h4c Container started 25m Normal Pulled pod/payload-processing-e2e-ipp-b-76d54a-5c57cb5785-lvdp9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal Started pod/payload-processing-e2e-ipp-b-76d54a-5c57cb5785-lvdp9 Container started 25m Normal Created pod/payload-pre-processing-e2e-ipp-b-76d54a-d97b6f8b7-p8h4c Container created 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-f88979-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-f88979-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-f94561-5c684bc9 Error creating: pods "payload-pre-processing-e2e-ipp-a-f94561-5c684bc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f94561: serviceaccount "payload-processing-e2e-ipp-a-f94561" not found 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-f94561-5c684bc9 Created pod: payload-pre-processing-e2e-ipp-a-f94561-5c684bc9-8ccfn 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-f94561 Scaled up replica set payload-pre-processing-e2e-ipp-a-f94561-5c684bc9 from 0 to 1 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-f94561-5ddf9d6659 Created pod: payload-processing-e2e-ipp-a-f94561-5ddf9d6659-rntqt 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-f94561-5ddf9d6659 Error creating: pods "payload-processing-e2e-ipp-a-f94561-5ddf9d6659-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f94561: serviceaccount "payload-processing-e2e-ipp-a-f94561" not found 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-f94561 Scaled up replica set payload-processing-e2e-ipp-a-f94561-5ddf9d6659 from 0 to 1 25m Normal Started pod/payload-processing-e2e-ipp-a-f94561-5ddf9d6659-rntqt Container started 25m Normal Created pod/payload-pre-processing-e2e-ipp-a-f94561-5c684bc9-8ccfn Container created 25m Normal Pulled pod/payload-processing-e2e-ipp-a-f94561-5ddf9d6659-rntqt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-f94561-5c684bc9-8ccfn Add eth0 [10.133.1.78/23] from ovn-kubernetes 25m Normal AddedInterface pod/payload-processing-e2e-ipp-a-f94561-5ddf9d6659-rntqt Add eth0 [10.132.0.242/23] from ovn-kubernetes 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-f94561-5c684bc9-8ccfn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal Created pod/payload-processing-e2e-ipp-a-f94561-5ddf9d6659-rntqt Container created 25m Normal Started pod/payload-pre-processing-e2e-ipp-a-f94561-5c684bc9-8ccfn Container started 25m Normal AddedLabel gateway/e2e-ipp-b-cdfa86 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-cdfa86 25m Normal ScalingReplicaSet deployment/e2e-ipp-b-cdfa86-openshift-default Scaled up replica set e2e-ipp-b-cdfa86-openshift-default-77bdfd8797 from 0 to 1 25m Normal SuccessfulCreate replicaset/e2e-ipp-b-cdfa86-openshift-default-77bdfd8797 Created pod: e2e-ipp-b-cdfa86-openshift-default-77bdfd8797-52tjw 25m Normal AddedInterface pod/e2e-ipp-b-cdfa86-openshift-default-77bdfd8797-52tjw Add eth0 [10.133.1.79/23] from ovn-kubernetes 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-cdfa86-b49f58999 Created pod: payload-pre-processing-e2e-ipp-b-cdfa86-b49f58999-qjqrs 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-cdfa86-8f44c5dc9 Created pod: payload-processing-e2e-ipp-b-cdfa86-8f44c5dc9-jgtzp 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-cdfa86-8f44c5dc9 Error creating: pods "payload-processing-e2e-ipp-b-cdfa86-8f44c5dc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-cdfa86: serviceaccount "payload-processing-e2e-ipp-b-cdfa86" not found 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-cdfa86-b49f58999 Error creating: pods "payload-pre-processing-e2e-ipp-b-cdfa86-b49f58999-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-cdfa86: serviceaccount "payload-processing-e2e-ipp-b-cdfa86" not found 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-cdfa86 Scaled up replica set payload-pre-processing-e2e-ipp-b-cdfa86-b49f58999 from 0 to 1 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-cdfa86 Scaled up replica set payload-processing-e2e-ipp-b-cdfa86-8f44c5dc9 from 0 to 1 25m Normal Pulled pod/payload-processing-e2e-ipp-b-cdfa86-8f44c5dc9-jgtzp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal Started pod/payload-processing-e2e-ipp-b-cdfa86-8f44c5dc9-jgtzp Container started 25m Normal Created pod/payload-processing-e2e-ipp-b-cdfa86-8f44c5dc9-jgtzp Container created 25m Normal AddedInterface pod/payload-processing-e2e-ipp-b-cdfa86-8f44c5dc9-jgtzp Add eth0 [10.132.0.244/23] from ovn-kubernetes 25m Normal Created pod/payload-pre-processing-e2e-ipp-b-cdfa86-b49f58999-qjqrs Container created 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-cdfa86-b49f58999-qjqrs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-cdfa86-b49f58999-qjqrs Add eth0 [10.133.1.80/23] from ovn-kubernetes 25m Normal Started pod/payload-pre-processing-e2e-ipp-b-cdfa86-b49f58999-qjqrs Container started 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-cdfa86-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-cdfa86-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-f94561-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-f94561-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Normal Killing pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-72ws5 Stopping container istio-proxy 24m Warning ProbeError pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-72ws5 Readiness probe error: HTTP probe failed with statuscode: 503... 24m Warning Unhealthy pod/e2e-worker-w4-0ad7c5-openshift-default-788b484d5c-72ws5 Readiness probe failed: HTTP probe failed with statuscode: 503 24m Warning ProbeError pod/e2e-ipp-b-76d54a-openshift-default-6bcffbc9bc-8d4d2 Readiness probe error: Get "http://10.132.0.238:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Warning Unhealthy pod/e2e-ipp-b-76d54a-openshift-default-6bcffbc9bc-8d4d2 Readiness probe failed: Get "http://10.132.0.238:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Normal Pulled pod/e2e-ipp-b-76d54a-openshift-default-6bcffbc9bc-8d4d2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Created pod/e2e-ipp-b-76d54a-openshift-default-6bcffbc9bc-8d4d2 Container created 24m Normal Started pod/e2e-ipp-b-76d54a-openshift-default-6bcffbc9bc-8d4d2 Container started 23m Normal Killing pod/payload-processing-e2e-ipp-a-e3a7dd-6b6b8b4cfd-9gg8g Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-ipp-a-e3a7dd-fc4fb5599-mg28w Stopping container payload-pre-processing 23m Warning Unhealthy pod/e2e-ipp-b-76d54a-openshift-default-6bcffbc9bc-8d4d2 Startup probe failed: Get "http://10.132.0.238:15021/healthz/ready": dial tcp 10.132.0.238:15021: connect: connection refused 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-76d54a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Warning ProbeError pod/e2e-ipp-b-76d54a-openshift-default-6bcffbc9bc-8d4d2 Startup probe error: Get "http://10.132.0.238:15021/healthz/ready": dial tcp 10.132.0.238:15021: connect: connection refused... 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-76d54a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Normal Killing pod/payload-processing-e2e-ipp-b-76d54a-5c57cb5785-lvdp9 Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-ipp-b-76d54a-d97b6f8b7-p8h4c Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-ipp-a-f94561-5ddf9d6659-rntqt Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-ipp-a-f94561-5c684bc9-8ccfn Stopping container payload-pre-processing 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-76d54a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-76d54a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Normal Killing pod/payload-processing-e2e-ipp-b-cdfa86-8f44c5dc9-jgtzp Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-ipp-b-cdfa86-b49f58999-qjqrs Stopping container payload-pre-processing 23m Warning ProbeError pod/e2e-ipp-b-cdfa86-openshift-default-77bdfd8797-52tjw Readiness probe error: Get "http://10.133.1.79:15021/healthz/ready": read tcp 10.133.0.2:38370->10.133.1.79:15021: read: connection reset by peer... 23m Normal Pulled pod/e2e-ipp-a-f94561-openshift-default-66c966685c-zcxmh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Started pod/e2e-ipp-a-f94561-openshift-default-66c966685c-zcxmh Container started 23m Normal Started pod/e2e-ipp-b-cdfa86-openshift-default-77bdfd8797-52tjw Container started 23m Normal Created pod/e2e-ipp-b-cdfa86-openshift-default-77bdfd8797-52tjw Container created 23m Normal Pulled pod/e2e-ipp-b-cdfa86-openshift-default-77bdfd8797-52tjw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Warning ProbeError pod/e2e-ipp-a-f94561-openshift-default-66c966685c-zcxmh Readiness probe error: Get "http://10.132.0.239:15021/healthz/ready": read tcp 10.132.0.2:60336->10.132.0.239:15021: read: connection reset by peer... 23m Normal Created pod/e2e-ipp-a-f94561-openshift-default-66c966685c-zcxmh Container created 23m Warning Unhealthy pod/e2e-ipp-b-cdfa86-openshift-default-77bdfd8797-52tjw Readiness probe failed: Get "http://10.133.1.79:15021/healthz/ready": read tcp 10.133.0.2:38370->10.133.1.79:15021: read: connection reset by peer 23m Warning Unhealthy pod/e2e-ipp-a-f94561-openshift-default-66c966685c-zcxmh Readiness probe failed: Get "http://10.132.0.239:15021/healthz/ready": read tcp 10.132.0.2:60336->10.132.0.239:15021: read: connection reset by peer 23m Warning ProbeError pod/e2e-ipp-b-cdfa86-openshift-default-77bdfd8797-52tjw Startup probe error: Get "http://10.133.1.79:15021/healthz/ready": dial tcp 10.133.1.79:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-ipp-b-cdfa86-openshift-default-77bdfd8797-52tjw Startup probe failed: Get "http://10.133.1.79:15021/healthz/ready": dial tcp 10.133.1.79:15021: connect: connection refused 23m Warning ProbeError pod/e2e-ipp-a-f94561-openshift-default-66c966685c-zcxmh Startup probe error: Get "http://10.132.0.239:15021/healthz/ready": dial tcp 10.132.0.239:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-ipp-a-f94561-openshift-default-66c966685c-zcxmh Startup probe failed: Get "http://10.132.0.239:15021/healthz/ready": dial tcp 10.132.0.239:15021: connect: connection refused 23m Normal Killing pod/e2e-ipp-a-e3a7dd-openshift-default-6c66b84dc9-v6hrm Stopping container istio-proxy 23m Warning Unhealthy pod/e2e-ipp-a-e3a7dd-openshift-default-6c66b84dc9-v6hrm Readiness probe failed: HTTP probe failed with statuscode: 503 23m Warning ProbeError pod/e2e-ipp-a-e3a7dd-openshift-default-6c66b84dc9-v6hrm Readiness probe error: HTTP probe failed with statuscode: 503... 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-76d54a Scaled up replica set payload-pre-processing-e2e-ipp-b-76d54a-fcfc75c9f from 0 to 1 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-76d54a Scaled up replica set payload-processing-e2e-ipp-b-76d54a-84746b7f6b from 0 to 1 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-76d54a-fcfc75c9f Error creating: pods "payload-pre-processing-e2e-ipp-b-76d54a-fcfc75c9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-76d54a: serviceaccount "payload-processing-e2e-ipp-b-76d54a" not found 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-76d54a-84746b7f6b Created pod: payload-processing-e2e-ipp-b-76d54a-84746b7f6b-br8pt 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-76d54a-84746b7f6b Error creating: pods "payload-processing-e2e-ipp-b-76d54a-84746b7f6b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-76d54a: serviceaccount "payload-processing-e2e-ipp-b-76d54a" not found 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-76d54a-fcfc75c9f Created pod: payload-pre-processing-e2e-ipp-b-76d54a-fcfc75c9f-lvgx2 22m Normal Created pod/payload-processing-e2e-ipp-b-76d54a-84746b7f6b-br8pt Container created 22m Normal Created pod/payload-pre-processing-e2e-ipp-b-76d54a-fcfc75c9f-lvgx2 Container created 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-76d54a-fcfc75c9f-lvgx2 Add eth0 [10.133.1.102/23] from ovn-kubernetes 22m Normal Started pod/payload-pre-processing-e2e-ipp-b-76d54a-fcfc75c9f-lvgx2 Container started 22m Normal Started pod/payload-processing-e2e-ipp-b-76d54a-84746b7f6b-br8pt Container started 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-76d54a-fcfc75c9f-lvgx2 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/payload-processing-e2e-ipp-b-76d54a-84746b7f6b-br8pt Add eth0 [10.133.1.103/23] from ovn-kubernetes 22m Normal Pulled pod/payload-processing-e2e-ipp-b-76d54a-84746b7f6b-br8pt Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-f94561 Scaled up replica set payload-pre-processing-e2e-ipp-a-f94561-66b74889b7 from 0 to 1 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-f94561 Scaled up replica set payload-processing-e2e-ipp-a-f94561-b76dcb6f6 from 0 to 1 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-f94561-b76dcb6f6 Error creating: pods "payload-processing-e2e-ipp-a-f94561-b76dcb6f6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f94561: serviceaccount "payload-processing-e2e-ipp-a-f94561" not found 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-f94561-b76dcb6f6 Created pod: payload-processing-e2e-ipp-a-f94561-b76dcb6f6-jvts2 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-f94561-66b74889b7 Error creating: pods "payload-pre-processing-e2e-ipp-a-f94561-66b74889b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f94561: serviceaccount "payload-processing-e2e-ipp-a-f94561" not found 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-f94561-66b74889b7 Created pod: payload-pre-processing-e2e-ipp-a-f94561-66b74889b7-j6lk5 22m Normal Started pod/payload-processing-e2e-ipp-a-f94561-b76dcb6f6-jvts2 Container started 22m Normal Started pod/payload-pre-processing-e2e-ipp-a-f94561-66b74889b7-j6lk5 Container started 22m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-cdfa86-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-f94561-66b74889b7-j6lk5 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-cdfa86-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Normal Created pod/payload-pre-processing-e2e-ipp-a-f94561-66b74889b7-j6lk5 Container created 22m Normal Created pod/payload-processing-e2e-ipp-a-f94561-b76dcb6f6-jvts2 Container created 22m Normal Pulled pod/payload-processing-e2e-ipp-a-f94561-b76dcb6f6-jvts2 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/payload-processing-e2e-ipp-a-f94561-b76dcb6f6-jvts2 Add eth0 [10.132.0.253/23] from ovn-kubernetes 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-f94561-66b74889b7-j6lk5 Add eth0 [10.133.1.104/23] from ovn-kubernetes 22m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-f94561-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-f94561-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-cdfa86 Scaled up replica set payload-processing-e2e-ipp-b-cdfa86-79959f8c88 from 0 to 1 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-cdfa86-5799cbf495 Error creating: pods "payload-pre-processing-e2e-ipp-b-cdfa86-5799cbf495-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-cdfa86: serviceaccount "payload-processing-e2e-ipp-b-cdfa86" not found 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-cdfa86-5799cbf495 Created pod: payload-pre-processing-e2e-ipp-b-cdfa86-5799cbf495-h927d 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-cdfa86-79959f8c88 Created pod: payload-processing-e2e-ipp-b-cdfa86-79959f8c88-hcpjf 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-cdfa86 Scaled up replica set payload-pre-processing-e2e-ipp-b-cdfa86-5799cbf495 from 0 to 1 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-cdfa86-79959f8c88 Error creating: pods "payload-processing-e2e-ipp-b-cdfa86-79959f8c88-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-cdfa86: serviceaccount "payload-processing-e2e-ipp-b-cdfa86" not found 22m Normal Started pod/payload-pre-processing-e2e-ipp-b-cdfa86-5799cbf495-h927d Container started 22m Normal Pulled pod/payload-processing-e2e-ipp-b-cdfa86-79959f8c88-hcpjf Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal Created pod/payload-processing-e2e-ipp-b-cdfa86-79959f8c88-hcpjf Container created 22m Normal Created pod/payload-pre-processing-e2e-ipp-b-cdfa86-5799cbf495-h927d Container created 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-cdfa86-5799cbf495-h927d Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-cdfa86-5799cbf495-h927d Add eth0 [10.133.1.106/23] from ovn-kubernetes 22m Normal Started pod/payload-processing-e2e-ipp-b-cdfa86-79959f8c88-hcpjf Container started 22m Normal AddedInterface pod/payload-processing-e2e-ipp-b-cdfa86-79959f8c88-hcpjf Add eth0 [10.132.0.254/23] from ovn-kubernetes 22m Normal SuccessfulCreate replicaset/e2e-ipp-b-bff950-openshift-default-5dd888d8fc Created pod: e2e-ipp-b-bff950-openshift-default-5dd888d8fc-46j6j 22m Normal ScalingReplicaSet deployment/e2e-ipp-b-bff950-openshift-default Scaled up replica set e2e-ipp-b-bff950-openshift-default-5dd888d8fc from 0 to 1 22m Normal AddedLabel gateway/e2e-ipp-b-bff950 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-bff950 22m Normal AddedInterface pod/e2e-ipp-b-bff950-openshift-default-5dd888d8fc-46j6j Add eth0 [10.133.1.107/23] from ovn-kubernetes 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-bff950-d477c7687 Created pod: payload-pre-processing-e2e-ipp-b-bff950-d477c7687-vm8sg 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-bff950 Scaled up replica set payload-pre-processing-e2e-ipp-b-bff950-d477c7687 from 0 to 1 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-bff950-746d89d77f Created pod: payload-processing-e2e-ipp-b-bff950-746d89d77f-w4559 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-bff950 Scaled up replica set payload-processing-e2e-ipp-b-bff950-746d89d77f from 0 to 1 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-bff950-746d89d77f Error creating: pods "payload-processing-e2e-ipp-b-bff950-746d89d77f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-bff950: serviceaccount "payload-processing-e2e-ipp-b-bff950" not found 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-bff950-d477c7687 Error creating: pods "payload-pre-processing-e2e-ipp-b-bff950-d477c7687-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-bff950: serviceaccount "payload-processing-e2e-ipp-b-bff950" not found 22m Normal AddedInterface pod/payload-processing-e2e-ipp-b-bff950-746d89d77f-w4559 Add eth0 [10.132.0.255/23] from ovn-kubernetes 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-bff950-d477c7687-vm8sg Add eth0 [10.132.1.0/23] from ovn-kubernetes 22m Normal Created pod/payload-pre-processing-e2e-ipp-b-bff950-d477c7687-vm8sg Container created 22m Normal Pulled pod/payload-processing-e2e-ipp-b-bff950-746d89d77f-w4559 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal Started pod/payload-processing-e2e-ipp-b-bff950-746d89d77f-w4559 Container started 22m Normal Created pod/payload-processing-e2e-ipp-b-bff950-746d89d77f-w4559 Container created 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-bff950-d477c7687-vm8sg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal Started pod/payload-pre-processing-e2e-ipp-b-bff950-d477c7687-vm8sg Container started 22m Normal Killing pod/payload-processing-e2e-shared-a-w5-6f2eba-5576597779-g8wwj Stopping container payload-processing 22m Normal Killing pod/payload-pre-processing-e2e-shared-a-w5-6f2eba-dc7c6b979-rb48s Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-ipp-b-a60e80-86d69656d4-9r2ls Stopping container payload-processing 22m Normal Killing pod/payload-pre-processing-e2e-ipp-b-a60e80-5db87c97f9-7rjpw Stopping container payload-pre-processing 22m Normal Killing pod/e2e-ipp-b-a60e80-openshift-default-c8f96989c-rvvlt Stopping container istio-proxy 22m Warning Unhealthy pod/e2e-ipp-b-a60e80-openshift-default-c8f96989c-rvvlt Readiness probe failed: HTTP probe failed with statuscode: 503 22m Warning ProbeError pod/e2e-ipp-b-a60e80-openshift-default-c8f96989c-rvvlt Readiness probe error: HTTP probe failed with statuscode: 503... 22m Normal SuccessfulDelete replicaset/e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949 Deleted pod: e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949-dr69x 22m Normal Killing pod/e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949-dr69x Stopping container istio-proxy 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-80ecb3-openshift-default New size: 1; reason: All metrics below target 22m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-80ecb3-openshift-default Scaled down replica set e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949 from 2 to 1 21m Normal Killing pod/payload-processing-e2e-shared-a-w6-0ff5b0-7c4b7489d7-xlk52 Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-shared-a-w6-0ff5b0-6f4dc7d5f-pmqnt Stopping container payload-pre-processing 21m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w5-6f2eba-dc7c6b979-rb48s Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Warning Unhealthy pod/payload-processing-e2e-shared-a-w5-6f2eba-5576597779-g8wwj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Normal Killing pod/e2e-shared-a-w6-0ff5b0-openshift-default-ff46db8d5-6m98h Stopping container istio-proxy 21m Normal Killing pod/payload-processing-e2e-ipp-a-f0a807-69fbbcbd9d-9r72m Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-ipp-a-f0a807-6b45d8f8f9-x5x2n Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-shared-b-w6-d9f1c2-6475ccbbb9-x7xgs Stopping container payload-processing 21m Warning Unhealthy pod/payload-processing-e2e-shared-a-w6-0ff5b0-7c4b7489d7-xlk52 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Normal Killing pod/payload-pre-processing-e2e-shared-b-w6-d9f1c2-5bf66857f-zbxhk Stopping container payload-pre-processing 21m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w6-0ff5b0-6f4dc7d5f-pmqnt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Normal Killing pod/e2e-ipp-a-f0a807-openshift-default-66f955d779-qv6wr Stopping container istio-proxy 21m Normal Killing pod/payload-processing-e2e-ipp-a-f88979-74fd6f8bd-vfz9j Stopping container payload-processing 21m Warning Unhealthy pod/e2e-ipp-a-f0a807-openshift-default-66f955d779-qv6wr Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning ProbeError pod/e2e-ipp-a-f0a807-openshift-default-66f955d779-qv6wr Readiness probe error: HTTP probe failed with statuscode: 503... 21m Normal Killing pod/payload-pre-processing-e2e-ipp-a-f88979-669bd98b7f-9lsrp Stopping container payload-pre-processing 21m Normal Killing pod/e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c-9ps8d Stopping container istio-proxy 21m Normal Killing pod/e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c-krxtr Stopping container istio-proxy 21m Warning ProbeError pod/e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c-krxtr Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning Unhealthy pod/e2e-shared-a-w5-6f2eba-openshift-default-5658f7689c-krxtr Readiness probe failed: HTTP probe failed with statuscode: 503 21m Normal Killing pod/e2e-ipp-a-f88979-openshift-default-674df6d98f-tr49g Stopping container istio-proxy 21m Warning ProbeError pod/e2e-ipp-a-f88979-openshift-default-674df6d98f-tr49g Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning Unhealthy pod/e2e-ipp-a-f88979-openshift-default-674df6d98f-tr49g Readiness probe failed: HTTP probe failed with statuscode: 503 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-6af86e-openshift-default New size: 1; reason: All metrics below target 21m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-6af86e-openshift-default Scaled down replica set e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c from 2 to 1 21m Normal Killing pod/e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c-2xvj7 Stopping container istio-proxy 21m Normal SuccessfulDelete replicaset/e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c Deleted pod: e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c-2xvj7 21m Normal Killing pod/payload-processing-e2e-shared-b-w5-510f80-c84b8b845-fpfjm Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-shared-b-w5-510f80-54b6b8c75-9w995 Stopping container payload-pre-processing 21m Warning Unhealthy pod/payload-processing-e2e-shared-b-w6-d9f1c2-6475ccbbb9-x7xgs Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-3cd77f-openshift-default Scaled down replica set e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c from 3 to 2 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-3cd77f-openshift-default New size: 2; reason: All metrics below target 21m Normal SuccessfulDelete replicaset/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c Deleted pod: e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-bmccc 21m Normal Killing pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-bmccc Stopping container istio-proxy 21m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w6-d9f1c2-5bf66857f-zbxhk Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Normal Killing pod/payload-pre-processing-e2e-ipp-b-cdfa86-5799cbf495-h927d Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-ipp-b-cdfa86-79959f8c88-hcpjf Stopping container payload-processing 21m Normal Pulled pod/e2e-ipp-b-bff950-openshift-default-5dd888d8fc-46j6j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Started pod/e2e-ipp-b-bff950-openshift-default-5dd888d8fc-46j6j Container started 21m Normal Created pod/e2e-ipp-b-bff950-openshift-default-5dd888d8fc-46j6j Container created 21m Normal Killing pod/e2e-ipp-b-cdfa86-openshift-default-77bdfd8797-52tjw Stopping container istio-proxy 21m Warning Unhealthy pod/e2e-ipp-b-bff950-openshift-default-5dd888d8fc-46j6j Startup probe failed: Get "http://10.133.1.107:15021/healthz/ready": dial tcp 10.133.1.107:15021: connect: connection refused 21m Warning ProbeError pod/e2e-ipp-b-bff950-openshift-default-5dd888d8fc-46j6j Startup probe error: Get "http://10.133.1.107:15021/healthz/ready": dial tcp 10.133.1.107:15021: connect: connection refused... 20m Normal Killing pod/payload-pre-processing-e2e-ipp-b-bff950-d477c7687-vm8sg Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-ipp-b-bff950-746d89d77f-w4559 Stopping container payload-processing 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w5-510f80-54b6b8c75-9w995 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-processing-e2e-shared-b-w5-510f80-c84b8b845-fpfjm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447-l89ql Stopping container istio-proxy 20m Warning ProbeError pod/e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447-l89ql Readiness probe error: HTTP probe failed with statuscode: 503... 20m Warning Unhealthy pod/e2e-shared-b-w6-d9f1c2-openshift-default-55dd7d9447-l89ql Readiness probe failed: HTTP probe failed with statuscode: 503 20m Normal Killing pod/payload-processing-e2e-ipp-b-76d54a-84746b7f6b-br8pt Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-ipp-b-76d54a-fcfc75c9f-lvgx2 Stopping container payload-pre-processing 20m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-80ecb3-7484c79f79-cndqx Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-shared-a-w2-80ecb3-67699fb85f-7tctg Stopping container payload-processing 20m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-bff950-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-bff950-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Normal Killing pod/payload-pre-processing-e2e-worker-w6-012ba0-669ff7bfff-8w9bc Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-worker-w6-012ba0-5669b88f55-swpg6 Stopping container payload-processing 20m Normal Killing pod/e2e-ipp-b-76d54a-openshift-default-6bcffbc9bc-8d4d2 Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-4qscf Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-b-w5-510f80-openshift-default-6465ccddcf-wjvgx Stopping container istio-proxy 20m Normal Killing pod/payload-processing-e2e-shared-a-w1-bed544-5b5dc64c47-5dwdv Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-shared-a-w1-bed544-6bc95f58bc-xmfrv Stopping container payload-pre-processing 20m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-bff950-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 20m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-bff950-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 20m Normal Killing pod/payload-pre-processing-e2e-shared-a-w0-5983ea-f76b4bb8c-xvp6k Stopping container payload-pre-processing 20m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-bff950 Scaled up replica set payload-processing-e2e-ipp-b-bff950-958c95556 from 0 to 1 20m Normal Killing pod/payload-processing-e2e-shared-a-w0-5983ea-65fbff47dc-kjmcc Stopping container payload-processing 20m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-bff950-958c95556 Error creating: pods "payload-processing-e2e-ipp-b-bff950-958c95556-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-bff950: serviceaccount "payload-processing-e2e-ipp-b-bff950" not found 20m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-bff950-958c95556 Created pod: payload-processing-e2e-ipp-b-bff950-958c95556-hrzxt 20m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-bff950-67c96cc479 Created pod: payload-pre-processing-e2e-ipp-b-bff950-67c96cc479-ll5pt 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-bff950-67c96cc479-ll5pt Add eth0 [10.132.1.2/23] from ovn-kubernetes 20m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-bff950 Scaled up replica set payload-pre-processing-e2e-ipp-b-bff950-67c96cc479 from 0 to 1 20m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-bff950-67c96cc479 Error creating: pods "payload-pre-processing-e2e-ipp-b-bff950-67c96cc479-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-bff950: serviceaccount "payload-processing-e2e-ipp-b-bff950" not found 20m Normal Killing pod/payload-pre-processing-e2e-worker-w5-54114e-7f6f89654c-sb4mv Stopping container payload-pre-processing 20m Normal Pulled pod/payload-processing-e2e-ipp-b-bff950-958c95556-hrzxt Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-bff950-67c96cc479-ll5pt Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal Started pod/payload-processing-e2e-ipp-b-bff950-958c95556-hrzxt Container started 20m Normal Created pod/payload-processing-e2e-ipp-b-bff950-958c95556-hrzxt Container created 20m Normal Created pod/payload-pre-processing-e2e-ipp-b-bff950-67c96cc479-ll5pt Container created 20m Normal AddedInterface pod/payload-processing-e2e-ipp-b-bff950-958c95556-hrzxt Add eth0 [10.132.1.3/23] from ovn-kubernetes 20m Normal Killing pod/payload-processing-e2e-worker-w5-54114e-5cf4644f-8htwc Stopping container payload-processing 20m Normal Started pod/payload-pre-processing-e2e-ipp-b-bff950-67c96cc479-ll5pt Container started 20m Normal Killing pod/e2e-shared-a-w2-80ecb3-openshift-default-75b8ddb949-2xr8n Stopping container istio-proxy 20m Warning Unhealthy pod/payload-processing-e2e-shared-a-w2-80ecb3-67699fb85f-7tctg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w2-80ecb3-7484c79f79-cndqx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w6-012ba0-669ff7bfff-8w9bc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-processing-e2e-worker-w6-012ba0-5669b88f55-swpg6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/e2e-shared-a-w0-5983ea-openshift-default-849df9ccdf-bbqjg Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-a-w0-5983ea-openshift-default-849df9ccdf-ffzk6 Stopping container istio-proxy 20m Warning Unhealthy pod/payload-processing-e2e-shared-a-w1-bed544-5b5dc64c47-5dwdv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w1-bed544-6bc95f58bc-xmfrv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/payload-processing-e2e-shared-b-w2-3cd77f-6d7cccccf5-gxpwd Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-3cd77f-799d6dfd99-2zkl9 Stopping container payload-pre-processing 20m Normal Killing pod/e2e-shared-a-w1-bed544-openshift-default-8659947965-dmmj5 Stopping container istio-proxy 20m Warning Unhealthy pod/payload-processing-e2e-shared-a-w0-5983ea-65fbff47dc-kjmcc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w0-5983ea-f76b4bb8c-xvp6k Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/payload-pre-processing-e2e-shared-b-w0-6af86e-64944b4ff9-4229b Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-shared-b-w0-6af86e-8bfcb675f-rgngw Stopping container payload-processing 20m Normal Killing pod/payload-processing-e2e-shared-b-w1-23aab2-67f75c4687-2wq29 Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-shared-b-w1-23aab2-ff485dcf-pzd2r Stopping container payload-pre-processing 20m Warning Unhealthy pod/payload-processing-e2e-worker-w5-54114e-5cf4644f-8htwc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w5-54114e-7f6f89654c-sb4mv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/payload-processing-e2e-ipp-a-f94561-b76dcb6f6-jvts2 Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-ipp-a-f94561-66b74889b7-j6lk5 Stopping container payload-pre-processing 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w2-3cd77f-799d6dfd99-2zkl9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-processing-e2e-shared-b-w2-3cd77f-6d7cccccf5-gxpwd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-cdc122-openshift-default New size: 2; reason: All metrics below target 19m Normal Killing pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-wsd2x Stopping container istio-proxy 19m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-cdc122-openshift-default Scaled down replica set e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc from 3 to 2 19m Normal SuccessfulDelete replicaset/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc Deleted pod: e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-wsd2x 19m Normal Killing pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-p2chp Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-52wf7 Stopping container istio-proxy 19m Warning Unhealthy pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-p2chp Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning ProbeError pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-p2chp Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning ProbeError pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-52wf7 Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning Unhealthy pod/e2e-shared-b-w2-3cd77f-openshift-default-6d76d5595c-52wf7 Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning ProbeError pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-kzdls Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning Unhealthy pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-kzdls Readiness probe failed: HTTP probe failed with statuscode: 503 19m Normal Killing pod/e2e-ipp-a-f94561-openshift-default-66c966685c-zcxmh Stopping container istio-proxy 19m Normal Killing pod/e2e-worker-w6-012ba0-openshift-default-846bcc5bfc-kzdls Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-b-w1-23aab2-openshift-default-86c4fcc59c-qvpxp Stopping container istio-proxy 19m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w1-23aab2-ff485dcf-pzd2r Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-processing-e2e-shared-b-w1-23aab2-67f75c4687-2wq29 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/e2e-shared-b-w0-6af86e-openshift-default-5bd446b89c-ztnmj Stopping container istio-proxy 19m Normal Killing pod/payload-processing-e2e-worker-w2-421583-7f9c569f65-xm89m Stopping container payload-processing 19m Normal Killing pod/payload-pre-processing-e2e-worker-w2-421583-bfd5dcc6c-7ztzj Stopping container payload-pre-processing 19m Warning Unhealthy pod/e2e-shared-b-w1-23aab2-openshift-default-86c4fcc59c-qvpxp Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning ProbeError pod/e2e-shared-b-w1-23aab2-openshift-default-86c4fcc59c-qvpxp Readiness probe error: HTTP probe failed with statuscode: 503... 19m Normal Killing pod/payload-pre-processing-e2e-worker-w1-2b1e88-65c766876c-4qz4v Stopping container payload-pre-processing 19m Normal Killing pod/payload-pre-processing-e2e-ipp-b-bff950-67c96cc479-ll5pt Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-worker-w1-2b1e88-6f4479c569-mltjd Stopping container payload-processing 19m Normal Killing pod/payload-processing-e2e-ipp-b-bff950-958c95556-hrzxt Stopping container payload-processing 19m Normal Killing pod/payload-pre-processing-e2e-worker-w0-09d0d5-75f8f97f55-6hq48 Stopping container payload-pre-processing 19m Normal Killing pod/e2e-worker-w5-54114e-openshift-default-7fc467ddd7-tdvdc Stopping container istio-proxy 19m Normal Killing pod/payload-processing-e2e-worker-w0-09d0d5-85cb95d7ff-n4lzb Stopping container payload-processing 19m Warning Unhealthy pod/e2e-ipp-b-bff950-openshift-default-5dd888d8fc-46j6j Readiness probe failed: HTTP probe failed with statuscode: 503 19m Normal Killing pod/e2e-ipp-b-bff950-openshift-default-5dd888d8fc-46j6j Stopping container istio-proxy 19m Warning ProbeError pod/e2e-ipp-b-bff950-openshift-default-5dd888d8fc-46j6j Readiness probe error: HTTP probe failed with statuscode: 503... 19m Normal Killing pod/e2e-worker-w2-421583-openshift-default-5cb767b447-k94dp Stopping container istio-proxy 19m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-421583-bfd5dcc6c-7ztzj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-processing-e2e-worker-w2-421583-7f9c569f65-xm89m Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/payload-pre-processing-e2e-shared-a-w3-cdc122-5b697dd967-7qll4 Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-shared-a-w3-cdc122-5584f9d75c-fljm5 Stopping container payload-processing 19m Normal Killing pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-bmvsb Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-6bvgh Stopping container istio-proxy 19m Warning Unhealthy pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-bmvsb Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning ProbeError pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-bmvsb Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning Unhealthy pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-6bvgh Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning ProbeError pod/e2e-shared-a-w3-cdc122-openshift-default-7b7d59b4dc-6bvgh Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning Unhealthy pod/payload-processing-e2e-shared-a-w3-cdc122-5584f9d75c-fljm5 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w3-cdc122-5b697dd967-7qll4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-cjkc9 Stopping container istio-proxy 18m Normal Killing pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-pmswk Stopping container istio-proxy 18m Warning Unhealthy pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-pmswk Readiness probe failed: HTTP probe failed with statuscode: 503 18m Warning ProbeError pod/e2e-worker-w0-09d0d5-openshift-default-fdf9b9665-pmswk Readiness probe error: HTTP probe failed with statuscode: 503... 18m Normal Killing pod/payload-pre-processing-e2e-shared-b-w3-e14e00-bdf56dd9f-67qgc Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-shared-b-w3-e14e00-56ff64bff5-ht6tb Stopping container payload-processing 18m Warning Unhealthy pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-cjkc9 Readiness probe failed: HTTP probe failed with statuscode: 503 18m Warning ProbeError pod/e2e-worker-w1-2b1e88-openshift-default-84bb47bf79-cjkc9 Readiness probe error: HTTP probe failed with statuscode: 503... 18m Normal Killing pod/e2e-shared-b-w3-e14e00-openshift-default-565cddf77c-7jg4k Stopping container istio-proxy 18m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w3-e14e00-bdf56dd9f-67qgc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-processing-e2e-shared-b-w3-e14e00-56ff64bff5-ht6tb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Normal Killing pod/payload-pre-processing-e2e-worker-w3-d52c59-56d5c877c5-zvrjx Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-worker-w3-d52c59-f75cff857-zdrn8 Stopping container payload-processing 18m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w3-d52c59-56d5c877c5-zvrjx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-processing-e2e-worker-w3-d52c59-f75cff857-zdrn8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Normal Killing pod/e2e-worker-w3-d52c59-openshift-default-5466968fb7-tbj69 Stopping container istio-proxy 17m Normal AddedInterface pod/maas-default-gateway-openshift-default-6db586dd6c-p2flk Add eth0 [10.132.1.9/23] from ovn-kubernetes 17m Normal Created pod/maas-default-gateway-openshift-default-6db586dd6c-p2flk Container created 17m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 17m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6db586dd6c from 2 to 4 17m Normal Pulled pod/maas-default-gateway-openshift-default-6db586dd6c-v8xjj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal Created pod/maas-default-gateway-openshift-default-6db586dd6c-v8xjj Container created 17m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6db586dd6c Created pod: maas-default-gateway-openshift-default-6db586dd6c-p2flk 17m Normal Started pod/maas-default-gateway-openshift-default-6db586dd6c-p2flk Container started 17m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6db586dd6c Created pod: maas-default-gateway-openshift-default-6db586dd6c-v8xjj 17m Normal Pulled pod/maas-default-gateway-openshift-default-6db586dd6c-p2flk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal Started pod/maas-default-gateway-openshift-default-6db586dd6c-v8xjj Container started 17m Normal AddedInterface pod/maas-default-gateway-openshift-default-6db586dd6c-v8xjj Add eth0 [10.132.1.8/23] from ovn-kubernetes 17m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-p2flk Startup probe failed: Get "http://10.132.1.9:15021/healthz/ready": dial tcp 10.132.1.9:15021: connect: connection refused 17m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-v8xjj Startup probe error: Get "http://10.132.1.8:15021/healthz/ready": dial tcp 10.132.1.8:15021: connect: connection refused... 17m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-p2flk Startup probe error: Get "http://10.132.1.9:15021/healthz/ready": dial tcp 10.132.1.9:15021: connect: connection refused... 17m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-v8xjj Startup probe failed: Get "http://10.132.1.8:15021/healthz/ready": dial tcp 10.132.1.8:15021: connect: connection refused 17m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-p2flk Startup probe failed: Get "http://10.132.1.9:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 17m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-p2flk Startup probe error: Get "http://10.132.1.9:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 17m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-v8xjj Startup probe error: Get "http://10.132.1.8:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 17m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-v8xjj Startup probe failed: Get "http://10.132.1.8:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 17m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: All metrics below target 17m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6db586dd6c Created pod: maas-default-gateway-openshift-default-6db586dd6c-hcq62 17m Normal AddedInterface pod/maas-default-gateway-openshift-default-6db586dd6c-hcq62 Add eth0 [10.133.1.146/23] from ovn-kubernetes 17m Normal Pulled pod/maas-default-gateway-openshift-default-6db586dd6c-hcq62 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6db586dd6c from 4 to 5 17m Normal Started pod/maas-default-gateway-openshift-default-6db586dd6c-hcq62 Container started 17m Normal Created pod/maas-default-gateway-openshift-default-6db586dd6c-hcq62 Container created 17m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-hcq62 Startup probe error: Get "http://10.133.1.146:15021/healthz/ready": dial tcp 10.133.1.146:15021: connect: connection refused... 17m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-hcq62 Startup probe failed: Get "http://10.133.1.146:15021/healthz/ready": dial tcp 10.133.1.146:15021: connect: connection refused 16m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6db586dd6c Created pod: maas-default-gateway-openshift-default-6db586dd6c-jp9lg 16m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6db586dd6c Created pod: maas-default-gateway-openshift-default-6db586dd6c-tbk5w 16m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6db586dd6c Created pod: maas-default-gateway-openshift-default-6db586dd6c-d8sgs 16m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6db586dd6c Created pod: maas-default-gateway-openshift-default-6db586dd6c-7qjgv 16m Normal AddedInterface pod/maas-default-gateway-openshift-default-6db586dd6c-d8sgs Add eth0 [10.134.0.150/23] from ovn-kubernetes 16m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-6db586dd6c Created pod: maas-default-gateway-openshift-default-6db586dd6c-h8mv5 16m Normal AddedInterface pod/maas-default-gateway-openshift-default-6db586dd6c-7qjgv Add eth0 [10.132.1.12/23] from ovn-kubernetes 16m Normal AddedInterface pod/maas-default-gateway-openshift-default-6db586dd6c-jp9lg Add eth0 [10.132.1.11/23] from ovn-kubernetes 16m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 16m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-6db586dd6c from 5 to 10 16m Normal Pulled pod/maas-default-gateway-openshift-default-6db586dd6c-7qjgv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal Created pod/maas-default-gateway-openshift-default-6db586dd6c-jp9lg Container created 16m Normal Started pod/maas-default-gateway-openshift-default-6db586dd6c-tbk5w Container started 16m Normal Created pod/maas-default-gateway-openshift-default-6db586dd6c-tbk5w Container created 16m Normal AddedInterface pod/maas-default-gateway-openshift-default-6db586dd6c-h8mv5 Add eth0 [10.133.1.148/23] from ovn-kubernetes 16m Normal Pulled pod/maas-default-gateway-openshift-default-6db586dd6c-h8mv5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal Created pod/maas-default-gateway-openshift-default-6db586dd6c-h8mv5 Container created 16m Normal Started pod/maas-default-gateway-openshift-default-6db586dd6c-h8mv5 Container started 16m Normal Pulled pod/maas-default-gateway-openshift-default-6db586dd6c-tbk5w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal AddedInterface pod/maas-default-gateway-openshift-default-6db586dd6c-tbk5w Add eth0 [10.133.1.149/23] from ovn-kubernetes 16m Normal Started pod/maas-default-gateway-openshift-default-6db586dd6c-jp9lg Container started 16m Normal Started pod/maas-default-gateway-openshift-default-6db586dd6c-d8sgs Container started 16m Normal Pulled pod/maas-default-gateway-openshift-default-6db586dd6c-jp9lg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal Created pod/maas-default-gateway-openshift-default-6db586dd6c-7qjgv Container created 16m Normal Created pod/maas-default-gateway-openshift-default-6db586dd6c-d8sgs Container created 16m Normal Pulled pod/maas-default-gateway-openshift-default-6db586dd6c-d8sgs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal Started pod/maas-default-gateway-openshift-default-6db586dd6c-7qjgv Container started 16m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-d8sgs Startup probe error: Get "http://10.134.0.150:15021/healthz/ready": dial tcp 10.134.0.150:15021: connect: connection refused... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-d8sgs Startup probe failed: Get "http://10.134.0.150:15021/healthz/ready": dial tcp 10.134.0.150:15021: connect: connection refused 16m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-7qjgv Startup probe error: Get "http://10.132.1.12:15021/healthz/ready": dial tcp 10.132.1.12:15021: connect: connection refused... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-7qjgv Startup probe failed: Get "http://10.132.1.12:15021/healthz/ready": dial tcp 10.132.1.12:15021: connect: connection refused 16m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-jp9lg Startup probe error: Get "http://10.132.1.11:15021/healthz/ready": dial tcp 10.132.1.11:15021: connect: connection refused... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-jp9lg Startup probe failed: Get "http://10.132.1.11:15021/healthz/ready": dial tcp 10.132.1.11:15021: connect: connection refused 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-h8mv5 Startup probe failed: Get "http://10.133.1.148:15021/healthz/ready": dial tcp 10.133.1.148:15021: connect: connection refused 16m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-h8mv5 Startup probe error: Get "http://10.133.1.148:15021/healthz/ready": dial tcp 10.133.1.148:15021: connect: connection refused... 16m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-tbk5w Startup probe error: Get "http://10.133.1.149:15021/healthz/ready": dial tcp 10.133.1.149:15021: connect: connection refused... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-tbk5w Startup probe failed: Get "http://10.133.1.149:15021/healthz/ready": dial tcp 10.133.1.149:15021: connect: connection refused 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-d8sgs Startup probe failed: Get "http://10.134.0.150:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 16m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-d8sgs Startup probe error: Get "http://10.134.0.150:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-jp9lg Startup probe failed: Get "http://10.132.1.11:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 16m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-7qjgv Startup probe error: Get "http://10.132.1.12:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 16m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-tbk5w Startup probe error: Get "http://10.133.1.149:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-tbk5w Startup probe failed: Get "http://10.133.1.149:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-7qjgv Startup probe failed: Get "http://10.132.1.12:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 16m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-jp9lg Startup probe error: Get "http://10.132.1.11:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-h8mv5 Startup probe failed: Get "http://10.133.1.148:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 16m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-h8mv5 Startup probe error: Get "http://10.133.1.148:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-hcq62 Readiness probe failed: Get "http://10.133.1.146:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 15m Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-hcq62 Readiness probe error: Get "http://10.133.1.146:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 12m Normal ScalingReplicaSet deployment/e2e-worker-single-main-2cad7e-openshift-default Scaled up replica set e2e-worker-single-main-2cad7e-openshift-default-6465775477 from 0 to 1 12m Normal AddedLabel gateway/e2e-worker-single-main-2cad7e Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-2cad7e 12m Normal SuccessfulCreate replicaset/e2e-worker-single-main-2cad7e-openshift-default-6465775477 Created pod: e2e-worker-single-main-2cad7e-openshift-default-6465775477q6cwj 12m Normal AddedInterface pod/e2e-worker-single-main-2cad7e-openshift-default-6465775477q6cwj Add eth0 [10.132.1.13/23] from ovn-kubernetes 12m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-2cad7e-758854df5f Error creating: pods "payload-processing-e2e-worker-single-main-2cad7e-758854df5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-2cad7e: serviceaccount "payload-processing-e2e-worker-single-main-2cad7e" not found 12m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-2cad7e-d85d5469f Created pod: payload-pre-processing-e2e-worker-single-main-2cad7e-d85d5hddg4 12m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-2cad7e-758854df5f Created pod: payload-processing-e2e-worker-single-main-2cad7e-758854df5pq2ws 12m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-2cad7e Scaled up replica set payload-processing-e2e-worker-single-main-2cad7e-758854df5f from 0 to 1 12m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-2cad7e Scaled up replica set payload-pre-processing-e2e-worker-single-main-2cad7e-d85d5469f from 0 to 1 12m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-2cad7e-d85d5469f Error creating: pods "payload-pre-processing-e2e-worker-single-main-2cad7e-d85d5469f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-2cad7e: serviceaccount "payload-processing-e2e-worker-single-main-2cad7e" not found 12m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-2cad7e-758854df5pq2ws Add eth0 [10.132.1.15/23] from ovn-kubernetes 12m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-2cad7e-d85d5hddg4 Add eth0 [10.132.1.16/23] from ovn-kubernetes 12m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-2cad7e-d85d5hddg4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 12m Normal Pulled pod/payload-processing-e2e-worker-single-main-2cad7e-758854df5pq2ws Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 12m Normal Created pod/payload-pre-processing-e2e-worker-single-main-2cad7e-d85d5hddg4 Container created 12m Normal Created pod/payload-processing-e2e-worker-single-main-2cad7e-758854df5pq2ws Container created 12m Normal Started pod/payload-processing-e2e-worker-single-main-2cad7e-758854df5pq2ws Container started 12m Normal Started pod/payload-pre-processing-e2e-worker-single-main-2cad7e-d85d5hddg4 Container started 12m Warning Unhealthy pod/e2e-worker-single-main-2cad7e-openshift-default-6465775477q6cwj Readiness probe failed: Get "http://10.132.1.13:15021/healthz/ready": read tcp 10.132.0.2:59974->10.132.1.13:15021: read: connection reset by peer 12m Normal Pulled pod/e2e-worker-single-main-2cad7e-openshift-default-6465775477q6cwj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 12m Normal Started pod/e2e-worker-single-main-2cad7e-openshift-default-6465775477q6cwj Container started 12m Warning ProbeError pod/e2e-worker-single-main-2cad7e-openshift-default-6465775477q6cwj Readiness probe error: Get "http://10.132.1.13:15021/healthz/ready": read tcp 10.132.0.2:59974->10.132.1.13:15021: read: connection reset by peer... 12m Normal Created pod/e2e-worker-single-main-2cad7e-openshift-default-6465775477q6cwj Container created 11m Warning Unhealthy pod/e2e-worker-single-main-2cad7e-openshift-default-6465775477q6cwj Startup probe failed: Get "http://10.132.1.13:15021/healthz/ready": dial tcp 10.132.1.13:15021: connect: connection refused 11m Warning ProbeError pod/e2e-worker-single-main-2cad7e-openshift-default-6465775477q6cwj Startup probe error: Get "http://10.132.1.13:15021/healthz/ready": dial tcp 10.132.1.13:15021: connect: connection refused... 11m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-2cad7e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 11m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-2cad7e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 11m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-2cad7e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 11m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-2cad7e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 11m Normal Killing pod/payload-processing-e2e-worker-single-main-2cad7e-758854df5pq2ws Stopping container payload-processing 11m Normal Killing pod/payload-pre-processing-e2e-worker-single-main-2cad7e-d85d5hddg4 Stopping container payload-pre-processing 10m Normal Killing pod/e2e-worker-single-main-2cad7e-openshift-default-6465775477q6cwj Stopping container istio-proxy 10m Warning ProbeError pod/e2e-worker-single-main-2cad7e-openshift-default-6465775477q6cwj Readiness probe error: HTTP probe failed with statuscode: 503... 10m Warning Unhealthy pod/e2e-worker-single-main-2cad7e-openshift-default-6465775477q6cwj Readiness probe failed: HTTP probe failed with statuscode: 503 10m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-2cad7e-758854df5pq2ws Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 10m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-2cad7e-d85d5hddg4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 8m35s Normal AddedLabel gateway/e2e-worker-single-main-ef9544 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-ef9544 8m35s Normal ScalingReplicaSet deployment/e2e-worker-single-main-ef9544-openshift-default Scaled up replica set e2e-worker-single-main-ef9544-openshift-default-68f86596d5 from 0 to 1 8m35s Normal SuccessfulCreate replicaset/e2e-worker-single-main-ef9544-openshift-default-68f86596d5 Created pod: e2e-worker-single-main-ef9544-openshift-default-68f86596d5hj74l 8m35s Normal AddedInterface pod/e2e-worker-single-main-ef9544-openshift-default-68f86596d5hj74l Add eth0 [10.132.1.18/23] from ovn-kubernetes 8m30s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-ef9544 Scaled up replica set payload-pre-processing-e2e-worker-single-main-ef9544-69bf697fb7 from 0 to 1 8m30s Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-ef9544 Scaled up replica set payload-processing-e2e-worker-single-main-ef9544-6849c57fc7 from 0 to 1 8m29s Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-ef9544-69bf697fb7 Error creating: pods "payload-pre-processing-e2e-worker-single-main-ef9544-69bf697fb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-ef9544: serviceaccount "payload-processing-e2e-worker-single-main-ef9544" not found 8m29s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-ef9544-69bf697fb7 Created pod: payload-pre-processing-e2e-worker-single-main-ef9544-69bf6pfp67 8m29s Normal Created pod/payload-processing-e2e-worker-single-main-ef9544-6849c57fcsxzg9 Container created 8m29s Normal AddedInterface pod/payload-processing-e2e-worker-single-main-ef9544-6849c57fcsxzg9 Add eth0 [10.132.1.21/23] from ovn-kubernetes 8m29s Normal Started pod/payload-processing-e2e-worker-single-main-ef9544-6849c57fcsxzg9 Container started 8m29s Normal Started pod/payload-pre-processing-e2e-worker-single-main-ef9544-69bf6pfp67 Container started 8m29s Normal Created pod/payload-pre-processing-e2e-worker-single-main-ef9544-69bf6pfp67 Container created 8m29s Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-ef9544-69bf6pfp67 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m29s Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-ef9544-6849c57fc7 Created pod: payload-processing-e2e-worker-single-main-ef9544-6849c57fcsxzg9 8m29s Normal Pulled pod/payload-processing-e2e-worker-single-main-ef9544-6849c57fcsxzg9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m29s Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-ef9544-69bf6pfp67 Add eth0 [10.132.1.20/23] from ovn-kubernetes 8m29s Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-ef9544-6849c57fc7 Error creating: pods "payload-processing-e2e-worker-single-main-ef9544-6849c57fc7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-ef9544: serviceaccount "payload-processing-e2e-worker-single-main-ef9544" not found 8m17s Warning ProbeError pod/e2e-worker-single-main-ef9544-openshift-default-68f86596d5hj74l Readiness probe error: Get "http://10.132.1.18:15021/healthz/ready": read tcp 10.132.0.2:55174->10.132.1.18:15021: read: connection reset by peer... 8m17s Normal Pulled pod/e2e-worker-single-main-ef9544-openshift-default-68f86596d5hj74l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 8m17s Warning Unhealthy pod/e2e-worker-single-main-ef9544-openshift-default-68f86596d5hj74l Readiness probe failed: Get "http://10.132.1.18:15021/healthz/ready": read tcp 10.132.0.2:55174->10.132.1.18:15021: read: connection reset by peer 8m17s Normal Created pod/e2e-worker-single-main-ef9544-openshift-default-68f86596d5hj74l Container created 8m16s Normal Started pod/e2e-worker-single-main-ef9544-openshift-default-68f86596d5hj74l Container started 8m11s Warning Unhealthy pod/e2e-worker-single-main-ef9544-openshift-default-68f86596d5hj74l Startup probe failed: Get "http://10.132.1.18:15021/healthz/ready": dial tcp 10.132.1.18:15021: connect: connection refused 8m11s Warning ProbeError pod/e2e-worker-single-main-ef9544-openshift-default-68f86596d5hj74l Startup probe error: Get "http://10.132.1.18:15021/healthz/ready": dial tcp 10.132.1.18:15021: connect: connection refused... 7m50s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-ef9544-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 7m50s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-ef9544-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 7m35s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-ef9544-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 7m35s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-ef9544-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 7m20s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-ef9544-69bf6pfp67 Stopping container payload-pre-processing 7m20s Normal Killing pod/payload-processing-e2e-worker-single-main-ef9544-6849c57fcsxzg9 Stopping container payload-processing 7m11s Normal Killing pod/e2e-worker-single-main-ef9544-openshift-default-68f86596d5hj74l Stopping container istio-proxy 7m10s Warning Unhealthy pod/e2e-worker-single-main-ef9544-openshift-default-68f86596d5hj74l Readiness probe failed: HTTP probe failed with statuscode: 503 7m10s Warning ProbeError pod/e2e-worker-single-main-ef9544-openshift-default-68f86596d5hj74l Readiness probe error: HTTP probe failed with statuscode: 503... 6m57s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-ef9544-6849c57fcsxzg9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 6m57s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-ef9544-69bf6pfp67 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 5m56s Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-24vbj Readiness probe error: Get "http://10.133.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 5m56s Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-24vbj Readiness probe failed: Get "http://10.133.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 5m25s Normal SuccessfulCreate replicaset/e2e-ait-1595eccb-openshift-default-84f7ff667 Created pod: e2e-ait-1595eccb-openshift-default-84f7ff667-58jtb 5m25s Normal ScalingReplicaSet deployment/e2e-ait-1595eccb-openshift-default Scaled up replica set e2e-ait-1595eccb-openshift-default-84f7ff667 from 0 to 1 5m25s Normal AddedLabel gateway/e2e-ait-1595eccb Added label istio.io/rev=openshift-gateway to gateway e2e-ait-1595eccb 5m24s Normal AddedInterface pod/e2e-ait-1595eccb-openshift-default-84f7ff667-58jtb Add eth0 [10.132.1.26/23] from ovn-kubernetes 5m20s Warning FailedCreate replicaset/payload-processing-e2e-ait-1595eccb-65879f7ddf Error creating: pods "payload-processing-e2e-ait-1595eccb-65879f7ddf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-1595eccb: serviceaccount "payload-processing-e2e-ait-1595eccb" not found 5m20s Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-1595eccb Scaled up replica set payload-processing-e2e-ait-1595eccb-65879f7ddf from 0 to 1 5m20s Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-1595eccb-65879f7ddf Created pod: payload-processing-e2e-ait-1595eccb-65879f7ddf-dxpjx 5m20s Normal Created pod/payload-processing-e2e-ait-1595eccb-65879f7ddf-dxpjx Container created 5m20s Normal AddedInterface pod/payload-pre-processing-e2e-ait-1595eccb-589744bfdc-m69n7 Add eth0 [10.132.1.28/23] from ovn-kubernetes 5m20s Normal Pulled pod/payload-processing-e2e-ait-1595eccb-65879f7ddf-dxpjx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m20s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-1595eccb-589744bfdc Created pod: payload-pre-processing-e2e-ait-1595eccb-589744bfdc-m69n7 5m20s Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-1595eccb-589744bfdc Error creating: pods "payload-pre-processing-e2e-ait-1595eccb-589744bfdc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-1595eccb: serviceaccount "payload-processing-e2e-ait-1595eccb" not found 5m20s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-1595eccb Scaled up replica set payload-pre-processing-e2e-ait-1595eccb-589744bfdc from 0 to 1 5m20s Normal AddedInterface pod/payload-processing-e2e-ait-1595eccb-65879f7ddf-dxpjx Add eth0 [10.132.1.27/23] from ovn-kubernetes 5m19s Normal Started pod/payload-pre-processing-e2e-ait-1595eccb-589744bfdc-m69n7 Container started 5m19s Normal Created pod/payload-pre-processing-e2e-ait-1595eccb-589744bfdc-m69n7 Container created 5m19s Normal Pulled pod/payload-pre-processing-e2e-ait-1595eccb-589744bfdc-m69n7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m19s Normal Started pod/payload-processing-e2e-ait-1595eccb-65879f7ddf-dxpjx Container started 5m2s Normal Pulled pod/e2e-ait-1595eccb-openshift-default-84f7ff667-58jtb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 5m2s Normal Created pod/e2e-ait-1595eccb-openshift-default-84f7ff667-58jtb Container created 5m2s Normal Started pod/e2e-ait-1595eccb-openshift-default-84f7ff667-58jtb Container started 4m57s Warning ProbeError pod/e2e-ait-1595eccb-openshift-default-84f7ff667-58jtb Startup probe error: Get "http://10.132.1.26:15021/healthz/ready": dial tcp 10.132.1.26:15021: connect: connection refused... 4m57s Warning Unhealthy pod/e2e-ait-1595eccb-openshift-default-84f7ff667-58jtb Startup probe failed: Get "http://10.132.1.26:15021/healthz/ready": dial tcp 10.132.1.26:15021: connect: connection refused 4m56s Normal Killing pod/payload-processing-e2e-ait-1595eccb-65879f7ddf-dxpjx Stopping container payload-processing 4m56s Normal Killing pod/payload-pre-processing-e2e-ait-1595eccb-589744bfdc-m69n7 Stopping container payload-pre-processing 4m55s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-1595eccb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m55s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-1595eccb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m50s Normal Killing pod/e2e-ait-1595eccb-openshift-default-84f7ff667-58jtb Stopping container istio-proxy 4m49s Normal SuccessfulCreate replicaset/e2e-authcfg-http-9cdd18-openshift-default-575d9dcd85 Created pod: e2e-authcfg-http-9cdd18-openshift-default-575d9dcd85-h972c 4m49s Normal ScalingReplicaSet deployment/e2e-authcfg-http-9cdd18-openshift-default Scaled up replica set e2e-authcfg-http-9cdd18-openshift-default-575d9dcd85 from 0 to 1 4m49s Normal AddedLabel gateway/e2e-authcfg-http-9cdd18 Added label istio.io/rev=openshift-gateway to gateway e2e-authcfg-http-9cdd18 4m48s Normal AddedInterface pod/e2e-authcfg-http-9cdd18-openshift-default-575d9dcd85-h972c Add eth0 [10.132.1.30/23] from ovn-kubernetes 4m43s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-authcfg-http-9cdd18 Scaled up replica set payload-pre-processing-e2e-authcfg-http-9cdd18-7f54b6fc9f from 0 to 1 4m43s Normal ScalingReplicaSet deployment/payload-processing-e2e-authcfg-http-9cdd18 Scaled up replica set payload-processing-e2e-authcfg-http-9cdd18-664b9ddfc from 0 to 1 4m43s Warning FailedCreate replicaset/payload-processing-e2e-authcfg-http-9cdd18-664b9ddfc Error creating: pods "payload-processing-e2e-authcfg-http-9cdd18-664b9ddfc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-9cdd18: serviceaccount "payload-processing-e2e-authcfg-http-9cdd18" not found 4m43s Normal SuccessfulCreate replicaset/payload-processing-e2e-authcfg-http-9cdd18-664b9ddfc Created pod: payload-processing-e2e-authcfg-http-9cdd18-664b9ddfc-56b6l 4m43s Warning FailedCreate replicaset/payload-pre-processing-e2e-authcfg-http-9cdd18-7f54b6fc9f Error creating: pods "payload-pre-processing-e2e-authcfg-http-9cdd18-7f54b6fc9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-9cdd18: serviceaccount "payload-processing-e2e-authcfg-http-9cdd18" not found 4m43s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-authcfg-http-9cdd18-7f54b6fc9f Created pod: payload-pre-processing-e2e-authcfg-http-9cdd18-7f54b6fc9f-9sg44 4m42s Normal Pulled pod/payload-processing-e2e-authcfg-http-9cdd18-664b9ddfc-56b6l Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 4m42s Normal AddedInterface pod/payload-processing-e2e-authcfg-http-9cdd18-664b9ddfc-56b6l Add eth0 [10.132.1.32/23] from ovn-kubernetes 4m42s Normal Started pod/payload-processing-e2e-authcfg-http-9cdd18-664b9ddfc-56b6l Container started 4m42s Normal Created pod/payload-processing-e2e-authcfg-http-9cdd18-664b9ddfc-56b6l Container created 4m42s Normal Created pod/payload-pre-processing-e2e-authcfg-http-9cdd18-7f54b6fc9f-9sg44 Container created 4m42s Normal Pulled pod/payload-pre-processing-e2e-authcfg-http-9cdd18-7f54b6fc9f-9sg44 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 4m42s Normal AddedInterface pod/payload-pre-processing-e2e-authcfg-http-9cdd18-7f54b6fc9f-9sg44 Add eth0 [10.132.1.33/23] from ovn-kubernetes 4m42s Normal Started pod/payload-pre-processing-e2e-authcfg-http-9cdd18-7f54b6fc9f-9sg44 Container started 4m30s Normal Pulled pod/e2e-authcfg-http-9cdd18-openshift-default-575d9dcd85-h972c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 4m30s Normal Created pod/e2e-authcfg-http-9cdd18-openshift-default-575d9dcd85-h972c Container created 4m30s Normal Started pod/e2e-authcfg-http-9cdd18-openshift-default-575d9dcd85-h972c Container started 4m30s Warning ProbeError pod/e2e-authcfg-http-9cdd18-openshift-default-575d9dcd85-h972c Readiness probe error: Get "http://10.132.1.30:15021/healthz/ready": read tcp 10.132.0.2:35092->10.132.1.30:15021: read: connection reset by peer... 4m30s Warning Unhealthy pod/e2e-authcfg-http-9cdd18-openshift-default-575d9dcd85-h972c Readiness probe failed: Get "http://10.132.1.30:15021/healthz/ready": read tcp 10.132.0.2:35092->10.132.1.30:15021: read: connection reset by peer 4m24s Warning Unhealthy pod/e2e-authcfg-http-9cdd18-openshift-default-575d9dcd85-h972c Startup probe failed: Get "http://10.132.1.30:15021/healthz/ready": dial tcp 10.132.1.30:15021: connect: connection refused 4m24s Warning ProbeError pod/e2e-authcfg-http-9cdd18-openshift-default-575d9dcd85-h972c Startup probe error: Get "http://10.132.1.30:15021/healthz/ready": dial tcp 10.132.1.30:15021: connect: connection refused... 4m18s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-9cdd18-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 4m18s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-9cdd18-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 3m48s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-9cdd18-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 3m48s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-9cdd18-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 2m59s Normal Killing pod/payload-pre-processing-e2e-authcfg-http-9cdd18-7f54b6fc9f-9sg44 Stopping container payload-pre-processing 2m59s Normal Killing pod/payload-processing-e2e-authcfg-http-9cdd18-664b9ddfc-56b6l Stopping container payload-processing 2m46s Normal Killing pod/e2e-authcfg-http-9cdd18-openshift-default-575d9dcd85-h972c Stopping container istio-proxy 2m32s Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-d8sgs Readiness probe failed: Get "http://10.134.0.150:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m32s Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-d8sgs Readiness probe error: Get "http://10.134.0.150:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 107s Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-p2flk Readiness probe failed: Get "http://10.132.1.9:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 107s Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-p2flk Readiness probe error: Get "http://10.132.1.9:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 106s Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-tbk5w Readiness probe error: Get "http://10.133.1.149:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 106s Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-tbk5w Readiness probe failed: Get "http://10.133.1.149:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 106s Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-jp9lg Readiness probe failed: Get "http://10.132.1.11:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 106s Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-h8mv5 Readiness probe failed: Get "http://10.133.1.148:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 106s Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-h8mv5 Readiness probe error: Get "http://10.133.1.148:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 106s Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-7qjgv Readiness probe error: Get "http://10.132.1.12:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 106s Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-7qjgv Readiness probe failed: Get "http://10.132.1.12:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 106s Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-v8xjj Readiness probe error: Get "http://10.132.1.8:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 106s Warning Unhealthy pod/maas-default-gateway-openshift-default-6db586dd6c-v8xjj Readiness probe failed: Get "http://10.132.1.8:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 106s Warning ProbeError pod/maas-default-gateway-openshift-default-6db586dd6c-jp9lg Readiness probe error: Get "http://10.132.1.11:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)...