=== events in openshift-ingress (2026-11-01T21:43:21Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 45m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w4-a9b439-76df769c99-g6df9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-a9b439-76df769c99-g6df9 to ip-10-0-141-35.us-west-1.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-api-a-b27e66-5db66b9467-z8rv7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-b27e66-5db66b9467-z8rv7 to ip-10-0-141-35.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lrl8n Successfully assigned openshift-ingress/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lrl8n to ip-10-0-131-188.us-west-1.compute.internal 58m Normal Scheduled pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lgsnj Successfully assigned openshift-ingress/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lgsnj to ip-10-0-133-242.us-west-1.compute.internal 63m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-zndkb Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-zndkb to ip-10-0-141-35.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lbcsf Successfully assigned openshift-ingress/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lbcsf to ip-10-0-141-35.us-west-1.compute.internal 56m Normal Scheduled pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-2bb5k Successfully assigned openshift-ingress/e2e-worker-w1-db0157-openshift-default-68856dc5c7-2bb5k to ip-10-0-131-188.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-jlkc9 Successfully assigned openshift-ingress/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-jlkc9 to ip-10-0-133-242.us-west-1.compute.internal 50m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-z4sdv Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-z4sdv to ip-10-0-141-35.us-west-1.compute.internal 8m54s Normal Scheduled pod/e2e-ait-49c82ebd-openshift-default-678b9f6cdc-mzsk8 Successfully assigned openshift-ingress/e2e-ait-49c82ebd-openshift-default-678b9f6cdc-mzsk8 to ip-10-0-141-35.us-west-1.compute.internal 56m Normal Scheduled pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-5bxs8 Successfully assigned openshift-ingress/e2e-worker-w1-db0157-openshift-default-68856dc5c7-5bxs8 to ip-10-0-133-242.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-gx9rt Successfully assigned openshift-ingress/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-gx9rt to ip-10-0-133-242.us-west-1.compute.internal 57m Normal Scheduled pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-brks2 Successfully assigned openshift-ingress/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-brks2 to ip-10-0-131-188.us-west-1.compute.internal 55m Normal Scheduled pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-6wrqk Successfully assigned openshift-ingress/e2e-worker-w1-db0157-openshift-default-68856dc5c7-6wrqk to ip-10-0-141-35.us-west-1.compute.internal 62m Normal Scheduled pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-blcnw Successfully assigned openshift-ingress/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-blcnw to ip-10-0-141-35.us-west-1.compute.internal 58m Normal Scheduled pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-784tj Successfully assigned openshift-ingress/e2e-worker-w1-db0157-openshift-default-68856dc5c7-784tj to ip-10-0-131-188.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-b9n9v Successfully assigned openshift-ingress/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-b9n9v to ip-10-0-131-188.us-west-1.compute.internal 56m Normal Scheduled pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-cc24r Successfully assigned openshift-ingress/e2e-worker-w1-db0157-openshift-default-68856dc5c7-cc24r to ip-10-0-141-35.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-9nlfq Successfully assigned openshift-ingress/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-9nlfq to ip-10-0-141-35.us-west-1.compute.internal 62m Normal Scheduled pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-gkhjr Successfully assigned openshift-ingress/e2e-worker-w1-db0157-openshift-default-68856dc5c7-gkhjr to ip-10-0-141-35.us-west-1.compute.internal 58m Normal Scheduled pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-mwvv2 Successfully assigned openshift-ingress/e2e-worker-w1-db0157-openshift-default-68856dc5c7-mwvv2 to ip-10-0-133-242.us-west-1.compute.internal 18m Normal Scheduled pod/e2e-worker-single-main-7249f4-openshift-default-5d6946dbbf95j6n Successfully assigned openshift-ingress/e2e-worker-single-main-7249f4-openshift-default-5d6946dbbf95j6n to ip-10-0-141-35.us-west-1.compute.internal 13m Normal Scheduled pod/e2e-worker-single-main-236f0f-openshift-default-689d7dd8c7jnbkk Successfully assigned openshift-ingress/e2e-worker-single-main-236f0f-openshift-default-689d7dd8c7jnbkk to ip-10-0-141-35.us-west-1.compute.internal 69m Normal Scheduled pod/openshift-ai-inference-openshift-ai-inference-bc8769f9-gvjgq Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-bc8769f9-gvjgq to ip-10-0-133-242.us-west-1.compute.internal 43m Normal Scheduled pod/e2e-shared-b-w4-dab736-openshift-default-5d5d779c5f-tcsxq Successfully assigned openshift-ingress/e2e-shared-b-w4-dab736-openshift-default-5d5d779c5f-tcsxq to ip-10-0-141-35.us-west-1.compute.internal 63m Normal Scheduled pod/payload-pre-processing-7b98bb678c-jdsm2 Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-jdsm2 to ip-10-0-131-188.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-ait-74ab02fe-openshift-default-577bf6d567-wqvkg Successfully assigned openshift-ingress/e2e-ait-74ab02fe-openshift-default-577bf6d567-wqvkg to ip-10-0-141-35.us-west-1.compute.internal 44m Normal Scheduled pod/e2e-shared-b-w3-bf3fa2-openshift-default-69d7c5c975-84bmk Successfully assigned openshift-ingress/e2e-shared-b-w3-bf3fa2-openshift-default-69d7c5c975-84bmk to ip-10-0-141-35.us-west-1.compute.internal 55m Normal Scheduled pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-psptj Successfully assigned openshift-ingress/e2e-worker-w1-db0157-openshift-default-68856dc5c7-psptj to ip-10-0-141-35.us-west-1.compute.internal 8m46s Normal Scheduled pod/payload-pre-processing-e2e-ait-49c82ebd-649fbdcc87-7rzw2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-49c82ebd-649fbdcc87-7rzw2 to ip-10-0-141-35.us-west-1.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-ait-74ab02fe-78888c8555-t9shv Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-74ab02fe-78888c8555-t9shv to ip-10-0-141-35.us-west-1.compute.internal 55m Normal Scheduled pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-xqm2p Successfully assigned openshift-ingress/e2e-worker-w1-db0157-openshift-default-68856dc5c7-xqm2p to ip-10-0-133-242.us-west-1.compute.internal 48m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-qg2c2 Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qg2c2 to ip-10-0-133-242.us-west-1.compute.internal 51m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-kmd5t Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-kmd5t to ip-10-0-131-188.us-west-1.compute.internal 48m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-gq6xw Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gq6xw to ip-10-0-141-35.us-west-1.compute.internal 69m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-z2jrj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-z2jrj to ip-10-0-133-242.us-west-1.compute.internal 55m Normal Scheduled pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-z5s97 Successfully assigned openshift-ingress/e2e-worker-w1-db0157-openshift-default-68856dc5c7-z5s97 to ip-10-0-131-188.us-west-1.compute.internal 45m Normal Scheduled pod/e2e-shared-b-w1-df700a-openshift-default-6cf7fb69dc-dg822 Successfully assigned openshift-ingress/e2e-shared-b-w1-df700a-openshift-default-6cf7fb69dc-dg822 to ip-10-0-141-35.us-west-1.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-ait-a0ef08b4-579f9b795-ffw68 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-a0ef08b4-579f9b795-ffw68 to ip-10-0-131-188.us-west-1.compute.internal 63m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-wwpvf Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-wwpvf to ip-10-0-141-35.us-west-1.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-ait-f955d283-6798d4f4dc-b22zs Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-f955d283-6798d4f4dc-b22zs to ip-10-0-141-35.us-west-1.compute.internal 43m Normal Scheduled pod/e2e-shared-b-w0-29baa8-openshift-default-84786bd7d7-q544h Successfully assigned openshift-ingress/e2e-shared-b-w0-29baa8-openshift-default-84786bd7d7-q544h to ip-10-0-141-35.us-west-1.compute.internal 66m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-4bw98 Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4bw98 to ip-10-0-141-35.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-ait-a0ef08b4-openshift-default-58c89679c-qf9jq Successfully assigned openshift-ingress/e2e-ait-a0ef08b4-openshift-default-58c89679c-qf9jq to ip-10-0-131-188.us-west-1.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-api-a-22ed7b-57f689dcd7-8vspb Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-22ed7b-57f689dcd7-8vspb to ip-10-0-141-35.us-west-1.compute.internal 67m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-c574b7b6f-4blzt Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-c574b7b6f-4blzt to ip-10-0-133-242.us-west-1.compute.internal 45m Normal Scheduled pod/e2e-shared-a-w4-a9b439-openshift-default-77b865d7d7-cfn9m Successfully assigned openshift-ingress/e2e-shared-a-w4-a9b439-openshift-default-77b865d7d7-cfn9m to ip-10-0-141-35.us-west-1.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-api-a-dd840a-5c9b4679f5-wspqf Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-dd840a-5c9b4679f5-wspqf to ip-10-0-141-35.us-west-1.compute.internal 63m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-vz6nd Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-vz6nd to ip-10-0-141-35.us-west-1.compute.internal 2m23s Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-vmxr9 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-vmxr9 to ip-10-0-133-242.us-west-1.compute.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-api-a-deff5e-77db945ff9-jtzp7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-deff5e-77db945ff9-jtzp7 to ip-10-0-141-35.us-west-1.compute.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-api-a-e0a6f5-68b97cb76c-sm88c Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-e0a6f5-68b97cb76c-sm88c to ip-10-0-141-35.us-west-1.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-api-b-670df9-6b57976fb7-rgk2d Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-670df9-6b57976fb7-rgk2d to ip-10-0-141-35.us-west-1.compute.internal 57m Normal Scheduled pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-2qv5b Successfully assigned openshift-ingress/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-2qv5b to ip-10-0-141-35.us-west-1.compute.internal 69m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-q97t2 Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-q97t2 to ip-10-0-131-188.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-shared-a-w3-24f923-openshift-default-645f69b989-dhxtq Successfully assigned openshift-ingress/e2e-shared-a-w3-24f923-openshift-default-645f69b989-dhxtq to ip-10-0-131-188.us-west-1.compute.internal 67m Normal Scheduled pod/kube-auth-proxy-85b9bf4598-hm78q Successfully assigned openshift-ingress/kube-auth-proxy-85b9bf4598-hm78q to ip-10-0-131-188.us-west-1.compute.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-api-b-a41242-5956bf7979-dsqpz Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-a41242-5956bf7979-dsqpz to ip-10-0-141-35.us-west-1.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-api-b-d08fd0-84d7f997dc-6zrkr Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-d08fd0-84d7f997dc-6zrkr to ip-10-0-141-35.us-west-1.compute.internal 57m Normal Scheduled pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-49zgd Successfully assigned openshift-ingress/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-49zgd to ip-10-0-131-188.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-ait-f955d283-openshift-default-775bfb5f57-x79dv Successfully assigned openshift-ingress/e2e-ait-f955d283-openshift-default-775bfb5f57-x79dv to ip-10-0-141-35.us-west-1.compute.internal 7m44s Normal Scheduled pod/payload-pre-processing-e2e-authcfg-http-e23c3b-75f45789f9-rwh6w Successfully assigned openshift-ingress/payload-pre-processing-e2e-authcfg-http-e23c3b-75f45789f9-rwh6w to ip-10-0-141-35.us-west-1.compute.internal 47m Normal Scheduled pod/e2e-shared-a-w1-dce296-openshift-default-84fd9bf7ff-htbpm Successfully assigned openshift-ingress/e2e-shared-a-w1-dce296-openshift-default-84fd9bf7ff-htbpm to ip-10-0-131-188.us-west-1.compute.internal 67m Normal Scheduled pod/kube-auth-proxy-85b9bf4598-mg8rp Successfully assigned openshift-ingress/kube-auth-proxy-85b9bf4598-mg8rp to ip-10-0-141-35.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-65cf4 Successfully assigned openshift-ingress/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-65cf4 to ip-10-0-131-188.us-west-1.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-derive-d0d57533-8598cb8b99-n7sv6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-d0d57533-8598cb8b99-n7sv6 to ip-10-0-141-35.us-west-1.compute.internal 56m Normal Scheduled pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-hlgfg Successfully assigned openshift-ingress/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-hlgfg to ip-10-0-131-188.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-nrzth Successfully assigned openshift-ingress/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-nrzth to ip-10-0-133-242.us-west-1.compute.internal 56m Normal Scheduled pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-sh9p6 Successfully assigned openshift-ingress/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-sh9p6 to ip-10-0-133-242.us-west-1.compute.internal 62m Normal Scheduled pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-tsxlv Successfully assigned openshift-ingress/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-tsxlv to ip-10-0-141-35.us-west-1.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-081c8e-9fdd659b7-8w2gx Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-081c8e-9fdd659b7-8w2gx to ip-10-0-131-188.us-west-1.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-3ab520-848dc5bd6c-k9zhf Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-3ab520-848dc5bd6c-k9zhf to ip-10-0-141-35.us-west-1.compute.internal 45m Normal Scheduled pod/e2e-shared-a-w0-de8145-openshift-default-57f9f8965-mpzk6 Successfully assigned openshift-ingress/e2e-shared-a-w0-de8145-openshift-default-57f9f8965-mpzk6 to ip-10-0-141-35.us-west-1.compute.internal 48m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-rfb2k Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-rfb2k to ip-10-0-133-242.us-west-1.compute.internal 58m Normal Scheduled pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-ttwc5 Successfully assigned openshift-ingress/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-ttwc5 to ip-10-0-133-242.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-mfk4k Successfully assigned openshift-ingress/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-mfk4k to ip-10-0-133-242.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-xzjrw Successfully assigned openshift-ingress/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-xzjrw to ip-10-0-141-35.us-west-1.compute.internal 41m Normal Scheduled pod/e2e-api-a-22ed7b-openshift-default-7fd8fd8b6f-twm7j Successfully assigned openshift-ingress/e2e-api-a-22ed7b-openshift-default-7fd8fd8b6f-twm7j to ip-10-0-133-242.us-west-1.compute.internal 72m Normal Scheduled pod/router-default-66ff46cf4c-rt6qk Successfully assigned openshift-ingress/router-default-66ff46cf4c-rt6qk to ip-10-0-131-188.us-west-1.compute.internal 73m Warning FailedScheduling pod/router-default-66ff46cf4c-rt6qk no nodes available to schedule pods 79m Warning FailedScheduling pod/router-default-66ff46cf4c-rt6qk no nodes available to schedule pods 46m Normal Scheduled pod/e2e-mt-a5ab4417-openshift-default-86cdc589-kztdw Successfully assigned openshift-ingress/e2e-mt-a5ab4417-openshift-default-86cdc589-kztdw to ip-10-0-131-188.us-west-1.compute.internal 50m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-qzw6j Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-qzw6j to ip-10-0-133-242.us-west-1.compute.internal 34m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-qswqs Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-qswqs to ip-10-0-141-35.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-mt-a5ab4417-openshift-default-86cdc589-g2rzx Successfully assigned openshift-ingress/e2e-mt-a5ab4417-openshift-default-86cdc589-g2rzx to ip-10-0-133-242.us-west-1.compute.internal 2m23s Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-qrbk7 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-qrbk7 to ip-10-0-141-35.us-west-1.compute.internal 34m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-qps6t Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-qps6t to ip-10-0-131-188.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-czlxh Successfully assigned openshift-ingress/e2e-worker-w3-944263-openshift-default-5f78997bc9-czlxh to ip-10-0-141-35.us-west-1.compute.internal 59m Normal Scheduled pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-dhjgg Successfully assigned openshift-ingress/e2e-worker-w3-944263-openshift-default-5f78997bc9-dhjgg to ip-10-0-133-242.us-west-1.compute.internal 61m Normal Scheduled pod/payload-processing-e2e-worker-w6-dd038b-58bf969f7c-5tbbt Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-dd038b-58bf969f7c-5tbbt to ip-10-0-141-35.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-mt-a5ab4417-openshift-default-86cdc589-9tm7w Successfully assigned openshift-ingress/e2e-mt-a5ab4417-openshift-default-86cdc589-9tm7w to ip-10-0-131-188.us-west-1.compute.internal 57m Normal Scheduled pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-fr9fj Successfully assigned openshift-ingress/e2e-worker-w3-944263-openshift-default-5f78997bc9-fr9fj to ip-10-0-131-188.us-west-1.compute.internal 45m Normal Scheduled pod/e2e-api-a-b27e66-openshift-default-5cccc745f7-nkvtl Successfully assigned openshift-ingress/e2e-api-a-b27e66-openshift-default-5cccc745f7-nkvtl to ip-10-0-141-35.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-mt-a5ab4417-openshift-default-86cdc589-8hn7h Successfully assigned openshift-ingress/e2e-mt-a5ab4417-openshift-default-86cdc589-8hn7h to ip-10-0-141-35.us-west-1.compute.internal 50m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-qnwxz Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-qnwxz to ip-10-0-131-188.us-west-1.compute.internal 24m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-qm8qv Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-qm8qv to ip-10-0-141-35.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-gjkgx Successfully assigned openshift-ingress/e2e-worker-w3-944263-openshift-default-5f78997bc9-gjkgx to ip-10-0-141-35.us-west-1.compute.internal 56m Normal Scheduled pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-gkcth Successfully assigned openshift-ingress/e2e-worker-w3-944263-openshift-default-5f78997bc9-gkcth to ip-10-0-131-188.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-mt-a5ab4417-openshift-default-86cdc589-6nzw2 Successfully assigned openshift-ingress/e2e-mt-a5ab4417-openshift-default-86cdc589-6nzw2 to ip-10-0-141-35.us-west-1.compute.internal 61m Normal Scheduled pod/payload-processing-e2e-worker-w5-c9ae21-6667d54cf5-twgq6 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-c9ae21-6667d54cf5-twgq6 to ip-10-0-141-35.us-west-1.compute.internal 62m Normal Scheduled pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-kndl9 Successfully assigned openshift-ingress/e2e-worker-w3-944263-openshift-default-5f78997bc9-kndl9 to ip-10-0-141-35.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-zttrt Successfully assigned openshift-ingress/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-zttrt to ip-10-0-141-35.us-west-1.compute.internal 46m Normal Scheduled pod/e2e-mt-a5ab4417-openshift-default-86cdc589-6b97v Successfully assigned openshift-ingress/e2e-mt-a5ab4417-openshift-default-86cdc589-6b97v to ip-10-0-133-242.us-west-1.compute.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-6b9305-864cc97d9f-dcsdn Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-6b9305-864cc97d9f-dcsdn to ip-10-0-141-35.us-west-1.compute.internal 33m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-044440-79875748f5-mwv7c Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-044440-79875748f5-mwv7c to ip-10-0-141-35.us-west-1.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-7091fd-6d79589dd7-mqf74 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-7091fd-6d79589dd7-mqf74 to ip-10-0-141-35.us-west-1.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-migrate-8ddfb06d-66db94bbb7-2fxht Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-8ddfb06d-66db94bbb7-2fxht to ip-10-0-141-35.us-west-1.compute.internal 42m Normal Scheduled pod/e2e-api-a-dd840a-openshift-default-54cd84677f-dvjrq Successfully assigned openshift-ingress/e2e-api-a-dd840a-openshift-default-54cd84677f-dvjrq to ip-10-0-131-188.us-west-1.compute.internal 56m Normal Scheduled pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-l7fxr Successfully assigned openshift-ingress/e2e-worker-w3-944263-openshift-default-5f78997bc9-l7fxr to ip-10-0-141-35.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-lxjj4 Successfully assigned openshift-ingress/e2e-worker-w3-944263-openshift-default-5f78997bc9-lxjj4 to ip-10-0-133-242.us-west-1.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-mt-11a069b1-6977f77fd7-kh7wl Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-11a069b1-6977f77fd7-kh7wl to ip-10-0-131-188.us-west-1.compute.internal 63m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-ppfz4 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-ppfz4 to ip-10-0-131-188.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-np65v Successfully assigned openshift-ingress/e2e-worker-w3-944263-openshift-default-5f78997bc9-np65v to ip-10-0-131-188.us-west-1.compute.internal 35m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-pg8rw Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-pg8rw to ip-10-0-133-242.us-west-1.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-mt-11a069b1-6977f77fd7-lvtg7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-11a069b1-6977f77fd7-lvtg7 to ip-10-0-131-188.us-west-1.compute.internal 38m Normal Scheduled pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-zqwvb Successfully assigned openshift-ingress/e2e-mt-11a069b1-openshift-default-7d5dd8f549-zqwvb to ip-10-0-131-188.us-west-1.compute.internal 56m Normal Scheduled pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-v5s7k Successfully assigned openshift-ingress/e2e-worker-w3-944263-openshift-default-5f78997bc9-v5s7k to ip-10-0-133-242.us-west-1.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-mt-a5ab4417-5677fb94bc-mg6v2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-a5ab4417-5677fb94bc-mg6v2 to ip-10-0-131-188.us-west-1.compute.internal 38m Normal Scheduled pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-xcsv8 Successfully assigned openshift-ingress/e2e-mt-11a069b1-openshift-default-7d5dd8f549-xcsv8 to ip-10-0-141-35.us-west-1.compute.internal 62m Normal Scheduled pod/payload-processing-e2e-worker-w3-944263-5cf85467cc-sl4cb Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-944263-5cf85467cc-sl4cb to ip-10-0-141-35.us-west-1.compute.internal 50m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-pfnvk Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-pfnvk to ip-10-0-133-242.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-vx9f5 Successfully assigned openshift-ingress/e2e-mt-11a069b1-openshift-default-7d5dd8f549-vx9f5 to ip-10-0-133-242.us-west-1.compute.internal 37m Normal Scheduled pod/e2e-api-a-deff5e-openshift-default-589bfb887f-hnvnf Successfully assigned openshift-ingress/e2e-api-a-deff5e-openshift-default-589bfb887f-hnvnf to ip-10-0-133-242.us-west-1.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w0-de8145-56fb74df5-8nbgw Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-de8145-56fb74df5-8nbgw to ip-10-0-141-35.us-west-1.compute.internal 38m Normal Scheduled pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-jswz2 Successfully assigned openshift-ingress/e2e-mt-11a069b1-openshift-default-7d5dd8f549-jswz2 to ip-10-0-131-188.us-west-1.compute.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w1-dce296-65f5556975-m2d7v Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dce296-65f5556975-m2d7v to ip-10-0-141-35.us-west-1.compute.internal 62m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-pcwvt Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-pcwvt to ip-10-0-133-242.us-west-1.compute.internal 38m Normal Scheduled pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-gkl98 Successfully assigned openshift-ingress/e2e-mt-11a069b1-openshift-default-7d5dd8f549-gkl98 to ip-10-0-141-35.us-west-1.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w3-24f923-869bcbd967-w25fl Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-24f923-869bcbd967-w25fl to ip-10-0-141-35.us-west-1.compute.internal 62m Normal Scheduled pod/payload-processing-e2e-worker-w2-cd248f-6c9889fc9c-wbd52 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-cd248f-6c9889fc9c-wbd52 to ip-10-0-141-35.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-8vhd5 Successfully assigned openshift-ingress/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-8vhd5 to ip-10-0-131-188.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-b95tc Successfully assigned openshift-ingress/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-b95tc to ip-10-0-141-35.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-m75bn Successfully assigned openshift-ingress/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-m75bn to ip-10-0-141-35.us-west-1.compute.internal 56m Normal Scheduled pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-nsqsb Successfully assigned openshift-ingress/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-nsqsb to ip-10-0-141-35.us-west-1.compute.internal 61m Normal Scheduled pod/payload-processing-e2e-worker-w4-1a1e9e-66d4f8c5f-cr9fp Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-1a1e9e-66d4f8c5f-cr9fp to ip-10-0-141-35.us-west-1.compute.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-4ac1ef-db4769dc-ffkcl Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-4ac1ef-db4769dc-ffkcl to ip-10-0-141-35.us-west-1.compute.internal 38m Normal Scheduled pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-85qkw Successfully assigned openshift-ingress/e2e-mt-11a069b1-openshift-default-7d5dd8f549-85qkw to ip-10-0-133-242.us-west-1.compute.internal 38m Normal Scheduled pod/e2e-api-a-e0a6f5-openshift-default-5b74475bdc-rrrbc Successfully assigned openshift-ingress/e2e-api-a-e0a6f5-openshift-default-5b74475bdc-rrrbc to ip-10-0-141-35.us-west-1.compute.internal 61m Normal Scheduled pod/payload-processing-e2e-worker-w1-db0157-654558c4b7-vsbx9 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-db0157-654558c4b7-vsbx9 to ip-10-0-141-35.us-west-1.compute.internal 48m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-nd97z Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-nd97z to ip-10-0-131-188.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-tb6nw Successfully assigned openshift-ingress/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-tb6nw to ip-10-0-141-35.us-west-1.compute.internal 35m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-m6rnq Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-m6rnq to ip-10-0-133-242.us-west-1.compute.internal 35m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-k7ndz Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-k7ndz to ip-10-0-133-242.us-west-1.compute.internal 25m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-hdvw9 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-hdvw9 to ip-10-0-141-35.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vcdwm Successfully assigned openshift-ingress/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vcdwm to ip-10-0-133-242.us-west-1.compute.internal 24m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-gnfzl Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-gnfzl to ip-10-0-133-242.us-west-1.compute.internal 61m Normal Scheduled pod/payload-processing-e2e-worker-w0-208f2b-7fb7755ddc-lx76l Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-208f2b-7fb7755ddc-lx76l to ip-10-0-141-35.us-west-1.compute.internal 18m Normal Scheduled pod/payload-processing-e2e-worker-single-main-7249f4-7568bd48ff597s Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-7249f4-7568bd48ff597s to ip-10-0-131-188.us-west-1.compute.internal 13m Normal Scheduled pod/payload-processing-e2e-worker-single-main-236f0f-58779bccbs4r97 Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-236f0f-58779bccbs4r97 to ip-10-0-131-188.us-west-1.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-shared-b-w4-dab736-5b68f8cd7c-947ht Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-dab736-5b68f8cd7c-947ht to ip-10-0-141-35.us-west-1.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-shared-b-w3-bf3fa2-79cb8b8f-w5wfb Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-bf3fa2-79cb8b8f-w5wfb to ip-10-0-141-35.us-west-1.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-shared-b-w1-df700a-788d57d789-n6jlx Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w1-df700a-788d57d789-n6jlx to ip-10-0-141-35.us-west-1.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-shared-b-w0-29baa8-6bbff57b65-6djp8 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-29baa8-6bbff57b65-6djp8 to ip-10-0-141-35.us-west-1.compute.internal 35m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-dggk5 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-dggk5 to ip-10-0-141-35.us-west-1.compute.internal 34m Normal Scheduled pod/e2e-api-b-670df9-openshift-default-6c5f7449f5-zvvdk Successfully assigned openshift-ingress/e2e-api-b-670df9-openshift-default-6c5f7449f5-zvvdk to ip-10-0-141-35.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vnzfv Successfully assigned openshift-ingress/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vnzfv to ip-10-0-133-242.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-wh67n Successfully assigned openshift-ingress/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-wh67n to ip-10-0-131-188.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-tt6hz Successfully assigned openshift-ingress/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-tt6hz to ip-10-0-141-35.us-west-1.compute.internal 62m Normal Scheduled pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-w8rd7 Successfully assigned openshift-ingress/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-w8rd7 to ip-10-0-131-188.us-west-1.compute.internal 48m Normal Scheduled pod/e2e-migrate-8ddfb06d-gw-openshift-default-7b966bd4bf-88hxr Successfully assigned openshift-ingress/e2e-migrate-8ddfb06d-gw-openshift-default-7b966bd4bf-88hxr to ip-10-0-141-35.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-wmg5x Successfully assigned openshift-ingress/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-wmg5x to ip-10-0-131-188.us-west-1.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-shared-a-w4-a9b439-76bf5ff49-qqkwc Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-a9b439-76bf5ff49-qqkwc to ip-10-0-141-35.us-west-1.compute.internal 53m Normal Scheduled pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-sbntr Successfully assigned openshift-ingress/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-sbntr to ip-10-0-133-242.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-lsbvv Successfully assigned openshift-ingress/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-lsbvv to ip-10-0-141-35.us-west-1.compute.internal 63m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-22bjk Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-22bjk to ip-10-0-133-242.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-kdwp6 Successfully assigned openshift-ingress/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-kdwp6 to ip-10-0-133-242.us-west-1.compute.internal 55m Normal Scheduled pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-zjkp8 Successfully assigned openshift-ingress/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-zjkp8 to ip-10-0-133-242.us-west-1.compute.internal 48m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-cxh5l Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-cxh5l to ip-10-0-133-242.us-west-1.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w0-29baa8-6c565db695-5m42b Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-29baa8-6c565db695-5m42b to ip-10-0-141-35.us-west-1.compute.internal 35m Normal Scheduled pod/e2e-api-b-a41242-openshift-default-86bb88f59-hn6s2 Successfully assigned openshift-ingress/e2e-api-b-a41242-openshift-default-86bb88f59-hn6s2 to ip-10-0-141-35.us-west-1.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-shared-a-w3-24f923-8474d84f5-kfvdm Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-24f923-8474d84f5-kfvdm to ip-10-0-141-35.us-west-1.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w1-df700a-789b6c9547-g9ps7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w1-df700a-789b6c9547-g9ps7 to ip-10-0-141-35.us-west-1.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w3-bf3fa2-5f4bb58487-jrpbx Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-bf3fa2-5f4bb58487-jrpbx to ip-10-0-141-35.us-west-1.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w4-dab736-7b55d47c59-lrvjr Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-dab736-7b55d47c59-lrvjr to ip-10-0-141-35.us-west-1.compute.internal 13m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-236f0f-59cf6t5j8h Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-236f0f-59cf6t5j8h to ip-10-0-141-35.us-west-1.compute.internal 18m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-7249f4-544589757t Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-7249f4-544589757t to ip-10-0-141-35.us-west-1.compute.internal 61m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-208f2b-76b7b8c9f-fxd8f Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-208f2b-76b7b8c9f-fxd8f to ip-10-0-141-35.us-west-1.compute.internal 56m Normal Scheduled pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-jwfhk Successfully assigned openshift-ingress/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-jwfhk to ip-10-0-141-35.us-west-1.compute.internal 47m Normal Scheduled pod/payload-processing-e2e-shared-a-w1-dce296-85d75dbb6f-rq556 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w1-dce296-85d75dbb6f-rq556 to ip-10-0-141-35.us-west-1.compute.internal 61m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-db0157-6689cd59bc-nd7kc Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-db0157-6689cd59bc-nd7kc to ip-10-0-141-35.us-west-1.compute.internal 30m Normal Scheduled pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-vpbgp Successfully assigned openshift-ingress/e2e-inf-b-7091fd-openshift-default-56c9644bc-vpbgp to ip-10-0-133-242.us-west-1.compute.internal 31m Normal Scheduled pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-qk7ll Successfully assigned openshift-ingress/e2e-inf-b-7091fd-openshift-default-56c9644bc-qk7ll to ip-10-0-131-188.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-hdc42 Successfully assigned openshift-ingress/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-hdc42 to ip-10-0-131-188.us-west-1.compute.internal 62m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-cd248f-7b8875bf8c-c8fvd Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-cd248f-7b8875bf8c-c8fvd to ip-10-0-131-188.us-west-1.compute.internal 25m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-2n5h6 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-2n5h6 to ip-10-0-141-35.us-west-1.compute.internal 63m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-2zkzj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-2zkzj to ip-10-0-131-188.us-west-1.compute.internal 42m Normal Scheduled pod/e2e-api-b-d08fd0-openshift-default-5fd6b47cd7-28k4z Successfully assigned openshift-ingress/e2e-api-b-d08fd0-openshift-default-5fd6b47cd7-28k4z to ip-10-0-141-35.us-west-1.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-shared-a-w0-de8145-9dcc65c7f-q6cff Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-de8145-9dcc65c7f-q6cff to ip-10-0-141-35.us-west-1.compute.internal 31m Normal Scheduled pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-hnhqh Successfully assigned openshift-ingress/e2e-inf-b-7091fd-openshift-default-56c9644bc-hnhqh to ip-10-0-141-35.us-west-1.compute.internal 62m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-944263-78999d46dc-m8ml8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-944263-78999d46dc-m8ml8 to ip-10-0-141-35.us-west-1.compute.internal 34m Normal Scheduled pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-h5bjz Successfully assigned openshift-ingress/e2e-inf-b-7091fd-openshift-default-56c9644bc-h5bjz to ip-10-0-133-242.us-west-1.compute.internal 61m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-1a1e9e-8fc6dcb7-bs6sm Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-1a1e9e-8fc6dcb7-bs6sm to ip-10-0-141-35.us-west-1.compute.internal 31m Normal Scheduled pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-7b6td Successfully assigned openshift-ingress/e2e-inf-b-7091fd-openshift-default-56c9644bc-7b6td to ip-10-0-141-35.us-west-1.compute.internal 61m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-c9ae21-7bc765fb9f-7tjsn Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-c9ae21-7bc765fb9f-7tjsn to ip-10-0-141-35.us-west-1.compute.internal 62m Normal Scheduled pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-h2t4l Successfully assigned openshift-ingress/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-h2t4l to ip-10-0-131-188.us-west-1.compute.internal 61m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-dd038b-c5cc45b55-nd59r Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-dd038b-c5cc45b55-nd59r to ip-10-0-141-35.us-west-1.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-mt-a5ab4417-6fcc7f8d97-gpp76 Successfully assigned openshift-ingress/payload-processing-e2e-mt-a5ab4417-6fcc7f8d97-gpp76 to ip-10-0-131-188.us-west-1.compute.internal 55m Normal Scheduled pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-dcbhv Successfully assigned openshift-ingress/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-dcbhv to ip-10-0-133-242.us-west-1.compute.internal 34m Normal Scheduled pod/e2e-inf-b-044440-openshift-default-7f6ddf4485-wjzfh Successfully assigned openshift-ingress/e2e-inf-b-044440-openshift-default-7f6ddf4485-wjzfh to ip-10-0-133-242.us-west-1.compute.internal 63m Normal Scheduled pod/payload-processing-55fbbbf9dc-mp5sx Successfully assigned openshift-ingress/payload-processing-55fbbbf9dc-mp5sx to ip-10-0-131-188.us-west-1.compute.internal 8m46s Normal Scheduled pod/payload-processing-e2e-ait-49c82ebd-59b45759bf-hjk4h Successfully assigned openshift-ingress/payload-processing-e2e-ait-49c82ebd-59b45759bf-hjk4h to ip-10-0-141-35.us-west-1.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-ait-74ab02fe-7c85d47c89-2fpz8 Successfully assigned openshift-ingress/payload-processing-e2e-ait-74ab02fe-7c85d47c89-2fpz8 to ip-10-0-141-35.us-west-1.compute.internal 2m8s Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-cfgkf Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-cfgkf to ip-10-0-133-242.us-west-1.compute.internal 7m51s Normal Scheduled pod/e2e-authcfg-http-e23c3b-openshift-default-5b9d4fcbbf-jt6tx Successfully assigned openshift-ingress/e2e-authcfg-http-e23c3b-openshift-default-5b9d4fcbbf-jt6tx to ip-10-0-141-35.us-west-1.compute.internal 36m Normal Scheduled pod/e2e-inf-a-6b9305-openshift-default-74c965c97c-7xs26 Successfully assigned openshift-ingress/e2e-inf-a-6b9305-openshift-default-74c965c97c-7xs26 to ip-10-0-141-35.us-west-1.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-mt-11a069b1-7c85476fd5-nqm4x Successfully assigned openshift-ingress/payload-processing-e2e-mt-11a069b1-7c85476fd5-nqm4x to ip-10-0-131-188.us-west-1.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-mt-11a069b1-7c85476fd5-mlmjd Successfully assigned openshift-ingress/payload-processing-e2e-mt-11a069b1-7c85476fd5-mlmjd to ip-10-0-131-188.us-west-1.compute.internal 25m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-bb8j6 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-bb8j6 to ip-10-0-133-242.us-west-1.compute.internal 2m8s Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-5jn27 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-5jn27 to ip-10-0-141-35.us-west-1.compute.internal 2m8s Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-5t72t Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-5t72t to ip-10-0-131-188.us-west-1.compute.internal 2m8s Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-9hj29 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-9hj29 to ip-10-0-141-35.us-west-1.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-ait-a0ef08b4-7bd7876d5f-jcbw7 Successfully assigned openshift-ingress/payload-processing-e2e-ait-a0ef08b4-7bd7876d5f-jcbw7 to ip-10-0-141-35.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-7rr6b Successfully assigned openshift-ingress/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-7rr6b to ip-10-0-133-242.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-8wf8c Successfully assigned openshift-ingress/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-8wf8c to ip-10-0-133-242.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-8xtk5 Successfully assigned openshift-ingress/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-8xtk5 to ip-10-0-141-35.us-west-1.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-migrate-8ddfb06d-54d7b96ff7-flgn4 Successfully assigned openshift-ingress/payload-processing-e2e-migrate-8ddfb06d-54d7b96ff7-flgn4 to ip-10-0-141-35.us-west-1.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-inf-b-7091fd-98896dbf7-6r5f6 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-7091fd-98896dbf7-6r5f6 to ip-10-0-141-35.us-west-1.compute.internal 33m Normal Scheduled pod/payload-processing-e2e-inf-b-044440-5678b676c-cg7nt Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-044440-5678b676c-cg7nt to ip-10-0-131-188.us-west-1.compute.internal 36m Normal Scheduled pod/payload-processing-e2e-inf-a-6b9305-6964565579-hwlhv Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-6b9305-6964565579-hwlhv to ip-10-0-141-35.us-west-1.compute.internal 37m Normal Scheduled pod/payload-processing-e2e-inf-a-4ac1ef-5bddb89485-dpbt4 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-4ac1ef-5bddb89485-dpbt4 to ip-10-0-141-35.us-west-1.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-inf-a-3ab520-7b7f4ff56c-vszhw Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-3ab520-7b7f4ff56c-vszhw to ip-10-0-141-35.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-derive-d0d57533-openshift-default-bb8564d89-mmmrv Successfully assigned openshift-ingress/e2e-derive-d0d57533-openshift-default-bb8564d89-mmmrv to ip-10-0-131-188.us-west-1.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-ait-f955d283-ddc87bc75-fdqxb Successfully assigned openshift-ingress/payload-processing-e2e-ait-f955d283-ddc87bc75-fdqxb to ip-10-0-141-35.us-west-1.compute.internal 37m Normal Scheduled pod/e2e-inf-a-4ac1ef-openshift-default-54d7ccf455-j89g4 Successfully assigned openshift-ingress/e2e-inf-a-4ac1ef-openshift-default-54d7ccf455-j89g4 to ip-10-0-133-242.us-west-1.compute.internal 2m23s Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-6xvdh Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-6xvdh to ip-10-0-141-35.us-west-1.compute.internal 25m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-6zwld Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-6zwld to ip-10-0-141-35.us-west-1.compute.internal 48m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-7b8v8 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-7b8v8 to ip-10-0-141-35.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-cznb8 Successfully assigned openshift-ingress/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-cznb8 to ip-10-0-141-35.us-west-1.compute.internal 62m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-8jpjl Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-8jpjl to ip-10-0-131-188.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-f7qth Successfully assigned openshift-ingress/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-f7qth to ip-10-0-133-242.us-west-1.compute.internal 57m Normal Scheduled pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-gf6ms Successfully assigned openshift-ingress/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-gf6ms to ip-10-0-141-35.us-west-1.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-api-a-22ed7b-5768c96547-5qpms Successfully assigned openshift-ingress/payload-processing-e2e-api-a-22ed7b-5768c96547-5qpms to ip-10-0-141-35.us-west-1.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-api-a-b27e66-6df49bd599-n7wtm Successfully assigned openshift-ingress/payload-processing-e2e-api-a-b27e66-6df49bd599-n7wtm to ip-10-0-141-35.us-west-1.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-api-a-dd840a-68df564877-hxfz7 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-dd840a-68df564877-hxfz7 to ip-10-0-141-35.us-west-1.compute.internal 36m Normal Scheduled pod/payload-processing-e2e-api-a-deff5e-bd664cbf7-wvr5w Successfully assigned openshift-ingress/payload-processing-e2e-api-a-deff5e-bd664cbf7-wvr5w to ip-10-0-141-35.us-west-1.compute.internal 37m Normal Scheduled pod/payload-processing-e2e-api-a-e0a6f5-56bbd4f6f7-5gtlm Successfully assigned openshift-ingress/payload-processing-e2e-api-a-e0a6f5-56bbd4f6f7-5gtlm to ip-10-0-141-35.us-west-1.compute.internal 41m Normal Scheduled pod/e2e-inf-a-081c8e-openshift-default-8cd89ddf-xx79m Successfully assigned openshift-ingress/e2e-inf-a-081c8e-openshift-default-8cd89ddf-xx79m to ip-10-0-141-35.us-west-1.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-inf-a-081c8e-86f4657b7-wc4tm Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-081c8e-86f4657b7-wc4tm to ip-10-0-131-188.us-west-1.compute.internal 34m Normal Scheduled pod/e2e-inf-a-3ab520-openshift-default-6765b98db9-jrzgk Successfully assigned openshift-ingress/e2e-inf-a-3ab520-openshift-default-6765b98db9-jrzgk to ip-10-0-141-35.us-west-1.compute.internal 62m Normal Scheduled pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-sz2sr Successfully assigned openshift-ingress/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-sz2sr to ip-10-0-131-188.us-west-1.compute.internal 52m Normal Scheduled pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-qjcxh Successfully assigned openshift-ingress/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-qjcxh to ip-10-0-133-242.us-west-1.compute.internal 48m Normal Scheduled pod/maas-default-gateway-openshift-default-58c6585447-82z67 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58c6585447-82z67 to ip-10-0-131-188.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-q5fbc Successfully assigned openshift-ingress/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-q5fbc to ip-10-0-131-188.us-west-1.compute.internal 51m Normal Scheduled pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-hnrhd Successfully assigned openshift-ingress/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-hnrhd to ip-10-0-131-188.us-west-1.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-api-b-670df9-86c5df874f-vm2ck Successfully assigned openshift-ingress/payload-processing-e2e-api-b-670df9-86c5df874f-vm2ck to ip-10-0-131-188.us-west-1.compute.internal 35m Normal Scheduled pod/payload-processing-e2e-api-b-a41242-6bfc8bf88f-cs272 Successfully assigned openshift-ingress/payload-processing-e2e-api-b-a41242-6bfc8bf88f-cs272 to ip-10-0-141-35.us-west-1.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-derive-d0d57533-7b666f8685-dlwfv Successfully assigned openshift-ingress/payload-processing-e2e-derive-d0d57533-7b666f8685-dlwfv to ip-10-0-141-35.us-west-1.compute.internal 7m43s Normal Scheduled pod/payload-processing-e2e-authcfg-http-e23c3b-c8996fd87-qfthz Successfully assigned openshift-ingress/payload-processing-e2e-authcfg-http-e23c3b-c8996fd87-qfthz to ip-10-0-131-188.us-west-1.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-api-b-d08fd0-7c95697f8f-sxbm5 Successfully assigned openshift-ingress/payload-processing-e2e-api-b-d08fd0-7c95697f8f-sxbm5 to ip-10-0-131-188.us-west-1.compute.internal 80m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-66ff46cf4c from 0 to 1 80m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 80m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 79m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 79m Normal SuccessfulCreate replicaset/router-default-66ff46cf4c Created pod: router-default-66ff46cf4c-rt6qk 73m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 72m Warning FailedMount pod/router-default-66ff46cf4c-rt6qk MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 72m Warning FailedMount pod/router-default-66ff46cf4c-rt6qk MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 72m Warning FailedMount pod/router-default-66ff46cf4c-rt6qk MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 72m Warning FailedMount pod/router-default-66ff46cf4c-rt6qk MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 72m Normal Created pod/router-default-66ff46cf4c-rt6qk Container created 72m Normal AddedInterface pod/router-default-66ff46cf4c-rt6qk Add eth0 [10.132.0.18/23] from ovn-kubernetes 72m Normal Started pod/router-default-66ff46cf4c-rt6qk Container started 72m Normal Pulled pod/router-default-66ff46cf4c-rt6qk Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 72m Warning ProbeError pod/router-default-66ff46cf4c-rt6qk Startup probe error: HTTP probe failed with statuscode: 500... 72m Warning Unhealthy pod/router-default-66ff46cf4c-rt6qk Startup probe failed: HTTP probe failed with statuscode: 500 69m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 69m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 69m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-q97t2 69m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-q97t2 Add eth0 [10.132.0.24/23] from ovn-kubernetes 69m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-q97t2 Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 69m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-q97t2 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.057s (4.057s including waiting). Image size: 185476477 bytes. 69m Normal Started pod/istiod-openshift-gateway-7876bfcb74-q97t2 Container started 69m Normal Created pod/istiod-openshift-gateway-7876bfcb74-q97t2 Container created 69m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-q97t2 Readiness probe error: HTTP probe failed with statuscode: 503... 69m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-q97t2 Readiness probe failed: HTTP probe failed with statuscode: 503 69m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-58c6585447 from 0 to 1 69m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-58c6585447 Created pod: maas-default-gateway-openshift-default-58c6585447-z2jrj 69m Normal ScalingReplicaSet deployment/openshift-ai-inference-openshift-ai-inference Scaled up replica set openshift-ai-inference-openshift-ai-inference-bc8769f9 from 0 to 1 69m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-z2jrj Add eth0 [10.133.0.29/23] from ovn-kubernetes 69m Normal Pulling pod/maas-default-gateway-openshift-default-58c6585447-z2jrj Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 69m Normal AddedInterface pod/openshift-ai-inference-openshift-ai-inference-bc8769f9-gvjgq Add eth0 [10.133.0.30/23] from ovn-kubernetes 69m Normal Pulling pod/openshift-ai-inference-openshift-ai-inference-bc8769f9-gvjgq Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 69m Normal SuccessfulCreate replicaset/openshift-ai-inference-openshift-ai-inference-bc8769f9 Created pod: openshift-ai-inference-openshift-ai-inference-bc8769f9-gvjgq 69m Normal EnsuringLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensuring load balancer 69m Normal EnsuredLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensured load balancer 68m Normal Pulled pod/openshift-ai-inference-openshift-ai-inference-bc8769f9-gvjgq Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 10.939s (10.939s including waiting). Image size: 183555333 bytes. 68m Normal Created pod/openshift-ai-inference-openshift-ai-inference-bc8769f9-gvjgq Container created 68m Normal Started pod/openshift-ai-inference-openshift-ai-inference-bc8769f9-gvjgq Container started 68m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-z2jrj Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 11.298s (11.298s including waiting). Image size: 183555333 bytes. 68m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 68m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 68m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 68m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 68m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 68m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 67m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-85b9bf4598 from 0 to 2 67m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-c574b7b6f from 0 to 1 67m Normal SuccessfulCreate replicaset/kube-auth-proxy-85b9bf4598 Created pod: kube-auth-proxy-85b9bf4598-mg8rp 67m Normal AddedInterface pod/kube-auth-proxy-85b9bf4598-hm78q Add eth0 [10.132.0.26/23] from ovn-kubernetes 67m Normal AddedInterface pod/kube-auth-proxy-85b9bf4598-mg8rp Add eth0 [10.134.0.24/23] from ovn-kubernetes 67m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-c574b7b6f Created pod: data-science-gateway-data-science-gateway-class-c574b7b6f-4blzt 67m Normal SuccessfulCreate replicaset/kube-auth-proxy-85b9bf4598 Created pod: kube-auth-proxy-85b9bf4598-hm78q 67m Normal Pulling pod/kube-auth-proxy-85b9bf4598-mg8rp Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 67m Normal Pulled pod/data-science-gateway-data-science-gateway-class-c574b7b6f-4blzt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 67m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-c574b7b6f-4blzt Add eth0 [10.133.0.39/23] from ovn-kubernetes 67m Normal Created pod/data-science-gateway-data-science-gateway-class-c574b7b6f-4blzt Container created 67m Normal Started pod/data-science-gateway-data-science-gateway-class-c574b7b6f-4blzt Container started 67m Normal Pulling pod/kube-auth-proxy-85b9bf4598-hm78q Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 67m Normal Pulled pod/kube-auth-proxy-85b9bf4598-mg8rp Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.97s (2.97s including waiting). Image size: 188904677 bytes. 67m Normal Pulled pod/kube-auth-proxy-85b9bf4598-hm78q Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 3.341s (3.341s including waiting). Image size: 188904677 bytes. 67m Normal Started pod/kube-auth-proxy-85b9bf4598-mg8rp Container started 67m Normal Created pod/kube-auth-proxy-85b9bf4598-hm78q Container created 67m Normal Started pod/kube-auth-proxy-85b9bf4598-hm78q Container started 67m Normal Created pod/kube-auth-proxy-85b9bf4598-mg8rp Container created 67m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 67m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 67m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 67m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 67m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 67m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 66m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-4bw98 66m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 66m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-4bw98 Add eth0 [10.134.0.29/23] from ovn-kubernetes 66m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-4bw98 Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 66m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-4bw98 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.979s (3.979s including waiting). Image size: 185476477 bytes. 66m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-4bw98 Container created 66m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-4bw98 Container started 66m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-4bw98 Readiness probe failed: HTTP probe failed with statuscode: 503 66m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-4bw98 Readiness probe error: HTTP probe failed with statuscode: 503... 66m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 66m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-q97t2 66m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-q97t2 Stopping container discovery 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-z2jrj Container created 65m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-z2jrj Container started 65m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-z2jrj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 65m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-z2jrj Startup probe failed: Get "http://10.133.0.29:15021/healthz/ready": dial tcp 10.133.0.29:15021: connect: connection refused 65m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-z2jrj Startup probe error: Get "http://10.133.0.29:15021/healthz/ready": dial tcp 10.133.0.29:15021: connect: connection refused... 64m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-58c6585447 from 1 to 2 63m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 63m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-zndkb Add eth0 [10.134.0.32/23] from ovn-kubernetes 63m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-58c6585447 Created pod: maas-default-gateway-openshift-default-58c6585447-zndkb 63m Normal Pulling pod/maas-default-gateway-openshift-default-58c6585447-zndkb Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 63m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-zndkb Container started 63m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-zndkb Container created 63m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-zndkb Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 7.361s (7.361s including waiting). Image size: 183555333 bytes. 63m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-zndkb Startup probe failed: Get "http://10.134.0.32:15021/healthz/ready": dial tcp 10.134.0.32:15021: connect: connection refused 63m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-58c6585447 Created pod: maas-default-gateway-openshift-default-58c6585447-ppfz4 63m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-zndkb Startup probe error: Get "http://10.134.0.32:15021/healthz/ready": dial tcp 10.134.0.32:15021: connect: connection refused... 63m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-58c6585447 from 2 to 4 63m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-58c6585447 Created pod: maas-default-gateway-openshift-default-58c6585447-wwpvf 63m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-ppfz4 Add eth0 [10.132.0.40/23] from ovn-kubernetes 63m Normal Pulling pod/maas-default-gateway-openshift-default-58c6585447-ppfz4 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 63m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-wwpvf Container started 63m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-wwpvf Container created 63m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-wwpvf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-wwpvf Add eth0 [10.134.0.33/23] from ovn-kubernetes 63m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-ppfz4 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.68s (3.68s including waiting). Image size: 183555333 bytes. 63m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-ppfz4 Container created 63m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-ppfz4 Container started 63m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-wwpvf Startup probe error: Get "http://10.134.0.33:15021/healthz/ready": dial tcp 10.134.0.33:15021: connect: connection refused... 63m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-wwpvf Startup probe failed: Get "http://10.134.0.33:15021/healthz/ready": dial tcp 10.134.0.33:15021: connect: connection refused 63m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-ppfz4 Startup probe failed: Get "http://10.132.0.40:15021/healthz/ready": dial tcp 10.132.0.40:15021: connect: connection refused 63m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-ppfz4 Startup probe error: Get "http://10.132.0.40:15021/healthz/ready": dial tcp 10.132.0.40:15021: connect: connection refused... 63m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-58c6585447 from 4 to 7 63m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 63m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-58c6585447 Created pod: maas-default-gateway-openshift-default-58c6585447-22bjk 63m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-58c6585447 Created pod: maas-default-gateway-openshift-default-58c6585447-2zkzj 63m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-58c6585447 Created pod: maas-default-gateway-openshift-default-58c6585447-vz6nd 63m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-2zkzj Container started 63m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-2zkzj Add eth0 [10.132.0.42/23] from ovn-kubernetes 63m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-vz6nd Container started 63m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-vz6nd Container created 63m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-2zkzj Container created 63m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-22bjk Add eth0 [10.133.0.51/23] from ovn-kubernetes 63m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-2zkzj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-vz6nd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-vz6nd Add eth0 [10.134.0.35/23] from ovn-kubernetes 63m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-22bjk Container started 63m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-22bjk Container created 63m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-22bjk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 63m Normal SuccessfulCreate replicaset/payload-processing-55fbbbf9dc Created pod: payload-processing-55fbbbf9dc-mp5sx 63m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 63m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-55fbbbf9dc from 0 to 1 63m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-jdsm2 63m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 63m Warning FailedCreate replicaset/payload-processing-55fbbbf9dc Error creating: pods "payload-processing-55fbbbf9dc-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 63m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-jdsm2 Add eth0 [10.132.0.43/23] from ovn-kubernetes 63m Normal Pulling pod/payload-processing-55fbbbf9dc-mp5sx Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 63m Normal Pulling pod/payload-pre-processing-7b98bb678c-jdsm2 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 63m Normal AddedInterface pod/payload-processing-55fbbbf9dc-mp5sx Add eth0 [10.132.0.44/23] from ovn-kubernetes 63m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-22bjk Startup probe failed: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused 63m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-2zkzj Startup probe error: Get "http://10.132.0.42:15021/healthz/ready": dial tcp 10.132.0.42:15021: connect: connection refused... 63m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-22bjk Startup probe error: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused... 63m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-vz6nd Startup probe error: Get "http://10.134.0.35:15021/healthz/ready": dial tcp 10.134.0.35:15021: connect: connection refused... 63m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-vz6nd Startup probe failed: Get "http://10.134.0.35:15021/healthz/ready": dial tcp 10.134.0.35:15021: connect: connection refused 63m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-2zkzj Startup probe failed: Get "http://10.132.0.42:15021/healthz/ready": dial tcp 10.132.0.42:15021: connect: connection refused 63m Normal Created pod/payload-processing-55fbbbf9dc-mp5sx Container created 63m Normal Pulled pod/payload-processing-55fbbbf9dc-mp5sx Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.94s (2.94s including waiting). Image size: 125353963 bytes. 63m Normal Started pod/payload-processing-55fbbbf9dc-mp5sx Container started 63m Normal Created pod/payload-pre-processing-7b98bb678c-jdsm2 Container created 63m Normal Pulled pod/payload-pre-processing-7b98bb678c-jdsm2 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.642s (3.642s including waiting). Image size: 125353963 bytes. 63m Normal Started pod/payload-pre-processing-7b98bb678c-jdsm2 Container started 62m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 62m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-58c6585447 from 7 to 9 62m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-58c6585447 Created pod: maas-default-gateway-openshift-default-58c6585447-8jpjl 62m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-58c6585447 Created pod: maas-default-gateway-openshift-default-58c6585447-pcwvt 62m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-pcwvt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-pcwvt Container started 62m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-pcwvt Container created 62m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-8jpjl Add eth0 [10.132.0.46/23] from ovn-kubernetes 62m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-pcwvt Add eth0 [10.133.0.52/23] from ovn-kubernetes 62m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-8jpjl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-8jpjl Container started 62m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-8jpjl Container created 62m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-pcwvt Startup probe error: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused... 62m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-pcwvt Startup probe failed: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused 62m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-8jpjl Startup probe failed: Get "http://10.132.0.46:15021/healthz/ready": dial tcp 10.132.0.46:15021: connect: connection refused 62m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-8jpjl Startup probe error: Get "http://10.132.0.46:15021/healthz/ready": dial tcp 10.132.0.46:15021: connect: connection refused... 62m Normal AddedLabel gateway/e2e-worker-w2-cd248f Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-cd248f 62m Normal ScalingReplicaSet deployment/e2e-worker-w6-dd038b-openshift-default Scaled up replica set e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 from 0 to 1 62m Normal ScalingReplicaSet deployment/e2e-worker-w3-944263-openshift-default Scaled up replica set e2e-worker-w3-944263-openshift-default-5f78997bc9 from 0 to 1 62m Normal AddedLabel gateway/e2e-worker-w3-944263 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-944263 62m Normal AddedInterface pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-kndl9 Add eth0 [10.134.0.36/23] from ovn-kubernetes 62m Normal SuccessfulCreate replicaset/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 Created pod: e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-h2t4l 62m Normal AddedLabel gateway/e2e-worker-w0-208f2b Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-208f2b 62m Normal AddedLabel gateway/e2e-worker-w4-1a1e9e Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-1a1e9e 62m Normal AddedLabel gateway/e2e-worker-w5-c9ae21 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-c9ae21 62m Normal ScalingReplicaSet deployment/e2e-worker-w2-cd248f-openshift-default Scaled up replica set e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 from 0 to 1 62m Normal SuccessfulCreate replicaset/e2e-worker-w3-944263-openshift-default-5f78997bc9 Created pod: e2e-worker-w3-944263-openshift-default-5f78997bc9-kndl9 62m Normal AddedLabel gateway/e2e-worker-w6-dd038b Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-dd038b 62m Normal AddedLabel gateway/e2e-worker-w1-db0157 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-db0157 62m Normal SuccessfulCreate replicaset/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 Created pod: e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-tsxlv 62m Normal AddedInterface pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-gkhjr Add eth0 [10.134.0.38/23] from ovn-kubernetes 62m Normal SuccessfulCreate replicaset/e2e-worker-w1-db0157-openshift-default-68856dc5c7 Created pod: e2e-worker-w1-db0157-openshift-default-68856dc5c7-gkhjr 62m Normal AddedInterface pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-h2t4l Add eth0 [10.132.0.47/23] from ovn-kubernetes 62m Normal AddedInterface pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-tsxlv Add eth0 [10.134.0.37/23] from ovn-kubernetes 62m Normal SuccessfulCreate replicaset/e2e-worker-w0-208f2b-openshift-default-5cc586d6f Created pod: e2e-worker-w0-208f2b-openshift-default-5cc586d6f-blcnw 62m Normal ScalingReplicaSet deployment/e2e-worker-w1-db0157-openshift-default Scaled up replica set e2e-worker-w1-db0157-openshift-default-68856dc5c7 from 0 to 1 62m Normal SuccessfulCreate replicaset/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 Created pod: e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-sz2sr 62m Normal ScalingReplicaSet deployment/e2e-worker-w5-c9ae21-openshift-default Scaled up replica set e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 from 0 to 1 62m Normal ScalingReplicaSet deployment/e2e-worker-w0-208f2b-openshift-default Scaled up replica set e2e-worker-w0-208f2b-openshift-default-5cc586d6f from 0 to 1 62m Normal ScalingReplicaSet deployment/e2e-worker-w4-1a1e9e-openshift-default Scaled up replica set e2e-worker-w4-1a1e9e-openshift-default-56c9685797 from 0 to 1 62m Normal AddedInterface pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-w8rd7 Add eth0 [10.132.0.49/23] from ovn-kubernetes 62m Normal AddedInterface pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-blcnw Add eth0 [10.134.0.39/23] from ovn-kubernetes 62m Normal SuccessfulCreate replicaset/e2e-worker-w4-1a1e9e-openshift-default-56c9685797 Created pod: e2e-worker-w4-1a1e9e-openshift-default-56c9685797-w8rd7 62m Normal AddedInterface pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-sz2sr Add eth0 [10.132.0.48/23] from ovn-kubernetes 62m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-cd248f-6c9889fc9c Error creating: pods "payload-processing-e2e-worker-w2-cd248f-6c9889fc9c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-cd248f: serviceaccount "payload-processing-e2e-worker-w2-cd248f" not found 62m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-cd248f-7b8875bf8c Error creating: pods "payload-pre-processing-e2e-worker-w2-cd248f-7b8875bf8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-cd248f: serviceaccount "payload-processing-e2e-worker-w2-cd248f" not found 62m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-cd248f-7b8875bf8c Created pod: payload-pre-processing-e2e-worker-w2-cd248f-7b8875bf8c-c8fvd 62m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-cd248f Scaled up replica set payload-pre-processing-e2e-worker-w2-cd248f-7b8875bf8c from 0 to 1 62m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-cd248f-6c9889fc9c Created pod: payload-processing-e2e-worker-w2-cd248f-6c9889fc9c-wbd52 62m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-cd248f Scaled up replica set payload-processing-e2e-worker-w2-cd248f-6c9889fc9c from 0 to 1 62m Normal Started pod/payload-pre-processing-e2e-worker-w2-cd248f-7b8875bf8c-c8fvd Container started 62m Normal Created pod/payload-pre-processing-e2e-worker-w2-cd248f-7b8875bf8c-c8fvd Container created 62m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-cd248f-7b8875bf8c-c8fvd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 62m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-cd248f-7b8875bf8c-c8fvd Add eth0 [10.132.0.50/23] from ovn-kubernetes 62m Normal Pulling pod/payload-processing-e2e-worker-w2-cd248f-6c9889fc9c-wbd52 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 62m Normal AddedInterface pod/payload-processing-e2e-worker-w2-cd248f-6c9889fc9c-wbd52 Add eth0 [10.134.0.40/23] from ovn-kubernetes 62m Normal Created pod/payload-processing-e2e-worker-w2-cd248f-6c9889fc9c-wbd52 Container created 62m Normal Pulled pod/payload-processing-e2e-worker-w2-cd248f-6c9889fc9c-wbd52 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.614s (2.614s including waiting). Image size: 125353963 bytes. 62m Normal Started pod/payload-processing-e2e-worker-w2-cd248f-6c9889fc9c-wbd52 Container started 62m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-944263 Scaled up replica set payload-processing-e2e-worker-w3-944263-5cf85467cc from 0 to 1 62m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-944263-78999d46dc Created pod: payload-pre-processing-e2e-worker-w3-944263-78999d46dc-m8ml8 62m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-944263-78999d46dc Error creating: pods "payload-pre-processing-e2e-worker-w3-944263-78999d46dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-944263: serviceaccount "payload-processing-e2e-worker-w3-944263" not found 62m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-944263 Scaled up replica set payload-pre-processing-e2e-worker-w3-944263-78999d46dc from 0 to 1 62m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-944263-5cf85467cc Error creating: pods "payload-processing-e2e-worker-w3-944263-5cf85467cc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-944263: serviceaccount "payload-processing-e2e-worker-w3-944263" not found 62m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-944263-5cf85467cc Created pod: payload-processing-e2e-worker-w3-944263-5cf85467cc-sl4cb 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-db0157-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-944263-78999d46dc-m8ml8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 62m Normal Pulled pod/payload-processing-e2e-worker-w3-944263-5cf85467cc-sl4cb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 62m Normal AddedInterface pod/payload-processing-e2e-worker-w3-944263-5cf85467cc-sl4cb Add eth0 [10.134.0.42/23] from ovn-kubernetes 62m Normal Started pod/payload-processing-e2e-worker-w3-944263-5cf85467cc-sl4cb Container started 62m Normal Started pod/payload-pre-processing-e2e-worker-w3-944263-78999d46dc-m8ml8 Container started 62m Normal Created pod/payload-processing-e2e-worker-w3-944263-5cf85467cc-sl4cb Container created 62m Normal Created pod/payload-pre-processing-e2e-worker-w3-944263-78999d46dc-m8ml8 Container created 62m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-944263-78999d46dc-m8ml8 Add eth0 [10.134.0.41/23] from ovn-kubernetes 62m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-db0157-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-dd038b Scaled up replica set payload-pre-processing-e2e-worker-w6-dd038b-c5cc45b55 from 0 to 1 62m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-dd038b-c5cc45b55 Error creating: pods "payload-pre-processing-e2e-worker-w6-dd038b-c5cc45b55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-dd038b: serviceaccount "payload-processing-e2e-worker-w6-dd038b" not found 62m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-dd038b Scaled up replica set payload-processing-e2e-worker-w6-dd038b-58bf969f7c from 0 to 1 62m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-dd038b-58bf969f7c Error creating: pods "payload-processing-e2e-worker-w6-dd038b-58bf969f7c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-dd038b: serviceaccount "payload-processing-e2e-worker-w6-dd038b" not found 61m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-dd038b-c5cc45b55-nd59r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/payload-processing-e2e-worker-w6-dd038b-58bf969f7c-5tbbt Add eth0 [10.134.0.44/23] from ovn-kubernetes 61m Normal Started pod/payload-pre-processing-e2e-worker-w6-dd038b-c5cc45b55-nd59r Container started 61m Normal Created pod/payload-pre-processing-e2e-worker-w6-dd038b-c5cc45b55-nd59r Container created 61m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-dd038b-58bf969f7c Created pod: payload-processing-e2e-worker-w6-dd038b-58bf969f7c-5tbbt 61m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-dd038b-c5cc45b55-nd59r Add eth0 [10.134.0.43/23] from ovn-kubernetes 61m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-dd038b-c5cc45b55 Created pod: payload-pre-processing-e2e-worker-w6-dd038b-c5cc45b55-nd59r 61m Normal Started pod/payload-processing-e2e-worker-w6-dd038b-58bf969f7c-5tbbt Container started 61m Normal Created pod/payload-processing-e2e-worker-w6-dd038b-58bf969f7c-5tbbt Container created 61m Normal Pulled pod/payload-processing-e2e-worker-w6-dd038b-58bf969f7c-5tbbt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 61m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-db0157-6689cd59bc Error creating: pods "payload-pre-processing-e2e-worker-w1-db0157-6689cd59bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-db0157: serviceaccount "payload-processing-e2e-worker-w1-db0157" not found 61m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-db0157 Scaled up replica set payload-pre-processing-e2e-worker-w1-db0157-6689cd59bc from 0 to 1 61m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-db0157 Scaled up replica set payload-processing-e2e-worker-w1-db0157-654558c4b7 from 0 to 1 61m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-db0157-654558c4b7 Created pod: payload-processing-e2e-worker-w1-db0157-654558c4b7-vsbx9 61m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-db0157-654558c4b7 Error creating: pods "payload-processing-e2e-worker-w1-db0157-654558c4b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-db0157: serviceaccount "payload-processing-e2e-worker-w1-db0157" not found 61m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-db0157-6689cd59bc Created pod: payload-pre-processing-e2e-worker-w1-db0157-6689cd59bc-nd7kc 61m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-db0157-6689cd59bc-nd7kc Add eth0 [10.134.0.45/23] from ovn-kubernetes 61m Normal Created pod/payload-processing-e2e-worker-w1-db0157-654558c4b7-vsbx9 Container created 61m Normal Started pod/payload-processing-e2e-worker-w1-db0157-654558c4b7-vsbx9 Container started 61m Normal Pulled pod/payload-processing-e2e-worker-w1-db0157-654558c4b7-vsbx9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 61m Normal Created pod/payload-pre-processing-e2e-worker-w1-db0157-6689cd59bc-nd7kc Container created 61m Normal AddedInterface pod/payload-processing-e2e-worker-w1-db0157-654558c4b7-vsbx9 Add eth0 [10.134.0.46/23] from ovn-kubernetes 61m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-db0157-6689cd59bc-nd7kc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 61m Normal Started pod/payload-pre-processing-e2e-worker-w1-db0157-6689cd59bc-nd7kc Container started 61m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-208f2b Scaled up replica set payload-processing-e2e-worker-w0-208f2b-7fb7755ddc from 0 to 1 61m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-208f2b-7fb7755ddc Error creating: pods "payload-processing-e2e-worker-w0-208f2b-7fb7755ddc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-208f2b: serviceaccount "payload-processing-e2e-worker-w0-208f2b" not found 61m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-208f2b-76b7b8c9f Error creating: pods "payload-pre-processing-e2e-worker-w0-208f2b-76b7b8c9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-208f2b: serviceaccount "payload-processing-e2e-worker-w0-208f2b" not found 61m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-208f2b-7fb7755ddc Created pod: payload-processing-e2e-worker-w0-208f2b-7fb7755ddc-lx76l 61m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-208f2b Scaled up replica set payload-pre-processing-e2e-worker-w0-208f2b-76b7b8c9f from 0 to 1 61m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-208f2b-76b7b8c9f Created pod: payload-pre-processing-e2e-worker-w0-208f2b-76b7b8c9f-fxd8f 61m Normal AddedInterface pod/payload-processing-e2e-worker-w0-208f2b-7fb7755ddc-lx76l Add eth0 [10.134.0.48/23] from ovn-kubernetes 61m Normal Started pod/payload-pre-processing-e2e-worker-w0-208f2b-76b7b8c9f-fxd8f Container started 61m Normal Started pod/payload-processing-e2e-worker-w0-208f2b-7fb7755ddc-lx76l Container started 61m Normal Created pod/payload-processing-e2e-worker-w0-208f2b-7fb7755ddc-lx76l Container created 61m Normal Pulled pod/payload-processing-e2e-worker-w0-208f2b-7fb7755ddc-lx76l Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-208f2b-76b7b8c9f-fxd8f Add eth0 [10.134.0.47/23] from ovn-kubernetes 61m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-208f2b-76b7b8c9f-fxd8f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 61m Normal Created pod/payload-pre-processing-e2e-worker-w0-208f2b-76b7b8c9f-fxd8f Container created 61m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-1a1e9e Scaled up replica set payload-pre-processing-e2e-worker-w4-1a1e9e-8fc6dcb7 from 0 to 1 61m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-1a1e9e Scaled up replica set payload-processing-e2e-worker-w4-1a1e9e-66d4f8c5f from 0 to 1 61m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-1a1e9e-8fc6dcb7 Error creating: pods "payload-pre-processing-e2e-worker-w4-1a1e9e-8fc6dcb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-1a1e9e: serviceaccount "payload-processing-e2e-worker-w4-1a1e9e" not found 61m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-1a1e9e-8fc6dcb7-bs6sm Add eth0 [10.134.0.49/23] from ovn-kubernetes 61m Normal Started pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-kndl9 Container started 61m Normal Created pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-kndl9 Container created 61m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-1a1e9e-66d4f8c5f Error creating: pods "payload-processing-e2e-worker-w4-1a1e9e-66d4f8c5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-1a1e9e: serviceaccount "payload-processing-e2e-worker-w4-1a1e9e" not found 61m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-1a1e9e-8fc6dcb7 Created pod: payload-pre-processing-e2e-worker-w4-1a1e9e-8fc6dcb7-bs6sm 61m Normal Pulled pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-kndl9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-1a1e9e-66d4f8c5f Created pod: payload-processing-e2e-worker-w4-1a1e9e-66d4f8c5f-cr9fp 61m Normal Created pod/payload-pre-processing-e2e-worker-w4-1a1e9e-8fc6dcb7-bs6sm Container created 61m Normal Created pod/payload-processing-e2e-worker-w4-1a1e9e-66d4f8c5f-cr9fp Container created 61m Normal Started pod/payload-processing-e2e-worker-w4-1a1e9e-66d4f8c5f-cr9fp Container started 61m Normal Pulled pod/payload-processing-e2e-worker-w4-1a1e9e-66d4f8c5f-cr9fp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/payload-processing-e2e-worker-w4-1a1e9e-66d4f8c5f-cr9fp Add eth0 [10.134.0.50/23] from ovn-kubernetes 61m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-1a1e9e-8fc6dcb7-bs6sm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 61m Normal Started pod/payload-pre-processing-e2e-worker-w4-1a1e9e-8fc6dcb7-bs6sm Container started 61m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-c9ae21 Scaled up replica set payload-processing-e2e-worker-w5-c9ae21-6667d54cf5 from 0 to 1 61m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-c9ae21 Scaled up replica set payload-pre-processing-e2e-worker-w5-c9ae21-7bc765fb9f from 0 to 1 61m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-c9ae21-7bc765fb9f Created pod: payload-pre-processing-e2e-worker-w5-c9ae21-7bc765fb9f-7tjsn 61m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-c9ae21-6667d54cf5 Error creating: pods "payload-processing-e2e-worker-w5-c9ae21-6667d54cf5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-c9ae21: serviceaccount "payload-processing-e2e-worker-w5-c9ae21" not found 61m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-c9ae21-7bc765fb9f-7tjsn Add eth0 [10.134.0.51/23] from ovn-kubernetes 61m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-c9ae21-6667d54cf5 Created pod: payload-processing-e2e-worker-w5-c9ae21-6667d54cf5-twgq6 61m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-c9ae21-7bc765fb9f Error creating: pods "payload-pre-processing-e2e-worker-w5-c9ae21-7bc765fb9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-c9ae21: serviceaccount "payload-processing-e2e-worker-w5-c9ae21" not found 61m Normal AddedInterface pod/payload-processing-e2e-worker-w5-c9ae21-6667d54cf5-twgq6 Add eth0 [10.134.0.52/23] from ovn-kubernetes 61m Warning ProbeError pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-kndl9 Startup probe error: Get "http://10.134.0.36:15021/healthz/ready": dial tcp 10.134.0.36:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-kndl9 Startup probe failed: Get "http://10.134.0.36:15021/healthz/ready": dial tcp 10.134.0.36:15021: connect: connection refused 61m Normal Pulled pod/payload-processing-e2e-worker-w5-c9ae21-6667d54cf5-twgq6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 61m Normal Started pod/payload-pre-processing-e2e-worker-w5-c9ae21-7bc765fb9f-7tjsn Container started 61m Normal Created pod/payload-pre-processing-e2e-worker-w5-c9ae21-7bc765fb9f-7tjsn Container created 61m Normal Created pod/payload-processing-e2e-worker-w5-c9ae21-6667d54cf5-twgq6 Container created 61m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-c9ae21-7bc765fb9f-7tjsn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 61m Normal Started pod/payload-processing-e2e-worker-w5-c9ae21-6667d54cf5-twgq6 Container started 61m Normal Pulled pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-h2t4l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Started pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-h2t4l Container started 61m Normal Created pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-h2t4l Container created 61m Normal Pulled pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-gkhjr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Started pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-gkhjr Container started 61m Normal Created pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-gkhjr Container created 61m Warning Unhealthy pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-h2t4l Startup probe failed: Get "http://10.132.0.47:15021/healthz/ready": dial tcp 10.132.0.47:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-h2t4l Startup probe error: Get "http://10.132.0.47:15021/healthz/ready": dial tcp 10.132.0.47:15021: connect: connection refused... 61m Warning ProbeError pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-gkhjr Startup probe error: Get "http://10.134.0.38:15021/healthz/ready": dial tcp 10.134.0.38:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-gkhjr Startup probe failed: Get "http://10.134.0.38:15021/healthz/ready": dial tcp 10.134.0.38:15021: connect: connection refused 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-dd038b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 61m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-dd038b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 61m Normal Pulled pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-blcnw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Created pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-blcnw Container created 61m Normal Started pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-blcnw Container started 61m Warning ProbeError pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-blcnw Readiness probe error: Get "http://10.134.0.39:15021/healthz/ready": read tcp 10.134.0.2:48894->10.134.0.39:15021: read: connection reset by peer... 61m Warning Unhealthy pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-blcnw Readiness probe failed: Get "http://10.134.0.39:15021/healthz/ready": read tcp 10.134.0.2:48894->10.134.0.39:15021: read: connection reset by peer 61m Warning ProbeError pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-blcnw Startup probe error: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-blcnw Startup probe failed: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused 61m Normal Started pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-tsxlv Container started 61m Normal Created pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-tsxlv Container created 61m Normal Pulled pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-tsxlv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-db0157-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-db0157-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 61m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-208f2b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-208f2b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning Unhealthy pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-tsxlv Startup probe failed: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused 61m Warning ProbeError pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-tsxlv Startup probe error: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused... 61m Normal Pulled pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-w8rd7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal Started pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-w8rd7 Container started 61m Normal Created pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-w8rd7 Container created 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-944263-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-944263-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-dd038b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-cd248f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-cd248f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-dd038b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-1a1e9e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 61m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-1a1e9e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 61m Warning ProbeError pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-w8rd7 Startup probe error: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused... 61m Warning Unhealthy pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-w8rd7 Startup probe failed: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused 60m Warning ProbeError pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-sz2sr Readiness probe error: Get "http://10.132.0.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 60m Warning Unhealthy pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-sz2sr Readiness probe failed: Get "http://10.132.0.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 60m Normal Created pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-sz2sr Container created 60m Normal Pulled pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-sz2sr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Started pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-sz2sr Container started 60m Warning ProbeError pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-sz2sr Startup probe error: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused... 60m Warning Unhealthy pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-sz2sr Startup probe failed: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-cd248f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-cd248f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-c9ae21-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-c9ae21-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-208f2b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-208f2b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-1a1e9e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-1a1e9e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-c9ae21-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-c9ae21-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-944263-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 59m Normal ScalingReplicaSet deployment/e2e-worker-w3-944263-openshift-default Scaled up replica set e2e-worker-w3-944263-openshift-default-5f78997bc9 from 1 to 2 59m Normal SuccessfulCreate replicaset/e2e-worker-w3-944263-openshift-default-5f78997bc9 Created pod: e2e-worker-w3-944263-openshift-default-5f78997bc9-dhjgg 59m Normal Pulled pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-dhjgg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-dhjgg Container created 59m Normal AddedInterface pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-dhjgg Add eth0 [10.133.0.57/23] from ovn-kubernetes 59m Normal Started pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-dhjgg Container started 59m Warning Unhealthy pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-dhjgg Startup probe failed: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused 59m Warning ProbeError pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-dhjgg Startup probe error: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused... 59m Normal ScalingReplicaSet deployment/e2e-worker-w1-db0157-openshift-default Scaled up replica set e2e-worker-w1-db0157-openshift-default-68856dc5c7 from 1 to 2 59m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-db0157-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 59m Normal SuccessfulCreate replicaset/e2e-worker-w1-db0157-openshift-default-68856dc5c7 Created pod: e2e-worker-w1-db0157-openshift-default-68856dc5c7-mwvv2 58m Normal Started pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-mwvv2 Container started 58m Normal Created pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-mwvv2 Container created 58m Normal AddedInterface pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-mwvv2 Add eth0 [10.133.0.59/23] from ovn-kubernetes 58m Normal Pulled pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-mwvv2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Warning ProbeError pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-mwvv2 Startup probe error: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-mwvv2 Startup probe failed: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused 58m Normal ScalingReplicaSet deployment/e2e-worker-w0-208f2b-openshift-default Scaled up replica set e2e-worker-w0-208f2b-openshift-default-5cc586d6f from 1 to 2 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-208f2b-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 58m Normal SuccessfulCreate replicaset/e2e-worker-w0-208f2b-openshift-default-5cc586d6f Created pod: e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lgsnj 58m Normal AddedInterface pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lgsnj Add eth0 [10.133.0.60/23] from ovn-kubernetes 58m Normal Created pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lgsnj Container created 58m Normal Pulled pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lgsnj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Started pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lgsnj Container started 58m Warning Unhealthy pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lgsnj Startup probe failed: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lgsnj Startup probe error: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused... 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-cd248f-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 58m Normal ScalingReplicaSet deployment/e2e-worker-w2-cd248f-openshift-default Scaled up replica set e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 from 1 to 2 58m Normal SuccessfulCreate replicaset/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 Created pod: e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-ttwc5 58m Normal Started pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-ttwc5 Container started 58m Normal AddedInterface pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-ttwc5 Add eth0 [10.133.0.64/23] from ovn-kubernetes 58m Normal ScalingReplicaSet deployment/e2e-worker-w1-db0157-openshift-default Scaled up replica set e2e-worker-w1-db0157-openshift-default-68856dc5c7 from 2 to 3 58m Normal SuccessfulCreate replicaset/e2e-worker-w1-db0157-openshift-default-68856dc5c7 Created pod: e2e-worker-w1-db0157-openshift-default-68856dc5c7-784tj 58m Normal Pulled pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-ttwc5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-ttwc5 Container created 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-db0157-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 58m Normal Started pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-784tj Container started 58m Normal Created pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-784tj Container created 58m Normal Pulled pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-784tj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-784tj Add eth0 [10.132.0.64/23] from ovn-kubernetes 58m Warning Unhealthy pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-ttwc5 Startup probe failed: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-ttwc5 Startup probe error: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-784tj Startup probe failed: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-784tj Startup probe error: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused... 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-c9ae21-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 57m Normal ScalingReplicaSet deployment/e2e-worker-w5-c9ae21-openshift-default Scaled up replica set e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 from 1 to 2 57m Normal Started pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-gf6ms Container started 57m Normal Created pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-gf6ms Container created 57m Normal Pulled pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-gf6ms Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal SuccessfulCreate replicaset/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 Created pod: e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-gf6ms 57m Normal AddedInterface pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-gf6ms Add eth0 [10.134.0.66/23] from ovn-kubernetes 57m Warning Unhealthy pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-gf6ms Startup probe failed: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-gf6ms Startup probe error: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused... 57m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-58c6585447 from 9 to 6 57m Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-8jpjl Stopping container istio-proxy 57m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-58c6585447 Deleted pod: maas-default-gateway-openshift-default-58c6585447-8jpjl 57m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-58c6585447 Deleted pod: maas-default-gateway-openshift-default-58c6585447-22bjk 57m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: All metrics below target 57m Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-22bjk Stopping container istio-proxy 57m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-58c6585447 Deleted pod: maas-default-gateway-openshift-default-58c6585447-2zkzj 57m Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-2zkzj Stopping container istio-proxy 57m Normal ScalingReplicaSet deployment/e2e-worker-w2-cd248f-openshift-default Scaled up replica set e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 from 2 to 4 57m Normal SuccessfulCreate replicaset/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 Created pod: e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-2qv5b 57m Normal SuccessfulCreate replicaset/e2e-worker-w3-944263-openshift-default-5f78997bc9 Created pod: e2e-worker-w3-944263-openshift-default-5f78997bc9-fr9fj 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-cd248f-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 57m Normal SuccessfulCreate replicaset/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 Created pod: e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-49zgd 57m Normal ScalingReplicaSet deployment/e2e-worker-w3-944263-openshift-default Scaled up replica set e2e-worker-w3-944263-openshift-default-5f78997bc9 from 2 to 3 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-944263-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 57m Normal SuccessfulCreate replicaset/e2e-worker-w0-208f2b-openshift-default-5cc586d6f Created pod: e2e-worker-w0-208f2b-openshift-default-5cc586d6f-brks2 57m Normal AddedInterface pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-fr9fj Add eth0 [10.132.0.65/23] from ovn-kubernetes 57m Normal AddedInterface pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-49zgd Add eth0 [10.132.0.66/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-49zgd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-49zgd Container created 57m Normal Started pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-49zgd Container started 57m Normal Pulled pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-fr9fj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-2qv5b Add eth0 [10.134.0.71/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-2qv5b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-208f2b-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 57m Normal ScalingReplicaSet deployment/e2e-worker-w0-208f2b-openshift-default Scaled up replica set e2e-worker-w0-208f2b-openshift-default-5cc586d6f from 2 to 3 57m Normal Started pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-fr9fj Container started 57m Normal Created pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-2qv5b Container created 57m Normal Created pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-fr9fj Container created 57m Normal Started pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-2qv5b Container started 57m Normal Started pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-brks2 Container started 57m Normal Created pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-brks2 Container created 57m Normal AddedInterface pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-brks2 Add eth0 [10.132.0.67/23] from ovn-kubernetes 57m Normal Pulled pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-brks2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Warning Unhealthy pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-fr9fj Startup probe failed: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-fr9fj Startup probe error: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-49zgd Startup probe error: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-2qv5b Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-2qv5b Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-49zgd Startup probe failed: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-brks2 Startup probe error: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-brks2 Startup probe failed: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused 56m Normal ScalingReplicaSet deployment/e2e-worker-w6-dd038b-openshift-default Scaled up replica set e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 from 1 to 2 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-dd038b-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 56m Normal SuccessfulCreate replicaset/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 Created pod: e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-jwfhk 56m Normal ScalingReplicaSet deployment/e2e-worker-w1-db0157-openshift-default Scaled up replica set e2e-worker-w1-db0157-openshift-default-68856dc5c7 from 3 to 4 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-db0157-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 56m Normal SuccessfulCreate replicaset/e2e-worker-w1-db0157-openshift-default-68856dc5c7 Created pod: e2e-worker-w1-db0157-openshift-default-68856dc5c7-cc24r 56m Normal AddedInterface pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-jwfhk Add eth0 [10.134.0.80/23] from ovn-kubernetes 56m Normal AddedInterface pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-cc24r Add eth0 [10.134.0.81/23] from ovn-kubernetes 56m Normal Pulled pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-cc24r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-cc24r Container created 56m Normal Started pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-jwfhk Container started 56m Normal Created pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-jwfhk Container created 56m Normal Pulled pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-jwfhk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-cc24r Container started 56m Warning ProbeError pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-kndl9 Readiness probe error: Get "http://10.134.0.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Warning Unhealthy pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-kndl9 Readiness probe failed: Get "http://10.134.0.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-jwfhk Startup probe error: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-jwfhk Startup probe failed: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-cc24r Startup probe error: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-cc24r Startup probe failed: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused 56m Normal ScalingReplicaSet deployment/e2e-worker-w2-cd248f-openshift-default Scaled up replica set e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 from 4 to 5 56m Normal AddedInterface pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-sh9p6 Add eth0 [10.133.0.68/23] from ovn-kubernetes 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-cd248f-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 56m Normal SuccessfulCreate replicaset/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 Created pod: e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-sh9p6 56m Normal Created pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-sh9p6 Container created 56m Normal Pulled pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-sh9p6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-sh9p6 Container started 56m Warning Unhealthy pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-sh9p6 Startup probe failed: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-sh9p6 Startup probe error: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused... 56m Normal Started pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-l7fxr Container started 56m Normal Pulled pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-v5s7k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-944263-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 56m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-58c6585447 Deleted pod: maas-default-gateway-openshift-default-58c6585447-vz6nd 56m Normal AddedInterface pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-gkcth Add eth0 [10.132.0.68/23] from ovn-kubernetes 56m Normal Pulled pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-gkcth Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-58c6585447 from 6 to 5 56m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: All metrics below target 56m Normal SuccessfulCreate replicaset/e2e-worker-w3-944263-openshift-default-5f78997bc9 Created pod: e2e-worker-w3-944263-openshift-default-5f78997bc9-gkcth 56m Normal SuccessfulCreate replicaset/e2e-worker-w3-944263-openshift-default-5f78997bc9 Created pod: e2e-worker-w3-944263-openshift-default-5f78997bc9-v5s7k 56m Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-vz6nd Stopping container istio-proxy 56m Normal AddedInterface pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-l7fxr Add eth0 [10.134.0.84/23] from ovn-kubernetes 56m Normal SuccessfulCreate replicaset/e2e-worker-w3-944263-openshift-default-5f78997bc9 Created pod: e2e-worker-w3-944263-openshift-default-5f78997bc9-l7fxr 56m Normal Pulled pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-l7fxr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-l7fxr Container created 56m Normal ScalingReplicaSet deployment/e2e-worker-w3-944263-openshift-default Scaled up replica set e2e-worker-w3-944263-openshift-default-5f78997bc9 from 3 to 6 56m Normal AddedInterface pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-v5s7k Add eth0 [10.133.0.69/23] from ovn-kubernetes 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-1a1e9e-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 56m Normal Pulled pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-2bb5k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-v5s7k Container started 56m Normal Created pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-v5s7k Container created 56m Normal Started pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-5bxs8 Container started 56m Normal AddedInterface pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-2bb5k Add eth0 [10.132.0.70/23] from ovn-kubernetes 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-db0157-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 56m Normal Created pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-5bxs8 Container created 56m Normal AddedInterface pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-5bxs8 Add eth0 [10.133.0.70/23] from ovn-kubernetes 56m Normal Started pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-gkcth Container started 56m Normal Created pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-2bb5k Container created 56m Normal Pulled pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-5bxs8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal SuccessfulCreate replicaset/e2e-worker-w1-db0157-openshift-default-68856dc5c7 Created pod: e2e-worker-w1-db0157-openshift-default-68856dc5c7-5bxs8 56m Normal SuccessfulCreate replicaset/e2e-worker-w1-db0157-openshift-default-68856dc5c7 Created pod: e2e-worker-w1-db0157-openshift-default-68856dc5c7-2bb5k 56m Normal Started pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-2bb5k Container started 56m Normal Created pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-gkcth Container created 56m Normal ScalingReplicaSet deployment/e2e-worker-w1-db0157-openshift-default Scaled up replica set e2e-worker-w1-db0157-openshift-default-68856dc5c7 from 4 to 6 56m Normal AddedInterface pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-nsqsb Add eth0 [10.134.0.85/23] from ovn-kubernetes 56m Normal Started pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-nsqsb Container started 56m Normal Created pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-nsqsb Container created 56m Normal SuccessfulCreate replicaset/e2e-worker-w4-1a1e9e-openshift-default-56c9685797 Created pod: e2e-worker-w4-1a1e9e-openshift-default-56c9685797-nsqsb 56m Normal ScalingReplicaSet deployment/e2e-worker-w4-1a1e9e-openshift-default Scaled up replica set e2e-worker-w4-1a1e9e-openshift-default-56c9685797 from 1 to 2 56m Normal Pulled pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-nsqsb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Warning Unhealthy pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-gkcth Startup probe failed: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-v5s7k Startup probe error: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused... 56m Warning ProbeError pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-gkcth Startup probe error: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-l7fxr Startup probe failed: Get "http://10.134.0.84:15021/healthz/ready": dial tcp 10.134.0.84:15021: connect: connection refused 56m Warning Unhealthy pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-v5s7k Startup probe failed: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-l7fxr Startup probe error: Get "http://10.134.0.84:15021/healthz/ready": dial tcp 10.134.0.84:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-2bb5k Startup probe failed: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused 56m Warning Unhealthy pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-5bxs8 Startup probe failed: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-2bb5k Startup probe error: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused... 56m Warning ProbeError pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-5bxs8 Startup probe error: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused... 56m Warning ProbeError pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-nsqsb Startup probe error: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-nsqsb Startup probe failed: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused 56m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-58c6585447 from 5 to 4 56m Normal ScalingReplicaSet deployment/e2e-worker-w2-cd248f-openshift-default Scaled up replica set e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 from 5 to 6 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-cd248f-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 56m Normal SuccessfulCreate replicaset/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 Created pod: e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-hlgfg 56m Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-z2jrj Stopping container istio-proxy 56m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-58c6585447 Deleted pod: maas-default-gateway-openshift-default-58c6585447-z2jrj 56m Normal Pulled pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-hlgfg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-hlgfg Add eth0 [10.132.0.72/23] from ovn-kubernetes 56m Normal Created pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-hlgfg Container created 56m Normal Started pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-hlgfg Container started 56m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-z2jrj Readiness probe error: HTTP probe failed with statuscode: 503... 56m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-z2jrj Readiness probe failed: HTTP probe failed with statuscode: 503 56m Warning ProbeError pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-hlgfg Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-hlgfg Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused 55m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-58c6585447 from 4 to 3 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-dd038b-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 55m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-58c6585447 Deleted pod: maas-default-gateway-openshift-default-58c6585447-zndkb 55m Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-zndkb Stopping container istio-proxy 55m Normal SuccessfulCreate replicaset/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 Created pod: e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-dcbhv 55m Normal ScalingReplicaSet deployment/e2e-worker-w6-dd038b-openshift-default Scaled up replica set e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 from 2 to 3 55m Normal AddedInterface pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-dcbhv Add eth0 [10.133.0.71/23] from ovn-kubernetes 55m Normal Started pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-dcbhv Container started 55m Normal Created pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-dcbhv Container created 55m Normal Pulled pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-dcbhv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Warning Unhealthy pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-dcbhv Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-dcbhv Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused... 55m Normal SuccessfulCreate replicaset/e2e-worker-w1-db0157-openshift-default-68856dc5c7 Created pod: e2e-worker-w1-db0157-openshift-default-68856dc5c7-xqm2p 55m Normal ScalingReplicaSet deployment/e2e-worker-w1-db0157-openshift-default Scaled up replica set e2e-worker-w1-db0157-openshift-default-68856dc5c7 from 6 to 10 55m Normal SuccessfulCreate replicaset/e2e-worker-w1-db0157-openshift-default-68856dc5c7 Created pod: e2e-worker-w1-db0157-openshift-default-68856dc5c7-6wrqk 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-db0157-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 55m Normal SuccessfulCreate replicaset/e2e-worker-w1-db0157-openshift-default-68856dc5c7 (combined from similar events): Created pod: e2e-worker-w1-db0157-openshift-default-68856dc5c7-psptj 55m Normal SuccessfulCreate replicaset/e2e-worker-w1-db0157-openshift-default-68856dc5c7 Created pod: e2e-worker-w1-db0157-openshift-default-68856dc5c7-z5s97 55m Normal Started pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-xqm2p Container started 55m Normal Started pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-psptj Container started 55m Normal Started pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-6wrqk Container started 55m Normal Created pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-6wrqk Container created 55m Normal Pulled pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-6wrqk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-6wrqk Add eth0 [10.134.0.90/23] from ovn-kubernetes 55m Normal AddedInterface pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-psptj Add eth0 [10.134.0.91/23] from ovn-kubernetes 55m Normal Pulled pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-psptj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-psptj Container created 55m Normal Started pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-z5s97 Container started 55m Normal Created pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-z5s97 Container created 55m Normal Pulled pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-z5s97 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-z5s97 Add eth0 [10.132.0.73/23] from ovn-kubernetes 55m Normal AddedInterface pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-xqm2p Add eth0 [10.133.0.73/23] from ovn-kubernetes 55m Normal Pulled pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-xqm2p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-xqm2p Container created 55m Warning ProbeError pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-z5s97 Startup probe error: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-z5s97 Startup probe failed: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-6wrqk Startup probe error: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-psptj Startup probe failed: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-psptj Startup probe error: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-6wrqk Startup probe failed: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused 55m Warning Unhealthy pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-xqm2p Startup probe failed: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-xqm2p Startup probe error: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused... 55m Normal SuccessfulCreate replicaset/e2e-worker-w4-1a1e9e-openshift-default-56c9685797 Created pod: e2e-worker-w4-1a1e9e-openshift-default-56c9685797-zjkp8 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-1a1e9e-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 55m Normal ScalingReplicaSet deployment/e2e-worker-w4-1a1e9e-openshift-default Scaled up replica set e2e-worker-w4-1a1e9e-openshift-default-56c9685797 from 2 to 3 55m Normal Pulled pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-zjkp8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Started pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-zjkp8 Container started 55m Normal Created pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-zjkp8 Container created 55m Normal AddedInterface pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-zjkp8 Add eth0 [10.133.0.75/23] from ovn-kubernetes 55m Warning Unhealthy pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-zjkp8 Startup probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-zjkp8 Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused... 53m Normal ScalingReplicaSet deployment/e2e-worker-w3-944263-openshift-default Scaled up replica set e2e-worker-w3-944263-openshift-default-5f78997bc9 from 6 to 10 53m Normal SuccessfulCreate replicaset/e2e-worker-w3-944263-openshift-default-5f78997bc9 Created pod: e2e-worker-w3-944263-openshift-default-5f78997bc9-np65v 53m Normal SuccessfulCreate replicaset/e2e-worker-w3-944263-openshift-default-5f78997bc9 Created pod: e2e-worker-w3-944263-openshift-default-5f78997bc9-czlxh 53m Normal AddedInterface pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-np65v Add eth0 [10.132.0.75/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-lxjj4 Add eth0 [10.133.0.76/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/e2e-worker-w3-944263-openshift-default-5f78997bc9 Created pod: e2e-worker-w3-944263-openshift-default-5f78997bc9-lxjj4 53m Normal SuccessfulCreate replicaset/e2e-worker-w3-944263-openshift-default-5f78997bc9 (combined from similar events): Created pod: e2e-worker-w3-944263-openshift-default-5f78997bc9-gjkgx 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-944263-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 53m Normal AddedInterface pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-czlxh Add eth0 [10.134.0.95/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-lxjj4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal SuccessfulCreate replicaset/e2e-worker-w0-208f2b-openshift-default-5cc586d6f Created pod: e2e-worker-w0-208f2b-openshift-default-5cc586d6f-9nlfq 53m Normal Started pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-gjkgx Container started 53m Normal ScalingReplicaSet deployment/e2e-worker-w0-208f2b-openshift-default Scaled up replica set e2e-worker-w0-208f2b-openshift-default-5cc586d6f from 3 to 6 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-208f2b-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 53m Normal Created pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-gjkgx Container created 53m Normal SuccessfulCreate replicaset/e2e-worker-w0-208f2b-openshift-default-5cc586d6f Created pod: e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lrl8n 53m Normal Started pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-lxjj4 Container started 53m Normal Pulled pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-np65v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-np65v Container created 53m Normal Started pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-np65v Container started 53m Normal SuccessfulCreate replicaset/e2e-worker-w0-208f2b-openshift-default-5cc586d6f Created pod: e2e-worker-w0-208f2b-openshift-default-5cc586d6f-jlkc9 53m Normal Pulled pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-czlxh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-czlxh Container created 53m Normal Started pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-czlxh Container started 53m Normal AddedInterface pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-gjkgx Add eth0 [10.134.0.96/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-gjkgx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-lxjj4 Container created 53m Normal Created pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-9nlfq Container created 53m Normal Pulled pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-9nlfq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-9nlfq Add eth0 [10.134.0.97/23] from ovn-kubernetes 53m Normal Started pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-9nlfq Container started 53m Normal AddedInterface pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-jlkc9 Add eth0 [10.133.0.78/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-jlkc9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lrl8n Container started 53m Normal AddedInterface pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lrl8n Add eth0 [10.132.0.76/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lrl8n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lrl8n Container created 53m Normal Created pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-jlkc9 Container created 53m Normal Started pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-jlkc9 Container started 53m Warning ProbeError pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lrl8n Startup probe error: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lrl8n Startup probe failed: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-gjkgx Startup probe error: Get "http://10.134.0.96:15021/healthz/ready": dial tcp 10.134.0.96:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-gjkgx Startup probe failed: Get "http://10.134.0.96:15021/healthz/ready": dial tcp 10.134.0.96:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-jlkc9 Startup probe error: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-lxjj4 Startup probe error: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-lxjj4 Startup probe failed: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-jlkc9 Startup probe failed: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-np65v Startup probe error: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-np65v Startup probe failed: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-czlxh Startup probe failed: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-czlxh Startup probe error: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-9nlfq Startup probe failed: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-9nlfq Startup probe error: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused... 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-dd038b-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 53m Normal Created pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-wh67n Container created 53m Normal Started pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-wh67n Container started 53m Normal Pulled pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-zttrt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-zttrt Container created 53m Normal ScalingReplicaSet deployment/e2e-worker-w6-dd038b-openshift-default Scaled up replica set e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 from 3 to 6 53m Normal Pulled pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-wh67n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-wh67n Add eth0 [10.132.0.78/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-zttrt Add eth0 [10.134.0.100/23] from ovn-kubernetes 53m Normal Started pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-zttrt Container started 53m Normal SuccessfulCreate replicaset/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 Created pod: e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-zttrt 53m Normal SuccessfulCreate replicaset/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 Created pod: e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-wh67n 53m Normal SuccessfulCreate replicaset/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 Created pod: e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-sbntr 53m Normal Created pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-sbntr Container created 53m Normal Started pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-sbntr Container started 53m Normal Pulled pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-sbntr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-sbntr Add eth0 [10.133.0.79/23] from ovn-kubernetes 53m Warning ProbeError pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-zttrt Startup probe error: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-sbntr Startup probe failed: Get "http://10.133.0.79:15021/healthz/ready": dial tcp 10.133.0.79:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-sbntr Startup probe error: Get "http://10.133.0.79:15021/healthz/ready": dial tcp 10.133.0.79:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-wh67n Startup probe error: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-wh67n Startup probe failed: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-zttrt Startup probe failed: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused 53m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 2; reason: All metrics below target 53m Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-pcwvt Stopping container istio-proxy 53m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-58c6585447 Deleted pod: maas-default-gateway-openshift-default-58c6585447-pcwvt 53m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-pcwvt Readiness probe error: HTTP probe failed with statuscode: 503... 53m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-pcwvt Readiness probe failed: HTTP probe failed with statuscode: 503 52m Normal ScalingReplicaSet deployment/e2e-worker-w4-1a1e9e-openshift-default Scaled up replica set e2e-worker-w4-1a1e9e-openshift-default-56c9685797 from 3 to 6 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-1a1e9e-openshift-default-56c9685797 Created pod: e2e-worker-w4-1a1e9e-openshift-default-56c9685797-wmg5x 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-1a1e9e-openshift-default-56c9685797 Created pod: e2e-worker-w4-1a1e9e-openshift-default-56c9685797-b95tc 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-1a1e9e-openshift-default-56c9685797 Created pod: e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vnzfv 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-1a1e9e-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 52m Normal Created pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-wmg5x Container created 52m Normal AddedInterface pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-b95tc Add eth0 [10.134.0.105/23] from ovn-kubernetes 52m Normal Created pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-b95tc Container created 52m Normal AddedInterface pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vnzfv Add eth0 [10.133.0.83/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-b95tc Container started 52m Normal Pulled pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-b95tc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Pulled pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vnzfv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Pulled pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-wmg5x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-wmg5x Add eth0 [10.132.0.79/23] from ovn-kubernetes 52m Normal Created pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vnzfv Container created 52m Normal Started pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vnzfv Container started 52m Normal Started pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-wmg5x Container started 52m Warning Unhealthy pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vnzfv Startup probe failed: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-b95tc Startup probe failed: Get "http://10.134.0.105:15021/healthz/ready": dial tcp 10.134.0.105:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-wmg5x Startup probe error: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vnzfv Startup probe error: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-b95tc Startup probe error: Get "http://10.134.0.105:15021/healthz/ready": dial tcp 10.134.0.105:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-wmg5x Startup probe failed: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-tsxlv Readiness probe failed: Get "http://10.134.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-tsxlv Readiness probe error: Get "http://10.134.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Normal SuccessfulCreate replicaset/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 Created pod: e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-cznb8 52m Normal SuccessfulCreate replicaset/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 Created pod: e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-f7qth 52m Normal ScalingReplicaSet deployment/e2e-worker-w5-c9ae21-openshift-default Scaled up replica set e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 from 2 to 4 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-c9ae21-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 52m Normal Started pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-f7qth Container started 52m Normal Pulled pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-cznb8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-cznb8 Add eth0 [10.134.0.110/23] from ovn-kubernetes 52m Normal Created pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-cznb8 Container created 52m Normal Started pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-cznb8 Container started 52m Normal AddedInterface pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-f7qth Add eth0 [10.133.0.84/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-f7qth Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-f7qth Container created 52m Warning Unhealthy pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-cznb8 Startup probe failed: Get "http://10.134.0.110:15021/healthz/ready": dial tcp 10.134.0.110:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-cznb8 Startup probe error: Get "http://10.134.0.110:15021/healthz/ready": dial tcp 10.134.0.110:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-f7qth Startup probe failed: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-f7qth Startup probe error: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused... 52m Normal SuccessfulCreate replicaset/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 Created pod: e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-lsbvv 52m Normal SuccessfulCreate replicaset/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 Created pod: e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-kdwp6 52m Normal ScalingReplicaSet deployment/e2e-worker-w2-cd248f-openshift-default Scaled up replica set e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 from 6 to 9 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-dd038b-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-cd248f-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 52m Normal ScalingReplicaSet deployment/e2e-worker-w6-dd038b-openshift-default Scaled up replica set e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 from 6 to 10 52m Normal SuccessfulCreate replicaset/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 Created pod: e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-hdc42 52m Normal SuccessfulCreate replicaset/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 (combined from similar events): Created pod: e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-tt6hz 52m Normal SuccessfulCreate replicaset/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 Created pod: e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-xzjrw 52m Normal SuccessfulCreate replicaset/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 Created pod: e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-65cf4 52m Normal SuccessfulCreate replicaset/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 Created pod: e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-nrzth 52m Normal Pulled pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-nrzth Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal ScalingReplicaSet deployment/e2e-worker-w5-c9ae21-openshift-default Scaled up replica set e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 from 4 to 5 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-208f2b-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 52m Normal ScalingReplicaSet deployment/e2e-worker-w0-208f2b-openshift-default Scaled up replica set e2e-worker-w0-208f2b-openshift-default-5cc586d6f from 6 to 10 52m Normal SuccessfulCreate replicaset/e2e-worker-w0-208f2b-openshift-default-5cc586d6f (combined from similar events): Created pod: e2e-worker-w0-208f2b-openshift-default-5cc586d6f-gx9rt 52m Normal SuccessfulCreate replicaset/e2e-worker-w0-208f2b-openshift-default-5cc586d6f Created pod: e2e-worker-w0-208f2b-openshift-default-5cc586d6f-mfk4k 52m Normal AddedInterface pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-hdc42 Add eth0 [10.132.0.81/23] from ovn-kubernetes 52m Normal SuccessfulCreate replicaset/e2e-worker-w0-208f2b-openshift-default-5cc586d6f Created pod: e2e-worker-w0-208f2b-openshift-default-5cc586d6f-b9n9v 52m Normal SuccessfulCreate replicaset/e2e-worker-w0-208f2b-openshift-default-5cc586d6f Created pod: e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lbcsf 52m Normal AddedInterface pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-nrzth Add eth0 [10.133.0.85/23] from ovn-kubernetes 52m Normal AddedInterface pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-xzjrw Add eth0 [10.134.0.116/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-xzjrw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal SuccessfulCreate replicaset/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 Created pod: e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-qjcxh 52m Normal Pulled pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-kdwp6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-xzjrw Container created 52m Normal AddedInterface pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-kdwp6 Add eth0 [10.133.0.86/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-65cf4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-65cf4 Add eth0 [10.132.0.80/23] from ovn-kubernetes 52m Normal Created pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-kdwp6 Container created 52m Normal AddedInterface pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-tt6hz Add eth0 [10.134.0.118/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-xzjrw Container started 52m Normal Started pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-nrzth Container started 52m Normal AddedInterface pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-lsbvv Add eth0 [10.134.0.117/23] from ovn-kubernetes 52m Normal Created pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-nrzth Container created 52m Normal Pulled pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-hdc42 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Started pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-kdwp6 Container started 52m Normal Created pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-hdc42 Container created 52m Normal Started pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-hdc42 Container started 52m Normal AddedInterface pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-b9n9v Add eth0 [10.132.0.82/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-65cf4 Container started 52m Normal Created pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-65cf4 Container created 52m Normal Pulled pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-tt6hz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-gx9rt Add eth0 [10.133.0.88/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-b9n9v Container started 52m Normal AddedInterface pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-mfk4k Add eth0 [10.133.0.87/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-mfk4k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Started pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-lsbvv Container started 52m Normal Started pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lbcsf Container started 52m Normal Created pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lbcsf Container created 52m Normal Pulled pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lbcsf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lbcsf Add eth0 [10.134.0.119/23] from ovn-kubernetes 52m Normal Created pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-lsbvv Container created 52m Normal Pulled pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-lsbvv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Pulled pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-gx9rt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Pulled pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-b9n9v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-b9n9v Container created 52m Normal Started pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-tt6hz Container started 52m Normal Created pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-tt6hz Container created 52m Normal Started pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-gx9rt Container started 52m Normal AddedInterface pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-qjcxh Add eth0 [10.133.0.89/23] from ovn-kubernetes 52m Normal Created pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-mfk4k Container created 52m Normal Pulled pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-qjcxh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Started pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-mfk4k Container started 52m Normal Created pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-gx9rt Container created 52m Normal Started pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-qjcxh Container started 52m Normal Created pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-qjcxh Container created 52m Warning Unhealthy pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-hdc42 Startup probe failed: Get "http://10.132.0.81:15021/healthz/ready": dial tcp 10.132.0.81:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-b9n9v Startup probe failed: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-b9n9v Startup probe error: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-65cf4 Startup probe failed: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-hdc42 Startup probe error: Get "http://10.132.0.81:15021/healthz/ready": dial tcp 10.132.0.81:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-65cf4 Startup probe error: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-xzjrw Startup probe failed: Get "http://10.134.0.116:15021/healthz/ready": dial tcp 10.134.0.116:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-xzjrw Startup probe error: Get "http://10.134.0.116:15021/healthz/ready": dial tcp 10.134.0.116:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-kdwp6 Startup probe error: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-tt6hz Startup probe error: Get "http://10.134.0.118:15021/healthz/ready": dial tcp 10.134.0.118:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lbcsf Startup probe failed: Get "http://10.134.0.119:15021/healthz/ready": dial tcp 10.134.0.119:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-lsbvv Startup probe failed: Get "http://10.134.0.117:15021/healthz/ready": dial tcp 10.134.0.117:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-lsbvv Startup probe error: Get "http://10.134.0.117:15021/healthz/ready": dial tcp 10.134.0.117:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lbcsf Startup probe error: Get "http://10.134.0.119:15021/healthz/ready": dial tcp 10.134.0.119:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-nrzth Startup probe error: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-nrzth Startup probe failed: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-tt6hz Startup probe failed: Get "http://10.134.0.118:15021/healthz/ready": dial tcp 10.134.0.118:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-kdwp6 Startup probe failed: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-kdwp6 Startup probe failed: Get "http://10.133.0.86:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-kdwp6 Startup probe error: Get "http://10.133.0.86:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-mfk4k Startup probe failed: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-mfk4k Startup probe error: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-gx9rt Startup probe failed: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-gx9rt Startup probe error: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused... 51m Normal SuccessfulCreate replicaset/e2e-worker-w4-1a1e9e-openshift-default-56c9685797 Created pod: e2e-worker-w4-1a1e9e-openshift-default-56c9685797-8vhd5 51m Warning ProbeError pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-qjcxh Startup probe error: Get "http://10.133.0.89:15021/healthz/ready": dial tcp 10.133.0.89:15021: connect: connection refused... 51m Normal Pulled pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-tb6nw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Warning Unhealthy pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-qjcxh Startup probe failed: Get "http://10.133.0.89:15021/healthz/ready": dial tcp 10.133.0.89:15021: connect: connection refused 51m Normal AddedInterface pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-8vhd5 Add eth0 [10.132.0.83/23] from ovn-kubernetes 51m Normal AddedInterface pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-m75bn Add eth0 [10.134.0.122/23] from ovn-kubernetes 51m Normal ScalingReplicaSet deployment/e2e-worker-w4-1a1e9e-openshift-default Scaled up replica set e2e-worker-w4-1a1e9e-openshift-default-56c9685797 from 6 to 10 51m Normal Pulled pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vcdwm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Pulled pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-8vhd5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-tb6nw Add eth0 [10.134.0.123/23] from ovn-kubernetes 51m Normal Started pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-8vhd5 Container started 51m Normal AddedInterface pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vcdwm Add eth0 [10.133.0.90/23] from ovn-kubernetes 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-1a1e9e-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/e2e-worker-w4-1a1e9e-openshift-default-56c9685797 (combined from similar events): Created pod: e2e-worker-w4-1a1e9e-openshift-default-56c9685797-tb6nw 51m Normal SuccessfulCreate replicaset/e2e-worker-w4-1a1e9e-openshift-default-56c9685797 Created pod: e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vcdwm 51m Normal Created pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-8vhd5 Container created 51m Normal SuccessfulCreate replicaset/e2e-worker-w4-1a1e9e-openshift-default-56c9685797 Created pod: e2e-worker-w4-1a1e9e-openshift-default-56c9685797-m75bn 51m Normal Started pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-m75bn Container started 51m Normal Created pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-m75bn Container created 51m Normal Pulled pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-m75bn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vcdwm Container started 51m Normal Created pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vcdwm Container created 51m Normal Created pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-tb6nw Container created 51m Normal Started pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-tb6nw Container started 51m Warning ProbeError pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-8vhd5 Startup probe error: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-tb6nw Startup probe error: Get "http://10.134.0.123:15021/healthz/ready": dial tcp 10.134.0.123:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-tb6nw Startup probe failed: Get "http://10.134.0.123:15021/healthz/ready": dial tcp 10.134.0.123:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-m75bn Startup probe error: Get "http://10.134.0.122:15021/healthz/ready": dial tcp 10.134.0.122:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-8vhd5 Startup probe failed: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vcdwm Startup probe error: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-m75bn Startup probe failed: Get "http://10.134.0.122:15021/healthz/ready": dial tcp 10.134.0.122:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vcdwm Startup probe failed: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-9nlfq Readiness probe error: Get "http://10.134.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-9nlfq Readiness probe failed: Get "http://10.134.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-b9n9v Readiness probe error: Get "http://10.132.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-b9n9v Readiness probe failed: Get "http://10.132.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-tt6hz Readiness probe error: Get "http://10.134.0.118:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-58c6585447 Deleted pod: maas-default-gateway-openshift-default-58c6585447-ppfz4 51m Warning Unhealthy pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-tt6hz Readiness probe failed: Get "http://10.134.0.118:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning Unhealthy pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-lsbvv Readiness probe failed: Get "http://10.134.0.117:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-lsbvv Readiness probe error: Get "http://10.134.0.117:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-ppfz4 Stopping container istio-proxy 51m Warning ProbeError pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lbcsf Readiness probe error: Get "http://10.134.0.119:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lbcsf Readiness probe failed: Get "http://10.134.0.119:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Normal ScalingReplicaSet deployment/e2e-ait-f955d283-openshift-default Scaled up replica set e2e-ait-f955d283-openshift-default-775bfb5f57 from 0 to 1 51m Normal AddedLabel gateway/e2e-ait-f955d283 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-f955d283 51m Normal SuccessfulCreate replicaset/e2e-ait-f955d283-openshift-default-775bfb5f57 Created pod: e2e-ait-f955d283-openshift-default-775bfb5f57-x79dv 51m Normal AddedInterface pod/e2e-ait-f955d283-openshift-default-775bfb5f57-x79dv Add eth0 [10.134.0.133/23] from ovn-kubernetes 51m Normal AddedLabel gateway/e2e-ait-a0ef08b4 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-a0ef08b4 51m Normal ScalingReplicaSet deployment/e2e-derive-d0d57533-openshift-default Scaled up replica set e2e-derive-d0d57533-openshift-default-bb8564d89 from 0 to 1 51m Normal SuccessfulCreate replicaset/e2e-ait-a0ef08b4-openshift-default-58c89679c Created pod: e2e-ait-a0ef08b4-openshift-default-58c89679c-qf9jq 51m Normal ScalingReplicaSet deployment/e2e-ait-a0ef08b4-openshift-default Scaled up replica set e2e-ait-a0ef08b4-openshift-default-58c89679c from 0 to 1 51m Normal AddedInterface pod/e2e-ait-a0ef08b4-openshift-default-58c89679c-qf9jq Add eth0 [10.132.0.84/23] from ovn-kubernetes 51m Normal AddedLabel gateway/e2e-derive-d0d57533 Added label istio.io/rev=openshift-gateway to gateway e2e-derive-d0d57533 51m Normal SuccessfulCreate replicaset/e2e-derive-d0d57533-openshift-default-bb8564d89 Created pod: e2e-derive-d0d57533-openshift-default-bb8564d89-mmmrv 51m Normal AddedInterface pod/e2e-derive-d0d57533-openshift-default-bb8564d89-mmmrv Add eth0 [10.132.0.85/23] from ovn-kubernetes 51m Warning FailedCreate replicaset/payload-processing-e2e-ait-f955d283-ddc87bc75 Error creating: pods "payload-processing-e2e-ait-f955d283-ddc87bc75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-f955d283: serviceaccount "payload-processing-e2e-ait-f955d283" not found 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-f955d283-6798d4f4dc Error creating: pods "payload-pre-processing-e2e-ait-f955d283-6798d4f4dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-f955d283: serviceaccount "payload-processing-e2e-ait-f955d283" not found 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-f955d283 Scaled up replica set payload-pre-processing-e2e-ait-f955d283-6798d4f4dc from 0 to 1 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-f955d283 Scaled up replica set payload-processing-e2e-ait-f955d283-ddc87bc75 from 0 to 1 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-f955d283-ddc87bc75 Created pod: payload-processing-e2e-ait-f955d283-ddc87bc75-fdqxb 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-f955d283-6798d4f4dc Created pod: payload-pre-processing-e2e-ait-f955d283-6798d4f4dc-b22zs 51m Normal Started pod/payload-pre-processing-e2e-ait-f955d283-6798d4f4dc-b22zs Container started 51m Normal Pulled pod/payload-pre-processing-e2e-ait-f955d283-6798d4f4dc-b22zs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Started pod/payload-processing-e2e-ait-f955d283-ddc87bc75-fdqxb Container started 51m Normal Created pod/payload-processing-e2e-ait-f955d283-ddc87bc75-fdqxb Container created 51m Normal Pulled pod/payload-processing-e2e-ait-f955d283-ddc87bc75-fdqxb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Created pod/payload-pre-processing-e2e-ait-f955d283-6798d4f4dc-b22zs Container created 51m Normal AddedInterface pod/payload-pre-processing-e2e-ait-f955d283-6798d4f4dc-b22zs Add eth0 [10.134.0.135/23] from ovn-kubernetes 51m Normal AddedInterface pod/payload-processing-e2e-ait-f955d283-ddc87bc75-fdqxb Add eth0 [10.134.0.136/23] from ovn-kubernetes 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-d0d57533 Scaled up replica set payload-pre-processing-e2e-derive-d0d57533-8598cb8b99 from 0 to 1 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-d0d57533-8598cb8b99 Created pod: payload-pre-processing-e2e-derive-d0d57533-8598cb8b99-n7sv6 51m Warning FailedCreate replicaset/payload-processing-e2e-derive-d0d57533-7b666f8685 Error creating: pods "payload-processing-e2e-derive-d0d57533-7b666f8685-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-d0d57533: serviceaccount "payload-processing-e2e-derive-d0d57533" not found 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-d0d57533 Scaled up replica set payload-processing-e2e-derive-d0d57533-7b666f8685 from 0 to 1 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-d0d57533-8598cb8b99 Error creating: pods "payload-pre-processing-e2e-derive-d0d57533-8598cb8b99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-d0d57533: serviceaccount "payload-processing-e2e-derive-d0d57533" not found 51m Normal SuccessfulCreate replicaset/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 Created pod: e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-q5fbc 51m Normal SuccessfulCreate replicaset/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 Created pod: e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-8xtk5 51m Normal SuccessfulCreate replicaset/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 (combined from similar events): Created pod: e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-8wf8c 51m Normal AddedInterface pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-hnrhd Add eth0 [10.132.0.86/23] from ovn-kubernetes 51m Normal AddedInterface pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-7rr6b Add eth0 [10.133.0.94/23] from ovn-kubernetes 51m Normal AddedInterface pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-q5fbc Add eth0 [10.132.0.87/23] from ovn-kubernetes 51m Normal SuccessfulCreate replicaset/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 Created pod: e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-7rr6b 51m Normal ScalingReplicaSet deployment/e2e-worker-w5-c9ae21-openshift-default Scaled up replica set e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 from 5 to 10 51m Normal SuccessfulCreate replicaset/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 Created pod: e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-hnrhd 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-c9ae21-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-d0d57533-7b666f8685 Created pod: payload-processing-e2e-derive-d0d57533-7b666f8685-dlwfv 51m Normal AddedInterface pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-8wf8c Add eth0 [10.133.0.95/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-q5fbc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Pulled pod/payload-pre-processing-e2e-derive-d0d57533-8598cb8b99-n7sv6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-q5fbc Container started 51m Normal AddedInterface pod/payload-processing-e2e-derive-d0d57533-7b666f8685-dlwfv Add eth0 [10.134.0.138/23] from ovn-kubernetes 51m Normal Pulled pod/payload-processing-e2e-derive-d0d57533-7b666f8685-dlwfv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Created pod/payload-processing-e2e-derive-d0d57533-7b666f8685-dlwfv Container created 51m Normal Started pod/payload-processing-e2e-derive-d0d57533-7b666f8685-dlwfv Container started 51m Normal AddedInterface pod/payload-pre-processing-e2e-derive-d0d57533-8598cb8b99-n7sv6 Add eth0 [10.134.0.137/23] from ovn-kubernetes 51m Normal Started pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-hnrhd Container started 51m Normal Started pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-8xtk5 Container started 51m Normal Created pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-8xtk5 Container created 51m Normal Pulled pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-8xtk5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-8xtk5 Add eth0 [10.134.0.139/23] from ovn-kubernetes 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-a0ef08b4 Scaled up replica set payload-processing-e2e-ait-a0ef08b4-7bd7876d5f from 0 to 1 51m Normal Created pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-hnrhd Container created 51m Normal Started pod/payload-pre-processing-e2e-derive-d0d57533-8598cb8b99-n7sv6 Container started 51m Normal Pulled pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-hnrhd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-8wf8c Container started 51m Normal Created pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-8wf8c Container created 51m Normal Pulled pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-8wf8c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Pulled pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-7rr6b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-7rr6b Container created 51m Normal Created pod/payload-pre-processing-e2e-derive-d0d57533-8598cb8b99-n7sv6 Container created 51m Normal Created pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-q5fbc Container created 51m Normal Started pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-7rr6b Container started 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-a0ef08b4-579f9b795 Error creating: pods "payload-pre-processing-e2e-ait-a0ef08b4-579f9b795-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-a0ef08b4: serviceaccount "payload-processing-e2e-ait-a0ef08b4" not found 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-a0ef08b4-7bd7876d5f Created pod: payload-processing-e2e-ait-a0ef08b4-7bd7876d5f-jcbw7 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-a0ef08b4 Scaled up replica set payload-pre-processing-e2e-ait-a0ef08b4-579f9b795 from 0 to 1 51m Warning FailedCreate replicaset/payload-processing-e2e-ait-a0ef08b4-7bd7876d5f Error creating: pods "payload-processing-e2e-ait-a0ef08b4-7bd7876d5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-a0ef08b4: serviceaccount "payload-processing-e2e-ait-a0ef08b4" not found 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-a0ef08b4-579f9b795 Created pod: payload-pre-processing-e2e-ait-a0ef08b4-579f9b795-ffw68 51m Normal AddedInterface pod/payload-pre-processing-e2e-ait-a0ef08b4-579f9b795-ffw68 Add eth0 [10.132.0.88/23] from ovn-kubernetes 51m Normal Pulled pod/payload-pre-processing-e2e-ait-a0ef08b4-579f9b795-ffw68 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Started pod/payload-processing-e2e-ait-a0ef08b4-7bd7876d5f-jcbw7 Container started 51m Normal Pulled pod/payload-processing-e2e-ait-a0ef08b4-7bd7876d5f-jcbw7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/payload-processing-e2e-ait-a0ef08b4-7bd7876d5f-jcbw7 Add eth0 [10.134.0.140/23] from ovn-kubernetes 51m Normal Created pod/payload-processing-e2e-ait-a0ef08b4-7bd7876d5f-jcbw7 Container created 51m Normal Created pod/payload-pre-processing-e2e-ait-a0ef08b4-579f9b795-ffw68 Container created 51m Normal Started pod/payload-pre-processing-e2e-ait-a0ef08b4-579f9b795-ffw68 Container started 51m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-kmd5t 51m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 51m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 51m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-kmd5t Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-kmd5t Add eth0 [10.132.0.89/23] from ovn-kubernetes 51m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-kmd5t Container created 51m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-kmd5t Container started 51m Warning Unhealthy pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-7rr6b Startup probe failed: Get "http://10.133.0.94:15021/healthz/ready": dial tcp 10.133.0.94:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-8wf8c Startup probe failed: Get "http://10.133.0.95:15021/healthz/ready": dial tcp 10.133.0.95:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-hnrhd Startup probe failed: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-7rr6b Startup probe error: Get "http://10.133.0.94:15021/healthz/ready": dial tcp 10.133.0.94:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-8wf8c Startup probe error: Get "http://10.133.0.95:15021/healthz/ready": dial tcp 10.133.0.95:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-q5fbc Startup probe failed: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-q5fbc Startup probe error: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-8xtk5 Startup probe failed: Get "http://10.134.0.139:15021/healthz/ready": dial tcp 10.134.0.139:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-hnrhd Startup probe error: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-8xtk5 Startup probe error: Get "http://10.134.0.139:15021/healthz/ready": dial tcp 10.134.0.139:15021: connect: connection refused... 51m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-kmd5t Readiness probe error: HTTP probe failed with statuscode: 503... 51m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-kmd5t Readiness probe failed: HTTP probe failed with statuscode: 503 50m Warning Unhealthy pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-mfk4k Readiness probe failed: Get "http://10.133.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning Unhealthy pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-gx9rt Readiness probe failed: Get "http://10.133.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-gx9rt Readiness probe error: Get "http://10.133.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-f955d283-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-f955d283-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning Unhealthy pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-qjcxh Readiness probe failed: Get "http://10.133.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-qjcxh Readiness probe error: Get "http://10.133.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning ProbeError pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-mfk4k Readiness probe error: Get "http://10.133.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-blcnw Readiness probe failed: Get "http://10.134.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-blcnw Readiness probe error: Get "http://10.134.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-psptj Readiness probe failed: Get "http://10.134.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-psptj Readiness probe error: Get "http://10.134.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 50m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-58c6585447 Created pod: maas-default-gateway-openshift-default-58c6585447-z4sdv 50m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-58c6585447 Created pod: maas-default-gateway-openshift-default-58c6585447-qnwxz 50m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-58c6585447 Created pod: maas-default-gateway-openshift-default-58c6585447-pfnvk 50m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-pfnvk Add eth0 [10.133.0.99/23] from ovn-kubernetes 50m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-qnwxz Add eth0 [10.132.0.90/23] from ovn-kubernetes 50m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-pfnvk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-pfnvk Container created 50m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-z4sdv Container started 50m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-qnwxz Container created 50m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-qnwxz Container started 50m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-qnwxz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-z4sdv Container created 50m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-z4sdv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-z4sdv Add eth0 [10.134.0.143/23] from ovn-kubernetes 50m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-pfnvk Container started 50m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-pfnvk Startup probe error: Get "http://10.133.0.99:15021/healthz/ready": dial tcp 10.133.0.99:15021: connect: connection refused... 50m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-qnwxz Startup probe error: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused... 50m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-pfnvk Startup probe failed: Get "http://10.133.0.99:15021/healthz/ready": dial tcp 10.133.0.99:15021: connect: connection refused 50m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-qnwxz Startup probe failed: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused 50m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-z4sdv Startup probe error: Get "http://10.134.0.143:15021/healthz/ready": dial tcp 10.134.0.143:15021: connect: connection refused... 50m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-z4sdv Startup probe failed: Get "http://10.134.0.143:15021/healthz/ready": dial tcp 10.134.0.143:15021: connect: connection refused 50m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-z4sdv Startup probe failed: Get "http://10.134.0.143:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-z4sdv Startup probe error: Get "http://10.134.0.143:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-58c6585447 Created pod: maas-default-gateway-openshift-default-58c6585447-qzw6j 50m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-qzw6j Container started 50m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-qzw6j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-qzw6j Add eth0 [10.133.0.100/23] from ovn-kubernetes 50m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-qzw6j Container created 50m Normal Started pod/e2e-ait-a0ef08b4-openshift-default-58c89679c-qf9jq Container started 50m Normal Pulled pod/e2e-ait-a0ef08b4-openshift-default-58c89679c-qf9jq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-ait-a0ef08b4-openshift-default-58c89679c-qf9jq Container created 50m Warning ProbeError pod/e2e-ait-f955d283-openshift-default-775bfb5f57-x79dv Readiness probe error: Get "http://10.134.0.133:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-qzw6j Startup probe failed: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused 50m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-qzw6j Startup probe error: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-ait-f955d283-openshift-default-775bfb5f57-x79dv Readiness probe failed: Get "http://10.134.0.133:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Normal Created pod/e2e-ait-f955d283-openshift-default-775bfb5f57-x79dv Container created 50m Normal Pulled pod/e2e-ait-f955d283-openshift-default-775bfb5f57-x79dv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-ait-f955d283-openshift-default-775bfb5f57-x79dv Container started 49m Warning ProbeError pod/e2e-ait-a0ef08b4-openshift-default-58c89679c-qf9jq Startup probe error: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-ait-a0ef08b4-openshift-default-58c89679c-qf9jq Startup probe failed: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-ait-f955d283-openshift-default-775bfb5f57-x79dv Startup probe failed: Get "http://10.134.0.133:15021/healthz/ready": dial tcp 10.134.0.133:15021: connect: connection refused 49m Warning ProbeError pod/e2e-ait-f955d283-openshift-default-775bfb5f57-x79dv Startup probe error: Get "http://10.134.0.133:15021/healthz/ready": dial tcp 10.134.0.133:15021: connect: connection refused... 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-a0ef08b4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-a0ef08b4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-wwpvf Readiness probe failed: Get "http://10.134.0.33:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-f955d283-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-wwpvf Readiness probe error: Get "http://10.134.0.33:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-f955d283-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-a0ef08b4-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-a0ef08b4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Normal Created pod/e2e-derive-d0d57533-openshift-default-bb8564d89-mmmrv Container created 49m Normal Started pod/e2e-derive-d0d57533-openshift-default-bb8564d89-mmmrv Container started 49m Normal Pulled pod/e2e-derive-d0d57533-openshift-default-bb8564d89-mmmrv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Warning ProbeError pod/e2e-derive-d0d57533-openshift-default-bb8564d89-mmmrv Startup probe error: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-derive-d0d57533-openshift-default-bb8564d89-mmmrv Startup probe failed: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-d0d57533-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-d0d57533-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Normal Killing pod/payload-processing-e2e-ait-f955d283-ddc87bc75-fdqxb Stopping container payload-processing 49m Normal Killing pod/payload-pre-processing-e2e-ait-f955d283-6798d4f4dc-b22zs Stopping container payload-pre-processing 49m Normal Killing pod/payload-pre-processing-e2e-ait-a0ef08b4-579f9b795-ffw68 Stopping container payload-pre-processing 49m Normal Killing pod/payload-processing-e2e-ait-a0ef08b4-7bd7876d5f-jcbw7 Stopping container payload-processing 49m Normal Killing pod/payload-processing-e2e-derive-d0d57533-7b666f8685-dlwfv Stopping container payload-processing 49m Normal Killing pod/payload-pre-processing-e2e-derive-d0d57533-8598cb8b99-n7sv6 Stopping container payload-pre-processing 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-d0d57533-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-d0d57533-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Normal Killing pod/e2e-ait-f955d283-openshift-default-775bfb5f57-x79dv Stopping container istio-proxy 48m Normal AddedLabel gateway/e2e-migrate-8ddfb06d-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-8ddfb06d-gw 48m Normal SuccessfulCreate replicaset/e2e-migrate-8ddfb06d-gw-openshift-default-7b966bd4bf Created pod: e2e-migrate-8ddfb06d-gw-openshift-default-7b966bd4bf-88hxr 48m Normal ScalingReplicaSet deployment/e2e-migrate-8ddfb06d-gw-openshift-default Scaled up replica set e2e-migrate-8ddfb06d-gw-openshift-default-7b966bd4bf from 0 to 1 48m Normal Killing pod/e2e-ait-a0ef08b4-openshift-default-58c89679c-qf9jq Stopping container istio-proxy 48m Normal Killing pod/e2e-derive-d0d57533-openshift-default-bb8564d89-mmmrv Stopping container istio-proxy 48m Normal AddedInterface pod/e2e-migrate-8ddfb06d-gw-openshift-default-7b966bd4bf-88hxr Add eth0 [10.134.0.149/23] from ovn-kubernetes 48m Warning ProbeError pod/e2e-ait-a0ef08b4-openshift-default-58c89679c-qf9jq Readiness probe error: HTTP probe failed with statuscode: 503... 48m Warning Unhealthy pod/e2e-ait-a0ef08b4-openshift-default-58c89679c-qf9jq Readiness probe failed: HTTP probe failed with statuscode: 503 48m Normal AddedLabel gateway/e2e-ait-74ab02fe Added label istio.io/rev=openshift-gateway to gateway e2e-ait-74ab02fe 48m Normal ScalingReplicaSet deployment/e2e-ait-74ab02fe-openshift-default Scaled up replica set e2e-ait-74ab02fe-openshift-default-577bf6d567 from 0 to 1 48m Normal SuccessfulCreate replicaset/e2e-ait-74ab02fe-openshift-default-577bf6d567 Created pod: e2e-ait-74ab02fe-openshift-default-577bf6d567-wqvkg 48m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-58c6585447 Created pod: maas-default-gateway-openshift-default-58c6585447-7b8v8 48m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-58c6585447 Created pod: maas-default-gateway-openshift-default-58c6585447-rfb2k 48m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-58c6585447 Created pod: maas-default-gateway-openshift-default-58c6585447-nd97z 48m Normal AddedInterface pod/e2e-ait-74ab02fe-openshift-default-577bf6d567-wqvkg Add eth0 [10.134.0.150/23] from ovn-kubernetes 48m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-nd97z Add eth0 [10.132.0.91/23] from ovn-kubernetes 48m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-rfb2k Add eth0 [10.133.0.101/23] from ovn-kubernetes 48m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-7b8v8 Add eth0 [10.134.0.151/23] from ovn-kubernetes 48m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-7b8v8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-7b8v8 Container created 48m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-nd97z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-nd97z Container started 48m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-7b8v8 Container started 48m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-rfb2k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-rfb2k Container created 48m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-rfb2k Container started 48m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-nd97z Container created 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-8ddfb06d-66db94bbb7 Created pod: payload-pre-processing-e2e-migrate-8ddfb06d-66db94bbb7-2fxht 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-8ddfb06d-66db94bbb7 Error creating: pods "payload-pre-processing-e2e-migrate-8ddfb06d-66db94bbb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-8ddfb06d: serviceaccount "payload-processing-e2e-migrate-8ddfb06d" not found 48m Warning FailedCreate replicaset/payload-processing-e2e-migrate-8ddfb06d-54d7b96ff7 Error creating: pods "payload-processing-e2e-migrate-8ddfb06d-54d7b96ff7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-8ddfb06d: serviceaccount "payload-processing-e2e-migrate-8ddfb06d" not found 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-8ddfb06d Scaled up replica set payload-pre-processing-e2e-migrate-8ddfb06d-66db94bbb7 from 0 to 1 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-8ddfb06d Scaled up replica set payload-processing-e2e-migrate-8ddfb06d-54d7b96ff7 from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-8ddfb06d-54d7b96ff7 Created pod: payload-processing-e2e-migrate-8ddfb06d-54d7b96ff7-flgn4 48m Normal AddedInterface pod/payload-processing-e2e-migrate-8ddfb06d-54d7b96ff7-flgn4 Add eth0 [10.134.0.153/23] from ovn-kubernetes 48m Normal Started pod/payload-pre-processing-e2e-migrate-8ddfb06d-66db94bbb7-2fxht Container started 48m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-8ddfb06d-66db94bbb7-2fxht Add eth0 [10.134.0.152/23] from ovn-kubernetes 48m Normal Pulled pod/payload-processing-e2e-migrate-8ddfb06d-54d7b96ff7-flgn4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/payload-processing-e2e-migrate-8ddfb06d-54d7b96ff7-flgn4 Container created 48m Normal Created pod/payload-pre-processing-e2e-migrate-8ddfb06d-66db94bbb7-2fxht Container created 48m Normal Started pod/payload-processing-e2e-migrate-8ddfb06d-54d7b96ff7-flgn4 Container started 48m Normal Pulled pod/payload-pre-processing-e2e-migrate-8ddfb06d-66db94bbb7-2fxht Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-nd97z Startup probe failed: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused 48m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-nd97z Startup probe error: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused... 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-74ab02fe-78888c8555 Created pod: payload-pre-processing-e2e-ait-74ab02fe-78888c8555-t9shv 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-74ab02fe-78888c8555 Error creating: pods "payload-pre-processing-e2e-ait-74ab02fe-78888c8555-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-74ab02fe: serviceaccount "payload-processing-e2e-ait-74ab02fe" not found 48m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-rfb2k Startup probe error: Get "http://10.133.0.101:15021/healthz/ready": dial tcp 10.133.0.101:15021: connect: connection refused... 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-74ab02fe Scaled up replica set payload-processing-e2e-ait-74ab02fe-7c85d47c89 from 0 to 1 48m Warning FailedCreate replicaset/payload-processing-e2e-ait-74ab02fe-7c85d47c89 Error creating: pods "payload-processing-e2e-ait-74ab02fe-7c85d47c89-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-74ab02fe: serviceaccount "payload-processing-e2e-ait-74ab02fe" not found 48m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-rfb2k Startup probe failed: Get "http://10.133.0.101:15021/healthz/ready": dial tcp 10.133.0.101:15021: connect: connection refused 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-74ab02fe Scaled up replica set payload-pre-processing-e2e-ait-74ab02fe-78888c8555 from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-74ab02fe-7c85d47c89 Created pod: payload-processing-e2e-ait-74ab02fe-7c85d47c89-2fpz8 48m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-7b8v8 Startup probe error: Get "http://10.134.0.151:15021/healthz/ready": dial tcp 10.134.0.151:15021: connect: connection refused... 48m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-7b8v8 Startup probe failed: Get "http://10.134.0.151:15021/healthz/ready": dial tcp 10.134.0.151:15021: connect: connection refused 48m Normal Created pod/payload-processing-e2e-ait-74ab02fe-7c85d47c89-2fpz8 Container created 48m Normal Pulled pod/payload-processing-e2e-ait-74ab02fe-7c85d47c89-2fpz8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/payload-pre-processing-e2e-ait-74ab02fe-78888c8555-t9shv Container created 48m Normal Started pod/payload-processing-e2e-ait-74ab02fe-7c85d47c89-2fpz8 Container started 48m Normal Started pod/payload-pre-processing-e2e-ait-74ab02fe-78888c8555-t9shv Container started 48m Normal AddedInterface pod/payload-pre-processing-e2e-ait-74ab02fe-78888c8555-t9shv Add eth0 [10.134.0.154/23] from ovn-kubernetes 48m Normal Pulled pod/payload-pre-processing-e2e-ait-74ab02fe-78888c8555-t9shv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-processing-e2e-ait-74ab02fe-7c85d47c89-2fpz8 Add eth0 [10.134.0.155/23] from ovn-kubernetes 48m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-82z67 Add eth0 [10.132.0.92/23] from ovn-kubernetes 48m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-58c6585447 Created pod: maas-default-gateway-openshift-default-58c6585447-cxh5l 48m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-58c6585447 Created pod: maas-default-gateway-openshift-default-58c6585447-82z67 48m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-cxh5l Add eth0 [10.133.0.102/23] from ovn-kubernetes 48m Normal AddedLabel gateway/e2e-mt-a5ab4417 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-a5ab4417 48m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-cxh5l Container started 48m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-82z67 Container started 48m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-82z67 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-82z67 Container created 48m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-cxh5l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-cxh5l Container created 48m Normal SuccessfulCreate replicaset/e2e-mt-a5ab4417-openshift-default-86cdc589 Created pod: e2e-mt-a5ab4417-openshift-default-86cdc589-8hn7h 48m Normal ScalingReplicaSet deployment/e2e-mt-a5ab4417-openshift-default Scaled up replica set e2e-mt-a5ab4417-openshift-default-86cdc589 from 0 to 1 48m Normal AddedLabel gateway/e2e-mt-11a069b1 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-11a069b1 48m Normal ScalingReplicaSet deployment/e2e-mt-11a069b1-openshift-default Scaled up replica set e2e-mt-11a069b1-openshift-default-7d5dd8f549 from 0 to 1 48m Normal AddedInterface pod/e2e-mt-a5ab4417-openshift-default-86cdc589-8hn7h Add eth0 [10.134.0.156/23] from ovn-kubernetes 48m Normal SuccessfulCreate replicaset/e2e-mt-11a069b1-openshift-default-7d5dd8f549 Created pod: e2e-mt-11a069b1-openshift-default-7d5dd8f549-vx9f5 48m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 4 48m Normal AddedInterface pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-vx9f5 Add eth0 [10.133.0.103/23] from ovn-kubernetes 48m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-qg2c2 48m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 4; reason: cpu resource utilization (percentage of request) above target 48m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-gq6xw Add eth0 [10.134.0.157/23] from ovn-kubernetes 48m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-gq6xw 48m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-gq6xw Container created 48m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-gq6xw Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-qg2c2 Add eth0 [10.133.0.104/23] from ovn-kubernetes 48m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-qg2c2 Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 48m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-82z67 Startup probe failed: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused 48m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-82z67 Startup probe error: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused... 48m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-gq6xw Container started 48m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-cxh5l Startup probe error: Get "http://10.133.0.102:15021/healthz/ready": dial tcp 10.133.0.102:15021: connect: connection refused... 48m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-gq6xw Readiness probe error: HTTP probe failed with statuscode: 503... 48m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-cxh5l Startup probe failed: Get "http://10.133.0.102:15021/healthz/ready": dial tcp 10.133.0.102:15021: connect: connection refused 48m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-gq6xw Readiness probe failed: HTTP probe failed with statuscode: 503 48m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-qg2c2 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.109s (4.109s including waiting). Image size: 185476477 bytes. 48m Warning FailedCreate replicaset/payload-processing-e2e-mt-a5ab4417-6fcc7f8d97 Error creating: pods "payload-processing-e2e-mt-a5ab4417-6fcc7f8d97-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-a5ab4417: serviceaccount "payload-processing-e2e-mt-a5ab4417" not found 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-a5ab4417-5677fb94bc Error creating: pods "payload-pre-processing-e2e-mt-a5ab4417-5677fb94bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-a5ab4417: serviceaccount "payload-processing-e2e-mt-a5ab4417" not found 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-a5ab4417-5677fb94bc Created pod: payload-pre-processing-e2e-mt-a5ab4417-5677fb94bc-mg6v2 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-a5ab4417 Scaled up replica set payload-pre-processing-e2e-mt-a5ab4417-5677fb94bc from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-a5ab4417-6fcc7f8d97 Created pod: payload-processing-e2e-mt-a5ab4417-6fcc7f8d97-gpp76 48m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-qg2c2 Container started 48m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-qg2c2 Container created 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-a5ab4417 Scaled up replica set payload-processing-e2e-mt-a5ab4417-6fcc7f8d97 from 0 to 1 48m Normal Pulled pod/payload-processing-e2e-mt-a5ab4417-6fcc7f8d97-gpp76 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Started pod/payload-pre-processing-e2e-mt-a5ab4417-5677fb94bc-mg6v2 Container started 48m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-qg2c2 Readiness probe failed: HTTP probe failed with statuscode: 503 48m Normal AddedInterface pod/payload-pre-processing-e2e-mt-a5ab4417-5677fb94bc-mg6v2 Add eth0 [10.132.0.93/23] from ovn-kubernetes 48m Normal AddedInterface pod/payload-processing-e2e-mt-a5ab4417-6fcc7f8d97-gpp76 Add eth0 [10.132.0.94/23] from ovn-kubernetes 48m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-qg2c2 Readiness probe error: HTTP probe failed with statuscode: 503... 48m Normal Started pod/payload-processing-e2e-mt-a5ab4417-6fcc7f8d97-gpp76 Container started 48m Normal Created pod/payload-processing-e2e-mt-a5ab4417-6fcc7f8d97-gpp76 Container created 48m Normal Created pod/payload-pre-processing-e2e-mt-a5ab4417-5677fb94bc-mg6v2 Container created 48m Normal Pulled pod/payload-pre-processing-e2e-mt-a5ab4417-5677fb94bc-mg6v2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-11a069b1-7c85476fd5 Created pod: payload-processing-e2e-mt-11a069b1-7c85476fd5-nqm4x 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-11a069b1 Scaled up replica set payload-pre-processing-e2e-mt-11a069b1-6977f77fd7 from 0 to 1 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-11a069b1-6977f77fd7 Error creating: pods "payload-pre-processing-e2e-mt-11a069b1-6977f77fd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-11a069b1: serviceaccount "payload-processing-e2e-mt-11a069b1" not found 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-11a069b1 Scaled up replica set payload-processing-e2e-mt-11a069b1-7c85476fd5 from 0 to 1 48m Warning FailedCreate replicaset/payload-processing-e2e-mt-11a069b1-7c85476fd5 Error creating: pods "payload-processing-e2e-mt-11a069b1-7c85476fd5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-11a069b1: serviceaccount "payload-processing-e2e-mt-11a069b1" not found 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-11a069b1-6977f77fd7 Created pod: payload-pre-processing-e2e-mt-11a069b1-6977f77fd7-kh7wl 48m Normal Created pod/payload-processing-e2e-mt-11a069b1-7c85476fd5-nqm4x Container created 48m Normal Created pod/payload-pre-processing-e2e-mt-11a069b1-6977f77fd7-kh7wl Container created 48m Normal AddedInterface pod/payload-pre-processing-e2e-mt-11a069b1-6977f77fd7-kh7wl Add eth0 [10.132.0.95/23] from ovn-kubernetes 48m Normal Pulled pod/payload-processing-e2e-mt-11a069b1-7c85476fd5-nqm4x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-processing-e2e-mt-11a069b1-7c85476fd5-nqm4x Add eth0 [10.132.0.96/23] from ovn-kubernetes 48m Normal Pulled pod/payload-pre-processing-e2e-mt-11a069b1-6977f77fd7-kh7wl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Started pod/payload-pre-processing-e2e-mt-11a069b1-6977f77fd7-kh7wl Container started 48m Normal Started pod/payload-processing-e2e-mt-11a069b1-7c85476fd5-nqm4x Container started 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-74ab02fe-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-74ab02fe-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-a5ab4417-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-a5ab4417-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Normal AddedLabel gateway/e2e-shared-a-w1-dce296 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w1-dce296 47m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-dce296-openshift-default Scaled up replica set e2e-shared-a-w1-dce296-openshift-default-84fd9bf7ff from 0 to 1 47m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-dce296-openshift-default-84fd9bf7ff Created pod: e2e-shared-a-w1-dce296-openshift-default-84fd9bf7ff-htbpm 47m Normal AddedInterface pod/e2e-shared-a-w1-dce296-openshift-default-84fd9bf7ff-htbpm Add eth0 [10.132.0.100/23] from ovn-kubernetes 47m Warning ProbeError pod/e2e-migrate-8ddfb06d-gw-openshift-default-7b966bd4bf-88hxr Readiness probe error: Get "http://10.134.0.149:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-migrate-8ddfb06d-gw-openshift-default-7b966bd4bf-88hxr Readiness probe failed: Get "http://10.134.0.149:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-ait-74ab02fe-openshift-default-577bf6d567-wqvkg Readiness probe error: Get "http://10.134.0.150:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-ait-74ab02fe-openshift-default-577bf6d567-wqvkg Readiness probe failed: Get "http://10.134.0.150:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w1-dce296-85d75dbb6f Error creating: pods "payload-processing-e2e-shared-a-w1-dce296-85d75dbb6f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-dce296: serviceaccount "payload-processing-e2e-shared-a-w1-dce296" not found 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w1-dce296-65f5556975 Error creating: pods "payload-pre-processing-e2e-shared-a-w1-dce296-65f5556975-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-dce296: serviceaccount "payload-processing-e2e-shared-a-w1-dce296" not found 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w1-dce296 Scaled up replica set payload-processing-e2e-shared-a-w1-dce296-85d75dbb6f from 0 to 1 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w1-dce296 Scaled up replica set payload-pre-processing-e2e-shared-a-w1-dce296-65f5556975 from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w1-dce296-65f5556975 Created pod: payload-pre-processing-e2e-shared-a-w1-dce296-65f5556975-m2d7v 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w1-dce296-85d75dbb6f Created pod: payload-processing-e2e-shared-a-w1-dce296-85d75dbb6f-rq556 47m Normal Created pod/payload-processing-e2e-shared-a-w1-dce296-85d75dbb6f-rq556 Container created 47m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w1-dce296-65f5556975-m2d7v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-pre-processing-e2e-shared-a-w1-dce296-65f5556975-m2d7v Container created 47m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w1-dce296-65f5556975-m2d7v Add eth0 [10.134.0.159/23] from ovn-kubernetes 47m Normal AddedInterface pod/payload-processing-e2e-shared-a-w1-dce296-85d75dbb6f-rq556 Add eth0 [10.134.0.160/23] from ovn-kubernetes 47m Normal Pulled pod/payload-processing-e2e-shared-a-w1-dce296-85d75dbb6f-rq556 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Started pod/payload-pre-processing-e2e-shared-a-w1-dce296-65f5556975-m2d7v Container started 47m Normal Started pod/payload-processing-e2e-shared-a-w1-dce296-85d75dbb6f-rq556 Container started 47m Normal Created pod/e2e-migrate-8ddfb06d-gw-openshift-default-7b966bd4bf-88hxr Container created 47m Normal Pulled pod/e2e-ait-74ab02fe-openshift-default-577bf6d567-wqvkg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-ait-74ab02fe-openshift-default-577bf6d567-wqvkg Container created 47m Normal Pulled pod/e2e-migrate-8ddfb06d-gw-openshift-default-7b966bd4bf-88hxr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Started pod/e2e-ait-74ab02fe-openshift-default-577bf6d567-wqvkg Container started 47m Normal Started pod/e2e-migrate-8ddfb06d-gw-openshift-default-7b966bd4bf-88hxr Container started 47m Normal Killing pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-hlgfg Stopping container istio-proxy 47m Normal ScalingReplicaSet deployment/e2e-worker-w2-cd248f-openshift-default Scaled down replica set e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 from 9 to 6 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-cd248f-openshift-default New size: 6; reason: All metrics below target 47m Normal SuccessfulDelete replicaset/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 Deleted pod: e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-xzjrw 47m Normal Killing pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-nrzth Stopping container istio-proxy 47m Normal SuccessfulDelete replicaset/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 Deleted pod: e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-nrzth 47m Normal SuccessfulDelete replicaset/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 Deleted pod: e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-hlgfg 47m Normal Killing pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-xzjrw Stopping container istio-proxy 47m Warning Unhealthy pod/e2e-ait-74ab02fe-openshift-default-577bf6d567-wqvkg Startup probe failed: Get "http://10.134.0.150:15021/healthz/ready": dial tcp 10.134.0.150:15021: connect: connection refused 47m Warning ProbeError pod/e2e-ait-74ab02fe-openshift-default-577bf6d567-wqvkg Startup probe error: Get "http://10.134.0.150:15021/healthz/ready": dial tcp 10.134.0.150:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-hlgfg Readiness probe failed: HTTP probe failed with statuscode: 503 47m Warning ProbeError pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-hlgfg Readiness probe error: HTTP probe failed with statuscode: 503... 47m Warning Unhealthy pod/e2e-migrate-8ddfb06d-gw-openshift-default-7b966bd4bf-88hxr Startup probe failed: Get "http://10.134.0.149:15021/healthz/ready": dial tcp 10.134.0.149:15021: connect: connection refused 47m Warning ProbeError pod/e2e-migrate-8ddfb06d-gw-openshift-default-7b966bd4bf-88hxr Startup probe error: Get "http://10.134.0.149:15021/healthz/ready": dial tcp 10.134.0.149:15021: connect: connection refused... 46m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-rfb2k Readiness probe error: Get "http://10.133.0.101:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-rfb2k Readiness probe failed: Get "http://10.133.0.101:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-cxh5l Readiness probe failed: Get "http://10.133.0.102:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-cxh5l Readiness probe error: Get "http://10.133.0.102:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-8ddfb06d-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-8ddfb06d-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal AddedLabel gateway/e2e-shared-a-w3-24f923 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-24f923 46m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-24f923-openshift-default Scaled up replica set e2e-shared-a-w3-24f923-openshift-default-645f69b989 from 0 to 1 46m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-24f923-openshift-default-645f69b989 Created pod: e2e-shared-a-w3-24f923-openshift-default-645f69b989-dhxtq 46m Normal AddedInterface pod/e2e-shared-a-w3-24f923-openshift-default-645f69b989-dhxtq Add eth0 [10.132.0.101/23] from ovn-kubernetes 46m Normal Created pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-vx9f5 Container created 46m Normal SuccessfulCreate replicaset/e2e-mt-a5ab4417-openshift-default-86cdc589 Created pod: e2e-mt-a5ab4417-openshift-default-86cdc589-6nzw2 46m Normal Pulled pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-vx9f5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal SuccessfulCreate replicaset/e2e-mt-a5ab4417-openshift-default-86cdc589 Created pod: e2e-mt-a5ab4417-openshift-default-86cdc589-kztdw 46m Normal SuccessfulCreate replicaset/e2e-mt-a5ab4417-openshift-default-86cdc589 Created pod: e2e-mt-a5ab4417-openshift-default-86cdc589-g2rzx 46m Normal ScalingReplicaSet deployment/e2e-mt-a5ab4417-openshift-default Scaled up replica set e2e-mt-a5ab4417-openshift-default-86cdc589 from 1 to 4 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-a5ab4417-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 46m Normal Started pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-vx9f5 Container started 46m Normal Created pod/e2e-mt-a5ab4417-openshift-default-86cdc589-6nzw2 Container created 46m Normal AddedInterface pod/e2e-mt-a5ab4417-openshift-default-86cdc589-6nzw2 Add eth0 [10.134.0.162/23] from ovn-kubernetes 46m Normal Created pod/e2e-mt-a5ab4417-openshift-default-86cdc589-g2rzx Container created 46m Normal AddedInterface pod/e2e-mt-a5ab4417-openshift-default-86cdc589-kztdw Add eth0 [10.132.0.102/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-mt-a5ab4417-openshift-default-86cdc589-kztdw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-mt-a5ab4417-openshift-default-86cdc589-kztdw Container created 46m Normal Started pod/e2e-mt-a5ab4417-openshift-default-86cdc589-kztdw Container started 46m Normal AddedInterface pod/e2e-mt-a5ab4417-openshift-default-86cdc589-g2rzx Add eth0 [10.133.0.106/23] from ovn-kubernetes 46m Normal Started pod/e2e-mt-a5ab4417-openshift-default-86cdc589-g2rzx Container started 46m Normal Pulled pod/e2e-mt-a5ab4417-openshift-default-86cdc589-6nzw2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-mt-a5ab4417-openshift-default-86cdc589-6nzw2 Container started 46m Normal Pulled pod/e2e-mt-a5ab4417-openshift-default-86cdc589-g2rzx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Pulled pod/e2e-mt-a5ab4417-openshift-default-86cdc589-8hn7h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-mt-a5ab4417-openshift-default-86cdc589-8hn7h Container created 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-11a069b1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-11a069b1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal Started pod/e2e-mt-a5ab4417-openshift-default-86cdc589-8hn7h Container started 46m Warning Unhealthy pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-vx9f5 Startup probe failed: Get "http://10.133.0.103:15021/healthz/ready": dial tcp 10.133.0.103:15021: connect: connection refused 46m Warning ProbeError pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-vx9f5 Startup probe error: Get "http://10.133.0.103:15021/healthz/ready": dial tcp 10.133.0.103:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-mt-a5ab4417-openshift-default-86cdc589-kztdw Startup probe failed: Get "http://10.132.0.102:15021/healthz/ready": dial tcp 10.132.0.102:15021: connect: connection refused 46m Warning ProbeError pod/e2e-mt-a5ab4417-openshift-default-86cdc589-kztdw Startup probe error: Get "http://10.132.0.102:15021/healthz/ready": dial tcp 10.132.0.102:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-mt-a5ab4417-openshift-default-86cdc589-g2rzx Startup probe failed: Get "http://10.133.0.106:15021/healthz/ready": dial tcp 10.133.0.106:15021: connect: connection refused 46m Warning ProbeError pod/e2e-mt-a5ab4417-openshift-default-86cdc589-g2rzx Startup probe error: Get "http://10.133.0.106:15021/healthz/ready": dial tcp 10.133.0.106:15021: connect: connection refused... 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w3-24f923 Scaled up replica set payload-pre-processing-e2e-shared-a-w3-24f923-869bcbd967 from 0 to 1 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w3-24f923 Scaled up replica set payload-processing-e2e-shared-a-w3-24f923-8474d84f5 from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w3-24f923-869bcbd967 Created pod: payload-pre-processing-e2e-shared-a-w3-24f923-869bcbd967-w25fl 46m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w3-24f923-8474d84f5 Error creating: pods "payload-processing-e2e-shared-a-w3-24f923-8474d84f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-24f923: serviceaccount "payload-processing-e2e-shared-a-w3-24f923" not found 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w3-24f923-869bcbd967 Error creating: pods "payload-pre-processing-e2e-shared-a-w3-24f923-869bcbd967-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-24f923: serviceaccount "payload-processing-e2e-shared-a-w3-24f923" not found 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w3-24f923-8474d84f5 Created pod: payload-processing-e2e-shared-a-w3-24f923-8474d84f5-kfvdm 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-8ddfb06d-gw-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning ProbeError pod/e2e-mt-a5ab4417-openshift-default-86cdc589-6nzw2 Startup probe error: Get "http://10.134.0.162:15021/healthz/ready": dial tcp 10.134.0.162:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-mt-a5ab4417-openshift-default-86cdc589-6nzw2 Startup probe failed: Get "http://10.134.0.162:15021/healthz/ready": dial tcp 10.134.0.162:15021: connect: connection refused 46m Normal AddedInterface pod/payload-processing-e2e-shared-a-w3-24f923-8474d84f5-kfvdm Add eth0 [10.134.0.165/23] from ovn-kubernetes 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-8ddfb06d-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning ProbeError pod/e2e-mt-a5ab4417-openshift-default-86cdc589-8hn7h Startup probe error: Get "http://10.134.0.156:15021/healthz/ready": dial tcp 10.134.0.156:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-mt-a5ab4417-openshift-default-86cdc589-8hn7h Startup probe failed: Get "http://10.134.0.156:15021/healthz/ready": dial tcp 10.134.0.156:15021: connect: connection refused 46m Normal Pulled pod/payload-processing-e2e-shared-a-w3-24f923-8474d84f5-kfvdm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Started pod/payload-pre-processing-e2e-shared-a-w3-24f923-869bcbd967-w25fl Container started 46m Normal Created pod/payload-pre-processing-e2e-shared-a-w3-24f923-869bcbd967-w25fl Container created 46m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w3-24f923-869bcbd967-w25fl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w3-24f923-869bcbd967-w25fl Add eth0 [10.134.0.163/23] from ovn-kubernetes 46m Normal Created pod/payload-processing-e2e-shared-a-w3-24f923-8474d84f5-kfvdm Container created 46m Normal Started pod/payload-processing-e2e-shared-a-w3-24f923-8474d84f5-kfvdm Container started 46m Normal Killing pod/payload-pre-processing-e2e-migrate-8ddfb06d-66db94bbb7-2fxht Stopping container payload-pre-processing 46m Normal Killing pod/payload-processing-e2e-migrate-8ddfb06d-54d7b96ff7-flgn4 Stopping container payload-processing 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-74ab02fe-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-74ab02fe-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-24f923-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-24f923-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal Killing pod/payload-pre-processing-e2e-ait-74ab02fe-78888c8555-t9shv Stopping container payload-pre-processing 46m Normal Killing pod/payload-processing-e2e-ait-74ab02fe-7c85d47c89-2fpz8 Stopping container payload-processing 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-pfnvk Readiness probe failed: Get "http://10.133.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-pfnvk Readiness probe error: Get "http://10.133.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-a5ab4417-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-a5ab4417-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-11a069b1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-11a069b1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal SuccessfulCreate replicaset/e2e-mt-a5ab4417-openshift-default-86cdc589 Created pod: e2e-mt-a5ab4417-openshift-default-86cdc589-6b97v 46m Normal SuccessfulCreate replicaset/e2e-mt-a5ab4417-openshift-default-86cdc589 Created pod: e2e-mt-a5ab4417-openshift-default-86cdc589-9tm7w 46m Normal ScalingReplicaSet deployment/e2e-mt-a5ab4417-openshift-default Scaled up replica set e2e-mt-a5ab4417-openshift-default-86cdc589 from 4 to 6 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-a5ab4417-openshift-default New size: 6; reason: All metrics below target 46m Normal Created pod/e2e-mt-a5ab4417-openshift-default-86cdc589-6b97v Container created 46m Normal AddedInterface pod/e2e-mt-a5ab4417-openshift-default-86cdc589-6b97v Add eth0 [10.133.0.108/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-mt-a5ab4417-openshift-default-86cdc589-6b97v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-mt-a5ab4417-openshift-default-86cdc589-6b97v Container started 46m Normal AddedInterface pod/e2e-mt-a5ab4417-openshift-default-86cdc589-9tm7w Add eth0 [10.132.0.103/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-mt-a5ab4417-openshift-default-86cdc589-9tm7w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-mt-a5ab4417-openshift-default-86cdc589-9tm7w Container created 46m Normal Started pod/e2e-mt-a5ab4417-openshift-default-86cdc589-9tm7w Container started 46m Normal Killing pod/payload-pre-processing-e2e-mt-a5ab4417-5677fb94bc-mg6v2 Stopping container payload-pre-processing 46m Warning ProbeError pod/e2e-mt-a5ab4417-openshift-default-86cdc589-9tm7w Startup probe error: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused... 46m Normal Killing pod/payload-processing-e2e-mt-a5ab4417-6fcc7f8d97-gpp76 Stopping container payload-processing 46m Warning Unhealthy pod/e2e-mt-a5ab4417-openshift-default-86cdc589-9tm7w Startup probe failed: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused 45m Warning Unhealthy pod/e2e-mt-a5ab4417-openshift-default-86cdc589-6b97v Startup probe failed: Get "http://10.133.0.108:15021/healthz/ready": dial tcp 10.133.0.108:15021: connect: connection refused 45m Warning ProbeError pod/e2e-mt-a5ab4417-openshift-default-86cdc589-6b97v Startup probe error: Get "http://10.133.0.108:15021/healthz/ready": dial tcp 10.133.0.108:15021: connect: connection refused... 45m Normal Killing pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-kndl9 Stopping container istio-proxy 45m Normal SuccessfulDelete replicaset/e2e-worker-w3-944263-openshift-default-5f78997bc9 Deleted pod: e2e-worker-w3-944263-openshift-default-5f78997bc9-gjkgx 45m Normal Killing pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-czlxh Stopping container istio-proxy 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-944263-openshift-default New size: 7; reason: All metrics below target 45m Normal ScalingReplicaSet deployment/e2e-worker-w3-944263-openshift-default Scaled down replica set e2e-worker-w3-944263-openshift-default-5f78997bc9 from 10 to 7 45m Normal SuccessfulDelete replicaset/e2e-worker-w3-944263-openshift-default-5f78997bc9 Deleted pod: e2e-worker-w3-944263-openshift-default-5f78997bc9-kndl9 45m Normal SuccessfulDelete replicaset/e2e-worker-w3-944263-openshift-default-5f78997bc9 Deleted pod: e2e-worker-w3-944263-openshift-default-5f78997bc9-czlxh 45m Normal Killing pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-gjkgx Stopping container istio-proxy 45m Normal Killing pod/e2e-migrate-8ddfb06d-gw-openshift-default-7b966bd4bf-88hxr Stopping container istio-proxy 45m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-de8145-openshift-default-57f9f8965 Created pod: e2e-shared-a-w0-de8145-openshift-default-57f9f8965-mpzk6 45m Normal AddedLabel gateway/e2e-shared-a-w0-de8145 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-de8145 45m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-de8145-openshift-default Scaled up replica set e2e-shared-a-w0-de8145-openshift-default-57f9f8965 from 0 to 1 45m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-a9b439-openshift-default Scaled up replica set e2e-shared-a-w4-a9b439-openshift-default-77b865d7d7 from 0 to 1 45m Normal Killing pod/e2e-ait-74ab02fe-openshift-default-577bf6d567-wqvkg Stopping container istio-proxy 45m Normal AddedInterface pod/e2e-shared-a-w0-de8145-openshift-default-57f9f8965-mpzk6 Add eth0 [10.134.0.167/23] from ovn-kubernetes 45m Normal AddedLabel gateway/e2e-shared-a-w4-a9b439 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-a9b439 45m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-a9b439-openshift-default-77b865d7d7 Created pod: e2e-shared-a-w4-a9b439-openshift-default-77b865d7d7-cfn9m 45m Normal AddedInterface pod/e2e-shared-a-w4-a9b439-openshift-default-77b865d7d7-cfn9m Add eth0 [10.134.0.168/23] from ovn-kubernetes 45m Warning ProbeError pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-czlxh Readiness probe error: Get "http://10.134.0.95:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning Unhealthy pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-czlxh Readiness probe failed: Get "http://10.134.0.95:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning ProbeError pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-gjkgx Readiness probe error: Get "http://10.134.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning Unhealthy pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-gjkgx Readiness probe failed: Get "http://10.134.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning Unhealthy pod/e2e-migrate-8ddfb06d-gw-openshift-default-7b966bd4bf-88hxr Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/e2e-migrate-8ddfb06d-gw-openshift-default-7b966bd4bf-88hxr Readiness probe error: HTTP probe failed with statuscode: 503... 45m Normal Pulled pod/e2e-shared-a-w1-dce296-openshift-default-84fd9bf7ff-htbpm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-shared-a-w1-dce296-openshift-default-84fd9bf7ff-htbpm Container created 45m Normal Started pod/e2e-shared-a-w1-dce296-openshift-default-84fd9bf7ff-htbpm Container started 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-dce296-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-dce296-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w0-de8145-9dcc65c7f Created pod: payload-processing-e2e-shared-a-w0-de8145-9dcc65c7f-q6cff 45m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w0-de8145-9dcc65c7f Error creating: pods "payload-processing-e2e-shared-a-w0-de8145-9dcc65c7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-de8145: serviceaccount "payload-processing-e2e-shared-a-w0-de8145" not found 45m Warning ProbeError pod/e2e-shared-a-w1-dce296-openshift-default-84fd9bf7ff-htbpm Startup probe error: Get "http://10.132.0.100:15021/healthz/ready": dial tcp 10.132.0.100:15021: connect: connection refused... 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w0-de8145 Scaled up replica set payload-pre-processing-e2e-shared-a-w0-de8145-56fb74df5 from 0 to 1 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w0-de8145-56fb74df5 Error creating: pods "payload-pre-processing-e2e-shared-a-w0-de8145-56fb74df5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-de8145: serviceaccount "payload-processing-e2e-shared-a-w0-de8145" not found 45m Warning Unhealthy pod/e2e-shared-a-w1-dce296-openshift-default-84fd9bf7ff-htbpm Startup probe failed: Get "http://10.132.0.100:15021/healthz/ready": dial tcp 10.132.0.100:15021: connect: connection refused 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w0-de8145 Scaled up replica set payload-processing-e2e-shared-a-w0-de8145-9dcc65c7f from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w0-de8145-56fb74df5 Created pod: payload-pre-processing-e2e-shared-a-w0-de8145-56fb74df5-8nbgw 45m Normal AddedInterface pod/payload-processing-e2e-shared-a-w0-de8145-9dcc65c7f-q6cff Add eth0 [10.134.0.169/23] from ovn-kubernetes 45m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w0-de8145-56fb74df5-8nbgw Add eth0 [10.134.0.170/23] from ovn-kubernetes 45m Normal ScalingReplicaSet deployment/e2e-worker-w0-208f2b-openshift-default Scaled down replica set e2e-worker-w0-208f2b-openshift-default-5cc586d6f from 10 to 7 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-c9ae21-openshift-default New size: 7; reason: All metrics below target 45m Normal SuccessfulDelete replicaset/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 Deleted pod: e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-8wf8c 45m Normal Killing pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-gx9rt Stopping container istio-proxy 45m Normal Started pod/payload-processing-e2e-shared-a-w0-de8145-9dcc65c7f-q6cff Container started 45m Normal ScalingReplicaSet deployment/e2e-worker-w5-c9ae21-openshift-default Scaled down replica set e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 from 10 to 7 45m Normal SuccessfulDelete replicaset/e2e-worker-w0-208f2b-openshift-default-5cc586d6f Deleted pod: e2e-worker-w0-208f2b-openshift-default-5cc586d6f-jlkc9 45m Normal Killing pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-f7qth Stopping container istio-proxy 45m Normal Killing pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-8wf8c Stopping container istio-proxy 45m Normal Killing pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-7rr6b Stopping container istio-proxy 45m Normal SuccessfulDelete replicaset/e2e-worker-w0-208f2b-openshift-default-5cc586d6f Deleted pod: e2e-worker-w0-208f2b-openshift-default-5cc586d6f-mfk4k 45m Normal SuccessfulDelete replicaset/e2e-worker-w0-208f2b-openshift-default-5cc586d6f Deleted pod: e2e-worker-w0-208f2b-openshift-default-5cc586d6f-gx9rt 45m Normal Created pod/payload-pre-processing-e2e-shared-a-w0-de8145-56fb74df5-8nbgw Container created 45m Normal Created pod/payload-processing-e2e-shared-a-w0-de8145-9dcc65c7f-q6cff Container created 45m Normal Started pod/payload-pre-processing-e2e-shared-a-w0-de8145-56fb74df5-8nbgw Container started 45m Normal Pulled pod/payload-processing-e2e-shared-a-w0-de8145-9dcc65c7f-q6cff Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal SuccessfulDelete replicaset/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 Deleted pod: e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-7rr6b 45m Normal SuccessfulDelete replicaset/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 Deleted pod: e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-f7qth 45m Normal Killing pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-mfk4k Stopping container istio-proxy 45m Normal Killing pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-jlkc9 Stopping container istio-proxy 45m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w0-de8145-56fb74df5-8nbgw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-208f2b-openshift-default New size: 7; reason: All metrics below target 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-1a1e9e-openshift-default New size: 9; reason: All metrics below target 45m Normal ScalingReplicaSet deployment/e2e-worker-w4-1a1e9e-openshift-default Scaled down replica set e2e-worker-w4-1a1e9e-openshift-default-56c9685797 from 10 to 9 45m Normal Killing pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-b95tc Stopping container istio-proxy 45m Normal SuccessfulDelete replicaset/e2e-worker-w4-1a1e9e-openshift-default-56c9685797 Deleted pod: e2e-worker-w4-1a1e9e-openshift-default-56c9685797-b95tc 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w4-a9b439-76df769c99 Created pod: payload-pre-processing-e2e-shared-a-w4-a9b439-76df769c99-g6df9 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w4-a9b439-76df769c99 Error creating: pods "payload-pre-processing-e2e-shared-a-w4-a9b439-76df769c99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-a9b439: serviceaccount "payload-processing-e2e-shared-a-w4-a9b439" not found 45m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w4-a9b439-76bf5ff49 Error creating: pods "payload-processing-e2e-shared-a-w4-a9b439-76bf5ff49-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-a9b439: serviceaccount "payload-processing-e2e-shared-a-w4-a9b439" not found 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w4-a9b439 Scaled up replica set payload-processing-e2e-shared-a-w4-a9b439-76bf5ff49 from 0 to 1 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w4-a9b439 Scaled up replica set payload-pre-processing-e2e-shared-a-w4-a9b439-76df769c99 from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w4-a9b439-76bf5ff49 Created pod: payload-processing-e2e-shared-a-w4-a9b439-76bf5ff49-qqkwc 45m Normal AddedInterface pod/payload-processing-e2e-shared-a-w4-a9b439-76bf5ff49-qqkwc Add eth0 [10.134.0.173/23] from ovn-kubernetes 45m Normal Created pod/payload-processing-e2e-shared-a-w4-a9b439-76bf5ff49-qqkwc Container created 45m Warning Unhealthy pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-jlkc9 Readiness probe failed: HTTP probe failed with statuscode: 503 45m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w4-a9b439-76df769c99-g6df9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Started pod/payload-processing-e2e-shared-a-w4-a9b439-76bf5ff49-qqkwc Container started 45m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w4-a9b439-76df769c99-g6df9 Add eth0 [10.134.0.172/23] from ovn-kubernetes 45m Normal Started pod/payload-pre-processing-e2e-shared-a-w4-a9b439-76df769c99-g6df9 Container started 45m Normal Created pod/payload-pre-processing-e2e-shared-a-w4-a9b439-76df769c99-g6df9 Container created 45m Normal Pulled pod/payload-processing-e2e-shared-a-w4-a9b439-76bf5ff49-qqkwc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Warning ProbeError pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-f7qth Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning Unhealthy pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-f7qth Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-jlkc9 Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning Unhealthy pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-b95tc Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-b95tc Readiness probe error: HTTP probe failed with statuscode: 503... 45m Normal Killing pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-tt6hz Stopping container istio-proxy 45m Normal SuccessfulDelete replicaset/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 Deleted pod: e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-lsbvv 45m Normal SuccessfulDelete replicaset/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 Deleted pod: e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-zttrt 45m Normal Killing pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-zttrt Stopping container istio-proxy 45m Normal Killing pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-lsbvv Stopping container istio-proxy 45m Normal SuccessfulDelete replicaset/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 Deleted pod: e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-tt6hz 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-dd038b-openshift-default New size: 7; reason: All metrics below target 45m Normal ScalingReplicaSet deployment/e2e-worker-w6-dd038b-openshift-default Scaled down replica set e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 from 10 to 7 45m Normal Killing pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-gkhjr Stopping container istio-proxy 45m Normal Killing pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-cc24r Stopping container istio-proxy 45m Normal SuccessfulDelete replicaset/e2e-worker-w1-db0157-openshift-default-68856dc5c7 Deleted pod: e2e-worker-w1-db0157-openshift-default-68856dc5c7-cc24r 45m Normal SuccessfulDelete replicaset/e2e-worker-w1-db0157-openshift-default-68856dc5c7 Deleted pod: e2e-worker-w1-db0157-openshift-default-68856dc5c7-6wrqk 45m Normal SuccessfulDelete replicaset/e2e-worker-w1-db0157-openshift-default-68856dc5c7 Deleted pod: e2e-worker-w1-db0157-openshift-default-68856dc5c7-gkhjr 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-db0157-openshift-default New size: 7; reason: All metrics below target 45m Normal ScalingReplicaSet deployment/e2e-worker-w1-db0157-openshift-default Scaled down replica set e2e-worker-w1-db0157-openshift-default-68856dc5c7 from 10 to 7 45m Normal Killing pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-6wrqk Stopping container istio-proxy 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-1a1e9e-openshift-default New size: 7; reason: All metrics below target 45m Normal Killing pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-wmg5x Stopping container istio-proxy 45m Normal SuccessfulDelete replicaset/e2e-worker-w4-1a1e9e-openshift-default-56c9685797 Deleted pod: e2e-worker-w4-1a1e9e-openshift-default-56c9685797-wmg5x 45m Normal SuccessfulDelete replicaset/e2e-worker-w4-1a1e9e-openshift-default-56c9685797 Deleted pod: e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vnzfv 45m Normal ScalingReplicaSet deployment/e2e-worker-w4-1a1e9e-openshift-default Scaled down replica set e2e-worker-w4-1a1e9e-openshift-default-56c9685797 from 9 to 7 45m Normal Killing pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vnzfv Stopping container istio-proxy 45m Warning Unhealthy pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-6wrqk Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-6wrqk Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning ProbeError pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-cc24r Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning Unhealthy pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-zttrt Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning Unhealthy pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-cc24r Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-zttrt Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-a9b439-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-a9b439-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning Unhealthy pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vnzfv Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vnzfv Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning Unhealthy pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-wmg5x Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-wmg5x Readiness probe error: HTTP probe failed with statuscode: 503... 45m Normal Killing pod/e2e-mt-a5ab4417-openshift-default-86cdc589-6nzw2 Stopping container istio-proxy 45m Normal Killing pod/e2e-mt-a5ab4417-openshift-default-86cdc589-kztdw Stopping container istio-proxy 45m Warning Unhealthy pod/e2e-mt-a5ab4417-openshift-default-86cdc589-9tm7w Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/e2e-mt-a5ab4417-openshift-default-86cdc589-9tm7w Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning Unhealthy pod/e2e-mt-a5ab4417-openshift-default-86cdc589-g2rzx Readiness probe failed: HTTP probe failed with statuscode: 503 45m Normal Killing pod/e2e-mt-a5ab4417-openshift-default-86cdc589-g2rzx Stopping container istio-proxy 45m Normal Killing pod/e2e-mt-a5ab4417-openshift-default-86cdc589-6b97v Stopping container istio-proxy 45m Warning ProbeError pod/e2e-mt-a5ab4417-openshift-default-86cdc589-g2rzx Readiness probe error: HTTP probe failed with statuscode: 503... 45m Normal Killing pod/e2e-mt-a5ab4417-openshift-default-86cdc589-9tm7w Stopping container istio-proxy 45m Normal Killing pod/e2e-mt-a5ab4417-openshift-default-86cdc589-8hn7h Stopping container istio-proxy 45m Normal SuccessfulCreate replicaset/e2e-api-a-b27e66-openshift-default-5cccc745f7 Created pod: e2e-api-a-b27e66-openshift-default-5cccc745f7-nkvtl 45m Warning Unhealthy pod/e2e-mt-a5ab4417-openshift-default-86cdc589-6b97v Readiness probe failed: HTTP probe failed with statuscode: 503 45m Normal AddedLabel gateway/e2e-api-a-b27e66 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-b27e66 45m Warning ProbeError pod/e2e-mt-a5ab4417-openshift-default-86cdc589-6b97v Readiness probe error: HTTP probe failed with statuscode: 503... 45m Normal ScalingReplicaSet deployment/e2e-api-a-b27e66-openshift-default Scaled up replica set e2e-api-a-b27e66-openshift-default-5cccc745f7 from 0 to 1 45m Warning ProbeError pod/e2e-mt-a5ab4417-openshift-default-86cdc589-6nzw2 Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning ProbeError pod/e2e-mt-a5ab4417-openshift-default-86cdc589-8hn7h Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning Unhealthy pod/e2e-mt-a5ab4417-openshift-default-86cdc589-8hn7h Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning Unhealthy pod/e2e-mt-a5ab4417-openshift-default-86cdc589-6nzw2 Readiness probe failed: HTTP probe failed with statuscode: 503 45m Normal AddedLabel gateway/e2e-shared-b-w1-df700a Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w1-df700a 45m Normal AddedInterface pod/e2e-api-a-b27e66-openshift-default-5cccc745f7-nkvtl Add eth0 [10.134.0.174/23] from ovn-kubernetes 45m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-df700a-openshift-default Scaled up replica set e2e-shared-b-w1-df700a-openshift-default-6cf7fb69dc from 0 to 1 45m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-df700a-openshift-default-6cf7fb69dc Created pod: e2e-shared-b-w1-df700a-openshift-default-6cf7fb69dc-dg822 45m Normal AddedInterface pod/e2e-shared-b-w1-df700a-openshift-default-6cf7fb69dc-dg822 Add eth0 [10.134.0.175/23] from ovn-kubernetes 45m Normal Started pod/e2e-shared-a-w3-24f923-openshift-default-645f69b989-dhxtq Container started 45m Normal Created pod/e2e-shared-a-w3-24f923-openshift-default-645f69b989-dhxtq Container created 45m Normal Pulled pod/e2e-shared-a-w3-24f923-openshift-default-645f69b989-dhxtq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Warning FailedCreate replicaset/payload-processing-e2e-api-a-b27e66-6df49bd599 Error creating: pods "payload-processing-e2e-api-a-b27e66-6df49bd599-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-b27e66: serviceaccount "payload-processing-e2e-api-a-b27e66" not found 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-b27e66 Scaled up replica set payload-pre-processing-e2e-api-a-b27e66-5db66b9467 from 0 to 1 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-b27e66 Scaled up replica set payload-processing-e2e-api-a-b27e66-6df49bd599 from 0 to 1 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-b27e66-5db66b9467 Error creating: pods "payload-pre-processing-e2e-api-a-b27e66-5db66b9467-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-b27e66: serviceaccount "payload-processing-e2e-api-a-b27e66" not found 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-b27e66-5db66b9467 Created pod: payload-pre-processing-e2e-api-a-b27e66-5db66b9467-z8rv7 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-b27e66-6df49bd599 Created pod: payload-processing-e2e-api-a-b27e66-6df49bd599-n7wtm 45m Normal Started pod/payload-pre-processing-e2e-api-a-b27e66-5db66b9467-z8rv7 Container started 45m Normal Started pod/payload-processing-e2e-api-a-b27e66-6df49bd599-n7wtm Container started 45m Normal AddedInterface pod/payload-processing-e2e-api-a-b27e66-6df49bd599-n7wtm Add eth0 [10.134.0.177/23] from ovn-kubernetes 45m Normal Created pod/payload-pre-processing-e2e-api-a-b27e66-5db66b9467-z8rv7 Container created 45m Normal Pulled pod/payload-pre-processing-e2e-api-a-b27e66-5db66b9467-z8rv7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-b27e66-5db66b9467-z8rv7 Add eth0 [10.134.0.176/23] from ovn-kubernetes 45m Normal Pulled pod/payload-processing-e2e-api-a-b27e66-6df49bd599-n7wtm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-processing-e2e-api-a-b27e66-6df49bd599-n7wtm Container created 45m Warning Unhealthy pod/e2e-shared-a-w3-24f923-openshift-default-645f69b989-dhxtq Startup probe failed: Get "http://10.132.0.101:15021/healthz/ready": dial tcp 10.132.0.101:15021: connect: connection refused 45m Warning ProbeError pod/e2e-shared-a-w3-24f923-openshift-default-645f69b989-dhxtq Startup probe error: Get "http://10.132.0.101:15021/healthz/ready": dial tcp 10.132.0.101:15021: connect: connection refused... 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w1-df700a Scaled up replica set payload-pre-processing-e2e-shared-b-w1-df700a-789b6c9547 from 0 to 1 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w1-df700a Scaled up replica set payload-processing-e2e-shared-b-w1-df700a-788d57d789 from 0 to 1 44m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w1-df700a-788d57d789 Error creating: pods "payload-processing-e2e-shared-b-w1-df700a-788d57d789-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-df700a: serviceaccount "payload-processing-e2e-shared-b-w1-df700a" not found 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w1-df700a-789b6c9547 Created pod: payload-pre-processing-e2e-shared-b-w1-df700a-789b6c9547-g9ps7 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w1-df700a-789b6c9547 Error creating: pods "payload-pre-processing-e2e-shared-b-w1-df700a-789b6c9547-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-df700a: serviceaccount "payload-processing-e2e-shared-b-w1-df700a" not found 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w1-df700a-788d57d789 Created pod: payload-processing-e2e-shared-b-w1-df700a-788d57d789-n6jlx 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-dce296-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal Created pod/payload-pre-processing-e2e-shared-b-w1-df700a-789b6c9547-g9ps7 Container created 44m Normal Created pod/payload-processing-e2e-shared-b-w1-df700a-788d57d789-n6jlx Container created 44m Normal Pulled pod/payload-processing-e2e-shared-b-w1-df700a-788d57d789-n6jlx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-processing-e2e-shared-b-w1-df700a-788d57d789-n6jlx Add eth0 [10.134.0.179/23] from ovn-kubernetes 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-dce296-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w1-df700a-789b6c9547-g9ps7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Started pod/payload-pre-processing-e2e-shared-b-w1-df700a-789b6c9547-g9ps7 Container started 44m Normal Started pod/payload-processing-e2e-shared-b-w1-df700a-788d57d789-n6jlx Container started 44m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w1-df700a-789b6c9547-g9ps7 Add eth0 [10.134.0.178/23] from ovn-kubernetes 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-df700a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-df700a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal AddedLabel gateway/e2e-shared-b-w3-bf3fa2 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-bf3fa2 44m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-bf3fa2-openshift-default Scaled up replica set e2e-shared-b-w3-bf3fa2-openshift-default-69d7c5c975 from 0 to 1 44m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-bf3fa2-openshift-default-69d7c5c975 Created pod: e2e-shared-b-w3-bf3fa2-openshift-default-69d7c5c975-84bmk 44m Normal AddedInterface pod/e2e-shared-b-w3-bf3fa2-openshift-default-69d7c5c975-84bmk Add eth0 [10.134.0.180/23] from ovn-kubernetes 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-24f923-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-24f923-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w3-bf3fa2-79cb8b8f Error creating: pods "payload-processing-e2e-shared-b-w3-bf3fa2-79cb8b8f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-bf3fa2: serviceaccount "payload-processing-e2e-shared-b-w3-bf3fa2" not found 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w3-bf3fa2-5f4bb58487 Error creating: pods "payload-pre-processing-e2e-shared-b-w3-bf3fa2-5f4bb58487-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-bf3fa2: serviceaccount "payload-processing-e2e-shared-b-w3-bf3fa2" not found 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w3-bf3fa2 Scaled up replica set payload-processing-e2e-shared-b-w3-bf3fa2-79cb8b8f from 0 to 1 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w3-bf3fa2 Scaled up replica set payload-pre-processing-e2e-shared-b-w3-bf3fa2-5f4bb58487 from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w3-bf3fa2-79cb8b8f Created pod: payload-processing-e2e-shared-b-w3-bf3fa2-79cb8b8f-w5wfb 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w3-bf3fa2-5f4bb58487 Created pod: payload-pre-processing-e2e-shared-b-w3-bf3fa2-5f4bb58487-jrpbx 44m Normal Pulled pod/payload-processing-e2e-shared-b-w3-bf3fa2-79cb8b8f-w5wfb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w3-bf3fa2-5f4bb58487-jrpbx Add eth0 [10.134.0.181/23] from ovn-kubernetes 44m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w3-bf3fa2-5f4bb58487-jrpbx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-pre-processing-e2e-shared-b-w3-bf3fa2-5f4bb58487-jrpbx Container created 44m Normal Started pod/payload-pre-processing-e2e-shared-b-w3-bf3fa2-5f4bb58487-jrpbx Container started 44m Normal Started pod/payload-processing-e2e-shared-b-w3-bf3fa2-79cb8b8f-w5wfb Container started 44m Normal Created pod/payload-processing-e2e-shared-b-w3-bf3fa2-79cb8b8f-w5wfb Container created 44m Normal AddedInterface pod/payload-processing-e2e-shared-b-w3-bf3fa2-79cb8b8f-w5wfb Add eth0 [10.134.0.182/23] from ovn-kubernetes 44m Normal Started pod/e2e-shared-a-w0-de8145-openshift-default-57f9f8965-mpzk6 Container started 44m Normal Pulled pod/e2e-shared-a-w0-de8145-openshift-default-57f9f8965-mpzk6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/e2e-shared-a-w0-de8145-openshift-default-57f9f8965-mpzk6 Container created 44m Warning Unhealthy pod/e2e-shared-a-w4-a9b439-openshift-default-77b865d7d7-cfn9m Readiness probe failed: Get "http://10.134.0.168:15021/healthz/ready": dial tcp 10.134.0.168:15021: connect: connection refused 44m Warning ProbeError pod/e2e-shared-a-w4-a9b439-openshift-default-77b865d7d7-cfn9m Readiness probe error: Get "http://10.134.0.168:15021/healthz/ready": dial tcp 10.134.0.168:15021: connect: connection refused... 43m Normal Created pod/e2e-shared-a-w4-a9b439-openshift-default-77b865d7d7-cfn9m Container created 43m Normal Pulled pod/e2e-shared-a-w4-a9b439-openshift-default-77b865d7d7-cfn9m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-shared-a-w4-a9b439-openshift-default-77b865d7d7-cfn9m Container started 43m Warning ProbeError pod/e2e-shared-a-w0-de8145-openshift-default-57f9f8965-mpzk6 Startup probe error: Get "http://10.134.0.167:15021/healthz/ready": dial tcp 10.134.0.167:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-shared-a-w0-de8145-openshift-default-57f9f8965-mpzk6 Startup probe failed: Get "http://10.134.0.167:15021/healthz/ready": dial tcp 10.134.0.167:15021: connect: connection refused 43m Warning Unhealthy pod/e2e-shared-a-w4-a9b439-openshift-default-77b865d7d7-cfn9m Startup probe failed: Get "http://10.134.0.168:15021/healthz/ready": dial tcp 10.134.0.168:15021: connect: connection refused 43m Warning ProbeError pod/e2e-shared-a-w4-a9b439-openshift-default-77b865d7d7-cfn9m Startup probe error: Get "http://10.134.0.168:15021/healthz/ready": dial tcp 10.134.0.168:15021: connect: connection refused... 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-de8145-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-de8145-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-de8145-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-de8145-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal ScalingReplicaSet deployment/e2e-worker-w6-dd038b-openshift-default Scaled down replica set e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 from 7 to 6 43m Normal SuccessfulDelete replicaset/e2e-worker-w3-944263-openshift-default-5f78997bc9 Deleted pod: e2e-worker-w3-944263-openshift-default-5f78997bc9-v5s7k 43m Normal SuccessfulDelete replicaset/e2e-worker-w3-944263-openshift-default-5f78997bc9 Deleted pod: e2e-worker-w3-944263-openshift-default-5f78997bc9-dhjgg 43m Normal Killing pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-dhjgg Stopping container istio-proxy 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-dd038b-openshift-default New size: 6; reason: All metrics below target 43m Normal Killing pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-v5s7k Stopping container istio-proxy 43m Normal SuccessfulDelete replicaset/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 Deleted pod: e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-kdwp6 43m Normal SuccessfulDelete replicaset/e2e-worker-w3-944263-openshift-default-5f78997bc9 Deleted pod: e2e-worker-w3-944263-openshift-default-5f78997bc9-fr9fj 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-944263-openshift-default New size: 4; reason: All metrics below target 43m Normal ScalingReplicaSet deployment/e2e-worker-w3-944263-openshift-default Scaled down replica set e2e-worker-w3-944263-openshift-default-5f78997bc9 from 7 to 4 43m Normal Killing pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-kdwp6 Stopping container istio-proxy 43m Normal Killing pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-fr9fj Stopping container istio-proxy 43m Normal ScalingReplicaSet deployment/e2e-worker-w0-208f2b-openshift-default Scaled down replica set e2e-worker-w0-208f2b-openshift-default-5cc586d6f from 7 to 5 43m Warning ProbeError pod/e2e-shared-b-w1-df700a-openshift-default-6cf7fb69dc-dg822 Readiness probe error: Get "http://10.134.0.175:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Normal Killing pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-b9n9v Stopping container istio-proxy 43m Normal Killing pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lbcsf Stopping container istio-proxy 43m Normal SuccessfulDelete replicaset/e2e-worker-w0-208f2b-openshift-default-5cc586d6f Deleted pod: e2e-worker-w0-208f2b-openshift-default-5cc586d6f-b9n9v 43m Normal ScalingReplicaSet deployment/e2e-worker-w5-c9ae21-openshift-default Scaled down replica set e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 from 7 to 5 43m Normal SuccessfulDelete replicaset/e2e-worker-w0-208f2b-openshift-default-5cc586d6f Deleted pod: e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lbcsf 43m Warning Unhealthy pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-mwvv2 Readiness probe failed: HTTP probe failed with statuscode: 503 43m Normal ScalingReplicaSet deployment/e2e-worker-w2-cd248f-openshift-default Scaled down replica set e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 from 6 to 4 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-208f2b-openshift-default New size: 5; reason: All metrics below target 43m Warning ProbeError pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-mwvv2 Readiness probe error: HTTP probe failed with statuscode: 503... 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-db0157-openshift-default New size: 6; reason: All metrics below target 43m Normal SuccessfulDelete replicaset/e2e-worker-w1-db0157-openshift-default-68856dc5c7 Deleted pod: e2e-worker-w1-db0157-openshift-default-68856dc5c7-mwvv2 43m Normal Killing pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-65cf4 Stopping container istio-proxy 43m Normal Killing pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-tsxlv Stopping container istio-proxy 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-c9ae21-openshift-default New size: 5; reason: All metrics below target 43m Normal SuccessfulDelete replicaset/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 Deleted pod: e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-hnrhd 43m Normal SuccessfulDelete replicaset/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 Deleted pod: e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-cznb8 43m Normal ScalingReplicaSet deployment/e2e-worker-w1-db0157-openshift-default Scaled down replica set e2e-worker-w1-db0157-openshift-default-68856dc5c7 from 7 to 6 43m Warning Unhealthy pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-fr9fj Readiness probe failed: HTTP probe failed with statuscode: 503 43m Normal SuccessfulDelete replicaset/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 Deleted pod: e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-65cf4 43m Normal SuccessfulDelete replicaset/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 Deleted pod: e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-tsxlv 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-cd248f-openshift-default New size: 4; reason: All metrics below target 43m Normal Killing pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-mwvv2 Stopping container istio-proxy 43m Warning Unhealthy pod/e2e-shared-b-w1-df700a-openshift-default-6cf7fb69dc-dg822 Readiness probe failed: Get "http://10.134.0.175:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Normal Killing pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-cznb8 Stopping container istio-proxy 43m Normal Killing pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-hnrhd Stopping container istio-proxy 43m Warning ProbeError pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-fr9fj Readiness probe error: HTTP probe failed with statuscode: 503... 43m Normal ScalingReplicaSet deployment/e2e-worker-w4-1a1e9e-openshift-default Scaled down replica set e2e-worker-w4-1a1e9e-openshift-default-56c9685797 from 7 to 6 43m Warning ProbeError pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-v5s7k Readiness probe error: HTTP probe failed with statuscode: 503... 43m Warning Unhealthy pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-v5s7k Readiness probe failed: HTTP probe failed with statuscode: 503 43m Normal SuccessfulDelete replicaset/e2e-worker-w4-1a1e9e-openshift-default-56c9685797 Deleted pod: e2e-worker-w4-1a1e9e-openshift-default-56c9685797-tb6nw 43m Normal Killing pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-tb6nw Stopping container istio-proxy 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-1a1e9e-openshift-default New size: 6; reason: All metrics below target 43m Normal AddedLabel gateway/e2e-shared-b-w0-29baa8 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-29baa8 43m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-29baa8-openshift-default-84786bd7d7 Created pod: e2e-shared-b-w0-29baa8-openshift-default-84786bd7d7-q544h 43m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-29baa8-openshift-default Scaled up replica set e2e-shared-b-w0-29baa8-openshift-default-84786bd7d7 from 0 to 1 43m Normal AddedInterface pod/e2e-shared-b-w0-29baa8-openshift-default-84786bd7d7-q544h Add eth0 [10.134.0.194/23] from ovn-kubernetes 43m Warning Unhealthy pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-cznb8 Readiness probe failed: HTTP probe failed with statuscode: 503 43m Warning ProbeError pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-cznb8 Readiness probe error: HTTP probe failed with statuscode: 503... 43m Warning Unhealthy pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-tb6nw Readiness probe failed: HTTP probe failed with statuscode: 503 43m Warning ProbeError pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-tb6nw Readiness probe error: HTTP probe failed with statuscode: 503... 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-a9b439-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-a9b439-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning Unhealthy pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-hnrhd Readiness probe failed: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused 43m Warning ProbeError pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-hnrhd Readiness probe error: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused... 43m Normal Pulled pod/e2e-api-a-b27e66-openshift-default-5cccc745f7-nkvtl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Created pod/e2e-shared-b-w1-df700a-openshift-default-6cf7fb69dc-dg822 Container created 43m Normal Started pod/e2e-shared-b-w1-df700a-openshift-default-6cf7fb69dc-dg822 Container started 43m Normal Started pod/e2e-api-a-b27e66-openshift-default-5cccc745f7-nkvtl Container started 43m Normal Pulled pod/e2e-shared-b-w1-df700a-openshift-default-6cf7fb69dc-dg822 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Created pod/e2e-api-a-b27e66-openshift-default-5cccc745f7-nkvtl Container created 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-b27e66-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-b27e66-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning Unhealthy pod/e2e-shared-b-w1-df700a-openshift-default-6cf7fb69dc-dg822 Startup probe failed: Get "http://10.134.0.175:15021/healthz/ready": dial tcp 10.134.0.175:15021: connect: connection refused 43m Warning ProbeError pod/e2e-shared-b-w1-df700a-openshift-default-6cf7fb69dc-dg822 Startup probe error: Get "http://10.134.0.175:15021/healthz/ready": dial tcp 10.134.0.175:15021: connect: connection refused... 43m Warning ProbeError pod/e2e-api-a-b27e66-openshift-default-5cccc745f7-nkvtl Startup probe error: Get "http://10.134.0.174:15021/healthz/ready": dial tcp 10.134.0.174:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-api-a-b27e66-openshift-default-5cccc745f7-nkvtl Startup probe failed: Get "http://10.134.0.174:15021/healthz/ready": dial tcp 10.134.0.174:15021: connect: connection refused 43m Normal Killing pod/payload-pre-processing-e2e-mt-11a069b1-6977f77fd7-kh7wl Stopping container payload-pre-processing 43m Normal Killing pod/payload-processing-e2e-mt-11a069b1-7c85476fd5-nqm4x Stopping container payload-processing 43m Normal AddedLabel gateway/e2e-shared-b-w4-dab736 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-dab736 43m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-dab736-openshift-default Scaled up replica set e2e-shared-b-w4-dab736-openshift-default-5d5d779c5f from 0 to 1 43m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-dab736-openshift-default-5d5d779c5f Created pod: e2e-shared-b-w4-dab736-openshift-default-5d5d779c5f-tcsxq 43m Normal AddedInterface pod/e2e-shared-b-w4-dab736-openshift-default-5d5d779c5f-tcsxq Add eth0 [10.134.0.195/23] from ovn-kubernetes 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w0-29baa8-6bbff57b65 Created pod: payload-processing-e2e-shared-b-w0-29baa8-6bbff57b65-6djp8 43m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w0-29baa8-6bbff57b65 Error creating: pods "payload-processing-e2e-shared-b-w0-29baa8-6bbff57b65-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-29baa8: serviceaccount "payload-processing-e2e-shared-b-w0-29baa8" not found 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w0-29baa8 Scaled up replica set payload-processing-e2e-shared-b-w0-29baa8-6bbff57b65 from 0 to 1 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w0-29baa8 Scaled up replica set payload-pre-processing-e2e-shared-b-w0-29baa8-6c565db695 from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w0-29baa8-6c565db695 Created pod: payload-pre-processing-e2e-shared-b-w0-29baa8-6c565db695-5m42b 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w0-29baa8-6c565db695 Error creating: pods "payload-pre-processing-e2e-shared-b-w0-29baa8-6c565db695-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-29baa8: serviceaccount "payload-processing-e2e-shared-b-w0-29baa8" not found 43m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w0-29baa8-6c565db695-5m42b Add eth0 [10.134.0.196/23] from ovn-kubernetes 43m Normal Started pod/payload-processing-e2e-shared-b-w0-29baa8-6bbff57b65-6djp8 Container started 43m Normal Pulled pod/payload-processing-e2e-shared-b-w0-29baa8-6bbff57b65-6djp8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-pre-processing-e2e-shared-b-w0-29baa8-6c565db695-5m42b Container started 43m Normal AddedInterface pod/payload-processing-e2e-shared-b-w0-29baa8-6bbff57b65-6djp8 Add eth0 [10.134.0.197/23] from ovn-kubernetes 43m Normal Created pod/payload-pre-processing-e2e-shared-b-w0-29baa8-6c565db695-5m42b Container created 43m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w0-29baa8-6c565db695-5m42b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-processing-e2e-shared-b-w0-29baa8-6bbff57b65-6djp8 Container created 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w4-dab736-7b55d47c59 Created pod: payload-pre-processing-e2e-shared-b-w4-dab736-7b55d47c59-lrvjr 42m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w4-dab736-5b68f8cd7c Error creating: pods "payload-processing-e2e-shared-b-w4-dab736-5b68f8cd7c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-dab736: serviceaccount "payload-processing-e2e-shared-b-w4-dab736" not found 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w4-dab736 Scaled up replica set payload-pre-processing-e2e-shared-b-w4-dab736-7b55d47c59 from 0 to 1 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-df700a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w4-dab736-7b55d47c59 Error creating: pods "payload-pre-processing-e2e-shared-b-w4-dab736-7b55d47c59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-dab736: serviceaccount "payload-processing-e2e-shared-b-w4-dab736" not found 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-df700a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w4-dab736 Scaled up replica set payload-processing-e2e-shared-b-w4-dab736-5b68f8cd7c from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w4-dab736-5b68f8cd7c Created pod: payload-processing-e2e-shared-b-w4-dab736-5b68f8cd7c-947ht 42m Normal AddedInterface pod/payload-processing-e2e-shared-b-w4-dab736-5b68f8cd7c-947ht Add eth0 [10.134.0.199/23] from ovn-kubernetes 42m Normal Started pod/payload-pre-processing-e2e-shared-b-w4-dab736-7b55d47c59-lrvjr Container started 42m Normal Started pod/payload-processing-e2e-shared-b-w4-dab736-5b68f8cd7c-947ht Container started 42m Normal Created pod/payload-pre-processing-e2e-shared-b-w4-dab736-7b55d47c59-lrvjr Container created 42m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w4-dab736-7b55d47c59-lrvjr Add eth0 [10.134.0.198/23] from ovn-kubernetes 42m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w4-dab736-7b55d47c59-lrvjr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Created pod/payload-processing-e2e-shared-b-w4-dab736-5b68f8cd7c-947ht Container created 42m Normal Pulled pod/payload-processing-e2e-shared-b-w4-dab736-5b68f8cd7c-947ht Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Warning Unhealthy pod/payload-processing-e2e-mt-11a069b1-7c85476fd5-nqm4x Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 42m Warning Unhealthy pod/payload-pre-processing-e2e-mt-11a069b1-6977f77fd7-kh7wl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-b27e66-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-b27e66-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Normal SuccessfulCreate replicaset/e2e-api-b-d08fd0-openshift-default-5fd6b47cd7 Created pod: e2e-api-b-d08fd0-openshift-default-5fd6b47cd7-28k4z 42m Normal AddedLabel gateway/e2e-api-b-d08fd0 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-d08fd0 42m Normal ScalingReplicaSet deployment/e2e-api-b-d08fd0-openshift-default Scaled up replica set e2e-api-b-d08fd0-openshift-default-5fd6b47cd7 from 0 to 1 42m Normal Started pod/e2e-shared-b-w3-bf3fa2-openshift-default-69d7c5c975-84bmk Container started 42m Normal Created pod/e2e-shared-b-w3-bf3fa2-openshift-default-69d7c5c975-84bmk Container created 42m Normal Pulled pod/e2e-shared-b-w3-bf3fa2-openshift-default-69d7c5c975-84bmk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/e2e-api-b-d08fd0-openshift-default-5fd6b47cd7-28k4z Add eth0 [10.134.0.201/23] from ovn-kubernetes 42m Normal AddedLabel gateway/e2e-api-a-dd840a Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-dd840a 42m Normal ScalingReplicaSet deployment/e2e-api-a-dd840a-openshift-default Scaled up replica set e2e-api-a-dd840a-openshift-default-54cd84677f from 0 to 1 42m Warning Unhealthy pod/e2e-shared-b-w3-bf3fa2-openshift-default-69d7c5c975-84bmk Startup probe failed: Get "http://10.134.0.180:15021/healthz/ready": dial tcp 10.134.0.180:15021: connect: connection refused 42m Warning ProbeError pod/e2e-shared-b-w3-bf3fa2-openshift-default-69d7c5c975-84bmk Startup probe error: Get "http://10.134.0.180:15021/healthz/ready": dial tcp 10.134.0.180:15021: connect: connection refused... 42m Normal SuccessfulCreate replicaset/e2e-api-a-dd840a-openshift-default-54cd84677f Created pod: e2e-api-a-dd840a-openshift-default-54cd84677f-dvjrq 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-bf3fa2-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-d08fd0 Scaled up replica set payload-pre-processing-e2e-api-b-d08fd0-84d7f997dc from 0 to 1 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-bf3fa2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-d08fd0 Scaled up replica set payload-processing-e2e-api-b-d08fd0-7c95697f8f from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-d08fd0-84d7f997dc Created pod: payload-pre-processing-e2e-api-b-d08fd0-84d7f997dc-6zrkr 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-d08fd0-84d7f997dc Error creating: pods "payload-pre-processing-e2e-api-b-d08fd0-84d7f997dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-d08fd0: serviceaccount "payload-processing-e2e-api-b-d08fd0" not found 42m Normal AddedInterface pod/e2e-api-a-dd840a-openshift-default-54cd84677f-dvjrq Add eth0 [10.132.0.110/23] from ovn-kubernetes 42m Warning FailedCreate replicaset/payload-processing-e2e-api-b-d08fd0-7c95697f8f Error creating: pods "payload-processing-e2e-api-b-d08fd0-7c95697f8f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-d08fd0: serviceaccount "payload-processing-e2e-api-b-d08fd0" not found 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-d08fd0-7c95697f8f Created pod: payload-processing-e2e-api-b-d08fd0-7c95697f8f-sxbm5 42m Normal Created pod/payload-processing-e2e-api-b-d08fd0-7c95697f8f-sxbm5 Container created 42m Normal AddedInterface pod/payload-processing-e2e-api-b-d08fd0-7c95697f8f-sxbm5 Add eth0 [10.132.0.111/23] from ovn-kubernetes 42m Normal Pulled pod/payload-processing-e2e-api-b-d08fd0-7c95697f8f-sxbm5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Started pod/payload-pre-processing-e2e-api-b-d08fd0-84d7f997dc-6zrkr Container started 42m Normal Started pod/payload-processing-e2e-api-b-d08fd0-7c95697f8f-sxbm5 Container started 42m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-d08fd0-84d7f997dc-6zrkr Add eth0 [10.134.0.202/23] from ovn-kubernetes 42m Normal Created pod/payload-pre-processing-e2e-api-b-d08fd0-84d7f997dc-6zrkr Container created 42m Normal Pulled pod/payload-pre-processing-e2e-api-b-d08fd0-84d7f997dc-6zrkr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-dd840a-5c9b4679f5 Error creating: pods "payload-pre-processing-e2e-api-a-dd840a-5c9b4679f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-dd840a: serviceaccount "payload-processing-e2e-api-a-dd840a" not found 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-dd840a Scaled up replica set payload-pre-processing-e2e-api-a-dd840a-5c9b4679f5 from 0 to 1 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-dd840a Scaled up replica set payload-processing-e2e-api-a-dd840a-68df564877 from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-dd840a-68df564877 Created pod: payload-processing-e2e-api-a-dd840a-68df564877-hxfz7 42m Warning FailedCreate replicaset/payload-processing-e2e-api-a-dd840a-68df564877 Error creating: pods "payload-processing-e2e-api-a-dd840a-68df564877-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-dd840a: serviceaccount "payload-processing-e2e-api-a-dd840a" not found 42m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-dd840a-5c9b4679f5-wspqf Add eth0 [10.134.0.204/23] from ovn-kubernetes 42m Normal Started pod/payload-processing-e2e-api-a-dd840a-68df564877-hxfz7 Container started 42m Normal Created pod/payload-processing-e2e-api-a-dd840a-68df564877-hxfz7 Container created 42m Normal Pulled pod/payload-pre-processing-e2e-api-a-dd840a-5c9b4679f5-wspqf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-processing-e2e-api-a-dd840a-68df564877-hxfz7 Add eth0 [10.134.0.203/23] from ovn-kubernetes 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-dd840a-5c9b4679f5 Created pod: payload-pre-processing-e2e-api-a-dd840a-5c9b4679f5-wspqf 42m Normal Pulled pod/payload-processing-e2e-api-a-dd840a-68df564877-hxfz7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Started pod/payload-pre-processing-e2e-api-a-dd840a-5c9b4679f5-wspqf Container started 42m Normal Created pod/payload-pre-processing-e2e-api-a-dd840a-5c9b4679f5-wspqf Container created 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-bf3fa2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-bf3fa2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal AddedLabel gateway/e2e-inf-a-081c8e Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-081c8e 41m Normal AddedInterface pod/e2e-inf-a-081c8e-openshift-default-8cd89ddf-xx79m Add eth0 [10.134.0.205/23] from ovn-kubernetes 41m Normal ScalingReplicaSet deployment/e2e-inf-a-081c8e-openshift-default Scaled up replica set e2e-inf-a-081c8e-openshift-default-8cd89ddf from 0 to 1 41m Normal SuccessfulCreate replicaset/e2e-inf-a-081c8e-openshift-default-8cd89ddf Created pod: e2e-inf-a-081c8e-openshift-default-8cd89ddf-xx79m 41m Normal AddedLabel gateway/e2e-api-a-22ed7b Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-22ed7b 41m Normal ScalingReplicaSet deployment/e2e-api-a-22ed7b-openshift-default Scaled up replica set e2e-api-a-22ed7b-openshift-default-7fd8fd8b6f from 0 to 1 41m Normal SuccessfulCreate replicaset/e2e-api-a-22ed7b-openshift-default-7fd8fd8b6f Created pod: e2e-api-a-22ed7b-openshift-default-7fd8fd8b6f-twm7j 41m Normal AddedInterface pod/e2e-api-a-22ed7b-openshift-default-7fd8fd8b6f-twm7j Add eth0 [10.133.0.122/23] from ovn-kubernetes 41m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-081c8e-86f4657b7 Error creating: pods "payload-processing-e2e-inf-a-081c8e-86f4657b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-081c8e: serviceaccount "payload-processing-e2e-inf-a-081c8e" not found 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-081c8e Scaled up replica set payload-processing-e2e-inf-a-081c8e-86f4657b7 from 0 to 1 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-081c8e Scaled up replica set payload-pre-processing-e2e-inf-a-081c8e-9fdd659b7 from 0 to 1 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-081c8e-9fdd659b7 Error creating: pods "payload-pre-processing-e2e-inf-a-081c8e-9fdd659b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-081c8e: serviceaccount "payload-processing-e2e-inf-a-081c8e" not found 41m Normal Pulled pod/payload-pre-processing-e2e-inf-a-081c8e-9fdd659b7-8w2gx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-081c8e-86f4657b7 Created pod: payload-processing-e2e-inf-a-081c8e-86f4657b7-wc4tm 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-081c8e-9fdd659b7 Created pod: payload-pre-processing-e2e-inf-a-081c8e-9fdd659b7-8w2gx 41m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-081c8e-9fdd659b7-8w2gx Add eth0 [10.132.0.112/23] from ovn-kubernetes 41m Normal AddedInterface pod/payload-processing-e2e-inf-a-081c8e-86f4657b7-wc4tm Add eth0 [10.132.0.113/23] from ovn-kubernetes 41m Normal Pulled pod/payload-processing-e2e-inf-a-081c8e-86f4657b7-wc4tm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-processing-e2e-inf-a-081c8e-86f4657b7-wc4tm Container created 41m Normal Started pod/payload-pre-processing-e2e-inf-a-081c8e-9fdd659b7-8w2gx Container started 41m Normal Started pod/payload-processing-e2e-inf-a-081c8e-86f4657b7-wc4tm Container started 41m Normal Created pod/payload-pre-processing-e2e-inf-a-081c8e-9fdd659b7-8w2gx Container created 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-22ed7b Scaled up replica set payload-processing-e2e-api-a-22ed7b-5768c96547 from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-22ed7b-57f689dcd7 Created pod: payload-pre-processing-e2e-api-a-22ed7b-57f689dcd7-8vspb 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-22ed7b-5768c96547 Created pod: payload-processing-e2e-api-a-22ed7b-5768c96547-5qpms 41m Warning FailedCreate replicaset/payload-processing-e2e-api-a-22ed7b-5768c96547 Error creating: pods "payload-processing-e2e-api-a-22ed7b-5768c96547-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-22ed7b: serviceaccount "payload-processing-e2e-api-a-22ed7b" not found 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-22ed7b-57f689dcd7 Error creating: pods "payload-pre-processing-e2e-api-a-22ed7b-57f689dcd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-22ed7b: serviceaccount "payload-processing-e2e-api-a-22ed7b" not found 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-22ed7b Scaled up replica set payload-pre-processing-e2e-api-a-22ed7b-57f689dcd7 from 0 to 1 41m Normal Created pod/payload-processing-e2e-api-a-22ed7b-5768c96547-5qpms Container created 41m Normal Pulled pod/payload-processing-e2e-api-a-22ed7b-5768c96547-5qpms Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/payload-processing-e2e-api-a-22ed7b-5768c96547-5qpms Add eth0 [10.134.0.207/23] from ovn-kubernetes 41m Normal Started pod/payload-processing-e2e-api-a-22ed7b-5768c96547-5qpms Container started 41m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-22ed7b-57f689dcd7-8vspb Add eth0 [10.134.0.206/23] from ovn-kubernetes 41m Normal Started pod/payload-pre-processing-e2e-api-a-22ed7b-57f689dcd7-8vspb Container started 41m Normal Pulled pod/payload-pre-processing-e2e-api-a-22ed7b-57f689dcd7-8vspb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-pre-processing-e2e-api-a-22ed7b-57f689dcd7-8vspb Container created 41m Warning Unhealthy pod/e2e-shared-b-w0-29baa8-openshift-default-84786bd7d7-q544h Readiness probe failed: Get "http://10.134.0.194:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Warning ProbeError pod/e2e-shared-b-w0-29baa8-openshift-default-84786bd7d7-q544h Readiness probe error: Get "http://10.134.0.194:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Normal Created pod/e2e-shared-b-w0-29baa8-openshift-default-84786bd7d7-q544h Container created 41m Normal Started pod/e2e-shared-b-w0-29baa8-openshift-default-84786bd7d7-q544h Container started 41m Normal Pulled pod/e2e-shared-b-w0-29baa8-openshift-default-84786bd7d7-q544h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Warning Unhealthy pod/e2e-shared-b-w0-29baa8-openshift-default-84786bd7d7-q544h Startup probe failed: Get "http://10.134.0.194:15021/healthz/ready": dial tcp 10.134.0.194:15021: connect: connection refused 41m Warning ProbeError pod/e2e-shared-b-w0-29baa8-openshift-default-84786bd7d7-q544h Startup probe error: Get "http://10.134.0.194:15021/healthz/ready": dial tcp 10.134.0.194:15021: connect: connection refused... 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-11a069b1-6977f77fd7 Error creating: pods "payload-pre-processing-e2e-mt-11a069b1-6977f77fd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-11a069b1: serviceaccount "payload-processing-e2e-mt-11a069b1" not found 41m Warning FailedCreate replicaset/payload-processing-e2e-mt-11a069b1-7c85476fd5 Error creating: pods "payload-processing-e2e-mt-11a069b1-7c85476fd5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-11a069b1: serviceaccount "payload-processing-e2e-mt-11a069b1" not found 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-11a069b1-7c85476fd5 Created pod: payload-processing-e2e-mt-11a069b1-7c85476fd5-mlmjd 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-11a069b1 Scaled up replica set payload-processing-e2e-mt-11a069b1-7c85476fd5 from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-11a069b1-6977f77fd7 Created pod: payload-pre-processing-e2e-mt-11a069b1-6977f77fd7-lvtg7 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-11a069b1 Scaled up replica set payload-pre-processing-e2e-mt-11a069b1-6977f77fd7 from 0 to 1 41m Normal Started pod/payload-processing-e2e-mt-11a069b1-7c85476fd5-mlmjd Container started 41m Normal AddedInterface pod/payload-pre-processing-e2e-mt-11a069b1-6977f77fd7-lvtg7 Add eth0 [10.132.0.115/23] from ovn-kubernetes 41m Normal Pulled pod/payload-pre-processing-e2e-mt-11a069b1-6977f77fd7-lvtg7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-pre-processing-e2e-mt-11a069b1-6977f77fd7-lvtg7 Container created 41m Normal Started pod/payload-pre-processing-e2e-mt-11a069b1-6977f77fd7-lvtg7 Container started 41m Normal Created pod/payload-processing-e2e-mt-11a069b1-7c85476fd5-mlmjd Container created 41m Normal Pulled pod/payload-processing-e2e-mt-11a069b1-7c85476fd5-mlmjd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/payload-processing-e2e-mt-11a069b1-7c85476fd5-mlmjd Add eth0 [10.132.0.116/23] from ovn-kubernetes 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-29baa8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-29baa8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal Killing pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-ttwc5 Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 Deleted pod: e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-ttwc5 40m Normal SuccessfulDelete replicaset/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 Deleted pod: e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-wh67n 40m Normal SuccessfulDelete replicaset/e2e-worker-w1-db0157-openshift-default-68856dc5c7 Deleted pod: e2e-worker-w1-db0157-openshift-default-68856dc5c7-2bb5k 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-db0157-openshift-default New size: 5; reason: All metrics below target 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-dd038b-openshift-default New size: 5; reason: All metrics below target 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-cd248f-openshift-default New size: 3; reason: All metrics below target 40m Normal ScalingReplicaSet deployment/e2e-worker-w2-cd248f-openshift-default Scaled down replica set e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 from 4 to 3 40m Normal Killing pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-wh67n Stopping container istio-proxy 40m Normal ScalingReplicaSet deployment/e2e-worker-w6-dd038b-openshift-default Scaled down replica set e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 from 6 to 5 40m Normal ScalingReplicaSet deployment/e2e-worker-w1-db0157-openshift-default Scaled down replica set e2e-worker-w1-db0157-openshift-default-68856dc5c7 from 6 to 5 40m Normal Killing pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-q5fbc Stopping container istio-proxy 40m Warning Unhealthy pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-2bb5k Readiness probe failed: HTTP probe failed with statuscode: 503 40m Normal SuccessfulDelete replicaset/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 Deleted pod: e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-q5fbc 40m Warning ProbeError pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-2bb5k Readiness probe error: HTTP probe failed with statuscode: 503... 40m Normal Killing pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-2bb5k Stopping container istio-proxy 40m Normal ScalingReplicaSet deployment/e2e-worker-w5-c9ae21-openshift-default Scaled down replica set e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 from 5 to 4 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-c9ae21-openshift-default New size: 4; reason: All metrics below target 40m Warning ProbeError pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-wh67n Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-wh67n Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-29baa8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-29baa8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-gq6xw 40m Warning Unhealthy pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-q5fbc Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning ProbeError pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-q5fbc Readiness probe error: HTTP probe failed with statuscode: 503... 40m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 4 to 3 40m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: All metrics below target 40m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-gq6xw Stopping container discovery 40m Normal Started pod/e2e-api-b-d08fd0-openshift-default-5fd6b47cd7-28k4z Container started 40m Normal Created pod/e2e-api-b-d08fd0-openshift-default-5fd6b47cd7-28k4z Container created 40m Warning Unhealthy pod/e2e-api-b-d08fd0-openshift-default-5fd6b47cd7-28k4z Readiness probe failed: Get "http://10.134.0.201:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Warning ProbeError pod/e2e-api-b-d08fd0-openshift-default-5fd6b47cd7-28k4z Readiness probe error: Get "http://10.134.0.201:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Normal Pulled pod/e2e-api-b-d08fd0-openshift-default-5fd6b47cd7-28k4z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Warning ProbeError pod/e2e-api-b-d08fd0-openshift-default-5fd6b47cd7-28k4z Startup probe error: Get "http://10.134.0.201:15021/healthz/ready": dial tcp 10.134.0.201:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-api-b-d08fd0-openshift-default-5fd6b47cd7-28k4z Startup probe failed: Get "http://10.134.0.201:15021/healthz/ready": dial tcp 10.134.0.201:15021: connect: connection refused 40m Normal SuccessfulDelete replicaset/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 Deleted pod: e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-sbntr 40m Normal SuccessfulDelete replicaset/e2e-worker-w1-db0157-openshift-default-68856dc5c7 Deleted pod: e2e-worker-w1-db0157-openshift-default-68856dc5c7-xqm2p 40m Normal ScalingReplicaSet deployment/e2e-worker-w1-db0157-openshift-default Scaled down replica set e2e-worker-w1-db0157-openshift-default-68856dc5c7 from 5 to 4 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-db0157-openshift-default New size: 4; reason: All metrics below target 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-dd038b-openshift-default New size: 4; reason: All metrics below target 40m Normal ScalingReplicaSet deployment/e2e-worker-w6-dd038b-openshift-default Scaled down replica set e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 from 5 to 4 40m Normal Killing pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-sbntr Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-xqm2p Stopping container istio-proxy 40m Normal ScalingReplicaSet deployment/e2e-worker-w4-1a1e9e-openshift-default Scaled down replica set e2e-worker-w4-1a1e9e-openshift-default-56c9685797 from 6 to 4 40m Normal SuccessfulDelete replicaset/e2e-worker-w4-1a1e9e-openshift-default-56c9685797 Deleted pod: e2e-worker-w4-1a1e9e-openshift-default-56c9685797-8vhd5 40m Normal SuccessfulDelete replicaset/e2e-worker-w4-1a1e9e-openshift-default-56c9685797 Deleted pod: e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vcdwm 40m Normal Killing pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vcdwm Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-8vhd5 Stopping container istio-proxy 40m Warning Unhealthy pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-xqm2p Readiness probe failed: HTTP probe failed with statuscode: 503 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-1a1e9e-openshift-default New size: 4; reason: All metrics below target 40m Warning ProbeError pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-xqm2p Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning ProbeError pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-sbntr Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-sbntr Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-d08fd0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-d08fd0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning ProbeError pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vcdwm Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-8vhd5 Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning Unhealthy pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-vcdwm Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning ProbeError pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-8vhd5 Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-d08fd0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-d08fd0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Normal Killing pod/payload-processing-e2e-shared-a-w0-de8145-9dcc65c7f-q6cff Stopping container payload-processing 39m Normal Killing pod/payload-pre-processing-e2e-shared-a-w0-de8145-56fb74df5-8nbgw Stopping container payload-pre-processing 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-944263-openshift-default New size: 3; reason: All metrics below target 39m Normal Killing pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-gkcth Stopping container istio-proxy 39m Normal Killing pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-z5s97 Stopping container istio-proxy 39m Normal SuccessfulDelete replicaset/e2e-worker-w3-944263-openshift-default-5f78997bc9 Deleted pod: e2e-worker-w3-944263-openshift-default-5f78997bc9-gkcth 39m Normal SuccessfulDelete replicaset/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 Deleted pod: e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-49zgd 39m Normal SuccessfulDelete replicaset/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 Deleted pod: e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-hdc42 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-cd248f-openshift-default New size: 2; reason: All metrics below target 39m Normal SuccessfulDelete replicaset/e2e-worker-w1-db0157-openshift-default-68856dc5c7 Deleted pod: e2e-worker-w1-db0157-openshift-default-68856dc5c7-z5s97 39m Normal ScalingReplicaSet deployment/e2e-worker-w2-cd248f-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 from 3 to 2 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-db0157-openshift-default New size: 3; reason: All metrics below target 39m Normal Killing pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-49zgd Stopping container istio-proxy 39m Normal ScalingReplicaSet deployment/e2e-worker-w3-944263-openshift-default Scaled down replica set e2e-worker-w3-944263-openshift-default-5f78997bc9 from 4 to 3 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-dd038b-openshift-default New size: 3; reason: All metrics below target 39m Warning ProbeError pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-gkcth Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-gkcth Readiness probe failed: HTTP probe failed with statuscode: 503 39m Normal Killing pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-hdc42 Stopping container istio-proxy 39m Normal ScalingReplicaSet deployment/e2e-worker-w1-db0157-openshift-default Scaled down replica set e2e-worker-w1-db0157-openshift-default-68856dc5c7 from 4 to 3 39m Normal ScalingReplicaSet deployment/e2e-worker-w0-208f2b-openshift-default Scaled down replica set e2e-worker-w0-208f2b-openshift-default-5cc586d6f from 5 to 3 39m Warning ProbeError pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-brks2 Readiness probe error: HTTP probe failed with statuscode: 503... 39m Normal SuccessfulDelete replicaset/e2e-worker-w0-208f2b-openshift-default-5cc586d6f Deleted pod: e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lrl8n 39m Warning Unhealthy pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-z5s97 Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning ProbeError pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-z5s97 Readiness probe error: HTTP probe failed with statuscode: 503... 39m Normal Killing pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lrl8n Stopping container istio-proxy 39m Normal SuccessfulDelete replicaset/e2e-worker-w0-208f2b-openshift-default-5cc586d6f Deleted pod: e2e-worker-w0-208f2b-openshift-default-5cc586d6f-brks2 39m Normal SuccessfulDelete replicaset/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 Deleted pod: e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-gf6ms 39m Warning Unhealthy pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-brks2 Readiness probe failed: HTTP probe failed with statuscode: 503 39m Normal SuccessfulDelete replicaset/e2e-worker-w4-1a1e9e-openshift-default-56c9685797 Deleted pod: e2e-worker-w4-1a1e9e-openshift-default-56c9685797-m75bn 39m Normal Killing pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-brks2 Stopping container istio-proxy 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-208f2b-openshift-default New size: 3; reason: All metrics below target 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-c9ae21-openshift-default New size: 3; reason: All metrics below target 39m Normal ScalingReplicaSet deployment/e2e-worker-w5-c9ae21-openshift-default Scaled down replica set e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 from 4 to 3 39m Normal Killing pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-gf6ms Stopping container istio-proxy 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-1a1e9e-openshift-default New size: 3; reason: All metrics below target 39m Normal Killing pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-m75bn Stopping container istio-proxy 39m Warning ProbeError pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lrl8n Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lrl8n Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning Unhealthy pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-m75bn Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning ProbeError pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-m75bn Readiness probe error: HTTP probe failed with statuscode: 503... 39m Normal Pulled pod/e2e-api-a-dd840a-openshift-default-54cd84677f-dvjrq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-api-a-dd840a-openshift-default-54cd84677f-dvjrq Container created 39m Normal Started pod/e2e-api-a-dd840a-openshift-default-54cd84677f-dvjrq Container started 39m Warning Unhealthy pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-hdc42 Readiness probe failed: HTTP probe failed with statuscode: 503 39m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 39m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 39m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-qg2c2 Stopping container discovery 39m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-qg2c2 39m Warning ProbeError pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-hdc42 Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-api-a-dd840a-openshift-default-54cd84677f-dvjrq Startup probe failed: Get "http://10.132.0.110:15021/healthz/ready": dial tcp 10.132.0.110:15021: connect: connection refused 39m Warning ProbeError pod/e2e-api-a-dd840a-openshift-default-54cd84677f-dvjrq Startup probe error: Get "http://10.132.0.110:15021/healthz/ready": dial tcp 10.132.0.110:15021: connect: connection refused... 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-dd840a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-dd840a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Normal Killing pod/payload-processing-e2e-shared-a-w4-a9b439-76bf5ff49-qqkwc Stopping container payload-processing 39m Normal Killing pod/payload-pre-processing-e2e-shared-a-w4-a9b439-76df769c99-g6df9 Stopping container payload-pre-processing 39m Normal Started pod/e2e-shared-b-w4-dab736-openshift-default-5d5d779c5f-tcsxq Container started 39m Normal Pulled pod/e2e-shared-b-w4-dab736-openshift-default-5d5d779c5f-tcsxq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-shared-b-w4-dab736-openshift-default-5d5d779c5f-tcsxq Container created 39m Warning Unhealthy pod/e2e-shared-b-w4-dab736-openshift-default-5d5d779c5f-tcsxq Startup probe failed: Get "http://10.134.0.195:15021/healthz/ready": dial tcp 10.134.0.195:15021: connect: connection refused 39m Warning ProbeError pod/e2e-shared-b-w4-dab736-openshift-default-5d5d779c5f-tcsxq Startup probe error: Get "http://10.134.0.195:15021/healthz/ready": dial tcp 10.134.0.195:15021: connect: connection refused... 39m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-58c6585447 Deleted pod: maas-default-gateway-openshift-default-58c6585447-qzw6j 39m Normal Created pod/e2e-api-a-22ed7b-openshift-default-7fd8fd8b6f-twm7j Container created 39m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-58c6585447 Deleted pod: maas-default-gateway-openshift-default-58c6585447-cxh5l 39m Normal SuccessfulDelete replicaset/e2e-worker-w3-944263-openshift-default-5f78997bc9 Deleted pod: e2e-worker-w3-944263-openshift-default-5f78997bc9-l7fxr 39m Normal Killing pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-l7fxr Stopping container istio-proxy 39m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-58c6585447 Deleted pod: maas-default-gateway-openshift-default-58c6585447-pfnvk 39m Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-qzw6j Stopping container istio-proxy 39m Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-cxh5l Stopping container istio-proxy 39m Normal ScalingReplicaSet deployment/e2e-worker-w3-944263-openshift-default Scaled down replica set e2e-worker-w3-944263-openshift-default-5f78997bc9 from 3 to 2 39m Warning ProbeError pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-l7fxr Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-l7fxr Readiness probe failed: HTTP probe failed with statuscode: 503 39m Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-pfnvk Stopping container istio-proxy 39m Normal Pulled pod/e2e-api-a-22ed7b-openshift-default-7fd8fd8b6f-twm7j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-82z67 Stopping container istio-proxy 39m Normal Started pod/e2e-api-a-22ed7b-openshift-default-7fd8fd8b6f-twm7j Container started 39m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-58c6585447 Deleted pod: maas-default-gateway-openshift-default-58c6585447-82z67 39m Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-rfb2k Stopping container istio-proxy 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-944263-openshift-default New size: 2; reason: All metrics below target 39m Normal Started pod/e2e-inf-a-081c8e-openshift-default-8cd89ddf-xx79m Container started 39m Normal Created pod/e2e-inf-a-081c8e-openshift-default-8cd89ddf-xx79m Container created 39m Normal Pulled pod/e2e-inf-a-081c8e-openshift-default-8cd89ddf-xx79m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-081c8e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-081c8e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-82z67 Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-pfnvk Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-pfnvk Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-82z67 Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-qzw6j Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-qzw6j Readiness probe error: HTTP probe failed with statuscode: 503... 39m Warning Unhealthy pod/e2e-api-a-22ed7b-openshift-default-7fd8fd8b6f-twm7j Startup probe failed: Get "http://10.133.0.122:15021/healthz/ready": dial tcp 10.133.0.122:15021: connect: connection refused 39m Warning ProbeError pod/e2e-api-a-22ed7b-openshift-default-7fd8fd8b6f-twm7j Startup probe error: Get "http://10.133.0.122:15021/healthz/ready": dial tcp 10.133.0.122:15021: connect: connection refused... 39m Normal Killing pod/payload-pre-processing-e2e-api-a-b27e66-5db66b9467-z8rv7 Stopping container payload-pre-processing 39m Warning ProbeError pod/e2e-inf-a-081c8e-openshift-default-8cd89ddf-xx79m Startup probe error: Get "http://10.134.0.205:15021/healthz/ready": dial tcp 10.134.0.205:15021: connect: connection refused... 39m Normal Killing pod/payload-processing-e2e-api-a-b27e66-6df49bd599-n7wtm Stopping container payload-processing 39m Warning Unhealthy pod/e2e-inf-a-081c8e-openshift-default-8cd89ddf-xx79m Startup probe failed: Get "http://10.134.0.205:15021/healthz/ready": dial tcp 10.134.0.205:15021: connect: connection refused 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-dd840a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-dd840a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal Killing pod/payload-pre-processing-e2e-api-a-dd840a-5c9b4679f5-wspqf Stopping container payload-pre-processing 39m Normal Killing pod/payload-processing-e2e-api-a-dd840a-68df564877-hxfz7 Stopping container payload-processing 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-dab736-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-dab736-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-22ed7b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-22ed7b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-081c8e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-081c8e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Normal ScalingReplicaSet deployment/e2e-mt-11a069b1-openshift-default Scaled up replica set e2e-mt-11a069b1-openshift-default-7d5dd8f549 from 1 to 4 38m Normal SuccessfulCreate replicaset/e2e-mt-11a069b1-openshift-default-7d5dd8f549 Created pod: e2e-mt-11a069b1-openshift-default-7d5dd8f549-xcsv8 38m Normal SuccessfulCreate replicaset/e2e-mt-11a069b1-openshift-default-7d5dd8f549 Created pod: e2e-mt-11a069b1-openshift-default-7d5dd8f549-85qkw 38m Normal SuccessfulCreate replicaset/e2e-mt-11a069b1-openshift-default-7d5dd8f549 Created pod: e2e-mt-11a069b1-openshift-default-7d5dd8f549-zqwvb 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-11a069b1-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 38m Normal AddedInterface pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-85qkw Add eth0 [10.133.0.126/23] from ovn-kubernetes 38m Normal Created pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-xcsv8 Container created 38m Normal Pulled pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-xcsv8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-xcsv8 Add eth0 [10.134.0.212/23] from ovn-kubernetes 38m Normal Created pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-zqwvb Container created 38m Normal Started pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-xcsv8 Container started 38m Normal Pulled pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-85qkw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Started pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-zqwvb Container started 38m Normal AddedInterface pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-zqwvb Add eth0 [10.132.0.119/23] from ovn-kubernetes 38m Normal Created pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-85qkw Container created 38m Normal Started pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-85qkw Container started 38m Normal Pulled pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-zqwvb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-22ed7b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-22ed7b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Normal Killing pod/e2e-shared-a-w0-de8145-openshift-default-57f9f8965-mpzk6 Stopping container istio-proxy 38m Warning Unhealthy pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-85qkw Startup probe failed: Get "http://10.133.0.126:15021/healthz/ready": dial tcp 10.133.0.126:15021: connect: connection refused 38m Warning ProbeError pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-85qkw Startup probe error: Get "http://10.133.0.126:15021/healthz/ready": dial tcp 10.133.0.126:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-xcsv8 Startup probe failed: Get "http://10.134.0.212:15021/healthz/ready": dial tcp 10.134.0.212:15021: connect: connection refused 38m Warning ProbeError pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-xcsv8 Startup probe error: Get "http://10.134.0.212:15021/healthz/ready": dial tcp 10.134.0.212:15021: connect: connection refused... 38m Warning ProbeError pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-zqwvb Startup probe error: Get "http://10.132.0.119:15021/healthz/ready": dial tcp 10.132.0.119:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-zqwvb Startup probe failed: Get "http://10.132.0.119:15021/healthz/ready": dial tcp 10.132.0.119:15021: connect: connection refused 38m Warning Unhealthy pod/e2e-shared-a-w0-de8145-openshift-default-57f9f8965-mpzk6 Readiness probe failed: Get "http://10.134.0.167:15021/healthz/ready": dial tcp 10.134.0.167:15021: connect: connection refused 38m Warning ProbeError pod/e2e-shared-a-w0-de8145-openshift-default-57f9f8965-mpzk6 Readiness probe error: Get "http://10.134.0.167:15021/healthz/ready": dial tcp 10.134.0.167:15021: connect: connection refused... 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-11a069b1-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 38m Normal ScalingReplicaSet deployment/e2e-mt-11a069b1-openshift-default Scaled up replica set e2e-mt-11a069b1-openshift-default-7d5dd8f549 from 4 to 6 38m Normal SuccessfulCreate replicaset/e2e-mt-11a069b1-openshift-default-7d5dd8f549 Created pod: e2e-mt-11a069b1-openshift-default-7d5dd8f549-jswz2 38m Normal SuccessfulCreate replicaset/e2e-mt-11a069b1-openshift-default-7d5dd8f549 Created pod: e2e-mt-11a069b1-openshift-default-7d5dd8f549-gkl98 38m Normal Created pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-gkl98 Container created 38m Normal Created pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-jswz2 Container created 38m Normal Pulled pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-jswz2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-jswz2 Add eth0 [10.132.0.120/23] from ovn-kubernetes 38m Normal AddedInterface pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-gkl98 Add eth0 [10.134.0.216/23] from ovn-kubernetes 38m Normal Pulled pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-gkl98 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Started pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-jswz2 Container started 38m Normal Killing pod/e2e-shared-a-w4-a9b439-openshift-default-77b865d7d7-cfn9m Stopping container istio-proxy 38m Normal Started pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-gkl98 Container started 38m Normal Killing pod/payload-processing-e2e-inf-a-081c8e-86f4657b7-wc4tm Stopping container payload-processing 38m Normal Killing pod/payload-pre-processing-e2e-inf-a-081c8e-9fdd659b7-8w2gx Stopping container payload-pre-processing 38m Normal Killing pod/payload-processing-e2e-api-a-22ed7b-5768c96547-5qpms Stopping container payload-processing 38m Warning Unhealthy pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-jswz2 Startup probe failed: Get "http://10.132.0.120:15021/healthz/ready": dial tcp 10.132.0.120:15021: connect: connection refused 38m Warning ProbeError pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-jswz2 Startup probe error: Get "http://10.132.0.120:15021/healthz/ready": dial tcp 10.132.0.120:15021: connect: connection refused... 38m Normal Killing pod/payload-pre-processing-e2e-api-a-22ed7b-57f689dcd7-8vspb Stopping container payload-pre-processing 38m Warning Unhealthy pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-gkl98 Startup probe failed: Get "http://10.134.0.216:15021/healthz/ready": dial tcp 10.134.0.216:15021: connect: connection refused 38m Warning ProbeError pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-gkl98 Startup probe error: Get "http://10.134.0.216:15021/healthz/ready": dial tcp 10.134.0.216:15021: connect: connection refused... 38m Normal Killing pod/payload-processing-e2e-shared-b-w0-29baa8-6bbff57b65-6djp8 Stopping container payload-processing 38m Normal Killing pod/payload-pre-processing-e2e-shared-b-w0-29baa8-6c565db695-5m42b Stopping container payload-pre-processing 38m Normal Killing pod/payload-pre-processing-e2e-shared-b-w4-dab736-7b55d47c59-lrvjr Stopping container payload-pre-processing 38m Normal Killing pod/payload-processing-e2e-shared-b-w4-dab736-5b68f8cd7c-947ht Stopping container payload-processing 38m Normal Killing pod/e2e-inf-a-081c8e-openshift-default-8cd89ddf-xx79m Stopping container istio-proxy 38m Normal Killing pod/e2e-shared-b-w4-dab736-openshift-default-5d5d779c5f-tcsxq Stopping container istio-proxy 38m Normal SuccessfulCreate replicaset/e2e-api-a-e0a6f5-openshift-default-5b74475bdc Created pod: e2e-api-a-e0a6f5-openshift-default-5b74475bdc-rrrbc 38m Normal ScalingReplicaSet deployment/e2e-api-a-e0a6f5-openshift-default Scaled up replica set e2e-api-a-e0a6f5-openshift-default-5b74475bdc from 0 to 1 38m Normal AddedLabel gateway/e2e-api-a-e0a6f5 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-e0a6f5 38m Normal AddedInterface pod/e2e-api-a-e0a6f5-openshift-default-5b74475bdc-rrrbc Add eth0 [10.134.0.220/23] from ovn-kubernetes 37m Normal Killing pod/payload-processing-e2e-worker-w5-c9ae21-6667d54cf5-twgq6 Stopping container payload-processing 37m Normal Killing pod/payload-pre-processing-e2e-worker-w5-c9ae21-7bc765fb9f-7tjsn Stopping container payload-pre-processing 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-e0a6f5 Scaled up replica set payload-processing-e2e-api-a-e0a6f5-56bbd4f6f7 from 0 to 1 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-e0a6f5-68b97cb76c Error creating: pods "payload-pre-processing-e2e-api-a-e0a6f5-68b97cb76c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-e0a6f5: serviceaccount "payload-processing-e2e-api-a-e0a6f5" not found 37m Warning FailedCreate replicaset/payload-processing-e2e-api-a-e0a6f5-56bbd4f6f7 Error creating: pods "payload-processing-e2e-api-a-e0a6f5-56bbd4f6f7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-e0a6f5: serviceaccount "payload-processing-e2e-api-a-e0a6f5" not found 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-e0a6f5-56bbd4f6f7 Created pod: payload-processing-e2e-api-a-e0a6f5-56bbd4f6f7-5gtlm 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-e0a6f5-68b97cb76c Created pod: payload-pre-processing-e2e-api-a-e0a6f5-68b97cb76c-sm88c 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-e0a6f5 Scaled up replica set payload-pre-processing-e2e-api-a-e0a6f5-68b97cb76c from 0 to 1 37m Normal AddedInterface pod/payload-processing-e2e-api-a-e0a6f5-56bbd4f6f7-5gtlm Add eth0 [10.134.0.222/23] from ovn-kubernetes 37m Normal Started pod/payload-pre-processing-e2e-api-a-e0a6f5-68b97cb76c-sm88c Container started 37m Normal Killing pod/payload-processing-e2e-mt-11a069b1-7c85476fd5-mlmjd Stopping container payload-processing 37m Normal Pulled pod/payload-processing-e2e-api-a-e0a6f5-56bbd4f6f7-5gtlm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Created pod/payload-pre-processing-e2e-api-a-e0a6f5-68b97cb76c-sm88c Container created 37m Normal Pulled pod/payload-pre-processing-e2e-api-a-e0a6f5-68b97cb76c-sm88c Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-e0a6f5-68b97cb76c-sm88c Add eth0 [10.134.0.221/23] from ovn-kubernetes 37m Normal Started pod/payload-processing-e2e-api-a-e0a6f5-56bbd4f6f7-5gtlm Container started 37m Normal Killing pod/payload-pre-processing-e2e-mt-11a069b1-6977f77fd7-lvtg7 Stopping container payload-pre-processing 37m Normal Created pod/payload-processing-e2e-api-a-e0a6f5-56bbd4f6f7-5gtlm Container created 37m Warning Unhealthy pod/payload-processing-e2e-worker-w5-c9ae21-6667d54cf5-twgq6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w5-c9ae21-7bc765fb9f-7tjsn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Normal Killing pod/e2e-api-a-dd840a-openshift-default-54cd84677f-dvjrq Stopping container istio-proxy 37m Normal ScalingReplicaSet deployment/e2e-inf-a-4ac1ef-openshift-default Scaled up replica set e2e-inf-a-4ac1ef-openshift-default-54d7ccf455 from 0 to 1 37m Normal SuccessfulCreate replicaset/e2e-inf-a-4ac1ef-openshift-default-54d7ccf455 Created pod: e2e-inf-a-4ac1ef-openshift-default-54d7ccf455-j89g4 37m Normal AddedLabel gateway/e2e-inf-a-4ac1ef Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-4ac1ef 37m Normal AddedInterface pod/e2e-inf-a-4ac1ef-openshift-default-54d7ccf455-j89g4 Add eth0 [10.133.0.127/23] from ovn-kubernetes 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-4ac1ef-db4769dc Error creating: pods "payload-pre-processing-e2e-inf-a-4ac1ef-db4769dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-4ac1ef: serviceaccount "payload-processing-e2e-inf-a-4ac1ef" not found 37m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-4ac1ef-5bddb89485 Error creating: pods "payload-processing-e2e-inf-a-4ac1ef-5bddb89485-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-4ac1ef: serviceaccount "payload-processing-e2e-inf-a-4ac1ef" not found 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-4ac1ef Scaled up replica set payload-pre-processing-e2e-inf-a-4ac1ef-db4769dc from 0 to 1 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-4ac1ef Scaled up replica set payload-processing-e2e-inf-a-4ac1ef-5bddb89485 from 0 to 1 37m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-4ac1ef-db4769dc-ffkcl Add eth0 [10.134.0.224/23] from ovn-kubernetes 37m Normal AddedInterface pod/payload-processing-e2e-inf-a-4ac1ef-5bddb89485-dpbt4 Add eth0 [10.134.0.223/23] from ovn-kubernetes 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-4ac1ef-5bddb89485 Created pod: payload-processing-e2e-inf-a-4ac1ef-5bddb89485-dpbt4 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-4ac1ef-db4769dc Created pod: payload-pre-processing-e2e-inf-a-4ac1ef-db4769dc-ffkcl 37m Normal Pulled pod/payload-processing-e2e-inf-a-4ac1ef-5bddb89485-dpbt4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Pulled pod/payload-pre-processing-e2e-inf-a-4ac1ef-db4769dc-ffkcl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Created pod/payload-pre-processing-e2e-inf-a-4ac1ef-db4769dc-ffkcl Container created 37m Normal Started pod/payload-pre-processing-e2e-inf-a-4ac1ef-db4769dc-ffkcl Container started 37m Normal Created pod/payload-processing-e2e-inf-a-4ac1ef-5bddb89485-dpbt4 Container created 37m Normal Started pod/payload-processing-e2e-inf-a-4ac1ef-5bddb89485-dpbt4 Container started 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-208f2b-openshift-default New size: 2; reason: All metrics below target 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-c9ae21-openshift-default New size: 2; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/e2e-worker-w5-c9ae21-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 from 3 to 2 37m Normal ScalingReplicaSet deployment/e2e-worker-w0-208f2b-openshift-default Scaled down replica set e2e-worker-w0-208f2b-openshift-default-5cc586d6f from 3 to 2 37m Normal SuccessfulDelete replicaset/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65 Deleted pod: e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-8xtk5 37m Normal SuccessfulDelete replicaset/e2e-worker-w0-208f2b-openshift-default-5cc586d6f Deleted pod: e2e-worker-w0-208f2b-openshift-default-5cc586d6f-9nlfq 37m Normal Killing pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-8xtk5 Stopping container istio-proxy 37m Normal Killing pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-9nlfq Stopping container istio-proxy 37m Normal Killing pod/e2e-shared-b-w0-29baa8-openshift-default-84786bd7d7-q544h Stopping container istio-proxy 37m Normal ScalingReplicaSet deployment/e2e-api-a-deff5e-openshift-default Scaled up replica set e2e-api-a-deff5e-openshift-default-589bfb887f from 0 to 1 37m Normal AddedLabel gateway/e2e-api-a-deff5e Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-deff5e 37m Warning Unhealthy pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-9nlfq Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-9nlfq Readiness probe error: HTTP probe failed with statuscode: 503... 37m Normal SuccessfulCreate replicaset/e2e-api-a-deff5e-openshift-default-589bfb887f Created pod: e2e-api-a-deff5e-openshift-default-589bfb887f-hnvnf 37m Normal AddedInterface pod/e2e-api-a-deff5e-openshift-default-589bfb887f-hnvnf Add eth0 [10.133.0.128/23] from ovn-kubernetes 37m Warning Unhealthy pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-8xtk5 Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-8xtk5 Readiness probe error: HTTP probe failed with statuscode: 503... 36m Normal Killing pod/e2e-api-a-22ed7b-openshift-default-7fd8fd8b6f-twm7j Stopping container istio-proxy 36m Normal AddedLabel gateway/e2e-inf-a-6b9305 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-6b9305 36m Normal SuccessfulCreate replicaset/e2e-inf-a-6b9305-openshift-default-74c965c97c Created pod: e2e-inf-a-6b9305-openshift-default-74c965c97c-7xs26 36m Normal ScalingReplicaSet deployment/e2e-inf-a-6b9305-openshift-default Scaled up replica set e2e-inf-a-6b9305-openshift-default-74c965c97c from 0 to 1 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-deff5e Scaled up replica set payload-pre-processing-e2e-api-a-deff5e-77db945ff9 from 0 to 1 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-deff5e Scaled up replica set payload-processing-e2e-api-a-deff5e-bd664cbf7 from 0 to 1 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-deff5e-77db945ff9 Error creating: pods "payload-pre-processing-e2e-api-a-deff5e-77db945ff9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-deff5e: serviceaccount "payload-processing-e2e-api-a-deff5e" not found 36m Normal AddedInterface pod/e2e-inf-a-6b9305-openshift-default-74c965c97c-7xs26 Add eth0 [10.134.0.225/23] from ovn-kubernetes 36m Warning FailedCreate replicaset/payload-processing-e2e-api-a-deff5e-bd664cbf7 Error creating: pods "payload-processing-e2e-api-a-deff5e-bd664cbf7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-deff5e: serviceaccount "payload-processing-e2e-api-a-deff5e" not found 36m Normal Pulled pod/payload-pre-processing-e2e-api-a-deff5e-77db945ff9-jtzp7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-deff5e-77db945ff9 Created pod: payload-pre-processing-e2e-api-a-deff5e-77db945ff9-jtzp7 36m Normal Started pod/payload-pre-processing-e2e-api-a-deff5e-77db945ff9-jtzp7 Container started 36m Normal Created pod/payload-pre-processing-e2e-api-a-deff5e-77db945ff9-jtzp7 Container created 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-deff5e-bd664cbf7 Created pod: payload-processing-e2e-api-a-deff5e-bd664cbf7-wvr5w 36m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-deff5e-77db945ff9-jtzp7 Add eth0 [10.134.0.226/23] from ovn-kubernetes 36m Normal Created pod/payload-processing-e2e-api-a-deff5e-bd664cbf7-wvr5w Container created 36m Normal Pulled pod/payload-processing-e2e-api-a-deff5e-bd664cbf7-wvr5w Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Started pod/payload-processing-e2e-api-a-deff5e-bd664cbf7-wvr5w Container started 36m Normal AddedInterface pod/payload-processing-e2e-api-a-deff5e-bd664cbf7-wvr5w Add eth0 [10.134.0.227/23] from ovn-kubernetes 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-6b9305-864cc97d9f Error creating: pods "payload-pre-processing-e2e-inf-a-6b9305-864cc97d9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-6b9305: serviceaccount "payload-processing-e2e-inf-a-6b9305" not found 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-6b9305-6964565579 Created pod: payload-processing-e2e-inf-a-6b9305-6964565579-hwlhv 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-6b9305 Scaled up replica set payload-pre-processing-e2e-inf-a-6b9305-864cc97d9f from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-6b9305-864cc97d9f Created pod: payload-pre-processing-e2e-inf-a-6b9305-864cc97d9f-dcsdn 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-6b9305 Scaled up replica set payload-processing-e2e-inf-a-6b9305-6964565579 from 0 to 1 36m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-6b9305-6964565579 Error creating: pods "payload-processing-e2e-inf-a-6b9305-6964565579-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-6b9305: serviceaccount "payload-processing-e2e-inf-a-6b9305" not found 36m Normal Created pod/payload-processing-e2e-inf-a-6b9305-6964565579-hwlhv Container created 36m Normal Pulled pod/payload-processing-e2e-inf-a-6b9305-6964565579-hwlhv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-6b9305-864cc97d9f-dcsdn Add eth0 [10.134.0.230/23] from ovn-kubernetes 36m Normal Pulled pod/payload-pre-processing-e2e-inf-a-6b9305-864cc97d9f-dcsdn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Created pod/payload-pre-processing-e2e-inf-a-6b9305-864cc97d9f-dcsdn Container created 36m Normal Started pod/payload-pre-processing-e2e-inf-a-6b9305-864cc97d9f-dcsdn Container started 36m Normal Started pod/payload-processing-e2e-inf-a-6b9305-6964565579-hwlhv Container started 36m Normal AddedInterface pod/payload-processing-e2e-inf-a-6b9305-6964565579-hwlhv Add eth0 [10.134.0.231/23] from ovn-kubernetes 36m Normal Killing pod/e2e-api-a-b27e66-openshift-default-5cccc745f7-nkvtl Stopping container istio-proxy 36m Normal Killing pod/payload-processing-e2e-api-b-d08fd0-7c95697f8f-sxbm5 Stopping container payload-processing 36m Normal Killing pod/payload-pre-processing-e2e-api-b-d08fd0-84d7f997dc-6zrkr Stopping container payload-pre-processing 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-db0157-openshift-default New size: 2; reason: All metrics below target 36m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: All metrics below target 36m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-58c6585447 Deleted pod: maas-default-gateway-openshift-default-58c6585447-z4sdv 36m Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-z4sdv Stopping container istio-proxy 36m Normal Killing pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-jwfhk Stopping container istio-proxy 36m Warning ProbeError pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-jwfhk Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning Unhealthy pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-jwfhk Readiness probe failed: HTTP probe failed with statuscode: 503 36m Normal SuccessfulDelete replicaset/e2e-worker-w1-db0157-openshift-default-68856dc5c7 Deleted pod: e2e-worker-w1-db0157-openshift-default-68856dc5c7-psptj 36m Normal Killing pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-psptj Stopping container istio-proxy 36m Normal ScalingReplicaSet deployment/e2e-worker-w1-db0157-openshift-default Scaled down replica set e2e-worker-w1-db0157-openshift-default-68856dc5c7 from 3 to 2 36m Normal SuccessfulDelete replicaset/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 Deleted pod: e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-jwfhk 36m Normal Created pod/e2e-api-a-e0a6f5-openshift-default-5b74475bdc-rrrbc Container created 36m Normal Started pod/e2e-api-a-e0a6f5-openshift-default-5b74475bdc-rrrbc Container started 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-e0a6f5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-e0a6f5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Normal Pulled pod/e2e-api-a-e0a6f5-openshift-default-5b74475bdc-rrrbc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-z4sdv Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-z4sdv Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning ProbeError pod/e2e-api-a-e0a6f5-openshift-default-5b74475bdc-rrrbc Startup probe error: Get "http://10.134.0.220:15021/healthz/ready": dial tcp 10.134.0.220:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-api-a-e0a6f5-openshift-default-5b74475bdc-rrrbc Startup probe failed: Get "http://10.134.0.220:15021/healthz/ready": dial tcp 10.134.0.220:15021: connect: connection refused 35m Normal Killing pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-85qkw Stopping container istio-proxy 35m Normal Killing pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-xcsv8 Stopping container istio-proxy 35m Normal Killing pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-vx9f5 Stopping container istio-proxy 35m Normal Killing pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-zqwvb Stopping container istio-proxy 35m Normal Killing pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-jswz2 Stopping container istio-proxy 35m Normal Killing pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-gkl98 Stopping container istio-proxy 35m Warning Unhealthy pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-jswz2 Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-jswz2 Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-zqwvb Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-xcsv8 Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-85qkw Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-gkl98 Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-gkl98 Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-85qkw Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-xcsv8 Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-mt-11a069b1-openshift-default-7d5dd8f549-zqwvb Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-e0a6f5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-e0a6f5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Normal Killing pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-zjkp8 Stopping container istio-proxy 35m Normal SuccessfulDelete replicaset/e2e-worker-w4-1a1e9e-openshift-default-56c9685797 Deleted pod: e2e-worker-w4-1a1e9e-openshift-default-56c9685797-zjkp8 35m Normal AddedLabel gateway/e2e-api-b-a41242 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-a41242 35m Normal Killing pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-sz2sr Stopping container istio-proxy 35m Normal Killing pod/e2e-worker-w5-c9ae21-openshift-default-5b788bdc65-qjcxh Stopping container istio-proxy 35m Normal ScalingReplicaSet deployment/e2e-api-b-a41242-openshift-default Scaled up replica set e2e-api-b-a41242-openshift-default-86bb88f59 from 0 to 1 35m Normal SuccessfulCreate replicaset/e2e-api-b-a41242-openshift-default-86bb88f59 Created pod: e2e-api-b-a41242-openshift-default-86bb88f59-hn6s2 35m Normal AddedInterface pod/e2e-api-b-a41242-openshift-default-86bb88f59-hn6s2 Add eth0 [10.134.0.233/23] from ovn-kubernetes 35m Normal Created pod/e2e-inf-a-4ac1ef-openshift-default-54d7ccf455-j89g4 Container created 35m Normal Pulled pod/e2e-inf-a-4ac1ef-openshift-default-54d7ccf455-j89g4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Pulled pod/e2e-api-a-deff5e-openshift-default-589bfb887f-hnvnf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-api-a-deff5e-openshift-default-589bfb887f-hnvnf Container created 35m Normal Started pod/e2e-inf-a-4ac1ef-openshift-default-54d7ccf455-j89g4 Container started 35m Normal Started pod/e2e-api-a-deff5e-openshift-default-589bfb887f-hnvnf Container started 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-a41242-6bfc8bf88f Created pod: payload-processing-e2e-api-b-a41242-6bfc8bf88f-cs272 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-a41242-5956bf7979 Error creating: pods "payload-pre-processing-e2e-api-b-a41242-5956bf7979-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-a41242: serviceaccount "payload-processing-e2e-api-b-a41242" not found 35m Warning FailedCreate replicaset/payload-processing-e2e-api-b-a41242-6bfc8bf88f Error creating: pods "payload-processing-e2e-api-b-a41242-6bfc8bf88f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-a41242: serviceaccount "payload-processing-e2e-api-b-a41242" not found 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-a41242 Scaled up replica set payload-processing-e2e-api-b-a41242-6bfc8bf88f from 0 to 1 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-a41242 Scaled up replica set payload-pre-processing-e2e-api-b-a41242-5956bf7979 from 0 to 1 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-a41242-5956bf7979 Created pod: payload-pre-processing-e2e-api-b-a41242-5956bf7979-dsqpz 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-4ac1ef-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Normal Pulled pod/payload-pre-processing-e2e-api-b-a41242-5956bf7979-dsqpz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Started pod/payload-pre-processing-e2e-api-b-a41242-5956bf7979-dsqpz Container started 35m Normal AddedInterface pod/payload-processing-e2e-api-b-a41242-6bfc8bf88f-cs272 Add eth0 [10.134.0.235/23] from ovn-kubernetes 35m Normal Pulled pod/payload-processing-e2e-api-b-a41242-6bfc8bf88f-cs272 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Created pod/payload-processing-e2e-api-b-a41242-6bfc8bf88f-cs272 Container created 35m Normal Started pod/payload-processing-e2e-api-b-a41242-6bfc8bf88f-cs272 Container started 35m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-a41242-5956bf7979-dsqpz Add eth0 [10.134.0.234/23] from ovn-kubernetes 35m Normal Created pod/payload-pre-processing-e2e-api-b-a41242-5956bf7979-dsqpz Container created 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-4ac1ef-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-pg8rw Add eth0 [10.133.0.132/23] from ovn-kubernetes 35m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-dggk5 Add eth0 [10.134.0.236/23] from ovn-kubernetes 35m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-dggk5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-k7ndz Container started 35m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-pg8rw Container started 35m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-m6rnq Container created 35m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-m6rnq Add eth0 [10.133.0.133/23] from ovn-kubernetes 35m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-m6rnq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-m6rnq Container started 35m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-k7ndz Container created 35m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-k7ndz Add eth0 [10.133.0.134/23] from ovn-kubernetes 35m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-pg8rw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-dggk5 Container started 35m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-dggk5 Container created 35m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-pg8rw Container created 35m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-k7ndz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Warning Unhealthy pod/e2e-inf-a-4ac1ef-openshift-default-54d7ccf455-j89g4 Startup probe failed: Get "http://10.133.0.127:15021/healthz/ready": dial tcp 10.133.0.127:15021: connect: connection refused 35m Warning ProbeError pod/e2e-api-a-deff5e-openshift-default-589bfb887f-hnvnf Startup probe error: Get "http://10.133.0.128:15021/healthz/ready": dial tcp 10.133.0.128:15021: connect: connection refused... 35m Warning ProbeError pod/e2e-inf-a-4ac1ef-openshift-default-54d7ccf455-j89g4 Startup probe error: Get "http://10.133.0.127:15021/healthz/ready": dial tcp 10.133.0.127:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-api-a-deff5e-openshift-default-589bfb887f-hnvnf Startup probe failed: Get "http://10.133.0.128:15021/healthz/ready": dial tcp 10.133.0.128:15021: connect: connection refused 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-deff5e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-deff5e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Normal Killing pod/e2e-api-b-d08fd0-openshift-default-5fd6b47cd7-28k4z Stopping container istio-proxy 34m Normal ScalingReplicaSet deployment/e2e-inf-a-3ab520-openshift-default Scaled up replica set e2e-inf-a-3ab520-openshift-default-6765b98db9 from 0 to 1 34m Normal SuccessfulCreate replicaset/e2e-inf-a-3ab520-openshift-default-6765b98db9 Created pod: e2e-inf-a-3ab520-openshift-default-6765b98db9-jrzgk 34m Normal AddedLabel gateway/e2e-inf-a-3ab520 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-3ab520 34m Normal AddedInterface pod/e2e-inf-a-3ab520-openshift-default-6765b98db9-jrzgk Add eth0 [10.134.0.237/23] from ovn-kubernetes 34m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-dggk5 Startup probe error: Get "http://10.134.0.236:15021/healthz/ready": dial tcp 10.134.0.236:15021: connect: connection refused... 34m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-dggk5 Startup probe failed: Get "http://10.134.0.236:15021/healthz/ready": dial tcp 10.134.0.236:15021: connect: connection refused 34m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-pg8rw Startup probe failed: Get "http://10.133.0.132:15021/healthz/ready": dial tcp 10.133.0.132:15021: connect: connection refused 34m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-pg8rw Startup probe error: Get "http://10.133.0.132:15021/healthz/ready": dial tcp 10.133.0.132:15021: connect: connection refused... 34m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-m6rnq Startup probe error: Get "http://10.133.0.133:15021/healthz/ready": dial tcp 10.133.0.133:15021: connect: connection refused... 34m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-m6rnq Startup probe failed: Get "http://10.133.0.133:15021/healthz/ready": dial tcp 10.133.0.133:15021: connect: connection refused 34m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-k7ndz Startup probe error: Get "http://10.133.0.134:15021/healthz/ready": dial tcp 10.133.0.134:15021: connect: connection refused... 34m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-k7ndz Startup probe failed: Get "http://10.133.0.134:15021/healthz/ready": dial tcp 10.133.0.134:15021: connect: connection refused 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-3ab520 Scaled up replica set payload-processing-e2e-inf-a-3ab520-7b7f4ff56c from 0 to 1 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-3ab520-848dc5bd6c Error creating: pods "payload-pre-processing-e2e-inf-a-3ab520-848dc5bd6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-3ab520: serviceaccount "payload-processing-e2e-inf-a-3ab520" not found 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-3ab520-848dc5bd6c Created pod: payload-pre-processing-e2e-inf-a-3ab520-848dc5bd6c-k9zhf 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-3ab520-7b7f4ff56c Created pod: payload-processing-e2e-inf-a-3ab520-7b7f4ff56c-vszhw 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-3ab520 Scaled up replica set payload-pre-processing-e2e-inf-a-3ab520-848dc5bd6c from 0 to 1 34m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-3ab520-848dc5bd6c-k9zhf Add eth0 [10.134.0.238/23] from ovn-kubernetes 34m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-3ab520-7b7f4ff56c Error creating: pods "payload-processing-e2e-inf-a-3ab520-7b7f4ff56c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-3ab520: serviceaccount "payload-processing-e2e-inf-a-3ab520" not found 34m Normal Pulled pod/payload-pre-processing-e2e-inf-a-3ab520-848dc5bd6c-k9zhf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Created pod/payload-pre-processing-e2e-inf-a-3ab520-848dc5bd6c-k9zhf Container created 34m Normal Started pod/payload-pre-processing-e2e-inf-a-3ab520-848dc5bd6c-k9zhf Container started 34m Normal Started pod/payload-processing-e2e-inf-a-3ab520-7b7f4ff56c-vszhw Container started 34m Normal Created pod/payload-processing-e2e-inf-a-3ab520-7b7f4ff56c-vszhw Container created 34m Normal AddedInterface pod/payload-processing-e2e-inf-a-3ab520-7b7f4ff56c-vszhw Add eth0 [10.134.0.239/23] from ovn-kubernetes 34m Normal Pulled pod/payload-processing-e2e-inf-a-3ab520-7b7f4ff56c-vszhw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-qswqs Add eth0 [10.134.0.240/23] from ovn-kubernetes 34m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-qswqs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-qps6t Add eth0 [10.132.0.122/23] from ovn-kubernetes 34m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-qps6t Container created 34m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-qps6t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-qps6t Container started 34m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-qswqs Container created 34m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-qswqs Container started 34m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-qps6t Startup probe error: Get "http://10.132.0.122:15021/healthz/ready": dial tcp 10.132.0.122:15021: connect: connection refused... 34m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-qps6t Startup probe failed: Get "http://10.132.0.122:15021/healthz/ready": dial tcp 10.132.0.122:15021: connect: connection refused 34m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-qswqs Startup probe failed: Get "http://10.134.0.240:15021/healthz/ready": dial tcp 10.134.0.240:15021: connect: connection refused 34m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-qswqs Startup probe error: Get "http://10.134.0.240:15021/healthz/ready": dial tcp 10.134.0.240:15021: connect: connection refused... 34m Normal SuccessfulCreate replicaset/e2e-inf-b-7091fd-openshift-default-56c9644bc Created pod: e2e-inf-b-7091fd-openshift-default-56c9644bc-h5bjz 34m Normal ScalingReplicaSet deployment/e2e-inf-b-7091fd-openshift-default Scaled up replica set e2e-inf-b-7091fd-openshift-default-56c9644bc from 0 to 1 34m Normal AddedLabel gateway/e2e-inf-b-7091fd Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-7091fd 34m Normal AddedInterface pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-h5bjz Add eth0 [10.133.0.135/23] from ovn-kubernetes 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-4ac1ef-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-4ac1ef-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-deff5e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-deff5e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-7091fd Scaled up replica set payload-pre-processing-e2e-inf-b-7091fd-6d79589dd7 from 0 to 1 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-7091fd Scaled up replica set payload-processing-e2e-inf-b-7091fd-98896dbf7 from 0 to 1 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-7091fd-6d79589dd7 Error creating: pods "payload-pre-processing-e2e-inf-b-7091fd-6d79589dd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-7091fd: serviceaccount "payload-processing-e2e-inf-b-7091fd" not found 34m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-7091fd-98896dbf7 Error creating: pods "payload-processing-e2e-inf-b-7091fd-98896dbf7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-7091fd: serviceaccount "payload-processing-e2e-inf-b-7091fd" not found 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-7091fd-98896dbf7 Created pod: payload-processing-e2e-inf-b-7091fd-98896dbf7-6r5f6 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-7091fd-6d79589dd7 Created pod: payload-pre-processing-e2e-inf-b-7091fd-6d79589dd7-mqf74 34m Normal Pulled pod/payload-processing-e2e-inf-b-7091fd-98896dbf7-6r5f6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Started pod/payload-processing-e2e-inf-b-7091fd-98896dbf7-6r5f6 Container started 34m Normal Created pod/payload-processing-e2e-inf-b-7091fd-98896dbf7-6r5f6 Container created 34m Normal Started pod/payload-pre-processing-e2e-inf-b-7091fd-6d79589dd7-mqf74 Container started 34m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-7091fd-6d79589dd7-mqf74 Add eth0 [10.134.0.241/23] from ovn-kubernetes 34m Normal Pulled pod/payload-pre-processing-e2e-inf-b-7091fd-6d79589dd7-mqf74 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/payload-processing-e2e-inf-b-7091fd-98896dbf7-6r5f6 Add eth0 [10.134.0.242/23] from ovn-kubernetes 34m Normal Created pod/payload-pre-processing-e2e-inf-b-7091fd-6d79589dd7-mqf74 Container created 34m Warning ProbeError pod/e2e-inf-a-6b9305-openshift-default-74c965c97c-7xs26 Readiness probe error: Get "http://10.134.0.225:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 34m Warning Unhealthy pod/e2e-inf-a-6b9305-openshift-default-74c965c97c-7xs26 Readiness probe failed: Get "http://10.134.0.225:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 34m Normal Pulled pod/e2e-inf-a-6b9305-openshift-default-74c965c97c-7xs26 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Created pod/e2e-inf-a-6b9305-openshift-default-74c965c97c-7xs26 Container created 34m Normal Started pod/e2e-inf-a-6b9305-openshift-default-74c965c97c-7xs26 Container started 34m Warning Unhealthy pod/e2e-inf-a-6b9305-openshift-default-74c965c97c-7xs26 Startup probe failed: Get "http://10.134.0.225:15021/healthz/ready": dial tcp 10.134.0.225:15021: connect: connection refused 34m Normal SuccessfulCreate replicaset/e2e-api-b-670df9-openshift-default-6c5f7449f5 Created pod: e2e-api-b-670df9-openshift-default-6c5f7449f5-zvvdk 34m Warning ProbeError pod/e2e-inf-a-6b9305-openshift-default-74c965c97c-7xs26 Startup probe error: Get "http://10.134.0.225:15021/healthz/ready": dial tcp 10.134.0.225:15021: connect: connection refused... 34m Normal ScalingReplicaSet deployment/e2e-api-b-670df9-openshift-default Scaled up replica set e2e-api-b-670df9-openshift-default-6c5f7449f5 from 0 to 1 34m Normal AddedLabel gateway/e2e-api-b-670df9 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-670df9 34m Normal AddedInterface pod/e2e-api-b-670df9-openshift-default-6c5f7449f5-zvvdk Add eth0 [10.134.0.244/23] from ovn-kubernetes 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-6b9305-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-6b9305-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-670df9 Scaled up replica set payload-processing-e2e-api-b-670df9-86c5df874f from 0 to 1 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-670df9-6b57976fb7 Created pod: payload-pre-processing-e2e-api-b-670df9-6b57976fb7-rgk2d 34m Normal AddedLabel gateway/e2e-inf-b-044440 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-044440 34m Warning FailedCreate replicaset/payload-processing-e2e-api-b-670df9-86c5df874f Error creating: pods "payload-processing-e2e-api-b-670df9-86c5df874f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-670df9: serviceaccount "payload-processing-e2e-api-b-670df9" not found 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-670df9-6b57976fb7 Error creating: pods "payload-pre-processing-e2e-api-b-670df9-6b57976fb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-670df9: serviceaccount "payload-processing-e2e-api-b-670df9" not found 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-670df9-86c5df874f Created pod: payload-processing-e2e-api-b-670df9-86c5df874f-vm2ck 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-670df9 Scaled up replica set payload-pre-processing-e2e-api-b-670df9-6b57976fb7 from 0 to 1 34m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-670df9-6b57976fb7-rgk2d Add eth0 [10.134.0.246/23] from ovn-kubernetes 34m Normal Started pod/payload-pre-processing-e2e-api-b-670df9-6b57976fb7-rgk2d Container started 34m Normal Created pod/payload-pre-processing-e2e-api-b-670df9-6b57976fb7-rgk2d Container created 34m Normal Started pod/payload-processing-e2e-api-b-670df9-86c5df874f-vm2ck Container started 34m Normal Pulled pod/payload-pre-processing-e2e-api-b-670df9-6b57976fb7-rgk2d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal ScalingReplicaSet deployment/e2e-inf-b-044440-openshift-default Scaled up replica set e2e-inf-b-044440-openshift-default-7f6ddf4485 from 0 to 1 34m Normal AddedInterface pod/payload-processing-e2e-api-b-670df9-86c5df874f-vm2ck Add eth0 [10.132.0.123/23] from ovn-kubernetes 34m Normal Pulled pod/payload-processing-e2e-api-b-670df9-86c5df874f-vm2ck Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Created pod/payload-processing-e2e-api-b-670df9-86c5df874f-vm2ck Container created 34m Normal SuccessfulCreate replicaset/e2e-inf-b-044440-openshift-default-7f6ddf4485 Created pod: e2e-inf-b-044440-openshift-default-7f6ddf4485-wjzfh 34m Normal AddedInterface pod/e2e-inf-b-044440-openshift-default-7f6ddf4485-wjzfh Add eth0 [10.133.0.136/23] from ovn-kubernetes 33m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-044440 Scaled up replica set payload-pre-processing-e2e-inf-b-044440-79875748f5 from 0 to 1 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-6b9305-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-6b9305-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-044440-79875748f5 Created pod: payload-pre-processing-e2e-inf-b-044440-79875748f5-mwv7c 33m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-044440 Scaled up replica set payload-processing-e2e-inf-b-044440-5678b676c from 0 to 1 33m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-044440-79875748f5 Error creating: pods "payload-pre-processing-e2e-inf-b-044440-79875748f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-044440: serviceaccount "payload-processing-e2e-inf-b-044440" not found 33m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-044440-5678b676c Error creating: pods "payload-processing-e2e-inf-b-044440-5678b676c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-044440: serviceaccount "payload-processing-e2e-inf-b-044440" not found 33m Normal Started pod/payload-pre-processing-e2e-inf-b-044440-79875748f5-mwv7c Container started 33m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-044440-5678b676c Created pod: payload-processing-e2e-inf-b-044440-5678b676c-cg7nt 33m Normal Killing pod/payload-pre-processing-e2e-worker-w6-dd038b-c5cc45b55-nd59r Stopping container payload-pre-processing 33m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-044440-79875748f5-mwv7c Add eth0 [10.134.0.247/23] from ovn-kubernetes 33m Normal Pulled pod/payload-pre-processing-e2e-inf-b-044440-79875748f5-mwv7c Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal Created pod/payload-pre-processing-e2e-inf-b-044440-79875748f5-mwv7c Container created 33m Normal Killing pod/payload-processing-e2e-worker-w6-dd038b-58bf969f7c-5tbbt Stopping container payload-processing 33m Normal Started pod/payload-processing-e2e-inf-b-044440-5678b676c-cg7nt Container started 33m Normal Created pod/payload-processing-e2e-inf-b-044440-5678b676c-cg7nt Container created 33m Normal Pulled pod/payload-processing-e2e-inf-b-044440-5678b676c-cg7nt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/payload-processing-e2e-inf-b-044440-5678b676c-cg7nt Add eth0 [10.132.0.124/23] from ovn-kubernetes 33m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-nd97z Readiness probe error: Get "http://10.132.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 33m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-nd97z Readiness probe failed: Get "http://10.132.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 33m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w6-dd038b-c5cc45b55-nd59r Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Warning Unhealthy pod/payload-processing-e2e-worker-w6-dd038b-58bf969f7c-5tbbt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Normal Started pod/e2e-api-b-a41242-openshift-default-86bb88f59-hn6s2 Container started 32m Normal Pulled pod/e2e-inf-a-3ab520-openshift-default-6765b98db9-jrzgk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Started pod/e2e-inf-a-3ab520-openshift-default-6765b98db9-jrzgk Container started 32m Normal Pulled pod/e2e-api-b-a41242-openshift-default-86bb88f59-hn6s2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-api-b-a41242-openshift-default-86bb88f59-hn6s2 Container created 32m Normal Created pod/e2e-inf-a-3ab520-openshift-default-6765b98db9-jrzgk Container created 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-3ab520-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-3ab520-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning Unhealthy pod/e2e-api-b-a41242-openshift-default-86bb88f59-hn6s2 Startup probe failed: Get "http://10.134.0.233:15021/healthz/ready": dial tcp 10.134.0.233:15021: connect: connection refused 32m Warning Unhealthy pod/e2e-inf-a-3ab520-openshift-default-6765b98db9-jrzgk Startup probe failed: Get "http://10.134.0.237:15021/healthz/ready": dial tcp 10.134.0.237:15021: connect: connection refused 32m Warning ProbeError pod/e2e-api-b-a41242-openshift-default-86bb88f59-hn6s2 Startup probe error: Get "http://10.134.0.233:15021/healthz/ready": dial tcp 10.134.0.233:15021: connect: connection refused... 32m Warning ProbeError pod/e2e-inf-a-3ab520-openshift-default-6765b98db9-jrzgk Startup probe error: Get "http://10.134.0.237:15021/healthz/ready": dial tcp 10.134.0.237:15021: connect: connection refused... 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-a41242-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-a41242-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-a41242-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-a41242-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-3ab520-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-3ab520-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Normal Started pod/e2e-inf-b-044440-openshift-default-7f6ddf4485-wjzfh Container started 32m Normal Pulled pod/e2e-inf-b-044440-openshift-default-7f6ddf4485-wjzfh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-inf-b-044440-openshift-default-7f6ddf4485-wjzfh Container created 32m Normal Pulled pod/e2e-api-b-670df9-openshift-default-6c5f7449f5-zvvdk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-api-b-670df9-openshift-default-6c5f7449f5-zvvdk Container created 32m Normal Started pod/e2e-api-b-670df9-openshift-default-6c5f7449f5-zvvdk Container started 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-044440-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Normal Killing pod/payload-pre-processing-e2e-api-a-e0a6f5-68b97cb76c-sm88c Stopping container payload-pre-processing 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-044440-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Normal Killing pod/payload-processing-e2e-api-a-e0a6f5-56bbd4f6f7-5gtlm Stopping container payload-processing 31m Warning Unhealthy pod/e2e-inf-b-044440-openshift-default-7f6ddf4485-wjzfh Startup probe failed: Get "http://10.133.0.136:15021/healthz/ready": dial tcp 10.133.0.136:15021: connect: connection refused 31m Warning ProbeError pod/e2e-inf-b-044440-openshift-default-7f6ddf4485-wjzfh Startup probe error: Get "http://10.133.0.136:15021/healthz/ready": dial tcp 10.133.0.136:15021: connect: connection refused... 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-670df9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-670df9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning Unhealthy pod/e2e-api-b-670df9-openshift-default-6c5f7449f5-zvvdk Startup probe failed: Get "http://10.134.0.244:15021/healthz/ready": dial tcp 10.134.0.244:15021: connect: connection refused 31m Warning ProbeError pod/e2e-api-b-670df9-openshift-default-6c5f7449f5-zvvdk Startup probe error: Get "http://10.134.0.244:15021/healthz/ready": dial tcp 10.134.0.244:15021: connect: connection refused... 31m Normal Killing pod/payload-processing-e2e-inf-a-3ab520-7b7f4ff56c-vszhw Stopping container payload-processing 31m Normal Killing pod/payload-pre-processing-e2e-inf-a-3ab520-848dc5bd6c-k9zhf Stopping container payload-pre-processing 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-944263-openshift-default New size: 1; reason: All metrics below target 31m Normal ScalingReplicaSet deployment/e2e-worker-w3-944263-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w3-944263-openshift-default-5f78997bc9 from 2 to 1 31m Normal SuccessfulDelete replicaset/e2e-worker-w3-944263-openshift-default-5f78997bc9 Deleted pod: e2e-worker-w3-944263-openshift-default-5f78997bc9-lxjj4 31m Normal Killing pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-lxjj4 Stopping container istio-proxy 31m Warning ProbeError pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-lxjj4 Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning Unhealthy pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-lxjj4 Readiness probe failed: HTTP probe failed with statuscode: 503 31m Normal SuccessfulDelete replicaset/e2e-worker-w0-208f2b-openshift-default-5cc586d6f Deleted pod: e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lgsnj 31m Normal ScalingReplicaSet deployment/e2e-worker-w0-208f2b-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w0-208f2b-openshift-default-5cc586d6f from 2 to 1 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-208f2b-openshift-default New size: 1; reason: All metrics below target 31m Normal Killing pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-lgsnj Stopping container istio-proxy 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-044440-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-044440-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-670df9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-670df9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Normal SuccessfulCreate replicaset/e2e-inf-b-7091fd-openshift-default-56c9644bc Created pod: e2e-inf-b-7091fd-openshift-default-56c9644bc-7b6td 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-7091fd-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 31m Normal SuccessfulCreate replicaset/e2e-inf-b-7091fd-openshift-default-56c9644bc Created pod: e2e-inf-b-7091fd-openshift-default-56c9644bc-qk7ll 31m Normal ScalingReplicaSet deployment/e2e-inf-b-7091fd-openshift-default Scaled up replica set e2e-inf-b-7091fd-openshift-default-56c9644bc from 1 to 4 31m Normal SuccessfulCreate replicaset/e2e-inf-b-7091fd-openshift-default-56c9644bc Created pod: e2e-inf-b-7091fd-openshift-default-56c9644bc-hnhqh 31m Normal Created pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-qk7ll Container created 31m Normal Pulled pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-qk7ll Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-7b6td Add eth0 [10.134.0.254/23] from ovn-kubernetes 31m Normal Pulled pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-hnhqh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Pulled pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-7b6td Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-qk7ll Add eth0 [10.132.0.128/23] from ovn-kubernetes 31m Normal AddedInterface pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-hnhqh Add eth0 [10.134.0.253/23] from ovn-kubernetes 31m Normal Created pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-7b6td Container created 31m Normal Started pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-7b6td Container started 31m Normal Started pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-hnhqh Container started 31m Normal Created pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-hnhqh Container created 31m Normal Started pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-qk7ll Container started 31m Normal Created pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-h5bjz Container created 31m Warning Unhealthy pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-h5bjz Readiness probe failed: Get "http://10.133.0.135:15021/healthz/ready": dial tcp 10.133.0.135:15021: connect: connection refused 31m Normal Killing pod/payload-processing-e2e-inf-a-4ac1ef-5bddb89485-dpbt4 Stopping container payload-processing 31m Normal Pulled pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-h5bjz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Warning ProbeError pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-h5bjz Readiness probe error: Get "http://10.133.0.135:15021/healthz/ready": dial tcp 10.133.0.135:15021: connect: connection refused... 31m Normal Killing pod/payload-pre-processing-e2e-inf-a-4ac1ef-db4769dc-ffkcl Stopping container payload-pre-processing 31m Normal Started pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-h5bjz Container started 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-dd038b-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 31m Normal Killing pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-h2t4l Stopping container istio-proxy 31m Normal SuccessfulDelete replicaset/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 Deleted pod: e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-h2t4l 31m Normal ScalingReplicaSet deployment/e2e-worker-w6-dd038b-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7 from 2 to 1 31m Warning Unhealthy pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-h2t4l Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning ProbeError pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-h2t4l Readiness probe error: HTTP probe failed with statuscode: 503... 31m Normal ScalingReplicaSet deployment/e2e-worker-w4-1a1e9e-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w4-1a1e9e-openshift-default-56c9685797 from 2 to 1 31m Warning ProbeError pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-qk7ll Startup probe error: Get "http://10.132.0.128:15021/healthz/ready": dial tcp 10.132.0.128:15021: connect: connection refused... 31m Normal Killing pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-nsqsb Stopping container istio-proxy 31m Normal SuccessfulDelete replicaset/e2e-worker-w4-1a1e9e-openshift-default-56c9685797 Deleted pod: e2e-worker-w4-1a1e9e-openshift-default-56c9685797-nsqsb 31m Warning Unhealthy pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-qk7ll Startup probe failed: Get "http://10.132.0.128:15021/healthz/ready": dial tcp 10.132.0.128:15021: connect: connection refused 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-1a1e9e-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 31m Warning ProbeError pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-hnhqh Startup probe error: Get "http://10.134.0.253:15021/healthz/ready": dial tcp 10.134.0.253:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-hnhqh Startup probe failed: Get "http://10.134.0.253:15021/healthz/ready": dial tcp 10.134.0.253:15021: connect: connection refused 31m Warning Unhealthy pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-7b6td Startup probe failed: Get "http://10.134.0.254:15021/healthz/ready": dial tcp 10.134.0.254:15021: connect: connection refused 31m Normal Killing pod/e2e-worker-w6-dd038b-openshift-default-7d9df7cfd7-dcbhv Stopping container istio-proxy 31m Warning ProbeError pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-7b6td Startup probe error: Get "http://10.134.0.254:15021/healthz/ready": dial tcp 10.134.0.254:15021: connect: connection refused... 31m Warning ProbeError pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-h5bjz Startup probe error: Get "http://10.133.0.135:15021/healthz/ready": dial tcp 10.133.0.135:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-h5bjz Startup probe failed: Get "http://10.133.0.135:15021/healthz/ready": dial tcp 10.133.0.135:15021: connect: connection refused 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-7091fd-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-7091fd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Normal Killing pod/payload-pre-processing-e2e-api-a-deff5e-77db945ff9-jtzp7 Stopping container payload-pre-processing 30m Normal Killing pod/payload-processing-e2e-api-a-deff5e-bd664cbf7-wvr5w Stopping container payload-processing 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-7091fd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-7091fd-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Normal Killing pod/payload-processing-e2e-inf-a-6b9305-6964565579-hwlhv Stopping container payload-processing 30m Normal Killing pod/payload-pre-processing-e2e-inf-a-6b9305-864cc97d9f-dcsdn Stopping container payload-pre-processing 30m Normal SuccessfulCreate replicaset/e2e-inf-b-7091fd-openshift-default-56c9644bc Created pod: e2e-inf-b-7091fd-openshift-default-56c9644bc-vpbgp 30m Normal ScalingReplicaSet deployment/e2e-inf-b-7091fd-openshift-default Scaled up replica set e2e-inf-b-7091fd-openshift-default-56c9644bc from 4 to 5 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-7091fd-openshift-default New size: 5; reason: All metrics below target 30m Normal AddedInterface pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-vpbgp Add eth0 [10.133.0.139/23] from ovn-kubernetes 30m Normal Pulled pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-vpbgp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Created pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-vpbgp Container created 30m Normal Started pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-vpbgp Container started 30m Warning Unhealthy pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-vpbgp Startup probe failed: Get "http://10.133.0.139:15021/healthz/ready": dial tcp 10.133.0.139:15021: connect: connection refused 30m Warning ProbeError pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-vpbgp Startup probe error: Get "http://10.133.0.139:15021/healthz/ready": dial tcp 10.133.0.139:15021: connect: connection refused... 30m Normal Killing pod/e2e-inf-a-3ab520-openshift-default-6765b98db9-jrzgk Stopping container istio-proxy 30m Normal Killing pod/e2e-inf-a-4ac1ef-openshift-default-54d7ccf455-j89g4 Stopping container istio-proxy 30m Normal Killing pod/payload-pre-processing-e2e-worker-w2-cd248f-7b8875bf8c-c8fvd Stopping container payload-pre-processing 30m Normal Killing pod/payload-processing-e2e-worker-w2-cd248f-6c9889fc9c-wbd52 Stopping container payload-processing 30m Normal Killing pod/payload-processing-e2e-inf-b-7091fd-98896dbf7-6r5f6 Stopping container payload-processing 30m Normal Killing pod/payload-pre-processing-e2e-inf-b-7091fd-6d79589dd7-mqf74 Stopping container payload-pre-processing 29m Warning Unhealthy pod/payload-processing-e2e-worker-w2-cd248f-6c9889fc9c-wbd52 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Normal SuccessfulDelete replicaset/e2e-worker-w1-db0157-openshift-default-68856dc5c7 Deleted pod: e2e-worker-w1-db0157-openshift-default-68856dc5c7-5bxs8 29m Normal SuccessfulDelete replicaset/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 Deleted pod: e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-2qv5b 29m Normal ScalingReplicaSet deployment/e2e-worker-w1-db0157-openshift-default Scaled down replica set e2e-worker-w1-db0157-openshift-default-68856dc5c7 from 2 to 1 29m Normal Killing pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-2qv5b Stopping container istio-proxy 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-db0157-openshift-default New size: 1; reason: All metrics below target 29m Normal Killing pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-5bxs8 Stopping container istio-proxy 29m Normal ScalingReplicaSet deployment/e2e-worker-w2-cd248f-openshift-default Scaled down replica set e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5 from 2 to 1 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-cd248f-openshift-default New size: 1; reason: All metrics below target 29m Normal Killing pod/e2e-inf-a-6b9305-openshift-default-74c965c97c-7xs26 Stopping container istio-proxy 29m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-cd248f-7b8875bf8c-c8fvd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Warning ProbeError pod/e2e-inf-a-6b9305-openshift-default-74c965c97c-7xs26 Readiness probe error: HTTP probe failed with statuscode: 503... 29m Warning Unhealthy pod/e2e-inf-a-6b9305-openshift-default-74c965c97c-7xs26 Readiness probe failed: HTTP probe failed with statuscode: 503 29m Normal Killing pod/e2e-api-a-e0a6f5-openshift-default-5b74475bdc-rrrbc Stopping container istio-proxy 29m Warning Unhealthy pod/payload-processing-e2e-worker-w2-cd248f-6c9889fc9c-wbd52 Readiness probe failed: timeout: failed to connect service "10.134.0.40:50052" within 1s: context deadline exceeded 29m Warning ProbeError pod/e2e-api-a-e0a6f5-openshift-default-5b74475bdc-rrrbc Readiness probe error: HTTP probe failed with statuscode: 503... 29m Warning Unhealthy pod/e2e-api-a-e0a6f5-openshift-default-5b74475bdc-rrrbc Readiness probe failed: HTTP probe failed with statuscode: 503 29m Normal Killing pod/payload-pre-processing-e2e-inf-b-044440-79875748f5-mwv7c Stopping container payload-pre-processing 29m Normal Killing pod/payload-processing-e2e-inf-b-044440-5678b676c-cg7nt Stopping container payload-processing 29m Normal Killing pod/payload-pre-processing-e2e-api-b-a41242-5956bf7979-dsqpz Stopping container payload-pre-processing 29m Normal Killing pod/payload-processing-e2e-api-b-a41242-6bfc8bf88f-cs272 Stopping container payload-processing 28m Normal Killing pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-h5bjz Stopping container istio-proxy 28m Normal Killing pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-hnhqh Stopping container istio-proxy 28m Normal Killing pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-vpbgp Stopping container istio-proxy 28m Normal Killing pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-qk7ll Stopping container istio-proxy 28m Normal Killing pod/e2e-inf-b-7091fd-openshift-default-56c9644bc-7b6td Stopping container istio-proxy 28m Normal Killing pod/e2e-api-a-deff5e-openshift-default-589bfb887f-hnvnf Stopping container istio-proxy 28m Normal Killing pod/payload-pre-processing-e2e-shared-a-w1-dce296-65f5556975-m2d7v Stopping container payload-pre-processing 28m Normal Killing pod/payload-processing-e2e-shared-a-w1-dce296-85d75dbb6f-rq556 Stopping container payload-processing 28m Normal Killing pod/e2e-inf-b-044440-openshift-default-7f6ddf4485-wjzfh Stopping container istio-proxy 28m Warning Unhealthy pod/e2e-inf-b-044440-openshift-default-7f6ddf4485-wjzfh Readiness probe failed: HTTP probe failed with statuscode: 503 28m Normal Killing pod/payload-processing-e2e-api-b-670df9-86c5df874f-vm2ck Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-api-b-670df9-6b57976fb7-rgk2d Stopping container payload-pre-processing 28m Warning ProbeError pod/e2e-inf-b-044440-openshift-default-7f6ddf4485-wjzfh Readiness probe error: HTTP probe failed with statuscode: 503... 28m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-kmd5t 28m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-kmd5t Stopping container discovery 28m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 28m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 28m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-58c6585447 Deleted pod: maas-default-gateway-openshift-default-58c6585447-dggk5 28m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-58c6585447 Deleted pod: maas-default-gateway-openshift-default-58c6585447-7b8v8 28m Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-dggk5 Stopping container istio-proxy 28m Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-qswqs Stopping container istio-proxy 28m Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-wwpvf Stopping container istio-proxy 28m Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-7b8v8 Stopping container istio-proxy 28m Normal Killing pod/payload-processing-e2e-shared-a-w3-24f923-8474d84f5-kfvdm Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-shared-a-w3-24f923-869bcbd967-w25fl Stopping container payload-pre-processing 28m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-7b8v8 Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-7b8v8 Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w3-24f923-869bcbd967-w25fl Readiness probe failed: timeout: failed to connect service "10.134.0.163:50052" within 1s: context deadline exceeded 28m Normal Killing pod/e2e-worker-w2-cd248f-openshift-default-d7fc5ccf5-sh9p6 Stopping container istio-proxy 27m Normal Killing pod/e2e-shared-a-w1-dce296-openshift-default-84fd9bf7ff-htbpm Stopping container istio-proxy 27m Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-pg8rw Stopping container istio-proxy 27m Normal Killing pod/e2e-api-b-670df9-openshift-default-6c5f7449f5-zvvdk Stopping container istio-proxy 27m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default (combined from similar events): Scaled down replica set maas-default-gateway-openshift-default-58c6585447 from 6 to 4 27m Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-m6rnq Stopping container istio-proxy 27m Normal Killing pod/payload-pre-processing-e2e-shared-b-w1-df700a-789b6c9547-g9ps7 Stopping container payload-pre-processing 27m Normal Killing pod/payload-processing-e2e-shared-b-w1-df700a-788d57d789-n6jlx Stopping container payload-processing 27m Normal Killing pod/e2e-api-b-a41242-openshift-default-86bb88f59-hn6s2 Stopping container istio-proxy 27m Warning ProbeError pod/e2e-api-b-a41242-openshift-default-86bb88f59-hn6s2 Readiness probe error: HTTP probe failed with statuscode: 503... 27m Warning Unhealthy pod/e2e-api-b-a41242-openshift-default-86bb88f59-hn6s2 Readiness probe failed: HTTP probe failed with statuscode: 503 27m Normal Killing pod/payload-processing-e2e-worker-w0-208f2b-7fb7755ddc-lx76l Stopping container payload-processing 27m Normal Killing pod/payload-pre-processing-e2e-worker-w0-208f2b-76b7b8c9f-fxd8f Stopping container payload-pre-processing 27m Normal Killing pod/e2e-shared-a-w3-24f923-openshift-default-645f69b989-dhxtq Stopping container istio-proxy 27m Normal Killing pod/payload-pre-processing-e2e-worker-w4-1a1e9e-8fc6dcb7-bs6sm Stopping container payload-pre-processing 27m Normal Killing pod/payload-processing-e2e-worker-w4-1a1e9e-66d4f8c5f-cr9fp Stopping container payload-processing 27m Warning Unhealthy pod/payload-processing-e2e-worker-w0-208f2b-7fb7755ddc-lx76l Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal Killing pod/e2e-shared-b-w1-df700a-openshift-default-6cf7fb69dc-dg822 Stopping container istio-proxy 27m Normal Killing pod/payload-pre-processing-e2e-shared-b-w3-bf3fa2-5f4bb58487-jrpbx Stopping container payload-pre-processing 27m Normal Killing pod/payload-processing-e2e-shared-b-w3-bf3fa2-79cb8b8f-w5wfb Stopping container payload-processing 27m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-208f2b-76b7b8c9f-fxd8f Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w4-1a1e9e-8fc6dcb7-bs6sm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Warning Unhealthy pod/payload-processing-e2e-worker-w4-1a1e9e-66d4f8c5f-cr9fp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal Killing pod/e2e-shared-b-w3-bf3fa2-openshift-default-69d7c5c975-84bmk Stopping container istio-proxy 27m Normal Killing pod/payload-pre-processing-e2e-worker-w1-db0157-6689cd59bc-nd7kc Stopping container payload-pre-processing 27m Normal Killing pod/payload-processing-e2e-worker-w1-db0157-654558c4b7-vsbx9 Stopping container payload-processing 26m Normal Killing pod/e2e-worker-w4-1a1e9e-openshift-default-56c9685797-w8rd7 Stopping container istio-proxy 26m Normal Killing pod/payload-pre-processing-e2e-worker-w3-944263-78999d46dc-m8ml8 Stopping container payload-pre-processing 26m Normal Killing pod/payload-processing-e2e-worker-w3-944263-5cf85467cc-sl4cb Stopping container payload-processing 26m Normal Killing pod/e2e-worker-w0-208f2b-openshift-default-5cc586d6f-blcnw Stopping container istio-proxy 26m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w1-db0157-6689cd59bc-nd7kc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Warning Unhealthy pod/payload-processing-e2e-worker-w1-db0157-654558c4b7-vsbx9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w3-944263-78999d46dc-m8ml8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Warning Unhealthy pod/payload-processing-e2e-worker-w3-944263-5cf85467cc-sl4cb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Normal Killing pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-784tj Stopping container istio-proxy 26m Warning ProbeError pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-784tj Readiness probe error: HTTP probe failed with statuscode: 503... 26m Warning Unhealthy pod/e2e-worker-w1-db0157-openshift-default-68856dc5c7-784tj Readiness probe failed: HTTP probe failed with statuscode: 503 26m Normal Killing pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-np65v Stopping container istio-proxy 26m Warning Unhealthy pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-np65v Readiness probe failed: HTTP probe failed with statuscode: 503 26m Warning ProbeError pod/e2e-worker-w3-944263-openshift-default-5f78997bc9-np65v Readiness probe error: HTTP probe failed with statuscode: 503... 25m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-hdvw9 Container created 25m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-2n5h6 Container created 25m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-bb8j6 Add eth0 [10.133.0.143/23] from ovn-kubernetes 25m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-2n5h6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-bb8j6 Container created 25m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-hdvw9 Add eth0 [10.134.1.32/23] from ovn-kubernetes 25m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-2n5h6 Add eth0 [10.134.1.34/23] from ovn-kubernetes 25m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-hdvw9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-hdvw9 Container started 25m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-6zwld Add eth0 [10.134.1.33/23] from ovn-kubernetes 25m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-bb8j6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-bb8j6 Container started 25m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-2n5h6 Container started 25m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default (combined from similar events): New size: 8; reason: cpu resource utilization (percentage of request) above target 25m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-58c6585447 from 4 to 8 25m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-6zwld Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-6zwld Container created 25m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-6zwld Container started 25m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-bb8j6 Startup probe failed: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused 25m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-bb8j6 Startup probe error: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused... 25m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-hdvw9 Startup probe error: Get "http://10.134.1.32:15021/healthz/ready": dial tcp 10.134.1.32:15021: connect: connection refused... 25m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-hdvw9 Startup probe failed: Get "http://10.134.1.32:15021/healthz/ready": dial tcp 10.134.1.32:15021: connect: connection refused 25m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-2n5h6 Startup probe error: Get "http://10.134.1.34:15021/healthz/ready": dial tcp 10.134.1.34:15021: connect: connection refused... 25m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-2n5h6 Startup probe failed: Get "http://10.134.1.34:15021/healthz/ready": dial tcp 10.134.1.34:15021: connect: connection refused 25m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-6zwld Startup probe error: Get "http://10.134.1.33:15021/healthz/ready": dial tcp 10.134.1.33:15021: connect: connection refused... 25m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-6zwld Startup probe failed: Get "http://10.134.1.33:15021/healthz/ready": dial tcp 10.134.1.33:15021: connect: connection refused 24m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-58c6585447 from 8 to 10 24m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-qm8qv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-gnfzl Container created 24m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-qm8qv Container started 24m Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-gnfzl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-gnfzl Add eth0 [10.133.0.144/23] from ovn-kubernetes 24m Normal Started pod/maas-default-gateway-openshift-default-58c6585447-gnfzl Container started 24m Normal Created pod/maas-default-gateway-openshift-default-58c6585447-qm8qv Container created 24m Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-qm8qv Add eth0 [10.134.1.36/23] from ovn-kubernetes 24m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-gnfzl Startup probe error: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused... 24m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-gnfzl Startup probe failed: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused 24m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-qm8qv Startup probe error: Get "http://10.134.1.36:15021/healthz/ready": dial tcp 10.134.1.36:15021: connect: connection refused... 24m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-qm8qv Startup probe failed: Get "http://10.134.1.36:15021/healthz/ready": dial tcp 10.134.1.36:15021: connect: connection refused 24m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-qm8qv Startup probe failed: Get "http://10.134.1.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-gnfzl Startup probe error: Get "http://10.133.0.144:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-gnfzl Startup probe failed: Get "http://10.133.0.144:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-qm8qv Startup probe error: Get "http://10.134.1.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 21m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-6zwld Readiness probe error: Get "http://10.134.1.33:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-hdvw9 Readiness probe failed: Get "http://10.134.1.32:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 21m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-hdvw9 Readiness probe error: Get "http://10.134.1.32:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 21m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-2n5h6 Readiness probe error: Get "http://10.134.1.34:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-6zwld Readiness probe failed: Get "http://10.134.1.33:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-2n5h6 Readiness probe failed: Get "http://10.134.1.34:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 18m Normal SuccessfulCreate replicaset/e2e-worker-single-main-7249f4-openshift-default-5d6946dbbf Created pod: e2e-worker-single-main-7249f4-openshift-default-5d6946dbbf95j6n 18m Normal ScalingReplicaSet deployment/e2e-worker-single-main-7249f4-openshift-default Scaled up replica set e2e-worker-single-main-7249f4-openshift-default-5d6946dbbf from 0 to 1 18m Normal AddedLabel gateway/e2e-worker-single-main-7249f4 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-7249f4 18m Normal AddedInterface pod/e2e-worker-single-main-7249f4-openshift-default-5d6946dbbf95j6n Add eth0 [10.134.1.50/23] from ovn-kubernetes 18m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-7249f4 Scaled up replica set payload-pre-processing-e2e-worker-single-main-7249f4-5445889db7 from 0 to 1 18m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-7249f4 Scaled up replica set payload-processing-e2e-worker-single-main-7249f4-7568bd48f9 from 0 to 1 18m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-7249f4-7568bd48f9 Error creating: pods "payload-processing-e2e-worker-single-main-7249f4-7568bd48f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-7249f4: serviceaccount "payload-processing-e2e-worker-single-main-7249f4" not found 18m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-7249f4-7568bd48f9 Created pod: payload-processing-e2e-worker-single-main-7249f4-7568bd48ff597s 18m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-7249f4-5445889db7 Created pod: payload-pre-processing-e2e-worker-single-main-7249f4-544589757t 18m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-7249f4-5445889db7 Error creating: pods "payload-pre-processing-e2e-worker-single-main-7249f4-5445889db7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-7249f4: serviceaccount "payload-processing-e2e-worker-single-main-7249f4" not found 18m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-7249f4-544589757t Add eth0 [10.134.1.51/23] from ovn-kubernetes 18m Normal Pulled pod/payload-processing-e2e-worker-single-main-7249f4-7568bd48ff597s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 18m Normal Started pod/payload-pre-processing-e2e-worker-single-main-7249f4-544589757t Container started 18m Normal Created pod/payload-pre-processing-e2e-worker-single-main-7249f4-544589757t Container created 18m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-7249f4-544589757t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 18m Normal Started pod/payload-processing-e2e-worker-single-main-7249f4-7568bd48ff597s Container started 18m Normal Created pod/payload-processing-e2e-worker-single-main-7249f4-7568bd48ff597s Container created 18m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-7249f4-7568bd48ff597s Add eth0 [10.132.0.133/23] from ovn-kubernetes 17m Normal Pulled pod/e2e-worker-single-main-7249f4-openshift-default-5d6946dbbf95j6n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal Started pod/e2e-worker-single-main-7249f4-openshift-default-5d6946dbbf95j6n Container started 17m Normal Created pod/e2e-worker-single-main-7249f4-openshift-default-5d6946dbbf95j6n Container created 17m Warning Unhealthy pod/e2e-worker-single-main-7249f4-openshift-default-5d6946dbbf95j6n Startup probe failed: Get "http://10.134.1.50:15021/healthz/ready": dial tcp 10.134.1.50:15021: connect: connection refused 17m Warning ProbeError pod/e2e-worker-single-main-7249f4-openshift-default-5d6946dbbf95j6n Startup probe error: Get "http://10.134.1.50:15021/healthz/ready": dial tcp 10.134.1.50:15021: connect: connection refused... 17m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-7249f4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 17m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-7249f4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 17m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-7249f4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 17m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-7249f4-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 16m Normal Killing pod/payload-pre-processing-e2e-worker-single-main-7249f4-544589757t Stopping container payload-pre-processing 16m Normal Killing pod/payload-processing-e2e-worker-single-main-7249f4-7568bd48ff597s Stopping container payload-processing 16m Normal Killing pod/e2e-worker-single-main-7249f4-openshift-default-5d6946dbbf95j6n Stopping container istio-proxy 16m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-7249f4-544589757t Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-7249f4-7568bd48ff597s Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 13m Normal AddedLabel gateway/e2e-worker-single-main-236f0f Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-236f0f 13m Normal SuccessfulCreate replicaset/e2e-worker-single-main-236f0f-openshift-default-689d7dd8c7 Created pod: e2e-worker-single-main-236f0f-openshift-default-689d7dd8c7jnbkk 13m Normal ScalingReplicaSet deployment/e2e-worker-single-main-236f0f-openshift-default Scaled up replica set e2e-worker-single-main-236f0f-openshift-default-689d7dd8c7 from 0 to 1 13m Normal AddedInterface pod/e2e-worker-single-main-236f0f-openshift-default-689d7dd8c7jnbkk Add eth0 [10.134.1.57/23] from ovn-kubernetes 13m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-236f0f-58779bccb9 Error creating: pods "payload-processing-e2e-worker-single-main-236f0f-58779bccb9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-236f0f: serviceaccount "payload-processing-e2e-worker-single-main-236f0f" not found 13m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-236f0f-59cf66f7c5 Error creating: pods "payload-pre-processing-e2e-worker-single-main-236f0f-59cf66f7c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-236f0f: serviceaccount "payload-processing-e2e-worker-single-main-236f0f" not found 13m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-236f0f Scaled up replica set payload-pre-processing-e2e-worker-single-main-236f0f-59cf66f7c5 from 0 to 1 13m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-236f0f Scaled up replica set payload-processing-e2e-worker-single-main-236f0f-58779bccb9 from 0 to 1 13m Normal Created pod/payload-processing-e2e-worker-single-main-236f0f-58779bccbs4r97 Container created 13m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-236f0f-59cf66f7c5 Created pod: payload-pre-processing-e2e-worker-single-main-236f0f-59cf6t5j8h 13m Normal Started pod/payload-pre-processing-e2e-worker-single-main-236f0f-59cf6t5j8h Container started 13m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-236f0f-58779bccb9 Created pod: payload-processing-e2e-worker-single-main-236f0f-58779bccbs4r97 13m Normal Created pod/payload-pre-processing-e2e-worker-single-main-236f0f-59cf6t5j8h Container created 13m Normal Pulled pod/payload-processing-e2e-worker-single-main-236f0f-58779bccbs4r97 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 13m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-236f0f-58779bccbs4r97 Add eth0 [10.132.0.136/23] from ovn-kubernetes 13m Normal Started pod/payload-processing-e2e-worker-single-main-236f0f-58779bccbs4r97 Container started 13m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-236f0f-59cf6t5j8h Add eth0 [10.134.1.59/23] from ovn-kubernetes 13m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-236f0f-59cf6t5j8h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 13m Normal Pulled pod/e2e-worker-single-main-236f0f-openshift-default-689d7dd8c7jnbkk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Normal Created pod/e2e-worker-single-main-236f0f-openshift-default-689d7dd8c7jnbkk Container created 13m Normal Started pod/e2e-worker-single-main-236f0f-openshift-default-689d7dd8c7jnbkk Container started 12m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-236f0f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 12m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-236f0f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 12m Warning Unhealthy pod/e2e-worker-single-main-236f0f-openshift-default-689d7dd8c7jnbkk Startup probe failed: Get "http://10.134.1.57:15021/healthz/ready": dial tcp 10.134.1.57:15021: connect: connection refused 12m Warning ProbeError pod/e2e-worker-single-main-236f0f-openshift-default-689d7dd8c7jnbkk Startup probe error: Get "http://10.134.1.57:15021/healthz/ready": dial tcp 10.134.1.57:15021: connect: connection refused... 12m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-236f0f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 12m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-236f0f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 11m Normal Killing pod/payload-pre-processing-e2e-worker-single-main-236f0f-59cf6t5j8h Stopping container payload-pre-processing 11m Normal Killing pod/payload-processing-e2e-worker-single-main-236f0f-58779bccbs4r97 Stopping container payload-processing 11m Normal Killing pod/e2e-worker-single-main-236f0f-openshift-default-689d7dd8c7jnbkk Stopping container istio-proxy 11m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-236f0f-59cf6t5j8h Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 11m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-236f0f-58779bccbs4r97 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 8m54s Normal ScalingReplicaSet deployment/e2e-ait-49c82ebd-openshift-default Scaled up replica set e2e-ait-49c82ebd-openshift-default-678b9f6cdc from 0 to 1 8m54s Normal SuccessfulCreate replicaset/e2e-ait-49c82ebd-openshift-default-678b9f6cdc Created pod: e2e-ait-49c82ebd-openshift-default-678b9f6cdc-mzsk8 8m54s Normal AddedLabel gateway/e2e-ait-49c82ebd Added label istio.io/rev=openshift-gateway to gateway e2e-ait-49c82ebd 8m53s Normal AddedInterface pod/e2e-ait-49c82ebd-openshift-default-678b9f6cdc-mzsk8 Add eth0 [10.134.1.65/23] from ovn-kubernetes 8m47s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-49c82ebd Scaled up replica set payload-pre-processing-e2e-ait-49c82ebd-649fbdcc87 from 0 to 1 8m47s Warning FailedCreate replicaset/payload-processing-e2e-ait-49c82ebd-59b45759bf Error creating: pods "payload-processing-e2e-ait-49c82ebd-59b45759bf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-49c82ebd: serviceaccount "payload-processing-e2e-ait-49c82ebd" not found 8m47s Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-49c82ebd Scaled up replica set payload-processing-e2e-ait-49c82ebd-59b45759bf from 0 to 1 8m47s Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-49c82ebd-649fbdcc87 Error creating: pods "payload-pre-processing-e2e-ait-49c82ebd-649fbdcc87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-49c82ebd: serviceaccount "payload-processing-e2e-ait-49c82ebd" not found 8m46s Normal AddedInterface pod/payload-processing-e2e-ait-49c82ebd-59b45759bf-hjk4h Add eth0 [10.134.1.66/23] from ovn-kubernetes 8m46s Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-49c82ebd-59b45759bf Created pod: payload-processing-e2e-ait-49c82ebd-59b45759bf-hjk4h 8m46s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-49c82ebd-649fbdcc87 Created pod: payload-pre-processing-e2e-ait-49c82ebd-649fbdcc87-7rzw2 8m46s Normal Created pod/payload-processing-e2e-ait-49c82ebd-59b45759bf-hjk4h Container created 8m46s Normal AddedInterface pod/payload-pre-processing-e2e-ait-49c82ebd-649fbdcc87-7rzw2 Add eth0 [10.134.1.67/23] from ovn-kubernetes 8m46s Normal Created pod/payload-pre-processing-e2e-ait-49c82ebd-649fbdcc87-7rzw2 Container created 8m46s Normal Started pod/payload-pre-processing-e2e-ait-49c82ebd-649fbdcc87-7rzw2 Container started 8m46s Normal Started pod/payload-processing-e2e-ait-49c82ebd-59b45759bf-hjk4h Container started 8m46s Normal Pulled pod/payload-pre-processing-e2e-ait-49c82ebd-649fbdcc87-7rzw2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m46s Normal Pulled pod/payload-processing-e2e-ait-49c82ebd-59b45759bf-hjk4h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m39s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-49c82ebd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 8m39s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-49c82ebd-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 8m28s Normal Pulled pod/e2e-ait-49c82ebd-openshift-default-678b9f6cdc-mzsk8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 8m28s Normal Created pod/e2e-ait-49c82ebd-openshift-default-678b9f6cdc-mzsk8 Container created 8m28s Normal Started pod/e2e-ait-49c82ebd-openshift-default-678b9f6cdc-mzsk8 Container started 8m22s Warning ProbeError pod/e2e-ait-49c82ebd-openshift-default-678b9f6cdc-mzsk8 Startup probe error: Get "http://10.134.1.65:15021/healthz/ready": dial tcp 10.134.1.65:15021: connect: connection refused... 8m22s Warning Unhealthy pod/e2e-ait-49c82ebd-openshift-default-678b9f6cdc-mzsk8 Startup probe failed: Get "http://10.134.1.65:15021/healthz/ready": dial tcp 10.134.1.65:15021: connect: connection refused 8m8s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-49c82ebd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 8m8s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-49c82ebd-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 8m2s Normal Killing pod/payload-processing-e2e-ait-49c82ebd-59b45759bf-hjk4h Stopping container payload-processing 8m2s Normal Killing pod/payload-pre-processing-e2e-ait-49c82ebd-649fbdcc87-7rzw2 Stopping container payload-pre-processing 7m53s Normal Killing pod/e2e-ait-49c82ebd-openshift-default-678b9f6cdc-mzsk8 Stopping container istio-proxy 7m52s Normal ScalingReplicaSet deployment/e2e-authcfg-http-e23c3b-openshift-default Scaled up replica set e2e-authcfg-http-e23c3b-openshift-default-5b9d4fcbbf from 0 to 1 7m52s Normal AddedLabel gateway/e2e-authcfg-http-e23c3b Added label istio.io/rev=openshift-gateway to gateway e2e-authcfg-http-e23c3b 7m51s Normal SuccessfulCreate replicaset/e2e-authcfg-http-e23c3b-openshift-default-5b9d4fcbbf Created pod: e2e-authcfg-http-e23c3b-openshift-default-5b9d4fcbbf-jt6tx 7m51s Normal AddedInterface pod/e2e-authcfg-http-e23c3b-openshift-default-5b9d4fcbbf-jt6tx Add eth0 [10.134.1.69/23] from ovn-kubernetes 7m51s Warning ProbeError pod/e2e-ait-49c82ebd-openshift-default-678b9f6cdc-mzsk8 Readiness probe error: HTTP probe failed with statuscode: 503... 7m51s Warning Unhealthy pod/e2e-ait-49c82ebd-openshift-default-678b9f6cdc-mzsk8 Readiness probe failed: HTTP probe failed with statuscode: 503 7m44s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-authcfg-http-e23c3b-75f45789f9 Created pod: payload-pre-processing-e2e-authcfg-http-e23c3b-75f45789f9-rwh6w 7m44s Warning FailedCreate replicaset/payload-pre-processing-e2e-authcfg-http-e23c3b-75f45789f9 Error creating: pods "payload-pre-processing-e2e-authcfg-http-e23c3b-75f45789f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-e23c3b: serviceaccount "payload-processing-e2e-authcfg-http-e23c3b" not found 7m44s Warning FailedCreate replicaset/payload-processing-e2e-authcfg-http-e23c3b-c8996fd87 Error creating: pods "payload-processing-e2e-authcfg-http-e23c3b-c8996fd87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-e23c3b: serviceaccount "payload-processing-e2e-authcfg-http-e23c3b" not found 7m44s Normal SuccessfulCreate replicaset/payload-processing-e2e-authcfg-http-e23c3b-c8996fd87 Created pod: payload-processing-e2e-authcfg-http-e23c3b-c8996fd87-qfthz 7m44s Normal ScalingReplicaSet deployment/payload-processing-e2e-authcfg-http-e23c3b Scaled up replica set payload-processing-e2e-authcfg-http-e23c3b-c8996fd87 from 0 to 1 7m44s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-authcfg-http-e23c3b Scaled up replica set payload-pre-processing-e2e-authcfg-http-e23c3b-75f45789f9 from 0 to 1 7m43s Normal Created pod/payload-processing-e2e-authcfg-http-e23c3b-c8996fd87-qfthz Container created 7m43s Normal Started pod/payload-pre-processing-e2e-authcfg-http-e23c3b-75f45789f9-rwh6w Container started 7m43s Normal Pulled pod/payload-processing-e2e-authcfg-http-e23c3b-c8996fd87-qfthz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 7m43s Normal Started pod/payload-processing-e2e-authcfg-http-e23c3b-c8996fd87-qfthz Container started 7m43s Normal Created pod/payload-pre-processing-e2e-authcfg-http-e23c3b-75f45789f9-rwh6w Container created 7m43s Normal AddedInterface pod/payload-processing-e2e-authcfg-http-e23c3b-c8996fd87-qfthz Add eth0 [10.132.0.143/23] from ovn-kubernetes 7m43s Normal Pulled pod/payload-pre-processing-e2e-authcfg-http-e23c3b-75f45789f9-rwh6w Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 7m43s Normal AddedInterface pod/payload-pre-processing-e2e-authcfg-http-e23c3b-75f45789f9-rwh6w Add eth0 [10.134.1.70/23] from ovn-kubernetes 7m23s Normal Pulled pod/e2e-authcfg-http-e23c3b-openshift-default-5b9d4fcbbf-jt6tx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 7m23s Normal Created pod/e2e-authcfg-http-e23c3b-openshift-default-5b9d4fcbbf-jt6tx Container created 7m23s Normal Started pod/e2e-authcfg-http-e23c3b-openshift-default-5b9d4fcbbf-jt6tx Container started 7m17s Warning ProbeError pod/e2e-authcfg-http-e23c3b-openshift-default-5b9d4fcbbf-jt6tx Startup probe error: Get "http://10.134.1.69:15021/healthz/ready": dial tcp 10.134.1.69:15021: connect: connection refused... 7m17s Warning Unhealthy pod/e2e-authcfg-http-e23c3b-openshift-default-5b9d4fcbbf-jt6tx Startup probe failed: Get "http://10.134.1.69:15021/healthz/ready": dial tcp 10.134.1.69:15021: connect: connection refused 6m36s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-e23c3b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 6m36s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-e23c3b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 3m54s Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-58c6585447 Deleted pod: maas-default-gateway-openshift-default-58c6585447-gnfzl 3m54s Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-hdvw9 Stopping container istio-proxy 3m54s Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: All metrics below target 3m54s Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-2n5h6 Stopping container istio-proxy 3m54s Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-bb8j6 Stopping container istio-proxy 3m54s Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-gnfzl Stopping container istio-proxy 3m54s Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-qm8qv Stopping container istio-proxy 3m54s Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-58c6585447 Deleted pod: maas-default-gateway-openshift-default-58c6585447-hdvw9 3m54s Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-58c6585447 from 10 to 3 3m54s Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-58c6585447 Deleted pod: maas-default-gateway-openshift-default-58c6585447-qps6t 3m54s Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-qps6t Stopping container istio-proxy 3m54s Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-58c6585447 Deleted pod: maas-default-gateway-openshift-default-58c6585447-2n5h6 3m53s Normal Killing pod/maas-default-gateway-openshift-default-58c6585447-6zwld Stopping container istio-proxy 3m51s Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-qm8qv Readiness probe failed: HTTP probe failed with statuscode: 503 3m51s Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-qm8qv Readiness probe error: HTTP probe failed with statuscode: 503... 3m51s Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-gnfzl Readiness probe failed: HTTP probe failed with statuscode: 503 3m51s Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-gnfzl Readiness probe error: HTTP probe failed with statuscode: 503... 3m5s Normal Killing pod/payload-pre-processing-e2e-authcfg-http-e23c3b-75f45789f9-rwh6w Stopping container payload-pre-processing 3m5s Normal Killing pod/payload-processing-e2e-authcfg-http-e23c3b-c8996fd87-qfthz Stopping container payload-processing 2m49s Warning FailedUpdateStatus horizontalpodautoscaler/e2e-authcfg-http-e23c3b-openshift-default horizontalpodautoscalers.autoscaling "e2e-authcfg-http-e23c3b-openshift-default" not found 2m49s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-e23c3b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 2m49s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-e23c3b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 2m49s Normal Killing pod/e2e-authcfg-http-e23c3b-openshift-default-5b9d4fcbbf-jt6tx Stopping container istio-proxy 2m46s Warning Unhealthy pod/e2e-authcfg-http-e23c3b-openshift-default-5b9d4fcbbf-jt6tx Readiness probe failed: HTTP probe failed with statuscode: 503 2m46s Warning ProbeError pod/e2e-authcfg-http-e23c3b-openshift-default-5b9d4fcbbf-jt6tx Readiness probe error: HTTP probe failed with statuscode: 503... 2m23s Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-58c6585447 Created pod: maas-default-gateway-openshift-default-58c6585447-qrbk7 2m23s Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-58c6585447 from 3 to 6 2m23s Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 2m22s Normal Created pod/maas-default-gateway-openshift-default-58c6585447-qrbk7 Container created 2m22s Normal Started pod/maas-default-gateway-openshift-default-58c6585447-qrbk7 Container started 2m22s Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-6xvdh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 2m22s Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-vmxr9 Add eth0 [10.133.0.148/23] from ovn-kubernetes 2m22s Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-vmxr9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 2m22s Normal Started pod/maas-default-gateway-openshift-default-58c6585447-6xvdh Container started 2m22s Normal Created pod/maas-default-gateway-openshift-default-58c6585447-6xvdh Container created 2m22s Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-qrbk7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 2m22s Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-qrbk7 Add eth0 [10.134.1.73/23] from ovn-kubernetes 2m22s Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-6xvdh Add eth0 [10.134.1.74/23] from ovn-kubernetes 2m22s Normal Created pod/maas-default-gateway-openshift-default-58c6585447-vmxr9 Container created 2m22s Normal Started pod/maas-default-gateway-openshift-default-58c6585447-vmxr9 Container started 2m15s Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-qrbk7 Startup probe error: Get "http://10.134.1.73:15021/healthz/ready": dial tcp 10.134.1.73:15021: connect: connection refused... 2m15s Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-6xvdh Startup probe failed: Get "http://10.134.1.74:15021/healthz/ready": dial tcp 10.134.1.74:15021: connect: connection refused 2m15s Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-vmxr9 Startup probe error: Get "http://10.133.0.148:15021/healthz/ready": dial tcp 10.133.0.148:15021: connect: connection refused... 2m15s Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-vmxr9 Startup probe failed: Get "http://10.133.0.148:15021/healthz/ready": dial tcp 10.133.0.148:15021: connect: connection refused 2m15s Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-qrbk7 Startup probe failed: Get "http://10.134.1.73:15021/healthz/ready": dial tcp 10.134.1.73:15021: connect: connection refused 2m15s Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-6xvdh Startup probe error: Get "http://10.134.1.74:15021/healthz/ready": dial tcp 10.134.1.74:15021: connect: connection refused... 2m8s Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-58c6585447 from 6 to 10 2m8s Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 2m7s Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-9hj29 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 2m7s Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-5t72t Add eth0 [10.132.0.145/23] from ovn-kubernetes 2m7s Normal Started pod/maas-default-gateway-openshift-default-58c6585447-5jn27 Container started 2m7s Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-5jn27 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 2m7s Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-5t72t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 2m7s Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-5jn27 Add eth0 [10.134.1.76/23] from ovn-kubernetes 2m7s Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-9hj29 Add eth0 [10.134.1.75/23] from ovn-kubernetes 2m7s Normal Started pod/maas-default-gateway-openshift-default-58c6585447-cfgkf Container started 2m7s Normal Created pod/maas-default-gateway-openshift-default-58c6585447-9hj29 Container created 2m7s Normal Started pod/maas-default-gateway-openshift-default-58c6585447-9hj29 Container started 2m7s Normal Created pod/maas-default-gateway-openshift-default-58c6585447-cfgkf Container created 2m7s Normal Pulled pod/maas-default-gateway-openshift-default-58c6585447-cfgkf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 2m7s Normal Created pod/maas-default-gateway-openshift-default-58c6585447-5t72t Container created 2m7s Normal AddedInterface pod/maas-default-gateway-openshift-default-58c6585447-cfgkf Add eth0 [10.133.0.149/23] from ovn-kubernetes 2m7s Normal Started pod/maas-default-gateway-openshift-default-58c6585447-5t72t Container started 2m7s Normal Created pod/maas-default-gateway-openshift-default-58c6585447-5jn27 Container created 2m Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-5t72t Startup probe failed: Get "http://10.132.0.145:15021/healthz/ready": dial tcp 10.132.0.145:15021: connect: connection refused 2m Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-5t72t Startup probe error: Get "http://10.132.0.145:15021/healthz/ready": dial tcp 10.132.0.145:15021: connect: connection refused... 119s Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-5jn27 Startup probe failed: Get "http://10.134.1.76:15021/healthz/ready": dial tcp 10.134.1.76:15021: connect: connection refused 119s Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-9hj29 Startup probe failed: Get "http://10.134.1.75:15021/healthz/ready": dial tcp 10.134.1.75:15021: connect: connection refused 119s Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-9hj29 Startup probe error: Get "http://10.134.1.75:15021/healthz/ready": dial tcp 10.134.1.75:15021: connect: connection refused... 119s Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-cfgkf Startup probe error: Get "http://10.133.0.149:15021/healthz/ready": dial tcp 10.133.0.149:15021: connect: connection refused... 119s Warning Unhealthy pod/maas-default-gateway-openshift-default-58c6585447-cfgkf Startup probe failed: Get "http://10.133.0.149:15021/healthz/ready": dial tcp 10.133.0.149:15021: connect: connection refused 119s Warning ProbeError pod/maas-default-gateway-openshift-default-58c6585447-5jn27 Startup probe error: Get "http://10.134.1.76:15021/healthz/ready": dial tcp 10.134.1.76:15021: connect: connection refused...