Time Namespace Component RelatedObject Reason Message

openshift-ingress

e2e-derive-8b33da08-openshift-default-6f7876d755-xhwlz

Scheduled

 Successfully assigned openshift-ingress/e2e-derive-8b33da08-openshift-default-6f7876d755-xhwlz to ip-10-0-131-71.ec2.internal

openshift-ingress

e2e-ait-38d19e47-openshift-default-8654df49cc-8tbck

Scheduled

 Successfully assigned openshift-ingress/e2e-ait-38d19e47-openshift-default-8654df49cc-8tbck to ip-10-0-131-71.ec2.internal

openshift-ingress

e2e-inf-b-ebaa5d-openshift-default-67b687fb47-smjfs

Scheduled

 Successfully assigned openshift-ingress/e2e-inf-b-ebaa5d-openshift-default-67b687fb47-smjfs to ip-10-0-130-239.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-5896d4dd49-mqc82

Scheduled

 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-mqc82 to ip-10-0-131-71.ec2.internal

openshift-ingress

payload-processing-58f86c7798-cm8wv

Scheduled

 Successfully assigned openshift-ingress/payload-processing-58f86c7798-cm8wv to ip-10-0-131-71.ec2.internal

openshift-ingress

e2e-mt-8bfb9318-openshift-default-5884775ff-vwnl9

Scheduled

 Successfully assigned openshift-ingress/e2e-mt-8bfb9318-openshift-default-5884775ff-vwnl9 to ip-10-0-130-239.ec2.internal

openshift-ingress

e2e-inf-a-cdb744-openshift-default-f64ff589c-58cfs

Scheduled

 Successfully assigned openshift-ingress/e2e-inf-a-cdb744-openshift-default-f64ff589c-58cfs to ip-10-0-130-239.ec2.internal

openshift-ingress

kube-auth-proxy-5548f7d4db-lcsdz

Scheduled

 Successfully assigned openshift-ingress/kube-auth-proxy-5548f7d4db-lcsdz to ip-10-0-135-40.ec2.internal

openshift-ingress

e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx

Scheduled

 Successfully assigned openshift-ingress/e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx to ip-10-0-130-239.ec2.internal

openshift-ingress

istiod-openshift-gateway-7574459656-5r4hf

Scheduled

 Successfully assigned openshift-ingress/istiod-openshift-gateway-7574459656-5r4hf to ip-10-0-131-71.ec2.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-7f964fb678gfmz6

Scheduled

 Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6 to ip-10-0-135-40.ec2.internal

openshift-ingress

e2e-shared-b-0ebdd9-openshift-default-7b5bd674cf-qkf4x

Scheduled

 Successfully assigned openshift-ingress/e2e-shared-b-0ebdd9-openshift-default-7b5bd674cf-qkf4x to ip-10-0-130-239.ec2.internal

openshift-ingress

e2e-mt-d72c5345-openshift-default-588974dd75-fklgw

Scheduled

 Successfully assigned openshift-ingress/e2e-mt-d72c5345-openshift-default-588974dd75-fklgw to ip-10-0-130-239.ec2.internal

openshift-ingress

router-default-87b54f4c7-4lpwt

FailedScheduling

 0/3 nodes are available: 3 node(s) had untolerated taint(s). no new claims to deallocate, preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling.

openshift-ingress

router-default-87b54f4c7-4lpwt

Scheduled

 Successfully assigned openshift-ingress/router-default-87b54f4c7-4lpwt to ip-10-0-135-40.ec2.internal

openshift-ingress

e2e-api-b-9c929f-openshift-default-d56778ccc-wk8lc

Scheduled

 Successfully assigned openshift-ingress/e2e-api-b-9c929f-openshift-default-d56778ccc-wk8lc to ip-10-0-130-239.ec2.internal

openshift-ingress

kube-auth-proxy-5548f7d4db-d8r2s

Scheduled

 Successfully assigned openshift-ingress/kube-auth-proxy-5548f7d4db-d8r2s to ip-10-0-131-71.ec2.internal

openshift-ingress

e2e-ait-5038f666-openshift-default-6599cb77dc-2hq48

Scheduled

 Successfully assigned openshift-ingress/e2e-ait-5038f666-openshift-default-6599cb77dc-2hq48 to ip-10-0-130-239.ec2.internal

openshift-ingress

payload-pre-processing-7964457696-8mnwv

Scheduled

 Successfully assigned openshift-ingress/payload-pre-processing-7964457696-8mnwv to ip-10-0-130-239.ec2.internal

openshift-ingress

istiod-openshift-gateway-94bb8fbfd-hqzt9

Scheduled

 Successfully assigned openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hqzt9 to ip-10-0-130-239.ec2.internal

openshift-ingress

e2e-api-a-ce033f-openshift-default-5f6f468755-vdjrw

Scheduled

 Successfully assigned openshift-ingress/e2e-api-a-ce033f-openshift-default-5f6f468755-vdjrw to ip-10-0-130-239.ec2.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

 Scaled up replica set router-default-87b54f4c7 from 0 to 1
(x9)

openshift-ingress

replicaset-controller

router-default-87b54f4c7

FailedCreate

 Error creating: pods "router-default-87b54f4c7-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

 There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

 Ensuring load balancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

 Ensured load balancer
(x16)

openshift-ingress

replicaset-controller

router-default-87b54f4c7

FailedCreate

 Error creating: pods "router-default-87b54f4c7-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

 Updated load balancer with new hosts

openshift-ingress

replicaset-controller

router-default-87b54f4c7

SuccessfulCreate

 Created pod: router-default-87b54f4c7-4lpwt
(x6)

openshift-ingress

kubelet

router-default-87b54f4c7-4lpwt

FailedMount

 MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x7)

openshift-ingress

kubelet

router-default-87b54f4c7-4lpwt

FailedMount

 MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt

openshift-ingress

multus

router-default-87b54f4c7-4lpwt

AddedInterface

 Add eth0 [10.132.0.9/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-87b54f4c7-4lpwt

Pulled

 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:32d6df0cb6282bc5ba571a8de322fe598afdf22e817f5a7dc92db3274dbdb9a1" already present on machine

openshift-ingress

kubelet

router-default-87b54f4c7-4lpwt

Created

 Created container: router

openshift-ingress

kubelet

router-default-87b54f4c7-4lpwt

Started

 Started container router

openshift-ingress

replicaset-controller

kube-auth-proxy-5548f7d4db

SuccessfulCreate

 Created pod: kube-auth-proxy-5548f7d4db-d8r2s

openshift-ingress

kubelet

kube-auth-proxy-5548f7d4db-lcsdz

Pulling

 Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

multus

kube-auth-proxy-5548f7d4db-lcsdz

AddedInterface

 Add eth0 [10.132.0.29/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-5548f7d4db-d8r2s

Pulling

 Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

multus

kube-auth-proxy-5548f7d4db-d8r2s

AddedInterface

 Add eth0 [10.134.0.21/23] from ovn-kubernetes

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

 Scaled up replica set kube-auth-proxy-5548f7d4db from 0 to 2

openshift-ingress

replicaset-controller

kube-auth-proxy-5548f7d4db

SuccessfulCreate

 Created pod: kube-auth-proxy-5548f7d4db-lcsdz
(x6)

openshift-ingress

replicaset-controller

kube-auth-proxy-5548f7d4db

FailedCreate

 Error creating: pods "kube-auth-proxy-5548f7d4db-" is forbidden: error looking up service account openshift-ingress/kube-auth-proxy: serviceaccount "kube-auth-proxy" not found

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

 Scaled up replica set istiod-openshift-gateway-7574459656 from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7574459656

SuccessfulCreate

 Created pod: istiod-openshift-gateway-7574459656-5r4hf

openshift-ingress

kubelet

istiod-openshift-gateway-7574459656-5r4hf

Pulling

 Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:0850242436e88f7d82f0f2126de064c7e0f09844f31d8ff0f53dc8d3908075d9"

openshift-ingress

multus

istiod-openshift-gateway-7574459656-5r4hf

AddedInterface

 Add eth0 [10.134.0.22/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-5548f7d4db-d8r2s

Pulled

 Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 3.741s (3.741s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

kube-auth-proxy-5548f7d4db-d8r2s

Started

 Started container kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-5548f7d4db-d8r2s

Created

 Created container: kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-5548f7d4db-lcsdz

Created

 Created container: kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-5548f7d4db-lcsdz

Started

 Started container kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-5548f7d4db-lcsdz

Pulled

 Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 5.591s (5.591s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7574459656-5r4hf

Pulled

 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:0850242436e88f7d82f0f2126de064c7e0f09844f31d8ff0f53dc8d3908075d9" in 3.113s (3.113s including waiting). Image size: 181034515 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7574459656-5r4hf

Created

 Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-7574459656-5r4hf

Started

 Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-7574459656-5r4hf

Unhealthy

 Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

istiod-openshift-gateway-7574459656-5r4hf

ProbeError

 Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

 Scaled up replica set data-science-gateway-data-science-gateway-class-7f964fb678 from 0 to 1

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-7f964fb678gfmz6

AddedInterface

 Add eth0 [10.132.0.32/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-7f964fb678

SuccessfulCreate

 Created pod: data-science-gateway-data-science-gateway-class-7f964fb678gfmz6

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-7f964fb678gfmz6

Pulling

 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8"

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

 Scaled up replica set istiod-openshift-gateway-94bb8fbfd from 0 to 1

openshift-ingress

multus

istiod-openshift-gateway-94bb8fbfd-hqzt9

AddedInterface

 Add eth0 [10.133.0.23/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

istiod-openshift-gateway-94bb8fbfd

SuccessfulCreate

 Created pod: istiod-openshift-gateway-94bb8fbfd-hqzt9

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-7f964fb678gfmz6

Pulled

 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" in 3.025s (3.025s including waiting). Image size: 181251556 bytes.

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-7f964fb678gfmz6

Started

 Started container istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-7f964fb678gfmz6

Created

 Created container: istio-proxy

openshift-ingress

kubelet

istiod-openshift-gateway-94bb8fbfd-hqzt9

Pulling

 Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:0850242436e88f7d82f0f2126de064c7e0f09844f31d8ff0f53dc8d3908075d9"

openshift-ingress

kubelet

istiod-openshift-gateway-94bb8fbfd-hqzt9

Created

 Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-94bb8fbfd-hqzt9

Started

 Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-94bb8fbfd-hqzt9

Pulled

 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:0850242436e88f7d82f0f2126de064c7e0f09844f31d8ff0f53dc8d3908075d9" in 2.161s (2.161s including waiting). Image size: 181034515 bytes.

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7574459656

SuccessfulDelete

 Deleted pod: istiod-openshift-gateway-7574459656-5r4hf

openshift-ingress

kubelet

istiod-openshift-gateway-7574459656-5r4hf

Killing

 Stopping container discovery

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

 Scaled down replica set istiod-openshift-gateway-7574459656 from 1 to 0
(x2)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

 failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

 invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

 invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

 failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

 invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

 failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

multus

maas-default-gateway-openshift-default-5896d4dd49-mqc82

AddedInterface

 Add eth0 [10.134.0.37/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5896d4dd49-mqc82

Pulling

 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8"

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5896d4dd49

SuccessfulCreate

 Created pod: maas-default-gateway-openshift-default-5896d4dd49-mqc82

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

 Scaled up replica set maas-default-gateway-openshift-default-5896d4dd49 from 0 to 1

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

 Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5896d4dd49-mqc82

Pulled

 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" in 3.417s (3.417s including waiting). Image size: 181251556 bytes.

openshift-ingress

kubelet

payload-processing-58f86c7798-cm8wv

Pulling

 Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:dc31b949d8c5ea8610b5c10ef53e2a199e00f0e0"
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-7964457696

FailedCreate

 Error creating: pods "payload-pre-processing-7964457696-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

kubelet

payload-pre-processing-7964457696-8mnwv

Pulling

 Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:dc31b949d8c5ea8610b5c10ef53e2a199e00f0e0"

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

 Scaled up replica set payload-processing-58f86c7798 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-58f86c7798

SuccessfulCreate

 Created pod: payload-processing-58f86c7798-cm8wv
(x5)

openshift-ingress

replicaset-controller

payload-processing-58f86c7798

FailedCreate

 Error creating: pods "payload-processing-58f86c7798-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

multus

payload-pre-processing-7964457696-8mnwv

AddedInterface

 Add eth0 [10.133.0.28/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-58f86c7798-cm8wv

AddedInterface

 Add eth0 [10.134.0.41/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

 Scaled up replica set payload-pre-processing-7964457696 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-7964457696

SuccessfulCreate

 Created pod: payload-pre-processing-7964457696-8mnwv

openshift-ingress

kubelet

payload-processing-58f86c7798-cm8wv

Pulled

 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:dc31b949d8c5ea8610b5c10ef53e2a199e00f0e0" in 1.696s (1.696s including waiting). Image size: 165432098 bytes.

openshift-ingress

kubelet

payload-processing-58f86c7798-cm8wv

Created

 Created container: payload-processing

openshift-ingress

kubelet

payload-processing-58f86c7798-cm8wv

Started

 Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-7964457696-8mnwv

Created

 Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-7964457696-8mnwv

Pulled

 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:dc31b949d8c5ea8610b5c10ef53e2a199e00f0e0" in 1.601s (1.601s including waiting). Image size: 165432098 bytes.

openshift-ingress

kubelet

payload-pre-processing-7964457696-8mnwv

Started

 Started container payload-pre-processing
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5896d4dd49-mqc82

Created

 Created container: istio-proxy
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5896d4dd49-mqc82

Started

 Started container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5896d4dd49-mqc82

Pulled

 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5896d4dd49-mqc82

ProbeError

 Startup probe error: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused body:
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5896d4dd49-mqc82

Unhealthy

 Startup probe failed: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused

openshift-ingress

gateway_labeler_controller

e2e-ait-38d19e47

AddedLabel

 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-38d19e47

openshift-ingress

kubelet

e2e-ait-38d19e47-openshift-default-8654df49cc-8tbck

Pulled

 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine

openshift-ingress

service-controller

e2e-ait-38d19e47-openshift-default

EnsuringLoadBalancer

 Ensuring load balancer

openshift-ingress

deployment-controller

e2e-ait-38d19e47-openshift-default

ScalingReplicaSet

 Scaled up replica set e2e-ait-38d19e47-openshift-default-8654df49cc from 0 to 1

openshift-ingress

replicaset-controller

e2e-ait-38d19e47-openshift-default-8654df49cc

SuccessfulCreate

 Created pod: e2e-ait-38d19e47-openshift-default-8654df49cc-8tbck

openshift-ingress

kubelet

e2e-ait-38d19e47-openshift-default-8654df49cc-8tbck

Started

 Started container istio-proxy

openshift-ingress

kubelet

e2e-ait-38d19e47-openshift-default-8654df49cc-8tbck

Created

 Created container: istio-proxy

openshift-ingress

multus

e2e-ait-38d19e47-openshift-default-8654df49cc-8tbck

AddedInterface

 Add eth0 [10.134.0.46/23] from ovn-kubernetes

openshift-ingress

service-controller

e2e-ait-38d19e47-openshift-default

EnsuredLoadBalancer

 Ensured load balancer

openshift-ingress

service-controller

e2e-ait-38d19e47-openshift-default

DeletingLoadBalancer

 Deleting load balancer

openshift-ingress

kubelet

e2e-ait-38d19e47-openshift-default-8654df49cc-8tbck

Killing

 Stopping container istio-proxy

openshift-ingress

service-controller

e2e-ait-38d19e47-openshift-default

DeletedLoadBalancer

 Deleted load balancer

openshift-ingress

multus

e2e-ait-5038f666-openshift-default-6599cb77dc-2hq48

AddedInterface

 Add eth0 [10.133.0.33/23] from ovn-kubernetes

openshift-ingress

service-controller

e2e-ait-5038f666-openshift-default

EnsuringLoadBalancer

 Ensuring load balancer

openshift-ingress

kubelet

e2e-ait-5038f666-openshift-default-6599cb77dc-2hq48

Pulling

 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8"

openshift-ingress

gateway_labeler_controller

e2e-ait-5038f666

AddedLabel

 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-5038f666

openshift-ingress

replicaset-controller

e2e-ait-5038f666-openshift-default-6599cb77dc

SuccessfulCreate

 Created pod: e2e-ait-5038f666-openshift-default-6599cb77dc-2hq48

openshift-ingress

deployment-controller

e2e-ait-5038f666-openshift-default

ScalingReplicaSet

 Scaled up replica set e2e-ait-5038f666-openshift-default-6599cb77dc from 0 to 1

openshift-ingress

kubelet

e2e-ait-5038f666-openshift-default-6599cb77dc-2hq48

Pulled

 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" in 2.794s (2.794s including waiting). Image size: 181251556 bytes.

openshift-ingress

service-controller

e2e-ait-5038f666-openshift-default

EnsuredLoadBalancer

 Ensured load balancer
(x2)

openshift-ingress

kubelet

e2e-ait-5038f666-openshift-default-6599cb77dc-2hq48

Started

 Started container istio-proxy

openshift-ingress

kubelet

e2e-ait-5038f666-openshift-default-6599cb77dc-2hq48

Pulled

 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ait-5038f666-openshift-default-6599cb77dc-2hq48

Created

 Created container: istio-proxy

openshift-ingress

kubelet

e2e-ait-5038f666-openshift-default-6599cb77dc-2hq48

Unhealthy

 Startup probe failed: Get "http://10.133.0.33:15021/healthz/ready": dial tcp 10.133.0.33:15021: connect: connection refused

openshift-ingress

kubelet

e2e-ait-5038f666-openshift-default-6599cb77dc-2hq48

Killing

 Stopping container istio-proxy

openshift-ingress

kubelet

e2e-ait-5038f666-openshift-default-6599cb77dc-2hq48

ProbeError

 Startup probe error: Get "http://10.133.0.33:15021/healthz/ready": dial tcp 10.133.0.33:15021: connect: connection refused body:

openshift-ingress

service-controller

e2e-ait-5038f666-openshift-default

DeletingLoadBalancer

 Deleting load balancer

openshift-ingress

service-controller

e2e-ait-5038f666-openshift-default

DeletedLoadBalancer

 Deleted load balancer

openshift-ingress

service-controller

e2e-derive-8b33da08-openshift-default

EnsuringLoadBalancer

 Ensuring load balancer

openshift-ingress

gateway_labeler_controller

e2e-derive-8b33da08

AddedLabel

 Added label istio.io/rev=openshift-gateway to gateway e2e-derive-8b33da08

openshift-ingress

replicaset-controller

e2e-derive-8b33da08-openshift-default-6f7876d755

SuccessfulCreate

 Created pod: e2e-derive-8b33da08-openshift-default-6f7876d755-xhwlz

openshift-ingress

deployment-controller

e2e-derive-8b33da08-openshift-default

ScalingReplicaSet

 Scaled up replica set e2e-derive-8b33da08-openshift-default-6f7876d755 from 0 to 1

openshift-ingress

kubelet

e2e-derive-8b33da08-openshift-default-6f7876d755-xhwlz

Pulled

 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine

openshift-ingress

multus

e2e-derive-8b33da08-openshift-default-6f7876d755-xhwlz

AddedInterface

 Add eth0 [10.134.0.47/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-derive-8b33da08-openshift-default-6f7876d755-xhwlz

Created

 Created container: istio-proxy

openshift-ingress

kubelet

e2e-derive-8b33da08-openshift-default-6f7876d755-xhwlz

Started

 Started container istio-proxy

openshift-ingress

service-controller

e2e-derive-8b33da08-openshift-default

EnsuredLoadBalancer

 Ensured load balancer

openshift-ingress

service-controller

e2e-derive-8b33da08-openshift-default

DeletingLoadBalancer

 Deleting load balancer

openshift-ingress

kubelet

e2e-derive-8b33da08-openshift-default-6f7876d755-xhwlz

Killing

 Stopping container istio-proxy

openshift-ingress

service-controller

e2e-derive-8b33da08-openshift-default

DeletedLoadBalancer

 Deleted load balancer

openshift-ingress

deployment-controller

e2e-mt-d72c5345-openshift-default

ScalingReplicaSet

 Scaled up replica set e2e-mt-d72c5345-openshift-default-588974dd75 from 0 to 1

openshift-ingress

multus

e2e-mt-d72c5345-openshift-default-588974dd75-fklgw

AddedInterface

 Add eth0 [10.133.0.36/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-mt-d72c5345-openshift-default-588974dd75

SuccessfulCreate

 Created pod: e2e-mt-d72c5345-openshift-default-588974dd75-fklgw

openshift-ingress

gateway_labeler_controller

e2e-mt-d72c5345

AddedLabel

 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-d72c5345
(x2)

openshift-ingress

kubelet

e2e-mt-d72c5345-openshift-default-588974dd75-fklgw

Pulled

 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
(x2)

openshift-ingress

kubelet

e2e-mt-d72c5345-openshift-default-588974dd75-fklgw

Created

 Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-d72c5345-openshift-default-588974dd75-fklgw

Started

 Started container istio-proxy
(x4)

openshift-ingress

kubelet

e2e-mt-d72c5345-openshift-default-588974dd75-fklgw

Unhealthy

 Startup probe failed: Get "http://10.133.0.36:15021/healthz/ready": dial tcp 10.133.0.36:15021: connect: connection refused
(x4)

openshift-ingress

kubelet

e2e-mt-d72c5345-openshift-default-588974dd75-fklgw

ProbeError

 Startup probe error: Get "http://10.133.0.36:15021/healthz/ready": dial tcp 10.133.0.36:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-mt-d72c5345-openshift-default-588974dd75-fklgw

Killing

 Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-mt-8bfb9318-openshift-default-5884775ff

SuccessfulCreate

 Created pod: e2e-mt-8bfb9318-openshift-default-5884775ff-vwnl9

openshift-ingress

gateway_labeler_controller

e2e-mt-8bfb9318

AddedLabel

 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-8bfb9318

openshift-ingress

deployment-controller

e2e-mt-8bfb9318-openshift-default

ScalingReplicaSet

 Scaled up replica set e2e-mt-8bfb9318-openshift-default-5884775ff from 0 to 1

openshift-ingress

GatewayAuthBootstrap

e2e-mt-8bfb9318

ReconcileError

 Failed to reconcile EnvoyFilter: failed to create EnvoyFilter: could not CREATE EnvoyFilter openshift-ingress/e2e-mt-8bfb9318-authn-ssl: envoyfilters.networking.istio.io "e2e-mt-8bfb9318-authn-ssl" already exists

openshift-ingress

multus

e2e-mt-8bfb9318-openshift-default-5884775ff-vwnl9

AddedInterface

 Add eth0 [10.133.0.38/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-mt-8bfb9318-openshift-default-5884775ff-vwnl9

Created

 Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-8bfb9318-openshift-default-5884775ff-vwnl9

Started

 Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-8bfb9318-openshift-default-5884775ff-vwnl9

Pulled

 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
(x5)

openshift-ingress

kubelet

e2e-mt-8bfb9318-openshift-default-5884775ff-vwnl9

Unhealthy

 Startup probe failed: Get "http://10.133.0.38:15021/healthz/ready": dial tcp 10.133.0.38:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-8bfb9318-openshift-default-5884775ff-vwnl9

ProbeError

 Startup probe error: Get "http://10.133.0.38:15021/healthz/ready": dial tcp 10.133.0.38:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-mt-8bfb9318-openshift-default-5884775ff-vwnl9

Killing

 Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-inf-a-cdb744-openshift-default

ScalingReplicaSet

 Scaled up replica set e2e-inf-a-cdb744-openshift-default-f64ff589c from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-inf-a-cdb744

AddedLabel

 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-cdb744

openshift-ingress

replicaset-controller

e2e-inf-a-cdb744-openshift-default-f64ff589c

SuccessfulCreate

 Created pod: e2e-inf-a-cdb744-openshift-default-f64ff589c-58cfs

openshift-ingress

multus

e2e-inf-a-cdb744-openshift-default-f64ff589c-58cfs

AddedInterface

 Add eth0 [10.133.0.41/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-inf-b-ebaa5d

AddedLabel

 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-ebaa5d

openshift-ingress

deployment-controller

e2e-inf-b-ebaa5d-openshift-default

ScalingReplicaSet

 Scaled up replica set e2e-inf-b-ebaa5d-openshift-default-67b687fb47 from 0 to 1

openshift-ingress

replicaset-controller

e2e-inf-b-ebaa5d-openshift-default-67b687fb47

SuccessfulCreate

 Created pod: e2e-inf-b-ebaa5d-openshift-default-67b687fb47-smjfs

openshift-ingress

multus

e2e-inf-b-ebaa5d-openshift-default-67b687fb47-smjfs

AddedInterface

 Add eth0 [10.133.0.43/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-inf-a-cdb744-openshift-default-f64ff589c-58cfs

Unhealthy

 Readiness probe failed: Get "http://10.133.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-inf-a-cdb744-openshift-default-f64ff589c-58cfs

ProbeError

 Readiness probe error: Get "http://10.133.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-inf-a-cdb744-openshift-default-f64ff589c-58cfs

Created

 Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-a-cdb744-openshift-default-f64ff589c-58cfs

Started

 Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-a-cdb744-openshift-default-f64ff589c-58cfs

Pulled

 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
(x4)

openshift-ingress

kubelet

e2e-inf-a-cdb744-openshift-default-f64ff589c-58cfs

ProbeError

 Startup probe error: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused body:
(x4)

openshift-ingress

kubelet

e2e-inf-a-cdb744-openshift-default-f64ff589c-58cfs

Unhealthy

 Startup probe failed: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-inf-b-ebaa5d-openshift-default-67b687fb47-smjfs

Pulled

 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
(x2)

openshift-ingress

kubelet

e2e-inf-b-ebaa5d-openshift-default-67b687fb47-smjfs

Created

 Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-b-ebaa5d-openshift-default-67b687fb47-smjfs

Started

 Started container istio-proxy
(x4)

openshift-ingress

kubelet

e2e-inf-b-ebaa5d-openshift-default-67b687fb47-smjfs

ProbeError

 Startup probe error: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused body:
(x4)

openshift-ingress

kubelet

e2e-inf-b-ebaa5d-openshift-default-67b687fb47-smjfs

Unhealthy

 Startup probe failed: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused

openshift-ingress

kubelet

e2e-inf-a-cdb744-openshift-default-f64ff589c-58cfs

Killing

 Stopping container istio-proxy

openshift-ingress

kubelet

e2e-inf-a-cdb744-openshift-default-f64ff589c-58cfs

Unhealthy

 Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-inf-a-cdb744-openshift-default-f64ff589c-58cfs

ProbeError

 Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-inf-b-ebaa5d-openshift-default-67b687fb47-smjfs

Killing

 Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-api-a-ce033f-openshift-default-5f6f468755

SuccessfulCreate

 Created pod: e2e-api-a-ce033f-openshift-default-5f6f468755-vdjrw

openshift-ingress

deployment-controller

e2e-api-a-ce033f-openshift-default

ScalingReplicaSet

 Scaled up replica set e2e-api-a-ce033f-openshift-default-5f6f468755 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-api-a-ce033f

AddedLabel

 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-ce033f

openshift-ingress

multus

e2e-api-a-ce033f-openshift-default-5f6f468755-vdjrw

AddedInterface

 Add eth0 [10.133.0.45/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-api-b-9c929f-openshift-default-d56778ccc

SuccessfulCreate

 Created pod: e2e-api-b-9c929f-openshift-default-d56778ccc-wk8lc

openshift-ingress

deployment-controller

e2e-api-b-9c929f-openshift-default

ScalingReplicaSet

 Scaled up replica set e2e-api-b-9c929f-openshift-default-d56778ccc from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-api-b-9c929f

AddedLabel

 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-9c929f

openshift-ingress

multus

e2e-api-b-9c929f-openshift-default-d56778ccc-wk8lc

AddedInterface

 Add eth0 [10.133.0.47/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-api-a-ce033f-openshift-default-5f6f468755-vdjrw

Unhealthy

 Readiness probe failed: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused

openshift-ingress

kubelet

e2e-api-a-ce033f-openshift-default-5f6f468755-vdjrw

ProbeError

 Readiness probe error: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-api-a-ce033f-openshift-default-5f6f468755-vdjrw

Started

 Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-a-ce033f-openshift-default-5f6f468755-vdjrw

Created

 Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-a-ce033f-openshift-default-5f6f468755-vdjrw

Pulled

 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
(x4)

openshift-ingress

kubelet

e2e-api-a-ce033f-openshift-default-5f6f468755-vdjrw

ProbeError

 Startup probe error: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused body:
(x4)

openshift-ingress

kubelet

e2e-api-a-ce033f-openshift-default-5f6f468755-vdjrw

Unhealthy

 Startup probe failed: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-api-b-9c929f-openshift-default-d56778ccc-wk8lc

Started

 Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-b-9c929f-openshift-default-d56778ccc-wk8lc

Created

 Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-b-9c929f-openshift-default-d56778ccc-wk8lc

Pulled

 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
(x5)

openshift-ingress

kubelet

e2e-api-b-9c929f-openshift-default-d56778ccc-wk8lc

ProbeError

 Startup probe error: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-api-b-9c929f-openshift-default-d56778ccc-wk8lc

Unhealthy

 Startup probe failed: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused

openshift-ingress

kubelet

e2e-api-a-ce033f-openshift-default-5f6f468755-vdjrw

Killing

 Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-shared-a-2cd1eb-openshift-default

ScalingReplicaSet

 Scaled up replica set e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-shared-a-2cd1eb

AddedLabel

 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-2cd1eb

openshift-ingress

replicaset-controller

e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7

SuccessfulCreate

 Created pod: e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx

openshift-ingress

kubelet

e2e-api-b-9c929f-openshift-default-d56778ccc-wk8lc

Killing

 Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx

FailedCreatePodSandBox

 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx_openshift-ingress_dfe27f9c-0f47-4c29-ac62-6da6d02f94a4_0(a1181b4051817d6775e9073134282333ebfd6ce1eb60824192d88f4521f97424): error adding pod openshift-ingress_e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"a1181b4051817d6775e9073134282333ebfd6ce1eb60824192d88f4521f97424" Netns:"/var/run/netns/cb27f5cc-e169-4c2d-89f5-9d214f87a886" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=openshift-ingress;K8S_POD_NAME=e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx;K8S_POD_INFRA_CONTAINER_ID=a1181b4051817d6775e9073134282333ebfd6ce1eb60824192d88f4521f97424;K8S_POD_UID=dfe27f9c-0f47-4c29-ac62-6da6d02f94a4" Path:"" ERRORED: error configuring pod [openshift-ingress/e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx] networking: Multus: [openshift-ingress/e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx/dfe27f9c-0f47-4c29-ac62-6da6d02f94a4]: error waiting for pod: Get "https://a63e3a650d7d6496d8a9d7a34922820d-ee5eddcae3b91cb3.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/openshift-ingress/pods/e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx?timeout=1m0s": context deadline exceeded ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}

openshift-ingress

kubelet

e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx

FailedCreatePodSandBox

 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx_openshift-ingress_dfe27f9c-0f47-4c29-ac62-6da6d02f94a4_0(6d48e99f551bc43a0d9f24ff59b7269a3f838086beb432c71669cd7527de300d): error adding pod openshift-ingress_e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"6d48e99f551bc43a0d9f24ff59b7269a3f838086beb432c71669cd7527de300d" Netns:"/var/run/netns/7449f5b9-53db-43cb-a2ac-99bf12862ba5" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=openshift-ingress;K8S_POD_NAME=e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx;K8S_POD_INFRA_CONTAINER_ID=6d48e99f551bc43a0d9f24ff59b7269a3f838086beb432c71669cd7527de300d;K8S_POD_UID=dfe27f9c-0f47-4c29-ac62-6da6d02f94a4" Path:"" ERRORED: error configuring pod [openshift-ingress/e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx] networking: Multus: [openshift-ingress/e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx/dfe27f9c-0f47-4c29-ac62-6da6d02f94a4]: error waiting for pod: Get "https://a63e3a650d7d6496d8a9d7a34922820d-ee5eddcae3b91cb3.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/openshift-ingress/pods/e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx?timeout=1m0s": context deadline exceeded ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}

openshift-ingress

kubelet

e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx

FailedCreatePodSandBox

 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx_openshift-ingress_dfe27f9c-0f47-4c29-ac62-6da6d02f94a4_0(1230ac767c904f889235966d200c8ff2287a4dcefd026d508f3e2aa5d5740702): error adding pod openshift-ingress_e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"1230ac767c904f889235966d200c8ff2287a4dcefd026d508f3e2aa5d5740702" Netns:"/var/run/netns/e7e861a7-2208-45a1-9461-ad413a8e915b" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=openshift-ingress;K8S_POD_NAME=e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx;K8S_POD_INFRA_CONTAINER_ID=1230ac767c904f889235966d200c8ff2287a4dcefd026d508f3e2aa5d5740702;K8S_POD_UID=dfe27f9c-0f47-4c29-ac62-6da6d02f94a4" Path:"" ERRORED: error configuring pod [openshift-ingress/e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx] networking: Multus: [openshift-ingress/e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx/dfe27f9c-0f47-4c29-ac62-6da6d02f94a4]: error waiting for pod: Get "https://a63e3a650d7d6496d8a9d7a34922820d-ee5eddcae3b91cb3.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/openshift-ingress/pods/e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx?timeout=1m0s": context deadline exceeded ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}

openshift-ingress

kubelet

e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx

FailedCreatePodSandBox

 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx_openshift-ingress_dfe27f9c-0f47-4c29-ac62-6da6d02f94a4_0(6d4eacd1b888f3a978afa690b50619ad8c8eb164420f84a078835cc9152e28ab): error adding pod openshift-ingress_e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"6d4eacd1b888f3a978afa690b50619ad8c8eb164420f84a078835cc9152e28ab" Netns:"/var/run/netns/afc8ba65-227a-4f38-8d71-ed60df7f9d16" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=openshift-ingress;K8S_POD_NAME=e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx;K8S_POD_INFRA_CONTAINER_ID=6d4eacd1b888f3a978afa690b50619ad8c8eb164420f84a078835cc9152e28ab;K8S_POD_UID=dfe27f9c-0f47-4c29-ac62-6da6d02f94a4" Path:"" ERRORED: error configuring pod [openshift-ingress/e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx] networking: Multus: [openshift-ingress/e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx/dfe27f9c-0f47-4c29-ac62-6da6d02f94a4]: error waiting for pod: Get "https://a63e3a650d7d6496d8a9d7a34922820d-ee5eddcae3b91cb3.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/openshift-ingress/pods/e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx?timeout=1m0s": context deadline exceeded ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}

openshift-ingress

multus

e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx

AddedInterface

 Add eth0 [10.133.0.49/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-shared-b-0ebdd9-openshift-default-7b5bd674cf

SuccessfulCreate

 Created pod: e2e-shared-b-0ebdd9-openshift-default-7b5bd674cf-qkf4x

openshift-ingress

deployment-controller

e2e-shared-b-0ebdd9-openshift-default

ScalingReplicaSet

 Scaled up replica set e2e-shared-b-0ebdd9-openshift-default-7b5bd674cf from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-shared-b-0ebdd9

AddedLabel

 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-0ebdd9

openshift-ingress

multus

e2e-shared-b-0ebdd9-openshift-default-7b5bd674cf-qkf4x

AddedInterface

 Add eth0 [10.133.0.51/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx

Started

 Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx

Created

 Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx

Pulled

 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
(x4)

openshift-ingress

kubelet

e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx

Unhealthy

 Startup probe failed: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused
(x4)

openshift-ingress

kubelet

e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx

ProbeError

 Startup probe error: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-shared-b-0ebdd9-openshift-default-7b5bd674cf-qkf4x

ProbeError

 Readiness probe error: Get "http://10.133.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-shared-b-0ebdd9-openshift-default-7b5bd674cf-qkf4x

Unhealthy

 Readiness probe failed: Get "http://10.133.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-shared-b-0ebdd9-openshift-default-7b5bd674cf-qkf4x

Pulled

 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-b-0ebdd9-openshift-default-7b5bd674cf-qkf4x

Created

 Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-0ebdd9-openshift-default-7b5bd674cf-qkf4x

Started

 Started container istio-proxy
(x4)

openshift-ingress

kubelet

e2e-shared-b-0ebdd9-openshift-default-7b5bd674cf-qkf4x

ProbeError

 Startup probe error: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused body:
(x4)

openshift-ingress

kubelet

e2e-shared-b-0ebdd9-openshift-default-7b5bd674cf-qkf4x

Unhealthy

 Startup probe failed: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused

openshift-ingress

kubelet

e2e-shared-a-2cd1eb-openshift-default-6bc4c5c9d7-7pxmx

Killing

 Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-0ebdd9-openshift-default-7b5bd674cf-qkf4x

Killing

 Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-0ebdd9-openshift-default-7b5bd674cf-qkf4x

ProbeError

 Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-shared-b-0ebdd9-openshift-default-7b5bd674cf-qkf4x

Unhealthy

 Readiness probe failed: HTTP probe failed with statuscode: 503