=== events in opendatahub (2026-10-29T23:17:55Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 26m Normal Scheduled pod/ai-gateway-controller-d74798bcb-jpzf4 Successfully assigned opendatahub/ai-gateway-controller-d74798bcb-jpzf4 to ip-10-0-136-187.ec2.internal 31m Normal Scheduled pod/kserve-controller-manager-776cc66d47-65h2w Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-65h2w to ip-10-0-139-130.ec2.internal 28m Normal Scheduled pod/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp Successfully assigned opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp to ip-10-0-136-228.ec2.internal 19m Normal Scheduled pod/test-curl-9038-bb78e1 Successfully assigned opendatahub/test-curl-9038-bb78e1 to ip-10-0-136-228.ec2.internal 19m Normal Scheduled pod/test-curl-9038-1e1933 Successfully assigned opendatahub/test-curl-9038-1e1933 to ip-10-0-136-228.ec2.internal 14m Normal Scheduled pod/maas-controller-848f956548-dgrbz Successfully assigned opendatahub/maas-controller-848f956548-dgrbz to ip-10-0-139-130.ec2.internal 27m Normal Scheduled pod/llmisvc-controller-manager-6466c554d8-ngv8z Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-ngv8z to ip-10-0-136-228.ec2.internal 32m Normal Scheduled pod/kserve-module-controller-manager-5cfb77bf8c-ldmzq Successfully assigned opendatahub/kserve-module-controller-manager-5cfb77bf8c-ldmzq to ip-10-0-139-130.ec2.internal 32m Normal Scheduled pod/opendatahub-operator-controller-manager-675b698459-nn6hx Successfully assigned opendatahub/opendatahub-operator-controller-manager-675b698459-nn6hx to ip-10-0-139-130.ec2.internal 31m Normal Scheduled pod/model-serving-api-75bd89fdfc-tnntd Successfully assigned opendatahub/model-serving-api-75bd89fdfc-tnntd to ip-10-0-136-228.ec2.internal 26m Normal Scheduled pod/ai-gateway-controller-7d6c696b49-4wcv6 Successfully assigned opendatahub/ai-gateway-controller-7d6c696b49-4wcv6 to ip-10-0-136-187.ec2.internal 28m Normal Scheduled pod/maas-controller-848f956548-92rkf Successfully assigned opendatahub/maas-controller-848f956548-92rkf to ip-10-0-136-228.ec2.internal 28m Normal Scheduled pod/maas-controller-577f596567-rktf7 Successfully assigned opendatahub/maas-controller-577f596567-rktf7 to ip-10-0-136-228.ec2.internal 31m Normal Scheduled pod/llmisvc-controller-manager-869d5dcc5d-tqwms Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-tqwms to ip-10-0-136-228.ec2.internal 32m Normal Scheduled pod/opendatahub-operator-controller-manager-675b698459-wwm4m Successfully assigned opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m to ip-10-0-136-228.ec2.internal 32m Normal Scheduled pod/opendatahub-operator-controller-manager-675b698459-ctkl2 Successfully assigned opendatahub/opendatahub-operator-controller-manager-675b698459-ctkl2 to ip-10-0-136-187.ec2.internal 15m Normal Scheduled pod/maas-controller-848f956548-9fnqq Successfully assigned opendatahub/maas-controller-848f956548-9fnqq to ip-10-0-139-130.ec2.internal 31m Normal Scheduled pod/odh-model-controller-f87fcf56c-nhwrp Successfully assigned opendatahub/odh-model-controller-f87fcf56c-nhwrp to ip-10-0-139-130.ec2.internal 32m Normal Scheduled pod/odh-observability-584f99c788-n78vt Successfully assigned opendatahub/odh-observability-584f99c788-n78vt to ip-10-0-136-187.ec2.internal 32m Normal RequirementsUnknown clusterserviceversion/opendatahub-operator.v3.6.0-ea.2 requirements not yet checked 32m Normal RequirementsNotMet clusterserviceversion/opendatahub-operator.v3.6.0-ea.2 one or more requirements couldn't be found 32m Normal AllRequirementsMet clusterserviceversion/opendatahub-operator.v3.6.0-ea.2 all requirements found, attempting install 32m Normal SuccessfulCreate replicaset/opendatahub-operator-controller-manager-675b698459 Created pod: opendatahub-operator-controller-manager-675b698459-wwm4m 32m Normal SuccessfulCreate replicaset/opendatahub-operator-controller-manager-675b698459 Created pod: opendatahub-operator-controller-manager-675b698459-nn6hx 32m Normal SuccessfulCreate replicaset/opendatahub-operator-controller-manager-675b698459 Created pod: opendatahub-operator-controller-manager-675b698459-ctkl2 32m Normal ScalingReplicaSet deployment/opendatahub-operator-controller-manager Scaled up replica set opendatahub-operator-controller-manager-675b698459 from 0 to 3 32m Normal AddedInterface pod/opendatahub-operator-controller-manager-675b698459-ctkl2 Add eth0 [10.133.0.15/23] from ovn-kubernetes 32m Normal AddedInterface pod/opendatahub-operator-controller-manager-675b698459-wwm4m Add eth0 [10.132.0.47/23] from ovn-kubernetes 32m Normal Pulling pod/opendatahub-operator-controller-manager-675b698459-wwm4m Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" 32m Normal Pulling pod/opendatahub-operator-controller-manager-675b698459-ctkl2 Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" 32m Normal AddedInterface pod/opendatahub-operator-controller-manager-675b698459-nn6hx Add eth0 [10.134.0.11/23] from ovn-kubernetes 32m Normal Pulling pod/opendatahub-operator-controller-manager-675b698459-nn6hx Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" 32m Warning AppliedWithWarnings installplan/install-5cp64 1 warning(s) generated during installation of operator "opendatahub-operator.v3.6.0-ea.2" (CustomResourceDefinition "hardwareprofiles.infrastructure.opendatahub.io"): infrastructure.opendatahub.io/v1alpha1 is deprecated; please use infrastructure.opendatahub.io/v1 32m Normal InstallSucceeded clusterserviceversion/opendatahub-operator.v3.6.0-ea.2 waiting for install components to report healthy 32m Normal InstallWaiting clusterserviceversion/opendatahub-operator.v3.6.0-ea.2 Webhook install failed: conversionWebhook not ready 32m Normal InstallWaiting clusterserviceversion/opendatahub-operator.v3.6.0-ea.2 installing: waiting for deployment opendatahub-operator-controller-manager to become ready: deployment "opendatahub-operator-controller-manager" not available: Deployment does not have minimum availability. 32m Normal Started pod/opendatahub-operator-controller-manager-675b698459-nn6hx Container started 32m Normal Pulled pod/opendatahub-operator-controller-manager-675b698459-nn6hx Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 2.797s (2.797s including waiting). Image size: 273709574 bytes. 32m Normal Started pod/opendatahub-operator-controller-manager-675b698459-wwm4m Container started 32m Normal Created pod/opendatahub-operator-controller-manager-675b698459-wwm4m Container created 32m Normal Pulled pod/opendatahub-operator-controller-manager-675b698459-wwm4m Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 2.823s (2.823s including waiting). Image size: 273709574 bytes. 32m Normal Pulled pod/opendatahub-operator-controller-manager-675b698459-ctkl2 Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 2.799s (2.799s including waiting). Image size: 273709574 bytes. 32m Normal Created pod/opendatahub-operator-controller-manager-675b698459-ctkl2 Container created 32m Normal Started pod/opendatahub-operator-controller-manager-675b698459-ctkl2 Container started 32m Normal Created pod/opendatahub-operator-controller-manager-675b698459-nn6hx Container created 32m Normal Created pod/opendatahub-operator-controller-manager-675b698459-nn6hx Container created 32m Normal Pulling pod/opendatahub-operator-controller-manager-675b698459-wwm4m Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" 32m Normal Pulling pod/opendatahub-operator-controller-manager-675b698459-nn6hx Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" 32m Normal Pulled pod/opendatahub-operator-controller-manager-675b698459-nn6hx Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 435ms (435ms including waiting). Image size: 273709574 bytes. 32m Normal Pulling pod/opendatahub-operator-controller-manager-675b698459-ctkl2 Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" 32m Normal Pulled pod/opendatahub-operator-controller-manager-675b698459-ctkl2 Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 371ms (371ms including waiting). Image size: 273709574 bytes. 32m Normal Created pod/opendatahub-operator-controller-manager-675b698459-ctkl2 Container created 32m Normal LeaderElection lease/07ed84f7.opendatahub.io opendatahub-operator-controller-manager-675b698459-ctkl2_f1cb0914-55ac-4138-9b97-e9fa9910ad68 became leader 32m Normal Started pod/opendatahub-operator-controller-manager-675b698459-wwm4m Container started 32m Normal Started pod/opendatahub-operator-controller-manager-675b698459-ctkl2 Container started 32m Normal Created pod/opendatahub-operator-controller-manager-675b698459-wwm4m Container created 32m Normal Pulled pod/opendatahub-operator-controller-manager-675b698459-wwm4m Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 317ms (317ms including waiting). Image size: 273709574 bytes. 32m Normal Started pod/opendatahub-operator-controller-manager-675b698459-nn6hx Container started 32m Normal InstallSucceeded clusterserviceversion/opendatahub-operator.v3.6.0-ea.2 install strategy completed with no errors 32m Normal SuccessfulCreate replicaset/odh-observability-584f99c788 Created pod: odh-observability-584f99c788-n78vt 32m Normal WaitingForApproval certificaterequest/odh-observability-webhook-cert-1 Not signing CertificateRequest until it is Approved 32m Normal ScalingReplicaSet deployment/odh-observability Scaled up replica set odh-observability-584f99c788 from 0 to 1 32m Warning FailedMount pod/odh-observability-584f99c788-n78vt MountVolume.SetUp failed for volume "webhook-certs" : secret "odh-observability-webhook-cert" not found 32m Normal Issuing certificate/odh-observability-webhook-cert The certificate has been successfully issued 32m Normal Requested certificate/odh-observability-webhook-cert Created new CertificateRequest resource "odh-observability-webhook-cert-1" 32m Normal Generated certificate/odh-observability-webhook-cert Stored new private key in temporary Secret resource "odh-observability-webhook-cert-2gshl" 32m Normal Issuing certificate/odh-observability-webhook-cert Issuing certificate as Secret does not exist 32m Normal CertificateIssued certificaterequest/odh-observability-webhook-cert-1 Certificate fetched from issuer successfully 32m Warning BadConfig certificaterequest/odh-observability-webhook-cert-1 Certificate will be issued with an empty Issuer DN, which contravenes RFC 5280 and could break some strict clients 32m Normal cert-manager.io certificaterequest/odh-observability-webhook-cert-1 Certificate request has been approved by cert-manager.io 32m Normal WaitingForApproval certificaterequest/odh-observability-webhook-cert-1 Not signing CertificateRequest until it is Approved 32m Normal WaitingForApproval certificaterequest/odh-observability-webhook-cert-1 Not signing CertificateRequest until it is Approved 32m Normal WaitingForApproval certificaterequest/odh-observability-webhook-cert-1 Not signing CertificateRequest until it is Approved 32m Normal WaitingForApproval certificaterequest/odh-observability-webhook-cert-1 Not signing CertificateRequest until it is Approved 32m Normal Pulling pod/odh-observability-584f99c788-n78vt Pulling image "quay.io/opendatahub/odh-observability@sha256:03689e71fb8879ea696366b83d2a4727eb76e696baf8c2be2b18121c0534db4e" 32m Normal AddedInterface pod/odh-observability-584f99c788-n78vt Add eth0 [10.133.0.18/23] from ovn-kubernetes 32m Normal Pulled pod/odh-observability-584f99c788-n78vt Successfully pulled image "quay.io/opendatahub/odh-observability@sha256:03689e71fb8879ea696366b83d2a4727eb76e696baf8c2be2b18121c0534db4e" in 2.238s (2.238s including waiting). Image size: 154660917 bytes. 32m Normal Started pod/odh-observability-584f99c788-n78vt Container started 32m Normal Created pod/odh-observability-584f99c788-n78vt Container created 32m Normal ScalingReplicaSet deployment/kserve-module-controller-manager Scaled up replica set kserve-module-controller-manager-5cfb77bf8c from 0 to 1 32m Normal SuccessfulCreate replicaset/kserve-module-controller-manager-5cfb77bf8c Created pod: kserve-module-controller-manager-5cfb77bf8c-ldmzq 32m Normal Pulling pod/kserve-module-controller-manager-5cfb77bf8c-ldmzq Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" 32m Normal AddedInterface pod/kserve-module-controller-manager-5cfb77bf8c-ldmzq Add eth0 [10.134.0.15/23] from ovn-kubernetes 32m Normal Created pod/kserve-module-controller-manager-5cfb77bf8c-ldmzq Container created 32m Normal Pulled pod/kserve-module-controller-manager-5cfb77bf8c-ldmzq Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 1.427s (1.427s including waiting). Image size: 195729000 bytes. 32m Normal Started pod/kserve-module-controller-manager-5cfb77bf8c-ldmzq Container started 32m Normal LeaderElection lease/kserve-module-controller-manager kserve-module-controller-manager-5cfb77bf8c-ldmzq_efa47dec-5efa-44ac-9140-1cbcdfa4198f became leader 31m Normal AddedInterface pod/llmisvc-controller-manager-869d5dcc5d-tqwms Add eth0 [10.132.0.49/23] from ovn-kubernetes 31m Normal ScalingReplicaSet deployment/odh-model-controller Scaled up replica set odh-model-controller-f87fcf56c from 0 to 1 31m Normal AddedInterface pod/kserve-controller-manager-776cc66d47-65h2w Add eth0 [10.134.0.16/23] from ovn-kubernetes 31m Normal Pulling pod/kserve-controller-manager-776cc66d47-65h2w Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" 31m Normal SuccessfulCreate replicaset/kserve-controller-manager-776cc66d47 Created pod: kserve-controller-manager-776cc66d47-65h2w 31m Normal ScalingReplicaSet deployment/kserve-controller-manager Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1 31m Normal Pulling pod/llmisvc-controller-manager-869d5dcc5d-tqwms Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 31m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-869d5dcc5d Created pod: llmisvc-controller-manager-869d5dcc5d-tqwms 31m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1 31m Normal Pulling pod/model-serving-api-75bd89fdfc-tnntd Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" 31m Normal SuccessfulCreate replicaset/odh-model-controller-f87fcf56c Created pod: odh-model-controller-f87fcf56c-nhwrp 31m Normal Pulling pod/odh-model-controller-f87fcf56c-nhwrp Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" 31m Normal AddedInterface pod/odh-model-controller-f87fcf56c-nhwrp Add eth0 [10.134.0.17/23] from ovn-kubernetes 31m Normal ScalingReplicaSet deployment/model-serving-api Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1 31m Normal SuccessfulCreate replicaset/model-serving-api-75bd89fdfc Created pod: model-serving-api-75bd89fdfc-tnntd 31m Normal AddedInterface pod/model-serving-api-75bd89fdfc-tnntd Add eth0 [10.132.0.50/23] from ovn-kubernetes 31m Normal Created pod/model-serving-api-75bd89fdfc-tnntd Container created 31m Normal Pulled pod/model-serving-api-75bd89fdfc-tnntd Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 1.873s (1.873s including waiting). Image size: 184925761 bytes. 31m Normal Started pod/model-serving-api-75bd89fdfc-tnntd Container started 31m Normal Started pod/llmisvc-controller-manager-869d5dcc5d-tqwms Container started 31m Normal Pulled pod/odh-model-controller-f87fcf56c-nhwrp Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 2.549s (2.549s including waiting). Image size: 236755567 bytes. 31m Normal Pulled pod/llmisvc-controller-manager-869d5dcc5d-tqwms Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 2.75s (2.75s including waiting). Image size: 235100981 bytes. 31m Normal Created pod/llmisvc-controller-manager-869d5dcc5d-tqwms Container created 31m Normal Started pod/kserve-controller-manager-776cc66d47-65h2w Container started 31m Normal Created pod/kserve-controller-manager-776cc66d47-65h2w Container created 31m Normal Pulled pod/kserve-controller-manager-776cc66d47-65h2w Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 2.224s (2.224s including waiting). Image size: 247676581 bytes. 31m Normal Started pod/odh-model-controller-f87fcf56c-nhwrp Container started 31m Normal Created pod/odh-model-controller-f87fcf56c-nhwrp Container created 31m Normal LeaderElection lease/kserve-controller-manager-leader-lock kserve-controller-manager-776cc66d47-65h2w_83d13420-ec2d-4029-b636-74f59b2b8ccc became leader 31m Normal LeaderElection lease/odh-model-controller.opendatahub.io odh-model-controller-f87fcf56c-nhwrp_29894a8b-03ec-4aa4-904b-896ba81f1f65 became leader 31m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-869d5dcc5d-tqwms_a74a8b3a-6b56-4f43-a065-058f519a78d1 became leader 28m Normal SuccessfulCreate replicaset/maas-controller-848f956548 Created pod: maas-controller-848f956548-92rkf 28m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-577f596567 from 0 to 1 28m Normal SuccessfulCreate replicaset/maas-controller-577f596567 Created pod: maas-controller-577f596567-rktf7 28m Normal AddedInterface pod/maas-controller-577f596567-rktf7 Add eth0 [10.132.0.65/23] from ovn-kubernetes 28m Normal Pulling pod/maas-controller-577f596567-rktf7 Pulling image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" 28m Normal AddedInterface pod/maas-controller-848f956548-92rkf Add eth0 [10.132.0.66/23] from ovn-kubernetes 28m Normal Pulling pod/maas-controller-848f956548-92rkf Pulling image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" 28m Normal Created pod/maas-controller-577f596567-rktf7 Container created 28m Normal Pulled pod/maas-controller-577f596567-rktf7 Successfully pulled image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" in 2.928s (2.928s including waiting). Image size: 199691795 bytes. 28m Normal Started pod/maas-controller-577f596567-rktf7 Container started 28m Normal Started pod/maas-controller-848f956548-92rkf Container started 28m Normal Created pod/maas-controller-848f956548-92rkf Container created 28m Normal Pulled pod/maas-controller-848f956548-92rkf Successfully pulled image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" in 2.631s (2.631s including waiting). Image size: 199691795 bytes. 28m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-848f956548-92rkf_14e0fff9-2e68-4ae3-a306-95469584da1e became leader 28m Normal ScalingReplicaSet deployment/usage-logs-tenancy-proxy Scaled up replica set usage-logs-tenancy-proxy-7d94f55f4b from 0 to 1 28m Normal SuccessfulCreate replicaset/usage-logs-tenancy-proxy-7d94f55f4b Created pod: usage-logs-tenancy-proxy-7d94f55f4b-w4nsp 28m Normal Pulling pod/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" 28m Normal AddedInterface pod/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp Add eth0 [10.132.0.67/23] from ovn-kubernetes 28m Normal ScalingReplicaSet deployment/maas-controller Scaled down replica set maas-controller-577f596567 from 1 to 0 28m Normal Killing pod/maas-controller-577f596567-rktf7 Stopping container manager 28m Normal SuccessfulDelete replicaset/maas-controller-577f596567 Deleted pod: maas-controller-577f596567-rktf7 28m Normal Pulled pod/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 10.172s (10.172s including waiting). Image size: 1089502687 bytes. 28m Normal Started pod/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp Container started 28m Normal Created pod/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp Container created 27m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-6466c554d8 Created pod: llmisvc-controller-manager-6466c554d8-ngv8z 27m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1 27m Normal Pulling pod/llmisvc-controller-manager-6466c554d8-ngv8z Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 27m Normal Pulled pod/llmisvc-controller-manager-6466c554d8-ngv8z Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 660ms (660ms including waiting). Image size: 235100981 bytes. 27m Normal AddedInterface pod/llmisvc-controller-manager-6466c554d8-ngv8z Add eth0 [10.132.0.69/23] from ovn-kubernetes 27m Normal Started pod/llmisvc-controller-manager-6466c554d8-ngv8z Container started 27m Normal Created pod/llmisvc-controller-manager-6466c554d8-ngv8z Container created 27m Normal Killing pod/llmisvc-controller-manager-869d5dcc5d-tqwms Stopping container manager 27m Normal SuccessfulDelete replicaset/llmisvc-controller-manager-869d5dcc5d Deleted pod: llmisvc-controller-manager-869d5dcc5d-tqwms 27m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0 27m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-6466c554d8-ngv8z_b60f38d4-02ae-49c3-8298-a333ef91c097 became leader 26m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-7d6c696b49 from 0 to 1 26m Normal SuccessfulCreate replicaset/ai-gateway-controller-7d6c696b49 Created pod: ai-gateway-controller-7d6c696b49-4wcv6 26m Normal AddedInterface pod/ai-gateway-controller-7d6c696b49-4wcv6 Add eth0 [10.133.0.26/23] from ovn-kubernetes 26m Normal Pulling pod/ai-gateway-controller-7d6c696b49-4wcv6 Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-146" 26m Normal Pulled pod/ai-gateway-controller-7d6c696b49-4wcv6 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-146" in 2.373s (2.373s including waiting). Image size: 183892910 bytes. 26m Normal Created pod/ai-gateway-controller-7d6c696b49-4wcv6 Container created 26m Normal Started pod/ai-gateway-controller-7d6c696b49-4wcv6 Container started 26m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-7d6c696b49-4wcv6_acb2e19a-c308-4a5d-9fc8-2b39f3053c62 became leader 26m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-d74798bcb from 0 to 1 26m Normal SuccessfulCreate replicaset/ai-gateway-controller-d74798bcb Created pod: ai-gateway-controller-d74798bcb-jpzf4 26m Normal Pulled pod/ai-gateway-controller-d74798bcb-jpzf4 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-146" in 291ms (291ms including waiting). Image size: 183892910 bytes. 26m Normal Created pod/ai-gateway-controller-d74798bcb-jpzf4 Container created 26m Normal Started pod/ai-gateway-controller-d74798bcb-jpzf4 Container started 26m Normal AddedInterface pod/ai-gateway-controller-d74798bcb-jpzf4 Add eth0 [10.133.0.29/23] from ovn-kubernetes 26m Normal Pulling pod/ai-gateway-controller-d74798bcb-jpzf4 Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-146" 25m Normal Killing pod/ai-gateway-controller-7d6c696b49-4wcv6 Stopping container manager 25m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled down replica set ai-gateway-controller-7d6c696b49 from 1 to 0 25m Normal SuccessfulDelete replicaset/ai-gateway-controller-7d6c696b49 Deleted pod: ai-gateway-controller-7d6c696b49-4wcv6 25m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-d74798bcb-jpzf4_809dc2be-9907-4f21-9fad-2a4726673b10 became leader 19m Normal Pulling pod/test-curl-9038-1e1933 Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" 19m Normal AddedInterface pod/test-curl-9038-1e1933 Add eth0 [10.132.0.98/23] from ovn-kubernetes 19m Normal Pulled pod/test-curl-9038-1e1933 Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 2.081s (2.081s including waiting). Image size: 106100986 bytes. 19m Normal Created pod/test-curl-9038-1e1933 Container created 19m Normal Started pod/test-curl-9038-1e1933 Container started 19m Normal Pulled pod/test-curl-9038-bb78e1 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 19m Normal AddedInterface pod/test-curl-9038-bb78e1 Add eth0 [10.132.0.99/23] from ovn-kubernetes 19m Normal Created pod/test-curl-9038-bb78e1 Container created 19m Normal Killing pod/test-curl-9038-1e1933 Stopping container test-curl-9038-1e1933 19m Normal Started pod/test-curl-9038-bb78e1 Container started 19m Normal Killing pod/test-curl-9038-bb78e1 Stopping container test-curl-9038-bb78e1 15m Normal Killing pod/maas-controller-848f956548-92rkf Stopping container manager 15m Normal SuccessfulDelete replicaset/maas-controller-848f956548 Deleted pod: maas-controller-848f956548-92rkf 15m Normal AddedInterface pod/maas-controller-848f956548-9fnqq Add eth0 [10.134.0.126/23] from ovn-kubernetes 15m Normal Pulling pod/maas-controller-848f956548-9fnqq Pulling image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" 15m Normal SuccessfulCreate replicaset/maas-controller-848f956548 Created pod: maas-controller-848f956548-9fnqq 14m Normal Pulled pod/maas-controller-848f956548-9fnqq Successfully pulled image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" in 2.206s (2.206s including waiting). Image size: 199691795 bytes. 14m Normal Created pod/maas-controller-848f956548-9fnqq Container created 14m Normal Started pod/maas-controller-848f956548-9fnqq Container started 14m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-848f956548-9fnqq_b631e9a0-9c76-4d18-a978-64a6ed3bc98c became leader 14m Normal SuccessfulDelete replicaset/maas-controller-848f956548 Deleted pod: maas-controller-848f956548-9fnqq 14m Normal Killing pod/maas-controller-848f956548-9fnqq Stopping container manager 14m Warning Unhealthy pod/maas-controller-848f956548-9fnqq Readiness probe failed: Get "http://10.134.0.126:8081/readyz": dial tcp 10.134.0.126:8081: connect: connection refused 14m Normal ScalingReplicaSet deployment/maas-controller Scaled down replica set maas-controller-848f956548 from 1 to 0 14m Normal SuccessfulCreate replicaset/maas-controller-848f956548 Created pod: maas-controller-848f956548-dgrbz 14m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-848f956548 from 0 to 1 14m Normal Pulled pod/maas-controller-848f956548-dgrbz Successfully pulled image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" in 716ms (716ms including waiting). Image size: 199691795 bytes. 14m Normal AddedInterface pod/maas-controller-848f956548-dgrbz Add eth0 [10.134.0.127/23] from ovn-kubernetes 14m Normal Started pod/maas-controller-848f956548-dgrbz Container started 14m Normal Pulling pod/maas-controller-848f956548-dgrbz Pulling image "quay.io/opendatahub/maas-controller:ccce7752af146eb3de9e99df107c26f414878726" 14m Normal Created pod/maas-controller-848f956548-dgrbz Container created 14m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-848f956548-dgrbz_1c99ea87-32da-4134-bdca-56e3df4976c6 became leader