TargetGroupBindings can now manipulate target groups from different aws accounts

[marcosdiez]

Issue #3634

Description

Now, every time a TargetGroupBinding has the alb.ingress.kubernetes.io/IamRoleArnToAssume (and optionally alb.ingress.kubernetes.io/AssumeRoleExternalId) annotations, we assume it before interacting to it (actually, the assume role operation is cached).

In real life, I changed:

• targetgroupbinding_validator
• targetgroupbinding_mutator
• target manager (which has RegisterTargets / DeregisterTargets / ListTargets

This way we can easily manipulate target groups from target group bindings for every single AWS account in the world, as long as there is a role we can assume. In order to prevent the confused deputy problem, external id is supported.

The way this was implemented was actually way simpler than initially imagined:

the service.elbv2Client object now supports a new method called AssumeRole which returns service.elbv2Client on the specified role. The catch is that if AssumeRole is called with a blank role, the object returns itself and life goes on.

In other words, every time we call some method from elbv2Client from something related to target groups, we also call AssumeRole and that's basically it.

Caching the assumed role was also trivial, a simple map that is stored on cloud.go

I decided to have cloud.go manage the caching becuase in the future it could assume role for other objects if needed and all the assume role logic would be in the same place.

The only unfortunate change that needed to be made is that aws.Cloud became services.Cloud else golang would complain about circular dependency :(

This is what the logs look like:

{"level":"info","ts":"2024-05-13T03:24:31Z","msg":"awsCloud","method":"GetAssumedRoleELBV2","AssumeRoleArn":"arn:aws:iam::7550XXXXXXXX:role/alb-controller-policy-to-impersonate","externalId":"XXXXXX"}
{"level":"info","ts":"2024-05-13T03:24:33Z","msg":"registering endpoints using the targetGroup's vpcID vpc-00XXXXXXXXXXXXXXX which is different from the cluster's vpcID vpc-04XXXXXXXXXXXXXXX"}
{"level":"info","ts":"2024-05-13T03:24:33Z","msg":"registering targets","arn":"arn:aws:elasticloadbalancing:us-east-1:7550XXXXXXXX:targetgroup/helloworld-8080-a8054/5741da332459b560","targets":[{"AvailabilityZone":"all","Id":"10.244.0.22","Port":8080}]}
{"level":"info","ts":"2024-05-13T03:24:34Z","msg":"registered targets","arn":"arn:aws:elasticloadbalancing:us-east-1:7550XXXXXXXX:targetgroup/helloworld-8080-a8054/5741da332459b560"}

Documentation and tests were properly updated in this PR

Checklist

• Added tests that cover your change (if possible)
• Added/modified documentation as required (such as the README.md, or the docs directory)
• Manually tested
• Made sure the title of the PR is a good description that can go into the release notes

BONUS POINTS checklist: complete for good vibes and maybe prizes?! 🤯

• Backfilled missing tests for code in same general area 🎉
• Refactored something and made the world a better place 🌟

A working amd64 container with this code is available here: marcosdiez/aws-load-balancer-controller:v2.7.2-m2

update: on 2024-09-26 I rebased the code. The new container image is marcosdiez/aws-load-balancer-controller:v2.8.3-m2

update: on 2024-10-28 I rebased the code again :(. The new container image is marcosdiez/aws-load-balancer-controller:v2.9.0-m1

[k8s-ci-robot]

Hi @marcosdiez. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[marcosdiez]

@shraddhabang I believe my code is ready. Feel free to review at will! Thank you.

Also, if you don't want the CRDs to be changed, here:
126c9c8

they are untouched.

[shraddhabang]

Thank you @marcosdiez for this PR. This looks excellent.
I will take a look and test it out. And would also ask our team to review it. Please note that we will have to go through our formal security process to review this. We will add this to our next minor release v2.9.0.
Please feel free to reach out to us.

[shraddhabang]

/ok-to-test

[yu-croco]

Hi @shraddhabang ,

We will add this to our next minor release v2.9.0

Could you please share us when the v2.9.0 would release? We are looking forward to this feature.

[yu-croco]

Hi, @johngmyers @M00nF1sh @shraddhabang

We will add this to our next minor release v2.9.0.

Does anyone know the release schedule for v2.9.0 ?

[marcosdiez]

I just rebased and merged my code. There is also a container available here: marcosdiez/aws-load-balancer-controller:v2.8.3-m1

[abr4xas]

Waited for this for a very long time! We need this ASAP.

[ellanos]

I'm waiting for this feature long time ago. Please approve and merge asap.

[cincinatino]

I've been eagerly awaiting this feature for a long time. Please approve and merge it as soon as possible. This feature is crucial for the efficient delivery of my infrastructure, as it will significantly enhance our operational capabilities.

[arawako]

I need this feature! please approve it.

[guerrerocarlos]

Looking forward for this to be merged! 🚀

[Skatox]

This is precisely what I was looking for. Is there an estimate for when this will be merged? I don't want to fork the repo just for this.

[zac-nixon]

Hello, thank you for this contribution. I apologize that we've made you wait so long for a proper review. I took a look at the code and it looks really good to me aside from small code duplication in the register targets path; but we can refactor that later. The biggest concern I currently have is that we are storing the external id in plain text. I am engaging with AWS AppSec to understand the best practices around storage of the external id.

[listellm]

Hello, thank you for this contribution. I apologize that we've made you wait so long for a proper review. I took a look at the code and it looks really good to me aside from small code duplication in the register targets path; but we can refactor that later. The biggest concern I currently have is that we are storing the external id in plain text. I am engaging with AWS AppSec to understand the best practices around storage of the external id.

Hi,

AWS IAM documentation state that:

It is not intended to be a 'secret'. Example Corp must provide the ExternalId value to each customer. What is crucial is that it must be generated by Example Corp and not their customers to ensure each external ID is unique.

This external ID is visible to anyone who has IAM read access and access to the AWS account. It is not stored as a secret

[vanetowers]

Hi! Great job done here. Looking forward to use this feature when approved.

[zac-nixon]

Thanks for contribution here, I have left some comments on stuff that I'd like changed. Since this PR has been sitting for a while I'm inclined to approve as-is and I can work on the changes in a separate PR. Let me know your thoughts or if you disagree with any of my suggestions.

Lastly, after engaging with my AWS AppSec team they would like to take a crack at testing an image prior to releasing for public consumption.

Thank you so much for your contribution! It is really well done.

[zac-nixon]

AWS AppSec did not find any problems with this approach. I'm going to engage a teammate to give this another review and then we'll merge it. I'll work on the comments in a follow up PR.

[zac-nixon]

/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: marcosdiez, wweiwei-li, zac-nixon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [wweiwei-li,zac-nixon]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment