Enhance subnet discovery

docs/deploy/subnet_discovery.md

@@ -1,38 +1,71 @@
-# Subnet auto-discovery
-By default, the AWS Load Balancer Controller (LBC) auto-discovers network subnets that it can create AWS Network Load Balancers (NLB) and AWS Application Load Balancers (ALB) in. ALBs require at least two subnets across Availability Zones by default, 
-set [Feature Gate ALBSingleSubnet](https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/deploy/configurations/#feature-gates) to "true" allows using only 1 subnet for provisioning ALB. NLBs require one subnet.
-The subnets must be tagged appropriately for auto-discovery to work. The controller chooses one subnet from each Availability Zone. During auto-discovery, the controller
-considers subnets with at least eight available IP addresses. In the case of multiple qualified tagged subnets in an Availability Zone, the controller chooses the first one in lexicographical 
-order by the subnet IDs.
-For more information about the subnets for the LBC, see [Application Load Balancers](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/application-load-balancers.html) 
-and [Network Load Balancers](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/network-load-balancers.html).
-If you used `eksctl` or an Amazon EKS AWS CloudFormation template to create your VPC after March 26, 2020, then the subnets are tagged appropriately when they're created. For 
-more information about the Amazon EKS AWS CloudFormation VPC templates, see [Creating a VPC for your Amazon EKS cluster](https://docs.aws.amazon.com/eks/latest/userguide/create-public-private-vpc.html).
+# Subnet Auto-Discovery
+The AWS Load Balancer Controller (LBC) automatically discovers subnets for creating AWS Network Load Balancers (NLB) and AWS Application Load Balancers (ALB). This auto-discovery process follows three main steps:
 
-## Public subnets
-Public subnets are used for internet-facing load balancers. These subnets must have the following tags:
+1. **Candidate Subnet Determination**: The controller identifies potential candidate subnets
+2. **Subnet Filtering**: The controller filters these candidates based on eligibility criteria
+3. **Final Selection**: The controller selects one subnet per availability zone
 
-| Key                                     | Value                 |
-| --------------------------------------- | --------------------- |
-| `kubernetes.io/role/elb`                | `1`  or ``            |
+## Candidate Subnet Determination
+The controller determines candidate subnets using the following process:
 
-## Private subnets
-Private subnets are used for internal load balancers. These subnets must have the following tags:
+1. **If tag filters are specified**: Only subnets matching these filters become candidates
+
+    !!!tip
+        You can only specify subnet tag filters for Ingress via IngressClassParams
 
-| Key                                     | Value                 |
-| --------------------------------------- | --------------------- |
-|  `kubernetes.io/role/internal-elb`      |  `1`  or ``           |
+2. **If no tag filters are specified**:
+    * If subnets with matching [role tag](#subnet-role-tag) exists: Only these become candidates
+    * [**For LBC version >= 2.12.1**] If no subnets have role tags: Candidates are subnets whose [reachability](#subnet-reachability) (public/private) matches the LoadBalancer's schema
 
 
-## Common tag
-In version v2.1.1 and older of the LBC, both the public and private subnets must be tagged with the cluster name as follows:
+### Subnet Role Tag
+Subnets can be tagged appropriately for auto-discovery selection:
 
-| Key                                     | Value                 |
-| --------------------------------------- | --------------------- |
-| `kubernetes.io/cluster/${cluster-name}` | `owned` or `shared`   |
+* **For internet-facing load balancers**, the controller looks for public subnets with following tags:
 
- `${cluster-name}` is the name of the Kubernetes cluster.
-
-The cluster tag is not required in versions v2.1.2 to v2.4.1, unless a cluster tag for another cluster is present.
+    | Key                                     | Value                 |
+    | --------------------------------------- | --------------------- |
+    | `kubernetes.io/role/elb`                | `1`  or ``            |
 
-With versions v2.4.2 and later, you can disable the cluster tag check completely by specifying the feature gate `SubnetsClusterTagCheck=false`
+* **For internal load balancers**, the controller looks for private subnets with following tags:
+
+    | Key                                     | Value                 |
+    | --------------------------------------- | --------------------- |
+    |  `kubernetes.io/role/internal-elb`      |  `1`  or ``           |
+
+### Subnet reachability
+The controller automatically discovers all subnets in your VPC and determines whether each is a public or private subnet based on its associated route table configuration.
+A subnet is classified as public if its route table contains a route to an Internet Gateway.
+
+!!!tip
+    You can disable this behavior via SubnetDiscoveryByReachability feature flag.
+
+## Subnet Filtering
+
+1. **Cluster Tag Check**: The controller checks for cluster tags on subnets. Subnets with ineligible cluster tags will be filtered out.
+
+    | Key                                     | Value                 |
+    | --------------------------------------- | --------------------- |
+    |  `kubernetes.io/cluster/${cluster-name}`      |  `owned` or `shared`          |
+
+    * If such cluster tag exists but no `<clusterName>` matches the current cluster, those subnets will be filtered out.
+    * [**For LBC version < 2.1.1**] subnets without a cluster tag matching cluster name will be filtered out.
+
+    !!! tip
+        You can disable this behavior via the `SubnetsClusterTagCheck` feature flag. When disabled, no cluster tag check will be performed against subnets.
+
+2. **IP Address Availability**: Subnets with insufficient available IP addresses(**<8**) are filtered out.
+
+## Final Selection
+
+The controller selects one subnet per availability zone. When multiple subnets exist per Availability Zone, the following priority order applies:
+
+1. Subnets with cluster tag for the current cluster (`kubernetes.io/cluster/<clusterName>`) are prioritized
+2. Subnets with lower lexicographical order of subnet ID are prioritized
+
+## Minimum Subnet Requirements
+
+* **ALBs**: Require at least two subnets across different Availability Zones by default
+
+    !!! tip
+        For customers allowlisted by the AWS Elastic Load Balancing team, you can enable the [ALBSingleSubnet feature gate](https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/deploy/configurations/#feature-gates). This allows provisioning an ALB with just one subnet instead of the standard requirement of two subnets.

docs/install/iam_policy.json

@@ -31,6 +31,7 @@
                 "ec2:DescribeCoipPools",
                 "ec2:GetSecurityGroupsForVpc",
                 "ec2:DescribeIpamPools",
+                "ec2:DescribeRouteTables",
                 "elasticloadbalancing:DescribeLoadBalancers",
                 "elasticloadbalancing:DescribeLoadBalancerAttributes",
                 "elasticloadbalancing:DescribeListeners",

docs/install/iam_policy_cn.json

@@ -31,6 +31,7 @@
                 "ec2:DescribeCoipPools",
                 "ec2:GetSecurityGroupsForVpc",
                 "ec2:DescribeIpamPools",
+                "ec2:DescribeRouteTables",
                 "elasticloadbalancing:DescribeLoadBalancers",
                 "elasticloadbalancing:DescribeLoadBalancerAttributes",
                 "elasticloadbalancing:DescribeListeners",

docs/install/iam_policy_iso.json

@@ -31,6 +31,7 @@
                 "ec2:DescribeCoipPools",
                 "ec2:GetSecurityGroupsForVpc",
                 "ec2:DescribeIpamPools",
+                "ec2:DescribeRouteTables",
                 "elasticloadbalancing:DescribeLoadBalancers",
                 "elasticloadbalancing:DescribeLoadBalancerAttributes",
                 "elasticloadbalancing:DescribeListeners",

docs/install/iam_policy_isob.json

@@ -31,6 +31,7 @@
                 "ec2:DescribeCoipPools",
                 "ec2:GetSecurityGroupsForVpc",
                 "ec2:DescribeIpamPools",
+                "ec2:DescribeRouteTables",
                 "elasticloadbalancing:DescribeLoadBalancers",
                 "elasticloadbalancing:DescribeLoadBalancerAttributes",
                 "elasticloadbalancing:DescribeListeners",

docs/install/iam_policy_isoe.json

@@ -31,6 +31,7 @@
                 "ec2:DescribeCoipPools",
                 "ec2:GetSecurityGroupsForVpc",
                 "ec2:DescribeIpamPools",
+                "ec2:DescribeRouteTables",
                 "elasticloadbalancing:DescribeLoadBalancers",
                 "elasticloadbalancing:DescribeLoadBalancerAttributes",
                 "elasticloadbalancing:DescribeListeners",

docs/install/iam_policy_isof.json

@@ -31,6 +31,7 @@
                 "ec2:DescribeCoipPools",
                 "ec2:GetSecurityGroupsForVpc",
                 "ec2:DescribeIpamPools",
+                "ec2:DescribeRouteTables",
                 "elasticloadbalancing:DescribeLoadBalancers",
                 "elasticloadbalancing:DescribeLoadBalancerAttributes",
                 "elasticloadbalancing:DescribeListeners",

docs/install/iam_policy_us-gov.json

@@ -31,6 +31,7 @@
                 "ec2:DescribeCoipPools",
                 "ec2:GetSecurityGroupsForVpc",
                 "ec2:DescribeIpamPools",
+                "ec2:DescribeRouteTables",
                 "elasticloadbalancing:DescribeLoadBalancers",
                 "elasticloadbalancing:DescribeLoadBalancerAttributes",
                 "elasticloadbalancing:DescribeListeners",

main.go

@@ -17,9 +17,10 @@ limitations under the License.
 package main
 
 import (
-	"k8s.io/client-go/util/workqueue"
 	"os"
 
+	"k8s.io/client-go/util/workqueue"
+
 	elbv2deploy "sigs.k8s.io/aws-load-balancer-controller/pkg/deploy/elbv2"
 
 	"github.com/go-logr/logr"
@@ -119,7 +120,11 @@ func main() {
 	sgReconciler := networking.NewDefaultSecurityGroupReconciler(sgManager, ctrl.Log)
 	azInfoProvider := networking.NewDefaultAZInfoProvider(cloud.EC2(), ctrl.Log.WithName("az-info-provider"))
 	vpcInfoProvider := networking.NewDefaultVPCInfoProvider(cloud.EC2(), ctrl.Log.WithName("vpc-info-provider"))
-	subnetResolver := networking.NewDefaultSubnetsResolver(azInfoProvider, cloud.EC2(), cloud.VpcID(), controllerCFG.ClusterName, ctrl.Log.WithName("subnets-resolver"))
+	subnetResolver := networking.NewDefaultSubnetsResolver(azInfoProvider, cloud.EC2(), cloud.VpcID(), controllerCFG.ClusterName,
+		controllerCFG.FeatureGates.Enabled(config.SubnetsClusterTagCheck),
+		controllerCFG.FeatureGates.Enabled(config.ALBSingleSubnet),
+		controllerCFG.FeatureGates.Enabled(config.SubnetDiscoveryByReachability),
+		ctrl.Log.WithName("subnets-resolver"))
 	multiClusterManager := targetgroupbinding.NewMultiClusterManager(mgr.GetClient(), mgr.GetAPIReader(), ctrl.Log)
 	tgbResManager := targetgroupbinding.NewDefaultResourceManager(mgr.GetClient(), cloud.ELBV2(), cloud.EC2(),
 		podInfoRepo, sgManager, sgReconciler, vpcInfoProvider, multiClusterManager, lbcMetricsCollector,

pkg/aws/services/ec2.go

@@ -2,6 +2,7 @@ package services
 
 import (
 	"context"
+
 	"github.com/aws/aws-sdk-go-v2/service/ec2"
 	"github.com/aws/aws-sdk-go-v2/service/ec2/types"
 	"sigs.k8s.io/aws-load-balancer-controller/pkg/aws/provider"
@@ -24,6 +25,9 @@ type EC2 interface {
 	// DescribeVPCsAsList wraps the DescribeVpcsPagesWithContext API, which aggregates paged results into list.
 	DescribeVPCsAsList(ctx context.Context, input *ec2.DescribeVpcsInput) ([]types.Vpc, error)
 
+	// DescribeRouteTablesAsList wraps the DescribeRouteTablesWithContext API, which aggregates paged results into list.
+	DescribeRouteTablesAsList(ctx context.Context, input *ec2.DescribeRouteTablesInput) ([]types.RouteTable, error)
+
 	CreateTagsWithContext(ctx context.Context, input *ec2.CreateTagsInput) (*ec2.CreateTagsOutput, error)
 	DeleteTagsWithContext(ctx context.Context, input *ec2.DeleteTagsInput) (*ec2.DeleteTagsOutput, error)
 	CreateSecurityGroupWithContext(ctx context.Context, input *ec2.CreateSecurityGroupInput) (*ec2.CreateSecurityGroupOutput, error)
@@ -141,6 +145,23 @@ func (c *ec2Client) DescribeVPCsAsList(ctx context.Context, input *ec2.DescribeV
 	return result, nil
 }
 
+func (c *ec2Client) DescribeRouteTablesAsList(ctx context.Context, input *ec2.DescribeRouteTablesInput) ([]types.RouteTable, error) {
+	var result []types.RouteTable
+	client, err := c.awsClientsProvider.GetEC2Client(ctx, "DescribeRouteTables")
+	if err != nil {
+		return nil, err
+	}
+	paginator := ec2.NewDescribeRouteTablesPaginator(client, input)
+	for paginator.HasMorePages() {
+		output, err := paginator.NextPage(ctx)
+		if err != nil {
+			return nil, err
+		}
+		result = append(result, output.RouteTables...)
+	}
+	return result, nil
+}
+
 func (c *ec2Client) CreateTagsWithContext(ctx context.Context, input *ec2.CreateTagsInput) (*ec2.CreateTagsOutput, error) {
 	client, err := c.awsClientsProvider.GetEC2Client(ctx, "CreateTags")
 	if err != nil {

pkg/config/feature_gates.go

@@ -11,18 +11,19 @@ import (
 type Feature string
 
 const (
-	ListenerRulesTagging         Feature = "ListenerRulesTagging"
-	WeightedTargetGroups         Feature = "WeightedTargetGroups"
-	ServiceTypeLoadBalancerOnly  Feature = "ServiceTypeLoadBalancerOnly"
-	EndpointsFailOpen            Feature = "EndpointsFailOpen"
-	EnableServiceController      Feature = "EnableServiceController"
-	EnableIPTargetType           Feature = "EnableIPTargetType"
-	EnableRGTAPI                 Feature = "EnableRGTAPI"
-	SubnetsClusterTagCheck       Feature = "SubnetsClusterTagCheck"
-	NLBHealthCheckAdvancedConfig Feature = "NLBHealthCheckAdvancedConfig"
-	NLBSecurityGroup             Feature = "NLBSecurityGroup"
-	ALBSingleSubnet              Feature = "ALBSingleSubnet"
-	LBCapacityReservation        Feature = "LBCapacityReservation"
+	ListenerRulesTagging          Feature = "ListenerRulesTagging"
+	WeightedTargetGroups          Feature = "WeightedTargetGroups"
+	ServiceTypeLoadBalancerOnly   Feature = "ServiceTypeLoadBalancerOnly"
+	EndpointsFailOpen             Feature = "EndpointsFailOpen"
+	EnableServiceController       Feature = "EnableServiceController"
+	EnableIPTargetType            Feature = "EnableIPTargetType"
+	EnableRGTAPI                  Feature = "EnableRGTAPI"
+	SubnetsClusterTagCheck        Feature = "SubnetsClusterTagCheck"
+	NLBHealthCheckAdvancedConfig  Feature = "NLBHealthCheckAdvancedConfig"
+	NLBSecurityGroup              Feature = "NLBSecurityGroup"
+	ALBSingleSubnet               Feature = "ALBSingleSubnet"
+	SubnetDiscoveryByReachability Feature = "SubnetDiscoveryByReachability"
+	LBCapacityReservation         Feature = "LBCapacityReservation"
 )
 
 type FeatureGates interface {
@@ -50,18 +51,19 @@ type defaultFeatureGates struct {
 func NewFeatureGates() FeatureGates {
 	return &defaultFeatureGates{
 		featureState: map[Feature]bool{
-			ListenerRulesTagging:         true,
-			WeightedTargetGroups:         true,
-			ServiceTypeLoadBalancerOnly:  false,
-			EndpointsFailOpen:            true,
-			EnableServiceController:      true,
-			EnableIPTargetType:           true,
-			EnableRGTAPI:                 false,
-			SubnetsClusterTagCheck:       true,
-			NLBHealthCheckAdvancedConfig: true,
-			NLBSecurityGroup:             true,
-			ALBSingleSubnet:              false,
-			LBCapacityReservation:        true,
+			ListenerRulesTagging:          true,
+			WeightedTargetGroups:          true,
+			ServiceTypeLoadBalancerOnly:   false,
+			EndpointsFailOpen:             true,
+			EnableServiceController:       true,
+			EnableIPTargetType:            true,
+			EnableRGTAPI:                  false,
+			SubnetsClusterTagCheck:        true,
+			NLBHealthCheckAdvancedConfig:  true,
+			NLBSecurityGroup:              true,
+			ALBSingleSubnet:               false,
+			SubnetDiscoveryByReachability: true,
+			LBCapacityReservation:         true,
 		},
 	}
 }