Skip to content

Conversation

d-vm
Copy link
Contributor

@d-vm d-vm commented Sep 25, 2023

Upgrade to terraform version 1.5.5 to meet the minimum requirements of the latest provider releases

@d-vm d-vm force-pushed the terraform-version-update branch from 0054bdb to 4a5e054 Compare September 25, 2023 17:52
@chivalryq
Copy link
Collaborator

We need a more updated version of terraform to use some modules. Can you be more specific? This PR looks good to me. But more information would be better.

@d-vm
Copy link
Contributor Author

d-vm commented Sep 26, 2023

Yes, you are right. Here is the detailed explanation.

For some demonstrations we have configured a new provisioning using OAM's ComponentDefinition from the official AWS module for VPC:

https://github.com/aws-ia/terraform-aws-vpc

Here is our ComponentDefinition:

apiVersion: core.oam.dev/v1beta1
kind: ComponentDefinition
metadata:
  annotations:
    definition.oam.dev/description: This module can be used to deploy a pragmatic VPC with various subnets types in AZs
  creationTimestamp: null
  labels:
    type: terraform
  name: aws-vpc-aws-orig
  namespace: vela-system
spec:
  schematic:
    terraform:
      configuration: https://github.com/aws-ia/terraform-aws-vpc.git
      providerRef:
        name: aws
        namespace: default
      type: remote
  workload:
    definition:
      apiVersion: terraform.core.oam.dev/v1beta1
      kind: Configuration
status: {}

This is the OAM application for deploying and provisioning on AWS:

apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
  name: provision-vpc-sample
  namespace: default
spec:
  components:
    - name: activaprefapp-sample-vpc
      type: aws-vpc-aws-orig
      properties:
        source: "aws-ia/vpc/aws"
        version: ">= 4.2.0"
        name: "multi-az-vpc"
        cidr_block: "10.0.0.0/16"
        vpc_assign_generated_ipv6_cidr_block: true
        vpc_egress_only_internet_gateway: true
        az_count: 3
        subnets:
          public:
            name_prefix: "my_public"
            netmask: 24
            assign_ipv6_cidr: true
            nat_gateway_configuration: "all_azs"
          private:
            netmask: 24
            connect_to_public_natgw: true
          private_ipv6:
            ipv6_native: true
            assign_ipv6_cidr: true
            connect_to_eigw: true
        writeConnectionSecretToRef:
          name: vpc-conn
        providerRef:
          name: aws3

Performing all this with the terraform controller, using this default image, we encounter the following error in the terraform-init container of the pod deployed by the OAM application:

The Terraform configuration must be valid before initialization so that
Terraform can determine which modules and providers need to be installed.
╷
│ Error: Invalid type specification
│
│   on variables.tf line 270, in variable "vpc_flow_logs":
│  270:     traffic_type         = optional(string, "ALL")
│
│ Optional attribute modifier expects only one argument: the attribute type.
╵

╷
│ Error: Invalid type specification
│
│   on variables.tf line 272, in variable "vpc_flow_logs":
│  272:       file_format                = optional(string, "plain-text")
│
│ Optional attribute modifier expects only one argument: the attribute type.
╵

╷
│ Error: Invalid type specification
│
│   on variables.tf line 273, in variable "vpc_flow_logs":
│  273:       hive_compatible_partitions = optional(bool, false)
│
│ Optional attribute modifier expects only one argument: the attribute type.
╵

╷
│ Error: Invalid type specification
│
│   on variables.tf line 274, in variable "vpc_flow_logs":
│  274:       per_hour_partition         = optional(bool, false)
│
│ Optional attribute modifier expects only one argument: the attribute type.

We have an error of interpretation of the following code of variables.tf of the official module:

variable "vpc_flow_logs" {
  description = "Whether or not to create VPC flow logs and which type. Options: \"cloudwatch\", \"s3\", \"none\". By default creates flow logs to `cloudwatch`. Variable overrides null value types for some keys, defined in defaults.tf."

  type = object({
    log_destination = optional(string)
    iam_role_arn    = optional(string)
    kms_key_id      = optional(string)

    log_destination_type = string
    retention_in_days    = optional(number)
    tags                 = optional(map(string))
    traffic_type         = optional(string, "ALL")
    destination_options = optional(object({
      file_format                = optional(string, "plain-text")
      hive_compatible_partitions = optional(bool, false)
      per_hour_partition         = optional(bool, false)
    }))
  })

  default = {
    log_destination_type = "none"
  }

  validation {
    condition     = contains(["cloud-watch-logs", "s3", "none"], var.vpc_flow_logs.log_destination_type)
    error_message = "Invalid input, options: \"cloud-watch-logs\", \"s3\", or \"none\"."
  }
}

We only got the terraform-init result correct by updating the terraform version in the image to meet the minimum requirements.

Results of terraform-init with docker-terraform image upgraded to the version proposed in the PR

Initializing the backend...

Successfully configured the backend "kubernetes"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing modules...
- calculate_subnets in modules/calculate_subnets
Downloading registry.terraform.io/drewmullen/subnets/cidr 1.0.2 for calculate_subnets.subnet_calculator...
- calculate_subnets.subnet_calculator in .terraform/modules/calculate_subnets.subnet_calculator
- calculate_subnets_ipv6 in modules/calculate_subnets_ipv6
Downloading registry.terraform.io/drewmullen/subnets/cidr 1.0.2 for calculate_subnets_ipv6.subnet_calculator...
- calculate_subnets_ipv6.subnet_calculator in .terraform/modules/calculate_subnets_ipv6.subnet_calculator
- flow_logs in modules/flow_logs
Downloading registry.terraform.io/aws-ia/cloudwatch-log-group/aws 1.0.0 for flow_logs.cloudwatch_log_group...
- flow_logs.cloudwatch_log_group in .terraform/modules/flow_logs.cloudwatch_log_group
- flow_logs.s3_log_bucket in modules/flow_logs/modules/s3_log_bucket
Downloading registry.terraform.io/aws-ia/label/aws 0.0.5 for subnet_tags...
- subnet_tags in .terraform/modules/subnet_tags
Downloading registry.terraform.io/aws-ia/label/aws 0.0.5 for tags...
- tags in .terraform/modules/tags
Downloading registry.terraform.io/aws-ia/label/aws 0.0.5 for vpc_lattice_tags...
- vpc_lattice_tags in .terraform/modules/vpc_lattice_tags

Initializing provider plugins...
- Finding hashicorp/aws versions matching ">= 3.68.0, >= 3.72.0, >= 4.0.0, >= 4.27.0"...
- Finding hashicorp/awscc versions matching "~> 0.9, >= 0.15.0"...
- Installing hashicorp/aws v5.17.0...
- Installed hashicorp/aws v5.17.0 (signed by HashiCorp)
- Installing hashicorp/awscc v0.61.0...
- Installed hashicorp/awscc v0.61.0 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

If you consider it necessary to check the image, we place it here provisionally for testing.

Copy link
Collaborator

@chivalryq chivalryq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@davma-io Thanks for the information. LGTM

@chivalryq chivalryq merged commit cd9937e into oam-dev:master Oct 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants