This repository contains IaC code based on Pulumi to provision dynamic test infrastructures for testing.
To run scripts and code in this repository, you will need:
-
Go 1.22 or later. You'll also need to set your
$GOPATH
and have$GOPATH/bin
in your path. -
Python 3.9+ along with development libraries for tooling.
-
account-admin
role on AWSagent-sandbox
account. Ensure it by runningaws-vault login sso-agent-sandbox-account-admin
-
dda installed on your laptop
aws-vault login sso-agent-sandbox-account-admin
This guide is tested on MacOS.
sudo apt install libnotify-bin
- Clone this repository
cd ~/dd && git clone [email protected]:DataDog/test-infra-definitions.git
- Install Python dependencies
cd ~/dd/test-infra-definitions && dda -v self dep sync -f legacy-e2e -f legacy-github
- Add a PULUMI_CONFIG_PASSPHRASE to your Terminal rc file. Create a random password using 1Password and store it there
export PULUMI_CONFIG_PASSPHRASE=<random password stored in 1Password>
- Run and follow the setup script
dda inv setup
Invoke tasks help deploying most common environments - VMs, Docker, ECS, EKS. Run dda inv -l
to learn more.
To get the list of available tasks
❯ dda inv -l
Run any -h
on any of the available tasks for more information
The setup.debug
invoke task will check for common mistakes such as key unavailable in configured AWS region, ssh-agent not running, invalid key format, and more.
aws-vault exec sso-agent-sandbox-account-admin -- dda inv setup.debug
aws-vault exec sso-agent-sandbox-account-admin -- dda inv setup --debug --no-interactive
Note: For most of the users interacting with Pulumi directly should not be required, you should be able to rely exclusively on the available tasks.
Pulumi requires to store/retrieve the state of your Stack
.
In Pulumi, Stack
objects represent your actual deployment:
- A
Stack
references aProject
(a folder with aPulumi.yaml
, for instance root folder of this repo) - A
Stack
references a configuration file calledPulumi.<stack_name>.yaml
This file holds yourStack
configuration. If it does not exist, it will be created. If it exists and you input some settings through the command line, using-c
, it will update theStack
file.
When performing operations on a Stack
, Pulumi will need to store a state somewhere (the Stack state).
Normally the state should be stored in a durable storage (e.g. S3-like), but for testing purposes
local filesystem could be used.
To choose a default storage provider, use pulumi login
(should be only done once):
# Using local filesystem (state will be stored in ~/.pulumi)
pulumi login --local
# Using storage on Cloud Storage (GCP)
# You need to create the bucket on sandbox.
# You also need to have sandbox as your current tenant in gcloud CLI.
pulumi login gs://<your_name>-pulumi
More information about state can be retrieved at: https://www.pulumi.com/docs/intro/concepts/state/
Finally, Pulumi is encrypting secrets in your Pulumi.<stack_name>.yaml
(if entered as such).
To do that, it requires a password. For dev purposes, you can simply store the password in the PULUMI_CONFIG_PASSPHRASE
variable in your ~/.zshrc
.
In this example, we're going to create an ECS Cluster:
# You need to have a DD APIKey in variable DD_API_KEY
pulumi up -c scenario=aws/ecs -c ddinfra:aws/defaultKeyPairName=<your_exisiting_aws_keypair_name> -c ddinfra:env=aws/agent-sandbox -c ddagent:apiKey=$DD_API_KEY -s <your_name>-ecs-test
In case of failure, you may update some parameters or configuration and run the command again.
Note that all -c
parameters have been set in your Pulumi.<stack_name>.yaml
file.
NOTE: Do not commit your Stack file.
Once you're finished with the test environment you've created, you can safely delete it.
To do this, we'll use the destroy
operation referencing our Stack
file:
pulumi destroy -s <your_name>-ecs-test
Note that we don't need to use -c
again as the configuration values were put into the Stack
file.
This will destroy all cloud resources associated to the Stack, but the state itself (mostly empty) will still be there.
To remove the stack state:
pulumi stack rm <your_name>-ecs-test
# You need to have a DD APIKey in variable DD_API_KEY
pulumi up -c scenario=aws/dockervm -c ddinfra:aws/defaultKeyPairName=<your_exisiting_aws_keypair_name> -c ddinfra:env=aws/agent-sandbox -c ddagent:apiKey=$DD_API_KEY -c ddinfra:aws/defaultPrivateKeyPath=$HOME/.ssh/id_rsa -s <your_name>-docker
# You need to have a DD APIKey in variable DD_API_KEY
pulumi up -c scenario=aws/ecs -c ddinfra:aws/defaultKeyPairName=<your_exisiting_aws_keypair_name> -c ddinfra:env=aws/agent-sandbox -c ddagent:apiKey=$DD_API_KEY -s <your_name>-ecs
# You need to have a DD APIKey AND APPKey in variable DD_API_KEY / DD_APP_KEY
pulumi up -c scenario=aws/eks -c ddinfra:aws/defaultKeyPairName=<your_exisiting_aws_keypair_name> -c ddinfra:env=aws/agent-sandbox -c ddagent:apiKey=$DD_API_KEY -c ddagent:appKey=$DD_APP_KEY -s <your_name>-eks
Prerequisites:
- Install the GKE authentication plugin:
gcloud components install gke-gcloud-auth-plugin
- Add the plugin to your PATH:
export PATH="/opt/homebrew/share/google-cloud-sdk/bin:$PATH"
- Authenticate with GCP:
gcloud auth application-default login
# You need to have a DD APIKey AND APPKey in variable DD_API_KEY / DD_APP_KEY
# GKE Standard
pulumi up -c scenario=gcp/gke -c ddinfra:env=gcp/agent-sandbox -c ddinfra:gcp/defaultPublicKeyPath=$HOME/.ssh/id_ed25519.pub -c ddagent:apiKey=$DD_API_KEY -c ddagent:appKey=$DD_APP_KEY -s <your_name>-gke
# GKE Autopilot
pulumi up -c scenario=gcp/gke -c ddinfra:env=gcp/agent-sandbox -c ddinfra:gcp/defaultPublicKeyPath=$HOME/.ssh/id_ed25519.pub -c ddinfra:gcp/gke/enableAutopilot=true -c ddagent:apiKey=$DD_API_KEY -c ddagent:appKey=$DD_APP_KEY -s <your_name>-gke-autopilot
# You need to have a DD APIKey AND APPKey in variable DD_API_KEY / DD_APP_KEY
pulumi up -c scenario=gcp/openshiftvm -c ddinfra:env=gcp/agent-sandbox -c ddinfra:gcp/openshift/pullSecretPath=<your_pull_secret_path -c ddinfra:gcp/enableNestedVirtualization=true -c ddinfra:gcp/defaultInstanceType=n2-standard-8 -c ddinfra:gcp/defaultPublicKeyPath=$HOME/.ssh/id_ed25519.pub -c ddagent:apiKey=$DD_API_KEY -c ddagent:appKey=$DD_APP_KEY -s <your_name>-openshift