Project Structure

Below is the structure of the go-ems project, reflecting its actual layout:

go-ems/
├── cmd/                # Application entry points
├── config/             # Configuration logic
├── conn/               # Database connection setup
├── domain/             # Repository and service interface definitions
├── env/                # Environment-specific configuration files
├── handlers/           # HTTP request handlers (controllers)
├── middlewares/        # Middleware functions for request processing
├── models/             # Data models and structures
├── repositories/       # Database interaction logic
├── routes/             # API route definitions
├── server/             # Server setup and initialization
├── services/           # Core business logic
├── types/              # Shared type definitions
├── utils/              # Shared utility functions
├── config.json         # JSON configuration file
├── Dockerfile          # Docker image definition
├── docker-compose.yml  # Docker Compose configuration
├── Makefile            # Build and automation commands
├── build-n-serve.sh    # Script to build and serve the application
├── go.mod              # Go module definition
├── go.sum              # Dependency checksum file
└── README.md           # Project documentation

Tech Stack

• Language : Golang
• Database : Mysql
• Cache: Redis
• Asynq Queue: Asynq
• Config Management: Consul
• Library
  • Cobra - Framework for building CLI applications
  • Viper - Library for managing configuration files and environment variables
  • GORM - ORM library for interacting with relational databases
  • Echo - Web framework used for building RESTful APIs and handling HTTP routing
  • Ozzo-Validation - Library used for validating input data, ensuring data integrity and consistency
  • golang-course-utils - Utility library providing shared helper functions and reusable components
  • Asynq - Library for managing background tasks and distributed task queues

Using Private Go Packages

To use private Go packages in your project, follow these steps:

1. Configure Git to use SSH for private repositories: Add the following to your ~/.gitconfig file:

   [url "ssh://[email protected]/"]
       insteadOf = https://github.com/

2. Set the GOPRIVATE environment variable: Add the private repository domain to the GOPRIVATE environment variable. For example:

   export GOPRIVATE=github.com/your-org-name

3. Authenticate with SSH: Ensure your SSH key is added to your SSH agent and linked to your GitHub account:

   ssh-add ~/.ssh/id_rsa

4. Get the private package: Use go get to fetch the private package:

   go get github.com/your-org-name/private-repo-name

This setup ensures that Go modules can fetch private repositories securely using SSH.

Install dependencies

go mod tidy
go mod vendor

Run the project

Make sure consul is up and running

Publish config to consul

curl --request PUT \
    --data-binary @"$CONFIG_PATH" \
    "$CONSUL_URL/v1/kv/$CONSUL_PATH"

Export environment variables

export CONSUL_URL="$CONSUL_URL"
export CONSUL_PATH="$CONSUL_PATH"

Build & Run

• build the project

go build -o app .

Run server

./app serve

Makefile

• with config.json

make run

• with env/config.local.json

make run-local

Docker

go mod vendor
docker compose up -d 

🚀 Production Deployment Guide (Docker + EC2 + ECR + Monitoring)

This guide walks through deploying a Dockerized application to an EC2 instance using Docker Compose, pulling images from AWS ECR, and enabling observability with Prometheus and Grafana.

---

🛠 Prerequisites

• EC2 instance (Amazon Linux 2 or similar)
• SSH access (.pem key file)
• Docker images pushed to AWS ECR
• AWS CLI configured
• Monitoring stack files (docker-compose.yml, prometheus.yml, etc.)

---

📦 Step 1: Setup Docker and Docker Compose

SSH into the EC2 instance:

ssh -i path/to/your-key.pem ec2-user@your-ec2-hostname

Then run:

sudo dnf update -y
sudo dnf install -y docker
sudo systemctl enable --now docker
sudo usermod -aG docker $USER
newgrp docker

Verify Docker is installed:

docker version
docker info

Install Docker Compose plugin:

mkdir -p ~/.docker/cli-plugins/
curl -SL https://github.com/docker/compose/releases/latest/download/docker-compose-linux-x86_64 \
  -o ~/.docker/cli-plugins/docker-compose
chmod +x ~/.docker/cli-plugins/docker-compose
docker compose version

---

📂 Step 2: Copy Files to the Server

From your local machine, copy required files to the EC2 instance:

scp -i path/to/your-key.pem ./docker-compose.yml ec2-user@your-ec2-hostname:/home/ec2-user/docker-compose.yml
scp -i path/to/your-key.pem ./init_db.sql ec2-user@your-ec2-hostname:/home/ec2-user/init_db.sql
scp -i path/to/your-key.pem ./prometheus.yml ec2-user@your-ec2-hostname:/home/ec2-user/prometheus.yml

---

🔐 Step 3: Authenticate Docker with AWS ECR

On the EC2 server:

1. Configure AWS CLI:

aws configure

2. Authenticate Docker to AWS ECR:

aws ecr get-login-password --region <your-region> \
  | docker login --username AWS --password-stdin <your-account-id>.dkr.ecr.<your-region>.amazonaws.com

---

Step 4: Deploy with Docker Compose

Build Docker image for Linux amd64 architecture

docker build -t <aws_account_id>.dkr.ecr.<region>.amazonaws.com/<repository_name>:latest . \
  --platform linux/amd64

Push the image to ECR

docker push <aws_account_id>.dkr.ecr.<region>.amazonaws.com/<repository_name>:latest

Authenticate to ECR before pushing:

aws ecr get-login-password --region <region> \
  | docker login --username AWS --password-stdin <aws_account_id>.dkr.ecr.<region>.amazonaws.com

Pull latest images:

docker compose -f docker-compose.yml pull

Start the containers:

docker compose -f docker-compose.yml up -d

---

Step 5: Access and Configure Grafana

• Visit: http://<your-ec2-public-ip>:3000

• Default login: admin / admin

• Add Prometheus as a data source:

  • URL: http://prometheus:9090

Sample Prometheus Queries

• CPU Usage:

  sum(rate(process_cpu_seconds_total[1m]))
  

• Custom request metric:

  sum(your_app_requests_total) by (url)
  

---

Monitoring & Troubleshooting

Check container status:

docker ps

Check logs:

docker compose logs -f

Restart services:

docker compose restart

Shutdown:

docker compose down

---

📌 Notes

• Open the necessary ports in your EC2 security group:
  • 22 for SSH
  • 80/443 for web access
  • 3000 for Grafana
  • 8080 for Backend service
• Do not expose Prometheus publicly without security controls.