Cloud File Storage

A cloud file storage service built with Go, using AWS S3 as the backend storage. This service provides a REST API for file operations including upload, download, listing, and deletion.

Prerequisites

• Go
• AWS Account with S3 access
• AWS CLI
• Terraform
• Docker (optional, for containerized deployment)

Project Structure

cloud-file-storage/
├── server/
│   ├── handlers/      # HTTP request handlers
│   ├── config/        # Configuration management
│   ├── main.go        # Server entry point
│   └── go.mod         # Go module file
├── Dockerfile         # Container definition
├── docker-compose.yml # Container orchestration
└── README.md          # This file

Getting Started

1. Clone the Repository

git clone https://github.com/yourusername/cloud-file-storage.git
cd cloud-file-storage

2. Local Development Setup

cd server
go mod download

1. Initial AWS Credentials Setup

Important

These admin credentials are only needed to run Terraform. The application will use a different IAM user that Terraform creates.

1. Use AWS root account credentials

   • Log into AWS Console
   • Go to IAM → Users
   • Create a new IAM user with admin access
   • Attach the AdministratorAccess policy
   • Create access keys for this user

2. Configure these admin credentials:

aws configure
# Enter the admin access key and secret

2. Infrastructure Setup

1. Navigate to the terraform directory:

   cd terraform

2. Initialize Terraform:

   terraform init

3. (Optional) Review planned changes:

   terraform plan

4. Apply the changes:

   terraform apply

5. After successful application, Terraform will output:

   • S3 bucket name
   • IAM user access key
   • IAM user secret key

The Terraform configuration automatically:

• Creates an S3 bucket with versioning and encryption
• Creates an IAM user (file-storage-app-user)
• Creates an IAM policy with S3 permissions
• Generates access keys for the IAM user
• Attaches the policy to the user
• Configures bucket policies for security

Configuration

Create a .env file in the server directory with the following variables:

AWS_ACCESS_KEY_ID=your_access_key
AWS_SECRET_ACCESS_KEY=your_secret_key
AWS_REGION=your_region
AWS_S3_BUCKET=your_bucket_name

Start the Server

go run main.go

The server will start on port 8080 by default.

Docker Deployment

Note

When using Docker, the .env file must be in the root directory of the project.

# Start the service
docker compose up --build

# View logs
docker compose logs -f

# Stop the service
docker compose down

API Endpoints

Upload File

curl -X POST http://localhost:8080/upload \
  -F "file=@/path/to/your/file.txt" \
  -F "folder=optional_folder_name"

Get File

curl -X GET http://localhost:8080/files/{id}

List All Files

curl -X GET http://localhost:8080/files

Delete File

curl -X DELETE http://localhost:8080/files/{id}

Delete All Files

curl -X DELETE http://localhost:8080/files