Prepare Databases for Manager

Guide variables

⚠️ Prepare the values of the following variables before working with this page and replace their occurrences with the values when you follow the guide.

{NS}	The etcd namespace
{ETCDADDR}	The etcd cluster address ({ETCDHOST}:{ETCDPORT}, localhost:8120 for development setup)
{DBADDR}	The PostgreSQL server address ({DBHOST}:{DBPORT}, localhost:8100 for development setup)
{DBUSER}	The database username (e.g., postgres for development setup)
{DBPASS}	The database password (e.g., develove for development setup)
{STRGMOUNT}	The path to the mounted network-shared storage. (Development setup: Use an arbitrary empty directory where Docker containers can also mount as volumes — e.g., Docker for Mac requires explicit configuration for mountable parent folders.)

Load initial etcd data

$ cd backend.ai-manager

Copy sample-configs/image-metadata.yml and sample-configs/image-aliases.yml and edit according to your setup.

$ cp sample-configs/image-metadata.yml image-metadata.yml
$ cp sample-configs/image-aliases.yml image-aliases.yml

By default you can pull the images listed in the sample via docker pull lablup/kernel-xxxx:tag(e.g. docker pull lablup/kernel-python-tensorflow:latest for the latest tensorflow) as they are hosted on the public Docker registry.

Load image registry metadata

(Instead of manually specifying environment variables, you may use scripts/run-with-halfstack.sh script in a development setup.)

$ BACKEND_NAMESPACE={NS} BACKEND_ETCD_ADDR={ETCDADDR} \
> python -m ai.backend.manager.cli etcd update-images \
>        -f image-metadata.yml

Load image aliases

$ BACKEND_NAMESPACE={NS} BACKEND_ETCD_ADDR={ETCDADDR} \
> python -m ai.backend.manager.cli etcd update-aliases \
>        -f image-aliases.yml

Set the default storage mount for virtual folders

$ BACKEND_NAMESPACE={NS} BACKEND_ETCD_ADDR={ETCDADDR} \
> python -m ai.backend.manager.cli etcd put \
>        volumes/_vfroot {STRGMOUNT}

Database Setup

Create a new database

In docker-compose based configurations, you may skip this step.

$ psql -h {DBHOST} -p {DBPORT} -U {DBUSER}

postgres=# CREATE DATABASE backend;
postgres=# \q

Install database schema

Backend.AI uses alembic to manage database schema and its migration during version upgrades. First, localize the sample config:

$ cp alembic.ini.sample alembic.ini

Modify the line where sqlalchemy.url is set. You may use the following shell command: (ensure that special characters in your password are properly escaped)

$ sed -i'' -e 's!^sqlalchemy.url = .*$!sqlalchemy.url = postgresql://{DBUSER}:{DBPASS}@{DBHOST}:{DBPORT}/backend!' alembic.ini

$ python -m ai.backend.manager.cli schema oneshot head

NOTE: All sub-commands under "schema" uses alembic.ini to establish database connections.

Load initial fixtures

Edit ai/backend/manager/models/fixtures.py so that you have a randomized admin keypair.

(TODO: automate here!)

Then pour it to the database:

$ python -m ai.backend.manager.cli \
>        --db-addr={DBHOST}:{DBPORT} --db-user={DBUSER} --db-password={DBPASS} --db-name=backend \
>        fixture populate example_keypair