|
| 1 | +# Run tutorials on an AWS EC2 instance using a SMCE Docker image |
| 2 | + |
| 3 | +(valid as of 2024-09-04; adapted from readme at `ECCO-v4-Python-Tutorial/Docker/README.md`) |
| 4 | + |
| 5 | + |
| 6 | +## Getting started |
| 7 | + |
| 8 | +To set this up you need an AWS EC2 instance with a Linux OS and Docker installed. The Dockerfile can be retrieved using |
| 9 | + |
| 10 | +```bash |
| 11 | +wget https://raw.githubusercontent.com/ECCO-GROUP/ECCO-v4-Python-Tutorial/master/Docker/smce/Dockerfile |
| 12 | +``` |
| 13 | + |
| 14 | +## Build the Docker image |
| 15 | + |
| 16 | +The `ECCO-v4-Python-Tutorial/Docker/smce` directory has the files that you need to build the Docker image. Make sure you are in the same directory as the Dockerfile and use: |
| 17 | + |
| 18 | +```bash |
| 19 | +docker build . -t ecco_tutorials |
| 20 | +``` |
| 21 | + |
| 22 | +The build process takes a few minutes, and the image will occupy a little over 10 GB of storage, so make sure your instance has sufficient storage. |
| 23 | + |
| 24 | +## Run the Docker image |
| 25 | + |
| 26 | +When the build completes, you will run the image, which will activate a container within your EC2 instance, and start running Jupyter lab in that container. This is done with the following command: |
| 27 | + |
| 28 | +```bash |
| 29 | +docker run -it -p 8888:8888 ecco_tutorials |
| 30 | +``` |
| 31 | + |
| 32 | +Note the port numbers specified under the `-p` option. The port listed after the colon is the container port, which is always 8888 unless this is changed manually in the `Dockerfile` (on the `EXPOSE 8888` line) prior to building the image. The port before the colon is what the host EC2 instance uses to communicate with the container, and this can be specified differently depending on the user's port availability. |
| 33 | + |
| 34 | +If you also want to use Pluto notebooks (running Julia code), you will need to also connect to port 1234 on the container, e.g., |
| 35 | + |
| 36 | +```bash |
| 37 | +docker run -it -p 8888:8888 -p 1234:1234 ecco_tutorials |
| 38 | +``` |
| 39 | + |
| 40 | +When one of the above commands above is run, you will first be queried for NASA Earthdata credentials if those are not already stored in a `~/.netrc` file under your user home directory. After entering the credentials, you will be queried for the container port number (8888 by default unless changed as described above), and an optional password (if no password is entered, none will be needed to log in to Jupyter lab). |
| 41 | + |
| 42 | +As Jupyter lab is launched, you will see a lot of output tagged `ServerApp` or `LabApp`. To free up this window you can press `Ctrl-p` `Ctrl-q`, and the window will escape the container...but importantly, the container is still running. To check the status of Docker containers, run `docker ps -a`. |
| 43 | + |
| 44 | +## Open Jupyter lab in your browser |
| 45 | + |
| 46 | +Now you need to open a connection between your local machine and the EC2 instance with the correct port forwarding. On your local machine you can use any unused port; note that if you are already running Jupyter lab/notebooks locally that port 8888 will likely already be in use. This example uses 9889 as the local port |
| 47 | + |
| 48 | +```bash |
| 49 | +ssh -i ~/.ssh/aws_ec2.pem -L 9889:localhost:8888 [email protected] |
| 50 | +``` |
| 51 | + |
| 52 | +and in a browser window on your local machine, access the port you specified before `localhost` above |
| 53 | + |
| 54 | +```bash |
| 55 | +http://localhost:9889 |
| 56 | +``` |
| 57 | + |
| 58 | +You will see a screen that asks for a password, but if you didn't enter any before, you can just go ahead and click `Login`. Now you have access to the tutorial repository, and the tutorials are in the directory `Tutorials_as_Jupyter_Notebooks`. |
| 59 | + |
| 60 | +## Re-connect to Jupyter lab in Docker container |
| 61 | + |
| 62 | +If the Docker container is stopped or exited, the Jupyter lab session will also exit. To restart a Docker container use `docker ps -a` to find the container name and ID, and then use |
| 63 | + |
| 64 | +```bash |
| 65 | +docker start <container-name or id> |
| 66 | +``` |
| 67 | + |
| 68 | +to re-start the container. Then you may need to run the following on your instance to re-start Jupyter lab within the container: |
| 69 | + |
| 70 | +```bash |
| 71 | +docker exec -it <container-name or id> ~/jupyter_lab_start_docker.sh |
| 72 | +``` |
| 73 | + |
| 74 | +and you should see the `ServerApp` and `LabApp` output appear indicating that the session has started. Then you can use `Ctrl-p` `Ctrl-q` to escape that window from the container. |
0 commit comments