2023-02-28

Docker for development of the colrc-v2

This tutorial is for those who are contributing code to the Coeur d'Alene Online Language Resource Center (COLRC). Our production site is at https://thecolrc.org

Getting Started on a Windows machine

If you are running Windows, you'll need to use WSL with Debian.
WSL is only fully supported on Windows10's education edition or better, not on the home edition. If you have the home edition and are a University of Arizona student, you can upgrade to the education edition at no additional charge.

After installing WSL/Debian, you'll work at the Debian command line primarily, which you can access by double-clicking the Debian icon in your toolbar.

When you use WSL/Debian, you and your files will live in a distinct mount (mnt) on your computer. If you type pwd at the command line, you'll see the directory you're in, which will probably look something like this:

home/[yourname]

at that point, you can either create a directory that you want to keep all of our project files (for example, Amy uses a directory called src, which has a subdirectory for the colrc materials), or just build our repository at the root.

you will also want to be sure you configure Docker Desktop to connect with WSL

Getting Started on a Mac or Linux machine

Mac and Linux users should use the terminal application that's native to their OS. For the most part, these systems require no special configuration beyond the default.

For all systems

Install these applications in the appropriate format for your machine.

You will need sudo/root access on your system at the command line.
docker
docker-compose
git
python3
node
We recommend using VSCode as your code editor for this project.

First Installation

You will do these steps at a command line - windows users will use the WSL/Debian command lime; Mac and Linux users will use your regular terminal application.

Create and/or switch to the directory where you want our application to live. Then, from the command line in that directory, clone or pull this repository:

git clone https://github.com/arizona-linguistics/colrc-v2
Afterward, change to the newly-created directory and pull to make sure you have all of the current changes to the repository. Note that the default branch, "main," is the branch you should clone and/or pull.

cd colrc-v2 && git pull
Create the external directories needed to run Odinson: Move up one directory, cd .., and create a new directory named 'data' mkdir data.

Move into the new directory, cd data, and make another directory called 'odinson', mkdir odinson.

Move back to the root of the colrc-v2 directory, cd .. to move up one level, then cd colrc-v2 to move into the root of the colrc-v2 dirctory.
Create your docker override yml file.

The purpose of this file is to tell our application where your external Odinson directory lives, so that it can map a drive to that directory. The override file will not be copied back into the public git repo, as it will have information in it that's specific to you.

You will make a new file at the command line: touch docker-compose.override.yml, OR open your VS Code editor from the command line by typing code .. This command will open VSCode in the directory where you are currently working, so you'll be able to see all the files there and manipulate them. If you open VSCode, you can right or control-click on the Explorer view and select 'new text file', and save the new file as docker-compose.override.yml.

No matter how you created the file, copy and paste the following code block into the file. Then change the [yourfilepath] to match your path to the data/odinson directory. If you're not sure what yourfilepath is, you can type pwd (print working directory) at the command line to reveal your current location.
```
services:
odinson-rest-api:
    volumes: 
    - /[yourfilepath]/data/odinson:/data/odinson
```
At the command line, build our development environment. Depending on your configuration, you may or may not need to sudo The initial build may take a while, but subsequent builds will go faster.

docker compose -f docker-compose.yml -f docker-compose.override.yml up --build
Once the build has finished, download our image/audio files from Dropbox. As files are updated in our Dropbox folder, you can run the script below while the development environment is down to keep your local filesystem up to date.

If you do not have the requests library, you will need to install it:
pip3 install requests or python3 -m pip install requests If you already have it or you have finished installing requests, you can run the script using the command below: python3 misc/dropbox-sync.py If you get a permissions error when running the script, you can use the command below to fix the permissions you need (if the file/folder that has the permissions issue is different, substitute file_data with that file/folder):
sudo chown -R $USER file_data
Then you may finally start our development environment as a background process!

docker compose -f docker-compose.yml -f docker-compose.override.yml up

Note that it may take a tiny bit after the command has completed in order for the environment to be fully up and running. To see if it is ready to go, check http://localhost:3000 and make sure you can see the website before proceeding!
When you want to bring the system down, you can either use control-C from the terminal where the application is running; or use the 'down' button to the right of the container in Docker Desktop's gui, or you can open a new terminal, navigate to the root of the project, and use this command:

docker compose down

To relaunch for a new work session, if you haven't done a new pull from the repo, you can just 'up' the system without rebuilding it like this:

docker compose -f docker-compose.yml -f docker-compose.override.yml up

To relaunch after a new pull or significant local changes to i.e. the backend, you can build and then up like this:

docker compose -f docker-compose.yml -f docker-compose.override.yml up --build

Trouble-shooting

metadata error on build

The error might look like this: rpc error failed to load metadata [some library] This can happen in unpredictable environments, and means that there's a network timeout that is preventing stable connections to the docker.io libraries.

You can prepend the docker compose command with DOCKER_BUILDKIT=0:

DOCKER_BUILDKIT=0 docker compose -f docker-compose.yml -f docker-compose.override.yml up --build

hasura exits with code 137

This is an indication that docker needs more memory allocated to it.
The error is most likely to occur on a Mac or Linux machine than on a Windows/WSL installation, because WSL and Docker work together to manage resource allocation automatically.

If hasura exits with code 137, that means that it's run out of memory available to docker. Go to the docker desktop > settings > resources tab and increase the memory allocation as much as you can.

Then you can try the upping process again, and (hopefully) hasura will be happy.