feature/EVG-191: Fix server not listening on the correct interface (#57)

* split up entrypoint into separate scripts for each env, use roarr logging framework

* bind on 0.0.0.0 to listen on all interfaces

* run npm in test and ci environments in shell, remove production configs

* remove prod entrypoint

* simply docker-compose exec syntax for CI

* add -T to exec command

* convert more console.log to debugging framework

* try only running functional tests

* use sh to run tests in CI

* convert more console.log to debugging framework

* convert more console.log to debugging framework

* try only running unit tests

* print free memory between test runs

* run tests outside of bin/sh

* wait for app to be running before executing tests

* print docker-compose service status and run wait4ports in separate container

* use entrypoints scripts to wait for services, start app detached using run in CI

* rename prod.sh to ci-after-db.sh and ci.sh to ci-after-app.sh

* run app in CI with NODE_ENV=production

* run app and tests in different containers in CI to avoid hostname overrides

* expose app environment variables to app env in CI

* run migrations in ci-after-db.sh

* give app more time to start in CI

* start tests in CI without recreating app container

* give the ephemeral app container a fixed name in CI

* start app in CI using up -d

* add command npm start to app in CI

.dockerignore

@@ -1 +1,3 @@
 node_modules
+.git
+Dockerfile

.github/workflows/workflow.yaml

@@ -22,10 +22,11 @@ jobs:
               run: docker-compose -f docker-compose.yml -f docker-compose.ci.yml build --build-arg NODE_ENV=development --parallel everglot-app
             - name: Start database
               run: docker-compose -f docker-compose.yml -f docker-compose.ci.yml up -d everglot-db
-            - name: Migrate database
-              run: docker-compose -f docker-compose.yml -f docker-compose.ci.yml run --entrypoint "npm run migrate up" everglot-app
+            # Run app as if it were in production.
+            - name: Migrate database and start app
+              run: docker-compose -f docker-compose.yml -f docker-compose.ci.yml up -d everglot-app
             - name: Run tests
-              run: docker-compose -f docker-compose.yml -f docker-compose.ci.yml run everglot-app
+              run: docker-compose -f docker-compose.yml -f docker-compose.ci.yml run --entrypoint entrypoints/ci-after-app.sh everglot-app-ci npm run test:pretty
     build:
         name: Build, Push and Deploy
         runs-on: ubuntu-latest

Dockerfile

@@ -10,20 +10,27 @@ RUN set -eux \
     	nodejs \
         yarn \
         git \
+        wait4ports \
         --repository=http://dl-cdn.alpinelinux.org/alpine/edge/community
 
-# Restrict app privileges.
-USER node
-RUN mkdir -p /home/node/app
-WORKDIR /home/node/app
-
 # Pass NODE_ENV as a build arg to change the NPM dependencies.
 # If NODE_ENV is development the image will also include dev dependencies.
 #ARG NODE_ENV=production
 # TODO: Default production instead (as soon as we can install prod dependencies without dev dependencies).
 ARG NODE_ENV=development
 ENV NODE_ENV=$NODE_ENV
 
+RUN echo "$NODE_ENV"
+
+RUN set -eux\
+    & [ "$NODE_ENV" != "development" ] || \
+        (npm i -g @roarr/cli)
+
+# Restrict app privileges.
+USER node
+RUN mkdir -p /home/node/app
+WORKDIR /home/node/app
+
 # Configure project dependencies.
 COPY --chown=node:node package*.json ./
 
@@ -52,4 +59,4 @@ RUN npm run build
 #            npm ci)
 
 EXPOSE 3000
-CMD [ "node", "__sapper__/build" ]
+CMD [ "npm", "start" ]

README.md

@@ -4,13 +4,13 @@ Everglot web application based on Express.js, Svelte/Sapper, Socket.io, PostgreS
 
 ## Configuration
 
-If you are going to use `docker-compose`, create an `.env` file in the same directory as this file. Otherwise set the following environment variables through some other means.
+If you are going to use `docker-compose` (highly recommended), create a file called `.env` in the same directory as this readme. Otherwise set the following environment variables through some other means.
 
 ```bash
 POSTGRES_USER=everglot_app_user
 POSTGRES_PASSWORD=everglot_app_pass
 POSTGRES_DB=everglot_app_db
-SESSION_COOKIE_VALIDATION_SECRETS=["SomeVeryLongRandomSecret"]
+SESSION_COOKIE_VALIDATION_SECRETS=["SomeVeryLongRandomSecret123"]
 ```
 
 In development you should keep the first three variables as they are. Definitely change the cookie validation secret(s).
@@ -40,14 +40,20 @@ Files from this directory are automatically mirrored. Sometimes you may find you
 
 You will likely want to run NPM commands outside of the containers. Install Node.js [through your package manager](https://nodejs.org/en/download/package-manager/) or with NVM (Node version manager).
 
-#### Dependencies
+### Dependencies
 
 Then install this project's development dependencies locally using
 
 ```bash
 npm i --also=dev
 ```
 
+It is also highly recommended you install the Roarr CLI globally so that you can prettify log outputs when necessary. This is done by default within our Docker images for development and testing but you may want this if you interact with production servers.
+
+```bash
+npm i -g @roarr/cli
+```
+
 ### Database Schema
 
 Before the app is fully operationable, the database schema must be updated by running all missing migrations. Migrations are essentially code describing how the database should be changed.
@@ -118,15 +124,51 @@ Note: PostGraphile Subscriptions are not configured, yet (2021-03-17).
 
 The `docker-compose.test.yml` configuration is designed for running the application tests locally, whereas `docker-compose.ci.yml` is for running them in CI environments.
 
+### Local execution
+
 It is important that you run commands regarding the testing environment in a different `docker-compose` project with the `-p` flag. That way your development containers for the app and for the database are not re-used for testing. The tests delete and add data to your database which you probably do not want during development.
 
-To execute the tests locally run:
+First start the database:
+
+```bash
+docker-compose -f docker-compose.yml -f docker-compose.test.yml -p test up -d everglot-db
+```
+
+To only run unit tests:
+
+```bash
+docker-compose -f docker-compose.yml -f docker-compose.test.yml -p test exec everglot-app npm run test src/__tests__/unit
+```
+
+To run functional tests the app must be running separately as well. This command can also be left out by simply not passing the `everglot-db` service in the `up -d` command above.
 
 ```bash
-docker-compose -f docker-compose.yml -f docker-compose.test.yml -p test run --entrypoint "npx jest --forceExit" everglot-app
+docker-compose -f docker-compose.yml -f docker-compose.test.yml -p test up -d everglot-app
 ```
 
-`--forceExit` is currently necessary because something is still running somewhere which prevents the tests from exiting normally within 1 second.
+Run all tests:
+
+```bash
+docker-compose -f docker-compose.yml -f docker-compose.test.yml -p test exec everglot-app npm run test
+```
+
+Only run functional tests:
+
+```bash
+docker-compose -f docker-compose.yml -f docker-compose.test.yml -p test exec everglot-app npm run test src/__tests__/functional
+```
+
+Jest's `--forceExit` option (as defined in the `test` script within `package.json`) is currently necessary because something is still running somewhere which prevents the tests from exiting normally within 1 second.
+
+### In CI
+
+CI environments don't need a separate project and should use the `docker-compose.ci.yml` configuration with `entrypoints/wait-for-db.sh` for setup and `entrypoints/wait-for-app.sh` for testing.
+
+```bash
+docker-compose -f docker-compose.yml -f docker-compose.ci.yml up -d everglot-db
+docker-compose -f docker-compose.yml -f docker-compose.ci.yml run -d --entrypoint entrypoints/wait-for-db.sh everglot-app
+docker-compose -f docker-compose.yml -f docker-compose.ci.yml run --entrypoint entrypoints/wait-for-app.sh everglot-app npm run test:pretty
+```
 
 ## Deployment
 

docker-compose.ci.yml

@@ -2,9 +2,33 @@ version: "3"
 
 services:
     everglot-app:
-        entrypoint: "npm run test -- --forceExit"
+        environment:
+            NODE_ENV: "production"
+            ROARR_LOG: "true"
+            HOST: "0.0.0.0"
+            PORT: "3000"
+        entrypoint: entrypoints/ci-after-db.sh
+        command: ["npm", "start"]
+        # Avoid building the same image twice.
+        image: everglot-app-image
+
+    everglot-app-ci:
         environment:
             NODE_ENV: "development"
+            ROARR_LOG: "true"
+            PGUSER: ${POSTGRES_USER:?Please set POSTGRES_USER}
+            PGPASSWORD: ${POSTGRES_PASSWORD:?Please set POSTGRES_PASSWORD}
+            PGDATABASE: ${POSTGRES_DB:?Please set POSTGRES_DB}
+            PGHOST: ${PGHOST:-everglot-db}
+            PGPORT: ${PGPORT:-5432}
+            DATABASE_URL: "postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${PGHOST:-everglot-db}:${PGPORT}/${POSTGRES_DB}"
+            SESSION_COOKIE_VALIDATION_SECRETS: ${SESSION_COOKIE_VALIDATION_SECRETS:?Please set SESSION_COOKIE_VALIDATION_SECRETS}
+        entrypoint: entrypoints/ci-after-app.sh
+        # Avoid building the same image twice.
+        image: everglot-app-image
+        depends_on:
+            - everglot-app
+
     everglot-db:
         environment:
             POSTGRES_USER: "everglot_app_user"

docker-compose.dev.yml

@@ -2,14 +2,19 @@ version: "3"
 
 services:
     everglot-app:
-        entrypoint: "npm run dev"
         volumes:
             - ./:/home/node/app
         ports:
             - "127.0.0.1:8002:3000"
             - "127.0.0.1:10000:10000"
         environment:
             NODE_ENV: "development"
+            ROARR_LOG: "true"
+            HOST: "0.0.0.0"
+            PORT: "3000"
+        entrypoint: entrypoints/dev.sh
+        command: ["/bin/sh", "-c", "npm run dev | roarr pretty-print"]
+
     everglot-db:
         environment:
             POSTGRES_USER: "everglot_app_user"

docker-compose.test.yml

@@ -2,11 +2,16 @@ version: "3"
 
 services:
     everglot-app:
-        entrypoint: "npm run test -- --forceExit"
         volumes:
             - ./:/home/node/app
         environment:
             NODE_ENV: "development"
+            ROARR_LOG: "true"
+            HOST: "0.0.0.0"
+            PORT: "3000"
+        entrypoint: entrypoints/test.sh
+        command: ["/bin/sh", "-c", "npm run dev | roarr pretty-print"]
+
     everglot-db:
         environment:
             POSTGRES_USER: "everglot_app_user"

docker-compose.yml

@@ -7,6 +7,8 @@ services:
             context: .
         restart: always
         working_dir: /home/node/app
+        depends_on:
+            - everglot-db
         environment:
             PGUSER: ${POSTGRES_USER:?Please set POSTGRES_USER}
             PGPASSWORD: ${POSTGRES_PASSWORD:?Please set POSTGRES_PASSWORD}

entrypoints/ci-after-app.sh

@@ -0,0 +1,7 @@
+#!/bin/sh
+set -e
+
+wait4ports -t 3 "tcp://everglot-db:5432"
+wait4ports -t 10 "tcp://everglot-app:3000"
+
+exec "$@"

entrypoints/ci-after-db.sh

@@ -0,0 +1,8 @@
+#!/bin/sh
+set -e
+
+wait4ports -t 10 "tcp://everglot-db:5432"
+
+npm run migrate up
+
+exec "$@"

entrypoints/dev.sh

@@ -0,0 +1,10 @@
+#!/bin/sh
+set -e
+
+wait4ports "tcp://everglot-db:5432"
+
+# Note: Migrations should be run manually in development!
+# Run them using:
+# PGHOST=localhost PGDATABASE=everglot_app_db PGUSER=everglot_app_user PGPASSWORD=everglot_app_pass docker-compose -f docker-compose.yml -f docker-compose.dev.yml exec everglot-app "npm run migrate up"
+
+exec "$@"

entrypoints/test.sh

@@ -0,0 +1,8 @@
+#!/bin/sh
+set -e
+
+wait4ports -t 5 "tcp://everglot-db:5432"
+
+npm run migrate up
+
+exec "$@"