About The Project

This Docker application was created by factoring out many reusable code artifacts from my various projects over a number of years. Since this work was not a part of a group effort, the test coverage is predictably abysmal 🤨 and Python documentation notably absent 😑. This package takes a submodule dependency on another one of my common packages. While this application is almost entirely boilerplate, it can run as a stand-alone application and serves the basis for any well-behaved Python application. The design is opinionated with the use of ZeroMQ but this is not a strict requirement. This project has a required dependency on 1Password for both build and run time (explained later). If this is unacceptable, you'll need to fork this project or send a pull request for a substitute like Bitwarden or equivalent.

Enough talk! What do I get?

• An Alpine Docker application that is specifically designed to act as a Docker base image for derived applications, or can be forked to run as is. This includes a variety of boilerplate entrypoint scripts that can be trivially overridden.
• Powerful threading and inter-thread data handling functions with significant resilience to unchecked thread death.
• Sample Healthchecks cron job with built-in container setup (you'd think that this would be simple and well documented).
• Pre-configured process control using supervisor.
• Automatic syslog configuration to log to the Docker host rsyslog.
• Support for AWS-CLI if appropriate AWS environment variables are present, like AWS_DEFAULT_REGION.
• Python dependency management using Poetry.

Here is a breakdown of some of the sample application features and structure:

• app.init.py: Sets a global for APP_NAME and WORK_DIR which currently assumes the location /opt/app.
• app.main.py: The naming of this entrypoint is intentional so that derived application containers can easily override this and existing application bootstrapping logic just works^TM. After basic imports, a 1Password connect SDK CredsConfig struct is instantiated which tells pylib which credentials to pull from the 1Password connect server. Next, a variety of pylib imports are done to bring the needed functionality into the application. A ZeroMQ URL is defined called URL_WORKER_APP for inter-thread communication using their "lockless programming" paradigm. DataReader demonstrates the use of the AppThread and Closeable functions which abstract away thread instantiation, thread death tracking and bring the context manager to gracefully handle errors and shutdown. A tenet of ZeroMQ is to not share sockets between threads and is managed for you with this implementation. A few lines of code bring a lot of powerful resilience features. DataReader pretends to fetch some useful data and then uses its ZeroMQ PUSH socket to forward any to whatever has a sink URL to URL_WORKER_APP. EventProcessor illustrates the consumer which is analogous to the main application loop. The main entrypoint instantiates these classes, echoes some environment variable keys to the log that are visible to the application which is useful for debugging environment bootstrap. All threads wait on a Python threading.Event object for quick shutdown should the signal arrive. When set, the main application and all non-daemon threads have a chance to complete, aided by a helper. Another helper routine zmq_term helps with getting ZeroMQ shutdown done. By design ALL ZeroMQ sockets must be closed gracefully in order for the application to exit. Any non-trivial applications need active management of sockets to avoid spending hours working out why the application shutdown is blocked. This is one of a few powerful features of this application and package.

(back to top)

Built With

Technologies that help make this package useful:

Also:

• Cronitor
• Healthchecks
• MessagePack

• Botoflow

(back to top)

Getting Started

Here is some detail about the intended use of this package.

Prerequisites

Beyond the Python dependencies defined in the Poetry configuration, the package init carries hardcoded dependencies on Sentry and 1Password in order to function. Unless you want these and are effectively extending my base project, you're likely better off forking this package and cutting out what you do not need.

Required Tools

Install these tools and make sure that they are on the environment $PATH.

• task for project build orchestration: https://taskfile.dev/installation/#install-script

• docker and docker-compose for container builds and execution: https://docs.docker.com/engine/install/

• mvn Maven for Java build orchestration: https://maven.apache.org/download.cgi

• poetry for Python dependency management: https://python-poetry.org/docs/#installation

• java and javac for Java build and runtime: https://aws.amazon.com/corretto/

• python is python3 for Python runtime: https://www.python.org/downloads/

Installation

0. 🛑 This project uses 1Password Secrets Automation to store both application key-value pairs as well as runtime secrets. It is assumed that the connect server containers are already running on your environment. If you do not want to use this, then you'll need to fork this package and make the changes as appropriate. It's actually very easy to set up, but note that 1Password is a paid product with a free-tier for secrets automation. Here is an example of how this looks for my application and the generation of the docker-compose.yml relies on this step. Your secrets automation vault must contain an entry called ENV.base_app with these keys:

• DEVICE_NAME: For naming the container. This project uses base-app.
• APP_NAME: Used for referencing the application's actual name for the logger. This project uses base_app.
• OP_CONNECT_HOST, OP_CONNECT_TOKEN, OP_CONNECT_VAULT: Used to specify the URL of the 1Password connect server with associated client token and Vault ID. See 1Password for more.
• HC_PING_URL: Healthchecks URL of this application's current health check status.

With these configured, you are now able to build the application.

In addition to this, additional runtime configuration is used by the application, and also need to be contained within the secrets vault. With these configured, you are now able to run the application.

1. Clone the repo

   git clone https://github.com/tailucas/base-app.git

2. Verify that the git submodule is present.

   git submodule init
   git submodule update

3. Make the Docker runtime user and set directory permissions. ✋ Be sure to first review the Makefile contents for assumptions around user IDs for Docker.

   make user

4. Now generate the docker-compose.yml:

   make setup

5. And generate the Docker image:

   make build

6. If successful and the local environment is running the 1Password connect containers, run the application. For foreground:

   make run

   For background:

   make rund

(back to top)

Usage

I have various projects that extend this Docker application. This Base Project serves as my Docker base image from which other projects are derived. While I may briefly run it locally in order to get basic functions, it it usually built, tagged and pushed to Docker so that the other applications can extend the functionality as needed.

(back to top)

License

Distributed under the MIT License. See LICENSE for more information.

(back to top)

Acknowledgments

• Template on which this README is based
• All the Shields

(back to top)