Juniper: Package lambda functions

Juniper is a packaging tool to stream and standardize the creation of a zip artifact for a set of AWS Lambda functions.

The zip artifacts generated include the source code of the dependencies defined in a given requirements.txt file as well as any shared libraries the function depends on. With the generated artifact, a developer can deploy a lambda function either manually, through the awscli or using a cloudformation/sam template.

Quickstart

With Python==3.6 and Docker installed, install juniper:

> pip install juniper

In order to package your lambda functions with juniper, you need to create a manifest file.

functions:
  # Name the zip file you want juni to create
  router:
    # The dependencies of the router function.
    requirements: ./src/requirements.txt.
    # Include this file in the generated zip artifact.
    include:
    - ./src/lambda_function.py

The folder structure this manifest refers to looks like:

.
├── manifest.yml
├── src
│   ├── requirements.txt
│   ├── lambda_function.py

Build it!

> juni build

Juniper creates the following artifact ./dist/router.zip 🎉

For a more comprehensive example, please take a look at our tutorial.

Python3.7 and Beyond

By default juniper uses docker containers to package your lambda functions. Behind the scenes, juniper creates a docker-compose file from your manifest. This file is used by the build command to spawn a build container per function definition.

Since the AWS Lambda service supports multiple python runtimes, it makes sense for juniper to give you the ability to specify a docker image. With the following manifest file, you can package the router lambda using a python3.7 image.

functions:
  router:
    # Use this docker image
    image: lambci/lambda:build-python3.7
    requirements: ./src/router/requirements.txt
    # Include these local modules in the artifact
    include:
    - ./src/commonlib/mylib
    - ./src/router_function/router

Keep in mind that not every single docker image works, for more information on the type of images supported read juniper and docker.

Lambda Layers

AWS Lambda layers is a recent service that gives a developer the ability to pre-package a set of dependencies. A lambda function can be built on top of multiple layers, either packaged by the developer, by AWS or by a third party.

To build a layer, the juniper manifest uses a new block:

layers:
  base:
    requirements: ./src/requirements/base.txt
  pg:
    requirements: ./src/requirements/postgres.txt

With this manifest, running juni build creates two layer artifacts. One with the name base and another one named pg. These artifacts are packaged along the lambda definitions and the zip files are stored in the artifacts directory.

The generated zip artifacts include the dependencies defined in the requirements file.

The layer section also supports the definition of a custom docker image. With this feature, a layer can be built using python3.7 and another one can be built using the default python interpreter; python3.6.

layers:
  base:
    image: lambci/lambda:build-python3.7
    requirements: ./src/requirements/base.txt

Juniper builds the artifact for you, you can either use the layers aws cli to upload it to AWS or you can use a SAM template definition. When declaring your layer in the SAM template, make sure you use the AWS::Serverless::LayerVersion resource.

To see an example on how to package lambda functions as well as layers, juniper includes a layers example in the codebase called fondolayers.

layers aws cli: https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html#configuration-layers-manage

Configuration

To update the default configuration of juniper, can use the the global section of the manifest. A sample configuration looks like:

global:
  image: lambci/lambda:build-python3.7
  output: ./build

functions:
  router:
    requirements: ./src/router/requirements.txt.
    include:
    - ./src/router_function/router/lambda_function.py

Setting a docker image at a global level tells juniper to package every lambda function using such image. In this example, the zip artifacts will be stored in the ./build folder instead of the ./dist; which is the default.

Features

This list defines the entire scope of Juniper. Nothing more, nothing else.

• Minimal manifest file to define packaging
• Using docker containers as a way to install dependencies and generate the artifacts
• Ability to tailor the requirements.txt per lambda
• Create an individual zip artifact for multiple lambda functions
• Ability to include shared dependencies (python modules relative to the function being packaged)
• Specify docker image to package lamdba functions using different python runtimes.

Contributing

For guidance on setting up a development environment and how to make a contribution to Juniper, see the contributing guidelines.

Links

• Documentation: https://eabglobal.github.io/juniper/
• License: Apache Software License
• Code: https://github.com/eabglobal/juniper
• Issue tracker: https://github.com/eabglobal/juniper/issues
• Test status:
  • Linux, Mac: https://circleci.com/gh/eabglobal/juniper