There are several dockerfiles to configure ONNX Runtime against different platform/runtime, but it's not a complete list, please refer to Docker Containers for ONNX Runtime to create Dockerfile for your own environment.
Dockerfiles
- CPU: Dockerfile, Instructions
- CUDA + CUDNN: Dockerfile, Instructions
- CPU from Source: Dockerfile
- CUDA + CUDNN from Source: Dockerfile
Published Docker Hub Images
Use docker pull with any of the images and tags below to pull an image.
| Build Flavor | Base Image | Docker Image tags | Latest |
|---|---|---|---|
| Default CPU | autodeployai/ai-serving | :0.9.0 | :latest |
| Default GPU | autodeployai/ai-serving | :0.9.0-cuda | :latest-cuda |
Use JAVA_OPTS environment
All docker images can use JAVA_OPTS to specify extra JVM options when starting JVM in a container, for example:
docker run -e JAVA_OPTS="-Xms1g -Xmx2g" ...
The JVM will be started with 1g amount of memory and will be able to use a maximum of 2g amount of memory.
Ubuntu 16.04, CPU (OpenMP), Java Bindings
- Build the docker image from the Dockerfile in this repository.
docker build -t ai-serving-cpu -f Dockerfile .
- Run the Docker image
docker run -it -v {local_writable_directory}:/opt/ai-serving -p {local_http_port}:9090 -p {local_grpc_port}:9091 ai-serving-cpu
Ubuntu 16.04, CUDA 10.1 and cuDNN 7.6.5
- Build the docker image from the Dockerfile in this repository.
docker build -t ai-serving-cuda -f Dockerfile.cuda .
- Run the Docker image
docker run -it -v {local_writable_directory}:/opt/ai-serving -p {local_http_port}:9090 -p {local_grpc_port}:9091 ai-serving-cuda