Skip to content

Commit

Permalink
Docker image for test and experiment Keras (keras-team#3035)
Browse files Browse the repository at this point in the history
* Docker image for test and experiment Keras

 - Docker image with CUDA support on ubuntu 14.04
 - nvidia-docker script to forward the GPU to the container
 - MakeFile to simplify docker commands for build, run, test, ..etc
 - Add useful tools like jupyter notebook, ipdb, sklearn for experiments

* update nvidia-docker plugin

* use .theanorc in Dockerfile

* Add tensorflow to the docker image

* update Docker image to cuDNN v5

* test fixes

* move docker to sub directory

* README for docker

* Fix typos

* Add visualization to Dockerfile
  • Loading branch information
dosht authored and fchollet committed Jul 27, 2016
1 parent 3726aba commit df84c69
Show file tree
Hide file tree
Showing 4 changed files with 135 additions and 0 deletions.
46 changes: 46 additions & 0 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
FROM nvidia/cuda:7.5-cudnn5-devel

ENV CONDA_DIR /opt/conda
ENV PATH $CONDA_DIR/bin:$PATH

RUN mkdir -p $CONDA_DIR && \
echo export PATH=$CONDA_DIR/bin:'$PATH' > /etc/profile.d/conda.sh && \
apt-get update && \
apt-get install -y wget git libhdf5-dev g++ graphviz && \
wget --quiet https://repo.continuum.io/miniconda/Miniconda3-3.9.1-Linux-x86_64.sh && \
echo "6c6b44acdd0bc4229377ee10d52c8ac6160c336d9cdd669db7371aa9344e1ac3 *Miniconda3-3.9.1-Linux-x86_64.sh" | sha256sum -c - && \
/bin/bash /Miniconda3-3.9.1-Linux-x86_64.sh -f -b -p $CONDA_DIR && \
rm Miniconda3-3.9.1-Linux-x86_64.sh

ENV NB_USER keras
ENV NB_UID 1000

RUN useradd -m -s /bin/bash -N -u $NB_UID $NB_USER && \
mkdir -p $CONDA_DIR && \
chown keras $CONDA_DIR -R && \
mkdir -p /src && \
chown keras /src

USER keras

# Python
ARG python_version=3.5.1
ARG tensorflow_version=0.9.0rc0-cp35-cp35m
RUN conda install -y python=${python_version} && \
pip install https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-${tensorflow_version}-linux_x86_64.whl && \
pip install git+git://github.com/Theano/Theano.git && \
pip install ipdb pytest pytest-cov python-coveralls coverage==3.7.1 pytest-xdist pep8 pytest-pep8 pydot_ng && \
conda install Pillow scikit-learn notebook pandas matplotlib nose pyyaml six h5py && \
pip install git+git://github.com/fchollet/keras.git && \
conda clean -yt

ADD theanorc /home/keras/.theanorc

ENV PYTHONPATH='/src/:$PYTHONPATH'

WORKDIR /src

EXPOSE 8888

CMD jupyter notebook --port=8888 --ip=0.0.0.0

26 changes: 26 additions & 0 deletions docker/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
help:
@cat Makefile

DATA?="${HOME}/Data"
GPU?=0
DOCKER_FILE=Dockerfile
DOCKER=GPU=$(GPU) nvidia-docker
BACKEND=tensorflow
TEST=tests/
SRC=$(shell dirname `pwd`)

build:
docker build -t keras --build-arg python_version=3.5 -f $(DOCKER_FILE) .

bash: build
$(DOCKER) run -it -v $(SRC):/src -v $(DATA):/data --env KERAS_BACKEND=$(BACKEND) keras bash

ipython: build
$(DOCKER) run -it -v $(SRC):/src -v $(DATA):/data --env KERAS_BACKEND=$(BACKEND) keras ipython

notebook: build
$(DOCKER) run -it -v $(SRC):/src -v $(DATA):/data --net=host --env KERAS_BACKEND=$(BACKEND) keras

test: build
$(DOCKER) run -it -v $(SRC):/src -v $(DATA):/data --env KERAS_BACKEND=$(BACKEND) keras py.test $(TEST)

58 changes: 58 additions & 0 deletions docker/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Using Keras via Docker

This directory contains `Dockerfile` to make it easy to get up and running with
Keras via [Docker](http://www.docker.com/).

## Installing Docker

General installation instructions are
[on the Docker site](https://docs.docker.com/installation/), but we give some
quick links here:

* [OSX](https://docs.docker.com/installation/mac/): [docker toolbox](https://www.docker.com/toolbox)
* [ubuntu](https://docs.docker.com/installation/ubuntulinux/)

## Running the container

We are using `Makefile` to simplify docker commands within make commands.

Build the container and start a jupyter notebook

$ make notebook

Build the container and start an iPython shell

$ make ipython

Build the container and start a bash

$ make bash

For GPU support install NVidia drivers (ideally latest) and
[nvidia-docker](https://github.com/NVIDIA/nvidia-docker). Run using

$ make notebook GPU=0 # or [ipython, bash]

Switch between Theano and TensorFlow

$ make notebook BACKEND=theano
$ make notebook BACKEND=tensorflow

Mount a volume for external data sets

$ make DATA=~/mydata

Prints all make tasks

$ make help

You can change Theano parameters by editing `/docker/theanorc`.


Note: If you would have a problem running nvidia-docker you may try the old way
we have used. But it is not recommended. If you find a bug in the nvidia-docker report
it there please and try using the nvidia-docker as described above.

$ export CUDA_SO=$(\ls /usr/lib/x86_64-linux-gnu/libcuda.* | xargs -I{} echo '-v {}:{}')
$ export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}')
$ docker run -it -p 8888:8888 $CUDA_SO $DEVICES gcr.io/tensorflow/tensorflow:latest-gpu
5 changes: 5 additions & 0 deletions docker/theanorc
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
[global]
floatX = float32
optimizer=None
device = gpu

0 comments on commit df84c69

Please sign in to comment.