Skip to content

Commit

Permalink
Eliminate GPU build; remove explicit TF dep., replace with checks in …
Browse files Browse the repository at this point in the history
…__init__.py

PiperOrigin-RevId: 219516463
  • Loading branch information
csuter authored and tensorflower-gardener committed Oct 31, 2018
1 parent 40fde73 commit 16ec0e2
Show file tree
Hide file tree
Showing 5 changed files with 91 additions and 59 deletions.
11 changes: 6 additions & 5 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,16 +46,17 @@ To run existing unit-tests on CPU, use the command:


```shell
bazel test --copt=-O3 --copt=-march=native \
//tensorflow_probability/...
bazel test --copt=-O3 --copt=-march=native //tensorflow_probability/...
```

from the root of the `tensorflow_probability` repository. To run tests on GPU,
use the command:
you just need to ensure the GPU-enabled version of TensorFlow is installed.
However, you will also need to include the flag `--jobs=1`, since by default
Bazel will run many tests in parallel, and each one will try to claim all the
GPU memory:

```shell
bazel test --copt=-O3 --copt=-march=native --config=cuda \
//tensorflow_probability/...
bazel test --jobs=1 --copt=-O3 --copt=-march=native //tensorflow_probability/...
```


Expand Down
29 changes: 12 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,20 +121,15 @@ To install the latest version, run the following:
pip install --user --upgrade tensorflow-probability # depends on tensorflow (CPU-only)
```

TensorFlow Probability depends on a recent stable release of TensorFlow
(pip package `tensorflow`); see [TFP release notes](
https://github.com/tensorflow/probability/releases) for details on the latest
version of TensorFlow Probability, and the version of TensorFlow it depends on.
TensorFlow Probability depends on a recent stable release of
[TensorFlow](https://www.tensorflow.org/install) (pip package `tensorflow`). See
the [TFP release notes](https://github.com/tensorflow/probability/releases) for
details about dependencies between TensorFlow and TensorFlow Probability.

We also provide a GPU-enabled package:

```shell
pip install --user --upgrade tensorflow-probability-gpu # depends on tensorflow-gpu
```

Currently, TensorFlow Probability does not contain any GPU-specific code. The
primary difference between these packages is that `tensorflow-probability-gpu`
depends on a GPU-enabled version of TensorFlow.
Note: Since TensorFlow is *not* included as a dependency of the TensorFlow
Probability package (in `setup.py`), you must explicitly install the TensorFlow
package (`tensorflow` or `tensorflow-gpu`). This allows us to maintain one
package instead of separate packages for CPU and GPU-enabled TensorFlow.

To force a Python 3-specific install, replace `pip` with `pip3` in the above
commands. For additional installation help, guidance installing prerequisites,
Expand All @@ -143,10 +138,10 @@ installation guide](https://www.tensorflow.org/install).

### Nightly Builds

We also release nightly builds, under the pip packages `tfp-nightly` and
`tfp-nightly-gpu`; these depend on `tf-nightly` and `tf-nightly-gpu`,
respectively. These builds include newer features, but may be less stable than
our versioned releases.
There are also nightly builds of TensorFlow Probability under the pip package
`tfp-nightly`, which depends on one of `tf-nightly` and `tf-nightly-gpu`.
Nightly builds include newer features, but may be less stable than the versioned
releases.

### Installing from Source

Expand Down
20 changes: 2 additions & 18 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,37 +33,21 @@
'numpy >= 1.13.3',
]

REQUIRED_TENSORFLOW_VERSION = '1.10.0'

if '--gpu' in sys.argv:
use_gpu = True
sys.argv.remove('--gpu')
else:
use_gpu = False

if '--release' in sys.argv:
release = True
sys.argv.remove('--release')
else:
# Build a nightly package by default.
release = False

maybe_gpu_suffix = '-gpu' if use_gpu else ''

if release:
project_name = 'tensorflow-probability' + maybe_gpu_suffix
tensorflow_package_name = 'tensorflow{}>={}'.format(
maybe_gpu_suffix, REQUIRED_TENSORFLOW_VERSION)
project_name = 'tensorflow-probability'
else:
# Nightly releases use date-based versioning of the form
# '0.0.1.dev20180305'
project_name = 'tfp-nightly' + maybe_gpu_suffix
project_name = 'tfp-nightly'
datestring = datetime.datetime.now().strftime('%Y%m%d')
__version__ += datestring
tensorflow_package_name = 'tf-nightly{}'.format(
maybe_gpu_suffix)

REQUIRED_PACKAGES.append(tensorflow_package_name)


class BinaryDistribution(Distribution):
Expand Down
58 changes: 58 additions & 0 deletions tensorflow_probability/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,64 @@
# Contributors to the `python/` dir should not alter this file; instead update
# `python/__init__.py` as necessary.


# We need to put some imports inside a function call below, and the function
# call needs to come before the *actual* imports that populate the
# tensorflow_probability namespace. Hence, we disable this lint check throughout
# the file.
#
# pylint: disable=g-import-not-at-top


# Ensure TensorFlow is importable and its version is sufficiently recent. This
# needs to happen before anything else, since the imports below will try to
# import tensorflow, too.
def _ensure_tf_install(): # pylint: disable=g-statement-before-imports
"""Attempt to import tensorflow, and ensure its version is sufficient.
Raises:
ImportError: if either tensorflow is not importable or its version is
inadequate.
"""
try:
import tensorflow as tf
except ImportError:
# Re-raise with more informative error message.
raise ImportError(
"Failed to import TensorFlow. Please note that TensorFlow is not "
"installed by default when you install TensorFlow Probability. This is "
"so that users can decide whether to install the GPU-enabled "
"TensorFlow package. To use TensorFlow Probability, please install the "
"most recent version of TensorFlow, by following instructions at "
"https://tensorflow.org/install.")

import distutils.version

#
# Update this whenever we need to depend on a newer TensorFlow release.
#
required_tensorflow_version = "1.11.0"

if (distutils.version.LooseVersion(tf.__version__) <
distutils.version.LooseVersion(required_tensorflow_version)):
raise ImportError(
"This version of TensorFlow Probability requires TensorFlow "
"version >= {required}; Detected an installation of version {present}. "
"Please upgrade TensorFlow to proceed.".format(
required=required_tensorflow_version,
present=tf.__version__))


_ensure_tf_install()


# Cleanup symbols to avoid polluting namespace.
import sys as _sys
for symbol in ["_ensure_tf_install", "_sys"]:
delattr(_sys.modules[__name__], symbol)


# from tensorflow_probability.google import staging # DisableOnExport
from tensorflow_probability.python import * # pylint: disable=wildcard-import
from tensorflow_probability.python.version import __version__
# pylint: enable=g-import-not-at-top
32 changes: 13 additions & 19 deletions tensorflow_probability/g3doc/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,24 +5,18 @@
Install the latest version of TensorFlow Probability:

<pre class="devsite-terminal devsite-click-to-copy prettyprint lang-shell">
pip install --upgrade tensorflow-probability # depends on tensorflow (CPU-only)
pip install --upgrade tensorflow-probability
</pre>

TensorFlow Probability depends on a recent stable release of TensorFlow
(pip package `tensorflow`), see the
[TFP release notes](https://github.com/tensorflow/probability/releases) for
details about the latest version of TensorFlow Probability and the version of
TensorFlow it depends on.
TensorFlow Probability depends on a recent stable release of
[TensorFlow](https://www.tensorflow.org/install) (pip package `tensorflow`). See
the [TFP release notes](https://github.com/tensorflow/probability/releases) for
details about dependencies between TensorFlow and TensorFlow Probability.

There is also a GPU-enabled package:

<pre class="devsite-terminal devsite-click-to-copy prettyprint lang-shell">
pip install --upgrade tensorflow-probability-gpu # depends on tensorflow-gpu
</pre>

Currently, TensorFlow Probability does not contain any GPU-specific code. The
primary difference between these packages is that `tensorflow-probability-gpu`
depends on a GPU-enabled version of TensorFlow.
Note: Since TensorFlow is *not* included as a dependency of the TensorFlow
Probability package (in `setup.py`), you must explicitly install the TensorFlow
package (`tensorflow` or `tensorflow-gpu`). This allows us to maintain one
package instead of separate packages for CPU and GPU-enabled TensorFlow.

To force a Python 3-specific install, replace `pip` with `pip3` in the above
commands. For additional installation help, guidance installing prerequisites,
Expand All @@ -31,10 +25,10 @@ installation guide](https://www.tensorflow.org/install).

## Nightly builds

There are also nightly builds of TensorFlow Probability under the pip packages
`tfp-nightly` and `tfp-nightly-gpu`, which depend on `tf-nightly` and
`tf-nightly-gpu`, respectively. These builds include newer features, but may be
less stable than the versioned releases.
There are also nightly builds of TensorFlow Probability under the pip package
`tfp-nightly`, which depend on one of `tf-nightly` and `tf-nightly-gpu`. Nightly
builds include newer features, but may be less stable than the versioned
releases.

## Install from source

Expand Down

0 comments on commit 16ec0e2

Please sign in to comment.