Skip to content

Commit

Permalink
Merge branch 'main' into datagen
Browse files Browse the repository at this point in the history
  • Loading branch information
stetef committed Mar 15, 2021
2 parents 18f4fac + 54d32c0 commit 073ceda
Show file tree
Hide file tree
Showing 25 changed files with 6,436 additions and 1,248 deletions.
4 changes: 1 addition & 3 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ language: python
matrix:
include:
- python: 3.6
- python: 3.7
- python: 3.8

# what branches should be evaluated
branches:
Expand All @@ -31,7 +29,7 @@ install:

# a list of commands to run before the main script
before_script:
- flake8 codebase
- flake8 gandy

# the actual commands to run
script:
Expand Down
56 changes: 51 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,63 @@
# GANdy
This repository estimates uncertainty using GANs and other machine learning models such as GPs and BNNs.
# GANdy [![Build Status](https://travis-ci.org/GANdy-team/GANdy.svg?branch=main)](https://travis-ci.org/GANdy-team/GANdy)
Automatically creating and comparing supervised machine learning models capable of returning uncertainty estimates in addition to predictions.

__Current Functionality__:
- [ ] Instantialize, train, and use uncertainty models
- - [ ] Gaussian Processes
- - [ ] Bayesian Neural Networks
- - [ ] uncertainty GANs
- [ ] Judge the quality of produced uncertainties with uncertainty metrics
- [ ] Model optimization to uncertainty metrics
- [ ] Comparison of model structures

## Installation
In console, execute the following command where <code>package_path</code> is the path to the folder containing this Readme (GANdy):
> <code>pip install package_path</code>
>
It can then be imported on the installed environment as <code>gandy</code>.
## Repo structure
```
GANdy
-----
setup.py # package installation
environment.yml # development environment
examples/
|-GPs_Showcase.ipynb # demo of gaussian processes as an uncertainty model
gandy/
|-tests/
|-models/
| |-models.py # package parent model class
| |-bnns.py # bayesian neural nets as an uncertainty model
| |-dcgan.py # helper functions for GANs
| |-gans.py # GANs as an uncertainty model
| |-gps.py # gaussian processes as an uncertainty model
|-quality_est/
| |-metrics.py # tools for evaluating returned uncertainties and predictions
```

## Justification
For a supervised machine learning task, one generally obtains deterministic predictions of a target variable based on a learned relationship between that target and a set of features. Such models make predictions on new quantities idependant of known variability or lack of knowledge, and there is no idication of the quality of a prediction. For many tasks, where the target variable is sensative to small changes, it is important not only to have a prediction but also the uncertainty associated with the prediction, in order to inform prediction costs.

Some models already exist that can approximate the uncertainty of a prediction, such as Gaussian Processes or Bayesian models, which have their own downsides including training cost. Recently (2020), it has been shown by Lee and Seok \[1\] that the relatively new architecture Generative Adversarial Networks (GAN) can formatted to produce distributions of a target conditions on features. Here, they invert the input and output of a traditional conditional GAN (cGAN) in order to make target predictions with uncertainty.

It is desirable to have a tool in python that allows for the formulation of such uncertainty GANs, but also a comparison with other tools capable of predicting uncertainty. GANdy aims to incorporate these different tools and allow for automated optimization and comparison such that a model ideal for a task's cost to quality ratio can be identified.

\[1\] M. Lee and J. Seok, “Estimation with Uncertainty via Conditional Generative Adversarial Networks.” ArXiv 2007.00334v1

## Examples
See <examples> for demonstrations on predicting uncertainties with the available tools.

## For developers
To install the development environment <code>conda env create --file devenv.yml</code>.
If any new installed development dependancies, add them to the devenv.yml environment file by Manually adding the dependency, eg.
### Installation
To install the development environment <code>conda env create --file environment.yml</code>.
If any new installed development dependancies, add them to the environment.yml environment file by Manually adding the dependency, eg.
> \- python=3.6.*
To update dev environment with new dependencies in the .yml file, <code>conda env update --file devenv.yml</code>
To update dev environment with new dependencies in the .yml file, <code>conda env update --file environment.yml</code>

./working/ is a workspace for notebooks/testing. It will be ignored by git by default, and will be removed upon release. To specifically "save" your files to git or to share work with other developers, use <code>git add --force working</code>.

### Testing
Tests located at <gandy/tests>
6 changes: 4 additions & 2 deletions environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,10 @@ dependencies:
- numpy=1.19.2
- pandas=1.1.3
- pip=20.3.3
- python=3.6.*
- scikit-learn=0.23.2
- scipy=1.5.2
- tensorflow=2.0.0
- optuna=2.5.0
- pip:
- tensorflow
- tensorflow-probability
- deepchem==2.5.0.dev20210311184720
Loading

0 comments on commit 073ceda

Please sign in to comment.