Skip to content

Commit

Permalink
Improved documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
EntilZha committed Jun 6, 2016
1 parent f9e33ff commit efc7b31
Show file tree
Hide file tree
Showing 4 changed files with 55 additions and 37 deletions.
22 changes: 17 additions & 5 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,23 @@
# Changelog
## Next Release

## Release 0.7.0
### New Features
* Re-architected code to allow for `pseq` to exist
* Implemented `pseq` by implementing `ParallelStream` and `ParallelExecutionEngine`
* `map`, `select`, `filter`, `filter_not`, `where`, `flatten`, and `flat_map` parallelized
* Many Thanks to `versae` for implementing the `pseq` feature!
* Cartesian product from `itertools.product` implemented as `cartesian`
* Auto parallelization by using `pseq` instead of `seq`. Details at https://github.com/EntilZha/PyFunctional/issues/47
* Parallel functions: `map`, `select`, `filter`, `filter_not`, `where`, `flatten`, and `flat_map`
* Compressed file IO support for `gzip`/`lzma`/`bz2` as detailed at https://github.com/EntilZha/PyFunctional/issues/54
* Cartesian product from `itertools.product` implemented as `Pipeline.cartesian`
* Website at [pyfunctional.org](http://www.pyfunctional.org) and docs at [docs.pyfunctional.org](http://docs.pyfunctional.org)

### Bug Fixes
* No option for encoding in `to_json` https://github.com/EntilZha/PyFunctional/issues/70

### Internal Changes
* Pinned versions of all dependencies

### Contributors
* Thanks to [versae](https://github.com/versae) for implementing most of the `pseq` feature!
* Thanks to [ChuyuHsu](https://github.com/ChuyuHsu) for implemented large parts of the compression feature!

## Release 0.6.0
### New Features
Expand Down
60 changes: 33 additions & 27 deletions DEVELOPER.md
Original file line number Diff line number Diff line change
@@ -1,40 +1,46 @@
# Developer Documentation
This file primarily contains useful information for developers of `ScalaFunctional`

## Release Process
For every release, the following process is used.

### Before Release
1. From the project directory, run `./run-tests.sh` to insure unit tests pass (on python 2 and 3),
and that pylint succeeds
2. Push commit which is the candidate release to Github master
3. Wait for tests to pass on [TravisCI](https://travis-ci.org/EntilZha/ScalaFunctional)
4. Wait for coverage to complete at 100% on [Coveralls](https://coveralls.io/github/EntilZha/ScalaFunctional)
5. Wait for docs to complete building successfully at [scalafunctional.readthedocs.org/en/latest](http://scalafunctional.readthedocs.org/en/latest/)
6. Create an empty `virtualenv` by running `virtualenv env_directory` and activate it by running
`source env_directory/bin/activate`
7. Install `ScalaFunctional` into the virtualenv by running `python setup.py install`
8. Run a python terminal session and insure that `import functional` returns with no errors
9. Deactivate the `virtualenv` by running `deactivate`
10. Repeat steps 6-9 but instead use a python3 interpreter
11. Build the source distribution using `python setup.py sdist`
12. Build the wheel distribution using `python bdist_wheel`
13. Assuming a `.pypirc` file like below, double check that `dist/` has the source and wheel
3. Wait for tests to pass on [TravisCI](https://travis-ci.org/EntilZha/PyFunctional)
4. Wait for coverage to complete at 100% on [Coveralls](https://coveralls.io/github/EntilZha/PyFunctional)
5. Wait for docs to complete building successfully at [docs.pyfunctional.org/en/latest](http://docs.pyfunctional.org/en/latest/)

### Testing Local Release
1. Run `docker run -it python bash` for clean python installation
2. Clone and install `PyFunctional` with `python setup.py install`
3. Run a python terminal session and insure that `import functional` returns with no errors
4. Repeat steps 6-9 but instead use a python3 docker image

### Building Release
1. Build the source distribution using `python setup.py sdist`
2. Build the wheel distribution using `python bdist_wheel`
3. Assuming a `.pypirc` file like below, double check that `dist/` has the source and wheel
distributions
14. Run `twine upload -r test dist/*` to upload `ScalaFunctional` to the test PyPi server
15. Browse to the [pypi test server](testpypi.python.org) and insure the webpage looks correct and

### Testing on Test PyPI
1. Run `twine upload -r test dist/*` to upload `PyFunctional` to the test PyPi server
2. Browse to the [pypi test server](testpypi.python.org) and insure the webpage looks correct and
that the upload was done correctly.
16. Create a new `virtualenv` and install the package using `pip install -i https://testpypi.python.org/pypi scalafunctional`.
Test that the install completes correctly and that `functional` is importable. This may require
installing dependencies from regular `pip` which are not on the test servers like `future`, `six`,
and `enum34` if running python 2.
17. Repeat step 16 using python 3.
18. If all these steps run, than the candidate release commit will become the new release which
means uploading to live pypi and tagging the commit as a release
19. Run `twine upload -r pypi dist/*` to publish `ScalaFunctional` to the live PyPi server.
20. Repeat steps 16 and 17 using the live pip repositories
21. Tag the release on git with `git tag -a vX.X.X`. Then run `git push` and `git push --tags`
22. On Github, create/edit the release page to match the changelog and add discussion
23. Celebrate!
3. Run `docker run -it python bash` and install the package using `pip install -i https://testpypi.python.org/pypi pyfunctional`.
4. Install dependencies not on the test PyPI instance: `future`, `six`, `dill`, and `backports.lzma`
5. Test that `functional` is importable
6. Repeat using python 3.

If all these steps run, than the candidate release commit will become the new release which
means uploading to live pypi and tagging the commit as a release.

### Publishing Release on Production PyPI
1. Run `twine upload -r pypi dist/*` to publish `PyFunctional` to the live PyPi server.
2. Repeat install tests from Test PyPI testing
3. Tag the release on git with `git tag -a vX.X.X`. Then run `git push` and `git push --tags`
4. On Github, create/edit the release page to match the changelog and add discussion
5. Celebrate!


### `.pypirc` file
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ food_cost = seq(transactions)\
.where(lambda x: x.reason == 'food')\
.select(lambda x: x.amount).sum()

# Using ScalaFunctional with fn
# Using PyFunctional with fn
from fn import _
food_cost = seq(transactions).filter(_.reason == 'food').map(_.amount).sum()
```
Expand Down Expand Up @@ -295,12 +295,12 @@ seq.sqlite3('filepath', 'select * from data')
```

For more information on the parameters that these functions can take, reference the
[streams documentation](http://scalafunctional.readthedocs.org/en/latest/functional.html#module-functional.streams)
[streams documentation](http://docs.pyfunctional.org/en/latest/functional.html#module-functional.streams)

### Transformations and Actions APIs
Below is the complete list of functions which can be called on a stream object from `seq`. For
complete documentation reference
[transformation and actions API](http://scalafunctional.readthedocs.org/en/latest/functional.html#module-functional.pipeline).
[transformation and actions API](http://docs.pyfunctional.org/en/latest/functional.html#module-functional.pipeline).

Function | Description | Type
------- | ----------- | ----
Expand Down
4 changes: 2 additions & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,15 @@

setup(
name='PyFunctional',
description='Package for creating data pipelines, LINQ, and chain functional programming',
description='Package for creating data pipelines with chain functional programming',
long_description=long_description,
url='https://github.com/EntilZha/PyFunctional',
author='Pedro Rodriguez',
author_email='[email protected]',
maintainer='Pedro Rodriguez',
maintainer_email='[email protected]',
license='MIT',
keywords='functional LINQ pipeline data collection rdd scala',
keywords='functional pipeline data collection chain rdd linq parallel',
packages=find_packages(exclude=['contrib', 'docs', 'tests*', 'test']),
version='0.6.0',
install_requires=install_requires,
Expand Down

0 comments on commit efc7b31

Please sign in to comment.