diff --git a/.gitignore b/.gitignore index 2ca8682..c3c8386 100644 --- a/.gitignore +++ b/.gitignore @@ -2,3 +2,4 @@ _site/ .sass-cache/ .jekyll-cache/ .jekyll-metadata +*~ \ No newline at end of file diff --git a/README.md b/README.md index 8758b1f..495e403 100644 --- a/README.md +++ b/README.md @@ -1,21 +1,22 @@ ## Lyncs, a Python API for Lattice QCD applications -Lyncs is a Python API for Lattice QCD currently under development. +Lyncs is a Python API for Lattice QCD, currently under development. Lyncs aims to bring several popular libraries for Lattice QCD under a common framework. Lyncs interfaces with libraries for GPUs and CPUs in a way that can accommodate additional computing architectures as these arise, achieving the best performance for the calculations while maintaining the same high-level workflow. -Lyncs distributes calculations using [Dask], with bindings to the +Lyncs distributes calculations using [mpi4py] and [Dask], with bindings to the libraries performed automatically via [cppyy]. Multiple distributed tasks can be executed in parallel and different computing units can be used at the same time to fully exploit the machine allocation. While Lyncs is designed to quite generally allow linking to multiple -libraries, we have focused on a set of targeted packages that include +libraries, we focus on a set of targeted packages that include [c-lime], [DDalphaAMG], [tmLQCD] and [quda]. Any contribution to link other libraries is very welcome! +[mpi4py]: https://mpi4py.readthedocs.io/en/stable/ [Dask]: https://dask.org/ [cppyy]: https://cppyy.readthedocs.io/ [c-lime]: https://github.com/usqcd-software/c-lime @@ -25,7 +26,7 @@ other libraries is very welcome! ### The Lyncs ecosystem -The Lyncs API is a top-level framework meant to be user-friendly, +The Lyncs-API is a top-level framework meant to be user-friendly, flexible, modular and extendable. Under the hood, the project is divided in many Python (sub-)modules that serve for a specific or generic purpose. These modules are collected in the [Lyncs-API] @@ -51,6 +52,10 @@ The modules part of lyncs are the following. - [lyncs.mpi](https://github.com/Lyncs-API/lyncs.mpi): utils for interfacing to MPI libraries using mpi4py and dask. +- [lyncs.io](https://github.com/Lyncs-API/lyncs.io): + IO functions (`load` and `save`) that support various formats + and also parallel IO via MPI and Dask. + ##### Interfaces to Lattice QCD libraries - [lyncs.clime](https://github.com/Lyncs-API/lyncs.clime): @@ -69,7 +74,7 @@ The modules part of lyncs are the following. These modules can be either installed as part of Lyncs, `pip install lyncs[NAME]` -or independently, +or independently (suggested for now), `pip install lyncs_NAME`. In the first case they can be imported with @@ -84,10 +89,55 @@ applications in Python. We seek for contributions under many aspect: linking to libraries, expanding the features of the API and writing of documentation and educational-oriented notebooks. -If you want to contribute, please read the following. +If you want to contribute, please read the [Developer guide]. + +For any news or being informed about planned events, +please consider registering to our mailing list. + +https://groups.google.com/g/lyncs-api #### Developer guide +When contributing to a package, clone the source from [github](https://github.com/Lyncs-API/) +and install the package in development mode: + +``` +pip install -e .[all] +``` + +Now you can run the test-suite for checking the correctness of the installation: + +``` +pytest -v +``` + +If everything goes well, you should see all the tests passed and obtain a coverage report. + +A main implementation requirement is a **high code-coverage**. +If you are going to implement something new, please, also add the respective +test files or functions in the `test/` directory. + +Another implementation requirement is to **format the code** via [black](https://github.com/ambv/black) +and to use [pylint](https://github.com/PyCQA/pylint) for improving the code standards. + +These packages can be installed via pip: + +``` +pip install black lyncs_setuptools[pylint] +``` + +Before any commit, run black from the source directory: + +``` +black . +``` + +When you are done with the implementation, try to resolve as many comments/warnings/errors +as possible brought up by `pylint`: + +``` +lyncs_pylint . +``` ### Funding diff --git a/papers/paper1/paper.bib b/papers/paper1/paper.bib new file mode 100644 index 0000000..b3da2b2 --- /dev/null +++ b/papers/paper1/paper.bib @@ -0,0 +1,17 @@ +@inproceedings{lavrijsen2016high, + title={High-performance Python-C++ bindings with PyPy and Cling}, + author={Lavrijsen, Wim TLP and Dutta, Aditi}, + booktitle={2016 6th Workshop on Python for High-Performance and Scientific Computing (PyHPC)}, + pages={27--35}, + year={2016}, + organization={IEEE} +} + +@inproceedings{rocklin2015dask, + title={Dask: Parallel computation with blocked algorithms and task scheduling}, + author={Rocklin, Matthew}, + booktitle={Proceedings of the 14th python in science conference}, + number={130-136}, + year={2015}, + organization={Citeseer} +} \ No newline at end of file diff --git a/papers/paper1/paper.md b/papers/paper1/paper.md new file mode 100644 index 0000000..fb6bfa1 --- /dev/null +++ b/papers/paper1/paper.md @@ -0,0 +1,68 @@ +--- +title: 'Lyncs-API: Python interfaces for Lattice QCD libraries' +tags: + - Python + - Lattice QCD + - MPI + - cppyy + - Dask +authors: + - name: Simone Bacchio + orcid: 0000-0002-5532-450X + affiliation: 1 + - name: Constantia Alexandrou + affiliation: "1, 2" + - name: Jacob Finkenrath + affiliation: 1 + - name: Giannis Koutsou + affiliation: 1 +affiliations: + - name: Computation-based Science and Technology Research Center, The Cyprus Institute, 20 Kavafi Str., Nicosia 2121, Cyprus + index: 1 + - name: Department of Physics, University of Cyprus, P.O. Box 20537, 1678 Nicosia, Cyprus + index: 2 + +date: 13 August 2017 +bibliography: paper.bib + +--- + +# Summary + +This is the first progress report of the Lyncs-API project, a Python API for Lattice +Quantum Chromodynamics (QCD) applications. +The Lyncs-API plans to support a large number of libraries widely used by the Lattice QCD community. +Common libraries for Lattice QCD are written in C/C++ and use MPI for data-distributed calculations. + +In this report we review our approach and experience in interfacing to these libraries. +We use the package `cppyy` [@lavrijsen2016high] for automatically creating Python bindings and +`dask` [@rocklin2015dask] for supporting distributed tasking and data manipulation. + +The Python interfaces we present are `lyncs.DDalphaAMG` interface to DDalphaAMG [@], +a multigrid solver library for Wilson and Twisted Mass fermions, `lyncs.quda` interface to QUDA [@], +a GPU-enabled library for Lattice QCD, and `lyncs.tmLQCD` interface to tmLQCD [@], +a legacy code of the Extended Twisted Mass collaboration. We use these first interfaces we have +created as an example of the flexibility our approach aims to, supporting from legacy codes to +fully-fledged community codes. + +# Introduction + +Lattice Quantum Chromodynamics (QCD) is a technologically driven field that benefits from the fast +evolution of computers and data technologies. It is a fundamental science research that studies +the strong interactions described by QCD, responsible for binding quarks and gluons within nucleons +of atoms. + +We are working on the + +- Flexibility +- Portability +- Modularity +- User-friendly +- + + +# Acknowledgements + +PRACE-6IP, Grant agreement ID: 823767, Project name: LyNcs. Lyncs is one of 10 applications supported by PRACE-6IP, WP8 "Forward Looking Software Solutions". + +# References