Apache SystemDS Documentation

The primary SystemDS documentation is written in markdown (*.md) and can be converted to HTML using Jekyll.

The Documentation is separated into different parts by sub folders.

.
├── _layouts
│   ├── global.html # The Default content layout and html file.
│   ├── site.html   # The Default content layout for docs pages contained in site folder
├── _site # If run locally a _site folder is generated containing the compiled site.
├── _config.yml # The configuration for jekyll to construct the site.
├── Gemfile.yml # The dependency list for the documentation
├── index.md #  The main entry File
├── css # The style files folder
├── img # The Images folder
├── js  # The JavaScript folder
├── site # The different pages in the documentation
├── api # Contains autogenerated files. Don't touch manually
│   ├── java # The generated Java docs from source code.
│   ├── python # The generated Python docs from source code.

Ubuntu Install

1. Install Ruby development environment Jekyll on Ubuntu.

2. Install Jekyll inside the Ruby environment.

   • gem install jekyll jekyll-redirect-from bundler pygments.rb
   • To do this you might need to change permissions on folders /var/lib/gems and /var/lib/gems/2.5.0.
   • The Pygments package is optional, and does require sudo permissions.

3. Install python dependencies (Optionally).

   • Install Sphinx and sphinx_rtd_theme.

   pip install Sphinx sphinx-rtd-theme

   • pip install Pygments numpy pandas py4j

   and install any other missing packages

4. Make sure Maven and Java 8 are installed.

   mvn --version

   Note: After Java 8, jdk.tools:jdk.tools:jar are removed from jdk

5. Now, update the API docs (Optional)

   ./updateAPI.sh

6. Launch the Documentation locally

   • jekyll serve -w
   • The serving will per default be on localhost port 4000 Link

Mac Install

1. Jekyll (and optionally Pygments) can be installed on the Mac OS in the following manner.

brew install ruby
gem install jekyll
gem install jekyll-redirect-from
gem install bundler
brew install python
gem install pygments.rb

2. Python depedencies

pip install Sphinx sphinx-rtd-theme Pygments numpy pandas py4j

and install any missing Python packages.

To generate SystemDS documentation in HTML, navigate to the docs folder, the root directory of the documentation. From there, you can have Jekyll convert the markdown files to HTML. If you run in server mode, Jekyll will serve up the generated documentation by default at http://127.0.0.1:4000. Modifications to *.md files will be converted to HTML and can be viewed in a web browser.

jekyll serve -w