Skip to content

Commit

Permalink
Update READMEs for doc building changes
Browse files Browse the repository at this point in the history
  • Loading branch information
@m57lyra @electrum
m57lyra authored and electrum committed Oct 27, 2020
1 parent 21708a1 commit 06cef37
Show file tree
Hide file tree
Showing 2 changed files with 43 additions and 53 deletions.
51 changes: 36 additions & 15 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,53 +15,71 @@
</a>
</p>

## Requirements
## Build requirements

* Mac OS X or Linux
* Java 11, 64-bit
* Java 11.0.8, 64-bit
* Python 2.6+ (for running with the launcher script)
* Docker (for building and running tests)
* Docker

## Building Presto

Presto is a standard Maven project. Simply run the following command from the project root directory:
Presto is a standard Maven project. Simply run the following command from the
project root directory:

./mvnw clean install

On the first build, Maven will download all the dependencies from the internet and cache them in the local repository (`~/.m2/repository`), which can take a considerable amount of time. Subsequent builds will be faster.
On the first build, Maven downloads all the dependencies from the internet
and caches them in the local repository (`~/.m2/repository`), which can take a
considerable amount of time. Subsequent builds are faster.

Presto has a comprehensive set of unit tests that can take several minutes to run. You can disable the tests when building:
Presto has a comprehensive set of unit tests that can also take a considerable
amount of time to run. You can disable the tests when building:

./mvnw clean install -DskipTests

## Running Presto in your IDE

### Overview

After building Presto for the first time, you can load the project into your IDE and run the server. We recommend using [IntelliJ IDEA](http://www.jetbrains.com/idea/). Because Presto is a standard Maven project, you can import it into your IDE using the root `pom.xml` file. In IntelliJ, choose Open Project from the Quick Start box or choose Open from the File menu and select the root `pom.xml` file.
After building Presto for the first time, you can load the project into your IDE
and run the server. We recommend using [IntelliJ
IDEA](http://www.jetbrains.com/idea/). Because Presto is a standard Maven
project, you can import it into your IDE using the root `pom.xml` file. In
IntelliJ, choose Open Project from the Quick Start box or choose Open from the
File menu and select the root `pom.xml` file.

After opening the project in IntelliJ, double check that the Java SDK is properly configured for the project:
After opening the project in IntelliJ, double check that the Java SDK is
properly configured for the project:

* Open the File menu and select Project Structure
* In the SDKs section, ensure that JDK 11 is selected (create one if none exist)
* In the Project section, ensure the Project language level is set to 8 (Presto does not yet use Java 11 language features)
* In the Project section, ensure the Project language level is set to 8 (Presto
does not yet use Java 11 language features)

Presto comes with sample configuration that should work out-of-the-box for development. Use the following options to create a run configuration:
Presto comes with sample configuration that should work out-of-the-box for
development. Use the following options to create a run configuration:

* Main Class: `io.prestosql.server.PrestoServer`
* VM Options: `-ea -XX:+UseG1GC -XX:G1HeapRegionSize=32M -XX:+UseGCOverheadLimit -XX:+ExplicitGCInvokesConcurrent -Xmx2G -Dconfig=etc/config.properties -Dlog.levels-file=etc/log.properties -Djdk.attach.allowAttachSelf=true`
* Working directory: `$MODULE_DIR$`
* Use classpath of module: `presto-server-main`

The working directory should be the `presto-server-main` subdirectory. In IntelliJ, using `$MODULE_DIR$` accomplishes this automatically.
The working directory should be the `presto-server-main` subdirectory. In
IntelliJ, using `$MODULE_DIR$` accomplishes this automatically.

Additionally, the Hive plugin must be configured with the location of your Hive metastore Thrift service. Add the following to the list of VM options, replacing `localhost:9083` with the correct host and port (or use the below value if you do not have a Hive metastore):
Additionally, the Hive plugin must be configured with the location of your Hive
metastore Thrift service. Add the following to the list of VM options, replacing
`localhost:9083` with the correct host and port (or use the below value if you
do not have a Hive metastore):

-Dhive.metastore.uri=thrift://localhost:9083

### Using SOCKS for Hive or HDFS

If your Hive metastore or HDFS cluster is not directly accessible to your local machine, you can use SSH port forwarding to access it. Setup a dynamic SOCKS proxy with SSH listening on local port 1080:
If your Hive metastore or HDFS cluster is not directly accessible to your local
machine, you can use SSH port forwarding to access it. Setup a dynamic SOCKS
proxy with SSH listening on local port 1080:

ssh -v -N -D 1080 server

Expand All @@ -80,10 +98,13 @@ Run a query to see the nodes in the cluster:

SELECT * FROM system.runtime.nodes;

In the sample configuration, the Hive connector is mounted in the `hive` catalog, so you can run the following queries to show the tables in the Hive database `default`:
In the sample configuration, the Hive connector is mounted in the `hive`
catalog, so you can run the following queries to show the tables in the Hive
database `default`:

SHOW TABLES FROM hive.default;

## Development

See [DEVELOPMENT](./DEVELOPMENT.md) for information about code style, development process & guidelines.
See [DEVELOPMENT](./DEVELOPMENT.md) for information about code style,
development process & guidelines.
45 changes: 7 additions & 38 deletions presto-docs/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# presto-docs - Presto Documentation
# presto-docs - Presto documentation

The presto-docs module contains the reference documentation for Presto.

Expand Down Expand Up @@ -48,7 +48,7 @@ Text](https://en.wikipedia.org/wiki/ReStructuredText) (`.rst`) format in
The engine used to create the documentation in HTML format is the Python-based
[Sphinx](https://www.sphinx-doc.org).

## Default Build
## Default build

The default build is using Apache Maven and Java like for the rest of the
Presto build. You just need to have built the current version from the root.
Expand All @@ -64,7 +64,7 @@ directly.
This also performs other checks, and it is the authoritative way to build the
docs, however it is somewhat also slower than using Sphinx directly.

## Faster Build for Authoring
## Faster build for authoring

For faster local build times when writing documentation, you can run the
Sphinx build directly. The build runs inside a Docker container and thus
Expand All @@ -82,7 +82,7 @@ by doing a Maven clean first:
./mvnw -pl presto-docs clean
```

## Viewing Documentation
## Viewing documentation

However you built the docs, the output HTML files can be found in the folder
`presto-docs/target/html/`.
Expand Down Expand Up @@ -119,22 +119,6 @@ python3 -m http.server 4000
In order to see any changes from the source files in the HTML output, simply
re-run the make command and refresh the browser.

## Using sphinx-autobuild

The optional setup of using
[sphinx-autobuild](https://pypi.org/project/sphinx-autobuild/) allows you to
have a running server with the docs and get incremental updates after saving any
changes. This is the fastest and best way to work on the documentation.

To use it, simply install sphinx-autobuild, and then run

```bash
make clean livehtml
```

From now on the docs are available at
[http://localhost:8000](http://localhost:8000).

## Versioning

The version displayed in the resulting HTML is read from the top level Maven
Expand All @@ -145,32 +129,17 @@ version you have to override the pom version with the `PRESTO_VERSION`
environment variable.

```bash
PRESTO_VERSION=327 make clean html
PRESTO_VERSION=327 presto-docs/build
```

If you work on the docs for more than one invocation, you can export the
variable and use it with sphinx as well as sphinx-autobuild.
variable and use it with sphinx.

```bash
export PRESTO_VERSION=327
make clean html
presto-docs/build
```

This is especially useful when deploying doc patches for a release where the
Maven pom has already moved to the next SNAPSHOT version.

## Known Issues

- Older Sphinx versions do not support the `-j auto` SPHINXOPTS in the makefile.
You can delete the option or upgrade Sphinx. The correct version of sphinx is
embedded in the Maven plugin used for the default build.
- Formats like `man` and others beyond the default `html` might have formatting
and content issues and are not actively maintained.
- Different installation methods for Sphinx result in different versions, and
hence in sometimes different problems. Especially when also using
sphinx-autobuild we recommend using the `pip`-based installation.
- Sphinx 2.x+ fails due to requiring parallel write support, which our sitemap extension
does not support. We recommend installing an older version by running
`pip3 install sphinx==1.8.2`. Alternatively, you clear `SPHINXOPTS` when running
Sphinx by using `make SPHINXOPTS="" clean html`, but this may result in other
compatibility issues or differences from the output produced by the Maven plugin.

0 comments on commit 06cef37

Please sign in to comment.