Add index page and navigation to documentation

README.md

@@ -1,20 +1,17 @@
+<!--intro-start-->
+# Holistic Evaluation of Language Models
+
 [comment]: <> (When using the img tag, which allows us to specify size, src has to be a URL.)
 <img src="https://github.com/stanford-crfm/helm/raw/main/src/helm/benchmark/static/images/helm-logo.png" alt=""  width="800"/>
 
-Welcome!  This repository contains all the assets for [Holistic Evaluation of Language Models](https://arxiv.org/abs/2211.09110), 
-which includes the following features:
+Welcome! The **`crfm-helm`** Python package contains code used in the **Holistic Evaluation of Language Models** project ([paper](https://arxiv.org/abs/2211.09110), [website](https://crfm.stanford.edu/helm/v1.0/)) by [Stanford CRFM](https://crfm.stanford.edu/). This package includes the following features:
 
 - Collection of datasets in a standard format (e.g., NaturalQuestions)
 - Collection of models accessible via a unified API (e.g., GPT-3, MT-NLG, OPT, BLOOM)
 - Collection of metrics beyond accuracy (efficiency, bias, toxicity, etc.)
 - Collection of perturbations for evaluating robustness and fairness (e.g., typos, dialect)
 - Modular framework for constructing prompts from datasets
 - Proxy server for managing accounts and providing unified interface to access models
+<!--intro-end-->
 
-To read more:
-
-- [Setup](docs/setup.md): how to run the code
-- [Code](docs/code.md): how to contribute new scenarios or models
-- [Running the proxy server](docs/proxy-server.md)
-- [Running the benchmark](docs/benchmark.md)
-- [Deployment](docs/deployment.md): for CRFM maintainers of the proxy server
+To get started, refer to [the documentation on Read the Docs](https://crfm-helm.readthedocs.io/) for how to install and run the package.

docs/code.md

@@ -1,4 +1,4 @@
-# Code structure
+# Code Structure
 
 Here's a birds-eye view of how the benchmarking process interacts with the main
 classes (see `benchmark`):

docs/index.md

@@ -0,0 +1,18 @@
+{%
+   include-markdown "../README.md"
+   start="<!--intro-start-->"
+   end="<!--intro-end-->"
+%}
+
+The code is [hosted on GitHub here](https://github.com/stanford-crfm/helm/).
+
+To run the code, refer to the User Guide's chapters:
+
+- [Installation](installation.md)
+- [Quick Start](quick_start.md)
+- [Tutorial](tutorial.md)
+
+To add new models and scenarios, refer to the Developer Guide's chapters:
+
+- [Developer Setup](developer_setup.md)
+- [Code Structure](code.md)

docs/proxy-server.md

@@ -1,4 +1,4 @@
-# Proxy access to language models
+# Proxy Access to Language Models
 
 We provide a single unified entry point into accessing large language models
 (e.g., GPT-3, Jurassic).  This provides both a web interface and a REST API.

docs/requirements.txt

@@ -1,3 +1,4 @@
 mkdocs==1.4.2
 mkdocstrings[python]==0.19.0
 mkdocs-macros-plugin==0.7.0
+mkdocs-include-markdown-plugin==4.0.0

mkdocs.yml

@@ -1,5 +1,6 @@
 site_name: CRFM HELM
-repo_url: https://github.com/example/repository/
+repo_url: https://github.com/stanford-crfm/helm/
+edit_uri: blob/main/docs/
 theme:
   name: readthedocs
   highlightjs: false
@@ -18,9 +19,28 @@ plugins:
             show_root_full_path: false
             show_if_no_docstring: true
             members_order: source
+  - include-markdown
 extra_css:
   - docstrings.css
 markdown_extensions:
   - pymdownx.magiclink
 watch:
   - src
+nav:
+  - 'Home': 'index.md'
+  - 'User Guide':
+    - 'installation.md'
+    - 'quick_start.md'
+    - 'tutorial.md'
+    - 'benchmark.md'
+    - 'proxy-server.md'
+  - 'Reference':
+    - 'models.md'
+    - 'metrics.md'
+    - 'perturbations.md'
+    - 'run_expanders.md'
+    - 'scenarios.md'
+    - 'schemas.md'
+  - 'Developer Guide':
+    - 'developer_setup.md'
+    - 'code.md'