Merge branch 'stanfordnlp:main' into main

bakebrain · Mar 3, 2024 · 6da72e3 · 6da72e3
2 parents 16be7a1 + 16fdf50
commit 6da72e3
Show file tree

Hide file tree

Showing 162 changed files with 1,229 additions and 3,484 deletions.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -1,4 +1,4 @@
-# Contibuting
+# Contributing
 
 ## Finding Issues
 

diff --git a/README.md b/README.md
@@ -114,7 +114,7 @@ If you're new to DSPy, it's probably best to go in sequential order. You will pr
 
 4. **[Optimizers (formerly Teleprompters)](https://dspy-docs.vercel.app/docs/building-blocks/optimizers)**
 
-6. **[DSPy Assertions](docs/assertions.md)**
+6. **[DSPy Assertions](https://dspy-docs.vercel.app/docs/building-blocks/assertions)**
 
 
 ### C) Examples
@@ -139,6 +139,8 @@ You can find other examples tweeted by [@lateinteraction](https://twitter.com/la
 - [DSPy on BIG-Bench Hard Example, by Chris Levy](https://drchrislevy.github.io/posts/dspy/dspy.html)
 - [Using Ollama with DSPy for Mistral (quantized) by @jrknox1977](https://gist.github.com/jrknox1977/78c17e492b5a75ee5bbaf9673aee4641)
 - [Using DSPy, "The Unreasonable Effectiveness of Eccentric Automatic Prompts" (paper) by VMware's Rick Battle & Teja Gollapudi, and interview at TheRegister](https://www.theregister.com/2024/02/22/prompt_engineering_ai_models/)
+- Typed DSPy (contributed by [@normal-computing](https://github.com/normal-computing))
+  - [Using DSPy to train Gpt 3.5 on HumanEval by @thomasahle](https://github.com/stanfordnlp/dspy/blob/main/examples/functional/functional.ipynb)
 
 There are also recent cool examples at [Weaviate's DSPy cookbook](https://github.com/weaviate/recipes/tree/main/integrations/dspy) by Connor Shorten. [See tutorial on YouTube](https://www.youtube.com/watch?v=CEuUG4Umfxs).
 
@@ -274,9 +276,32 @@ compiled_rag = teleprompter.compile(RAG(), trainset=my_rag_trainset)
 If we now use `compiled_rag`, it will invoke our LM with rich prompts with few-shot demonstrations of chain-of-thought retrieval-augmented question answering on our data.
 
 
+## 5) Pydantic Types
 
+Sometimes you need more than just string inputs/outputs.
+Assume, for example, you need to find 
 
-## 5) FAQ: Is DSPy right for me?
+```python
+from pydantic import BaseModel, Field
+
+class TravelInformation(BaseModel):
+    origin: str = Field(pattern=r"^[A-Z]{3}$")
+    destination: str = Field(pattern=r"^[A-Z]{3}$")
+    date: datetime.date
+    confidence: float = Field(gt=0, lt=1)
+
+class TravelSignature(Signature):
+    """ Extract all travel information in the given email """
+    email: str = InputField()
+    flight_information: list[TravelInformation] = OutputField()
+
+predictor = dspy.TypedPredictor(TravelSignature)
+predictor(email='...')
+```
+
+Which will output a list of `TravelInformation` objects.
+
+## 6) FAQ: Is DSPy right for me?
 
 The **DSPy** philosophy and abstraction differ significantly from other libraries and frameworks, so it's usually straightforward to decide when **DSPy** is (or isn't) the right framework for your usecase.
 

diff --git a/docs-page/README.md b/docs-page/README.md
diff --git a/docs-page/api/hosting_language_models_locally/HFModel.md b/docs-page/api/hosting_language_models_locally/HFModel.md
diff --git a/docs-page/api/hosting_language_models_locally/MLC.md b/docs-page/api/hosting_language_models_locally/MLC.md
diff --git a/docs-page/api/hosting_language_models_locally/Ollama.md b/docs-page/api/hosting_language_models_locally/Ollama.md
diff --git a/docs-page/api/hosting_language_models_locally/TGI.md b/docs-page/api/hosting_language_models_locally/TGI.md
diff --git a/docs-page/api/hosting_language_models_locally/_category_.json b/docs-page/api/hosting_language_models_locally/_category_.json
diff --git a/docs-page/api/hosting_language_models_locally/vLLM.md b/docs-page/api/hosting_language_models_locally/vLLM.md
diff --git a/docs-page/api/intro.md b/docs-page/api/intro.md
diff --git a/docs-page/docs/deep-dive/language_model_clients/local_models/_category_.json b/docs-page/docs/deep-dive/language_model_clients/local_models/_category_.json
diff --git a/docs-page/.gitignore → docs/.gitignore b/docs-page/.gitignore → docs/.gitignore
diff --git a/docs/DSPy-preprint.pdf b/docs/DSPy-preprint.pdf
diff --git a/docs/README.md b/docs/README.md
@@ -0,0 +1,23 @@
+# DSPy Documentation
+
+This website is built using [Docusaurus](https://docusaurus.io/), a modern static website generator.
+
+## Contributing to the `docs` Folder
+
+This guide is for contributors looking to make changes to the documentation in the `dspy/docs` folder. 
+
+1. **Pull the up-to-date version of the website**: Please pull the latest version of the live documentation site via its subtree repository with the following command:
+
+```bash
+#Ensure you are in the top-level dspy/ folder
+git subtree pull --prefix=docs https://github.com/krypticmouse/dspy-docs master
+```
+
+2. **Push your new changes on a new branch**: Feel free to add or edit existing documentation and open a PR for your changes. Once your PR is reviewed and approved, the changes will be ready to merge into main. 
+
+3. **Updating the website**: Once your changes are merged to main, they need to be pushed to the subtree repository that hosts the live documentation site. This step will eventually be done automatically, but for now, please run the following command to push the updated `docs` content to the website subtree repository:
+
+```bash
+#Ensure you are in the top-level dspy/ folder
+git subtree push --prefix=docs https://github.com/krypticmouse/dspy-docs master
+```
diff --git a/docs/assertions.md → docs/api/assertions.md b/docs/assertions.md → docs/api/assertions.md
@@ -1,11 +1,14 @@
-# DSPy Assertions 
-## Introduction
+---
+sidebar_position: 7
+---
+
+# DSPy Assertions
 
 Language models (LMs) have transformed how we interact with machine learning, offering vast capabilities in natural language understanding and generation. However, ensuring these models adhere to domain-specific constraints remains a challenge. Despite the growth of techniques like fine-tuning or “prompt engineering”, these approaches are extremely tedious and rely on heavy, manual hand-waving to guide the LMs in adhering to specific constraints. Even DSPy's modularity of programming prompting pipelines lacks mechanisms to effectively and automatically enforce these constraints. 
 
 To address this, we introduce DSPy Assertions, a feature within the DSPy framework designed to automate the enforcement of computational constraints on LMs. DSPy Assertions empower developers to guide LMs towards desired outcomes with minimal manual intervention, enhancing the reliability, predictability, and correctness of LM outputs.
 
-### dspy.Assert and dspy.Suggest API
+## dspy.Assert and dspy.Suggest API
 
 We introduce two primary constructs within DSPy Assertions:
 
@@ -255,4 +258,4 @@ compiled_with_assertions_baleen = teleprompter.compile(student = baleen, teacher
 #Compilation + Inference with Assertions
 compiled_baleen_with_assertions = teleprompter.compile(student=baleen_with_assertions, teacher = baleen_with_assertions, trainset=trainset, valset=devset)
 
-```
+```
diff --git a/docs/api/intro.md b/docs/api/intro.md
@@ -0,0 +1,7 @@
+---
+sidebar_position: 1
+---
+
+# API References
+
+Welcome to the API References for DSPy! This is where you'll find easy-to-understand information about all the parts of DSPy that you can use in your projects. We've got guides on different tools and helpers that DSPy has, like modules and optimizers. Everything is sorted so you can quickly find what you need. If you're making something and need to quickly get started with DSPy to do certain tasks, this place will show you how to set it up and get it working just right.