forked from langchain-ai/langchainjs
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Some improvements to the docs (langchain-ai#176)
* Improve getting started instructions * Add llm overview doc * Improve order of prompts docs
- Loading branch information
Showing
5 changed files
with
73 additions
and
3 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
--- | ||
sidebar_position: 1 | ||
--- | ||
|
||
# LLM Overview | ||
|
||
Large Language Models (LLMs) are a core component of LangChain. LangChain is not a provider of LLMs, but rather provides a standard interface through which you can interact with a variety of LLMs. | ||
|
||
See the documentation for each LLM on the left sidebar for more information on how to use them. | ||
|
||
## Caching | ||
|
||
LangChain provides an optional caching layer for LLMs. This is useful for two reasons: | ||
|
||
1. It can save you money by reducing the number of API calls you make to the LLM provider, if you're often requesting the same completion multiple times. | ||
2. It can speed up your application by reducing the number of API calls you make to the LLM provider. | ||
|
||
Currently, the cache is stored in-memory. This means that if you restart your application, the cache will be cleared. We're working on adding support for persistent caching. | ||
|
||
To enable it you can pass `cache: true` when you instantiate the LLM. For example: | ||
|
||
```typescript | ||
import { OpenAI } from "langchain/llms"; | ||
|
||
const model = new OpenAI({ cache: true }); | ||
``` | ||
|
||
## Dealing with rate limits | ||
|
||
Some LLM providers have rate limits. If you exceed the rate limit, you'll get an error. To help you deal with this, LangChain provides a `concurrency` option when instantiating an LLM. This option allows you to specify the maximum number of concurrent requests you want to make to the LLM provider. If you exceed this number, LangChain will automatically queue up your requests to be sent as previous requests complete. | ||
|
||
For example, if you set `concurrency: 5`, then LangChain will only send 5 requests to the LLM provider at a time. If you send 10 requests, the first 5 will be sent immediately, and the next 5 will be queued up. Once one of the first 5 requests completes, the next request in the queue will be sent. | ||
|
||
To use this feature, simply pass `concurrency: <number>` when you instantiate the LLM. For example: | ||
|
||
```typescript | ||
import { OpenAI } from "langchain/llms"; | ||
|
||
const model = new OpenAI({ concurrency: 5 }); | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters