forked from langchain-ai/langchainjs
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs[minor]: Query analysis (langchain-ai#4598)
* cr * quickstart in nb * dropped mdx in favor of ipynb * few shot * cr * expansion * hyde * routing * nits and step back * pull in how to from py * high cardinality * multi query * Multi retriever * no queries * constructing filters * cleanup and structure * chore: lint files * fix install deps * fixes
- Loading branch information
1 parent
fa1b07f
commit dc6b883
Showing
26 changed files
with
5,134 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -14,6 +14,11 @@ | |
| "@langchain/anthropic": "npm:@langchain/anthropic", | ||
| "node-llama-cpp": "npm:/node-llama-cpp", | ||
| "readline": "https://deno.land/x/[email protected]/mod.ts", | ||
| "ml-distance": "npm:/ml-distance" | ||
| "ml-distance": "npm:/ml-distance", | ||
| "youtubei.js": "npm:/youtubei.js", | ||
| "youtube-transcript": "npm:/youtube-transcript", | ||
| "chromadb": "npm:/chromadb", | ||
| "uuid": "npm:/uuid", | ||
| "@faker-js/faker": "npm:@faker-js/faker" | ||
| } | ||
| } | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| position: 3 | ||
| label: 'Query Analysis' |
2 changes: 2 additions & 0 deletions
2
docs/core_docs/docs/use_cases/query_analysis/how_to/_category_.yml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| position: 3 | ||
| label: 'How-To Guides' |
185 changes: 185 additions & 0 deletions
185
docs/core_docs/docs/use_cases/query_analysis/how_to/constructing_filters.ipynb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,185 @@ | ||
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "raw", | ||
| "id": "df7d42b9-58a6-434c-a2d7-0b61142f6d3e", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "---\n", | ||
| "sidebar_position: 6\n", | ||
| "---" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "id": "f2195672-0cab-4967-ba8a-c6544635547d", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# Construct Filters\n", | ||
| "\n", | ||
| "We may want to do query analysis to extract filters to pass into retrievers. One way we ask the LLM to represent these filters is as a Zod schema. There is then the issue of converting that Zod schema into a filter that can be passed into a retriever. \n", | ||
| "\n", | ||
| "This can be done manually, but LangChain also provides some \"Translators\" that are able to translate from a common syntax into filters specific to each retriever. Here, we will cover how to use those translators." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "id": "bc1302ff", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Setup\n", | ||
| "#### Install dependencies\n", | ||
| "\n", | ||
| "```{=mdx}\n", | ||
| "import Npm2Yarn from \"@theme/Npm2Yarn\";\n", | ||
| "\n", | ||
| "<Npm2Yarn>\n", | ||
| " langchain zod\n", | ||
| "</Npm2Yarn>\n", | ||
| "```\n" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "id": "70cd3b6d", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "\n", | ||
| "In this example, `year` and `author` are both attributes to filter on." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 3, | ||
| "id": "64055006", | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "import { z } from \"zod\";\n", | ||
| "\n", | ||
| "const searchSchema = z.object({\n", | ||
| " query: z.string(),\n", | ||
| " startYear: z.number().optional(),\n", | ||
| " author: z.string().optional(),\n", | ||
| "})" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 4, | ||
| "id": "44eb6d98", | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "const searchQuery: z.infer<typeof searchSchema> = {\n", | ||
| " query: \"RAG\",\n", | ||
| " startYear: 2022,\n", | ||
| " author: \"LangChain\"\n", | ||
| "}" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 6, | ||
| "id": "e8ba6705", | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "import { Comparison, Comparator } from \"langchain/chains/query_constructor/ir\";\n", | ||
| "\n", | ||
| "function constructComparisons(query: z.infer<typeof searchSchema>): Comparison[] {\n", | ||
| " const comparisons: Comparison[] = [];\n", | ||
| " if (query.startYear !== undefined) {\n", | ||
| " comparisons.push(\n", | ||
| " new Comparison(\n", | ||
| " \"gt\" as Comparator,\n", | ||
| " \"start_year\",\n", | ||
| " query.startYear,\n", | ||
| " )\n", | ||
| " );\n", | ||
| " }\n", | ||
| " if (query.author !== undefined) {\n", | ||
| " comparisons.push(\n", | ||
| " new Comparison(\n", | ||
| " \"eq\" as Comparator,\n", | ||
| " \"author\",\n", | ||
| " query.author,\n", | ||
| " )\n", | ||
| " );\n", | ||
| " }\n", | ||
| " return comparisons;\n", | ||
| " }\n" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 7, | ||
| "id": "6a79c9da", | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "const comparisons = constructComparisons(searchQuery);" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 9, | ||
| "id": "2d0e9689", | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "import {\n", | ||
| " Operation,\n", | ||
| " Operator,\n", | ||
| "} from \"langchain/chains/query_constructor/ir\";\n", | ||
| "\n", | ||
| "const _filter = new Operation(\"and\" as Operator, comparisons)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 10, | ||
| "id": "d75455ae", | ||
| "metadata": {}, | ||
| "outputs": [ | ||
| { | ||
| "data": { | ||
| "text/plain": [ | ||
| "{\n", | ||
| " \u001b[32m\"$and\"\u001b[39m: [\n", | ||
| " { start_year: { \u001b[32m\"$gt\"\u001b[39m: \u001b[33m2022\u001b[39m } },\n", | ||
| " { author: { \u001b[32m\"$eq\"\u001b[39m: \u001b[32m\"LangChain\"\u001b[39m } }\n", | ||
| " ]\n", | ||
| "}" | ||
| ] | ||
| }, | ||
| "execution_count": 10, | ||
| "metadata": {}, | ||
| "output_type": "execute_result" | ||
| } | ||
| ], | ||
| "source": [ | ||
| "import { ChromaTranslator } from \"langchain/retrievers/self_query/chroma\";\n", | ||
| "\n", | ||
| "new ChromaTranslator().visitOperation(_filter)" | ||
| ] | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "kernelspec": { | ||
| "display_name": "Deno", | ||
| "language": "typescript", | ||
| "name": "deno" | ||
| }, | ||
| "language_info": { | ||
| "file_extension": ".ts", | ||
| "mimetype": "text/x.typescript", | ||
| "name": "typescript", | ||
| "nb_converter": "script", | ||
| "pygments_lexer": "typescript", | ||
| "version": "5.3.3" | ||
| } | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 5 | ||
| } |
Oops, something went wrong.