[DOCS] Add full-text search overview #119462
Conversation
leemthompo
added
>docs
|
Documentation preview: |
|
Pinging @elastic/es-docs (Team:Docs) |
| ==== | ||
| {es}'s text analysis produces meaningful _linguistic_ tokens (like words and phrases) optimized for search relevance scoring. | ||
| This differs from neural tokenizers, which break text into smaller subword units and numerical vectors for machine learning models. | ||
| For example, "searching" becomes the searchable word token "search" in {es}, while a neural tokenizer might split it into ["sea", "##rch", "##ing"] for model consumption. |
There was a problem hiding this comment.
these ## rendered into a highlight. not sure what your intent was here but you might have to escape the chars
There was a problem hiding this comment.
need to use backticks
There was a problem hiding this comment.
Removing the example as unnecessary detail
There was a problem hiding this comment.
I really love this - it's so great at explaining the landscape and giving people the confidence to implement ft search.
wonder if we should update the quickstart w/ a link to this new overview as well (in the intro and also in "learn more").
There was a problem hiding this comment.
I think this diagram is very helpful, but it needs to be polished up so the text placement is more consistent / there's consistent padding in the cells. we could prob leverage the figma auto-layout tools for this.
We could also consider paring back colors that don't add a lot of meaning - I'd suggest doing greyscale for most of these and then maybe using a different shape for search results
There was a problem hiding this comment.
💯
Not working in Figma because I'm visually illiterate but will try to fix those color/layout issues
|
|
||
| Built on decades of information retrieval research, full-text search in {es} is a compute-efficient, deterministic approach that scales predictably with data volume. | ||
| Full-text search is the cornerstone of production-grade search solutions. | ||
| Combine full-text search with <<semantic-search,semantic search using vectors>> to build modern hybrid search applications. |
There was a problem hiding this comment.
| Combine full-text search with <<semantic-search,semantic search using vectors>> to build modern hybrid search applications. | |
| You can combine full-text search with <<semantic-search,semantic search using vectors>> to build modern hybrid search applications. |
There was a problem hiding this comment.
This was an ORDER!
| Documents and search queries are transformed to enable returning https://www.elastic.co/what-is/search-relevance[relevant] results instead of simply exact term matches. | ||
| Fields of type <<text-field-type,`text`>> are analyzed and indexed for full-text search. | ||
|
|
||
| Built on decades of information retrieval research, full-text search in {es} is a compute-efficient, deterministic approach that scales predictably with data volume. |
There was a problem hiding this comment.
This sentence is pretty dense - deterministic is doing a lot of heavy lifting here. Can we be more explicit about the benefits, or alternatively, weigh the value of this sentence to the reader?
There was a problem hiding this comment.
agree, rewording/reshaping
docs/reference/search/search-your-data/full-text-search.asciidoc
Outdated
Show resolved
Hide resolved
docs/reference/search/search-your-data/full-text-search.asciidoc
Outdated
Show resolved
Hide resolved
docs/reference/search/search-your-data/full-text-search.asciidoc
Outdated
Show resolved
Hide resolved
docs/reference/search/search-your-data/full-text-search.asciidoc
Outdated
Show resolved
Hide resolved
|
|
||
| [discrete] | ||
| [[full-text-search-learn-more]] | ||
| === Learn more |
There was a problem hiding this comment.
this section works as a v1 but it might be nice to guide people through what resources we want them to check out next, or help them to understand the context of a topic (e.g. "To learn how to optimize the relevance of your search results, refer to <<Search relevance optimizations>>")
would also consider pulling out the "get started" into its own CTA - it's the most important thing people should be looking at next. I'm also curious to know if there's a resource we can provide to move this into a prod world (guess that would be explained in our references to API clients)
There was a problem hiding this comment.
Adding some more context.
I hinted at prod world in the intro paragraph revision— to concretize the compute efficiency wording, with link to moving to prod section.
docs/reference/search/search-your-data/full-text-search.asciidoc
Outdated
Show resolved
Hide resolved
docs/reference/search/search-your-data/full-text-search.asciidoc
Outdated
Show resolved
Hide resolved
|
Thanks @shainaraskas! I think I've addressed most things |
| Combine full-text search with <<semantic-search,semantic search using vectors>> to build modern hybrid search applications. | ||
| Built on decades of information retrieval research, full-text search delivers reliable results that scale predictably as your data grows. Because it runs efficiently on CPUs, {es}'s full-text search requires minimal computational resources compared to GPU-intensive vector operations. | ||
|
|
||
| This translates to lower infrastructure costs and predictable scaling requirements. You can scale horizontally by adding more nodes with standard CPU cores and RAM - no specialized hardware needed. A typical deployment will start with 2-3 nodes and grow incrementally as search volume increases. Learn more about <<scalability, moving to production>>. |
There was a problem hiding this comment.
I think this is a bit of a red herring in this doc. I'd just remove the whole paragraph. it also sends the wrong signals to people on serverless who use ft search (the paragraph immediately before it also has references to hardware but I'm less concerned about it because it mostly just sells that this is a performant design)
when I mentioned prod in this context, I mostly meant the idea of making these calls from an app or site (this comment likely also a red herring)
There was a problem hiding this comment.
hmmm yes good point about serverless and the basic message is clear in preceding paragraph anyways
👁️ URL preview
Adds a full-text search overview/entrypoint in "Search your data". Eventually, with additional nesting capabilities, the Text analysis section should be a child of this.
Summary of changes:
full-text-search.asciidoc: Adds new section on full-text search:tokenizers.asciidoc: Clarifies distinction between Elasticsearch's linguistic tokenization and neural tokenizers because tokenizers in ML-context are different beasts.full-text-search-overview.svgAdds new SVG diagram to help visualize full-text search workflow in Elasticsearch.analysis.asciidoc: Adds link to new FTS section