Add index_options to semantic_text field mappings #119967

kderusso · 2025-01-10T16:15:38Z

Adds index_options support for semantic_text fields using dense models.

Example:

PUT _inference/text_embedding/my-e5-model
{
  "service": "elasticsearch",
  "service_settings": {
    "num_allocations": 1,
    "num_threads": 1,
    "model_id": ".multilingual-e5-small"
  }
}

PUT my-semantic-index
{
  "mappings": {
    "properties": {
      "inference_field": {
        "type": "semantic_text",
        "inference_id": "my-e5-model",
        "index_options": {
          "type": "bbq_hnsw",
          "ef_construction": 100
        }
      }
    }
  }
}

elasticsearchmachine · 2025-01-10T16:38:05Z

Hi @kderusso, I've created a changelog YAML for you.

elasticsearchmachine · 2025-01-10T16:39:12Z

Pinging @elastic/es-search-relevance (Team:Search Relevance)

elasticsearchmachine · 2025-01-10T16:39:12Z

Pinging @elastic/search-eng (Team:SearchOrg)

elasticsearchmachine · 2025-01-10T16:39:12Z

Pinging @elastic/search-relevance (Team:Search - Relevance)

Mikep86

Good start to this! I have a bunch of comments, but they're mostly interrelated, so it's not as much as it seems.

Mikep86 · 2025-01-10T19:21:47Z

docs/changelog/119967.yaml

@@ -0,0 +1,5 @@
+pr: 119967
+summary: Add `index_options` to `semantic_text` field mappings
+area: Relevance


Mapping is probably a more appropriate area for this change

Mikep86 · 2025-01-10T19:24:30Z

docs/reference/mapping/types/semantic-text.asciidoc

+For dense vector models, the configuration of `index_options` models the <<dense-vector-index-options>> configuration and defaults for dense vectors.
+There are currently no supported index options for sparse models.


Suggested change

For dense vector models, the configuration of `index_options` models the <<dense-vector-index-options>> configuration and defaults for dense vectors.

There are currently no supported index options for sparse models.

For dense vector models, the configuration of `index_options` specifies the <<dense-vector-index-options>> configuration and defaults for dense vectors.

There are currently no supported index options for sparse vector models.

Mikep86 · 2025-01-10T19:31:16Z

...ugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextField.java

+                    try {
+                        builder.field(key, value);
+                    } catch (IOException e) {
+                        throw new UncheckedIOException(e);


I get why you're doing this, but it would be nice if we didn't have to use UncheckedIOException here. The method already throws IOException. Can we iterate over the map without using forEach so that we can throw IOException directly?

Mikep86 · 2025-01-10T19:38:16Z

...ugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextField.java

+            return asMap().toString();
+        }
+
+        public abstract Map<String, Object> asMap();


Can we implement asMap() in the base class and handle type in there? We could have an abstract asMap(Map<String, Object> map) method for writing the rest of the params that vary by implementation.

Mikep86 · 2025-01-10T19:39:31Z

...ugin/inference/src/main/java/org/elasticsearch/xpack/inference/mapper/SemanticTextField.java

@@ -186,6 +194,101 @@ private void validateFieldNotPresent(String field, Object fieldValue) {
        }
    }

+    public abstract static class IndexOptions implements ToXContentObject {
+
+        protected final String type;


Can we make this private?

Mikep86 · 2025-01-10T22:00:01Z