[Inference API] Fix Azure AI Studio Integration for Completions and Embeddings

[brendan-jugan-elastic]

This draft PR fixes the Inference API integration with Azure AI Foundry (previously Azure AI Studio). The previous integration was broken for both completions and embeddings models due to API changes from Microsoft.

Core Changes:

• no longer referencing an approved list of providers
• introducing a required AzureAiStudioDeploymentType to the service settings
  • either azure_ai_model_inference_service or serverless_api
• modifying auth configuration for each deployment type
• slight request format modifications after API changes
• testing and rebranding changes are in-progress, wanted to get some eyes on the implementation while I complete them

Once testing is complete, I will add more detailed docs describing the deployment types, their configurations, and describe how to use this integration with screenshots from the Azure console.

Local Testing:

Embeddings:

PUT http://localhost:9200/_inference/text_embedding/cohere_serverless_embed

curl --location --request PUT 'http://localhost:9200/_inference/text_embedding/cohere_serverless_embed' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic *****' \
--data '{
  "service": "azureaistudio",
  "service_settings": {
    "api_key": "*****",
    "target": "https://example-target.eastus.models.ai.azure.com/embeddings",
    "deployment_type": "serverless_api",
    "deployment_name": "Cohere-embed-v3-english-hmcek"
  }
}

POST http://localhost:9200/_inference/text_embedding/cohere_serverless_embed

curl --location 'http://localhost:9200/_inference/text_embedding/cohere_serverless_embed' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic *****' \
--data '{
  "input": "What is Elastic?"
}'



PUT http://localhost:9200/_inference/text_embedding/cohere_amlis_embed

curl --location --request PUT 'http://localhost:9200/_inference/text_embedding/cohere_amlis_embed' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic *****' \
--data '{
  "service": "azureaistudio",
  "service_settings": {
    "api_key": "*****",
    "target": "https://example-target/models/embeddings",
    "deployment_type": "azure_ai_model_inference_service",
    "deployment_name": "Cohere-embed-v3-english"
  }
}'

curl --location 'http://localhost:9200/_inference/text_embedding/cohere_amlis_embed' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic *****' \
--data '{
  "input": "What is Elastic?"
}'

Completions:

PUT http://localhost:9200/_inference/completion/cohere_serverless_completion

curl --location --request PUT 'http://localhost:9200/_inference/completion/cohere_serverless_completion' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic *****' \
--data '{
  "service": "azureaistudio",
  "service_settings": {
    "api_key": "*****",
    "target": "https://example-target.eastus.models.ai.azure.com/chat/completions",
    "deployment_type": "serverless_api",
    "deployment_name": "Cohere-command-r"
  }
}'

POST http://localhost:9200/_inference/completion/cohere_serverless_completion

curl --location 'http://localhost:9200/_inference/completion/cohere_serverless_completion' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic *****' \
--data '{
  "input": "What is Elastic?"
}'



PUT http://localhost:9200/_inference/completion/cohere_amlis_completion

curl --location --request PUT 'http://localhost:9200/_inference/completion/cohere_amlis_completion' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic *****' \
--data '{
  "service": "azureaistudio",
  "service_settings": {
    "api_key": "*****",
    "target": "https://example-target.services.ai.azure.com/models/chat/completions?api-version=2024-05-01-preview",
    "deployment_type": "azure_ai_model_inference_service",
    "deployment_name": "Cohere-command-r"
  }
}'

POST http://localhost:9200/_inference/completion/cohere_amlis_completion

curl --location 'http://localhost:9200/_inference/completion/cohere_amlis_completion' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic *****' \
--data '{
  "input": "What is Elastic?"
}'

Related Issues:

• https://github.com/elastic/search-team/issues/8893
• https://github.com/elastic/search-team/issues/8895
• https://github.com/elastic/search-team/issues/8894

Helpful Links:

• https://learn.microsoft.com/en-us/azure/ai-studio/reference/reference-model-inference-chat-completions
• https://learn.microsoft.com/en-us/azure/ai-studio/reference/reference-model-inference-embeddings

[timgrein]

Nice, good stuff! 👏 Gave it a first pass and left some comments, already looking good

[brendan-jugan-elastic]

Note: I'm waiting to complete the Azure AI Studio -> Azure AI Foundry renaming until tomorrow. The above commits contain all of the functional/test changes for this Inference API fix.

[timgrein]

Mainly left comments around changes we need to address with regards to the transport level changes - we can also sync on that, it can be a bit confusing in the beginning