Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] EIS Unified chat completions integration #118301

Closed
wants to merge 9 commits into from

Conversation

jonathan-buttner
Copy link
Contributor

WIP

This PR implements the changes to add chat completion support to EIS within the inference plugin.

The current state of this PR is hard coded to work against OpenAI.

Testing

Run elasticsearch with a few feature flags enabled:

run-es -Des.inference_unified_feature_flag_enabled=true -Des.elastic_inference_service_feature_flag_enabled=true

Create an endpoint

PUT http://localhost:9200/_inference/completion/test
{
    "service": "elastic",
    "service_settings": {
        "api_key": "key",
        "model_id": "gpt-4o"
    }
}

Send request

POST http://localhost:9200/_inference/completion/test/_unified
{
    "model": "gpt-4o",
    "messages": [
        {
            "role": "user",
            "content": "What is the weather like in Boston today?"
        }
    ],
    "stop": ["none"],
    "tools": [
        {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
            "type": "object",
            "properties": {
                "location": {
                "type": "string",
                "description": "The city and state, e.g. San Francisco, CA"
                },
                "unit": {
                "type": "string",
                "enum": ["celsius", "fahrenheit"]
                }
            },
            "required": ["location"]
            }
        }
        }
    ],
    "tool_choice": "auto"
}

@jonathan-buttner jonathan-buttner added >enhancement :ml Machine learning Team:ML Meta label for the ML team v9.0.0 v8.18.0 labels Dec 9, 2024
@elasticsearchmachine
Copy link
Collaborator

Hi @jonathan-buttner, I've created a changelog YAML for you.

@jonathan-buttner
Copy link
Contributor Author

Closing in favor of: #118871

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>enhancement :ml Machine learning Team:ML Meta label for the ML team v8.18.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants