Custom URL ability or generic bridges?

[wiretail]

The API URL's appear to be hard-coded, such as in /src/Bridge/OpenAI/GPT/ModelClient.php for example. These API surfaces are widely compatible with many other providers and some local/private options, and it also seems less than ideal for the project to curate the "supported providers" while also making it less easy to use others.

Would it make sense to optionally specify the API URL in the constructor? (similar to /src/platform/src/Bridge/Ollama/LlamaModelHandler.php but with a default)

Not my favorite approach, as it not being an OpenAI GPT model while called that is a bit ugly -- but alternatively a \Bridge\OpenAI\Generic\ModelClient could be added for this specific purpose, so it's obvious in code an alternative provider is being used.

You can add your own of course, but this would keep it easier to use other models, and also probably keep the flood of "add every provider" type requests down. I know OpenRouter is included, but there are many other aggregators too. (And while its easy to automatically swap OpenRouter targets in different environments, this can also be handy with direct models.)

[chr-hertel]

Hi @wiretail, thanks for raising this, and I'm aware - this is totally valid.
I've done a larger refactoring already to have a default contract, that is basically the OpenAI one, available without any bridge, and I think there should be an easy-to-use generic ModelClient and ResponseConverter.

Def a valid feature request, agreed 👍

[wiretail]

Awesome, thank you — and this is great stuff.

[welcoMattic]

We could rethink something for local platforms (Ollama, Docker Model Runner, LMStudio, LocalAI, ...). If all are OpenAI API compatible, we could configure them using env vars only (url, port)?

[OskarStark]

Maybe let's use a DSN approach here, like we do for Mailer and Notifier bridges?

[welcoMattic]

I change my mind about a generic platform, as local model runners (Ollama, LMStudio, Docker Model Runner, etc) have different API endpoints for same features (completion, embeddings, ...).

So, the DSN approach does not fit well here. Instead I think we have to maintain dedicated platform bridge for each one.

Ollama and LMStudio already exist (Ollama could be improve with Completion and Embeddings ModelClients).

I can work on Docker Model Runner.

---

Docs ref

Ollama Embeddings endpoint: https://github.com/ollama/ollama/blob/main/docs/api.md#generate-embeddings
LMStudio Embeddings enpoint: https://github.com/ollama/ollama/blob/main/docs/api.md#generate-embeddings

Ollama Completion endpoint: https://github.com/ollama/ollama/blob/main/docs/api.md#generate-a-completion
LMStudio Completion endpoint: https://github.com/ollama/ollama/blob/main/docs/api.md#generate-a-completion

Docker Model Runner endpoints: https://docs.docker.com/ai/model-runner/#what-api-endpoints-are-available