use remote ai model for Tabby itself

[itforxp]

I have rent GPU server and want point local installed Tabby to that server. Is it possible at the moment?
Something like :
export TABBY_BACKEND_LLAMA=http://<REMOTE_MODEL_URL>
export TABBY_BACKEND_AUTHORIZATION="Bearer <YOUR_API_KEY>"
docker run -it -p 8080:8080 -v $HOME/.tabby:/data registry.tabbyml.com/tabbyml/tabby server --remote

---

Please reply with a 👍 if you want this feature.

[wsxiaoys]

Hi - https://tabby.tabbyml.com/docs/references/models-http-api/llama.cpp/ contains an example connecting tabby to a remote model HTTP server

[itforxp]

thanks. how do i run tabby to use the remote model instead of the local one. the current tabby binary options force me to run the local model

[itforxp]

also i see :

~/.tabby/config.toml

[model.completion.http]
kind = "llama.cpp/completion"
api_endpoint = "http://localhost:8888"
prompt_template = "<PRE> {prefix} <SUF>{suffix} <MID>" 

how can i provide Bearer Token to authorize on remote ollama side?

[wsxiaoys]

You could set api_key field for authorization. https://tabby.tabbyml.com/docs/references/models-http-api/openai/ contains relevant examples.

[itforxp]

Thanks! But what about : how to disable local tabby's served ai model? I tried to kill local tabby model, but it restarted rapidly

[wsxiaoys]

Tabby initiates three default models if they are not configured remotely. For more information, please refer to our documentation at: https://tabby.tabbyml.com/docs/administration/model/

Could you please confirm if you have set up all three models?

[itforxp]

i still have only local models