Support for Local/Private models on SuperAGI

[cognitivebot]

We're working to bring you a feature that has been highly requested by the community - Support for local/private models on SuperAGI

We're exploring possible integrations with Oogabooga, GPT4ALL, LM Studio & Fastchat. Please share your thoughts & insights, which one works best for you, and why?

[mrgebien]

A very much needed features. Other people have also expressed interest. Local LLMS are the future!

+1

[pdramos1]

I'd like to add in my support for local models support. I think it can be as simple as an interface where you can put the URL of the app running the local model. Then super AGI simply uses that vs the default openAI one. etc.

[izevar]

+1 on this feature, as integrations LM studio and Oogabooga are the most common ones.
I do use LM Studio on my workstation due to the very good support on Apple M2 systems.

[jmherrera11]

LM Studio is very easy to use and it combines really well GPU and CPU processing. It shoul be a hit combine it with superagi

[cbluth95]

I am being asked for local llm auth token? the video release showed nothing about that.

[jonny255]

Have no idea if LiteLLM has been mentioned in other forums on the discussion page, but just got SuperAGI to work with LiteLLM. LiteLLM has pretty much become the sole proxy that I use now. Just want to say that I'm highly supportive of integrating such frameworks with local/self-hosted models and that I appreciate the hard work!!

[ishaan-jaff]

LiteLLM has pretty much become the sole proxy that I use now.

Thanks for using LiteLLM

[alfi4000]

Yeah nice I tried to run a new llm on superagi but it wasn't able to load it is that correct that not all elms are supported? that one I used was a gruff dolphin 2.5 mistral 8x7b!?