This is a fork of April 11th of Auto-GPT. I hacked the llama.cpp support in an hour without knowing much about how Auto-GPT actually works (yay for AI safety ;-)). So this is more of a proof of concept. I tried it with the vicuna-13B-4bit model. It's sloooow and most of the time you're fighting with the too small context window size, but sometimes it works and it's really quite astonishing what even such a small model comes up with. But obviously don't expect GPT-4 brilliance here. Running the model with gptq on the GPU might make the latency a little more bearable. I'll add a better README soon. When in doubt how to do something look in the Auto-GPT repo.
forked from rhohndorf/Auto-Llama-cpp
-
Notifications
You must be signed in to change notification settings - Fork 0
Uses Auto-GPT with Llama.cpp
License
p3nj/Auto-Llama-cpp
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
About
Uses Auto-GPT with Llama.cpp
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published
Languages
- Python 99.8%
- Dockerfile 0.2%