New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Llama.cpp 30B runs with only 6GB of RAM now - update ? #182

Closed

magicmars35 opened this issue Apr 1, 2023 · 3 comments

magicmars35 commented Apr 1, 2023

Llama.cpp 30B runs with only 6GB of RAM now :

anotherwanwanyi commented Apr 2, 2023

It appears that this was just a misreading of how memory usage was being reported.

Take a look at jart's reply.
You still need 20GB of RAM.

https://news.ycombinator.com/item?id=35400066

antimatter15 closed this as completed

Openaicn commented Apr 6, 2023

please make clear 20GB RAM from GPU or motherboard?

anotherwanwanyi commented Apr 6, 2023

@Openaicn Llama.cpp runs on the CPU.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment