Skip to content

Commit

Permalink
docs: Update MODEL_SPEC.md (TabbyML#504)
Browse files Browse the repository at this point in the history
  • Loading branch information
wsxiaoys authored Oct 4, 2023
1 parent f44e87f commit 93b6766
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion MODEL_SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,6 @@ ct2-transformers-converter --model ./ --output_dir ctranslate2 --quantization=fl
*Note that the model itself must be compatible with ctranslate2.*

### ggml/
This directory contains binary files used by the [llama.cpp](https://github.com/ggerganov/llama.cpp) inference engine. Tabby utilizes ctranslate2 for inference on the `metal` device.
This directory contains binary files used by the [llama.cpp](https://github.com/ggerganov/llama.cpp) inference engine. Tabby utilizes ggml for inference on the `metal` device.

Currently, only `q8_0.gguf` in this directory is in use. You can refer to the instructions in llama.cpp to learn how to acquire it.

0 comments on commit 93b6766

Please sign in to comment.