Skip to content

Commit

Permalink
docs: add faq on how to convert own model for tabby (TabbyML#592)
Browse files Browse the repository at this point in the history
  • Loading branch information
wsxiaoys authored Oct 18, 2023
1 parent fc63dd7 commit 7818f64
Show file tree
Hide file tree
Showing 2 changed files with 14 additions and 0 deletions.
6 changes: 6 additions & 0 deletions MODEL_SPEC.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# Tabby Model Specification (Unstable)

> 💁 **INFO**
> Tabby currently operates with two inference backends: [ctranslate2](https://github.com/OpenNMT/CTranslate2) and [llama.cpp](https://github.com/ggerganov/llama.cpp).
> The CUDA/CPU device utilizes ctranslate2 when the `--device cuda` or `--device cpu` options are specified, while the Metal (M1/M2) device employs llama.cpp with the `--device metal` option.
>
> It is possible to create a model that is only usable for a specific inference backend. However, in general, the Tabby team will maintain models that are usable on all devices.
Tabby organizes the model within a directory. This document provides an explanation of the necessary contents for supporting model serving. An example model directory can be found at https://huggingface.co/TabbyML/StarCoder-1B

The minimal Tabby model directory should include the following contents:
Expand Down
8 changes: 8 additions & 0 deletions website/docs/faq.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -31,3 +31,11 @@ import CodeBlock from '@theme/CodeBlock';
</CodeBlock>
</div>
</details>

<details>
<summary>How can I convert my own model for use with Tabby?</summary>
<div>
<p>Follow the instructions provided in the <a href="https://github.com/TabbyML/tabby/blob/main/MODEL_SPEC.md">Model Spec</a>.</p>
<p>Please note that the spec is unstable and does not adhere to semver.</p>
</div>
</details>

0 comments on commit 7818f64

Please sign in to comment.