Skip to content

Latest commit

 

History

History
 
 

tabby-loadtest

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
README.md
README.md
 
 
loadtest.js
loadtest.js
 
 
record.csv
record.csv
 
 
run.sh
run.sh
 
 
server.py
server.py
 
 

README.md

Run loadtest with tabby on modal GPUs

Steps:

  1. Adjust TABBY_API_HOST in run.sh to match your modal deployment url.
  2. Add models you're interested in to benchmark at end of run.sh
  3. Run run.sh, output will be appended to record.csv