Name		Name	Last commit message	Last commit date
parent directory ..
kernels		kernels
README.md		README.md
backend_request_func.py		backend_request_func.py
benchmark_latency.py		benchmark_latency.py
benchmark_prefix_caching.py		benchmark_prefix_caching.py
benchmark_serving.py		benchmark_serving.py
benchmark_throughput.py		benchmark_throughput.py
launch_tgi_server.sh		launch_tgi_server.sh
sonnet.txt		sonnet.txt

README.md

Benchmarking vLLM

Downloading the ShareGPT dataset

You can download the dataset by running:

wget https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered/resolve/main/ShareGPT_V3_unfiltered_cleaned_split.json