#

co-serving

Here is 1 public repository matching this topic...

llm-db / FineInfer

Deferred Continuous Batching in Resource-Efficient Large Language Model Serving (EuroMLSys 2024)

inference pytorch lora fine-tuning peft llm co-serving

Updated May 28, 2024
Python

Improve this page

Add a description, image, and links to the co-serving topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the co-serving topic, visit your repo's landing page and select "manage topics."