Skip to content
#

model-inference-service

Star

Here are 4 public repositories matching this topic...

Language: All
Filter by language
All 4 Jupyter Notebook 2 Python 2

ksm26 / Efficiently-Serving-LLMs

Star 9

Learn the ins and outs of efficiently serving Large Language Models (LLMs). Dive into optimization techniques, including KV caching and Low Rank Adapters (LoRA), and gain hands-on experience with Predibase’s LoRAX framework inference server.

text-generation batch-processing server-optimization model-serving model-acceleration inference-optimization optimization-techniques machine-learning-operations deep-learning-techniques model-inference-service performance-enhancement scalability-strategies serving-infrastructure large-scale-deployment
  • Updated Apr 12, 2024
  • Jupyter Notebook

Improve this page

Add a description, image, and links to the model-inference-service topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the model-inference-service topic, visit your repo's landing page and select "manage topics."

Learn more