Code for Neurips24 paper: QuaRot, an end-to-end 4-bit inference of large language models.

Model Compression Toolbox for Large Language Models and Diffusion Models

DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads

Repository of LV-Eval Benchmark

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

A high-throughput and memory-efficient inference and serving engine for LLMs

[MLSys'22] TorchSparse: Efficient Point Cloud Inference Engine

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.