Pinned Loading

1. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 32.5k 5k

2. bytedance/lightseq bytedance/lightseq Public

   LightSeq: A High Performance Library for Sequence Processing and Generation

   C++ 3.2k 329

3. microsoft/Megatron-DeepSpeed microsoft/Megatron-DeepSpeed Public

   Forked from NVIDIA/Megatron-LM

   Ongoing research training transformer language models at scale, including: BERT & GPT-2

   Python 1.9k 344

4. AniZpZ/AutoSmoothQuant AniZpZ/AutoSmoothQuant Public

   An easy-to-use package for implementing SmoothQuant for LLMs

   Python 88 7

5. QQQ QQQ Public

   QQQ is an innovative and hardware-optimized W4A8 quantization solution for LLMs.

   Python 92 8

6. sglang sglang Public

   Forked from sgl-project/sglang

   SGLang is a fast serving framework for large language models and vision language models.

   Python