Skip to content

Pull requests: vllm-project/vllm

New pull request New
Clear current search query, filters, and sorts
444 Open 4,981 Closed
444 Open 4,981 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Add Encoder-decoder model support and T5 Model support new model Requests to new models
#3117 opened Feb 29, 2024 by js8544 • Draft
1
21 🚀 18
A more memory-efficient(1/9) and faster (10×) cuda kernel for performing top-k and top-p operations. ci/build needs-rebase unstale
#1416 opened Oct 19, 2023 by SuperCB Loading…
16 🚀 14
Support generation from input embedding frontend needs-rebase unstale
#1265 opened Oct 5, 2023 by pfldy2850 Loading…
4 tasks done
18 🚀 10
FP8 MoE implementation, take 2 needs-rebase unstale
#3954 opened Apr 10, 2024 by pcmoritz Loading…
2 🚀 10
[WIP] Deepseek V2 MLA ci/build
#10927 opened Dec 5, 2024 by simon-mo • Draft
4 of 11 tasks
9 🚀 10
[Frontend] Add option for LLMEngine to return model hidden states. ready ONLY add when PR is ready to merge/full CI is needed
#7892 opened Aug 27, 2024 by jdvin Loading…
1
15 🚀 9
[Model]: Add transformers backend support ci/build documentation Improvements or additions to documentation
#11330 opened Dec 19, 2024 by ArthurZucker Loading…
25 🚀 9
[docs] add load balancing examples
#1837 opened Nov 29, 2023 by imoneoi Loading…
9 🚀 8
[Kernel][Quantization] Custom Floating-Point Runtime Quantization ci/build needs-rebase
#8751 opened Sep 23, 2024 by AlpinDale Loading…
4 tasks
2
52 🚀 7
AutoQuant: Automatic quantization model for INT8/INT4 inference, INT4 runs faster than AWQ and FP16. ci/build documentation Improvements or additions to documentation frontend needs-rebase unstale
#2801 opened Feb 7, 2024 by ChengcanWang-zte Loading…
5 🚀 7
[Kernel] Add ExLlamaV2 Weight Quantization Support ci/build kernel quantization
#11348 opened Dec 19, 2024 by AlpinDale • Draft
1
5 🚀 5
GPTQ & AWQ Fused MOE needs-rebase unstale
#2761 opened Feb 5, 2024 by chu-tianxiang Loading…
3 tasks done
41 🚀 4
[RFC/WIP] First steps towards FP8 for Mixtral ci/build needs-rebase unstale
#3208 opened Mar 5, 2024 by pcmoritz Loading…
4 🚀 4
[DO NOT MERGE] Asynchronous Output Processing POC [using asyncio]
#6958 opened Jul 30, 2024 by megha95 • Draft
1 🚀 4
[Core][Frontend] Add faster-outlines as guided decoding backend ci/build needs-rebase structured-output
#10277 opened Nov 13, 2024 by unaidedelf8777 Loading…
11 🚀 3
[Kernel][Core][WIP] Tree attention and parallel decoding needs-rebase unstale
#4325 opened Apr 24, 2024 by yukavio Loading…
4 🚀 3
[DOC] Correct warning about performance
#6654 opened Jul 22, 2024 by casper-hansen Loading…
1
13 🚀 3
torch.compile based model optimizer
#6377 opened Jul 12, 2024 by bnellnm • Draft
8 🚀 3
[Model] Add moondream vision language model documentation Improvements or additions to documentation needs-rebase unstale
#4228 opened Apr 20, 2024 by vikhyat Loading…
10 🚀 3
[torch.compile] A simple solution to recursively compile loaded model: using phi3-small as an example
#8398 opened Sep 12, 2024 by wschin Loading…
6 🚀 2
[Kernel] Add prefix-caching support for phi-3-small-8k/128k model triton kernel needs-rebase
#8345 opened Sep 10, 2024 by congcongchen123 Loading…
19 🚀 2
[Frontend] Support Tool and RAG
#3971 opened Apr 10, 2024 by esmeetu • Draft
2 tasks
4 🚀 2
[Core] Added streaming support to LLM Class
#7648 opened Aug 19, 2024 by speggioale Loading…
5 🚀 1
[Kernels] Add an inductor pass to rewrite and fuse collective communication ops with gemms frontend needs-rebase
#9886 opened Oct 31, 2024 by bnellnm Loading…
27 🚀 1
[WIP][Model][Kernel][Bugfix] Commits for new MSFT PhiMoE model ci/build needs-rebase unstale
#7691 opened Aug 20, 2024 by wenxcs Loading…
5 🚀 1
ProTip! Find all pull requests that aren't related to any open issues with -linked:issue.