Commits

Branch selector

 main

User selector

All users

Datepicker

All time

Commit History

Commits on Nov 15, 2024

• Add DP/TP/SP/CP papers with codes (DefTruth#92)

  Show description for 6e6f5ce
  DefTruth

  authored
  6e6f5ce

Commits on Nov 12, 2024

• 🔥[VL-CACHE] VL-CACHE: SPARSITY AND MODALITY-AWARE KV CACHE COMPRESSION FOR VISION-LANGUAGE MODEL INFERENCE ACCELERATION

  DefTruth

  authored
  f3f27a7

• 🔥[BitNet] BitNet a4.8: 4-bit Activations for 1-bit LLMs (DefTruth#91)

  DefTruth

  authored
  b23559e

Commits on Oct 31, 2024

• 🔥[Tensor Product] Acceleration of Tensor-Product Operations with Tensor Cores (DefTruth#90)

  DefTruth

  authored
  a854d6c

• 🔥[Fast Best-of-N] Fast Best-of-N Decoding via Speculative Rejection (DefTruth#89)

  DefTruth

  authored
  567f07c

• 🔥[ShadowKV] ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference

  Show description for db15936
  DefTruth

  authored
  db15936

Commits on Oct 28, 2024

• 🔥[FastAttention] FastAttention: Extend FlashAttention2 to NPUs and Low-resource GPUs for Efficient Inference (DefTruth#88)

  DefTruth

  authored
  613300d

Commits on Oct 21, 2024

• added hybrid inference paper (DefTruth#87)

  aharshms

  authored
  4184e26

Commits on Oct 15, 2024

• Add paper AdaKV (DefTruth#86)

  FFY0

  authored
  4fa6d91

• early exit of LLM inference (DefTruth#85)

  Show description for 53ca177
  boyi-liu

  authored
  53ca177

Commits on Oct 10, 2024

• 🔥[PARALLELSPEC] PARALLELSPEC: PARALLEL DRAFTER FOR EFFICIENT SPECULATIVE DECODING (DefTruth#84)

  DefTruth

  authored
  7ba03a6

• [LLM Inference] LARGE LANGUAGE MODEL INFERENCE ACCELERATION: A COMPREHENSIVE HARDWARE PERSPECTIVE (DefTruth#83)

  DefTruth

  authored
  9ceedc7

Commits on Oct 7, 2024

• Large Language Model Performance Benchmarking on Mobile Platforms: A Thorough Evaluation (DefTruth#82)

  DefTruth

  authored
  8704a95

• 🔥[LORC] Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategy (DefTruth#81)

  DefTruth

  authored
  e87eafe

Commits on Oct 6, 2024

• Link CacheGen and CacheBlend to LMCache (DefTruth#80)

  KuntaiDu

  authored
  8831f39

Commits on Oct 3, 2024

• Bump up to v2.6 (DefTruth#79)

  DefTruth

  authored
  c3f1409

• 🔥[LayerKV] Optimizing Large Language Model Serving with Layer-wise KV Cache Management (DefTruth#78)

  DefTruth

  authored
  a3a8242

• 🔥[KV-COMPRESS] PAGED KV-CACHE COMPRESSION WITH VARIABLE COMPRESSION RATES PER ATTENTION HEAD (DefTruth#77)

  DefTruth

  authored
  fc5b335

Commits on Sep 27, 2024

• 🔥🔥[Tensor Cores]Efficient Arbitrary Precision Acceleration for Large Language Models on GPU Tensor Cores (DefTruth#76)

  DefTruth

  authored
  94e89c5

Commits on Sep 26, 2024

• 🔥[AlignedKV] AlignedKV: Reducing Memory Access of KV-Cache with Precision-Aligned Quantization (DefTruth#75)

  DefTruth

  authored
  d44a51e

• 🔥🔥[HiFloat8] Ascend HiFloat8 Format for Deep Learning (DefTruth#74)

  DefTruth

  authored
  a19f39d

• [Low-bit] A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithms (DefTruth#73)

  DefTruth

  authored
  7b46a89

• 🔥🔥[INT-FLASHATTENTION] INT-FLASHATTENTION: ENABLING FLASH ATTENTION FOR INT8 QUANTIZATION (DefTruth#72)

  DefTruth

  authored
  fbb3c1f

• fix typo (DefTruth#71)

  DefTruth

  authored
  005ecbb

• 🔥[VPTQ] VPTQ: EXTREME LOW-BIT VECTOR POST-TRAINING QUANTIZATION FOR LARGE LANGUAGE MODELS (DefTruth#70)

  DefTruth

  authored
  752a035

• Bump up to v2.5 (DefTruth#69)

  DefTruth

  authored
  3e43647

Commits on Sep 25, 2024

• 🔥🔥[CRITIPREFILL] CRITIPREFILL: A SEGMENT-WISE CRITICALITYBASED APPROACH FOR PREFILLING ACCELERATION IN LLMS (DefTruth#68)

  DefTruth

  authored
  4f1fd58

Commits on Sep 23, 2024

• move RetrievalAttention -> long context (DefTruth#67)

  DefTruth

  authored
  6150e1f

Commits on Sep 19, 2024

• Update codebase of paper "parallel speculative decoding with adaptive draft length" (DefTruth#66)

  smart-lty

  authored
  f98202a

Commits on Sep 18, 2024

• 🔥[InstInfer] InstInfer: In-Storage Attention Offloading for Cost-Effective Long-Context LLM Inference (DefTruth#65)

  DefTruth

  authored
  5dc2039

• Bump up to v2.4 (DefTruth#64)

  DefTruth

  authored
  829da5a

Commits on Sep 17, 2024

• 🔥[Inf-MLLM] Inf-MLLM: Efficient Streaming Inference of Multimodal Large Language Models on a Single GPU (DefTruth#63)

  DefTruth

  authored
  7247770

• 🔥[RetrievalAttention] Accelerating Long-Context LLM Inference via Vector Retrieval (DefTruth#62)

  DefTruth

  authored
  efb983b

Commits on Sep 9, 2024

• Bump up to v2.3 (DefTruth#61)

  DefTruth

  authored
  f0860e8

• 🔥[TEE]Confidential Computing on nVIDIA H100 GPU: A Performance Benchmark Study

  DefTruth

  authored
  9361bbf

Pagination

PreviousNext