Machine Learning Journal for Intermediate to Advanced Topics.

Silicon Layout Wizard

Berkeley's Spatial Array Generator

QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving

[NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.

KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache

[ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.

[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

GPGPU processor supporting RISCV-V extension, developed with Chisel HDL

GPGPU supporting RISCV-V, developed with verilog HDL

PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT

TinyChatEngine: On-Device LLM Inference Library

Fast and memory-efficient exact attention

AISystem 主要是指AI系统，包括AI芯片、AI编译器、AI推理和训练框架等AI全栈底层技术

Major project - kannada LLM for farmers

An easy-to-use package for implementing SmoothQuant for LLMs

BaseJump STL: A Standard Template Library for SystemVerilog

[HPCA'21] SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning

[ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models

A minimal GPU design in Verilog to learn how GPUs work from the ground up

Repo for external large-scale work

A curated list of practical guide resources of LLMs (LLMs Tree, Examples, Papers)

KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization

Running large language models on a single GPU for throughput-oriented scenarios.

The official Meta Llama 3 GitHub site

Inference code for Llama models

GPGPU-Sim provides a detailed simulation model of contemporary NVIDIA GPUs running CUDA and/or OpenCL workloads. It includes support for features such as TensorCores and CUDA Dynamic Parallelism as…