📚Tensor/CUDA Cores, 📖150+ CUDA Kernels, ⚡️⚡️toy-hgemm library with WMMA, MMA and CuTe (98%~100% TFLOPS of cuBLAS 🎉🎉).

CUDA Templates for Linear Algebra Subroutines

📰 Must-read papers on KV Cache Compression (constantly updating 🤗).

Public repo for HF blog posts

A fast communication-overlapping library for tensor parallelism on GPUs.

GPU programming related news and material links

Development repository for the Triton language and compiler

CUDA Library Samples

flash attention tutorial written in python, triton, cuda, cutlass

APPy (Annotated Parallelism for Python) enables users to annotate loops and tensor expressions in Python with compiler directives akin to OpenMP, and automatically compiles the annotated code to GP…

A standalone GEMM kernel for fp16 activation and quantized weight, extracted from FasterTransformer

Kernl lets you run PyTorch transformer models several times faster on GPU with a single line of code, and is designed to be easily hackable.

This is a cross-chip platform collection of operators and a unified neural network library.

C++11/14/17/20 templates and generic programming, the most complex and difficult technical details of C++, indispensable in building infrastructure libraries.

Get down and dirty with FlashAttention2.0 in pytorch, plug in and play no complex CUDA kernels

MiniOB is one mini database, helping developers to learn how database works.

A fast multi-producer, multi-consumer lock-free concurrent queue for C++11

TensorRT-in-Action 是一个 GitHub 代码库，提供了使用 TensorRT 的代码示例，并有对应 Jupyter Notebook。

State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure.

Deep Learning System core principles introduction.

8-bit CUDA functions for PyTorch

llama/alpaca onnx models, quantization and testcase

row-major matmul optimization

tvm learn

高性能计算相关知识学习笔记，包含学习笔记和相关知识的代码demo，在持续完善中。 如果有帮助的话请Star一下，对作者帮助很大，谢谢！

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a toolkit of libraries (Ray AIR) for accelerating ML workloads.