FlashInfer: Kernel Library for LLM Serving

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficie…

CUDA Templates for Linear Algebra Subroutines

📚150+ Tensor/CUDA Cores Kernels, ⚡️flash-attn-mma, ⚡️hgemm with WMMA, MMA and CuTe (98%~100% TFLOPS of cuBLAS/FA2 🎉🎉).

A highly optimized inference acceleration engine for Llama and its variants.

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

Arm NN ML Software. The code here is a read-only mirror of https://review.mlplatform.org/admin/repos/ml/armnn

Composable Kernel: Performance Portable Programming Model for Machine Learning Tensor Operators

Training materials associated with NVIDIA's CUDA Training Series (www.olcf.ornl.gov/cuda-training-series/)

Fast and memory-efficient exact attention

Tensors and Dynamic neural networks in Python with strong GPU acceleration

Numbers every LLM developer should know

A list of awesome compiler projects and papers for tensor computation and deep learning.

MegEngine 是一个快速、可拓展、易于使用且支持自动求导的深度学习框架

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies. Note: the repository does not accept github pull requests at this moment. Please submit your patches at…

MNN is a blazing fast, lightweight deep learning framework, battle-tested by business-critical use cases in Alibaba

Assembler for NVIDIA Maxwell architecture

This is an online course where you can learn and master the skill of low-level performance analysis and tuning.

Modular visual interface for GDB in Python

An open-source C++ library developed and used at Facebook.

开发者边车，github打不开，github加速，git clone加速，git release下载加速，stackoverflow加速