A lightweight task engine for building stateful AI agents that prioritizes simplicity and flexibility.

KV cache compression for high-throughput LLM inference

InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management (OSDI'24)

A fast communication-overlapping library for tensor parallelism on GPUs.

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilizatio…

Sequence Parallel Attention for Long Context LLM Model Training and Inference

flash attention tutorial written in python, triton, cuda, cutlass

Open-sourced codes for MiniGPT-4 and MiniGPT-v2 (https://minigpt-4.github.io, https://minigpt-v2.github.io/)

🌟 The Multi-Agent Framework: First AI Software Company, Towards Natural Language Programming

☁️ Build multimodal AI applications with cloud-native stack

A high-throughput and memory-efficient inference and serving engine for LLMs

Generative Models by Stability AI

The official implementation of Latte: Latent Diffusion Transformer for Video Generation.

⚡ Building applications with LLMs through composability ⚡

Enable Next-Gen Large Language Model Applications. Join our Discord: https://discord.gg/pAbnFJrkgZ

Reference implementation of Mistral AI 7B v0.1 model.

Fast and memory-efficient exact attention

include LLM open framework measurement

Benchmarking LLMs on GPUs

12 Lessons, Get Started Building with Generative AI 🔗 https://microsoft.github.io/generative-ai-for-beginners/

Batched LORA + Continuous Batching

Batched Lora + Continuous Batching

Enable everyone to develop, optimize and deploy AI models natively on everyone's devices.

GPU Profiling

《代码随想录》LeetCode 刷题攻略：200道经典题目刷题顺序，共60w字的详细图解，视频难点剖析，50余张思维导图，支持C++，Java，Python，Go，JavaScript等多语言版本，从此算法学习不再迷茫！🔥🔥 来看看，你会发现相见恨晚！🚀

C++ Tutorial

The Compute Library is a set of computer vision and machine learning functions optimised for both Arm CPUs and GPUs using SIMD technologies.