A lightweight data processing framework built on DuckDB and 3FS.

DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling

A bidirectional pipeline parallelism algorithm for computation-communication overlap in V3/R1 training.

FlashMLA: Efficient MLA decoding kernels

VideoVAE+: Large Motion Video Autoencoding with Cross-modal Video VAE

Implementation of Bit Diffusion, Hinton's group's attempt at discrete denoising diffusion, in Pytorch

Implementation of Flash Attention in Jax

Implementation of TiTok, proposed by Bytedance in "An Image is Worth 32 Tokens for Reconstruction and Generation"

Implementation of GigaGAN, new SOTA GAN out of Adobe. Culmination of nearly a decade of research into GANs

Fast and memory-efficient exact attention

Implementation of Lumiere, SOTA text-to-video generation from Google Deepmind, in Pytorch

Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch

Implementation of Autoregressive Diffusion in Pytorch

Implementation of the proposed MaskBit from Bytedance AI

Implementation of rectified flow and some of its followup research / improvements in Pytorch

Vector (and Scalar) Quantization, in Pytorch

Implementation of E2-TTS, "Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTS", in Pytorch

A generative world for general-purpose robotics & embodied AI learning.

Focused on fast experimentation and simplicity

Tackling the Generative Learning Trilemma with Denoising Diffusion GANs https://arxiv.org/abs/2112.07804

LLM training in simple, raw C/CUDA

[NeurIPS 2024 Best Paper][GPT beats diffusion🔥] [scaling laws in visual generation📈] Official impl. of "Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction". An *ult…

[WACV 2025] Official implementation of "RAW-Diffusion: RGB-Guided Diffusion Models for High-Fidelity RAW Image Generation"

PyTorch implementation of VQ-VAE-2 from "Generating Diverse High-Fidelity Images with VQ-VAE-2"

Implementation of Generating Diverse High-Fidelity Images with VQ-VAE-2 in PyTorch

SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformer