A Unified Tokenizer for Visual Generation and Understanding

Official implementation of OneDiffusion paper

Triton implementation of FlashAttention2 that adds Custom Masks.

Lumina-T2X is a unified framework for Text to Any Modality Generation

Modeling, training, eval, and inference code for OLMo

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

Fine-tune Segment-Anything Model with Lightning Fabric.

Official pytorch implementation of the paper: "An Edit Friendly DDPM Noise Space: Inversion and Manipulations". CVPR 2024.

Source code for "Taming Visually Guided Sound Generation" (Oral at the BMVC 2021)

Stable diffusion for real-time music generation

Code for the Interspeech 2021 paper "AST: Audio Spectrogram Transformer".

Kandinsky 2 — multilingual text2image latent diffusion model

An 16kHz implementation of HiFi-GAN for soft-vc.

HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis

A faster pytorch implementation of faster r-cnn

Instant neural graphics primitives: lightning fast NeRF and more

Model parallel transformers in JAX and Haiku

Unofficial implementation of Pix2SEQ

An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.

Open-AI's DALL-E for large scale training in mesh-tensorflow.

Task-based datasets, preprocessing, and evaluation for sequence models.

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

A TensorFlow implementation of Invertible Residual Networks

Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

[CVPR'21 Oral] Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learning

A PyTorch Lightning solution to training OpenAI's CLIP from scratch.