Learn how to design large-scale systems. Prep for the system design interview. Includes Anki flashcards.

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

Magnificent app which corrects your previous console command.

Tensors and Dynamic neural networks in Python with strong GPU acceleration

Models and examples built with TensorFlow

Inference code for Llama models

TensorFlow code and pre-trained models for BERT

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

Pretrain, finetune ANY AI model of ANY size on multiple GPUs, TPUs with zero code changes.

Repository to track the progress in Natural Language Processing (NLP), including the datasets and the current state-of-the-art for the most common NLP tasks.

A set of examples around pytorch in Vision, Text, Reinforcement Learning, etc.

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

🤗 The largest hub of ready-to-use datasets for ML models with fast, easy-to-use and efficient data manipulation tools

Fully open reproduction of DeepSeek-R1

Open source code for AlphaFold 2.

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

🏄 Scalable embedding, reasoning, ranking for images and sentences with CLIP

Ongoing research training transformer models at scale

A PyTorch-based Speech Toolkit

End-to-End Speech Processing Toolkit

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

A Python implementation of global optimization with gaussian processes.

Chinese version of GPT2 training code, using BERT tokenizer.

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

Open Source Neural Machine Translation and (Large) Language Models in PyTorch

Model parallel transformers in JAX and Haiku

Google AI 2018 BERT pytorch implementation

Code for the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"