AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

Tensors and Dynamic neural networks in Python with strong GPU acceleration

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

A high-throughput and memory-efficient inference and serving engine for LLMs

A generative speech model for daily dialogue.

The Memory layer for AI Agents

Official inference repo for FLUX.1 models

A high-performance, zero-overhead, extensible Python compiler with built-in NumPy support

Fast and memory-efficient exact attention

Mamba SSM architecture

text and image to video generation: CogVideoX (2024) and CogVideo (ICLR 2023)

Hackable and optimized Transformers building blocks, supporting a composable construction.

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch

An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.

SGLang is a fast serving framework for large language models and vision language models.

HunyuanVideo: A Systematic Framework For Large Video Generation Model

More relighting!

Example models using DeepSpeed

Firefly: 大模型训练工具，支持训练Qwen2.5、Qwen2、Yi1.5、Phi-3、Llama3、Gemma、MiniCPM、Yi、Deepseek、Orion、Xverse、Mixtral-8x7B、Zephyr、Mistral、Baichuan2、Llma2、Llama、Qwen、Baichuan、ChatGLM2、InternLM、Ziya2、Vicuna、Bloom等大模型

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

A large-scale 7B pretraining language model developed by BaiChuan-Inc.

High-Resolution 3D Assets Generation with Large Scale Hunyuan3D Diffusion Models.

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

AITemplate is a Python framework which renders neural network into high performance CUDA/HIP C++ code. Specialized for FP16 TensorCore (NVIDIA GPU) and MatrixCore (AMD GPU) inference.

A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)

Efficient Triton Kernels for LLM Training

PyTorch extensions for high performance and large scale training.