Fully open reproduction of DeepSeek-R1

Foundational Models for State-of-the-Art Speech and Text Translation

Inference Llama 2 in one file of pure 🔥

The Mojo Programming Language

The RedPajama-Data repository contains code for preparing large datasets for training large language models.

Inference code for Llama models

The Introduction of the OLKAVS Dataset

A playbook for systematically maximizing the performance of deep learning models.

Speedy Wavenet generation using dynamic programming ⚡

Letting computers listen to you and really care

You like pytorch? You like micrograd? You love tinygrad! ❤️

A wrapper around speech quality metrics MOSNet, BSSEval, STOI, PESQ, SRMR, SISDR

Convmelspec: Convertible Melspectrograms via 1D Convolutions

Robust Speech Recognition via Large-Scale Weak Supervision

The People’s Speech Dataset

Layerwise Batch Entropy Regularization

Accessible large language models via k-bit quantization for PyTorch.

Unofficial Pytorch Implementation of WaveGrad2

A collection of resources and papers on Diffusion Models

Automatic Speech Recognition (ASR), Speaker Verification, Speech Synthesis, Text-to-Speech (TTS), Language Modelling, Singing Voice Synthesis (SVS), Voice Conversion (VC)

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Pretrain, finetune ANY AI model of ANY size on multiple GPUs, TPUs with zero code changes.

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

Model Quantization Benchmark

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RN…

Supercharge Your Model Training

MatMul Performance Benchmarks for a Single CPU Core comparing both hand engineered and codegen kernels.