Rust bindings for the C++ api of PyTorch.

Sample Python extension using Rust/PyO3/tch to interact with PyTorch

Sampling profiler for Python programs

Pax is a Jax-based machine learning framework for training large scale models. Pax allows for advanced and fully configurable experimentation and parallelization, and has demonstrated industry lead…

A simple, performant and scalable Jax LLM!

LLM <-> Python agentic runtime prototype

Tile primitives for speedy kernels

Arena-Hard-Auto: An automatic LLM benchmark.

AICI: Prompts as (Wasm) Programs

Flash Attention in ~100 lines of CUDA (forward pass only)

SGLang is a fast serving framework for large language models and vision language models.

A guidance language for controlling large language models.

Structured Text Generation

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

[ICML 2024] Break the Sequential Dependency of LLM Inference Using Lookahead Decoding

MathVista: data, code, and evaluation for Mathematical Reasoning in Visual Contexts

Machine Learning Engineering Open Book

Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.

Solve puzzles. Learn CUDA.

An Open Source text-to-speech system built by inverting Whisper.

Perf monitoring CLI tool for Apple Silicon

GGUF implementation in C as a library and a tools CLI program

A small utility library for parsing GGUF file info

Code for the paper "Visual Anagrams: Generating Multi-View Optical Illusions with Diffusion Models"

MLX: An array framework for Apple silicon

Examples in the MLX framework

Rust native ready-to-use NLP pipelines and transformer-based models (BERT, DistilBERT, GPT2,...)

Accessible large language models via k-bit quantization for PyTorch.