A curated list of resources for learning and exploring Triton, OpenAI's programming language for writing efficient GPU code.

Expert Specialized Fine-Tuning

OpenRFT: Adapting Reasoning Foundation Model for Domain-specific Tasks with Reinforcement Fine-Tuning

An Easy-to-use, Scalable and High-performance RLHF Framework (70B+ PPO Full Tuning & Iterative DPO & LoRA & RingAttention & RFT)

A Lightweight Recommendation System

A generative world for general-purpose robotics & embodied AI learning.

LayerNorm(SmallInit(Embedding)) in a Transformer to improve convergence

Python tool for converting files and office documents to Markdown.

SONAR, a new multilingual and multimodal fixed-size sentence embedding space, with a full suite of speech and text encoders and decoders.

Large Concept Models: Language modeling in a sentence representation space

Code for BLT research paper

A simple, easy-to-hack GraphRAG implementation

General technology for enabling AI capabilities w/ LLMs and MLLMs

Implementation of papers in 100 lines of code.

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

Agent Framework / shim to use Pydantic with LLMs

A course on aligning smol models.

DeMo: Decoupled Momentum Optimization

When it comes to optimizers, it's always better to be safe than sorry

LLM inference in C/C++

prime is a framework for efficient, globally distributed training of AI models over the internet.

HVM3

Powerful unsupervised domain adaptation method for dense retrieval. Requires only unlabeled corpus and yields massive improvement: "GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptatio…

The official implementation of MARS: Unleashing the Power of Variance Reduction for Training Large Models

Everything about the SmolLM & SmolLM2 family of models

supporting pytorch FSDP for optimizers