Experiments on Multi-Head Latent Attention

Train transformer language models with reinforcement learning.

You like pytorch? You like micrograd? You love tinygrad! ❤️

OpenR: An Open Source Framework for Advanced Reasoning with Large Language Models

High-quality single file implementation of Deep Reinforcement Learning algorithms with research-friendly features (PPO, DQN, C51, DDPG, TD3, SAC, PPG)

systems is a set of tools for describing, running and visualizing systems diagrams.

A high-throughput and memory-efficient inference and serving engine for LLMs

A suite of test scenarios for multi-agent reinforcement learning.

Implementation of Dreamer v3 in pytorch.

OpenSpiel is a collection of environments and algorithms for research in general reinforcement learning and search/planning in games.

Development repository for the Triton language and compiler

Repository for the paper Stream of Search: Learning to Search in Language

Mastering Diverse Domains through World Models

Monte Carlo tree search in JAX

SWE-agent takes a GitHub issue and tries to automatically fix it, using GPT-4, or your LM of choice. It solves 12.47% of bugs in the SWE-bench evaluation set and takes just 1 minute to run.

Cost aware hyperparameter tuning algorithm

An algorithm that generalizes the paradigm of self-play reinforcement learning and search to imperfect-information games.

Code for Cicero, an AI agent that plays the game of Diplomacy with open-domain natural language negotiation.

Tensors and Dynamic neural networks in Python with strong GPU acceleration

Code for magnetic mirror descent.

A clean implementation based on AlphaZero for any game in any framework + tutorial + Othello/Gobang/TicTacToe/Connect4 and more

A tiny scalar-valued autograd engine and a neural net library on top of it with PyTorch-like API

Solve puzzles. Improve your pytorch.

A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training

A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)