Implementation of the sparse attention pattern proposed by the Deepseek team in their "Native Sparse Attention" paper

Fully open reproduction of DeepSeek-R1

🐙 Guides, papers, lecture, notebooks and resources for prompt engineering

Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs.

Entropy Based Sampling and Parallel CoT Decoding

Code for RATIONALYST: Pre-training Process-Supervision for Improving Reasoning https://arxiv.org/pdf/2410.01044

Deep learning for dummies. All the practical details and useful utilities that go into working with real models.

A comprehensive repository of reasoning tasks for LLMs (and beyond)

LLM101n: Let's build a Storyteller

LLM Analytics

OCR, layout analysis, reading order, table recognition in 90+ languages

Convert PDF to markdown + JSON quickly with high accuracy

🔎 Monitor deep learning model training and hardware usage from your mobile phone 📱

Curate better data for LLMs

Code for Quiet-STaR

Grok open release

A multi-programming language benchmark for LLMs

MLX: An array framework for Apple silicon

DeepSeek LLM: Let there be answers

A quick guide (especially) for trending instruction finetuning datasets

AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.

📝 CodeEdit App for macOS – Elevate your code editing experience. Open source, free forever.

A terminal for a more modern age

Benchmarking large language models' complex reasoning ability with chain-of-thought prompting

Parsers for scientific papers (PDF2JSON, TEX2JSON, JATS2JSON)

The GeoV model is a large langauge model designed by Georges Harik and uses Rotary Positional Embeddings with Relative distances (RoPER). We have shared a pre-trained 9B parameter model.

High-Resolution Image Synthesis with Latent Diffusion Models

🧑‍🏫 60+ Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), ga…

Fast and memory-efficient exact attention