The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.

Video+code lecture on building nanoGPT from scratch

nanoGPT style version of Llama 3.1

🍁 Sycamore is an LLM-powered search and analytics platform for unstructured data.

BabyGPT: Build Your Own GPT Large Language Model from Scratch Pre-Training Generative Transformer Models: Building GPT from Scratch with a Step-by-Step Guide to Generative AI in PyTorch and Python

MLLM for On-Demand Spatial-Temporal Understanding at Arbitrary Resolution