OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training

Differentiable ODE solvers with full GPU support and O(1)-memory backpropagation.

A collection of resources regarding the interplay between differential equations, deep learning, dynamical systems, control and numerical methods.

An LM forked from my transformer-train-script repo that replaces attention with a novel idea called "matrix recurrent units."

Code for the paper "Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning"

[ICML 2024 Best Paper] Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution (https://arxiv.org/abs/2310.16834)

Emergent world representations: Exploring a sequence model trained on a synthetic task

Bootstrapping ARC

Latent Program Network (from the "Searching Latent Program Spaces" paper)

Our solution for the arc challenge 2024

Implementation of the proposed minGRU in Pytorch

Material for lectures on Diffusion models at IE university

【PyTorch】Easy-to-use,Modular and Extendible package of deep-learning based CTR models.

🧬 Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human Genomics

Evaluating genomic sequence models for explaining personalized expression variation

Implementation of Enformer, Deepmind's attention network for predicting gene expression, in Pytorch

GTEx & TOPMed data production and analysis pipelines

For fine-tuning Enformer using paired WGS & gene expression data

Demonstrations of Loss of Plasticity and Implementation of Continual Backpropagation

treemind interprets ensemble tree models by analyzing individual trees and their predictions, providing insights into the decision-making process.

Pretrain, finetune ANY AI model of ANY size on multiple GPUs, TPUs with zero code changes.

Mamba for Multivariate Time Series Forecasting

LLM-powered multiagent persona simulation for imagination enhancement and business insights.

A bibliography and survey of the papers surrounding o1

Biological foundation modeling from molecular to genome scale