An Extensible Deep Learning Library

Understand and test language model architectures on synthetic tasks.

Ongoing research training transformer models at scale

Fast implementations of causal linear attention for autogressive language modeling (Pytorch)

LLM training code for MosaicML foundation models

higher is a pytorch library allowing users to obtain higher order gradients over losses spanning training loops rather than individual training steps.

A large annotated semantic parsing corpus for developing natural language interfaces.

An open-source NLP research library, built on PyTorch.

Sparse and structured neural attention mechanisms

A flexible, high-performance serving system for machine learning models