A framework for few-shot evaluation of autoregressive language models.

LightEval is a lightweight LLM evaluation suite that Hugging Face has been using internally with the recently released LLM data processing library datatrove and LLM training library nanotron.

This project studies the performance and robustness of language models and task-adaptation methods.

Holistic Evaluation of Language Models (HELM), a framework to increase the transparency of language models (https://arxiv.org/abs/2211.09110).

An open-source NLP research library, built on PyTorch.

🤗 The largest hub of ready-to-use NLP datasets for ML models with fast, easy-to-use and efficient data manipulation tools

PyTorch version of Google AI's BERT model with script to load Google's pre-trained models

Mesh TensorFlow: Model Parallelism Made Easier

code for demo.allennlp.org

Resources for the MRQA 2019 Shared Task

An example submission to the AllenAI DROP leaderboard

Probabilistic Neural Programming

A deep NLP library, based on Keras / tf, focused on question answering (but useful for other NLP too)

Dynamic neural network library

Jayant Krishnamurthy's (machine) Learning and Optimization Library

Library for building reproducible data pipelines to support experimentation