Distribute and run LLMs with a single file.

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

The fastest way to develop full-stack web apps with React & Node.js.

KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization

KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache

A Comprehensive Assessment of Trustworthiness in GPT Models

GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM

Retrieval-Augmented Theorem Provers for Lean

A high-throughput and memory-efficient inference and serving engine for LLMs

Knowledge Graph Retrieval Augmented Generation (KG-RAG) Eval Datasets

Holistic Evaluation of Language Models (HELM), a framework to increase the transparency of language models (https://arxiv.org/abs/2211.09110).

Tilted Empirical Risk Minimization (ICLR '21)

LlamaIndex (GPT Index) is a data framework for your LLM applications

A gradio web UI for running Large Language Models like LLaMA, llama.cpp, GPT-J, Pythia, OPT, and GALACTICA.

C++ implementation of ChatGLM-6B & ChatGLM2-6B

Scalable Python data science, in an API compatible & lightning fast way.

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.

React + 3JS Game