Expert Parallelism Load Balancer

DeepEP: an efficient expert-parallel communication library

[COLM 2024] TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding

A PyTorch Native LLM Training Framework

Development repository for the Triton language and compiler

A curated list for Efficient Large Language Models

Trainable, memory-efficient, and GPU-friendly PyTorch reproduction of AlphaFold 2

Repository for benchmarking graph neural networks (JMLR 2023)

paper and its code for AI System

Benchmark datasets, data loaders, and evaluators for graph machine learning

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RN…

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

Running large language models on a single GPU for throughput-oriented scenarios.

SparseTIR: Sparse Tensor Compiler for Deep Learning

Resources of Neural Rendering

Large scale graph learning on a single machine.

Release of stream-specialization software/hardware stack.

A list of awesome GNN systems.

Automatically citation parsing from Google Scholar and Researchgate for creating Github badges

[ICML 2022] "DepthShrinker: A New Compression Paradigm Towards Boosting Real-Hardware Efficiency of Compact Neural Networks", by Yonggan Fu, Haichuan Yang, Jiayi Yuan, Meng Li, Cheng Wan, Raghurama…

Cyberduck is a libre FTP, SFTP, WebDAV, Amazon S3, Backblaze B2, Microsoft Azure & OneDrive and OpenStack Swift file transfer client for Mac and Windows.

PyTorch tutorials.

Federated learning on graph, especially on graph neural networks (GNNs), knowledge graph, and private GNN.

A beautiful, simple, clean, and responsive Jekyll theme for academics

OTOv1-v3, NeurIPS, ICLR, TMLR, DNN Training, Compression, Structured Pruning, Erasing Operators, CNN, Diffusion, LLM

[NeurIPS 2021] "Drawing Robust Scratch Tickets: Subnetworks with Inborn Robustness Are Found within Randomly Initialized Networks" by Yonggan Fu, Qixuan Yu, Yang Zhang, Shang Wu, Xu Ouyang, David C…

Infrastructure of SJTUG mirrors

A tool for learning vector representations of words and entities from Wikipedia