An Open Source Machine Learning Framework for Everyone

LLM inference in C/C++

Productive, portable, and performant GPU programming in Python.

Cloud-native high-performance edge/middle/service proxy

Unsupervised text tokenizer for Neural Network-based text generation.

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and support state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorR…

中文的C++ Template的教学指南。与知名书籍C++ Templates不同，该系列教程将C++ Templates作为一门图灵完备的语言来讲授，以求帮助读者对Meta-Programming融会贯通。(正在施工中)

CUDA Templates for Linear Algebra Subroutines

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

HElib is an open-source software library that implements homomorphic encryption. It supports the BGV scheme with bootstrapping and the Approximate Number CKKS scheme. HElib also includes optimizati…

校招、秋招、春招、实习好项目！带你从零实现一个高性能的深度学习推理库，支持大模型 llama2 、Unet、Yolov5、Resnet等模型的推理。Implement a high-performance deep learning inference library step by step

DeepRec is a high-performance recommendation deep learning framework based on TensorFlow. It is hosted in incubation in LF AI & Data Foundation.

A fast communication-overlapping library for tensor/expert parallelism on GPUs.

A scalable pipeline for designing reconfigurable organisms

Composable Kernel: Performance Portable Programming Model for Machine Learning Tensor Operators

Perplexity GPU Kernels

SPU (Secure Processing Unit) aims to be a provable, measurable secure computation device, which provides computation ability while keeping your private data protected.

A tree-based federated learning system (MLSys 2023)

A simple, light-weight C++ library for unstructured mesh generation in 2-D using Delaunay refinement algorithms