Take a screenshot online and compresses images in browser with Webassembly

A python module to repair invalid JSON, commonly used to parse the output of LLMs

LLM Inference on consumer devices

The easiest tool for fine-tuning LLM models, synthetic data generation, and collaborating on datasets.

DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including CUDA, x86 and ARMv9.

Advanced Quantization Algorithm for LLMs/VLMs.

A generative world for general-purpose robotics & embodied AI learning.

Tritonbench is a collection of PyTorch custom operators with example inputs to measure their performance.

A throughput-oriented high-performance serving framework for LLMs

🛠 A lite C++ toolkit of 100+ Awesome AI models, support ORT, MNN, NCNN, TNN and TensorRT. 🎉🎉

Efficient Triton Kernels for LLM Training

SOTA Open Source TTS

BELLE: Be Everyone's Large Language model Engine（开源中文对话大模型）

A highly optimized LLM inference acceleration engine for Llama and its variants.

🔗 Some useful websites for programmers.

Materials for learning SGLang

leetcode 刷题总结

OpenOCR: A general OCR system with accuracy and efficiency. Supporting 24 Scene Text Recognition methods trained from scratch on large-scale real datasets, and will continue to add the latest methods.

A collection of (mostly) technical things every software developer should know about

Porting of Pillow resize method in C++ and OpenCV.

GLM Series Edge Models

A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper

LLM notes, including model inference, transformer model structure, and llm framework code analysis notes.

FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.