A lightweight data processing framework built on DuckDB and 3FS.

Expert Parallelism Load Balancer

Analyze computation-communication overlap in V3/R1.

A bidirectional pipeline parallelism algorithm for computation-communication overlap in V3/R1 training.

My learning notes/codes for ML SYS.

DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling

DeepEP: an efficient expert-parallel communication library

FlashMLA: Efficient MLA decoding kernels

Production-tested AI infrastructure tools for efficient AGI development and community-driven innovation

SGLang is a fast serving framework for large language models and vision language models.

🪄 Turns your machine learning code into microservices with web API, interactive GUI, and more.

A collection of community maintained NRI plugins

OpenAI ChatGPT, GPT-3, GPT-4, DALL·E, Whisper API wrapper for Go

SciLifeLab Serve is a platform offering machine learning model serving, data science app hosting (Shiny, Gradio, Streamlit, Dash, etc.), and other tools to life science researchers affiliated with …

Examples of models deployable with Truss

BERT classification model for processing texts longer than 512 tokens. Text is first divided into smaller chunks and after feeding them to BERT, intermediate results are pooled. The implementation …

A high-throughput and memory-efficient inference and serving engine for LLMs

Llama中文社区，Llama3在线体验和微调模型已开放，实时汇总最新Llama3学习资料，已将所有代码更新适配Llama3，构建最好的中文Llama大模型，完全开源可商用

Free ChatGPT API Key，免费ChatGPT API，支持GPT4 API（免费），ChatGPT国内可用免费转发API，直连无需代理。可以搭配ChatBox等软件/插件使用，极大降低接口使用成本。国内即可无限制畅快聊天。

Slim(toolkit): Don't change anything in your container image and minify it by up to 30x (and for compiled languages even more) making it secure too! (free and open source)

A high-quality tool for convert PDF to Markdown and JSON.一站式开源高质量数据提取工具，将PDF转换成Markdown和JSON格式。

本项目旨在收集开源的表格智能任务数据集（比如表格问答、表格-文本生成等），将原始数据整理为指令微调格式的数据并微调LLM，进而增强LLM对于表格数据的理解，最终构建出专门面向表格智能任务的大型语言模型。

GPU-Jupyter: Your GPU-accelerated JupyterLab with a rich data science toolstack, TensorFlow and PyTorch for your reproducible deep learning experiments.

This repository contains a Python implementation that allows you to use gorilla-llm/gorilla-openfunctions-v2 LLM to perform function calling using the OpenAI protocol. It provides a way to extend t…

SparkClient 是一个Go语言库，用于与讯飞星火Spark AI的聊天API进行交互。它封装了创建请求、处理响应和WebSocket通信的逻辑，使得在Go应用程序中集成Spark AI服务变得简单。

OpenAI 接口接入适配，支持千帆大模型平台、讯飞星火大模型、腾讯混元以及MiniMax、Deep-Seek，等兼容OpenAI接口，仅单可执行文件，配置超级简单，一键部署，开箱即用. Seamlessly integrate with OpenAI and compatible APIs using a single executable for quick setup and depl…

YOLOv10: Real-Time End-to-End Object Detection [NeurIPS 2024]

The first AI Agent Server, Eidolon is a pluggable Agent SDK and enterprise ready, deployment server for Agentic applications

中文羊驼大模型三期项目 (Chinese Llama-3 LLMs) developed from Meta Llama 3