A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.

A Gradio web UI for Large Language Models with support for multiple inference backends.

Official community-driven Azure Machine Learning examples, tested with GitHub Actions.

GNU/Linux 更换系统软件源脚本及 Docker 安装脚本

A high-throughput and memory-efficient inference and serving engine for LLMs

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficie…

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Production-Grade Container Scheduling and Management

A process for automating Docker container base image updates.

Connect, secure, control, and observe services.

OpenTelemetry Python API and SDK

The Kubernetes Package Manager

Bitnami Helm Charts

The open source Firebase alternative. Supabase gives you a dedicated Postgres database to build your web, mobile, and AI applications.