SGLang is a fast serving framework for large language models and vision language models.

Structured Text Generation

A language for constraint-guided and efficient LLM programming.

Instant, controllable, local pre-trained AI models in Rust