s1: Simple test-time scaling

macOS packaging for ungoogled-chromium

Everything you need to build state-of-the-art foundation models, end-to-end.

A cross-platform command-line utility that creates projects from cookiecutters (project templates), e.g. Python package projects, C projects.

High-performance SGEMM on CUDA devices

Minimalist ML framework for Rust

Community-maintained Kubernetes config and Helm chart for Langfuse

Disaggregated serving system for Large Language Models (LLMs).

Go microservice template for Kubernetes

Open source Loom alternative. Beautiful, shareable screen recordings.

Blazingly fast LLM inference.

🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23

A high-throughput and memory-efficient inference and serving engine for LLMs

Build Conversational AI in minutes ⚡️

A guidance language for controlling large language models.

SGLang is a fast serving framework for large language models and vision language models.

Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks

Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]

An open source developer portal for API productization and monetization

Kubebuilder - SDK for building Kubernetes APIs using CRDs

🪽hydra-zen configuration of flyte workflows 🧘

DevOps Roadmap for 2024. with learning resources

Crawl a site to generate knowledge files to create your own custom GPT from a URL

S-LoRA: Serving Thousands of Concurrent LoRA Adapters

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model

All-in-one chatbot client

notes for software engineers getting up to speed on new AI developments. Serves as datastore for https://latent.space writing, and product brainstorming, but has cleaned up canonical references und…

⚖️ A highly scalable, distributed, real-time chat application built with GO, websockets, cloud native NATS for messaging between microservices and a Next.js frontend.

H2O LLM Studio - a framework and no-code GUI for fine-tuning LLMs. Documentation: https://docs.h2o.ai/h2o-llmstudio/