A Datacenter Scale Distributed Inference Serving Framework

vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization

🚀 Awesome System for Machine Learning ⚡️ AI System Papers and Industry Practice. ⚡️ System for Machine Learning, LLM (Large Language Model), GenAI (Generative AI). 🍻 OSDI, NSDI, SIGCOMM, SoCC, MLSy…

A high-throughput and memory-efficient inference and serving engine for LLMs

eTran: Extensible Kernel Transport with eBPF

Service Weaver port of the GCP Online Boutique demo application. See https://github.com/GoogleCloudPlatform/microservices-demo.

AppNet Compiler

Expressive, Easy-to-build, and High-performance Application Networks

A curated list for awesome cloud native tools, software and tutorials. - https://jimmysong.io/awesome-cloud-native/

A Redis HTTP interface with JSON output

A terminal UI for tshark, inspired by Wireshark

WebAssembly for Proxies (Go SDK)

WebAssembly for Proxies (Rust SDK)

Packet, where are you? -- eBPF-based Linux kernel networking debugger

Serving multiple LoRA finetuned LLM as one

The Cloud-Native Network Proxy Platform

Kivi: verifying your Kubernetes clusters

Instant Kubernetes-Native Application Observability

gRPC proxy is a Go reverse proxy that allows for rich routing of gRPC calls with minimum overhead.

Compilation of P4 exercises, examples, documentation, slides for learning or teaching

"How to Do Great Research" Course for Ph.D. Students

🐎 Benchmarks for Inter-Process-Communication Techniques

The Patterns of Scalable, Reliable, and Performant Large-Scale Systems

Batfish is a network configuration analysis tool that can find bugs and guarantee the correctness of (planned or current) network configurations. It enables network engineers to rapidly and safely …

Phoenix dataplane system service

Programming framework for writing and deploying cloud applications.

Example xDS load balancing for go gRPC

gRPC xDS Loadbalancing

A suite of gRPC debugging tools. Like Fiddler/Charles but for gRPC.

Go bindings for creating BPF programs.