Efficient Triton Kernels for LLM Training

Train transformer language models with reinforcement learning.

DPO, but faster 🚀

A cross platform way to express data transformation, relational algebra, standardized record expression and plans.

Apache Arrow DataFusion SQL Query Engine

Distributed DataFrame for Python designed for the cloud, powered by Rust

SGLang is a fast serving framework for large language models and vision language models.

Official Rust implementation of Apache Arrow

Allow DataFusion to resolve queries across remote query engines while pushing down as much compute as possible down.

Apache DataFusion Benchmarks

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

Apache DataFusion Ray

Apache Arrow DataFusion Python Bindings

Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.

Extensible Python SDK for developing Flyte tasks and workflows. Simple to get started and learn and highly extensible.

Open source implementation of AlphaFold3

Patched version of dbgen

Apache DataFusion Ballista Distributed Query Engine

Distributed SQL Query Engine in Python using Ray

Build and publish crates with pyo3, cffi and uniffi bindings as well as rust binaries as python packages

GitHub Action to install and run a custom maturin command with built-in support for cross compilation

Rust bindings for the Python interpreter

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

PROST! a Protocol Buffers implementation for the Rust Language

The Rust Reference

Ongoing research training transformer models at scale

A modern TLS library in Rust