Skip to content

A distributed KV store for disaggregated LLM inference

Notifications You must be signed in to change notification settings

bd-iaas-us/infiniStore

Repository files navigation

Run pre-commit checks

Slack

pre-required

install

pip install infinistore

run

infinistore

or

python -m infinistore.server

build

apt install libuv1-dev
apt install libgtest-dev
apt install libflatbuffers-dev
apt install libspdlog-dev libfmt-dev
apt install ibverbs-utils libibverbs-dev
pip install -e .
pip install pre-commit
pre-commit install

client example

check example code infinistore/example/client.py

unit test

pytest infinistore/test_infinistore.py

pre-commit

pre-commit run --all-files

About

A distributed KV store for disaggregated LLM inference

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •