This repository contains the code for QuaRot, an end-to-end 4-bit inference of large language models.
- cmake
- C++ compiler (GCC/clang/...)
- nvcc
git clone https://github.com/sashkboos/QuaRot.git
cd QuaRot
pip install -e . # or pip install .spcl/QuaRot
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|
![@dependabot[bot]](https://avatars.githubusercontent.com/in/29110?s=64&v=4){kind=small}
