Skip to content

Commit

Permalink
Update readme
Browse files Browse the repository at this point in the history
  • Loading branch information
bartowski1182 committed Jul 22, 2023
1 parent cd2ac01 commit 3ed45b0
Show file tree
Hide file tree
Showing 2 changed files with 36 additions and 1 deletion.
2 changes: 1 addition & 1 deletion Dockerfile_cpu
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ RUN apt-get update && apt-get install -y git \
build-essential \
libclblast-dev \
libopenblas-dev \
&& git clone https://github.com/LostRuins/koboldcpp.git --branch v1.33 ./ \
&& git clone https://github.com/LostRuins/koboldcpp.git --branch v1.36 ./ \
&& pip install --no-cache-dir --trusted-host pypi.python.org -r requirements.txt \
&& make LLAMA_OPENBLAS=1 LLAMA_CLBLAST=1 \
&& apt-get clean s&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
Expand Down
35 changes: 35 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ docker build -t koboldcpp-docker:latest .

# Running the image with docker run

(add -d for detached)

```sh
docker run --gpus all -p 80:80 -v /media/teamgroup/models:/app/models koboldcpp-docker:latest --model /app/models/wizardlm-13b-v1.1.ggmlv3.q4_1.bin --port 80 --threads 6 --usecublas --gpulayers 43
```
Expand All @@ -34,3 +36,36 @@ Feel free to modify both to fit your needs, for example I use lowvram for bigger
Pre-built images are provided at https://hub.docker.com/r/noneabove1182/koboldcpp-gpu

Follow the same command as above except with noneabove1182/koboldcpp-gpu:(version)

# Quirks and features

If you're having trouble saving info across sessions, try adding a docker volume (which may have to be removed between updates)

add:

```
- kobold:/koboldcpp
```

to the volumes section of docker-compose.yml and

```
volumes:
kobold:
```

at the bottom

or for docker run:

```
docker volume create kobold
```

and add

```
-v kobold:/koboldcpp
```

to your run command (will make these instructions better soontm)

0 comments on commit 3ed45b0

Please sign in to comment.