llmcord.py

Talk to LLMs with your friends!

Features

• Collaborative prompting

  @ the bot and it will reply to your message. Reply to the bot's message to continue from that point. Build conversations with reply chains!

  You can reply to any of the bot's messages to continue from any point. Or reply to your friend's message and @ the bot to ask a question about it. There are no limits to this functionality.

  Additionally:

  • Back-to-back messages from the same user are automatically chained together. Just reply to the latest one and the bot will see all of them.
  • You can seamlessly move any conversation into a thread. Just create a thread from any message and @ the bot inside to continue.

• Choose your LLM

  Supports models from OpenAI API, Mistral API, ollama and many more thanks to LiteLLM.

  Also supports:

  • oobabooga/text-generation-webui
  • Jan
  • LM Studio

• Vision support

  The bot can see image attachments when you choose a vision model.

• Streamed responses

  The bot's responses are dynamically generated and turn green when complete.

And more...

• Easily set a custom personality
• DM the bot for private access (no @ required)
• User identity aware
• Fully asynchronous
• 1 Python file, ~200 lines of code

Instructions

Before you start, install Python and clone this git repo.

1. Install Python requirements:

pip install -r requirements.txt

2. Create a copy of .env.example named .env and set it up:

Setting	Instructions
DISCORD_BOT_TOKEN	Create a new Discord bot at discord.com/developers/applications and generate a token under the Bot tab. Also enable MESSAGE CONTENT INTENT.
DISCORD_CLIENT_ID	Found under the OAuth2 tab of the Discord bot you just made.
LLM	For LiteLLM supported providers (OpenAI API, Mistral API, ollama, etc.), follow the LiteLLM instructions for its model name formatting. For Jan, set to local/openai/<MODEL_NAME> where <MODEL_NAME> is the name of the model you have loaded. For oobabooga and LM Studio, set to local/openai/model regardless of the model you have loaded.
CUSTOM_SYSTEM_PROMPT	Write practically anything you want to customize the bot's behavior!
CUSTOM_DISCORD_STATUS	Set a custom message that displays on the bot's Discord profile. Max 128 characters.
ALLOWED_CHANNEL_IDS	Discord channel IDs where the bot can send messages, separated by commas. Leave blank to allow all channels.
ALLOWED_ROLE_IDS	Discord role IDs that can use the bot, separated by commas. Leave blank to allow everyone. Specifying at least one role also disables DMs.
MAX_COMPLETION_TOKENS	Limit for the maximum response length of the bot. Suggested to not set this higher than the default. (Default: 1024)
MAX_IMAGES	The maximum number of image attachments allowed in a single message. Only applicable when using a vision model. (Default: 5)
MAX_MESSAGES	The maximum number of messages allowed in a reply chain. (Default: 20)
TEMPERATURE	Controls LLM response, higher = more creative, lower = more deterministic. (Default: 1)
TOP_P	Controls LLM response, higher = more diverse output, lower = more precise. (Default: 1)
LOCAL_SERVER_URL	The URL of your local API server. Only applicable when LLM starts with local/. (Default: http://localhost:5000/v1)
LOCAL_API_KEY	The API key to use with your local API server. Only applicable when LLM starts with local/. Usually safe to leave blank.
OOBABOOGA_CHARACTER	Your oobabooga character that you want to use. Only applicable when using oobabooga. Leave blank to use CUSTOM_SYSTEM_PROMPT instead.
OPENAI_API_KEY	Only required if you choose an OpenAI API model. Generate an OpenAI API key at platform.openai.com/account/api-keys. You must also add a payment method to your OpenAI account at platform.openai.com/account/billing/payment-methods.
MISTRAL_API_KEY	Only required if you choose a Mistral API model. Generate a Mistral API key at console.mistral.ai/user/api-keys. You must also add a payment method to your Mistral account at console.mistral.ai/billing.

OPENAI_API_KEY and MISTRAL_API_KEY are provided as examples. Add more as needed for other providers.

3. Run the bot:

python llmcord.py

Notes

• Vision support is currently limited to gpt-4-vision-preview from OpenAI API. Support for local vision models like llava is planned.

• Only models from OpenAI API are user identity aware (excluding gpt-4-vision-preview for now). This is because only OpenAI API supports the name property in the user message object. I tried the alternate approach of prepending user's names in the message body but this doesn't seem to work well with all models.

• PRs are welcome :)

Star History