Enhancement: Custom Plugins (Langchain Tools) as External Modules - Dependency Injection

[MoezGholami]

What features would you like to see added?

With the current implementation of plugins, adding any custom plugin will require changing the project code. Let's make the plugins external dependencies.

More details

According to the documents (make_your_own.md), adding a new plugin to Librechat requires modifying the repository files including handle_tools.js, index.js, and manifest.json.

If anyone wants to implement their own plugin, they have to fork the repository and manually keep up with the updates, git conflicts, and breaking changes.

An alternative approach would be having Librechat to look for external Langchain tools and do the imports/exports automatically (specially once manifest.json goes away).

Which components are impacted by your request?

No response

Pictures

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[MoezGholami]

Honestly, I can't think of any clean way to implement this.

[danny-avila]

Honestly, I can't think of any clean way to implement this.

I've thought a lot about this and know of a way, but I want to retire the old way of handling plugins and build it from the ground up anyway. I would implement this first for Assistants, then a new agent system that is also compatible with other endpoints (Anthropic, google, etc.)

[danny-avila]

The Assistants endpoint is most of the way there for dependency injection of tools.

[MoezGholami]

just a random idea: you can put expect a gitignored directory where people can go and develop their plugins there. similar to how comfyui (for example) discovers its custom nodes; allowing each node to be maintained in a separate repository.

[moritzfl]

I implemented a tool yesterday and found the overall process and documentation pretty confusing.
If I want to add something through a plugin system to an existing product, I really do not expect to have to rebuild the product. In the end, I threw the work away and ended up adding a file to the Docker container as api\app\clients\tools.well-known\my_search.json

That way, I do not need to sync changes and check out the entire LibreChat repository to build and deploy a system with plugins.

Is there really no better way to do all this separately from the LibreChat core application currently?

FROM ghcr.io/danny-avila/librechat:v0.7.2 AS librechat

WORKDIR /app

USER node

COPY my_search.json /app/api/app/clients/tools/.well-known/my_search.json

RUN \
    npm config set fetch-retry-maxtimeout 600000 ; \
    npm config set fetch-retries 5 ; \
    npm config set fetch-retry-mintimeout 15000 ; \
    npm install --no-audit; \
    # React client buildcd
    NODE_OPTIONS="--max-old-space-size=2048" npm run frontend; \
    npm prune --production; \
    npm cache clean --force

# Node API setup
EXPOSE 3080
ENV HOST=0.0.0.0
CMD ["npm", "run", "backend"]

[danny-avila]

The current plugins system will be discontinued in favor of something more modular, including using different LLM providers.

Until then, yeah you don't have much options other than following the example of an implemented tool, with TraversaalSearch being the most recent example (works for both assistants and plugins endpoint)

You will be able to add openapi specs just like you can now with assistants:

Most of the heavy lifting for this is developed in-house and is not specific to Assistants, and you can try this out now, too.

[danny-avila]

I implemented a tool yesterday and found the overall process and documentation pretty confusing.

Note some of those docs go back before OpenAI had function calling via their API. We have a lot more frameworks now to build systems like this, and I'm excited to improve the current system. Working on the OpenAI assistants has also taught me a lot on building agents.

[MoezGholami]

The current plugins system will be discontinued in favor of something more modular, including using different LLM providers

Danny, can we have an ETA on when this new implementation will be available? and perhaps a sneak peak of the new approach?

[starascendin]

+1. Can definitely add in some help to get this out the door :)