[Enhancement]: if user_provided API KEY value is used, fetch models for user

[ZentixUA]

Contact Details

[email protected]

What happened?

if user_provided API KEY value is used, the model list will not be loaded.

Screenshot:

But setting the global key fixes it:

Steps to Reproduce

1. Go to https://chat.naga.ac/
2. Try it yourself

OR

1. Use OPENAI_REVERSE_PROXY=https://api.naga.ac/v1/chat/completions
2. Use OPENAI_API_KEY=user_provided
3. Enter your NagaAI key on the web page
4. Enjoy this issue xDDD

What browsers are you seeing the problem on?

Firefox, Chrome

Relevant log output

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

This is by design so not a bug but a feature request. OpenAI doesn't allow model calls without API key, so this will need an added feature

[danny-avila]

there's two potential solutions, and both can be added. # 1 might be more useful in your case (using reverse proxies)

1. add a env var for an API KEY exclusively for fetching available models
2. if user_provided is set, change the fetch to a different controller, which will read the encrypted key the user provided for the official api

[ArakiSatoshi]

It should be considered a bug, or at least a design flaw.

It's insecure to hardcode API keys into the config files over properly hashing them individually for each user (I haven't confirmed this is how LibreChat handles it, just hopefully assume), so using user_provided should be prioritized. However, the fetch mechanism does not use the user-provided key to fetch available models, although it is expected to do so, making the issue a bug.

The only workaround for the user is to specify the models using the default: list, but this could be quite troublesome for remotes like OpenRouter which has 100+ models available and it is desirable to both have all of them and have access to the most recent ones without rewriting the librechat.yaml file.

[nitanmarcel]

there's two potential solutions, and both can be added. # 1 might be more useful in your case (using reverse proxies)

1. add a env var for an API KEY exclusively for fetching available models
2. if user_provided is set, change the fetch to a different controller, which will read the encrypted key the user provided for the official api

You can also try to fetch them anyway either if it's an user_provided key or not. Some proxies including the ones above allow to get models without an API key

[danny-avila]

It's insecure to hardcode API keys into the config files over properly hashing them individually for each user (I haven't confirmed this is how LibreChat handles it, just hopefully assume), so using user_provided should be prioritized.

This is not how we encourage users to define API keys in the LibreChat config file.

While you can directly put in API keys in the config file, it's discouraged, and references to the environment variable names are encouraged instead.

i.e this is a valid setting: apiKey: "${WESTUS_API_KEY}" , where WESTUS_API_KEY is meant to be defined in the .env file.

However, the fetch mechanism does not use the user-provided key to fetch available models, although it is expected to do so, making the issue a bug.

The models are fetched on startup, which is why it doesn't happen. They would need to be fetched upon user requests for this to be possible, which is planned.

[normunds-wipo]

guys I do not understand where this discussion is going. I have submitted a PR that fixes this problem. It's around since June and I keep updating so it corresponds to other changes.

We are using this long time and are happy, so what is the problem to merge it? Why "plan" something that is already available?