Enhancement: send user name / user id + Conversation thread id to litellm

[pi-infected]

What features would you like to see added?

Hi,

I am developing a SaaS with LibreChat & Litellm. It would be really beneficial to pass a user id and a conversation thread id to litellm.

Use cases for the user id : monitoring token usage by user

Use cases for the conversation thread id : I am developing with the openai assistant API inside Litellm, and I need to keep track of the conversations, so with a conversation thread id I could make a mapping between what's inside the UI and what OpenAI requires.

If that's not in your priorities, that's fine, I understand. If you could at least point me into the right direction in the code base, I'll implement the features and share them with you if you want.

Best regards,
Paul Irolla

More details

N/A

Which components are impacted by your request?

No response

Pictures

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

User ID is already getting sent by default. Check the incoming request on litellm side: https://docs.librechat.ai/install/configuration/custom_config.html#breakdown-of-default-params

I am developing with the openai assistant API inside Litellm

LibreChat already uses the Assistants API but I suppose you have a different use case. The thread_id's are stored in messages when using Assistants through LibreChat.

However, I don't see the point of doing this if you are managing assistants outside of LibreChat

[pi-infected]

I use LibreChat as a 'frontend' and LiteLLM to implement the logic : function calls to vector database & others. I do know Python but JS is new to me. It would too difficult for me to implement the logic I have in mind in the time frame I've got, in JS .

Ok I've seen the user ID used in LiteLLM, I'll check out how to get the conversation ID. Thanks.

[pi-infected]

Could you give me the file name where the text message and parameters are sent to litellm ? I would like to add the conversation id to it (even in a hacky way). I am on the code, but it's hard to find where

[pi-infected]

Well, I found my way into OpenAIClient.js by concatenating it to the first message role string. Quite ugly but working like a charm. You can close the discussion. Thanks for your time.

[ccasgar]

User ID is already getting sent by default. Check the incoming request on litellm side: https://docs.librechat.ai/install/configuration/custom_config.html#breakdown-of-default-params

I am developing with the openai assistant API inside Litellm

LibreChat already uses the Assistants API but I suppose you have a different use case. The thread_id's are stored in messages when using Assistants through LibreChat.

However, I don't see the point of doing this if you are managing assistants outside of LibreChat

Sending the thread_id (or conversationId) as an additional parameter would be extremely helpful for tracking context in custom llm / agentic server implementations (langchain / langgraph). For example, I have users that are using combinations of custom tools to conduct research. Maintaining context for a single thread is important to support coherent UI chats and also manage overall context size. Subsequently joining contexts on the back-end for advanced composition and other use cases (like presentation-building) is typical.

Happy to share real world use cases if helpful.

[bendavis78]

I have the same use case. My custom endpoint is an openapi-compatible service that itself uses the OpenAI Assistants API with specficic function calling tools, vector db, etc. Because of this I need to be able to keep track of a thread identifier.

Perhaps this could be tracked in a request/response header?