Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Group Chat usage summary (tokens + cost) #173

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Conversation

marklysze
Copy link
Collaborator

@marklysze marklysze commented Dec 9, 2024

UPDATE: We are exploring the native integration of telemetry to provide a more holistic capture of the key events and costs of your AG2 workflow. So, this PR may be supeseded by that integration. Exploration continues...

Why are these changes needed?

An issue raised by @bassilkhilo, #103, identified a bug with the ChatResult of a group chat not including the costs.

After investigating, I've found that when calculating the costs in initiate_chat/a_initiate_chat it is not looking at the costs of the agents in the group chat (only the initiating agent and the group chat manager).

This addresses that.

The status of this PR is that it is pulling through the costs of the agents in the group chat, however it is not including the costs of the speaker selection agent when in auto speaker selection mode. This needs to be addressed as well.

As it is, this will work for a swarm, which doesn't use that.

TODO: Review nested chats to ensure those costs are coming through as well.

Related issue number

Closes #103

Checks

Copy link
Collaborator

@bassilkhilo bassilkhilo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @marklysze This works great for non SwarmResult agent transfers!

However as you've mentioned, if one is using SwarmResult to transfer to another agent within a function, this wouldn't take that agent's cost into account since it's with the temporary AssistantAgent.

Great work! Looking forward to having this include the costs of SwarmResult transfers :) Would be epic

@bassilkhilo
Copy link
Collaborator

Also, @marklysze had a great solution on using initiate_chat with a dummy user proxy agent (max turns 1) to get the cost of any inner chat that would normally use generate_reply. This can be summed with the Swarm costs to get a total cost.

@marklysze marklysze mentioned this pull request Dec 27, 2024
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: chat_result.cost for Swarm Agents not working
2 participants