invalid_request_error when using any of the tools

[ChrisBurgdorff]

I cloned the project, installed dependencies, filled out the env vars, started the relay server and the React frontend.

Everything seems to work fine until it tries to access one of the tools. If I say "What is the weather in Chicago?" (or any location) it will output the following in the logs:

{
  "type": "error",
  "event_id": "event_AQJyeAN9SM0lxdiYtgtXf",
  "error": {
    "type": "invalid_request_error",
    "code": null,
    "message": "Conversation already has an active response",
    "param": null,
    "event_id": null
  }
}

And say something to the effect "I am having trouble accessing the weather information right now".

The strange thing is the conversation output window on the bottom actually displays a successful result from the weather API call as "function call output". Here is an example:

t
{"latitude":40.710335,"longitude":-73.99309,"generationtime_ms":0.02205371856689453,"utc_offset_seconds":0,"timezone":"GMT","timezone_abbreviation":"GMT","elevation":32,"current_units":{"time":"iso8601","interval":"seconds","temperature_2m":"°C","wind_speed_10m":"km/h"},"current":{"time":"2024-11-05T19:45","interval":900,"temperature_2m":19.9,"wind_speed_10m":19.9}}

So it's calling the function, getting the weather from the API, returning it, but somehow the voice agent does not recognize this.

Anyway this is running locally on Chrome (latest version) on a Windows 11 machine, but I think it's a problem with the code and not on my end.

If anyone else has encountered this or has a solution let me know please.

[marcelvanworkum]

Possibly related to #474

[assembled-jake]

@ChrisBurgdorff I've run into the same error.

I've seen this happen if you have server VAD on and then sometimes, in the same response event (I'll call this Response A), the response eventually has two items: one of type function_call and one of type message. My hypothesis on what causes your error is that the function_call item is handled first, which attempts to send response.create client event. The issue is that the item of message is not output yet, which means Response A is not done yet, so you get the active response error message even though the function call was made.

If I'm correct in this being the issue, the tricky aspect in fixing this is that you'd have to modify the client to either cancel or wait for the current response to be done before then triggering the new response after the function call has been made. I've found that cancelling the current response and then triggering the new response only when you know Response A is done works most of the time, but there are still some edge cases. For example, sometimes I see response.done from Response A once and then I see my code sending a response.cancel event, which then triggers another response.done from Response A for some reason.

If you have any ideas of better fixes here, I'd be happy to discuss - I'm still working through this problem myself.