The ChatCompletionService.GetChatMessageContentAsync method should be extended

[takashiuesaka]

In my opinion, when communicating with an LLM using SemanticKernel, there are two main methods to use:

1. ChatCompletionService.GetChatMessageContentAsync
2. Kernel.InvokePromptAsync

The first is used when utilizing ChatHistory, while the second is used when not utilizing it.

However, GetChatMessageContentAsync has an overload that accepts a string instead of ChatHistory, making both methods almost identical in usage. I believe this situation can be quite confusing.

In my view, the current partial implementation of both methods is problematic.

• Issues with ChatCompletionService.GetChatMessageContentAsync:

  1. IPromptRenderFilter is not called.
  2. There is no overload that accepts KernelArguments when taking a string as a prompt.

• Issues with Kernel.InvokePromptAsync:

  1. There is no overload that accepts ChatHistory.

My suggestion is to either add an overload to ChatCompletionService.GetChatMessageContentAsync that accepts KernelArguments and ensure it calls IPromptRenderFilter, or plan to deprecate the overload that accepts a string as a prompt. This is because it would be unnatural to create an overload for Kernel.InvokePromptAsync that accepts ChatHistory, an object meant for chat completion.

[mornante-basson-1nebula]

Also noticed while using the ChatCompletionService and invoking the GetStreamingChatMessageContentsAsync function that any filters (IFunctionInvocationFilter, IPromptRenderFilter, IAutoFunctionInvocationFilter) that you have registered are not invoked.

I have tested and when invoking the InvokePromptStreamingAsync function from the Kernel class, that filters are indeed triggered if you have any registered. But I would prefer to rather use the ChatCompletionService.

Do we know if this is supported currently or if I am just configuring something wrong?

[RogerBarreto]

Hi @takashiuesaka, thanks for bringing in this discussion.

You can use a chat history with "Kernel.InvokePrompt" if you use the ChatPrompt meta language as shown in this sample

One important thing that may be topic for discussion is to in the future when using our Kernel.Invoke methods to accept KernelContents as input, where if you pass a list of ChatMessageContents they could be reflected as a ChatHistory and would enable multi modality access thru Kernel invocations.

Thanks for the subject and this surely will be topic for further discussion.

[chesnelg]

A follow-up on this topic: according to the to the documentation, the filters should now be supported by the ChatCompletionService - https://learn.microsoft.com/en-us/semantic-kernel/concepts/enterprise-readiness/filters?pivots=programming-language-csharp#using-filters-with-ichatcompletionservice
However, my finding is that on IFunctionInvocationFilter is invoked, not IPromptRenderFilter. Is that a problem with the documentation?

[dmytrostruk]

@chesnelg If you use IChatCompletionService directly instead of Kernel, there is no prompt rendering involved, that's why IPromptRenderFilter is not invoked. If you want to use prompt template engine and prompt rendering including filter, you can use Kernel.

The reason why IFunctionInvocationFilter is invoked with IChatCompletionService is because function calling operation is present in both Kernel and IChatCompletionService.