Repository navigation
.Net: Bug: semantic kernel injecting user message between tool calls response messages #7626
Description
Describe the bug
Semantic Kernel is adding a user message between tool response messages, which causes the LLM to fail the request with 400 since its expecting to receive response from all tools first.
To Reproduce
Steps to reproduce the behavior:
- Create a flow which the LLM requests multiple tools to be called
- one of the tools should fail (except the last one) by throwing an exception (which should be handled here)
- Semantic kernel will handle the exception by adding a tool message with the exception and a following user message indicating tool failure
- A following tool should succeed. We should have a chat history similar to:
- tool_message
- user_message
- tool_message
- tool_message
- LLM will return the error below
---> Azure.RequestFailedException: An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_iE1vjQZ6NXoB3qJC0q2wtgAE, call_JZ7RjkHTMVjXrGw0hz5MjAHM
Status: 400 (model_error)
Content:
{
"error": {
"message": "An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_iE1vjQZ6NXoB3qJC0q2wtgAE, call_JZ7RjkHTMVjXrGw0hz5MjAHM",
"type": "invalid_request_error",
"param": "messages.[7].role",
"code": null
}
}
Expected behavior
Semantic kernel should send all tool responses in sequence, and just after that add any specific user message related to tool failure. This could be achieved by initially aggregating all user messages related to tool failures, and add them to the history only after the function responses loop
Platform
- OS: Windows
- IDE: Visual Studio
- Language: C#
- Source: 1.8.0
Additional context
- We are using
FunctionCallingStepwisePlanner - I know we are not using the latest version, but I checked it and the problem seems to be there as well.
Reacted by MithrilMan
Metadata
Metadata
Assignees
Labels
Type
Projects
- Status
Activity
[-]Bug: semantic kernel injecting user message between tool calls response messages[/-][+].Net: Bug: semantic kernel injecting user message between tool calls response messages[/+]dmytrostruk commentedon Aug 5, 2024
Franklin Guimaraes (@franklinlindemberg) Thanks for reporting this issue!
Could you please try to use automatic function calling instead of
FunctionCallingStepwisePlannerand see if you can reproduce the problem? We recommend automatic function calling for planning capabilities since it's more efficient approach in terms of performance and cost.Here is more information:
franklinlindemberg commentedon Aug 8, 2024
Dmytro Struk (@dmytrostruk) Thanks for the reply!
I'm gonna try using the automatic function calling and will let you know!
Also just wanted to clarify that this is not really a blocker for us (the try catch on each tool implementation kinda fixed it) but I decided to open the issue anyway since I had already debugged this and had all the details (it might be helpful in case other folks face the same issue)
sergey-ko commentedon Aug 8, 2024
Franklin Guimaraes (@franklinlindemberg) thanks for identifying the cause!
Dmytro Struk (@dmytrostruk) i can confirm it for
ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctionsdmytrostruk commentedon Sep 9, 2024
Franklin Guimaraes (@franklinlindemberg) It would be great to learn about your experience using automatic function calling and if you can reproduce this issue with it. As for now, I'm going to close the issue, since we no longer recommend using
FunctionCallingStepwisePlannerand suggest using automatic function calling instead. But in case you can still reproduce it with automatic function calling - feel free to re-open this issue or create a new one. Thanks a lot!