Skip to content

Java: Bug: An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id' #9443

Closed

Description

@miladhub

Describe the bug
Got the following error using Semantic Kernel in Java with automatic function calling using 4o* OpenAI models: "An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'".

To Reproduce
Steps to reproduce the behavior:

  1. Unzip provided example, build it and execute it:
mvn clean install
export OPENAI_API_KEY=sk-...
java -jar target/lights-example-1.0-SNAPSHOT.jar
  1. Enter two consecute inputs: "get the lights" and "turn them on"
  2. See error
SLF4J(W): No SLF4J providers were found.
SLF4J(W): Defaulting to no-operation (NOP) logger implementation
SLF4J(W): See https://www.slf4j.org/codes.html#noProviders for further details.
User > get the lights
Getting lights
Assistant > Here are the current states of the lights:

1. **Table Lamp**: Off
2. **Porch Light**: Off
3. **Chandelier**: On
User > turn them on
Exception in thread "main" com.azure.core.exception.HttpResponseException: Status code 400, "{
  "error": {
    "message": "An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_Mor7WOBzlb1Ao9ryIczup9mS, call_ejzgT6RAATaWBqmJbrzbzAMP",
    "type": "invalid_request_error",
    "param": "messages",
    "code": null
  }
}"
	at com.azure.core.implementation.http.rest.RestProxyBase.instantiateUnexpectedException(RestProxyBase.java:388)
	at com.azure.core.implementation.http.rest.AsyncRestProxy.lambda$ensureExpectedStatus$1(AsyncRestProxy.java:135)
	at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:113)
	at reactor.core.publisher.MonoUsing$MonoUsingSubscriber.onNext(MonoUsing.java:231)
	at reactor.core.publisher.FluxHandle$HandleSubscriber.onNext(FluxHandle.java:126)
	at reactor.core.publisher.FluxMap$MapConditionalSubscriber.onNext(FluxMap.java:224)
	at reactor.core.publisher.FluxDoFinally$DoFinallySubscriber.onNext(FluxDoFinally.java:113)
	at reactor.core.publisher.FluxHandleFuseable$HandleFuseableSubscriber.onNext(FluxHandleFuseable.java:191)
	at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onNext(FluxContextWrite.java:107)
	at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1839)
	at reactor.core.publisher.MonoCollectList$MonoCollectListSubscriber.onComplete(MonoCollectList.java:129)
	at reactor.core.publisher.FluxPeek$PeekSubscriber.onComplete(FluxPeek.java:260)
	at reactor.core.publisher.FluxMap$MapSubscriber.onComplete(FluxMap.java:144)
	at reactor.netty.channel.FluxReceive.onInboundComplete(FluxReceive.java:415)
	at reactor.netty.channel.ChannelOperations.onInboundComplete(ChannelOperations.java:439)
	at reactor.netty.channel.ChannelOperations.terminate(ChannelOperations.java:493)
	at reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:789)
	at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:114)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at com.azure.core.http.netty.implementation.AzureSdkHandler.channelRead(AzureSdkHandler.java:224)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1475)
	at io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1349)
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1389)
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:530)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:469)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1407)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:918)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:994)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:1583)
	Suppressed: java.lang.Exception: #block terminated with an error
		at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:100)
		at reactor.core.publisher.Mono.block(Mono.java:1742)
		at org.example.LightsExample.main(LightsExample.java:86)

Expected behavior
The lights should be turned on.

Platform

  • OS: Mac M1
  • IDE: IntelliJ Idea or command line
  • Language: Java
  • Source: Maven repository, version 1.3.0

Additional context
The problem seems to be reproduced with models 4o and 4o-mini, but not with "gpt-3.5-turbo-0125". Change constant MODEL_ID in class LightsExample to see the difference.

Activity

  1. changed the title [-]Bug: Java - An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'[/-] [+].Net: Bug: Java - An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'[/+] on Oct 27, 2024
  2. changed the title [-].Net: Bug: Java - An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'[/-] [+]Java: Bug: Java - An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'[/+] on Oct 27, 2024
  3. changed the title [-]Java: Bug: Java - An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'[/-] [+]Java: Bug: An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'[/+] on Oct 27, 2024
  4. moonbox3 commented on Oct 28, 2024

    @moonbox3
    Collaborator

    Hi Milad (@miladhub), the SK Java repo has moved here: https://github.com/microsoft/semantic-kernel-java. Can you please open your issue in that repo, so the appropriate team members can help? Thanks.

  5. miladhub commented on Oct 28, 2024

    @miladhub
    Author
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    .NETIssue or Pull requests regarding .NET codebugSomething isn't workingjavaIssue or PR regarding Java codetriage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions