|
2 | 2 | title: Creating and managing a chat history object |
3 | 3 | description: Use chat history to maintain a record of messages in a chat session |
4 | 4 | zone_pivot_groups: programming-languages |
5 | | -author: matthewbolanos |
| 5 | +author: evanmattson |
6 | 6 | ms.topic: conceptual |
7 | | -ms.author: mabolan |
8 | | -ms.date: 07/12/2023 |
| 7 | +ms.author: evmattso |
| 8 | +ms.date: 01/20/2025 |
9 | 9 | ms.service: semantic-kernel |
10 | 10 | --- |
11 | 11 |
|
@@ -415,6 +415,103 @@ chatHistory.addAll(results); |
415 | 415 |
|
416 | 416 | ::: zone-end |
417 | 417 |
|
| 418 | +## Chat History Reduction |
| 419 | + |
| 420 | +Managing chat history is essential for maintaining context-aware conversations while ensuring efficient performance. As a conversation progresses, the history object can grow beyond the limits of a model’s context window, affecting response quality and slowing down processing. A structured approach to reducing chat history ensures that the most relevant information remains available without unnecessary overhead. |
| 421 | + |
| 422 | +### Why Reduce Chat History? |
| 423 | +- Performance Optimization: Large chat histories increase processing time. Reducing their size helps maintain fast and efficient interactions. |
| 424 | +- Context Window Management: Language models have a fixed context window. When the history exceeds this limit, older messages are lost. Managing chat history ensures that the most important context remains accessible. |
| 425 | +- Memory Efficiency: In resource-constrained environments such as mobile applications or embedded systems, unbounded chat history can lead to excessive memory usage and slow performance. |
| 426 | +- Privacy and Security: Retaining unnecessary conversation history increases the risk of exposing sensitive information. A structured reduction process minimizes data retention while maintaining relevant context. |
| 427 | + |
| 428 | +### Strategies for Reducing Chat History |
| 429 | + |
| 430 | +Several approaches can be used to keep chat history manageable while preserving essential information: |
| 431 | + |
| 432 | +- Truncation: The oldest messages are removed when the history exceeds a predefined limit, ensuring only recent interactions are retained. |
| 433 | +- Summarization: Older messages are condensed into a summary, preserving key details while reducing the number of stored messages. |
| 434 | +- Selective Retention: Only specific message types (such as user queries and key responses) are retained, while redundant or lower-priority messages are discarded. |
| 435 | +- Threshold-Based Reduction: A reduction process is triggered only when the chat history reaches a certain threshold, preventing premature truncation while maintaining efficiency. |
| 436 | + |
| 437 | +A Chat History Reducer automates these strategies by evaluating the history’s size and reducing it based on configurable parameters such as target_count (the desired number of messages to retain) and threshold_count (the point at which reduction is triggered). By integrating these reduction techniques, chat applications can remain responsive and performant without compromising conversational context. |
| 438 | + |
| 439 | +::: zone pivot="programming-language-csharp" |
| 440 | + |
| 441 | +> Content About Chat History Reduction in C# is Coming Soon. |
| 442 | + |
| 443 | +::: zone-end |
| 444 | + |
| 445 | +::: zone pivot="programming-language-python" |
| 446 | + |
| 447 | +In this section, we cover the implementation details of chat history reduction in Python. The approach involves creating a ChatHistoryReducer that integrates seamlessly with the ChatHistory object, allowing it to be used and passed wherever a chat history is required. |
| 448 | + |
| 449 | +- Integration: In Python, the `ChatHistoryReducer` is designed to be a subclass of the `ChatHistory` object. This inheritance allows the reducer to be interchangeable with standard chat history instances. |
| 450 | +- Reduction Logic: Users can invoke the `reduce` method on the chat history object. The reducer evaluates whether the current message count exceeds `target_count` plus `threshold_count` (if set). If it does, the history is reduced to `target_count` either by truncation or summarization. |
| 451 | +- Configuration: The reduction behavior is configurable through parameters like `target_count` (the desired number of messages to retain) and threshold_count (the message count that triggers the reduction process). |
| 452 | + |
| 453 | +The supported history reducers are `ChatHistorySummarizationReducer` and `ChatHistoryTruncationReducer`. As part of the reducer configuration, `auto_reduce` can be enabled to automatically apply history reduction when used with `add_message_async`, ensuring the chat history stays within the configured limits. |
| 454 | + |
| 455 | +The following example demonstrates how to use ChatHistoryTruncationReducer to retain only the last two messages while maintaining conversation flow. |
| 456 | + |
| 457 | +```python |
| 458 | +import asyncio |
| 459 | + |
| 460 | +from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion |
| 461 | +from semantic_kernel.contents import ChatHistoryTruncationReducer |
| 462 | +from semantic_kernel.kernel import Kernel |
| 463 | + |
| 464 | + |
| 465 | +async def main(): |
| 466 | + kernel = Kernel() |
| 467 | + kernel.add_service(AzureChatCompletion()) |
| 468 | + |
| 469 | + # Keep the last two messages |
| 470 | + truncation_reducer = ChatHistoryTruncationReducer( |
| 471 | + target_count=2, |
| 472 | + ) |
| 473 | + truncation_reducer.add_system_message("You are a helpful chatbot.") |
| 474 | + |
| 475 | + is_reduced = False |
| 476 | + |
| 477 | + while True: |
| 478 | + user_input = input("User:> ") |
| 479 | + |
| 480 | + if user_input.lower() == "exit": |
| 481 | + print("\n\nExiting chat...") |
| 482 | + break |
| 483 | + |
| 484 | + is_reduced = await truncation_reducer.reduce() |
| 485 | + if is_reduced: |
| 486 | + print(f"@ History reduced to {len(truncation_reducer.messages)} messages.") |
| 487 | +
|
| 488 | + response = await kernel.invoke_prompt( |
| 489 | + prompt="{{$chat_history}}{{$user_input}}", user_input=user_input, chat_history=truncation_reducer |
| 490 | + ) |
| 491 | +
|
| 492 | + if response: |
| 493 | + print(f"Assistant:> {response}") |
| 494 | + truncation_reducer.add_user_message(str(user_input)) |
| 495 | + truncation_reducer.add_message(response.value[0]) |
| 496 | +
|
| 497 | + if is_reduced: |
| 498 | + for msg in truncation_reducer.messages: |
| 499 | + print(f"{msg.role} - {msg.content}\n") |
| 500 | + print("\n") |
| 501 | +
|
| 502 | +
|
| 503 | +if __name__ == "__main__": |
| 504 | + asyncio.run(main()) |
| 505 | +``` |
| 506 | +
|
| 507 | +::: zone-end |
| 508 | +
|
| 509 | +::: zone pivot="programming-language-java" |
| 510 | +
|
| 511 | +> Chat History Reduction is currently unavailable in Java. |
| 512 | +
|
| 513 | +::: zone-end |
| 514 | +
|
418 | 515 | ## Next steps |
419 | 516 | Now that you know how to create and manage a chat history object, you can learn more about function calling in the [Function calling](./function-calling/index.md) topic. |
420 | 517 |
|
|
0 commit comments