Why is Chat memory limit (top K) is removed in GA release?

[anupsajjan]

I was using chatMemory custom implementation in 1.0-M6 version with 'lastN' parameter to fetch only the last 'N' messages as chat history. But now I see that the interface itself doesnt support 'limit' and I see that a MessageWindowChatMemory implementation has been provided, but I have use case where I want user to be able to see their chat history as long as needed, but to LLM I want to pass last N messages as 'ChatMemory' where LIMIT would come handy. Why even fetch all messages?

Previously this method supported 'lastN' parameter List get(String conversationId, int lastN);

Manually removing additional messages.

[sunyuhan1998]

Because in fact, according to the current Spring AI definition of ChatMemory, it is actually a "chat memory", not a "conversation history".The concept of "chat memory" is closer to the concept of LLM conversation token context, which means it is a sliding window.I don't think there is any specific support for "conversation history" in Spring AI yet, so if you need such a feature, you'll need to implement it yourself for now.

[anupsajjan]

@sunyuhan1998 Thanks for the response !! I agree with you on the definition of chatMemory. But just one question though, if we do implement separate 'conversation history' wont it simply duplicate messages in both chatMemory and conversation history?

[sunyuhan1998]

What you mean is: if we implement an additional "Conversation History", will the "Conversation History" and the "Chat Memory" each store a duplicate copy of the chat data?I think so, at least so far, there is no perfect solution.

[anupsajjan]

@sunyuhan1998 Yeah right, I meant that.
Okay got it thanks!

[fanxt0218]

@sunyuhan1998 Thanks for the response !! I agree with you on the definition of chatMemory. But just one question though, if we do implement separate 'conversation history' wont it simply duplicate messages in both chatMemory and conversation history?

I think it is possible to manually store and load session history, but it is not just about copying from the conversation. You can filter the memory yourself during storage or loading, which can effectively make LLM answers more in line with your ideas. Max_cessages only limits the maximum number of conversations for the current session, but you can still save all the history in the database and select valid ones to load into memory. At least that's what I did when session memory didn't support JDBC in the M7 version, and even now it can still be used normally and maintain high-performance response