I think you can build another package on top of this that reduces prompt/messages length or filters based on in-memory vector search or your matching algo. This would be advertised as - "make sure your prompts don't go over ___", or reduce your prompt input tokens by __ based on the last message ny tbe user.
I think you can build another package on top of this that reduces prompt/messages length or filters based on in-memory vector search or your matching algo. This would be advertised as - "make sure your prompts don't go over ___", or reduce your prompt input tokens by __ based on the last message ny tbe user.