Ask HN: Best Practices for LLM Chatbot that references user account details?

[olives]

What's the current best practice for including personalized user account details (e.g. for queries like "How much did I spend on this credit card last month?") into a LLM chatbot?

Specifically, assuming the user has already been authenticated properly, how is the user details retrieved, how is the data re-formatted, and how is the data ultimately fed into the LLM? This is a function in many popular banking chatbots, e.g. Bank of America's Erica, but there is scarce information about the actual implementation.

[brianjking]

This is often called Function calling or tool use where you provide the LLM the ability to execute API calls to external tools.

[olives → brianjking]

Is function calling the only way to do this? If the user account has some associated text, for example, historical medical files, is there a way to pass that text in such that the user could query "Write a short summary about my medical history?"

[fanweixiao → olives]

You can create a function calling, the parameter is the user_id or a fingerprint of the user for backend. then, get all data from your database, returned as a JSON object, then, you will get the result.

The prompt will looks like:

""" you are a professional medical history recorder, write a short summary about the user's medical history, use tool_call when possible.

the current user_id is: abcd12345 """

your tool_call description should be set clearly, looks like:

""" get medical history by given user_id """