Thanks, yes we're seeing a lot of use cases being unlocked by function calling.
To answer you first question, function calling models are trained to detect when a function should to be called (depending on the input) and to respond with JSON that adheres to the function signature.
A thoughtful prompt and a loop where they parse outputs and check if it corresponds to the arguments of the function is the ReAct model of building agents, which I've found starts to deteriorate in quality after about 5 tools.
Whereas with function calling, because the LLM is trained to detect/call functions it can work very well up-to 30-40 functions -- after which we need to start using techniques like delegation.
Hi HN, chat with PDFs is the first AI App everyone builds but quickly we realize that basic prompt stuffing doesn’t work. So I put together a PDF AI that uses function calling: https://pdf.aidev.run
It intelligently figures out if:
> The question needs retrieval or web search
> If it needs retrieval, does it need to search the latest doc, a specific doc or all docs
> Produces an answer with context.
Give it a spin at: https://pdf.aidev.run and let me know what you think. Its a prototype so expect flaws, but if you share feedback i'll make sure to improve it.
