Neat, this is like a SaaS version of Langchain[1]. For those who haven't already seen it, Langchain is a framework that gives you the ability to do this as well as integrate out of the box solutions for other common problems like memory.
Langchain is awesome. Another great library is GPT Index. This field is apparently being called "prompt orchestration" (getting data from external sources, chaining going back and forth with an LLM, etc).
[1] https://langchain.readthedocs.io/en/latest/