If you are building a state machine then you should use tools that enable you to do that (temporal, langgraph, etc) by orchestrating multple LLM calls - in that case archgw offers intelligent routing to your COT agent and enables you to transparently add rich observability and metrics for all calls made to LLMs in the COT/state machine scenario.