On: If you don't run your own LLM proxy, how do you know whats going on?

[pavelmelnichuk]

not really, for example relay gives you model "groups" ie: fallbacks if one is down, or prefer to use a free-endpoint while you have daily capacity, etc.

Maybe you need a decision model to route, or maybe you want to have guardrail-type models pre or post response for an enterprise client.