Service to auto route LLM/Model traffic

[aikin-nivedit]

I suggest creating a service that monitors all the LLMs/AI model deployment services on Azure, AWS, Google Cloud, Groq, Krutrim, and other cloud deployments and routes your traffic based on availability, latency, rate-limiting, and other parameters. Manage everything at the backend, just pay a single unified bill, with no deployments or accounts on any other service. Sounds cool?

[yorwba]

Sounds similar to SkyPilot: https://docs.skypilot.co

But you still need to create your own accounts for each service.