Also to add- the one service that was fast enough on the LLM side was Cerebras. ...

hassaanr 63 days ago | parent | context | favorite | on: Show HN: A real time AI video agent with under 1 s...

Also to add- the one service that was fast enough on the LLM side was Cerebras. The time to first token (ttft) is incredibly fast (200-300ms) and the t/s is 2000t/s for 8B- combined making for a great conversational experience.