Also curious about the potential users of your product, do you target individual users, small businesses, or large enterprises? Pursuing SOC2 and HIPPA make me think about the large ones; but aren't they already happy using hyperscalers?
Not to mention GCP and Azure both have confidential GPU offerings. How do you compete against them, as well as some startups mentioned in other comments like Edgeless Systems and Opaque Systems?
Just noticed Tinfoil runs Deepseek-R1 "70b". Technically this is not the original 671b Deepseek R1; it's just a Llama-70b trained by Deepseek R1 (called "distillation").
GCP has confidential VMs with H100 GPUs; I'm not sure if Google would be interested. And they get huge discount buying GPUs in bulk. The trade-off between cost and privacy is obvious for most users imo.
Great work! I'm interested to know where the GPU servers are located. Are they in the US; do you run your own datacenter or rent servers on the hyperscalers?
Yes, in the US right now. We don't run our own datacenters, though we sometimes consider it in a moment of frustration when the provider is not able to get the correct hardware configuration and firmware versions. Currently renting bare metal servers from neoclouds. We can't use hyperscalers because we need bare metal access to the machine.
That's the best part, you don't. You only need to trust NVIDIA and AMD/Intel.
Modulo difficult to mount physical attacks and side channels, which we wrote more about here: https://tinfoil.sh/blog/2025-05-15-side-channels
Not to mention GCP and Azure both have confidential GPU offerings. How do you compete against them, as well as some startups mentioned in other comments like Edgeless Systems and Opaque Systems?