
The Challenges of Building Inferencing Chips - rajnathani
https://semiengineering.com/the-challenges-of-building-inferencing-chips/
======
MeteorMarc
The paper is not very clear about what they mean with inferencing. It seems
they just mean calculating a prediction for new data using a trained machine
learning model. Readers outside the field, like me, are likely to confuse
inferencing with inferring (using logic), but that is out of scope, here.

Edit: find the real explanation here:
[https://blogs.nvidia.com/blog/2016/08/22/difference-deep-
lea...](https://blogs.nvidia.com/blog/2016/08/22/difference-deep-learning-
training-inference-ai/)

~~~
shmageggy
Also not to be confused with probabilistic inference.

There are too many overloaded words in statistics and related fields.

------
anonymousDan
I've played around with some of the new 'Edge' accelerators (specifically Edge
TPU and Intel NCS 2). The performance and energy efficiency of the Edge TPU
especially is really amazing for the price. Does anyone know of any other edge
devices worth evaluating at a similar price point (other than Nvidia Jetson)?

~~~
rajnathani
If your model can fit within 10MB, and is based on CNNs, then Gyrfalcon’s [0]
‘compute in memory’ inference chips can in theory provide 3-4x higher
TOPS/watt compared to the above inference chips you’ve mentioned, with around
12.6 TOPS/watt for one of their latest chips [1]. Their chips already seem to
be in some new devices [2] [3].

[0] [https://www.gyrfalcontech.ai/](https://www.gyrfalcontech.ai/)

[1] [https://www.eetimes.com/gyrfalcon-unveils-fourth-ai-
accelera...](https://www.eetimes.com/gyrfalcon-unveils-fourth-ai-accelerator-
chip/)

[2] [https://www.zdnet.com/article/ai-comes-to-the-edge-with-
soli...](https://www.zdnet.com/article/ai-comes-to-the-edge-with-solidrun-and-
gyrfalcons-ai-inference-server/)

[3] [https://www.prnewswire.com/news-releases/lg-launched-
phone-p...](https://www.prnewswire.com/news-releases/lg-launched-phone-
powered-by-gyrfalcon-technology-incs-ai-chip-300933178.html)

