
How Transformers Work – Model Used by Open AI and DeepMind - giacaglia
https://medium.com/@giacaglia/transformers-141e32e69591
======
rasmi
Here are two more great articles about Transformers:

The Illustrated Transformer (referenced in the parent):
[http://jalammar.github.io/illustrated-
transformer/](http://jalammar.github.io/illustrated-transformer/)

The Annotated Transformer:
[http://nlp.seas.harvard.edu/2018/04/03/attention.html](http://nlp.seas.harvard.edu/2018/04/03/attention.html)

~~~
giacaglia
I reference one of the articles but I hadn’t look at the other one! Very
interesting. Thanks for sharing

~~~
panic
You do more than reference it -- you've copied a bunch of text and figures
from it as well. Search for "The encoder’s inputs first flow through a self-
attention layer" and read on from there. Most of the article is a word-for-
word copy.

~~~
giacaglia
I’ve tried to use a bunch of the figures and information from these articles.
I hope it was useful for some people

~~~
throwawaymath
Regardless of whether or not it's useful, it's substantially plagiarized from
another source. You don't have an inline citation or visual indication that
many of the figures are copied from another article on the topic. The same
goes for copying paragraphs with extremely minimal modifications.

Slightly changing sentence structure is not paraphrasing or stating in your
own words. Pointing a reader to an article for further reading is not the same
as a citation.

To put it bluntly, your arrangement of the material, substantial paragraphs,
and a significant number of your figures/graphics are copied from elsewhere
without citation.

~~~
giacaglia
I'm adding a note to the _self-attention_ section adding the fact that this
was taken from another blog post

~~~
danieldk
Note that if the other blog post is not licensed under a license that allows
you to do so (such as Creative Commons Share-Alike), you are simply not
permitted to copy images or text without explicit permission from the authors.
It's not enough to state that it was taken from another post.

Of course, you are allowed to cite excerpts, but then the text should be
clearly marked up as a citation.

------
FactolSarin
I'm going to be honest, I thought this was going to be about Transformer
characters desired by a Deep Learning AI

~~~
inamberclad
I thought this was about the electrical components. I was like, why would you
need a neural network when there are physical laws?

~~~
seotut2
I too was expecting to see some derivations of Maxwell's equations, inductor
behavior etc.

