
Tensor considered harmful - singhrac
http://nlp.seas.harvard.edu/NamedTensor
======
londons_explore
I believe historically dimensions were indexed because the exact order made a
_huge difference_ for performance. Hand coded CUDA would only work for the
exact order it was designed for.

Now that nearly all frameworks have one/many optimisation passes which can
transpose/rearrange/split data transparently for performance, this reason
seems to go away.

------
aportnoy
How does this compare with xarray?

------
The_suffocated
Can those developers who write deep learning libraries stop calling multi-
dimensional arrays "tensors"? This marketing gimmick is really annoying. One
may represent a tensor as a multi-dimensional array, but multi-dimensional
arrays per se are not tensors. Calling them tensors is like calling every
object a "vector". While technically correct (given any field, every set can
be viewed as a vector space and hence every object --- whatever it is --- is
technically a vector in some vector space), this is not how we think in normal
contexts. The case with arrays vs tensors is similar. Calling them tensors is
just bending the language to make arrays sound fashionable.

Actually, I suspect that those "tensors" in TensorFlow or the like have
nothing to do with the tensors in physics/mathematics at all. The only
connection of arrays to "tensors" that I can find in various deeping learning
tutorials is low-rank approximation of a matrix (but even this can be put in
matrix language without mentioning any tensor). Apparently all other
operations in those "tensor" libraries are ordinary array operations.

~~~
singhrac
Out of curiosity, what would you prefer? I think MultiDimensionalArray is a
bit verbose, and numpy already has a hold on just `array`.

I don't think the notation comes from any claims or delusions about what's
being represented, it's just a convenient term. I can understand the
frustration though.

~~~
The_suffocated
I haven't any preferred term. You may call it multiarray, mdarray, ndarray,
hdarray, MDA etc..

I don't agree that "tensor" is a good convenient term. E.g. C++'s std::vector
does not really support vector operations, but most programmers can guess that
it is a 1D array in terms of memory layout. "Tensor"? It's just a meaningless
term to most beginners. If you think MultiDimensionalArray is too verbose, you
may call it "MDA". It is shorter, conveys the meaning of the term better than
"tensor" does and it's by no means misleading.

~~~
Chinjut
To precisely the extent that a vector is an array, we have also that a tensor
is a multidimensional array. A multidimensional array-of-scalars type is
precisely the tensor product of the appropriate one-dimensional array-of-
scalars types, in the standard math sense of "tensor product": linear maps out
of the former are in natural correspondence with multilinear maps out of the
latter.

The only things I see to complain about are that vector spaces don't have to
come with a choice of ordered finite basis and that arrays don't have to be of
scalars from a field. But if you're already happy with the conflation of
"vector" and "array", nothing new is happening in then speaking of
multidimensional arrays as "tensors"; it's just continuing the same
conflation.

~~~
The_suffocated
_But if you 're already happy with the conflation of "vector" and "array"_

Hmm, no, I am not happy with that. All I'm saying is that while std::vector is
a bad name, most people can immediately get that it has an array-like memory
layout. So, there is at least some merit with this inaccurate name. This is
not the case with "tensor", a term that most programmers have never heard of
(at least before the advent of TensorFlow).

