
Python 3.5 to Ship with Matrix Multiplication Operator - madisonmay
http://legacy.python.org/dev/peps/pep-0465/#implementation-details
======
zurn
To everyone arguing against special-casing matrix multiplication, please base
your arguments on the PEP's "Motivation" section to avoid too much of
rehashing the obvious. It even has a subsection "But isn't matrix
multiplication a pretty niche requirement?"

~~~
wyuenho
It seems like I can boil down the rationale as:

1\. Numpy people overload * for element-wise mul and matmul inconsistently and
confusing people 2\. The prefix func calls convention for matmul makes the
formulas difficult to read 3\. Python's precedence for splitting / into / and
// can't apply to matmul because * * is already taken. 4\. ` is banned, ?!
lend unrelated meanings to context, $ is Perl and PHP baggage, so @ is the
only thing left?

I got lost on the choice for @. If ?! lends unrelated meaings, and $ is
Perlism, why doesn't @ suggest some kind of concat ops or Bashish/Perlish
array sigils?

I'd personally much prefer >< . It looks like x, so it's much clearer. I don't
understand the PEP's reason for not using ><.

~~~
kzrdude
Yes `backticks` are banned since Python 3 because 1) They are hard to type
many on common keyboard layouts 2) They are hard to read, especially in Python
books.

~~~
tomp
I don't understand the hate for backticks either. I used to dislike them too,
when I was just a hobby programmer, and used a Slovenian keyboard layout (even
for programming). Needless to say, after switching to English layout, my
programming skills quadrupled - ` isn't the only symbol barely accessible
(without finger acrobatics) on international keyboards (other examples
include: {}][|~\^ ).

~~~
kzrdude
I've stuck to Swedish layout but I've remapped all of `{}[]()\ in fact! :-)

------
iskander
This is surprisingly detailed design rationale which weighs many alternatives
and gives careful consideration to possible tradeoffs. Great job by Nathaniel
Smith and the numerical Python community.

------
willvarfar
The nice thing is that it makes a nice symbol `@` available to objects that
aren't matrices, to do with as they want ...

The (ab)use seems endless :)

~~~
gkya
I'd love it to be used as value membership operator for dictionaries.
Something like this in pseudo-code:

    
    
      class object:
          def __matmul__(self, dict):
              return self in dict.values()
    
      a = {'asdf': 1}
      if 1 @ a: # Evaluates to True.
          print('Lovely!')

~~~
ecma
That seems too esoteric to ever go into the language unfortunately. I love the
idea of some sort of __in__ targeting values in mappings but have never been
able to think of an elegant mechanism that uses the existing grammar sanely or
introduces useful new grammar.

I've always thought that maps and a matching __maps__ would have been a nicer
way of referring to the keys of a mapping with in/__in__ referring to the
values. Although the semantics are a bit odd in that case. Technically the
mapping maps a key, but the language semantic to to have it the other way
around as in key is mapped by mapping.

    
    
        foo = {'bar': 42}
        assert 42 in foo
    

but which makes more sense below?

    
    
        assert 42 maps foo
        assert foo maps 42
    

or even

    
    
        assert 42 mapped by foo
    

The benefits of including by in the grammar could be of consequence here too.

~~~
kzrdude
What's wrong with this?

    
    
       if 42 in d.values(): pass

~~~
ecma
That's why none of the alternative solutions are attractive. They don't really
improve on the existing semantics.

What bugs me is the implicitness of __in__ defaulting to keys for a mapping.
We have .keys() and .values() which are nice and clear since they explicitly
grab an object which has an unambiguous definition for __in__.

Double Edit: It's an implementation decision that had to be made and I'm not
aware of the rationale or debate behind the original choice. It's directly
attributable to the decision to make __iter__ return the keys for a mapping
which is the root of my issue. It's presumably useful and convenient (from a
language writer's perspective) for iteration of a mapping to be along the keys
in whatever order they may be traversed but if that's all it comes down to,
why should a mapping be iterable at all when there is obvious ambiguity in
what may be iterated? Maybe there is some history I'm not aware of where the
.keys(), .values() and .items() methods were introduced post-hoc and the
previous behaviour was such due to their non-existence and the need to iterate
and then index to get all values in the object.

~~~
knome
Looking up the value using "in" in a dict would be inefficient as it would
have to iterate over the contained values.

If you want to efficiently look for values, use a set() or store a second dict
that goes from values to keynames.

If you're not worried about the efficiency, do the "in d.values()" you've
suggested.

Using "in d.keys()" takes a quick hash lookup and turns it into a slower
iterative list lookup.

~~~
ecma
I think I'm combining issues in a problematic way here and I don't even know
what was going on when I typed __in__ (twice! :/) instead of __contains__ -
I'll just blame the lack of coffee. I realise why the membership 'in' should
operate on the hash table in memory for O(1) performance and it makes perfect
sense. I've just never been fully comfortable with the idea that keys are
considered the 'members' of a 'mapping' in and of themselves. Maybe because I
usually consider one-way mapping as a special case.

The more reasonable (I think) problem I have is one of language semantics
which is introduced by _in_ being applicable to a mapping in the first place -
why should a mapping be iterable at all if it is ambiguous (as I believe) as
to what it should return? Members should be key:val pairs but membership
checks refer only to keys and it's probably not unreasonable for a user to
want any of .keys(), .values() or .items() when iterating. That's obviously
why they're made available so why should __iter__ special case one of them?

I assume this is to match semantics introduced by the membership check and
probably historical because without a .values() method, key iteration and
lookups would be the usual way to get all values out of a mapping. This just
seems like the kind of thing that could have been changed in Python3 (although
2to3 would probably not have been able to handle the syntax changes
automatically).

~~~
maxerickson
The code to test for membership of a key value pair is already straightforward
for any mapping:

    
    
        map[key]==value
    

Which is a weak reason in support of the use of containment testing for
something else.

------
madisonmay
Guido has accepted the proposal and work has now begun to add support for the
new operator:
[http://bugs.python.org/issue21176](http://bugs.python.org/issue21176)

------
lmm
After a few years in Scala, where all operators are methods and vice versa (2
+ 3 is just sugar for 2.+(3)), the method-operator distinction just seems
_weird_.

~~~
zephyrfalcon
Hm, in Python, 2 + 3 is just sugar for (2).__add__(3). Not that different;
Python just doesn't allow symbols like '+' in method names.

~~~
lmm
> in Python, 2 + 3 is just sugar for (2).__add__(3).

Sure, but it doesn't work the other way; there's no nicer way to write
2.florble(3). So you have a few operator methods, and then other methods are
second-class citizens. Which in turn encourages people to abuse the operators
as shortcuts for commonly-used operations, á la C++'s <<

~~~
klodolph
> Which in turn encourages people to abuse the operators...

In practice, very few languages have suffered problems from operator
overloading. I can't think of another language where this is a problem (other
than C++), which makes me think it's a cultural/code style problem, rather
than a problem with the language itself. C++ seems to encourage abuse because
everyone learns its IO library, and its IO has an API design so horrible it
should make you retch.

By comparison, I'm not sure that adding more operators makes the resulting
code any more palatable. Look at Haskell: libraries define their own
operators, and the operators in Haskell have a steep learning curve. Can you
tell me, without looking at a chart, whether $ or >>= has higher precedence,
and what kind of associativity they have?

~~~
lmm
> Can you tell me, without looking at a chart, whether $ or >>= has higher
> precedence, and what kind of associativity they have?

No, which is why I don't think that should be allowed. Scala doesn't do that -
all "custom operators" have the same precedence, and the global rule is that
anything ending in : associates to the right (and is defined by the thing on
its right), otherwise to the left. So I could answer those questions easily
for a Scala library.

~~~
klodolph
Those weren't custom operators in Haskell, they are part of the prelude.

------
lutorm
Since this is just a special case of a multiply-and-add indexing loop, maybe
they should just introduce some form of tensor notation, so that A[i,j]*B[j,k]
is the matrix product of A and B? That would extend to so many more use cases
than just a 2d matrix product.

~~~
gaius
U+2297 CIRCLED TIMES is already the logarithm operator in APL, so you'd want
to avoid that confusion. Phew!

------
JadeNB
The PEP claims that:

> Matrix multiplication is more of a special case. It's only defined on 2d
> arrays (also known as "matrices")

but this is not true. Matrix multiplication is just a special case of
contraction of indices in a tensor
([https://en.wikipedia.org/wiki/Tensor_contraction)—probably](https://en.wikipedia.org/wiki/Tensor_contraction\)—probably)
the most frequently used case, but not the only one. I'm certainly not arguing
for the inclusion of general tensor-manipulating operators in Python, but it
does seem to suggest a sensible alternative to:

> For inputs with more than 2 dimensions, we treat the last two dimensions as
> being the dimensions of the matrices to multiply, and 'broadcast' across the
> other dimensions.

namely, just contract on the inner indices. That is, arr(n1, ..., nk, m) @
arr(m, p1, ... pl) = arr(n1, ..., nk, p1, ..., pl).

EDIT: scythe
([https://news.ycombinator.com/item?id=7554013](https://news.ycombinator.com/item?id=7554013))
already pointed this out in passing.

~~~
onedognight
> namely, just contract on the inner indices. That is, arr(n1, ..., nk, m) @
> arr(m, p1, ... pl) = arr(n1, ..., nk, p1, ..., pl).

No. This is what a mathematician might assume the PEP proposes without
actually reading it. It instead proposes an entirely non-obvious definition
which not equivalent to what you wrote.

In particular consider this example from the PEP.

    
    
        arr(10, 2, 3) @ arr(10, 3, 4) = arr(10, 2, 4)

~~~
JadeNB
I know it is not equivalent—that is why I proposed it as a sensible
alternative to what the PEP proposes (which I quoted). The punctuation may
have made it unclear, but what I was trying to say was:

> … a sensible alternative to [PEP proposal]; namely, just contract on the
> inner indices.

and not

> … a sensible alternative to contracting on the inner indices.

My argument for why it's sensible is precisely what you mentioned, namely,
that it is what a mathematician would expect.

------
JadeNB
My issue is not with the matrix-multiplication operator—I'm a mathematician
before I'm a programmer, and so am all for it—but with the vector-to-matrix
promotions: why not just consistently promote vectors to _columns_ (or rows,
if BDFL swings that way)? This would achieve almost the same effect as
promoting to rows or columns as needed, and would avoid the non-associativity
problem that the PEP mentions.

This PEP seems to imply that the cost would be a profusion of superfluous
`newaxis`s, but I can't see that: it seems to me that you would need only to
remember which kind of promotion works by default, and sprinkle in a `.T`
whenever you need the other kind. (Anyone who's uncomfortable with lots of
`.T`s in matrix-crunching code is not, I submit, someone who writes or reads
lots of matrix-crunching code.)

------
segphault
There are a few handy things that I'd like to see added to Python's syntax. A
dedicated operator for matrix multiplication isn't one of them.

~~~
aberrant
What would you like to see?

~~~
shadowmint
multi-line lambdas.

~~~
jdpage
Not happening, because it would mean multi-line function arguments, which have
been declared "ugly".

[http://legacy.python.org/dev/peps/pep-3099/](http://legacy.python.org/dev/peps/pep-3099/)

------
pattern
I really like this snippet from the justification for the symbol chosen: "APL
apparently used +.×, which by combining a multi-character token, confusing
attribute-access-like . syntax, and a unicode character, ranks somewhere below
U+2603 SNOWMAN on our candidate list".

One of the reasons I have been such a fan of Python for so long is the
relatively no-nonsense approach to design decisions that many others would
have rushed through.

------
pekk
Python's syntax is already pretty ponderous, this kind of thing pushes it
rapidly toward bash-like incomprehensibility. I've been a big fan of the
language for years but this kind of thing makes me consider jumping to Lua or
something which is more sparing with new complicated sigils.

~~~
Borogove
How does this reduce comprehensibility? In cases where you're doing matrix
muls, "a @ b" is way clearer than "a.dot(b)". In other cases, infix "@" won't
appear.

------
NamTaf
Matlab's approach to using * as the matrix multiplier makes sense, because
every numerical variable is an array with integers/floats simply being a 1x1
array. Using .* then as the elementwise version works.

I'd have personally preferred to see Python do type testing on the variables -
it already does. For example:

'quick' \+ 'fox' = 'quickfox' 3 + 5 = 8 [3] + [5] = [3,5]

So why not make it a case where * on an int or float does the 'standard'
multiplication that already exists whereas * on an array does matrix
multiplication?

You arrive at the problem of then not having an element-wise version of the
multiplication but it's not as if this solves that problem anyway.

What am I missing to make that a problem?

~~~
apl

      > You arrive at the problem of then not having an element-
      > wise version of the multiplication but it's not as if this
      > solves that problem anyway.
    

Please reread the text. It's exactly what it does, and an exhaustive rationale
is given for why both operators are useful and necessary.

    
    
      # Matrix multiplication:
      a = b @ c
    
      # Element-wise multiplication:
      a = b * c

~~~
NamTaf
My apologies, I misremembered that from when I first read it. I'm still on
python 2.7 - in 3, does:

[3] * [5] = [15]

? Because really it needs to reduce down to that if it's going to make sense
from a continuity point of view.

edit: To clarify: Is that already part of 3.x, or shall it become part of 3.x
once this is implemented?

~~~
apl
A Python list is not a matrix, so I don't see how this change would entail
your statement. Of course, for very specific use cases, a list looks like a
numerical vector; the semantics are very different, though.

------
BoppreH
I think "Matrix Multiplication" is a little too specific for a syntax change,
but I welcome the addition of a new operator to be overloaded.

Am I missing some useful applications of matrix multiplication on a general
purpose language?

~~~
heron87
Ehm, Python is not a "scripting language". It is a — as you said — general
purpose programming language. And it is becoming more and more popular for
scientific applications, where matrix multiplication plays a key role.

~~~
BoppreH
I honestly didn't expect a downvote for calling Python a "scripting language".
I use the term loosely and always thought Python fit spot on. I still think it
applies here, but I've edited the parent post to correct this possibly
incorrect usage of the word.

Just for the record, these are the characteristics of Python that made me call
it "scripting":

\- Interpreted

\- Highly expressive

\- No boilerplate

\- Vast majority of programs are short

\- Lots of libraries for gluing stuff together

\- Used for turing-complete customization of software in other languages
(e.g.: Sublime Text)

\- Recommended as replacement for Bash in some cases

Anyway, I'm happy to see the language evolving to meet its users' needs.

~~~
wisty
Stolen from
[https://www.python.org/doc/humor/](https://www.python.org/doc/humor/) (in
response to a claim that it's not a true OO language):

> Indeed, and because it doesn't support closures, it's not a true functional
> programming language either. And because you have to import all sorts of
> modules to do the simplest things (e.g., regular expressions), neither is it
> a true scripting language. Indeed, because it doesn't support labeled break
> or continue statements, it's not even a true structured programming
> language.

I think the point is, trying to classify programming languages (except domain
specific ones like SQL or maybe PHP) is kind of pointless.

But to answer your real question, Python is one of the top languages in number
crunching. After web work, that's Python's second biggest niche.

~~~
TillE
> it doesn't support closures

It does now, and thank goodness, because even C++ has closures these days.

------
Aardwolf
They gave a rationale for @ compared to other not yet used ascii characters.

But they didn't consider a combination of multiple characters, like e.g. 3 *
s?

(NOTE: Hacker news does not allow typing 3 * in a row)

~~~
ecma

      I suspect this was to distance matrix mult
      from traditional scalar mult operators.
      Remember that ** is just shorthand for n *s.
      whereas matrix mult is a different beast
      entirely.
    
      There is also added clutter with 3* where
      typos may be non-obvious (already a potential
      issue with ** I guess).
    

Nice, using verbatim to get around markup...

------
JoshTriplett
Personally, I'd really like to see an equivalent in Python to Haskell's
`binop` , to turn an arbitrary two-argument function into a binary operator.

~~~
JadeNB
I think that you're not alone, but this is specifically considered and
rejected (in the section on "Rejected alternatives …"):

> Add lots of new operators, or add a new generic syntax for defining infix
> operators: In addition to being generally un-Pythonic and repeatedly
> rejected by BDFL fiat, this would be using a sledgehammer to smash a fly.

~~~
JoshTriplett
Anyone have a reference for where that in particular was rejected by BDFL
fiat?

~~~
takluyver
PEP 225 was an attempt to add a load of new operators that was rejected (well,
deferred 14 years ago). I don't know of a proposal to define arbitrary binary
operators.

------
mckoss
Why not use the existing convention for names of infix operators like __mmul__
but allow then to be infix as well.. So, we could write A __mmul__ B.

I think this is more clear than @ and is already familiar to programmers that
deal with operator overloading methods. It would also enable the addition of
arbitrary infix operators to python.

~~~
atlbeer
__ methods are private and not meant to be called outside of the class

The @ decorator vs @ MMUL seems pretty syntactically clear though.

@decorator()

def function():

variable @ variable

------
fest
I hope this is not a first step towards transforming python syntax into perl.

~~~
jdpage
It won't be. The Python parser has to remain LL(1) by BDFL fiat.[1]

1:
[http://legacy.python.org/dev/peps/pep-3099/](http://legacy.python.org/dev/peps/pep-3099/)

~~~
lamby
OP was clearly referring to style and aesthetics, not the grammar's level in
the Chomksy hierarchy.

------
beefsack
You've got to wonder, does matrix multiplication happen so often and is it so
inconvenient to call as a function that it's worth complicating the syntax?
I'm really not sure about this one.

~~~
yen223
I'd say matrix multiplication is far more common than complex numbers, and
Python natively supports those.

~~~
baq
yeah but complex numbers don't introduce new punctuation aka symbolic noise.

------
riffraff
one question I could not see from the PEP page: why not use a spelling "dot"
instead of a symbol? i.e. "a dot b"

~~~
alexchamberlain
When dealing with vectors, the dot product has a specific definition, so it
would be very confusing.

------
waitingkuo
Currently, we can use * to multiply two numpy matrix. We'll have two choices,
* and @, to do the matrix multiply things in numpy. Is it pythonic? Or should
we deprecate * ?

~~~
apl
No, we should deprecate numpy.matrix which is the culprit here. If there's
only numpy.ndarray, the semantics are absolutely clear: * for element-wise
multiplication and @ for "proper" multiplication of two matrices.

------
bsaul
from the pep as to why not using a type that defines _mul_ as matrix
multiplication : " The result is a strong consensus among both numpy
developers and developers of downstream packages that numpy.matrix should
essentially never be used, because of the problems caused by having
conflicting duck types for arrays "

looks to me like an issue with python lack of static type checking. if you
need to define an operator every time two types conflict, you're in for a long
list...

------
0xdeadbeefbabe
Dear Guido,

You lost me at 3.

