
The Reasonable Effectiveness of the Multiplicative Weights Update Algorithm - seycombi
https://jeremykun.com/2017/02/27/the-reasonable-effectiveness-of-the-multiplicative-weights-update-algorithm/
======
orasis
In the context of bandits, how does this perform compared to Thompson
Sampling?

~~~
j2kun
For bandit learning MWUA is called EXP3, and it's close to optimal (off from
the information-theoretical lower bound by a factor of log n, I believe).
Various improvements like EXP4 and EXP4p keep improving this bound.

The difference is it's optimizing for a different mathematical model, one
which assumes no stochasticity and the rewards can be arbitrary, and
adversarially chosen with full knowledge of the algorithm. This "full
knowledge" guarantee is what makes MWUA all about "adaptability" (and it's
what makes the linear programming example work so spectacularly in the post:
the roles are flipped and we as the algorithm designers get to be MWUA's
adversaries!)

I don't have a direct empirical comparison of EXP3 and Thompson sampling, but
I am delighted to say my blog post [1] comes up first when you google "Exp3"

[1]: [https://jeremykun.com/2013/11/08/adversarial-bandits-and-
the...](https://jeremykun.com/2013/11/08/adversarial-bandits-and-the-
exp3-algorithm/)

------
amelius
This is the first time I'm reading pseudocode of an algorithm before it is
explained what the algorithm is supposed to do.

~~~
aljungberg
I'd like if this were the norm. I found the pseudocode much more readable than
the mathematical description of what it was supposed to do.

I'm (obviously) biased as a programmer rather than a mathematician but
sometimes I think the study of mathematics would benefit tremendously from the
same kind of "readability is more important than conciseness" thinking we have
in software development.

Here's my proposal for a "contributing style" for mathematics:

Don't use short cryptic variable names like "wt". Do use informative full
variable names like "weight" and "step".

Don't use a subscript for something like "step": it's just tiny and hard to
read, especially once you start doing math in it. You have a series and you
want to talk about the nth element? Use programming like array indexing,
"weight[step]". It has the added benefit of being writable in pure ASCII
without a math formula editor.

Don't omit your binary operators, e.g. "wt(1 + G)". Write it out:
"weight[step] * (1 + reward)". You only write your formula once, but it will
be read many times. Saving 1 character is not a win in total, especially not
when it creates an ambiguous reading (are you using the function wt on 1 + G
or are you multiplying? No way to know without resolving wt which can involve
backtracking in the text for your reader.) Optimise for reading, not for
writing.

Don't use sigma for sums. Again readability trumps terseness. Instead of "S"
with a little "t from 1 to T" and a formula like "Rt/B" on the right, just
write it out: "sum(R[t] / B for t in range(1, T))".

Actually it would be neat if there existed a Chrome or Safari extension that
converted math notation to pseudo-Python. It wouldn't be able to assign
sensible variable names automatically but at least it'd save me so much time
and leaning in to stare at tiny subscripts.

I might be exceptionally lazy but sometimes when I read a math paper and the
dense piles of tiny script formulas begin to pile up my eyes just glaze over
even that in theory I understand every component. It's just so much effort to
mentally parse it. Like reading sample code in assembly. If readability was
improved I'm certain me and others like me would finish reading many more
papers and we'd all get a little smarter.

~~~
BuuQu9hu
You only found the pseudocode readable because you know Python and don't know
maths notation. It's not unreasonable to have to learn maths notation in order
to read maths papers and blogs.

You aren't exceptionally lazy. You are normally lazy, and additionally not
used to reading maths notation.

~~~
emn13
So I'm not a mathemetician, certainly not anymore.

But for a while I did read a lot of this stuff, and wrote things like it.

Aaand... to some extent you get used to it. But it really is simply a worse
syntax. It's not just habit.

For one thing, there does not exist such a thing as "mathematical notation".
There are a bunch of different, semi-overlapping conventions. You can't
trivially read something someone wrote if the math was for another field. In
fact, I'm pretty sure the variance is even larger than for most (seriously
used) computer languages. It's easier for a pythonista to grok C++ templates
than it is to read physics-inspired math with a modal-logic background.

This is particularly true when it comes to conventional symbols; there just
aren't enough, or perhaps it's history, but there are different symbols for
the same concept, and different concepts for the same symbol. Another pet
peeve is subscripts - are those formally meaningless tags to distinguish
variations the author talks about? are they arguments of some other class
(think type arguments vs. value arguments)? Are they something elses? It's
quite common for articles to use various forms, and sometimes even on the same
symbols; for an example (albeit with superscripts) you probably know: X^T as a
transpose, but X^2 as a power.

Then there are things like context sensitivity. It's not uncommon for some
notation not to be self-contained in the sense that an expression depends on
some variable, but that that variable is never specified in the expression
itself. It's taken for granted you know which those are, and that can get
quite tricky. It's the mutable global state of programming.

Mathematics could definitely benefit from a more disciplined and rigorous
approach when it comes to notation, at least, IMNSHO.

------
hervature
> Sanjeev Arora and his coauthors consider it “a basic tool [that should be]
> taught to all algorithms students together with divide-and-conquer, dynamic
> programming, and random sampling.”

As someone who took Sanjeev's Advanced Algorithms course this past semester, I
can confirm that he himself does indeed teach this.

