

A New Way To Solve Linear Equations (over finite fields) - ColinWright
http://rjlipton.wordpress.com/2012/08/09/a-new-way-to-solve-linear-equations?utm_source=twitter

======
colanderman
This is very neat, in that it uses random combinations of vectors to maintain
constant entropy over each iteration. Unfortunately this method does not
improve on the O(n^2*m) asymptotic run-time of Gaussian elimination, nor does
it address numerical stability issues (since it applies only to finite
fields). Still neat though.

~~~
bilbo0s
OK Cool.

I thought I was just being dense until I came back to HN from the article and
read this comment.

------
andrewcooke
...over finite fields (<http://en.wikipedia.org/wiki/Finite_field_arithmetic>)

which is cool for many things, but it's probably not what you expected from
the title.

tl;dr: filter a collection of random guesses one equation at a time. new trick
is expanding the set of guesses as you work through, so you don't need to
start with 2^n (if modulo 2) for example.

~~~
jps359
As I understand it, the conjugate gradient method works over the reals.

~~~
stephencanon
Conjugate Gradient works over the reals _if the matrix is symmetric positive
definite_.

~~~
jmount
Which can be arranged. Instead of solving for x in A x = b solve for x in t(A)
A x = t(A) b (t() being transpose, and you may not need to explicitly form
t(A) A).

~~~
stephencanon
... which works, but the condition number of AtA is the square of the
condition number of A, so convergence may be very slow, or the results may be
numerically unusable.

You can often work around _that_ with a carefully chosen pre-conditioner, but
you're rapidly going down a rabbit hole that takes you away from the extreme
simplicity of conjugate-gradient.

~~~
jmount
True. But this is all a rabbit hole. I think the upper comment was trying to
point out that real linear algebra already has work from a trial vector
methods of solution. So there is some precedent for non-factorization based
solutions. But even that detracts that this really does seem to be a new
method for finite fields.

------
beagle3
I wonder if there's an underlying "random attempts" theory that can tie these
successes (that have intrigued me for a long time):

1\. Shannon's random code book construction (for achieving the noisy channel
rate limit)

2\. Johnson-Lindenstrauss lemma
[http://en.wikipedia.org/wiki/Johnson%E2%80%93Lindenstrauss_l...](http://en.wikipedia.org/wiki/Johnson%E2%80%93Lindenstrauss_lemma),
the result that underlies most if not all of the compressive sensing field
(and all of the recent superfast large scale SVD, PCA algorithms)

And of course this recent result might also be related somehow.

~~~
eru
There's also the (bigger?) question whether randomness helps at all, e.g.
whether P is weaker than P + a source of random bits. In practice,
randomization does help, but like with P vs NP as far as I know nobody has
proven anything definite.

~~~
pmb
The complexity classes that involve randomness have names like BPP, ZPP, and
RP. BPP is usually the one people mean when they talk about practical problems
that randomness might help solve. Although the second level of PH contains
both NP and BPP, which both contain P, it is still not known if any of those
containments are proper, and the relationship between BPP and NP is still
open. Even more interesting is the relationship between BPP and its quantum
version BQP. The Complexity Zoo has more for the interested reader...
<http://qwiki.stanford.edu/index.php/Complexity_Zoo>

~~~
eru
Thanks for giving a less hand-wavy account.

------
pmb
This stuff has just been in the air the past few years - Network coding (
<http://www.eecs.harvard.edu/~michaelm/CS222/contentdist.pdf> ) does the
inverse operation to increase efficiency in Bittorrent-like distribution
schemes. It is quite cool that randomness can help in both the creation and
solving of these problems.

------
rm999
I never took abstract algebra, can someone explain what is going on? I read
about half the article before realizing I have no idea what they mean.

~~~
acomar
A field is a structure that behaves very much like the reals (under the usual
operations). That is, it's a set of elements with an addition operation with
an identity called 0 (special elements are named after their numeric
equivalents, but that doesn't mean they're always the same thing), a
multiplication with an identity called 1, and the two interact "properly".
That is, they distribute and the additive identity is the multiplicative zero.
Further, inverses exist for all elements, with respect to both addition and
multiplication (except 0).

So very reals-like. Finite fields are then fields with a finite number of
elements. Can you think of any examples?

The easiest way to read the OP is to read the field F as if it were the reals,
except with a finite number of elements.

Let me know if that helps.

~~~
sadga
Nit: A closer analogy is rationals, more so than reals (as reals have other
unrelated properties like uncountability)

That is, given your simple rings (the integers) and finite rings (modulo n),
you get fields by introducing just division (integers -> rationals; mod-n ->
mod-p for prime p).

------
azakai
> He starts with a random set of vectors {V_{0}}. Then he checks which ones
> satisfy the first equation. If the vectors are random, then about half of
> them will satisfy the equation.

I don't follow this. Shouldn't it be 1/n, not 1/2?

~~~
ColinWright
They're using the specific example of GF(2) as the field. Any given assignment
of values to variables then gives a value of 0 or 1, so there's a 50:50 chance
of getting it right.

If you're working over F_p then about 1/p of the randomly chosen vectors will
satisfy the equation.

~~~
azakai
Thanks, I didn't realize it was a specific example there.

------
bascule
Does this have any applications to elliptic curve cryptography?

~~~
ColinWright
I suspect not. This technique works by assigning values to variables at
random, lots of times, and then keeping the ones that happen to make the first
equation true. You then somehow breed them into another large set, and winnow
with the second equation.

However, the number of elements in the fields involved there are so large that
you need a huge number of equations to have even the smallest chance of any of
them happening to be true "by chance", so I can't see how it could work as
stated, but it's certainly an interesting question.

------
propercoil
And no one mentioned a simple matrix?

