

CMU Researchers Break Speed Barrier In Solving Important Class of Linear Systems - yarapavan
http://www.cmu.edu/news/archive/2010/October/oct21_speedyalgorithm.shtml

======
drats
As much as the cult of genius is often overwhelming and contributing people
have been written out of histories of discovery I still feel uneasy when I see
a Prof and a PhD student listed as discovering something as you never know
what the Prof contributed with the current state of the academy. It can always
be a collaboration yes, so I am not passing judgment on this case, but there
seem to be many cases where it is not a collaboration but a payment of dues to
seniority. Then again, at my faculty in another discipline we have so little
support or contact with the Profs it's embarrassing: so I suppose having any
kind of working relationship as peers would be better than what goes on in
some places.

~~~
Qz
As a recent CMU Alum, I can say that in this case you needn't be suspect.

~~~
gojomo
Do you mean, "you need not just suspect, I can confirm that in my experience
at CMU the professor _is_ over-credited", or "you need not be suspect, in my
experience at CMU the credit is properly allocated"?

~~~
jgershen
I interpreted Qz's comment as "this professor is deserving of the allocated
credit." As a recent CMU CS grad (and student of Professor Miller) I would
agree and say that this is also generally the case.

~~~
gojomo
I was leaning towards that positive interpretation, given CMU's positive
reputation.

But given the thread root, and not knowing any more about Qz -- indeed being
unable to trivially associate that handle with a real name/project via the
profile page -- a cynical comment from a disgruntled grad couldn't be ruled
out.

~~~
Qz
Apologies for the lack of clarity -- my original wording was possibly more
ambiguous than the one I posted and I thought I had fixed that, but I guess I
failed.

------
d4ft
The article mentions a few fairly vague examples of what this type of
algorithm is used for. To all of you non-neophytes, what are some specific
implementations of this algorithm and where are they used?

~~~
dmlorenzetti
If you write the equations describing the flows in a mass-conserving system,
you can end up with a symmetric diagonally-dominant system.

As you might expect, there's a physical interpretation.

Diagonal dominance basically means that the diagonal entry of each row of the
matrix is at least as big (in magnitude) as the sum of the off-diagonal
entries.

Suppose the diagonal entry gives the rate of change of flow (say of a fluid,
or of electric charge) into some physical location as you change some other
property of that location (like its pressure, or voltage). Then the off-
diagonals in the same column reflect the rates of change of flows to other
locations as you change its pressure. And the off-diagonals on the same row
reflect the rates of change of flows out of that location as you change the
pressures in other locations.

If the flows are driven by pressure or voltage differences-- which is often
the case, especially when you've linearized the system mathematically-- you
get symmetry (because adding 1 volt to node A has the same effect on the
A-to-B flow as subtracting 1 volt from node B).

If the system conserves mass (or electric charge or whatever), then the
diagonals must at least add up to off-diagonals (in magnitude).

So then all you need is a connection to an outside node of known pressure or
voltage (like ground), that doesn't change and hence doesn't contribute to the
off-diagonals. That kicks one node over into having a diagonal entry greater
than its off-diagonals, and then you have a nonsingular matrix and you can
apply the algorithm.

Disclaimer-- it's been a while since I worked on this, so I probably messed up
at least one of the directions or row-wise vs. column-wise relationships.

~~~
mturmon
And it's also worth saying that diagonal dominance is a stronger property than
positive definiteness. That is, something can easily be PD but not DD.

------
jmckib
Link to the actual paper, for the mathematically inclined:
[http://www.cs.cmu.edu/~glmiller/Publications/Papers/KoutisAp...](http://www.cs.cmu.edu/~glmiller/Publications/Papers/KoutisApproaching-2010.pdf)

------
jedbrown
Funny that they don't mention multigrid which is O(n) and very reliable for
M-matrices (more general than SDD, which is a rarely used term in the
multigrid literature). It's hard to take this seriously if they don't
demonstrate that the constants are good enough to beat the various multigrids
for this problem. Also note that most physical problems which lack optimal
sparsity are also not M-matrices, and indeed lack H-ellipticity (necessary and
sufficient condition for the existence of a pointwise smoother), even though
they are often still SPD.

~~~
wtallis
Wouldn't this method at least have the advantage of being a drop-in
replacement, where multigrid methods require a bit more work to integrate in
to your solver?

~~~
jedbrown
Algebraic multigrid is usually called in the same way as a direct solver. For
example, there are three distributed-memory parallel algebraic multigrid
packages (and a bunch of other methods, direct and iterative) that may be used
as runtime options with PETSc's same Solve call.

------
jasondavies
> The current theoretically best max flow algorithm uses, at its core, an SDD
> solver.

Does anyone know how this improvement in solving SDD affects the complexity of
the current best max-flow algorithm [1] i.e. (N+L)^(4/3) ?

[1]: [http://web.mit.edu/newsoffice/2010/max-flow-
speedup-0927.htm...](http://web.mit.edu/newsoffice/2010/max-flow-
speedup-0927.html)

------
fbcocq
I'm terribly sorry if this is an ignorant question but does this somehow apply
to manually solving a, say, 5x5?

~~~
dmlorenzetti
Probably not worth the effort on such a small system.

They specifically target large systems, because their approach (I think)
iteratively approximates the matrix, then uses the solution of each simpler
version as the initial guess to the solution of the next closest one to the
original.

Presumably all that overhead pays off in the long run, but not on small
systems.

------
joshuasmyth
In the article it mentions the runtime is s[log(s)]2 - Does this mean s
_log(s)_ log(s) or 2 _s_ log(s) ?

~~~
jessriedel
It's the former, and it looks fine to me: <http://i.imgur.com/g38Ix.jpg>

