

Statistical Learning as the Ultimate Agile Development Tool (Peter Norvig) - jbr
http://videolectures.net/cikm08_norvig_slatuad/

======
andrewpbrett
It's a very interesting talk - but the title is misleading (title is the same
in the submission as in the source). Norvig focuses on statistical learning
and how that can be used to infer the meaning of words and the appropriate
results in searches, particularly image searches. How these approaches tie
into agile development, or any software development really, is given at best a
cursory pass with a couple mentions that I, for one, wasn't able to follow. I
kept waiting for the other shoe to drop and it never did.

I'd love to be told that I'm wrong or that I'm missing something here, so if
anyone wants to enlighten me on the connection, please go for it.

~~~
jbr
My take: What I extracted from the talk regarding agile software development
is his point about keeping specifics out of the code. If the operational
specifics of your application are kept in "data form" instead of precise a
priori knowledge, it is easier to be "agile" (adaptive). Agile is an
incredibly loaded word, but I think he was referring to lightweight, working
software that is easily adaptable.

As far as how it's software development: It's an architectural choice whether
you're going to a) hardcode domain specifics into your code, b) keep some of
the domain specifics out of the code, but neatly organized and structured, or
c) leave the data messy but extract meaning from a large corpus statistically.

~~~
andrewpbrett
That makes a bit more sense - I'll buy it. Thanks.

------
jbr
I'm particularly interested in the idea of distribution/statistical-test
driven software testing. Instead of concrete assertions (truth assertions,
equality assertions), we would make assertions about descriptive statistics or
other tests. Or just assert that if we do something stochastic N times, we
should expect some truth/equality test to pass at least M times.

I'm not sure what to do with that idea, but that's actually the core of why I
posted this. Any other hackers out there excited by this idea?

~~~
tristan_juricek
Yes; I'm a lot weaker in my statistical understanding, but it seems like it
would be a fantastic way of slicing up the testing pie.

Unit tests are kind of handy if you've got some sort of "fencepost" style
error where you need to figure out a piece of logic that was overlooked. But
that's about it.

This sort of testing would way more relevant, because it's a way of focusing
on "what (data) do we 'well' right now"?

I'm thinking that such sort of systems might be useful as a basis for
performance or concurrency testing for example.

------
jwr
The videolectures.net site is frustrating. On the one hand, it is a great
collection of interesting talks. On the other hand, its annoying use of
Windows Media means that I have to watch the talks in the browser, while I
would much rather download them for watching on my iPhone.

Does anybody have a solution that would allow me to download these, convert
(possibly with VisualHub) and watch on an iPhone/iPod?

~~~
macmac
VLC will capture the stream and convert it to h.264 at the same time.

