
Crowdsourcing Bruteforcing: Help me win the EngineYard challenge - sil3ntmac
http://rustyengines.silentmac.com/jsengine.php
======
randomwalker
Let's keep some perspective here folks. The idea is cool and all, but as
discussed in the other thread, SHA-1 throughput on GPUs is between 200 million
and 600 million per second, depending on the GPU
([http://forums.nvidia.com/index.php?s=&showtopic=102349&#...</a>). On the
other hand, the maximum throughput that people are getting with JS is about
3000/second. Which is a hundred thousand times slower.<p>I think it is safe to
conclude that browser-based crowdsourcing is not a compelling approach to
numerical computation.

~~~
tlrobinson
Yeah, AFAICT the leader right now is <http://twitter.com/seibert> with a
hamming distance of 31
([http://forums.nvidia.com/index.php?s=87acc31e73d939b72dffbf7...](http://forums.nvidia.com/index.php?s=87acc31e73d939b72dffbf79f60584d1&showtopic=102349&st=120)),
and he says he's pushing 800M/sec across 4 GPUs
(<http://twitter.com/seibert/status/2748691432>).

For comparison, this guy's distributed JS implementation just reached 1B a few
hours ago: <http://twitter.com/wondersquirrel/status/2750527789>

If Engine Yard was trying to promote their cloud services with this contest
it's rather ironic it appears it will probably be won be a few desktop
computers with high end GPUs, which cloud hosts usually don't even have.

~~~
Torn
Even on my lowly 8800GT with quad-core I'm hitting 83+ megahashes per second
-- my lowest distance is currently 37.

CUDA is pretty neat :)

------
fsniper
This idea is a really well thought and implemented. This may be a new way of
cloud sourcing. Have you considered making a facebook app? It would definitely
attract many more people than this. Of course you shou ld offer some carrots
for users that do not have any idea what's going on.

~~~
PStamatiou
Agreed. running it on my computer with the slider set to max. Hope you win!

~~~
mikeyur
Ditto. Got 10 tabs in Chrome and 10 tabs in Safari all running at 1ms. Using
~60% CPU. I'll let it run for the next 4-5hrs until I go to work.

Good luck.

------
philfreo
I hope you aren't disappointed when you a see a new update on your AJAX
postback page that looks like a new winning string, when you see it is simply
a note from me. :)

(I also generated a few hundred thousand hashes for you)

~~~
philfreo
uhh.. sorry if I broke it. It now says the smallest hamming distance found is
zero, so either you had a _major_ breakthrough or something went wrong.

Edit: it's back up to 43, a nice improvement from 48 from a few minutes ago.

------
andreyf
To get more people excited, might be a good idea to say how many cycles each
browser has contributed - make it a fight of Chrome vs Firefox vs Safari ;)

~~~
ars
He can't.

As far as I can tell, the browsers only contact him if they find a string with
a shorter hamming distance, otherwise it's entirely offline, including picking
random words.

------
lukas
Haha, nice work guys -- great minds. We made:
<http://shacontest.doloreslabs.com> \- same approach but server side.

I upvoted you :).

------
judofyr
Did you just get a hamming distance of 0, or is something broken?

~~~
sil3ntmac
Nah, some script kiddie messin with me. Fixed and patching the hole now...

~~~
barredo
Well, 43 isn't that bad :-)

~~~
sil3ntmac
Haha, he left a message in my database.

"Oh but for your trouble I generated a few hundred thousand hashes for you.
--Phil Freo"

Thanks Phil :-) Your message's hash had a hamming distance of 73

------
rarestblog
Please increase delta range down to 1ms, Chrome barely uses CPU power at 10ms.
6 tabs @10ms uses 25% of Core2Duo.

~~~
rarestblog
Also decrease rate at which HTML stats are updated - DOM update slows stuff
down, it should update stats only once a second or so.

~~~
sil3ntmac
Updated to 1ms! I'll work on the HTML stat updates

Edit: fixed and fixed

~~~
rarestblog
You might also consider removing UTF8Encoding from SHA1 implementation, I
might be wrong, but I think contest states no encoding should be used - i.e.
ASCII only. If that's a care, that's a useless O(N) operation.

~~~
sp332
UTF8(ASCII) == ASCII

Unless you're using characters > 255, UTF-8 is identical to ASCII.

~~~
m_eiman
Actually it's UTF8(ASCII) == ASCII iff ASCII < 128, since the MSB signifies a
multi-byte character.

<http://en.wikipedia.org/wiki/UTF-8>

Hm, on second thought UTF8(ASCII) is -always- == ASCII, since ASCII is only
defined for 0-127, unless you count the extended variants :P

<http://en.wikipedia.org/wiki/ASCII>

------
jacquesm
Have you compared running your code in JS on an ad-hoc cluster like this vs
running it in optimized C on a local machine ?

~~~
Periodic
It's going to be a lot slower, particularly with the goal of not slowing down
a person's browser. By default, at least for me, their page is doing 1 hash
per 200ms, or 5/second.

On the other hand, a quick Haskell implementation using Data.Digest.Pure.SHA
is doing about 6,000 hashes per second (might have been 8000, I forget now)
per core on an Intel(R) Core(TM)2 Duo CPU E6750 @ 2.66GHz

So you're looking at 1000 people to equal one core of a decent desktop. It's
certainly an interesting approach, and has a lot of potential. I'm just not
sure the marketing is there with the "help us win" motivation.

~~~
mbrubeck
This JavaScript version: <http://news.ycombinator.com/item?id=715014> (which
doesn't do any throttling) is doing ~1500 hashes per second in one Chrome tab
on my fairly old (2.8 GHz Pentium D) desktop. I'm curious how fast it would be
on your Core 2 Duo in two Chrome tabs.

~~~
ErrantX
Mine peaked at 3,450 or so on a Core2 Quad @ 3.2Ghz

~~~
Retric
I think GPU's are going to win this "Yeah, my 5 GPUs are currently cranking at
a total rate of 800 M/sec" they are are already looking at the low 30's.

------
sil3ntmac
Just want to add that there is trick to make this incredibly easy to run in
parallel - just open the page in a new tab!

~~~
fsniper
Well on my dual core, opening a new tab could not fire up the other core.
Opening opera side by the firefox did the trick.

~~~
mbrubeck
I've got two tabs in Chrome using two CPU cores.

~~~
fsniper
Chrome uses different processes per tab. It's their sand boxing tech for
security.

------
Davertron
I would be interested to see if you could get something like this running
using the Google NACL project (<http://code.google.com/p/nativeclient/>).
Wouldn't that give you then benefit of having native code running on any
machine that went to the page and had the plugin installed? You could have the
user install a plugin when they went to the page and then reload the page.
Just popped into my head, probably too late to implement anything now, but
might be fun to give it a go.

------
silvio
Would you be willing to write up the stats of your experiment once the
challenge is over? I'd be really interested to know how much compute power you
were able to harvest and what was the rate of compute acquisition.

Terrific idea and implementation. I have 10 tabs on Chrome. Good luck.

~~~
ars
He can't.

As far as I can tell, the browsers only contact him if they find a string with
a shorter hamming distance, otherwise it's entirely offline, including picking
random words.

------
Klonoar
Eh, I'll leave it on my PowerMac overnight. Just fyi, I've donated the
following browsers...

OS X: Safari, Firefox 3.5, Webkit Nightly, Chromium Nightly, Opera 10 beta 2,
Shiira, Camino 2, Seamonkey, Sunrise, Flock

Then I have 3 virtual machines open...

Windows XP #1: Firefox 3.5, Chrome, Internet Explorer 6 (Dev testing bed, not
upgrading, but take it anyway), Opera 9, Maxthon 2 (Surprised I even had it)

Windows XP #2: Firefox 3.5, Chrome, Internet Explorer 7, Opera 9, Maxthon 2

Windows XP #3: Firefox 3.5, Chrome, Internet Explorer 8, Opera 9, Maxthon

I'd donate an Ubuntu VM or two, but... they don't wanna start up at this
point. _shrug_

This may not help, I dunno, but I'm certainly not using the computer. Have a
blast, and good luck!

------
alexgartrell
As a CPU contributor, I demand that you give us stats as to which browsers
were the fastest for your computation! This could be an interesting comparison
point for the bundled javascript engines. (Maybe :))

~~~
andreyf
Heh, I'm not sure if it would be representative of a good benchmark, but it
might be exciting to see how many hashes each browser has contributed ;)

------
mrtron
I wouldn't install an app to help you out.

I wouldn't leave it running if you killed my processor.

But you made it painless, so you get at least a few hours of runtime on me @
1ms.

------
imok20
Fantastic! 10 tabs open in Safari 4 @1ms using < 1core of my 2.66GHz
Core2Dueo. Cheers and good luck!

(I second, or third, the browser-war idea.)

------
Davertron
Also, are you logging IP's for your best results? Would be cool to know if my
machine was the one that came up with the shortest hamming distance (obviously
IP's aren't a guaranteed way to figure this out, but still...)

------
petercooper
Someone on Twitter claimed this entry has already got the 0:

<http://twitter.com/rikfaith/statuses/2743945177>

I have no way to check. True or not?

~~~
rikfaith
Just to be clear, _I_ never claimed it got a 0 :)

~~~
petercooper
Oh right, just someone I was following pointed to it and said it was. My bad
:) (Well theirs, I guess.. :P)

~~~
rikfaith
(The tweets I'm sending to contesttracker have the hamming distance at the
front.)

------
barredo
When I started it said: The smallest Hamming distance we've achieved so far
is: 48

Now, I opened it also in Safari and says: The smallest Hamming distance we've
achieved so far is: 0

0? Does that mean you got one?

Edit: Ok. Solved.

------
sil3ntmac
I'm going to set the script up to tweet whenever I find a closer hash, and
include the IP... you can follow me @silentmac

------
simonista
I'd be curious to hear how many people/cycles you get to help with this.

------
infinity
Very cool idea, I wish you good luck!

~~~
infinity
I have just opened another 6 tabs on a different Duo Core machine, all sliders
set to max. I will tell some more people ...

------
3pt14159
17M hashes overnight. :) Good luck.

------
Oompa
Fantastic concept. Good luck!

