
8x Nvidia GTX 1080 Hashcat Benchmarks - biggerfisch
https://gist.github.com/epixoip/a83d38f412b4737e99bbef804a270c40
======
nikcub
I'm going to wait for the 1070 benchmarks as in other series the x70 models
had a better price/performance ratio:

[http://www.videocardbenchmark.net/high_end_gpus.html#value](http://www.videocardbenchmark.net/high_end_gpus.html#value)

The 1080 is $599 while the 1070 will be $379 while the Titan X is ~$1000.

$1,500 for a 4x 1070 setup could be _very_ attractive considering the
performance should be on par with 4x Titan X (at $4k)

edit: forgot to mention power consumption as well, which should be a component
of price/performance

~~~
dogma1138
PCIE slots are limited so when you are talking about actual compute scale the
1070 can be more expensive considering you'll have to get another case, cpu,
ram, and an PSU that can handle 4 cards at max load. And of course the more
components you have in a given system the more likely is that one of the will
fail, granted if you are designing a system to be resilient against attrition
a system with more lower powered components can be more resilient than a
system with fewer more powerful ones based on the performance difference
between them and the MTBF for each component.

But bang for buck charts don't mean much even for Gaming I never understood
why people even use them, spend what you can and want to get the performance
you want. Spending 50% more to get only 20% more performance is a valid option
because those 20% can mean the difference between a playable game at high
resolution and a laggy mess. Traditionally SLI setups for mid range cards
could out-perform and be cheaper than high end cards but with a huge caveat -
being dependant on the SLI profile for the game, if it come out to be shitty
or non-existent you are just stuck with 2 cards that can't really run the game
at the settings/resolution you want.

Also unless you are building a small datacenter power consumption can be
ignored, the 50-75W difference in total system draw between a mid range and a
high end card when the rest of the hardware is the same isn't going to have
any financial impact on anyone even if we take the most expensive power in
Europe that's going to come out to about 30 cents per KWh.

~~~
nikcub
I agree that for gaming it makes sense getting the higher end cards rather
than dealing with SLI, but my use case is password cracking that doesn't
require five 9's uptime.

The GTX 1070 is 150W power consumption while the 1080 is 180W. Rather than get
another case i'd just scale up the case size based on the number of cards - 2
cards ($100[0]) 4 cards ($200[1]), 6 cards ($400[2])

1200W PSU $300[3], riser cards $10[4] and then motherboard + cpu + SSD about
$200-250 so it works out $700 base for 2 GPU system, $900 base for 4 GPU
system and $1400 for 6 GPU (2 PSU's)

If you work that out with the card options the sweet spot is 4x 1070 GPU's for
$1500. It really isn't worth spending $4k on cards to get equiv performance
from Titan X's or 15-30% more performance with the 1080 (although
power/performance is better).

[0] [http://www.coolermaster.com/case/mid-tower-
cm690-series/cm69...](http://www.coolermaster.com/case/mid-tower-
cm690-series/cm693/) [1] [http://www.corsair.com/en/obsidian-
series-900d-super-tower-c...](http://www.corsair.com/en/obsidian-
series-900d-super-tower-case) [2]
[http://www.miningrigs.net/?product=graymatter-6gpu-server-
ca...](http://www.miningrigs.net/?product=graymatter-6gpu-server-case) [3]
[http://www.newegg.com/Product/Product.aspx?Item=N82E16817139...](http://www.newegg.com/Product/Product.aspx?Item=N82E16817139039)
[4] [http://www.amazon.com/Fixable-Adapter-Flexible-Extension-
Con...](http://www.amazon.com/Fixable-Adapter-Flexible-Extension-
Connector/dp/B00IMYODGS)

~~~
dogma1138
What motherboard can you buy for under 50 with 6 PCIeX16 (power) slots? The
cheap "bitcoin" motherboard that go for about 50-75$(old intel express
chipset) won't work in this case, not sure about the 1070 but the 980 (non-ti)
and the Titan X do not want to work with any PCIe slot that isn't a true
PCIeX16 power slot. The PCIe x1-4 to x6 riser converter boards do not work,
even the "good" ones with a SATA power connector and an actual power PCB for
some reason, my guess is that there is some actual sensing going on in the
card itself.

Unless you buy something like this [http://www.amazon.com/ASRock-
Motherboard-H81-PRO-BTC/dp/B00H...](http://www.amazon.com/ASRock-
Motherboard-H81-PRO-
BTC/dp/B00HEUIS2G?ie=UTF8&keywords=asrock%20bitcoin%20motherboard&qid=1465300259&ref_=sr_1_1&sr=8-1)
which which is i think the only non-server motherboard which is commonly
available you aren't getting more than 4-4 PCIe slots, and even to get 4
(PCIe16) you need to spend about 200$ on a motherboard these days and the H81
chipset mobo's don't want to work with modern nVIDIA cards (or the other way
around) with all but the main PCIe16 slot.

Pretty much to get more than 2 PCIx16 slots these days you have to buy a good
gaming Z170/Z97/X99 motherboard, that ain't cheap, nor are the CPU's that go
into them. While the AMD CPU's might be a bit cheaper again the motherboards
won't be.

P.S. my personal password cracking setup at home is 4 5970's you could get
them for about 100$ even a couple of years ago (today they are much cheaper 2
for 130 or b/o: [http://goo.gl/yplISa](http://goo.gl/yplISa)) and for most
hashes they go 40-50% of the perofmrnace of a Titan X (I do have 2 Titan X's
in my personal gaming rig which i can use in a pinch). I got them setup on an
ASROCK motherboard (h61 iirc) that was designed for bitcoin mining so it comes
with 8 PCIE slots and powered converter ribbons connect the cards do it all of
this sits in a al cheapo rackmount case of a 20 year old compaq server.

But if you are looking for more scalable compute instance AWS CUDA instances
are probably a better solution and I've been using them more and more lately.

~~~
Bedon292
I have been wondering about the AWS GPU instances. They are something like
$2.60 / hour. At what point do they make sense monetarily? You can build a
machine like described here for ~$5000 total. Thats not that many hours of AWS
time. So if anyone is doing any long running cracking, or machine learning,
does it ever make sense? I am truly curious about what people's experience has
been.

~~~
ryanlol
>At what point do they make sense monetarily?

For password cracking? Never, especially considering you can just go on
insidepro or something and rent access to someones cluster.

~~~
dogma1138
Insidepro is a great resource, but you usually aren't going to get a VAT
invoice to either bill your client for or deduct in your taxes ;)

AWS is simple, it's not overly expensive for most password cracking ventures
(e.g. password quality audit that certain organization like to do on their
AD/LDAP) and more importantly you aren't giving away potentially very
sensitive information to people that by default cannot be trusted.

Inside pro is ok if you are going mostly non-legit / off the books to begin
with, it's also ok for basic research on non-real world data, but never use it
for actual real world data not to mention B2B services.

------
Freaky
The scrypt and bcrypt rates are a bit misleading because they're with
unrealistically low work factors:

* bcrypt: 2^5, when 2^10 or 2^11 is a much more typical figure

* scrypt: N=1024 r=1 -> 128KiB. For interactive logins the recommendation is N=16384 r=8 -> 16MiB.

LastPass is also off by a factor of 10 (500 rounds, when they default to
5000).

~~~
Justsignedup
It would help if they indicated number of iterations for all algorithms.

~~~
Freaky
Most of them are defined here:

[https://github.com/hashcat/oclHashcat/blob/master/include/sh...](https://github.com/hashcat/oclHashcat/blob/master/include/shared.h#L1071-L1123)

------
nailer
(entrepreneur hat on)

The hashcat project could make a bunch of money with cracking as-a-service:
you supply hashes, hashcat runs on a dedicated multi-GPU instance (SoftLayer
etc provides these), and get outputs.

For users: no setting up hardware or software and much better price /
performance than DIY

For hashcat developers: money to pay their rent and work on hashcat

Edit: looks like the company sells SW/HW but not as-a-service. Massive
opportunity for them there:
[https://sagitta.pw/software/](https://sagitta.pw/software/)

Edit 2: they're doing it: Q1 2017
[https://twitter.com/jmgosney/status/740146970254147584](https://twitter.com/jmgosney/status/740146970254147584)

~~~
robk
I think you're talking about what gpuhash.me does now

~~~
JorgeGT
Also hashcrack.org where you can use paypal, visa, bitcoin or even fancy
ethereum for your shady cracking needs.

~~~
ryanlol
And the good old cmd5.org, which at least in the past has been the best such
service available.

------
Zenst
Interesting that the GOST-R 512bit seems no slower than the 256bit hashing.

Does this mean the 256bit is underperforming or is the 512bit flavour just as
compute intensive as its lesser counterpart with half the number of bits.
Least for me seems to stick out a bit(sic).

"Hashtype: GOST R 34.11-2012 (Streebog) 256-bit

Speed.Dev.#1.: 50018.8 kH/s (334.18ms) Speed.Dev.#2.: 49784.4 kH/s (332.43ms)
Speed.Dev.#3.: 51323.1 kH/s (325.52ms) Speed.Dev.#4.: 50947.0 kH/s (327.87ms)
Speed.Dev.#5.: 51510.6 kH/s (329.13ms) Speed.Dev.#6.: 50417.3 kH/s (331.39ms)
Speed.Dev.#7.: 50825.4 kH/s (333.57ms) Speed.Dev.#8.: 50853.3 kH/s (333.38ms)
Speed.Dev.# _.: 405.7 MH /s

Hashtype: GOST R 34.11-2012 (Streebog) 512-bit

Speed.Dev.#1.: 49979.4 kH/s (329.40ms) Speed.Dev.#2.: 49849.7 kH/s (330.26ms)
Speed.Dev.#3.: 50315.4 kH/s (336.13ms) Speed.Dev.#4.: 50305.3 kH/s (328.99ms)
Speed.Dev.#5.: 51486.7 kH/s (326.69ms) Speed.Dev.#6.: 49709.2 kH/s (328.72ms)
Speed.Dev.#7.: 51328.8 kH/s (330.30ms) Speed.Dev.#8.: 51530.7 kH/s (326.41ms)
Speed.Dev.#_.: 404.5 MH/s"

------
fivesigma
>WPA/WPA2: 3177.6 kH/s

22.5 minutes (11.25 on average) to crack a randomized 8 hex digit password
that a good number of wifi modem/routers come with (mine certainly did)

~~~
exhilaration
Can you point me to some resources for learning how to do this? Starting with
capturing my wi-fi traffic to running the right tools to crack it? I'd like to
see how long it takes me to crack my own password.

~~~
efraim
This[0] guide goes over how to use WiFite in Kali Linux[1] to capture a wpa
handshake and how to use hashcat to crack it. You have to have a wifi-card
that is compatible with aircrack-ng[2].

[0][https://www.blackmoreops.com/2014/03/27/cracking-wpa-
wpa2-wi...](https://www.blackmoreops.com/2014/03/27/cracking-wpa-wpa2-with-
hashcat-kali-linux/)
[1][https://www.kali.org/downloads/](https://www.kali.org/downloads/)
[2][http://www.aircrack-
ng.org/doku.php?id=compatible_cards](http://www.aircrack-
ng.org/doku.php?id=compatible_cards)

------
gh2k
I'm surprised that the primary use case for a machine with 8 of these things
in would be password cracking. ([https://sagitta.pw/hardware/gpu-compute-
nodes/brutalis/](https://sagitta.pw/hardware/gpu-compute-nodes/brutalis/))

There's not even any mention of training neuralnets, rendering things or doing
science. Are there many non-shady reasons for purchasing the above device
based on the purpose described?

~~~
agency
FWIW password cracking is not necessarily a shady use-case. At my old job the
infosec team had a GPU cluster continuously trying to crack employees'
passwords and would force users to change passwords if they succeeded.

------
binarray2000
Fascinating hardware with great processing speed.

One question for the knowledgeable: Why are speeds for PostgreSQL hashes so
high (~ 25k MH/s) compared with, say, MSSQL(2012) (~ 1k MH/s)?

~~~
tacos
Postgres is MD5; MSSQL(2012) is SHA-512.

~~~
stouset
And they're both terrible.

------
znebby
It just goes to show that in 2016, you really can't be using SHA1/256/512 for
password hashing anymore, even salted. Bcrypt/Scrypt or similar is a must.

~~~
inglor
From what I can tell with reasonable password length SHA512 still seems
relatively secure. I'm not saying there is any reason you _should_ use it -
but if it does 8624.7Mh/s and your password is JUST numbers, lowercase and
uppercase English letters and it's only 10 characters it would still take it
(26+26+10)^10 / 8624.7Mh/s ~= 83929936 seconds which is still about 3 years
for a single hash.

If it's 12 characters - then you're up to 10,000 years.

~~~
znebby
True - reasonable password length being the important factor. I was basing my
calculations on 8 character passwords.

At my university we were forced to use passwords that are exactly 8 characters
long, for some ridiculous reasons. And this was meant to be one of the top
universities in Australia...

~~~
inglor
It's because the internet is full of monkeys:
[http://security.stackexchange.com/questions/33470/what-
techn...](http://security.stackexchange.com/questions/33470/what-technical-
reasons-are-there-to-have-low-maximum-password-lengths)

> Originally, some developer, somewhere, was working on an old Unix system
> from the previous century, which used the old DES-based "crypt", actually a
> password hashing function derived from the DES block cipher. In that hashing
> function, only the first eight characters of the password are used (and only
> the low 7 bits of each character, as well). Subsequent characters are
> ignored. That's the banana.

------
chrisra
Time to get a new password.

~~~
riskable
No. Time to start using proper _password_ hashing algorithms (e.g. scrypt)
with added protections such as requiring _n_ rounds of hashing.

Tip: To require 65535 rounds of hashing on a Debian or Ubuntu system just run
this:

    
    
        sed -i -e 's/sha512$/sha512 rounds=65535\n/g' /etc/pam.d/common-password
    

(Note: Assumes you're still using the default hashing SHA-512 hashing
algorithm)

~~~
distances
There was an earlier discussion about KeePassX, where I mentioned I used 10
million rounds of the AES encryption. Someone commented that it doesn't really
add much additional strength.

I'm not familiar enough with the topic to understand why it would be so. I
wonder if there is some basis for that claim?

~~~
dogma1138
[Disclaimer: This is assuming you aren't talking about key schedule, since you
use odd terminology, and even then how do you get to 10M rounds is beyond me]
Because you use the same key, not to mention the same IV most likely, unless
you are storing 10M IV's some where... Doing 10M rounds of AES is just
stupid(ly) expensive.

Also verifying the decryption with 10M rounds, means, 10M CRC checks, and if
you are using it on a non-ECC system there is a good chance that many attempt
to decrypt anything stored like that will fail due to single bit errors.

~~~
Freaky
He's talking about KeePass's key derivation algorithm - SHA256 the password,
make a random 256 bit key, AES-encrypt the hash N times using the key, and
SHA256 the result.

10 million iterations is about a second of work on a typical desktop.

~~~
mioelnir
Typical being AESNI accelerated?

~~~
distances
I don't have expertise on the subject as was probably already evident. For me
it's a setting in KeePassX, and that's it.

With the 10 million config, opening the password database takes just a bit
over a second on i7-4800MQ which should come with AES-NI. I can't say if the
software is using that or not.

On my Android it takes 4 seconds, so very acceptable still.

------
danielsamuels
I'd be interested in seeing that the numbers are like for Argon2.

------
izietto
I wonder how many fps runs Minesweeper...

~~~
stephengillie
I wonder how many shaders can be run in Minecraft and still get 60fps with
this setup.

~~~
cridenour
Probably still none.

------
boromi
Is double precision performance crippled?

~~~
mrb
Yes: DP runs at 1/32 the rate of SP on GP104:
[https://en.wikipedia.org/wiki/Pascal_(microarchitecture)#Per...](https://en.wikipedia.org/wiki/Pascal_\(microarchitecture\)#Performance)

However it is uncrippled on GP100 (ie. Tesla P100).

------
hathym
Can you do some gaming performance in FPS?

~~~
nikcub
1080:
[http://www.hardocp.com/article/2016/05/17/nvidia_geforce_gtx...](http://www.hardocp.com/article/2016/05/17/nvidia_geforce_gtx_1080_founders_edition_review)

1070:
[http://www.hardocp.com/article/2016/06/06/nvidia_geforce_gtx...](http://www.hardocp.com/article/2016/06/06/nvidia_geforce_gtx_1070_founders_edition_review/4)

Other benchmarks: [http://www.trustedreviews.com/nvidia-geforce-
gtx-1080-review...](http://www.trustedreviews.com/nvidia-geforce-
gtx-1080-review-performance-benchmarks-and-conclusion-page-2)

~~~
rasz_pl
Im sure he meant 8-way sli gaming benchmarks

