On: The first chosen-prefix collision for SHA-1

[joeyh]

Hardened sha1 does detect this new attack. Easy to test: Check their pair of files into a git repo and see that they have different checksums, while sha1sum(1) generates the same for both.

[mzs → joeyh]

checks-out, thanks

    $ mkdir sha1
    $ cd sha1
    $ curl -O https://sha-mbles.github.io/messageA
    ...
    $ curl -O https://sha-mbles.github.io/messageB
    ...
    $ echo foo > bar
    $ echo foo > baz
    $ openssl sha1 *
    SHA1(bar)= f1d2d2f924e986ac86fdf7b36c94bcdf32beec15
    SHA1(baz)= f1d2d2f924e986ac86fdf7b36c94bcdf32beec15
    SHA1(messageA)= 8ac60ba76f1999a1ab70223f225aefdc78d4ddc0
    SHA1(messageB)= 8ac60ba76f1999a1ab70223f225aefdc78d4ddc0
    $ git init
    Initialized empty Git repository in ...
    $ git add *
    $ git commit
    [master (root-commit) b274c88] sha1 collision test
    ...
     4 files changed, 2 insertions(+)
     create mode 100644 bar
     create mode 100644 baz
     create mode 100644 messageA
     create mode 100644 messageB
    $ git ls-files -s *
    100644 257cc5642cb1a054f08cc83f2d943e56fd3ebe99 0 bar
    100644 257cc5642cb1a054f08cc83f2d943e56fd3ebe99 0 baz
    100644 5a7c30e97646c66422abe0a9793a5fcb9f1cf8d6 0 messageA
    100644 fe39178400a7ebeedca8ccfd0f3a64ceecdb9cda 0 messageB
    $

[Thorrez → mzs]

No, you and joeyh are incorrect about the test (but correct about the result). As can be seen in the output, SHA1(bar)= f1d2d2f924e986ac86fdf7b36c94bcdf32beec15 but git_SHA1(bar) = 257cc5642cb1a054f08cc83f2d943e56fd3ebe99 . Why is there a difference? Not because of hardened SHA1. Hardened SHA1 essentially always produces identical outputs to SHA1

> git doesn't really use SHA-1 anymore, it uses Hardened-SHA-1 (they just so happen to produce the same outputs 99.99999999999...% of the time).[1]

https://stackoverflow.com/questions/10434326/hash-collision-...

There's essentially no chance that the string "foo\n" fell into that tiny probability of difference. The reason there's a difference is because before git hashes something, git will do various processing to it (maybe appending and prepending various things) and those things broke the carefully created collision. But a chosen-prefix attack might mean those various things can be accounted for, and a collision could still be found.

So we need to directly run hardened SHA1 on the data, which I believe is located at https://github.com/cr-marcstevens/sha1collisiondetection

As seen in https://github.com/git/git/blob/master/sha1dc_git.c

So I tested that one:

    $ sha1collisiondetection-master/bin/sha1dcsum bar baz messageA messageB shattered-1.pdf shattered-2.pdf
    f1d2d2f924e986ac86fdf7b36c94bcdf32beec15  bar
    f1d2d2f924e986ac86fdf7b36c94bcdf32beec15  baz
    4f3d9be4a472c4dae83c6314aa6c36a064c1fd14 *coll* messageA
    9ed5d77a4f48be1dbf3e9e15650733eb850897f2 *coll* messageB
    16e96b70000dd1e7c85b8368ee197754400e58ec *coll* shattered-1.pdf
    e1761773e6a35916d99f891b77663e6405313587 *coll* shattered-2.pdf

So it does protect against the new attack.

[mzs → Thorrez]

I really appreciate this, thanks!