Hacker News new | comments | ask | show | jobs | submit login
Crypto 101 (crypto101.io)
352 points by tptacek on Mar 19, 2014 | hide | past | web | favorite | 86 comments



This is a preview release of Crypto 101, an introductory course on cryptography. It's a follow-up to a talk I gave last year at PyCon.

To paraphrase David Reid, abstinence-only crypto education isn't working. We need easily accessible crypto education for developers. This book, and, once they're done, the included exercises, hopes to help.

I will happily answer all your questions here, by e-mail (see profile) or on twitter (@lvh).

In case the website breaks down, here's the direct download URL: https://9d0df72831e4b345bb93-4b37fd03e6af34f2323bb971f72f0c0...

here's a magnet link: magnet:?xt=urn:btih:e4af18f490672c6f7982a03f427e099014013774&dn=Crypto 101March2014.pdf&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A8 0%2Fannounce&tr=udp%3A%2F%2Ftracker.publicbt.com%3A80&tr=udp%3 A%2F%2Ftracker.ccc.de%3A80&tr=udp%3A%2F%2Ftracker.istole.it%3A 80


Do you want patches/prs for copy editing? I've noticed 2 or 3 of them.


Absolutely! All the stuff is on Github, website and book (and source code for the exercises which I'm still hacking on): https://www.github.com/crypto101

Thanks in advance for your contributions!


Had a look through and really enjoyed the picture examples, can even remember some of those examples from uni.

And I see you are using vectorisation of hand drawn diagrams (maybe you write on a whiteboard and then take a photo and vectorise it). If you have an ipad I would recommend GoodNotes and a good stylus (maybe the Adonit Jot Script).


Thank you for the kind words!

Due to a stupid oversight my iPad is stuck in storage somewhere a few countries over :( I really wish I could, but right now it's just me, white paper, a pen, a scanner and potrace. It's tedious, but workable :)


The book looks great. I don't have any experience in crypto really but will certainly be taking a look at the early chapters here for a foundation.

edit: ignore the below, just got to this explanation in the book!

> The entire Crypto 101 project is publicly developed on Github under the crypto101 organization, including this book. https://github.com/crypto101/book

----

Excuse me if this info's on the website (I can't get to it right now, HN "hug of death" it appears), will you be open sourcing contribution to the book? I'm not saying it's necessary or that I even recommend it, just curious about your authorship model (and noticed the CC license at the beginning).

Also, did you make the book in LaTeX (or some TeX variant)? If so, will you be sharing the source? I always love looking at other people's TeX documents.


> Latex source

Check the github link you posted, the book is right there. I also like looking at other people's tex files. The book was written with orgmode, so this could be a double treat, you get to see the tex as well as the orgmode used to generate the pdf.

UPDATE: The choice of non-free fonts is a peculiar choice. The math text looks a little strange in places. I am not sure if there is a package to fix up the math font for caslon, ie. one similar to the minion package. The euler fonts would work for a mathfont. Why not use something from texlive so that anyone can build a copy of the book that looks like the original and/or submit improvements? Everything looks nicer when you turn on microtype.


Thanks for the suggestions! I think that being able to build on standard free texlive installations would be a wonderful feature, but this the first time I've heard of the microtype package. Are you saying I can just turn it on and it'd be better? I've skimmed the CTAN entry and it suggests that it'd work with most fonts; does that include the ones I'm using now? (If so, does that mean that "use free fonts" and "use microtype" are two different tickets?)

Thank you very much in advance. I've created two tickets for this: https://github.com/crypto101/book/issues/60 and https://github.com/crypto101/book/issues/61.


Microtype is different from " use free fonts." Microtype allows for:

  "character protrusion and font expansion, furthermore the adjustment 
  of interword spacing and additional kerning, as well as hyphenatable 
  letterspacing (tracking) and the possibility  to disable all or 
  selected ligatures."
I will follow up on github. Are you planning to have the book printed? The recto/verso margins can be distracting.


It would be nice if you collect email addresses so that you could inform interested individuals(that's me!) when the final version is released.


I'm actually trying to, but at the last minute I changed the magnet link to a HTTPS PDF link, which appears to open in a new window for most people :-/ Any help from people with frontend chops who could tell me how to fix that (apparently Content-Disposition isn't the answer?) would be much appreciated!


As a quick fix you can add a target attribute to the big anchor tag:

<a href="..." target="_blank">

It'll still open the PDF in a new window but the underlying site with email prompt will still exist!


Excellent suggestion. Not the OP, but this suggestion allowed me to sign up.


I think this should be up now. Thanks, GP!


Cool stuff, man.

Where do you think your book stands in relation to other books like Cryptography Engineering? Should people new to crypto read your book before or after CE, or instead of?


Thanks!

This is definitely in the same space as CE. When I just started writing this, tptacek wrote: http://sockpuppet.org/blog/2013/07/22/applied-practical-cryp...

... and that was a huge motivator and inspiration to me. So:

1. (Unlike Applied Cryptography) focus on things that you are actually likely to hit in the wild

2. (Unlike Cryptography Engineering) Written today, and hopefully perpetually updated; including things like ECC, all sorts of CBC attacks, et cetera.

3. (Unlike either) Free of charge, and freely redistributable :)


About #3, have you considered hosting a git repo with your (presumably LaTeX) source so that others can contribute?


Yeah; there's a link somewhere in the "Development" section. It's org-mode, though: https://github.com/crypto101/book

Everything about this project (except the TLS keys) is on Github.


Looks great (haven't read all of it yet).

Have you considered using something like Ipe to make diagrams for you?


Nope! I just started drawing some stuff and it stuck. I had never even heard of ipe. How's it compare to TikZ? (That seems like the closest thing I have played with.)


Both coursera and udacity have amazing courses on crypto.

Udacity: https://www.udacity.com/course/cs387

Coursera crypto I: https://www.coursera.org/course/crypto

Coursera crypto II: https://www.coursera.org/course/crypto2

I took coursera crypto I myself. It was a lot of work, but I learned a ton.


Good luck trying to take Coursera's Crypto II: I've been signed up since August 2012, and every 3-6 months it has been delayed another 3-6 months. At this point I'm no longer expecting it to be offered.

Crypto I is not vaporware and is excellent.


I have hopes for Crypto II. Based on Crypto I, Boneh likes to do a good job with the course and being who he is, he's probably just incredibly busy (the original Crypto I itself had two minor delays in the middle of the class), so it keeps getting postponed. I wouldn't be surprised to see it materialize eventually.


Makes you wonder why aren't online classes kept, at least a year or something, in case the information is out of date (for technology)?


Udacity has a model in which every class is self paced and they have not deleted a single one since uploading them.


This is my favorite feature that Udacity has and the others don't.


That's because the staff needs a schedule similar to the academic schedule so they can answer questions, correct things, participate in the forum discussions...

Crypto I has been offered several times though (at least 4 or 5). If you ever signed up for one of the offerings, you can still access to the full course (videos, lectures, and I think even the automated grader) as well as the forums (but the forum activity usually fades down after the end of the course).


It's not because of some practical reasons, just university policies.


I'm not sure what you're saying. As far as I know, Crypto II has never been offered, so the problem is probably that they haven't developed any course material for it.


Here. I'll start you on better class on cryptography than any video, pdf, or slide deck you're going to find.

Go to this repository, grab the code, and figure out how you can attack it: https://github.com/jackjack-jj/jeeq (of course, along the way go learn whatever you need to). There are several interesting weaknesses in this code, and yet it's not just a toy: it was briefly deployed in a widely used application.

(If you google for it, you might find some of my analysis on it, which would potentially spoil the learning experience, so I suggest you don't. Though if you finish with this one I can dig up some other weak cryptosystems.)


Anyone else only get:

<html><head><title>Processing Failed</title></head><body><b>Processing Failed</b></body></html>


Yep. Too many open file descriptors. I'm trying to figure out how to fix that without taking the site down now.

EDIT: It should be hopefully slightly better right now...



The "processing failed" banner is quite ironic considering our recent collective experiences with cryptography. Why the Mt.Gox failure and so many other hackings and exposures of supposedly-private data? "Processing Failed"


I don't read "Processing Failed" as an indication of hacking or a data breach.

Sounds to me more like an overloaded website falling back to a small static error page.


Yep, that's exactly what happened. Combination of:

1) me not understanding docker and forgetting to increase the host nofiles;

2) me not understanding upstart (because I don't normally use ubuntu...) and not realizing that it conveniently ignores /etc/security/limits.conf :)

It's obviously not a data breach: I don't actually store anyone's data :)


Who is the author? I can't find anything about him except for his real name and a self-description of "hacker". I'm confused why he thinks it's reasonable to create something so authoritative sounding as "crypto 101" without bothering to explain his credentials to do so.


This looks helpful for people who are already very code-literate. But it seems like "101" should start with the absolute basics, stuff like classical cyphers, the history of cryptography, and definitions of basic terms. I realize that's not what the HN crowd really needs, but it would be a good place for hundreds of millions who are interested in learning what it really means for something to be secure, encrypted, hashed, etc. Anyone have a favorite resource to this end?


I'm interested in what you think the value of studying classic ciphers is.


Well I personally find clarity in the history of things — they provide context and a narrative for the topic in question. You don't start a class on warfare with Afghanistan, for instance, or even WWII. It's worth going back to the beginning, if only for a cursory look at how wars were fought, how innovations came about and when, and which of them is still influential.

Similarly, with crypto, I think it's about understanding how and why cryptography emerged, and learning about the arms race it developed into as people went from decoder-ring level encryption to the present state of the art. Knowing when and why key exchanges were first created, how they were broken back then, why one-time pads are still useful in some circumstances, and so on I think contributes to a larger understanding, something that exceeds a technical one - and I also believe that will lead to better implementation (though I don't have much to back that up).

Also, there are interesting stories in there. Learning why Enigma was unbreakable, and subsequently how it was broken and the attending circumstances, informed as they were by politics and strategic warfare, is a lesson not in cyphers, but in cryptography. I think that's important, though obviously that is not OP's goal! I just think it should the way one starts — at the beginning.


"The Code Book", by Simon Singh, would be a good companion read. It covers much of the above, and is both compelling and well-written.

Edit: Fixed from "Code" to "The Code Book". Although "Code" (Charles Petzold), on a different subject, is also excellent.


Thanks, I will.


Well said my friend, well said.

The comment you responded to reminds me of my favorite quote of Alan Kay where he compares software culture to pop culture, and expounds on the importance of history to rise to the higher levels.

I was lucky enough to learn cryptography from Len Adleman at USC, and guess what he started with? Classical ciphers.


History is great as a topic in its own right. But in my view, we already have lots of books that cover crypto through World War II.

Very little of it is relevant to modern ciphers. Some of it is even actively harmful, witness old advice about "RNG entropy depletion" and "stale keys" resulting in overly complex and failure-prone crypto systems.

What we need more of relevant, modern, best-practices-in-light-of-latest-attacks information suitable for careful, conscientious developers who have to build actual systems out of it.


FWIW, I got way more value out of the Matasano challenges than studying classic ciphers.

Do you think it'd be a not-terrible idea for newcomers to focus entirely on studying this? http://www.daemonology.net/blog/2009-06-11-cryptographic-rig...


That is a pretty good list of recommendations, but I have a couple criticisms.

The recommendations are mostly low-level. None of them are wrong, but they put undue burden on developers to get details right. For example, the AES-CTR recommendation doesn't talk about nonce management, but this is critical to the security of the construction. Application developers should always use the highest-level cryptographic constructions they can get away with. As such, many of these bullet points could be replaced with a recommendation to use PGP or NaCl.

Also, the list skimps on random number recommendations. It talks a bit about how big numbers should be, but it doesn't discuss sources. This is really important as RNG is a weak point in many systems. Short answer: use /dev/urandom.


I think it depends on what your end-goal is.

If you're interested in creating a production application that uses Crypto, you should use existing high-level systems, and learn how to minimize your own areas of vulnerability. (Similar to your link)

But some people enjoy algorithms, and enjoy playing with information. That's a different goal, and a different objective, and would understandably have a different reading list.

For example, I went through a period a while ago when I was really enjoying learning about the development of the Apple II. As part of that, I read through the reference manual, and enjoyed learning various memory locations and what they did (None of which I remember, fwiw..)

I had a fun and relaxing Sunday afternoon reading the manual.. But I wouldn't expect those "skills" in Apple II memory locations to be particularly applicable to modern application development.

If you want to write an app, use tools. If you want to learn for the sake of enjoyment, that's awesome! Just remember the historical context, rather than trying to re-implement older bad practices.


I've emailed for the Matasano challenges three times from two different email address and never received anything.

Is it okay if I ask somebody here to send me a copy?


> I come from a proud academic background and am sufficiently optimistic about humankind that I think it's a good idea to spread some knowledge around.

This is a funny way of pointing out the arrogance in the opposing position: "You aren't smart enough to do this right (and I am talking to you specifically), so don’t even waste your time."


For someone that is truly a beginner in cryptography it is helpful to understand at a very basic level how encryption/decryption works.

Using something simple as a Caesar cipher or ROT13 you can show how plaintext undergoes a transformation using an algorithm to create the cipher text and how that same algorithm is used to convert the cipher back to plaintext.

This simple idea provides the basis to discuss other cipher techniques (and their shortcomings); as well as provide an early example of symmetric-key algorithms.


There is zero cryptography value in spending more than 5 minutes on a Caesar cipher or ROT13. It's a shame that so many university courses on cryptography even waste such time on such topics. It would be like teaching times-tables in a course on calculus.


Vigenere is useful in practice for breaking real constructions, but that's all I can think of.


They are easy to understand and provide a venue to use the cryoptographic terms that the students should learn to use.

Having to understand a complicated bit of cryptography while at the same time coming to terms with a big batch of nomenclature you haven't committed to memory is hard. Harder than I can handle anyway.


Besides "cipher" and, perhaps, "key", is there a lot of nomenclature/terminology to be found in the study of pre-Shannon cryptography?


... yes.

don't ever become a teacher.


Your comment would be better without the sad little barb, and with an actual example of nomenclature learned more easily from classic ciphers than from modern ones.

I suspect you didn't provide one because you don't have one. Prove me wrong.

(I can think of two potential terms already.)


"Those who don't know history are doomed to repeat it."


One of the interesting things about classic ciphers is that (as far as I know) they're obviously insecure by modern standards.

Most of the time what people want out of modern crypto is something called "semantic security" -- in other words, an attacker should not be able to learn even a single bit of the plaintext for any message, even if they are allowed to see the encryption of polynomially many chosen plaintexts (ignoring length, of course.)

What this means is that deterministic encryption schemes are all broken. Consider a game:

The attacker asks to see the encryption of messages a and b, receiving a' and b' as ciphertexts of a and b.

Now, the attacker is "challenged" with a new ciphertext of a or b, c. If the attacker can (with reasonable probability) distinguish which plaintext c is the encryption of, then the encryption scheme is not "semantically secure."

Notably, since our encryption scheme is deterministic, c = a' or b'. Therefore, with probability = 1.0 the attacker can distinguish which plaintext encrypted to c.

So, by modern security standards all deterministic (and therefore all classic, I think) ciphers are broken.

This is the sort of thing which is covered in the coursera class. Modern crypto is (usually provably) substantially more secure than what used to be used, and the kinds of techniques used to do modern crypto don't have much overlap with things like Caesar ciphers.


Thanks! This is definitely targeted towards developers. There's certainly some sections that require you to be able to read some code, but I'd hope to eventually tone those down to a minimum. My personal popular science favorite for that is probably the Code Book by Simon Singh.


Just looking at the table of contents is a bit worrying. :-(

Why are block cipher modes like CBC and CTR, and issues like padding listed in the stream cipher section? Those aren't relevant to stream ciphers (though you can regard counter mode as turning a block cipher into a stream cipher).

Putting pbkdf2, scrypt bcrypt under key derivation functions but omitting them from the password storage section while technically accurate isn't really helping anyone.

Reading the text in enough detail to see if this is any good would take longer than I've spent, but the organisation of the material at least definitely needs some work.


Hi! Thanks for your comments. I feel that a more detailed reading would most likely address your concerns.

The book explicitly addresses why modes of operation (and their related bits, like padding) are in the stream cipher section. I've flip-flopped between putting them in one or the other a few times now, but I'm increasingly convinced that doing it this way (and having the book explicitly say that I'm doing it this way) makes the storyline, similar to the one I tried to keep in the talk, work better.

The password storage section talks about a lot of broken password stores, as a subsection of the chapter on hash functions. It explicitly refers to the key derivation function section at the end. This pattern comes back through the entire book: "we want to do X, and it may look like we can do X already with the tools P and Q we have already, but you actually still need R and S; here's why".


Fair enough. I'll read it properly and then comment. :-)


It's really refreshing to have a site just give you the material without making you make an account, or sign up via email first.


One can also learn crypto by doing with Matasano Crypto Challenges:

http://www.matasano.com/articles/crypto-challenges/


Twice sent an email and did not receive any response


I'm very sorry for the availability issues. All of my usual tricks for increasing the fd limit didn't work; then I realized this is because of two things:

1. I'm running inside a docker container 2. I'm using ubuntu instead of debian, and upstart conveniently ignores all the usual fd limits places like /etc/security/limits.conf.

TL;DR, be careful when experimenting with new fancy technology you don't understand.

I hope it's resolved now ;-)


Host it on something like GitHub?


This seems like a great base. I'd love if there was a step-by-step guide to creating a cryptocurrency though. I'm trying to learn about it (without intending to release another onto the crap filled market) and there's not many guides. It seems like a well kept secret for now. Simply cloning one is giving me enough trouble, I can't generate the merkel root and move on from there. No documentation.


In Chrome pressing the "Get the pre-release now" button immediately opens the pdf in the same tab, which means it's not possible to subscribe for updates. I had to use Firefox instead for it to work as intended.


The "call to action" button "get the prerelease now" redirs me to https://www.crypto101.io/testfile ... obviously b0rken.


I have no idea why; the Docker container just started randomly serving a really old version (several hours ago) for no apparent reason, and then started serving the right thing. I seriously have no clue what happened.

I am very sorry. Have a direct PDF link: https://9d0df72831e4b345bb93-4b37fd03e6af34f2323bb971f72f0c0...


...this is why one uses servers setup by hand on real, dedicated machines instead of five-level-virtualized piles of dung heap. Not unless the dung heap has refined over the course of a couple years.</rant>

Thanks for the link, it'll be a night lecture for me for some weeks, I can tell :)


> While there are also modes of operation (like OFB and CFB) that can produce self-synchronizing stream ciphers, these are far less common, and not discussed here.

Aw. :( Aren't there any noteworthy attacks against these modes?


While CFB can be considered a self-synchronizing stream cipher, OFB is a typical synchronous stream cipher. Being stream ciphers, these modes suffer from the usual bit-flipping attacks and nonce-reuse problems, same as CTR.

One problem specific to these feedback modes (and also to sponge functions) is the possibility of falling into a short cycle. A random permutation is expected to have log n cycles, with one big cycle taking around half of the values and a few shorter ones. Falling into a short cycle would imply quickly repeating the stream, which is catastrophic. The good news is that for a good block cipher the probability of this happening is overwhelmingly small, i.e., 1/2^(n-1) for block size n.


First time I hit the page, I got a https connection error, ssl_error_no_cypher_overlap

Tried it again and it's fine.


Likewise (running Firefox)


So, is the kindle edition in the making, or not available anymore, or what is going on with it?


Any crypto guide that doesn't include practical attacks against DES is a waste.


Then contribute to the guide.


I already took Crypto 101. Where are the next ones? :)


tptacek, are you involved with this?


STOP !


hahaha.... bad link sorry


Webpage not available - nice touch!


I am very sorry. Have a direct PDF link: https://9d0df72831e4b345bb93-4b37fd03e6af34f2323bb971f72f0c0...


> .io

not sure if legit




Applications are open for YC Summer 2019

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: