Personally I don't see any reason for having them completely randomly generated.
'thIs1smyp4ssw0rd19%2' isn't any less secure than another 20 character password that includes lower+upper case letters, numbers and special characters.
Obviously, if you do something like 'c0r1np4ssw0rd" then it may get to the stage where enough people do that for crackers to expect it (maybe it already is, but as long as you follow his third and fourth rules ("Use mixed-case, numbers and special characters" and "Use passwords of at least 12 characters") you really should be fine, and you'll have an easier time memorising them.
(I can remember multiple 20+ character passwords that would be very difficult to crack, and have no need to write them down.)
That's not quite true. When we're talking about the security of a password, we're talking about how long it's going to take someone to crack a possibly salted hash (Gawker's weren't) of your password. When attempting to crack a hash, the attacker begins with dictionary words, then with leetspeak permutations of dictionary words, then starts adding random numbers in with random words.
It's hard to say in what order specifically they're going to try it, but generally speaking, they go from fewer bits to higher bits. thIs 1s my p4ssw0rd - that's not quite as many bits as you think it is. That's 4 permutations of dictionary words. That password probably hits the requirement of 80 bits, but it is less secure than other 20 character passwords. thIs1smyp4ssw0rd is going to be tried in its various permutations fairly early on in the cracking, and tacking on a few extra characters, while an order of magnitude increase in compute time, is not outside the realm of possibility.
Realistically if you take into account the possibility of using up to four works, with some but not all vowels replaced with numbers, a mixture of upper/lower, and some other random numbers and special characters in an undisclosed place... that creates such a huge number of possibilities that it might as well not be dictionary based from the cracker's point of view.
Write your scheme down, compute the number of permutations it makes available, then divide by the total number of possibilities allowed by your password set (a straight [possible_chars]^[length] computation).
The results will surprise you. The space of things you describe is far smaller relative to the available space than you realize.
You can indeed make this big enough to work, and it's easy to make it big enough that only a dedicated cracker could get through it, if that. My point isn't that you're wrong, but that you may be less right than you realize.
The thing is, even if it's true for the one scheme I suggested, unless you know the format a password will be in you can't aim directly for that scheme, meaning you can't limit the number of possibilities that much.
I could pick a password that is "coRInStaNdr3ws19@90" which is my first name, my first school, my year of birth, and a few capitals/numbers/special characters. If an attacker knows that's my type of password then it might make it easier for him, but 'corin' isn't in dictionaries (sure they might think to find out my name and add it as a dictionary word), and I doubt "st" and "andrews" would be. Realistically, unless I give away my rough password format, they won't be able to narrow it down and will be left with a password that's just as hard to crack as if it was completely random.
You're making a false assumption that the hacker must somehow divine your pattern, but that is not necessary. An attacker could mount a Markov-chain based attack based on common phonetic patterns (and even common numbers, since some will show up more often than others) and radically cut down the search space even without assuming any aspect of your "pattern", and what you cited will get caught in that orders of magnitude before a truly random password. Along with enormous numbers of patterns that you could cite, as well, this is pretty much a superset of them all.
When you cut the possibility space down, you have cut the possibility space down. You can't fool entropy. The math is quite vicious that way.
You're encountering the "everybody can create an encryption scheme that they themselves can't crack" problem. You may not be able to think of how to abuse low-entropy passwords to crack something far longer than you "ought" to be able to, but that doesn't make it impossible, or even necessarily hard for an attacker.
The problem with this approach is that you can only remember a few passwords like this, which means that you are going to reuse passwords on multiple sites. One site compromise can compromise multiple accounts for you.
I don't have any problem remembering them personally (then again, I still remember root passwords I used when I was thirteen and they were completely random characters).
Sites like HN and Reddit I'm happy to use the same password (I'd be minorly annoyed, but realistically I wouldn't be that fussed if I lost control of either account, I can either recover it or even create a new one), but any site where someone finding my password could cause a problem has its own password, and I can remember them all.
Occasionally I forget which password goes with which site, but not often - actually I find it harder to remember which username/email goes with each site.
actually I find it harder to remember which username/email goes with each site
So very true. My password creation process is fairly memorable, and tends to meet the requirements of any site (password length, numbers, capitals etc). But user names - it's an email address, it's the name on your email address, they do accept spaces, they don't accept spaces, they only have 8 characters ... drives me mad !
'thIs1smyp4ssw0rd19%2' isn't any less secure than another 20 character password that includes lower+upper case letters, numbers and special characters.
Obviously, if you do something like 'c0r1np4ssw0rd" then it may get to the stage where enough people do that for crackers to expect it (maybe it already is, but as long as you follow his third and fourth rules ("Use mixed-case, numbers and special characters" and "Use passwords of at least 12 characters") you really should be fine, and you'll have an easier time memorising them.
(I can remember multiple 20+ character passwords that would be very difficult to crack, and have no need to write them down.)