
Twitter urges users to change passwords after computer 'glitch' - petethomas
https://www.reuters.com/article/us-twitter-passwords/twitter-says-glitch-exposed-substantial-number-of-users-passwords-idUSKBN1I42JG?il=0
======
fatratchet
Actual twitter post:
[https://blog.twitter.com/official/en_us/topics/company/2018/...](https://blog.twitter.com/official/en_us/topics/company/2018/keeping-
your-account-secure.html)

"Due to a bug, passwords were written to an internal log before completing the
hashing process. We found this error ourselves, removed the passwords, and are
implementing plans to prevent this bug from happening again."

Exact same thing that github did just recently.

~~~
ams6110
So best practice would be that the cleartext password is never sent to the
server, so they could never log it even accidentally. That means the hashing
needs to be done client side, probably with JavaScript. Is there any safe way
to do that?

~~~
jrochkind1
nah, that just makes the "hashed password" the equivalent of the cleartext
password. Whatever it is your client sends to the server for auth is the thing
that needs to be protected. If the client sends a "hashed password", that's
just... the password. Which now needs to be protected. Since if someone has
it, they can just send it to the server for auth.

But you can do fancy cryptographic things where the server never sees the
password and it's still secure. like the entire field of public key
cryptography, diffie-hellman key exchange, etc.

~~~
hunter2_
"Since if someone has it, they can just send it to the server for auth" unless
it's only good for a few moments (the form you type it into constantly polling
for a new nonce).

~~~
deepbreath
The server would not be able to verify a changing hash without knowing the
password

------
tzs
A couple of steps you can take to reduce the chances of accidentally putting
sensitive information in a log.

1\. Make a list of all sensitive information that the test users in your test
environment will be giving to your application.

As part of your test procedure, search all logs for that information. This can
be as simple as having a text file with all the sensitive information, and
doing a 'grep -F -f sensitive.txt' on all your log files.

2\. If you are working in some sort of object oriented language, make separate
classes for sensitive data, such as a class Password to hold passwords, a
class CardNum to hold credit card numbers, and so on.

Make the method that whatever you use for logging uses to convert objects to
strings return a placeholder string for these objects, such as "Password",
"Credit Card Number", and so on.

~~~
geofft
Probably better than 2 is to use actual privilege separation between sensitive
data and as much of your service as possible. Handle passwords and password
changes in a separate microservice which exchanges it quickly for a session
cookie, so that the bulk of your application logic doesn't have passwords in
memory at all. For credit card numbers, do something like what Stripe does
where one API endpoint exchanges the card number for a token and every other
endpoint just needs the token, which means that Stripe can route that one URL
to services they pay extra attention to.

This assumes that passwords and credit card numbers are more sensitive than
session cookies or tokens, which is usually true because people tend to share
passwords across sites and definitely share credit card numbers across sites.
So the risk of logging cookies/tokens is much lower. Also, you can revoke all
cookies and token much more easily than you can force everyone to change
passwords or credit card numbers.

~~~
sbov
While good advice, your proposal is not necessarily better than #2 because #2
is something that happens automatically once the password hits your object
model.

If rather than a naked string you have a Password class with literally no way
to extract the plain text password then you can be positive that any code that
uses it will never accidentally log the password.

In contrast, if you rely on microservice tokenization you can still
accidentally log your password before your tokenization happens, just like
people are accidentally logging passwords before they are bcrypted.

Both proposals have a problem of logging raw requests or raw service calls
(outside of your object model).

Where plausible it would probably be best for microservice calls to only take
already bcrypted passwords, and error out if it detects one that isn't, so
there is zero chance of accidentally logging a plain text password when
calling a microservice.

Again, an actual object (e.g. HashedPassword) shines here, because your code
can automatically detect bad values the instant it hits your object model, and
refuse to properly log or give access to anything that looks like it isn't
already hashed.

~~~
geofft
You have to get the password _into_ the object model first, though—and your
object model can simply not contain a Password type in this process (and only
in the password-handling one). You shouldn't pass the password on as a naked
string and then drop it—you should prevent the password from getting past the
auth service at all.

I think the risk of logging raw passwords in the auth service model is lower
because logging in a password-specific microservice is an intuitively
dangerous thing to do, so both code authors and code reviewers will pay
heightened attention to it. Meanwhile, "log all requests" is a common thing to
want to do and will raise fewer alarms in a primarily-business-logic service.
(In fact, another usually reasonable thing to do is "log all requests that
don't parse properly and return 500...")

------
calebh
I highly recommend using a password manager. I finally bit the bullet and
started using 1Password a few weeks ago, and I haven't looked back since. It's
just so much better than having to remember a thousand different passwords.

Besides securely managing passwords, you can also use a password manager to
secure your digital legacy. 1Password has a feature where you can print out
"emergency kit" sheets that has the information required to access your
password vault. I printed out two of these sheets and gave them to trusted
family members in sealed envelopes. In the event that I become incapacitated,
they will be able to access my accounts.

~~~
racer-v
I keep all my passwords in a text file. I can't imagine remembering them all.
I suppose I should keep that file encrypted and synced to multiple devices
with rsync or so. Would a password manager give me any advantage over this
scheme?

~~~
freeone3000
No, that's basically what they do, but in a more user-friendly format.

~~~
sushid
Well if he's not keeping that text file encrypted, I'd argue that there is a
very significant difference in his methodology vs 1Password et al.

------
jwilliams
The Tweet from the Twitter CTO on this:
[https://twitter.com/paraga/status/992135139994943488](https://twitter.com/paraga/status/992135139994943488)

"We are sharing this information to help people make an informed decision
about their account security. We didn’t have to, but believe it’s the right
thing to do."

The "we didn't have to" is a little jarring given the scale of this.

~~~
bomb199
Well, nothing ever left Twitter's servers. The logs themselves would probably
be uninteresting to outside parties and inaccessible.

~~~
MBCook
Last year a contractor deleted the president’s account.

The fact it didn’t leave Twitter doesn’t mean everything is good. There are
still a LOT of people who may have had some kind of access to this data.

~~~
Alex3917
> Last year a contractor deleted the president’s account.

The fact that they undeleted it is strong evidence that he didn't have
discretion in how he performed his job, and thus was actually an employee and
not a contractor.

~~~
lstamour
I’m not sure how that follows. Are you suggesting they don’t keep backups or
use a “deletion” flag temporarily, e.g. as part of spam account removal?

~~~
SmellyGeekBoy
Indeed. I deleted my Twitter account recently, there was a message that data
is retained for 30 days to facilitate un-deletion. I assume their internal
process is the same.

------
fpgaminer
Back in my younger, move naive days when I first started using a password
manager and such a practice was not as widespread, I had the same question I
think most other people have: "If my computer gets hacked or infected, won't
the attacker instantly grab all the passwords from the manager?"

I chose to use a manager anyway, with the logic that if my system was
compromised I was owned either way; memorizing passwords provided very little,
if any, "softening" of the damage.

It's funny looking back on that fear now. Password managers have become more
mainstream, and we, collectively as users, have learned something interesting.
One of the assumptions of that hacking fear was that it was more likely for a
user's machine to become compromised than it was for the service's the user
uses to be compromised.

Well, as it turns out, the opposite was true and the fear was unfounded. The
vast majority of password leaks we've seen over the past decade have been due
not to malware but rather server compromises.

Perhaps that makes sense in retrospect. But it wasn't obvious a decade ago.
Back then viruses and malware were rampant on the internet. In many ways they
still are, but not like it was back then. Those were the days of pandemics
like the Blaster worm. So it made sense to be more afraid of your own machine
being compromised.

But the tides have turned. The landscape of OS security has improved
dramatically. The single points of failure offered by servers are far more
valuable now than the unwashed masses of networked user computers.

Just a funny retrospective that I thought I'd share.

~~~
hyder_m29
> The vast majority of password leaks we've seen over the past decade have
> been due not to malware but rather server compromises.

Something to consider is that malware-based compromises of personal systems
don't raise as much brouhaha as corporate-level compromises.

~~~
fpgaminer
My statement is based not on frequency of news, but rather on my understanding
of the provenance of passwords in password lists. The vast majority of
passwords in password lists are sourced from service hacks; not from user
malware. And that makes sense. Malware events net maybe thousands of user
passwords? On a good day maybe 100k. But hacks of major services like LinkedIn
... those yielded hundreds of millions of passwords. The two just don't
compare in magnitude. Remember that HIBP's list is sitting at something like
half a billion password combos right now.

~~~
geofft
One thing to note is that a remotely-compromised computer gets you access to
the actual browser session and not solely to the password. In fact, most of
the time the user has already logged in, and so the session is accessible but
the password isn't (users without password managers have no persistent store,
and password managers try to keep the secrets encrypted except when the master
password has been recently typed).

So it's not often useful to take a password from an individual user and put it
in a password list, but it _is_ often useful to maintain persistence on their
machine. Tools for doing that (RATs) are definitely common, both from random
internet attackers and from e.g. angry exes with physical access to your
device. But those aren't the attacks that e.g. HIBP is interested in, so if
you're looking at data from people interested in password compromises, the
effect is that you'll undercount client-side compromises.

------
wyldfire
Stupid question department:

W3c and IETF or other similar clever folks really like security stuff and do
lots of clever things to make us safer. So why couldn't we create a http
browser/server authentication method that has something closer to a nonce-
based challenge/response mechanism? If it were standardized, the browsers
could even do some clever hashing of some peer addresses or other things that
we think should be static. All of the browsers could still present a
"username/password" field that looks remarkably similar to existing forms but
never reveals the plaintext password to the peer.

As long as I could type my password into other browsers on other computers and
still get the same result, it seems like it's moving things forward. It would
be opt-in, traditional username/password forms would continue to live on for
decades to follow.

So what am I missing? Presumably something like this was considered and ruled
out?

EDIT: yes, of course, I forgot about the existing HTTP authentication
mechanism(s) in RFC 2069. I don't know why it never caught on but the fact
that the browser uses modal dialogues for these means a significantly
different context/user experience.

~~~
rocqua
These challenge response mechanisms still require a shared secret. This means
the server still needs to know either your password or a hashed version of it.
TLS covers the problems a challenge-response method is supposed to solve. That
is, TLS prevents replay attacks because the shared secret is sent under
encryption.

Really, the solution to exposing passwords to the endpoint is to do key-
derivation client-side, with a server-provided salt.

~~~
MarkMc
> Really, the solution to exposing passwords to the endpoint is to do key-
> derivation client-side, with a server-provided salt.

That sounds a lot like Secure Remote Password protocol:
[https://en.wikipedia.org/wiki/Secure_Remote_Password_protoco...](https://en.wikipedia.org/wiki/Secure_Remote_Password_protocol)

------
CiPHPerCoder
I've seen this scenario unfold before:

    
    
      1. User provides username/password.
      2. An exception occurs somewhere.
      3. The stack trace from the exception is logged.
      4. The stack trace includes the credentials.
      5. The exception ends up in a ticketing system (Trac, JIRA, etc.)
      6. Nobody notices for years.

~~~
testplzignore
As a bonus, sometimes the stack trace will be returned to the user, and maybe
it'll contain server-side secrets, too!

------
madrox
These comments are everything from eliminating passwords (good luck) to
“congress should fix it with law” (lol).

Having been in the identity space for a while and seen how various companies
think about it, these kinds of problems will keep coming up...mostly at
companies for whom identity is commodity. These companies will always give
account security the minimum requisite attention. No one at Twitter gets
excited about working on the login form.

I know OpenID was a bust and everyone hates Facebook Connect, but as an
industry we need to figure out how platforms that view account security as a
necessary evil can vendor that to people who take it seriously. Trying legal
avenues to get people to take it seriously or finding alternative methods is
what we’ve been trying for the last 15 years and it hasn’t worked.

~~~
cbhl
Security Keys are often used as a second factor, but the original vision was
that they could also be a primary factor in lieu of a password. Humans are bad
at passwords, but they carry around house keys every day. Why not also carry
around computer keys on their keychain?

For the average person (not worried about a megacorp or nation state attacking
them), TouchID is also a possible "password replacement" for the primary
authentication factor, although it comes with the disadvantage of not being
rotatable.

------
deallocator
At work one of our most commonly used libraries prints it's connection string
(including the plain text password, username and database) in the log files on
debug level (which I often see as the configured level). When I pointed it out
they told me it was intentional, and that attackers wouldn't go to the log
files anyway if they could acces the system. I gave up on the discussion at
that point

~~~
KeitIG
Maybe you should explain them there is a reason why passwords input is hidden
in terminals: to avoid them being kept in ~/.bash_history.

It is exactly the same problem.

~~~
ComodoHacker
There's another argument that might work in GP's case. A fair share of
successful attacks were successful not because the attackers actually broke
something on their well-guarded target. But because they used credentials
obtained from other poorly defended systems.

------
minimaxir
Twitter's CTO had an odd tweet about this disclosure (emphasis mine):

> We are sharing this information to help people make an informed decision
> about their account security. We didn’t have to, but believe it’s the right
> thing to do.

[https://twitter.com/paraga/status/992135139994943488](https://twitter.com/paraga/status/992135139994943488)

~~~
jms703
"I should not have said we didn’t have to share. I have felt strongly that we
should. My mistake."

[https://twitter.com/paraga/status/992146630232043520](https://twitter.com/paraga/status/992146630232043520)

~~~
gowld
Felt strongly that [Twitter] should have... "said we didn't have to share"?
"shared"?

Parag needs a Comms director to handle is Twitter account.

~~~
bradjohnson
Pretty obviously "shared" given the context.

------
spike021
>Due to a bug, passwords were written to an internal log before completing the
hashing process.

Hate to speculate, but it sounds possibly like perhaps a debug statement/log
level had been enabled for testing and forgotten about?

~~~
mithr
You wouldn't even need that -- just a request log that included the request
body without sanitizing it first :)

~~~
Jare
That would have never happened to me. Not ever!

------
vmarsy
The reuters article is pretty succint.

Here's the tweet about it:
[https://twitter.com/TwitterSupport/status/992132808192634881](https://twitter.com/TwitterSupport/status/992132808192634881)

which links to that blog:

[https://blog.twitter.com/official/en_us/topics/company/2018/...](https://blog.twitter.com/official/en_us/topics/company/2018/keeping-
your-account-secure.html)

------
gingernaut
They don't say what the timeframe for this issue is. Have passwords been
logged for the last 6 months? Last 3 years? Was this a bug found and fixed
last year, and only now are they reporting it?

~~~
softawre
The article does say:

> that they were exposed for “several months.”

~~~
lucb1e
Same as Github it seems.

I wonder what library was used, and which other companies use it which hasn't
told their users. That'd let us know who isn't as transparent...

------
djadmin
Are Github & Twitter using the same logging tools/service? Btw, Happy World
Password Day!

~~~
senoroink
For context: [https://www.zdnet.com/article/github-says-bug-exposed-
accoun...](https://www.zdnet.com/article/github-says-bug-exposed-account-
passwords/)

------
acjohnson55
This seems like the sort of problem that typed programming can vastly reduce.
In a language where string representation is controlled by the data type
itself, one could protect against emitting passwords in plaintext
accidentally. You'd still be vulnerable until deserialization happens, but
that's a lot less surface area to worry about.

------
dschuetz
Twitter hasn't figured out how to properly handle passwords after over a
decade of its existence? No, I'm not changing my password, I'm deleting my
Twitter account for good.

I'm tired of big shot Internet companies getting away with such bland
disregard of basic security and privacy rules.

~~~
cryptozeus
Its a mistake , it happens. Software is hard. Change your password.

~~~
dschuetz
It would be fine, but people who claim to be pro in software and are being
paid premium refuse to learn from mistakes, neither from their own nor from
others'. They just mitigate the fallout by saying things "It's was a mistake,
sorry about that, it happens, software is hard".

Brain surgery is hard. Mistakes happen. But after a few mistakes you probably
should stop doing brain surgery altogether. At least the patients will get a
higher chance to survive your surgeries by avoiding _you_.

In terms of security, handling passwords should be considered analogous to
brain surgery. A single mistake undermines the whole thing. If you can't
handle that, stop doing it, and let people do it who can handle it better.

~~~
richardwhiuk
Paying people more doesn't prevent mistakes.

If you banned every developer the first time they made a mistake their
wouldn't be any developers.

~~~
dschuetz
No, it doesn't. And that's not the issue. It's OK to make mistakes if you
learn from them to prevent them from happening twice or trice. But apparently
even if paid premium they don't. There were soooo many password db hacks and
cracks in the recent years, I can't believe people still defend bad software
engineering as if mistakes is a natural occurrence which cannot be prevented.
You can't prevent singular mistakes, but you well damn can make it hard to
exploit them!

If you keep and let bad developers in software security even if they made the
same mistakes (and mistakes of others) repeatedly there won't be any
_security_ left. So what's the point?

~~~
scrollaway
Would you relax? If what Twitter says is true (and there's no reason to think
it's not), these were passwords which were logged to plaintext logs, which
only people internal to the company can read.

We're not talking about a massive password breach, a bunch of script kiddies
who found a database of plaintext credit cards by going to /admin.php and
logging in with "admin / admin", or anything like that. We're talking about a
mistake Github themselves made (and if you think Github doesn't know what
they're doing in terms of security, I question your judgement).

Furthermore, _when_ was the last time there was a major security breach at
Twitter? You're claiming they're "keeping" bad developers and not learning
from their mistakes as if this was a regular occurence for them.

And coming from me, I don't usually defend security breaches and malpractice.
This doesn't really qualify. They made an official announcement, notified
_all_ users, even unaffected ones, both by email and on first login; that's
more than you can ask them to do.

What bothers me about reactionary posts like yours is they give negative
feedback to companies who actually do right by their breaches, which as is
well known in the security field, _is a matter of when, not if_.

------
sundvor
Why should I change my "passwords"? I only have one entry for Twitch in my
password manager.

It's 2018, no sane person would reuse their passwords across multiple sites
which can and do get hacked.

~~~
rudedogg
> It's 2018, no sane person would reuse their passwords across multiple sites
> which can and do get hacked.

All of my family does this :(. I think it's really common outside of tech
savvy people. I've tried pushing them to use a password manager on their
phone, or writing them down so they can use multiple passwords, but they'd
rather just use a single one.

Years of bad advice didn't help either:

    
    
        - Never write your password down
        - Make sure you use at least one symbol!
        - Make sure you use one capital letter!
    

[https://xkcd.com/936/](https://xkcd.com/936/)

~~~
sundvor
That's an awesome xkcd, thanks.

------
dmourati
Changed mine, just logged back in to this warning:

Keeping your account secure When you set a password for your Twitter account,
we use technology that masks it so no one at the company can see it. We
recently identified a bug that stored passwords unmasked in an internal log.
We have fixed the bug, and our investigation shows no indication of breach or
misuse by anyone.

Out of an abundance of caution, we ask that you consider changing your
password on all services where you’ve used this password. Learn more

------
oneeyedpigeon
> The glitch was related to Twitter’s use of a technology known as “hashing”
> that masks passwords as a user enters them by replacing them with numbers
> and letters, according to the blog.

Sigh. They appear to have confused hashing with asterisks.

> A bug caused the passwords to be written on an internal computer log before
> the hashing process was completed, the blog said.

So "related" in almost no way whatsoever, then? The state of technology
reporting in the mainstream press really makes me despair sometimes.

~~~
Johnny555
_The state of technology reporting in the mainstream press really makes me
despair sometimes._

The problem is that the mainstream press writes for mainstream users, and the
state of science/technology understanding in the general public is the real
problem.

~~~
oneeyedpigeon
I hear what you're saying, but I think — in this case — the article's just
giving too much irrelevant information. I think most mainstream users I know
would understand it if it were written something like:

> A bug caused passwords to be written on an internal computer log, the blog
> said.

~~~
stordoff
I'm not sure many users would understand the issue from just that - a
perfectly reasonable (albeit inaccurate) reaction to that would be "Of course
Twitter store my passwords - otherwise how can I log in?".

------
aylons
A honest question from a hardware guy:

Why isn't the password hashed and salted client-side, with the salt being sent
from server to client?

I understand that hashing in the client side would only substitute the user
password for a new one. This alone sounds like a win to me, as it at least
contains the damage somewhat in case of leakage, specially if salted.

EDIT:Sorry, I just found the exact same discussion in this thread.

------
Rjevski
It would be nice if they forced a password reset for all affected users and
submitted the old passwords to the Pwned Passwords list.

I have trouble believing _no_ passwords have been misused by insiders - sure,
there wasn't any large-scale misuse but I am sure someone poaching a few
passwords here and there for later mischief (once everything has settled down)
would have gone unnoticed.

~~~
ghazak
Agreed. I am assuming they will not do it for the same reason yahoo didn’t.
They are simply afraid of losing too many active users.

Yahoo’s case was worse though, since it was hacked.

~~~
Rjevski
If the previous breaches are of any indication, it's that users don't give a
shit - many major websites have leaked passwords
([https://haveibeenpwned.com/PwnedWebsites](https://haveibeenpwned.com/PwnedWebsites))
and they're still alive and kicking; for the ones that have gone down the
drain (Yahoo!) it was more because the service itself faded into irrelevance.

Based on that I'd say it would be pretty safe to disclose a breach and reset
all passwords; if your service is relevant your users will stay with you, and
if not then not disclosing a breach will only buy you time before the
inevitable happens anyway.

~~~
scrollaway
I suspect by "Losing active users", GP meant losing people who get confused or
stuck throughout the password reset phase. Twitter is used by a _lot_ of
computer novices.

Twitter did disclose this, through email and on first login. Anyone they'd
lose because of the breach is long gone and I also think it's probably next to
nobody.

~~~
ghazak
Yes, I’m purely talking about losing people if they forced a password reset.

------
fiddlerwoaroof
I once had configured a mail server to log passwords while I was figuring out
how to set it up. Then, I turned on logwatch and started using it “for real”.
Sometime later, while reading through the logs that logwatch was sending to
gmail, I discovered that I had been emailing myself login credentials via
plain-text emails.

------
unilynx
Can't we reduce the risk of these accidental disclosures by using
public/private key encryption? Ie:

\- server publishes a public key on a fixed url

\- your JS framework (react/angular/form handling/whatever) is modified to,
whenever you query the value of a input[type=password], to return a value
encrypted with the private key. (as close to the reading of the field as
possible, except for eg. 'repeat password' handling)

\- your serverside code decrypt the password as close to their
'VerifyPassword' and 'UpdatePassord' implementation as possible. and fail if
they receive an unencrypted password.

This would make it very hard for intermediate steps (middleware, RPC/REST/JSON
decoding) to accidentally spill the password, even if they fully log the
traffic, and is something frameworks should be able to help you enforce.

------
TangoTrotFox
I don't understand why servers don't just store passwords as a public/private
key schema. A private key is algorithmically generated from a password with
the public key stored on the server. When logging in the server sends a one-
use-only 'challenge' as an encrypted blob to the person logging in. Locally
they use the algorithmic private key generated from their password to decrypt
the blob and send the response back. And you're logged in.

Seems like a pretty simple system and absolutely nothing that has to be
secured is stored server side. Is there some clever reason I'm missing that
this system fails?

\----

Even better you can also salt the password->key schema based on something like
the username, making table based attacks infeasible.

------
TTPrograms
Naive question - wouldn't a lot of these issues be much better if passwords
were (on top of server side processing) salted and hashed client-side? Then in
principle you couldn't do these cross-site attacks where people reuse their
passwords.

~~~
zimlu
Cool idea, I like it.

------
dawnerd
So why are they not invalidating exposed passwords like Github did the other
day?

At the very least they should have a security alert at the top of your feed or
something.

Edit: Looks like they are alerting users.

~~~
lucb1e
Github didn't invalidate passwords. They sent an email to affected users (or
just everyone?) with a generic password reset link
([https://news.ycombinator.com/item?id=16972050](https://news.ycombinator.com/item?id=16972050)
)

~~~
dawnerd
I got forced logged and and required a password reset. That was an
invalidation.

~~~
lucb1e
Oh, then we must have gotten different emails (the one I got was in the linked
thread). I didn't know some people got something else!

~~~
dawnerd
Same email I got. Strange if not everyone was logged out.

------
md224
Might just be a weird coincidence, but at 2:14 PM today I got an e-mail from
Twitter with the subject "Security alert: new or unusual Twitter login". I
hadn't heard anything about the Twitter bug at that point (was it even public
at 2:14?) but I changed my password as a response to that e-mail.

Now I'm wondering if I need to change it _again_... does anyone know what time
the bug was patched?

------
ConcernedCoder
Thanks Twitter for making it easy to change my password. Took me all of 10
seconds to do that and review apps with twitter privileges.

------
zeec
I think asymmetric encryption would solve all problems. The sever only ever
stores the users public key and uses challenge-response to prevent replay
attacks.

This is a bad idea, but if you are required that your uses have passwords, you
could use their password to seed an elliptic curve private key to create the
user key on the fly.

~~~
isostatic
So you go to a site, you enter your password, but then javascript creates a
private key and public key based on the userid and password (which is a unique
tuple). The key will always be the same regardless of the password.

When you first set your password, it sends the public key to the server, which
is then stored. If that key was intercepted, it's not a problem.

Then the server challenges by sending a random code, which the client uses the
private key to encode, sends back, the server then decodes with the public
key. That guarantees that the private key is known by the client, and thus the
password is known, but the private key never leaves the machine.

The second time you go to the site, you get the challenge, and respond.
Neither private nor public key is transferred.

Do I understand that right?

~~~
zeec
> "The key will always be the same regardless of the password." The keypair is
> different for each userid and password. Everything else is correct.

------
beirut_bootleg
No amount of entropy will protect against plain security goofs. "It would take
a computer about 115 OCTILLION YEARS to crack your password" some analysis
tells me. Guess how many engineers it takes to downgrade any password's
strength to "123456".

------
PakG1
I have two-factor authentication with Twitter and wherever else I can using
authy. Isn't that enough for me to not change my password? It truly gets
bothersome. Maybe I'll need to break down one day and finally get a password
manager. :(

------
mrguyorama
I haven't actively logged into twitter in almost a year. Does this affect me?

~~~
aviv
Tinfoil in me says this is exactly the point - to bring people like you back
for a second chance.

~~~
a_imho
Or get people to accept some change in toc/pp by/after signin-in, possibly due
to GDPR pressure.

------
oneeyedpigeon
If this happens in a month's time, will they have to pay out $96,000,000 via
the GDPR? This came up at my place of work the other day: are data leaks as a
result of bugs breaches of the act?

~~~
beberlei
IANAL: No, there is a process of stuff before you will get fined. A human
error does not suddenly lead to heavy fines, if you can prove that you put
everything you can to prevent this from happening, i.e. follow all the
processes GDPR defines like data processing agreements with third party
processors, technical and organizational measures to protect against privacy
violations, documentation of personal data stored with type of processing,
type of consent, and such. what example would a data privacy agency set to
fine a company that is fully compliant? I imagine you could even get court to
side with you on not having to pay a fine.

data leaks of course are always a breach and must be reported. that doesnt
mean you get a fine though.

~~~
stordoff
Specifically, you'd look to Article 83[1]. It being an unintentional act
(83(2)(b)), otherwise following best practices (83(2)(d)), taking steps to
mitigate the damage (83(2)(c)), and that Twitter announced the breach
(83(2)(h)) weigh heavily in favour of the fine being minimal, or there being
no fine at all. Article 83(1) also notes that fines must be "proportionate and
dissuasive" \-- there is nothing proportionate about imposing the maximum fine
for a simple error, nor dissuasive about fining a company that is otherwise
compliant and following best practices.

[1] [https://gdpr-info.eu/art-83-gdpr/](https://gdpr-info.eu/art-83-gdpr/)

------
cryptozeus
Announcement is underplaying the situation. Absolutely change the
password..anyone could have seen it. Just because there ia no evidence of log
reading does not mean anything.

------
mtgx
I think it's ridiculous that Twitter still allows users to reset their
passwords through SMS. How long are they going to pretend that SMS is secure?

------
speedplane
The partial answer to this is to dump detailed logs after a few days or weeks.
That way, even if disaster strikes like this, it can be somewhat contained.

------
gsich
Why even hash the password on the remote side, shouldn't some challenge-
response hashing be more secure?

The most likely downside is that without JS it won't work.

~~~
LyndsySimon
If you hash the password on the client, the hash made on the client is now the
password. It's easy enough to send the hash directly, and the server is none
the wiser.

That's not to say it's a bad idea, though. I have used client-side hashing in
the past to allow passwords of arbitrary length while using finite network
resources.

~~~
John_KZ
That's not true if you ask the client to hash both the password and a one-time
token you just send. Knowing both the password/key (which was send during
registration) and the token, you can calculate the hash. Otherwise you can't.
Password/key is never transmitted. You can also use asymmetric encryption. I
always cringe with amateur devs sending plaintext.

~~~
yorwba
Your scheme requires the server to store the password in plaintext, and it's
transmitted at least during registration.

~~~
John_KZ
It does. You can do the same with with the salted hash of the password and not
the password the user entered. It's still worse than real authentication
protocols where you don't store the shared secret at all, but better than
those diy schemes described earlier.

Also you can't get away from sending the shared secret during registration.
Even with asymmetric cryptography you have to rely on the TLS to make sure you
exchange the real untampered public keys.

------
propman
We need a regulatory rulebook codified in law by congress that fines companies
that make these "mistakes". Enough of a fine will force companies to take
these "mistakes" seriously.

In Yahoo's case, that might have forced Marissa to actually keep a
cybersecurity team and not cut them when she knew the systems were in danger
of being compromised. We aren't getting any jail time, but hefty fines that
don't stifle growth, just punish negligence and carelessness that are codified
and don't need long court hearings to pass are a must.

~~~
zerostar07
"Technology by legislation" \- that's the new trend it seems. How about
replacing passwords with something technologically superior instead?

~~~
aaronbrethorst
It isn't a new trend.

Section 508 of the Rehabilitation Act legislated that the government purchase
accessible software.

HIPAA legislated that your medical data be kept secure.

Minnesota, Nevada, and Washington have enshrined some or all of PCI DSS into
law:
[https://en.wikipedia.org/wiki/Payment_Card_Industry_Data_Sec...](https://en.wikipedia.org/wiki/Payment_Card_Industry_Data_Security_Standard#Mandated_compliance)

A little farther afield, seat belt technology has been legally mandated to be
included in most automobiles sold in the United States since 1968:
[https://en.wikipedia.org/wiki/Seat_belt_laws_in_the_United_S...](https://en.wikipedia.org/wiki/Seat_belt_laws_in_the_United_States)

~~~
zerostar07
Those are great examples of regulations that actually benefit the user. When
the regulation however addresses the wrong problem, then it becomes burdensome
and dumb. If the user uses a simplistic password, his account will be hacked
even if it's never leaked. Studies show that bad passwords and phishing are
more destructive than leaks.

------
applekicks
Did not GitHub do the exact same thing yesterday?

------
mundanevoice
Exactly the same thing happened with GitHub couple of days back. Funny both
faced the same issue within days of each other.

------
samfisher83
Wouldn't you have to be an employee of the company to get the passwords since
you need access to the log files?

------
cgb223
So why arent they hashing passwords on the user side?

Can't leak a password if the only thing you have to begin with is a hash...

~~~
inimino
Asked and answered in the thread above:

[https://news.ycombinator.com/item?id=16990570](https://news.ycombinator.com/item?id=16990570)

------
honoredb
It's really hard to avoid logging passwords when there's a parse error on
structured data that includes a password, for example if there's malformed
JSON in the authentication HTTP request. A weird error like that is exactly
when verbose logs are essential, but it's also a time when your system doesn't
know what it's logging. Amazon has a machine-learning based service to try to
deal with this.

------
llccbb
Legitimate, non-cynical, non-opportunistic question. Will this boost Twitter's
monthly active users?

------
jonjojr
must be the same logging library/framework as github. I want to know which
library it is.

~~~
lucb1e
And which other companies use(d) it, so then we know whom to trust (i.e. those
who came forward after finding it).

------
mancerayder
How exactly does one manage in a major company to set up an app in production
that spits passwords, or even less sensitive customer data, to a logfile?

Is that the state of infrastructure engineering? The cynic in me wonders if
it's related to the trend to take developers with some systems knowledge and
turn them into Ops/SA's.

------
rocky1138
While you're in there, enable 2-factor authentication if you haven't already.

~~~
bm5k
it seems like they only offer SMS 2fa which is a bummer.

~~~
rocky1138
That isn't true. I use Google Authenticator with them.

------
flying_sheep
This will be my excuse of simply logging everything :-)

// logger.log(request.payload);

------
iamgopal
And I thought first hashing done at browser level.

------
asdsa5325
Did GitHub's post make them realize this?

------
aurelien
That is just to force users to accept their f __k __g rules. S __k my d __k b
__*h Twitter!

------
supergirl
if they can leak these, god knows what else they leak and where and for how
long

------
pishpash
For the thousandth time, can we be done with passwords and move to physical
tokens please?

------
mbid
Reminder that none of this would be necessary if nobody reused their
passwords.

~~~
davidcbc
Sure it would. People using Twitter still need to change their password even
if it is the only place they use their password.

~~~
mbid
To defend against what? That some guy at twitter who saw the logs can login
and change somebody's status?

Anyway, I guess you're right, I simplified. It is kind of a valid point,
although if I was this paranoid then I would never leave the house. I just
wanted to say that if the web wouldn't have been built on a giant piece of s
__ __ __ __*, for example password authentication, none of this would have
been necessary.

------
dmourati
“We are very sorry this happened,” the Twitter blog said.

~~~
abootstrapper
That stuck out to me too. They skirt admitting blame with that carefully
crafted apology.

------
uptown
The "right" way to handle this is to revoke access to all logged-in devices,
revoke all previous passwords, send an email notifying users of the
revocations, and force a password change upon next login. But Twitter won't do
that because a non-trivial percentage of their user base would never jump
through the hoops to get things up and running again and Twitter's MAU numbers
would sink.

~~~
49bc
This doesn’t make a big impact in the end because of how prevalent password
reuse is.

~~~
quickthrower2
This is an opportunity to run Troy's filter on the new password.

------
77pt77
How is something like this even possible?

It just sounds so incompetent.

~~~
thln666
seriously? pretty easily. somebody probably left a debug log message in place
or something. guaranteed that this happens all the time and most people don't
report it.

~~~
chatmasta
I doubt anyone left something that logged the plaintext password. No
reasonable architecture necessitates holding onto a plaintext password for
more than one line of code.

One possibility is an HTTP server on the request path after TLS termination.
But then why is an HTTP server logging the request body?

My guess would be some sort of instrumentation process was blindly reading
data in memory without distinguishing _what_ the data was, but produced logs
that incidentally included passwords.

~~~
testplzignore
In my experience, I've seen both of the following scenarios:

POST request comes in from the client. Full URL and request body is logged.
Sometimes for simply troubleshooting, sometimes for security reasons (e.g.,
wanting to know all data coming in so that it's possible to identify security
holes after they've been exploited).

POST request comes in from client. Frontend server makes a GET request to a
backend server, and the password ends up in the standard request logs. In one
case, I've seen this happen because the developer thought path variables were
cool, so every API they wrote looked like /a/b/c/d/e. Sigh.

------
anonytrary
> Due to a bug, passwords were written to an internal log before completing
> the hashing process

This isn't a bug, it's incompetence. Remember folks, never, ever use the same
password for two different websites. Assume all websites store your password
in plain text even if they hash it at some point. With logging and tracking on
steroids these days, I expect "bugs" like this to be commonplace.

------
aviv
This smells like an attempt by Twitter to get as many users as possible to log
in to Twitter so they can report XXXX% increase in monthly active users. With
the added bonus of perhaps converting some who gave up on Twitter back into
actual active users going forward.

