
0day vulnerability full disclosure: American Express - michiel3
http://qnrq.se/full-disclosure-american-express/
======
duncan_bayne
Typing this up in real time ...

I called American Express Australia to report the defect & I was transferred
through to the American call centre.

The CSR to whom I spoke transferred me through to a different department,
after I explained that I didn't have an account. She did ask whether "I
received an email" which I assume was some sort of inquiry as to whether I had
been phished.

I then spoke to an online services rep., who after asking for my card number,
listened to my report. She then put me on hold.

(The call had taken 10 minutes by this time).

After a few more minutes on hold, the CSR came back on the line, asked me to
repeat the information, and confirmed for the umpteenth time that I don't have
an American Express card. I explained that it wasn't my find, but that it had
been published online & so was by now _very_ public.

(15 minutes by this time, most of that on hold listening to advertising for
American Express, including some ironic praise for their website).

CSR comes back on the line. She's spoken to her 'technical team' who assure me
that there's nothing insecure going on because it's all over HTTPS. So I
politely walked her through the process - visit the page, add ?debug to the
URL, click the admin link & behold: lots of should-be-secure stuff.

At this point she thanks me profusely, & asks that I hold while she speaks to
her supervisor. Back to the American Express ads ...

(20 minutes at this point).

The CSR came back on the line, thanked me again, & said that her supervisor
had taken a screenshot of the issue & escalated it. Job done.

So, yeah, I can totally understand the frustration experienced by the guy who
discovered the vulnerability. But it certainly wasn't impossible for me to
report the issue, & I'm in Australia.

~~~
T-hawk
Do remember that for every real serious problem such as this, American Express
and any other large company receives a hundred or a thousand calls for non-
issues like phishing or minor issues like a compromised card number to cancel
and reissue. The system is optimized for these common cases; the edge case of
a real public vulnerability will require extra effort and it's not a failing
of Amex that the system is so.

~~~
leif
when the cost of a false negative is so damn high, you'd think they would know
not to filter so aggressively

~~~
bnr
at some point, all the people dealing with the false positives will be
costlier.

~~~
stygianguest
Seriously? I won't take American Express anytime soon, I can tell you that. If
this gets picked up in mainstream media, it is devastating.

~~~
bdr
More likely: people will be assured the problem is fixed, and then not care.

------
jgrahamc
Some years ago when I was doing more stuff in spam and phishing I came across
a phishing site for a small US bank. The list of phished card details was
available through the interface and it was clear that there were some real
people local to the bank who had given their name, address, card number, PIN,
SSN, ... everything.

I decided to contact the bank. After filling in the form for contact on their
web site giving all the details of the site, I did get an email back and
eventually I got someone on the phone. This person (who said they were in
charge of bank computer security) thanked me and said that they were going to
try to deal with it (I had also contacted the school district whose computer
was hosting the site to get it shut down).

I then told this person that there were real account details on the phisher
site and would they like the list of people's account numbers so they could
inform their customer/shut down their debit card etc. The bank officer
replied, "No." As far as they were concerned the people who were that stupid
got what they deserved.

I was flabbergasted, but couldn't do much to make the bank do something.

So, using the names and addresses of the people from the phishing site I
managed to track a couple of them down (they were small businesses whose
business addresses were available on the web) and phoned them up so they would
be alerted. They took it pretty well considering that some weird British guy
was calling them from France to tell them their US bank account details were
at risk.

~~~
CWuestefeld
A few years back, when "Verified by Visa" first came out, I was taken aback
the first time I saw it. It's not at all hard to imagine that you're being
phished by this strange page.

I called the customer service number for my Visa card and asked if this was a
_real_ Visa card "feature". After spending a couple of minutes asking around,
nobody knew what the heck it was.

If Visa has a division that takes security seriously, they certainly need to
work hard on the customer-facing aspects of it.

~~~
shabble
VbV has to be some of the worst security engineering I've ever seen. iframe
content, arbitrary domain (securesuite?!), trivially guessable or resettable
details.

~~~
mattmanser
And to add on top it _looks_ like a con, the design is horrific.

------
maxniederhofer
They knew this was open. They even took it out of their robots.txt :)

<https://www.americanexpress.com/robots.txt>

User-agent: * Disallow: /us/admin/ Disallow: /us/heroes/ Allow:

~~~
coenhyde
I apologise in advance for a lack luster comment, but seeing incompetence on
so many levels like this on a monthly basis from financial institutions makes
me want to be sick.

This is like putting a sign out the front of your house saying please do not
enter though the back window, it's open.

~~~
jrockway
I look at this as a good thing. I know that if I am ever injured in such a way
as to receive severe brain damage, I'll still be able to get a high-paying
programming job.

------
demetris
The first three Twitter messages by the vulnerability reporter are:

“@AmericanExpress Who can I contact regarding security vulnerabilities in your
system? I'm not available through phone, physical mail or fax”

“@AmericanExpress Just to clarify: I have vulnerabilities. This should be
"urgent", so no technical support jungle please :-)”

“@AmericanExpress I've been trying to get in touch with AMEX regarding
security vulnerabilities in your system for a while. Who do I speak to?”

I think this is not ideally expressive language when you talk to a lay-person
representative on Twitter. I believe a better result could be achieved with
simpler and clearer language:

“@AmericanExpress I have discovered a serious security issue in your web
system (money can be stolen). Please help me report it to someone
responsible.”

~~~
danvideo
yep, credit that the guy partially tried - but prefacing your first
interaction about a serious issue by "I'm not available [to contact through
most of the usual communication methods]" is sort of self-defeating.

~~~
nknight
He's reporting a vulnerability on a _website_. It is absolutely reasonable to
expect to be able to report it through email, and utterly ridiculous of AMEX
to refuse. That's where the conversation ends, not with "well, you should
spend your time fighting through these costly and obsolete mechanisms so you
can do us a favor".

------
uptown
Here's something I learned from AMEX last week ... if one of your cards gets
compromised and you cancel the card, AMEX will continue to allow charges to
flow through that old "canceled" number to your newly issued number if those
charges are coming from a "trusted recurring entity". I discovered that
charges were continuing to flow through a number that I'd canceled due to it
being compromised even though I thought it'd been nullified. AMEX explained
that their policy is to allow these charges to continue, and it took a number
of months before I caught the problem because the charge was coming from a
business I continued to have business with. Apparently the person that stole
my number had setup a recurring charge with this business as well. To their
credit, AMEX removed all of these charges even though they spanned a number of
months ... but it caught me completely by surprise that a number I though was
canceled was still allowing charges to flow through it.

~~~
sjtgraham
I have an AMEX card that expired in 2007 and it is still successfully charged
by AWS each month. Apparently, it's a big pain to get customers to re-enter
new payment details when cards expire, as a result I believe merchants are
often allowed to charge to cards that have long since expired.

~~~
gcp
I was once very close to a server (that was in active use) getting
disconnected/wiped by a hosting provider because my CC expired, and their mail
to inform me of that got lost.

Only figured that one when the site went offline, didn't come back, and I
started bitching at their support.

So I have some sympathy for the CC company being lax with recurring charges on
expired cards. Would be a nice service if they went ahead and called you up in
such a situation.

~~~
click170
One would think it would be common sense to double check recurring charges to
a canceled card with the account holder. Then again, when have financial
institutions ever followed common sense..

------
Nitramp
The author should have contacted the email addresses given in the DNS WHOIS
(amexdns@aexp.com, gtld@aexp.com) and the obvious aliases (security@...).

However I can understand and sympathize, it's enraging how hard it is to get
into contact with a person of any kind at certain companies (KLM/Air France,
I'm looking at you). I understand they want to save money, but if you run a
business, you have to be contactable in one way or another. And snail mail as
the last option really doesn't cut it in the 21st century.

~~~
asto
Extremely hard to get in touch with Google as well. And you only tend to
realise it when something goes horribly wrong - like when your adsense account
gets suspended.

~~~
eli
Yeah, but it's not hard to report a security problem:
[https://www.google.com/appserve/security-
bugs/new?rl=usrwf3z...](https://www.google.com/appserve/security-
bugs/new?rl=usrwf3z65ebo2rey87mtsxmr)

------
jgrahamc
Wow. All you need to do to activate this is append ?debug to the main American
Express URL: <https://www.americanexpress.com/?debug>

------
epenn
When a major company, especially a financial services company, is subject to
public security vulnerability disclosures like this, it should really make
other companies stand up and take notice. There is absolutely no excuse for
these kinds of vulnerabilities to exist on a production system. When Citibank
was recently hacked by simply changing the account number in URLs, that should
have been enough for other financial institutions to do an internal security
audit to make sure they weren't susceptible to anything similar. Don't wait
until it's too late. For the sake of their customers I hope this is resolved
swiftly.

~~~
viraptor
It seems the bigger the company is the more irresponsible they become. In UK
in the bank I use, you can activate protection of your debit card / current
account (usage analysis, higher insurance), but to do that you need to
register with Experian (credit rating company). The process for that is: put
your recent bill, bank statement and photocopy of ID in an envelope and post
it to them via normal mail.

I decided to ignore that great offer and keep my account secure in traditional
way. Apparently ignorance with regards to the internet sites is not what
causes big companies to act in stupid ways. It's the whole mindset...

~~~
sliverstorm
I'm pretty sure normal mail is generally quite secure. Sure, there's very
little barrier to someone opening your envelope, but perhaps because the ratio
of sensitive stuff vs. letters to grandma is so low, I'm not aware of it ever
happening much.

~~~
viraptor
There's one big difference in those letters though. The letter to grandma will
be addressed to a person. The letter with documents for Experian will be
addressed to... Experian, which is a known company dealing with money and
personal data.

~~~
sliverstorm
True, but the sheer volume of mail makes sorting through it a formidable task.
The US Postal Service has sorting capacity like you wouldn't believe.

A criminal could in theory target Experian's mailing address, so perhaps it
simply comes down to whether you believe they can secure their property.

------
aiham
// don't ask me how exactly, but this gets the main domain froma hostname;

This explains a lot. What I don't understand though, is why this guy, who
doesn't understand basic regular expressions (the expression is also wrong),
is working on the American Express website.

~~~
danso
The regex:

// don't ask me how exactly, but this gets the main domain froma hostname;

    
    
      var hostArray = /([^.]+(.com))$/ 
    
    

LOL.

------
chaz
Next time, I would try reaching their Public Relations group for help. PR
people are almost always accessible by name, phone, and email -- they're
usually on the bottom of every press release that goes out. They also have
good internal channels to every part of the company and know who to contact.

Googling for "american express public relations" turns up a page with three
NY-based vice presidents, with direct lines and email addresses listed:
<http://about.americanexpress.com/news/media_contacts.aspx>

------
InclinedPlane
Unrelated, it looks like someone at AmEx finally improved their crazy, broken
password system at least, this used to be the password requirement:

 _"Your Password should contain 6 to 8 characters . at least one letter and
one number (not case sensitive), contain no spaces or special characters (e.g.
&, >, _, $, @) and be different from your User ID."*

Now it's this:

 _"Your Password must be different from your User ID, must contain 8 to 20
characters, including one letter and number, may include the following
characters: %, &, _, ?, #, =, -, cannot have any spaces and will not be case
sensitive."_

~~~
desigooner
Is the "will not be case sensitive" just a typo or do they enforce case
insensitivity?! If they do, that's horrendous.

~~~
aidenn0
Try logging into various sights with a case-flipped version of your password,
you'd be surprised(horrified?) how often it works

------
rgarcia
Can someone explain the origin or meaning of the word "hero" to describe
primary marketing/call to action sections? I saw it first in the twitter
bootstrap code [1], and now here.

[1] view-source: <http://twitter.github.com/bootstrap/examples/hero.html>

~~~
showerst
I'm not so sure about the origin, but it's commonly used in design/UX to
showcase one primary or "Hero" product, and refers to a large space front-and-
center above the fold on a page. Think apple's site putting up a huge iPhone
image on release day (<http://www.sprint.com> is another good example).

I think this likely started in physical product sites and the lingo just
stuck.

------
ch0wn
Oh wow, unprotected admin tools and an XSS vulnerability on their main
homepage that is used for customer logins. That's pretty bad.

~~~
JoshTriplett
The utter lack of a mechanism to report bugs, particularly security bugs,
seems far worse.

I've encountered this problem frequently when interacting with various
organizations. The pervasive availability of bug-tracking systems and/or bug-
reporting email addresses makes the absence of one quite conspicuous.

~~~
michiel3
I've many seen organizations applying spam filtering on their security@org.org
address, leading to tons of reports ending up in spam boxes without being
noticed by the company. The researcher doesn't receive any feedback on his
responsible disclosure and multiple reminders, and finally submits the
vulnerability to a full disclosure list.

~~~
JoshTriplett
Even worse: some businesses apply spam filtering to their abuse@ address,
which thus rejects reports of spam as...spam.

------
viraptor
This is crazy... when you go to the admin panel
<https://www.americanexpress.com/us/admin/> you actually get access to user
cookies (session ids) which probably allow you to hijack their session
(haven't tried it in case it's going to be traced back...)

------
Robin_Message
Surely a DM message to the AskAmex account, with some actual details written
in clear English, not jargon or "hacker lingo stuff" would have been more
suitable? Or asking someone on here like Thomas to make a phone call?

I understand the argument between full disclosure and responsible disclosure,
but if the author could have DM'd it on Twitter. Or posted it on Twitter
wholesale, since its now public anyway.

~~~
redthrowaway
The operator of the AskAmex account seemed completely clueless on security-
related matters. I doubt saying, "visit this URL:
[https://www.americanexpress.com/?debug=true&heroOverride...](https://www.americanexpress.com/?debug=true&heroOverride=%3c%73%63%72%69%70%74%3e%61%6c%65%72%74%28%27%68%61%78%27%29%3c%2f%73%63%72%69%70%74%3e)
would have registered as a problem for her.

AMEX made it incredibly difficult for this guy to report the issue to anyone
who had the slightest clue as to its severity. Banging his head against the
wall until someone finally clued in would not have fixed that communication
issue. Full disclosure just might.

~~~
Robin_Message
All the more reason to make as clear and straightforward a declaration as
possible. Not "I have vulnerabilities", but a DM saying "American Express is
leaking customer information at this URL and it is imperative this is reported
to your security department." It's their problem to escalate if they don't
understand, but you have to give enough information to make escalation
possible.

~~~
danvideo
Agreed Robin - it's likely that the person operating the twitter account for
most huge companies has minimal, if any, interaction with IT/security and its
lingo.

Speak plainly people.

~~~
redthrowaway
How much more plain than "Who can I contact regarding security vulnerabilities
in your system" can you get? When she asked what kind of vulnerabilities,
would saying, "unsecured admin panel and xss allowing for session jacking and
spoofing" really have been more meaningful than what he said? Even saying
"unsecured admin panel" on twitter would have sent people scrambling for it.
He was attempting responsible disclosure before he turned to full disclosure.

~~~
darklajid
Right.

All you guys (not targeted specifically at you here) that say 'He tried it in
a clear way': Call one of the lesser technical inclined people in your
family/among your friends. Tell them you've just read about a security
vulnerability and wonder if they could describe what that is to one (possibly
less technical inclined) people in their family/among their friends.

That's essentially what you're looking at if you throw these words at a
corporate marketing (with some links to support) drone that needs to fill in
his/her supervisors to make anything special happen.

------
avree
It's amazing that such a huge oversight can be made. I hope American Express
doesn't try to sue this guy.

~~~
toyg
The admin page is completely unprotected, you don't even get a notice about
the system being private. They can't sue him, they wouldn't have a leg to
stand on.

To me it looks like somebody left a "DEBUG = True" somewhere on the site and
went to the beach :)

~~~
mootothemax
_They can't sue him, they wouldn't have a leg to stand on._

Why not? I've been clicking around various US laws and have yet to see any
mention about a login screen or "Go away, private!" messages being required
before it counts as unauthorized access:

[http://www.irongeek.com/i.php?page=computerlaws/state-
hackin...](http://www.irongeek.com/i.php?page=computerlaws/state-hacking-laws)

~~~
rufibarbatus
The question is, is the disclosure of this (technically public) resource
location litigation material?

IANAL, but I can definitely see both sides of this debate being highly
defensible.

~~~
onemoreact
In the US the first time you sue someone can be for any reason. It's only
considered abusive if you do it _several_ times without reason.

------
sudonim
Does going to the url <https://www.americanexpress.com/us/admin/> constitute
"computer hacking"? It's not protected in any way, shape or form.

~~~
mootothemax
_Does going to the url<https://www.americanexpress.com/us/admin/> constitute
"computer hacking"? It's not protected in any way, shape or form._

I believe that the level of hacking/cracking required is irrelevant to most
laws around the world; if you're not meant to be there, you're guilty of an
offence.

I'm sure lawyers could argue intent all day long, but whether or not a logic
screen appears is irrelevant.

~~~
Evgeny
_if you're not meant to be there, you're guilty of an offence._

But how would I know? If someone's private property is not marked as such in
any way, would I be a trespasser if I wander into it? Let's say it's part of a
field or a forest, not a building with doors ...

~~~
mootothemax
_But how would I know? If someone's private property is not marked as such in
any way, would I be a trespasser if I wander into it?_

According to this page (the first result I found in Google - there may be more
reliable information out there), it's not an easy question to answer:

<http://www.ucc.ie/law/odg/messages/060222b.htm>

 _Can trespass to land be committed without fault? The answer should be
obvious but I have found it surprisingly difficult to track down. I am
referring, not to cases of involuntary entry onto land (there are clear cases
saying no liability if you get pushed or fall unconscious), but to the sort of
case where you (without carelessness) cross over someone's boundary in the
bush (maybe more likely in Australia than the UK!) without knowing it_

~~~
3pt14159
In Canada there is assumed permission unless no trespassing signs are put up
(in the bush you even have right of way, they can't legally stop you!).

Also, in areas where you are legally allowed to hunt the land owner has to put
up markings that mean "no, I do not give you permission to hunt" if they do
not want you to hunt.

------
yahelc
This is kind of a cool debug interface. Anyone feel like forking it and
putting it on Github?

------
slpollack
I work at AXP and have escalated internally

~~~
0x12
I'll bet you that reporting this on HN is a more effective way than going
through channels.

------
danso
For the longest time, American Express had a password system that only allowed
8 alphanumeric characters and was case-INSENSITIVE.

Moreover, sometimes the AJAX used to submit your payments did not activate,
and often, no feedback at all was given if a payment did go through.

This kind of vulnerability seems par for course for their tech team.

------
gcp
FWIW, on his homepage there's also a nice small vulnerability in reCAPTCHA.
The Google developer who wrote the buggy code actually had to do a hack to
shut up PHP warnings about it. Duuuh...

------
jcromartie
The admin page is still there. Amazing.

~~~
gulbrandr
It seems to be fixed: <https://www.americanexpress.com/us/admin/>

------
gospelwut
google + "Amex security response team" = eirp@aexp.com

also

[http://www.reddit.com/r/netsec/comments/l2uzj/0day_full_disc...](http://www.reddit.com/r/netsec/comments/l2uzj/0day_full_disclosure_american_express/c2pbt39)

~~~
pbhjpbhj
Don't you think then that if you asked their customer services via twitter for
a way to report a security issue that the customer service rep should have
sent that address?

~~~
gospelwut
Should have? Maybe. But, with corporations that size it's unlikely. I'm not
saying this fall entirely on him, but I feel he didn't exactly do his fullest
before puling the trigger on the full disclosure.

------
rdl
I don't consider telephone contact for security vulnerabilities to be that
unreasonable. They should support PGP encrypted email, yes, and have a page
about how to report incidents, issue tracking numbers, etc., but it took me ~3
minutes on the phone to get the right info for Amex corporate security.

------
simon_weber
Unfortunately, I've had this kind of difficulty far too often when reaching
out to large companies with disclosures. Most recently, the only thing that
worked was blasting off an email to all the internal people I could find
through google: the CTO, vp of engineering, and head of support were on the
list, as were a few lower level employees. The lower level got back to me
right away, eager to cc the CTO on their response =)

------
eric-hu
08:39 PST: the page says it's removed for me.

<https://www.americanexpress.com/us/admin/>

------
eykanal
Check out the site now, it looks like this has been fixed. At the very least,
not bad response time on their part once they got wind of it.

------
nyellin
I empathize with the developer, but this disclosure is wildly irresponsible.

It's a pain contacting live representatives at any large corporation. When
you're dealing with the financial industry, you should grit your teeth and
find a way to do it anyway. If you have no choice, publish a warning about the
exploit, but don't release all the details without a long warning period.

~~~
mustpax
No. It's about time we stop letting the financial industry get away with
incompetence. Every other software vendor would be raked over coals for not
having a publicly available security disclosure email address and utterly
failing to properly route a request via Twitter.

Responsible disclosure exists so that vendors have an incentive to respond to
vulnerability reports in a timely manner. In fact, it is the responsible thing
to publicly disclose vulnerabilities so that AmEx learns to implement a proper
security reporting process.

~~~
saucetenuto
No. I agree with almost everything you wrote, but this sort of disclosure
doesn't punish the company, it punishes its _users_, and doesn't give them an
easy way to make the causal connection. Unless this story is picked up by the
mainstream media, how are any victims of this exploit to know that it happened
because AmEx is incompetent, instead of e.g. because credit cards are risky?

------
jrockway
So 90 comments and no mention of "didn't he try emailing
security@americanexpress.com". That would be my first step, not harassing a
marketing account on Twitter. Marketing campaigns are often run by third-party
companies. Whoever gets security@ emails, not so much.

If you want to inflate your ego, post to full-disclosure; don't annoy people
on Twitter and blog about it.

------
mml
Target.com had an almost identical problem on their newly designed site (years
in the making).

------
john_b
Since AMEX caters to wealthier customers you would think that they would be on
top of this kind of thing...

------
funkah
Ugh, it would just be easier to sell the vuln than try to inform one of these
clueless dinosaur companies about it. I know why companies like Amex build
these giant fortresses around their communications, but they should be more
cognizant of the damage that can cause.

------
clistctrl
Wow. This is a huge vulnerability. I hope they fix this very soon. The
cognitive dissonance going on with that twitter conversation makes me think he
was talking to a bot. Also I love the "These cookies are secure" bit on the
admin interface.

------
mkramlich
protip: if you're a bank or credit card company you need top security folks
and procedures. just a thought.

------
fred10
I don't think this is anything dangerous. All the data is static, its just
some sort of demo. It doesn't matter who goes to the page, they will always
get the same data, it never changes. I'm not a customer so can't try once
logged in. If I was to wildly speculate, I'd say honeypot.

~~~
djwelch666
This is dangerous! Someone has left the debug=true in the config somewhere.
Anything could be possible on the site, not just the script injection in the
url and the debug page, but a lot of other stuff as well. When the debug flag
is true on our sites, we have a link which will authenticate us as an admin
without any credentials for example!

~~~
rufibarbatus
> _When the debug flag is true on our sites, we have a link which will
> authenticate us as an admin without any credentials for example!_

Well, get rid of that and push for a change in your company's workflow. This
kind of control shouldn't be deployable to the main servers _at all._

Have separate, staging servers and run your tests and debugging interfaces on
it, but as much as possible, don't deploy administrator interfaces to the
servers that talk to the customer. [1]

[1] I'm undecided which kinds of heisenbugs would justify breaking that lemma.

