
I was a senior VP of tech at Starwood: here’s my take on the guest data breach - valiant-comma
https://www.phocuswire.com/Marriott-data-breach-ex-Starwood-perspective
======
abhorrence
This article seems full of points that a lay person might nod along with, yet
don't hold up to scrutiny.

> The fact is, if we accept Marriott’s statement that the breach began in
> 2014, the system would already have been operating securely for five years.

It does not mean that. It means that we don't know of any exploited
vulnerabilities before that point.

> If the detection tool was used prior to this September, why hadn’t the
> breach been detected earlier? And if the tool was not used earlier, how can
> they be so sure the breach occurred in 2014?

It isn't unthinkable that the new tool alerted them to a problem, and during
investigation discovered evidence that the vulnerability had been abused in
the past.

> It is almost impossible to imagine a scenario in which an external hacker is
> able to gain access to the primary encryption keys.

Why? The argument seems to be: the primary encryption key is important, and
thus will be most carefully guarded, so it is unthinkable that it would
actually be exposed.

Ultimately the article strikes me as an article written by someone who has a
beef with Marriott, and he ends noting that it's possible that the breach
occurred not due to issues with design, but due to the layoffs of Starwood's
technical staff.

~~~
heartbreak
> Ultimately the article strikes me as an article written by someone who has a
> beef with Marriott, and he ends noting that it's possible that the breach
> occurred not due to issues with design, but due to the layoffs of Starwood's
> technical staff.

I agree with your first several points, but a lay-off beef is unlikely since
the author hasn’t worked for Starwood in over a decade.

~~~
ec109685
Marriot replaced his system at Starwood with theirs (and he didn’t agree with
that) so his technology was laid off :)

------
kenneth
I'm amongst the most frequent guests at Starwood, spending >100 nights a year
in their hotels. I wasn't thrilled when it was announced they'd be acquired by
Marriott. This year, they began the switchover process to migrating to
Marriott's technology, and the full switch officially happened in mid-August.

It was a complete and utter disaster.

Everything was buggy, points mysteriously disappeared, reservations
disappeared. Inconsistent UI, a mix of old and new systems. A truly awful
experience dealing with support agents who were incapable of comprehending
what was happening. I'm still waiting for a handful of stays to be credited to
my account months later and nobody can help me because the systems are broken.

I found myself staying mostly at Hyatt hotels while the dust settled. I'll end
the year with another 100 nights with Starwood/Marriott, and 80 with Hyatt.
But, given the direction the company has taken since the merger, that number
will likely be going down on the Marriott side.

After hearing that Marriott laid off the majority of Starwood's technical
staff before attempting this migration, I'm not surprised it went this way.
I'm also very much inclined to believe that the data breach happened during
this migration.

~~~
verst
Agreed that their migration from SPG to Marriott has been painful. It ended up
creating 3 new logins for me before finally consolidating everything under a
new number.

Out of curiosity, do you find your Marriott / SPG Ambassador to be useful? I
reached Platinum Premier Elite with Ambassador status in November but my
ambassador hasn't been helpful at all. The Your24 perk also rarely works in
practice. It's a nice marketing gimmick, but definitely not worth it so far.

~~~
kenneth
Ambassador status is highly dependent on who you get. Overall, my ambassador
is a nice guy who's professional, friendly, and helpful. I also really
appreciate having a dedicated line of support considering how awful Marriott's
customer support otherwise are. They're organized a few upgrades for me when
they really mattered, and are my main point of contact for support, especially
for some pretty complex issues which would've been much more painful through
traditional channels. Beyond that, I can't say it's made that meaningful of a
difference.

The quality of the service has gone markedly down since the August
integration. My understanding is that ambassadors have been bogged down with
technical issues, an influx of new customers (going from 50 guests per
ambassador to >300), and a big drop in morale. A lot of the issues they can't
help with aren't their fault, but rather problems with Marriott policies and
technology.

And oh: on Your24, I've only tried to use it a couple times, and had it work
about 50% of the time. I've checked in before 3pm many times and pretty much
always get a room. (Except one frustrating time after a red-eye where I needed
a shower desperately and the hotel had no rooms available or any showers in
the gym. I was not a happy camper.)

------
Aloha
The article he wrote about Marriott's choice to continue using its z/TPF based
platform over migrating to Starwood is also telling.

[https://www.linkedin.com/pulse/marriottstarwood-back-
future-...](https://www.linkedin.com/pulse/marriottstarwood-back-future-
technology-decision-israel-del-rio/)

See this quote "To better understand the resulting Starwood’s technology
compared to industry legacy systems, think Tesla Model S versus a gas-guzzling
1975 Buick Electra.

Then along came Marriott . . .

When Marriott announced its interest in acquiring Starwood, one would have
believed that they factored in a $500 million Starwood IP technology value
within their $13.6 Billion offer, and that they would have been salivating at
the prospect of having their hands on the fruits of the multi-year
transformation experience this IP represented. After all, while stable as a
rock, Marriott’s own system today centers around 1970’s Mainframe TPF
technology (MARSHA) suitably kept current via the judicious use of the scotch-
tape and wires represented by a cornucopia of front-end gateways and the labor
intense support of inflexible legacy code, eclectic data bases, hard-coded
interfaces, and a veritable zoo of different property management systems
crying for better integration. "

It reads as sour-grapes to me.

If you wanna read more about MARSHA - this seems to be a good source:
[http://ibmsystemsmag.com/mainframe/casestudies/miscellaneous...](http://ibmsystemsmag.com/mainframe/casestudies/miscellaneous/marriott_agility/?page=1)

------
kcorbitt
> if [...] the breach began in 2014, the system would already have been
> operating securely for five years. It is difficult to imagine how an
> architectural or platform vulnerability would not have been discovered or
> exploited sooner.

I mean, that seems very easy to imagine? Just last year Wannacry exposed an
RCE exploit in Windows that has been present since at least Windows XP
([https://docs.microsoft.com/en-us/security-
updates/securitybu...](https://docs.microsoft.com/en-us/security-
updates/securitybulletins/2017/ms17-010)). And there are orders of magnitude
more people looking for exploits in Windows than Marriott's internal systems.
I don't find this article particularly credible.

------
lawnchair_larry
This guy appears to have no clue what he is talking about, and is painfully
ignorant of both security and technology.

I realize that’s not a very substantive comment, but wow.

 _”The Valhalla system was fully activated in 2009, and my understanding is
that all best practices were followed in its design (firewalls, DMZs,
encryption, etc.).”_

 _”It is difficult to imagine how an architectural or platform vulnerability
would not have been discovered or exploited sooner.”_

 _”It is almost impossible to imagine a scenario in which an external hacker
is able to gain access to the primary encryption keys.”_

This is just painful to read from someone so senior on their soapbox. It’s
probably exactly why they got hacked. Also note that this isn’t the first
major starwood breach:
[https://www.starwoodhotels.com/html/HTML_Blocks/Corporate/Co...](https://www.starwoodhotels.com/html/HTML_Blocks/Corporate/Confidential/Letter.htm)

~~~
adrr
You don't get to SVP of a big corporation by being an expert in technology or
security. That high up, your skill set is on organization, process,
prioritization/planning, and budgeting. Sometimes you do see technical CTOs.

------
packetized
This is an exceptionally self-serving take on the matter at hand. So much so
that it’s frankly breathtaking that it’s been upvoted to #1.

Dear Israel del Rio,

As a Mariott and SPG member since history, kindly focus on not disclaiming
responsibility in a public forum, since you almost assuredly aren’t as
innocent as you claim.

------
agotterer
> It is almost impossible to imagine a scenario in which an external hacker is
> able to gain access to the primary encryption keys.

I was reasonably sold on what was being said until that comment. Impossible is
a strong word to use when it comes to computer security. It seems that
everyone who has claimed that there system is unhackable, always ends up being
hacked.

~~~
scarcely
He said 'almost impossible'. Big difference.

~~~
hawkice
He said almost impossible to imagine. It really was an extremely hyperbolic
statement. I've never done security work but I've helped develop plans for
when primary and secondary keys are stolen. It's something that should have
been imagined, not "almost impossible to imagine".

------
Cyclone_
"The fact is, if we accept Marriott’s statement that the breach began in 2014,
the system would already have been operating securely for five years.

It is difficult to imagine how an architectural or platform vulnerability
would not have been discovered or exploited sooner."

Not really. There's been vulnerabilities that have been out in the wild for
quite some time and took years to be found. Sometimes it just comes down to
luck/what people are trying to exploit.

~~~
brabara
Don't disagree, but how much more insecure is Marriott to say, Motel 8?

------
nazca
Worth pointing out that at the time of this guy's tenure, and for many years
afterwards, the way you authenticated yourself while booking a rewards
reservation with SPG via the phone was to verbally tell the agent your online
password. Like, WTF.

~~~
rdl
The SPG password for phone has been a different password from the web login
password for as long as I can remember (2014?)

~~~
nazca
Yes, I remember them being the same from 2005-ish through to about 2011-2012
ish. This guy was at SPG until 2006.

------
bogomipz
This person has not worked at Starwood in 12 years. Their qualifications for
making any kind of insightful analysis are basically nil at this point. As if
this weren't weren't absurd enough he goes on to make the following laughable
statements:

>"It is almost impossible to imagine a scenario in which an external hacker is
able to gain access to the primary encryption keys."

>"The fact is, if we accept Marriott’s statement that the breach began in
2014, the system would already have been operating securely for five years."

>"Israel del Rio is executive technology consultant and CTO at Quilmach."

As someone who has family affected by this breach it's upsetting to see to see
this individual using this incident for their own self-promotion. However at
least his new company Quilmach now knows he is completely clueless about
technology. So I guess he did everyone a favor here. Israel del Rio -
Executive Idiot.

------
dbt00
“It would be irresponsible to speculate at this time, unless I can point the
finger at systems I wasn’t personally responsible for.”

------
Spooky23
There’s no meaningful security in travel companies. They share with everyone
and controls are a joke.

Hell, Hilton allowed for 4-digit numeric passwords until a few years ago.

~~~
rdl
IHG (Holiday Inn, InterContinental) is still a 4 digit PIN and a numeric (or
email) userid.

------
tdb7893
He seemed to say the database wouldn't have 500 million records in it at a
time since they are deleted but that seems irrelevant with how the breach took
place over 4 years. Anyway the article seems to be just speculation, which is
disappointing.

Edit: this article has gotten a lot more upvotes than I would expect if
something this quality, is there something about it I'm missing that makes it
particularly insightful?

~~~
codycraven
This whole article reads to me like an SVP who doesn't actually understand
technology and security (which is definitely not uncommon).

------
docker_up
This is a masterclass of CYA. He takes a lot of effort to try to prove that
the system he was in charge of wasn't the cause, even though his arguments are
absurd ("if we accept Marriott’s statement that the breach began in 2014, the
system would already have been operating securely for five years." It was
operating for 5 years but there's nothing to prove that it was operating
securely for 5 years.) He also handwaves and leads the readers down some
detective story ("ergo it must have been the data warehouse!") and says "We
really don't know if it was Valhalla and may never know!"

No wonder he's an executive, he is an expert CYA-ers!

------
arduanika
Also see extensive discussion of Marriott/Starwood situation within
[https://news.ycombinator.com/item?id=18651676](https://news.ycombinator.com/item?id=18651676)

------
busterarm
There's a lot of sour-grapes among Starwood employees in the lead-up to and
after the merger. A lot of dedicated middle management folks got forced out
and a lot of managers in well-performing hotels were forced to move. None of
the grunt employees seemed happy with it on either side.

------
neil_s
This is a surprisingly poor understanding of tech. Do hotels hand out titles
like investment banks (there were hundreds of VPs at JP Morgan), or was this
author actually in a position of responsibility? Is it common for non-tech
companies to have such people in high up places?

~~~
laurentl
2 factors I think:

* the CTO title is the new project manager. It seems as soon as someone is in a position to choose whether to use Postgre or MySql they break out the CTO moniker. You can now be the CTO of just about any organizational unit, no matter how small.

* CTO != CISO (quite the contrary usually). Being tech-savy doesn't mean you grok security, especially its operational, wetware/social aspects. Obligatory xkcd: [https://www.xkcd.com/538/](https://www.xkcd.com/538/)

I would tend to go with the second explanation in this case. When I read the
quote "It is almost impossible to imagine a scenario in which an external
hacker is able to gain access to the primary encryption keys", I thought it
pointed to a lack of imagination (and probably a bit of Dunning-Kruger
syndrome) rather than security expertise.

------
licebmi__at__
> Still, most commonly, breaches occur when someone obtains an administrative
> password via deceitful means (e.g., phishing attacks), enabling them to log
> into the system and install Trojan software to extract data or to manipulate
> the system.

> This is the method the Russians used to hack into the Democratic National
> Committee emails, for example.

AFAIK, phishing was the main attack for Podesta's emails, but I'm not aware
that this was used on the DNC hack. I think the author is mixing scenarios.

