
Gone in Six Characters: Short URLs Considered Harmful for Cloud Services - ajdlinux
https://freedom-to-tinker.com/blog/vitaly/gone-in-six-characters-short-urls-considered-harmful-for-cloud-services/
======
jiiam
Also notice the different attitudes of affected services:

\- OneDrive "[...] reiterated that the issues we discovered do not qualify as
a security vulnerability"

\- Google Maps "[...] responded immediately. All newly generated goo.gl/maps
URLs have 11- or 12-character tokens, and Google deployed defenses to limit
the scanning of the existing URLs."

Well done, Microsoft!

~~~
patcheudor
The fact that anyone considers a shortened URL as a means to secure a piece of
data outside of authentication saddens me. I think that should be the key
point here. Did you just get a URL that gives you a bit of data without the
need to authenticate? Then it's not secure.

~~~
matt4077
How is a URL with that's not linked anywhere different from a password?

~~~
jakobegger
It is not. The problem is that short URLs are often only six characters long,
so they are just as safe as a six character password (ie. not very safe)

~~~
patcheudor
Far less safe because you use a password in combination with a userID so to
crack someone's account you must have their login name and password. Six
characters are six characters, easy.

------
d--b
This is breathtaking. This article is not only very important for the
vulnerability it uncovers, but also it is one of very few articles that shows
with very specific examples why breaches of privacy do matter.

Even if you 'don't have anything to hide', you don't want anyone to know that
you sent someone the directions to a planned parenthood center. Not because
you think it's a bad thing to do, but because the publicity of this
information could be harmful to you.

~~~
theseatoms
I mean, doesn't that count as 'having something to hide'? I think we're in
agreement though. As has been rehashed on HN many times, it's a bogus
argument.

~~~
IanCal
That's probably quite technically correct, although I think the wording "hide"
implies you've done something wrong.

"If you've got nothing to protect..." definitely has a different feeling to
it.

~~~
theseatoms
Absolutely. That's a much fairer phrasing of the 'nothing to hide' argument.

Next time someone presents the 'if you have nothing to hide argument', tell
them to send you a video of each of their family members on the toilet.

~~~
fapjacks
Right! The ole toilet argument seems to win over everybody. I was in the Army
for a long time, and right after 9/11 during our pre-mobilization for our
first deployment, we mobilized out of Fort Stewart, in Georgia. We were put up
in these awful billets from WW2. Just awful. Anyways, the bathrooms only had
rows of toilets immediately behind rows of sinks. So if you went to shit in
the morning (literally knee-to-knee with the guy shitting next to you),
someone's ass was in your face as they brushed their teeth. It was at once one
of the most enlightening moments of my life about the cultural concept of
privacy that we have these days. We have an expectation of privacy that
obviously they did not enjoy in days of yore. And we have _got_ to fight to
keep it.

------
barrkel
Sharing a link with someone via email or chat - a private channel - suddenly
becomes a share in a public channel because of the lack of entropy in the
shortened link.

Even more surprising is the number of people on here who don't understand why
this is problematic, essentially blaming the victim for not understanding that
their private channel is leaking information. It certainly is not obvious to
the general public, and wasn't obvious to the people who implemented these
services, that a side-effect of a shortener with insufficient entropy is
leaking information from private channels.

~~~
Pxtl
Yup. Even if Google and MS disabled their built-in shorteners or made them
not-actually-short, if I send a URL over a Twitter DM, it will get
shortened... and I doubt that twitter's url-shortening forwarder will restrict
access to the url to sender and recipient.

Fundamentally, long-urls-as-security makes mathematical sense but doesn't
actually work with the way the web behaves. URLs are shared constantly,
freely.

If we had everybody using the same auth systems so we could do whitelists
without "oh you need a google account" or "oh you need an MS account" then
whitelists would be a solition, but In Real Life whitelists get in the way so
instead people just rip out the security altogether and count on the security
of the URL.

Imho, public-with-shared-password would be the right feature to add, even
though it's anachronistic.

~~~
barrkel
Is it reasonable to expect driving directions to be password protected? That
example is used in the article and I think it's an excellent test case for
thinking about alternative approaches.

Driving directions by themselves are usually pretty innocuous. When one end is
something like an abortion clinic, it gets a bit more sensitive; when the
other end is a specific house, even more so. And I don't think you can expect
driving directions to ever get password protected.

Some links are like words - merely a reference; some are like sentences -
describing some relation or fact; and some are like passwords - describing how
to find or access something that you wouldn't otherwise know.

The first are not revealing when out of context; the last are always dubious.
It's the middle case, describing a relation (like driving directions) that's
most problematic.

~~~
Pxtl
It would be reasonable to offer the option when you hit the "share" button. I
would assume people who care about privacy but are sharing with non-Google
people would like the ability to add a shared password to the map.

------
stephenr
I don't understand the use of URL shorteners for 99% of what people use them
for.

Unless you're sending your link over a relatively short fixed-length limited
medium like Twitter or SMS, there is no fucking point.

I've seen people post links to download apps, which go from their own site >
some random bit.ly/etc URL > dropbox. I'm already seriously doubting if I want
to run your app if you can't manage something better than dropbox file
sharing, but to then rely on a bit.ly URL that could go fucking anywhere, when
you're just putting the link on a webpage is beyond belief.

~~~
gedrap
1) Let's say I am doing a presentation, the computer has internet connection
but can't use USB drives, etc. So I would shorten the url (to, e.g., google
drive file) and write down the token. Much better than logging into personal
account on shared machine.

2) Analytics. I've seen many times different shortened urls used in different
locations for the same campaign. I assume that makes it much easier to track
your offline marketing channels.

~~~
soared
Analytics Analytics Analytics. Shortened urls automatically have very decent
analytics making it WAY easier than setting up UTM params or other campaign
identifies.

Add a "+" to any bitly or goo.gl link and see what I mean.
[https://goo.gl/forms/9fA366pQ1f+](https://goo.gl/forms/9fA366pQ1f+)

I use goo.gl to see how many people click a link I share on facebook too.

~~~
jessaustin
Wow that seems like information that someone might occasionally want to keep
private.

~~~
soared
Yeah I agree. Anytime I see a bitly or goo.gl I check the stats just for fun.
I've seen some other shorteners that track other stuff, one even had ip
addresses!

------
simonw
The ability to traverse the full content of a OneDrive account starting with a
short URL and in some cases /upload malware to them/ which gets synced back to
the user's computers is shocking. Even more shocking is that Microsoft
apparently declared this to be as designed, not a security bug. That's some
terrible software design.

~~~
rileymat2
As a developer who helped out with QA one of the most annoying things was
reporting bugs and then getting into a debate about wether or not it was a bug
as it was working as designed with the functionality was clearly
terrible/broken.

~~~
specialist
It's like pushing rope. Then to really get the party started, throw in some
"business analysts" and "scrum masters".

------
AndyMcConachie
If your security depends on someone not walking your DNS zone, you're doing
something wrong.

If your security depends on someone not guessing a URL, shortened or not,
you're doing something wrong.

~~~
morgante
> If your security depends on someone not guessing a URL, shortened or not,
> you're doing something wrong.

Why? Done properly, a secure URL should be as long and contain as much entropy
as a strong password.

Are all systems which depend on someone not guessing your strong password
wrong? Are practically all encryption schemes wrong?

~~~
danielweber
URLs leak in a bunch of ways. Cached in browsers and proxies, sent in
referers, sent in toolbar requests, logged in access.log files.

I'm not saying that it's necessarily wrong to use a long URL as security;
sometimes the problem constraint means that is the only way to do it. I've
done it on rare occasion. But I also made sure that management knew there was
a risk here.

~~~
rkangel
If that URL is being sent over an HTTPS link (which of course it should be)
then the only two points that can log/cache it are the browser at the local
end, and the server at the remote end.

The latter is up to server design, the former is an interesting point.

Browsers are unlikely to cache get parameters, but these things might end up
in history etc..

~~~
jessaustin
The "Referer" header can also leak URLs.

------
stevetrewick
Scary, but isn't really more like 'using a trivially computable string as a
trampoline for live authorisation tokens considered harmful for supposedly
secured cloud services'.

Not as catchy, I admit.

------
neogodless
I think short URLs are great for Twitter, sending people to public URLs, for
other services where you're literally _just shortening a public URL_ , if they
want to include a tracking / redirect to harvest all your juicy habit
information - but I don't think it's a great thing for _private_ URLs.

~~~
TheSmiddy
Twitter considers every link, regardless of length, as 23 characters so
they're not even that useful there.

~~~
phn
I think that's precisely because they are always shortened automatically on
twitter's side.

------
jcrawfordor
A long time ago, perhaps 6-7 years, I used an OS X app that took screenshots
and put a link on the clipboard. I noticed that it used very short paths and
they appeared to be sequential, so in a moment of boredom I made a little PHP
script that just gave you next/previous buttons to iterate through them. It
was amusing and I considered actually scraping them and then trying to OCR for
sensitive information or something out of curiosity, but I never got around to
it (pity, I could have scooped this article!).

Well, fast forward about two years, and that script is still sitting around on
a forlorn webserver of mine. Somehow, I have no idea why, some random person
ended up tweeting a link to it and it spread around a bit until the software
vendor got wind of it. They ended up sending me a probably too-polite email
asking if I could do something about it, and after a bit of back-and-forth I
got instructions from them on how to enable more secure "long URLs" in the
software (an option that I think was new since I made it, so I wonder if I may
have actually inspired it...) and added those to the bottom of the page.

It's long gone now, and to be honest I can't remember which app was affected.
Possibly tinygrab.

The point of this anecdote is that the problem is not at all new, and the
problem of how to deal with it isn't new either. I suggested to the developer
at the time that they should probably use long URLs by default, but it seems
users just like those short URLs too much. Going to non-sequential assignment
would have helped, but the space was still just too small.

Really, I think the fix is just communication. Microsoft's workflow used to be
sensible in that OneDrive gave you a long URL and then you had to click
another button to get a short one. That second click should come with a
warning that there should be no sensitive information in the document, and it
will potentially become public after shortening. Users will have to be trusted
with the judgment, at least you've CYAd.

------
ryanswapp
Speaking of short urls, am I the only one that refuses to click on them? I
hate that I can't see where the link is going to take me.

~~~
soared
I've never heard this before. What would be an example of somewhere you don't
want to go? I've never clicked a shortened URL and had unexpected results.

~~~
ryanswapp
Most of the time it happens on Twitter. Someone will tweet a bitly link with
some enticing headline but I refuse to click it because I don't know where the
link is taking me. Perhaps I'm just overly cautious...

------
spullara
I pointed this out when I was at Twitter when they were still using the normal
short t.co URLs inside DMs. We quickly switch to using very long tokens for
those generated URLs. To me it seems completely obvious and I struggle with
the developer that stored private information in something so eminently
scannable.

------
pdkl95
When you use a URL shortener, you are effectively encrypting the URL and
telling someone they have to go to some 3rd party to get the plaintext.
Without any checksum to verify that the 3rd party didn't send an incorrect
URL, either maliciously or by accident.

------
Sleaker
Well... The article is a little dis-engenuous about the shared folder stuff.
If a user selected to share the folders with anyone that has the link and also
allows write into the folder publicly from anyone, then that's by design.
Obscurity on the url part isn't necessarily required, and it may even be a
feature to allow easy dumping.. This is on the end user to make sure they
aren't auto-downloading public data that has been dumped there.. I can see why
this may not be ideal from a security standpoint, and allowing data
mining/unauthenticated file drops may not be a great way to handle it, but I
don't think the article actually gives the full details. Unless I'm completely
wrong, and there is no options in OneDrive for sharing permissions (public,
select group, etc) then yes it's a security vulnerability.

------
whatever_dude
"...For Cloud Services" doesn't seem like an appropriate title to me. I would
say it's more like "For Cloud Service _Users_ ".

------
jessegreathouse
This is an article about how people are using url shorteners for the wrong
reasons and/or not using security on private data.

~~~
maxerickson
It shouldn't be easy to misuse a shortener that is built into a service like
OneDrive.

~~~
jessegreathouse
Until today I had no idea what OneDrive is, so I'll take your word for it. I
would suggest that the problem is that access to the OneDrive resource starts
and stops at the URL. Conventional security protocols are conventional for a
reason.

------
downandout
_" After an email exchange that lasted over two months, “Brian” informed us on
August 1, 2015, that the ability to share documents via short URLs “appears by
design” and “does not currently warrant an MSRC case.”_

What is it with these large companies ignoring serious security issues while
paying attention to smaller ones? I reported something to Facebook that was a
moderate privacy concern and got a bug bounty. A few months later, I
discovered that I could make Facebook falsely report the domain that a posted
URL goes to, and they denied that it was even a bug. So I could share a URL on
mydomain.com, customize the contents of the share posting ("Obama says he's
going to nuke Russia"), and Facebook would show users in the post that the
link goes to Whitehouse.gov or CNN.com or any other domain I choose. This
still works perfectly.

These companies really need to take a look at the analytical abilities of
those they are employing to screen bug reports.

~~~
cyber
One also needs to take into account how these larger companies' internal
groups function.

"Brian" probably looked into it, knowing that obscurity != security, but got a
response back from the group responsible that was the way they intended it to
work, and that group's management wasn't going to do anything about it.
"Brian" may have even put messages in the right ear up the management chain
such that it would actually effect the outcome.

The fact that the email exchange lasted 2 months before "Brian" said "Sorry,
not a case." probably means that "Brian" was trying to make it happen and had
actually done an analysis.

* Note: I'm NOT Brian. I've never worked for Microsoft.

~~~
cmdrfred
Probably some metric says that if "Bob" gets less than X cases per year he
gets a bonus. Problem is "Bob" determines "Frank's" wage and "Frank" is
"Brian's" boss. It is probably more complicated than that but that's what it
always boils down to.

------
Animats
OneDrive _publicly writeable_? Why is that even possible?

------
tedmiston
IMO the author is conflating two separate "issues".

> TL;DR: short URLs produced by bit.ly, goo.gl, and similar services are so
> short that they can be scanned by brute force.

This is not the issue for OneDrive. Everyone knew this already, right?

For Google Maps, it's definitely more nuanced. I'm glad Google acted swiftly.

> Our scan discovered a large number of Microsoft OneDrive accounts with
> private documents. Many of these accounts are unlocked and allow anyone to
> inject malware that will be automatically downloaded to users’ devices.

This is the issue for OneDrive. I'm not a OneDrive user, but if the documents
are publicly editable per a setting the user controls, this isn't a
"vulnerability" either.

~~~
Normal_gaussian
> if the documents are publicly editable per a setting the user controls, this
> isn't a "vulnerability" either.

These services advertise it as "editable by anybody with the link" not
"editable by anybody"

There is an implication that people can't get the link without you giving it
to them.

------
bognition
Honestly this title feels a bit like FUD. Sure restricting the space of
possible URLs decreases the difficulty of brute forcing urls, but honestly if
you don't want something publicly accessible put it behind a auth wall.

~~~
tremon
Yes, the title does. But let's take a look at the actual content:

 _OneDrive generates short URLs for documents and folders using [..] the same
tokens as bit.ly. [..] In our sample scan of 100,000,000 bit.ly URLs with
randomly chosen 6-character tokens [..] 19,524 URLs lead to OneDrive /SkyDrive
files and folders, most of them live. [..] From the URL to a single shared
document (“seed”), one can construct the root URL [from which] it is easy to
automatically discover URLs of other shared files and folders in the account_

In other words, the links provided by these shorteners contain authorization
information. And:

 _Around 7% of the OneDrive folders discovered in this fashion allow writing._

And in the case of Google Maps:

 _goo.gl /maps URLs used 5-character tokens. Our sample random scan of these
URLs yielded 23,965,718 live links, of which 10% were for maps with driving
directions. These include directions to and from many sensitive locations:
clinics for specific diseases (including cancer and mental diseases),
addiction treatment centers, abortion providers, correctional and juvenile
detention facilities, payday and car-title lenders, gentlemen’s clubs, etc.
The endpoints of driving directions often contain enough information (e.g.,
addresses of single-family residences) to uniquely identify the individuals
who requested the directions_

~~~
blowski
Thanks. This was a good summary of the problem.

Do Google et al not have any kind of rate limiting which looks at suspicious
behaviour like scanning lots of short URLs?

~~~
abalos
Did you read the article? At the bottom it said that they have both increased
the key size to 11 or 12 characters and deployed methods for preventing the
brute forcing of these URL's. I think that it's safe to assume that one of
these methods is rate limiting.

~~~
blowski
Thanks. I asked specifically about rate limiting as the article didn't
specifically mention it.

------
nxzero
To me, this is like saying Base64 encoding is dangerous; sure, if you think
Base64 is encryption and are using it to store passwords, please stop.

Almost all tech can be used in the wrong way, this does not make the tech bad
if use correctly.

~~~
Scarblac
The story is that products as huge as OneDrive and Google Maps use it in
terrible ways.

------
Grue3
Seems like the problem is not with the URL shorteners, but with OneDrive
braindead security model. Somehow having a link to one file allows the
attacker to see all the user's files? What were they thinking?

------
0xf005ba11
Instead of trying to make brute force more costly, couldn't these services
make it impossible, by forcing the fulfilment of a CAPTCHA when trying to
expand/follow a shortened URL?

------
based2
[http://arxiv.org/pdf/1604.02734v1.pdf](http://arxiv.org/pdf/1604.02734v1.pdf)

[http://www.icir.org/vern/papers/monarch-
oak11.pdf](http://www.icir.org/vern/papers/monarch-oak11.pdf)

------
aokyler
I agree this could become a big issue - but I wouldn't consider it a "security
vulnerability" per se.

URLs aren't secure, and shouldn't really be considered so.

~~~
csydas
I think it's less about the URL itself and more about the services which
automatically generate them for content without the user being fully cognizant
of what that implies/means. It has the potential to publicize information
without the user's intent or knowledge; in the case of OneDrive from the
article, it exposed documents with sensitive information when the URL itself
wasn't even shared, it was just brute-forced. Prior to Google's changes to the
URL shortening, it sounds like it was possible to get quite a bit of personal
identifying information just by guessing at the shortened URL, even if the URL
itself was never shared.

Even if URL shortening is a feature that users are aware exists, the
consequences of it certainly aren't immediately clear, and to my knowledge,
not many of these services include a way to disable the generation of a
shortened link or have a means to prevent this type of scanning from
happening.

------
e0m
This probably can be significantly helped from a UI design perspective. As
more and more services auto-shrink links for readability there may not be as
much of a compelling need to shorten links. Why in 2016 must we keep
displaying everything as "raw text". This is a perfect example of the power of
richer displays.

------
dorfsmay
As a very late adopter of twitter, I was shock when I first run into shortened
URL, not understanding their values and seeing only the risks.

Interestingly, a lot of email client also replace links by their own creating
similar risks, but nobody talks about those...

I have to applaud Slack for displaying and linking to links the they were
meant to.

------
neil_s
What's more worrying to me than the enumerating of all short URLs, is the
directory traversal when you know one URL. This is someone I know, and have
specifically shared one file with, being able to see ALL my other documents.
Glad to see that's gone now.

------
rkeene2
My URL shortener makes the user come up with their own short URL so they can
decide how long to make it.

[http://oc9.org/<insertYourShortURLHere>](http://oc9.org/<insertYourShortURLHere>)

~~~
slig
It's possible to create a redirect loop, i.e, a URL that redirects to itself.
I don't know if it's something that should happen or not, but Chrome gives me
an error after too many redirects.

------
finnn
Is anyone else getting the following JSON for all requests to 1drv.ms?

{"error":{"code":"generalException","message":"General Exception While
Processing"}}

------
elwell
One solution [0]: Password Protected URL Shortener
[http://thinfi.com/](http://thinfi.com/)

[0] - at the cost of usability

------
wickedlogic
I'd still like to see delete added goo.gl

------
xpda
It's hard to believe so many people consider the use of shortened URLs a
security measure. It is not, and was never intended to be. A URL is exposed,
by definition, whether long or shortened. A shortened URL is a convenience,
not a security tool. Some people misuse base64 encoding for "security" as
well, but it does not mean we should get rid of base64 encoding.

~~~
NelsonMinar
URLs aren't secrets.

~~~
pjc50
Why not, if you've tried to keep it secret? They're an incredibly common means
of giving out things to a limited set of people without having to explicitly
authenticate them. They're used by a lot of mailing lists for e.g. unsubscribe
links. If there's enough entropy and they're once-only it's a reasonable
approach.

Otherwise you end up with people just sending around "go to this url with this
login and password" through email and chat instead, which is slightly harder
to automatically exploit but not really more secure.

------
cyc115
simple script to brute force goog.gl urls for fun. :)
[https://gist.github.com/cyc115/f22db26de6a5d723ef6094a97f0ed...](https://gist.github.com/cyc115/f22db26de6a5d723ef6094a97f0edc6d)

------
nilved
This vuln probably also exists on imgur

~~~
btym
Yeah, imgur's predictable too. Go look up "imgur roulette".

~~~
tedmiston
Well the URLs are predictable...

But I don't think you can find out one's imgur username or more of their
photos from just having a link to one of their photos.

I also tested from an arbitrary image in an album, trying to go backwards to
find out what album it belongs to, but that doesn't seem possible either.

------
makecheck
There is nothing inherently wrong with a short URL, provided that supporting
infrastructure has proper security.

Even now, if you connect to your favorite “trusted” long domain names, nothing
stops that from being totally hijacked by an untrustworthy Internet service
provider or other entity that has access (or more insidiously, inserting crap
like ads that were not in the original source).

And heck, long URLs are suspicious as well. I’m sure by now everyone has
received one of those ridiculous
"important@facebook.com.kdsjfksdjfkdsfjdskfjdskfjdskfjdskfjdskfjdskfjdskfs.oopsmalware.com".

The push should be for broader adoption of mechanisms that make it hard to
subvert what you download, and easy to verify what will happen when you click
a link.

------
athenot
This article is more interesting from a UX point of view. It seems that many
people think the gibberish-ly looking URL, however short, is nice and safe.
That confers a false sense of privacy.

The part about automatically-generated short URLs in MS docs is (was?)
worriesome. Few users understand the implication of having a public URL that
directly references their document which, in all likelihood, were intended for
a restricted audience.

I should point out that Slack has the same bug, though their URLs are simply
obfuscated with a longer token. Shameless plug: Cisco Spark[0] has solved that
with end-to-end encryption.

[0]
[https://support.ciscospark.com/customer/en/portal/articles/1...](https://support.ciscospark.com/customer/en/portal/articles/1339052-how-
secure-is-the-cisco-spark-app-?b_id=8722)

------
Pxtl
Sounds like the web-office tools need a "partially public" option. I mean, I
want to do whatever, whenever to my private documents, but when I open a
document for public editing because my collaborators don't have a google
account and so I just share out a long-URL with them, the expectation is that
the worst thing that coudl happen is that a malicious actor that finds the URL
could mangle the document and I'd have to revert it.

Giving my collaborators enough power to inject executables is far beyond my
needs and my intent when I make the doc "open". At worst I'd expect to find a
document edited with a link to an external malware exe, not some horrifying
autorun problem.

You could also do warnings when the user clicks a link ina publicly editable
doc "this document is publicly editable, which means that any rando on the
internet might have set this link, not just your buddy who made the doc. Are
you really sure you want to go there?"

