
Show HN: I built a FOSS service to protect against link shorteners tracking us - l1am0
https://unshort.link/
======
myself248
Related, the Archive Team realized a while ago, that when a link shortening
service goes dark, all the links that went through it instantly break. This
has bad implications for future historians trying to figure out how the
present-day web was interlinked.

So there's a project called URLteam to index and save them. I just started
running a Warrior instance on the URLteam project, and it's really
straightforward. I figure folks reading this may be interested. More info:

[https://www.archiveteam.org/index.php?title=URLTeam](https://www.archiveteam.org/index.php?title=URLTeam)

[https://www.archiveteam.org/index.php?title=ArchiveTeam_Warr...](https://www.archiveteam.org/index.php?title=ArchiveTeam_Warrior)

------
Wowfunhappy
Where and why are you guys still seeing/using shortened URLs? I don't think
I've personally made a shortened URL since Twitter built one into their
service, and I stopped _seeing_ shortened URLs around the same time.

There are two exceptions:

• I still see service-specific short URLs, e.g. goo.gl, nyti.ms, etc. Because
these can only be created by the service in question, they don't introduce
security/privacy concerns.

• When a URL shortener is used to create a sort of vanity link to e.g. a
Google doc. (OP's service could be useful for these!)

Outside of those times... why would you use a URL shortener? When are you
limited by number of characters?

~~~
kick
_• I still see service-specific short URLs, e.g. goo.gl, nyti.ms, etc. Because
these can only be created by the service in question, they don 't introduce
security/privacy concerns._

This isn't necessarily true; I've seen companies that offer to do the 'custom
shortlink' thing for various companies. It's safe to assume that there are
privacy concerns with at least some of them.

~~~
tyingq
Bitly runs nyti.ms, so yes, you aren't trusting just the NYT.

~~~
edoceo
Yes, and nyti.ms only ever pints to New York Times, a well known entity (well,
that's what's intended, right?)

~~~
tyingq
Sure. Noting that you have to trust that bitly isn't fingerprinting your
browser and tracking you across their domains.

------
iudqnolq
Reminds me when security researchers used data Bit.ly exposes to go from one
url shortened by Russian military intelligence to the hundreds of other short
URLs they made for phishing emails, many of which had state embedded in query
parameters that gave away who they were targeting. All because the GRU forgot
to set their accounts to private.

[https://www.vice.com/en_us/article/mg7xjb/how-hackers-
broke-...](https://www.vice.com/en_us/article/mg7xjb/how-hackers-broke-into-
john-podesta-and-colin-powells-gmail-accounts)

------
l1am0
Oh wow thanks for all the feedback. Should not post a link on HN and than
sleep directly after.

I came to work on this as I wantend to not get bothered with the short link
services (and the links behind them) tracking me. So unshort.link also tries
to "learn" (by trying different url parameters) which paramters on the long
url are required and which are not and are only tracking you (with a HTML
diff).

The service learns new shortlink services the moment you enter them on
unshort.link and they than are also automatically used in the extension.

------
OJFord
Yes yes yes! I hate URL shorteners. It's not just privacy - I can't decide if
I want to click it or jot before I do.

I suppose that issue could also be solved with a browser extension.

It'd be nice to do this with DNS rather than an extension though, above point
aside, e.g. if already running Pi-hole or similar, send shortener URLs to
unshort.link, and redirect immediately.

~~~
tyingq
Pi-hole would only see the hostname, and not the "slug", so that wouldn't
work.

~~~
edoceo
Pi-Hole sees the bit.ly DNS lookup,that responds with HTTP/30x to
full.domain.tls, which is a second DNS lookup the Pi could catch/filter.

~~~
tyingq
But bitly.com/xyz can be mapped to full.domain/abcdef

The pi-hole doesn't see the /xyz, or any of the http redirects either.

------
tyingq
Pretty cool. You might consider adding some of the more popular vanity names
that bitly runs for big sites. Like nyti.ms (New York Times), wapo.st
(Washington Post), etc.

~~~
l1am0
You can add them yourself :D Just unshort them on unshort.link (not the
browser extensions) and the service learns that it found a new shortlink
service and it will be deployed right away.

After the next browser restart also the extension knows about that new
service. (No updated required)

You can see if your updated worked if e.g. wapo.st is listed in
[https://unshort.link/providers](https://unshort.link/providers)

------
pmoriarty
How do we know this site is not tracking us?

~~~
notduncansmith
You can run it yourself, and audit the source if you like:
[https://github.com/simonfrey/unshort.link/](https://github.com/simonfrey/unshort.link/)

~~~
eat_veggies
If you run it yourself, do you not lose some of the privacy benefits?

~~~
notduncansmith
I believe you gain privacy by running it yourself, since you're no longer
submitting a portion of your browsing history to a third party. The README for
the server claims "You can build & run it yourself for even better privacy".

~~~
OJFord
Shorteners are frequently used to hide/prettify query parameters used for
tracking; so if you run your own you're losing the aggregate anonymity-ish,
and gaining that OP can't see what URLs you're unshortening.

I suppose it depends on your use, but IMO if you run it yourself as the only
user, you lose more privacy than you gain.

~~~
notduncansmith
If the website you're ultimately visiting can see your IP address anyways, I
fail to see the lost privacy. Their traffic will be sliced and diced into
cohorts along many axes, and this is only one axis on which you are slightly
anomalous. If anything, you'll be targeted less as a result since you won't
show up in their funnels.

------
zzo38computer
Is it possible to make such a thing working if using command-line programs
such as curl or wget to download a file rather than opening it in the web
browser?

~~~
l1am0
It should totaly be possible. Same as with the DNS solution you can get the
data via the api of unshort.link and than build a small bash script for it. If
I have time I may try to do such a script, but not sure if and when I will be
able to make it.

Would be awesome if you build something like that! You can get the currently
to unshort.link known short link providers in a nice json format from
[https://unshort.link/providers](https://unshort.link/providers)

You can get the redirect info from unshort link also via its api without
redirecting, just replace /d/ with /api/ in the GET request: e.g.
[https://unshort.link/api/https://tinyurl.com/unshortchromeex...](https://unshort.link/api/https://tinyurl.com/unshortchromeex..).

If you build a solution, please do not forget to add the documentation on how
to do it via a pull request to the unshort.link repo, so everyone could profit
from it

------
zelon88
This is really cool. And it's all written in Go? Interesting.

Do you have any plans on releasing the static JS/HTML? Or is there a way this
could be run on standard Apache/Nginx/IIS web servers?

~~~
l1am0
No I do not play to make it runnable on Apache/Nginx/ISS as there is more
logic in it, than just the redirect.

The js/html assets are in the subfolder server/static/ for your use

------
gesman
Add: Replacement of affiliate or referral links with your user' affiliate link
and you can charge for this service.

------
gesman
I wanted to make something like this with my C.GG domain but couldn't refuse
an offer to sell the name for $10k

