I worked with an org that ran their own link shortener... and used it for confirmation links! I'm not even kidding, you'd go to reset you password and as expected the link would be something like:
I hope you've managed to fix this, because this is an obvious security issue. A long token is used precisely because it is long and unguessable. The shortened URL is subject to enumeration attacks which can be used to hijack accounts.
> A long token is used precisely because it is long and unguessable
This. So much fun can be had by enumerating link shortener URLs. I've experimented with enumerating some services' URL schema. Most of the time the link pointed to innocuous things like Amazon affiliate links or whatnot. Sometimes you would find interesting content that made you go 'wow!', but that was very rare.
The first link is a hex encoded token, 80 chars, 4 bits per char = 320 bits of information. The second shortened link key is likely base 64 encoded, 5 chars, 64 bits per char = 320 bits of information. These should be basically the same from a security perspective. Is there something I'm missing here that you're suggesting?
Edit: This is wrong and should be 30 bits of information not 320 bits for the shortened form. 64 values = 6 bits not 64 bits.
Probably is. Base64 includes + and / which I believe need to be URL encoded, plus the = padding can mean an extra step if you want to remove them to make the URL pretty.
Wow. And makes it easier to brute-force (Which I think you're insinuating). If the links have an auto-expire of 10 minutes, is the risk sufficiently mitigated? Or am I missing something else?
I'm not sure if this is still the case but there was a period of time when email clients would not transform the entire URL into a clickable link if it was too long. These were generally email clients which supported plain text only.
Anyway, I think that I'd be okay with a shortener like your example as long as it:
1. Required me to enter my user id again
2. Was only valid for 30 minutes
Yeah, we do that. It's simple and easy, allows us to tweak the destination of a published link if the site shuffles, lets us print simple short links that are quick to type but still are obviously our own domain.
You can't do it for everything... for one thing, you don't gain search engine karma from the links. But it's often very useful.
Do you also do that for links with secret tokens like the reset password link the op mentioned? Because -spoiler- that makes those links very easy to guess/brute force
No, I should have made that clear. Don't do it for links including tokens, user accounts, or anything like that. (Obviously.) Only on links you'd put out for mass consumption.
There are many valid use cases for URL shortener, I am not sure if this is one of them.
IMHO, this is a display layer issue that only affects human eyes, and should be handled on display layer (html, email rendering...etc) -- just don't display the whole URL somehow. Machines won't have any problem processing that long link.
Some platforms do this. My understanding is that they're especially motivated by the fact that many people don't really distinguish between mybank.com/changepassword and mybank.attacker.com/changepassword.
However, it really infuriates some vocal technical folks (e.g. https://www.androidpolice.com/2020/08/13/google-resumes-its-...). I think the compromise is good: hide the full URL by default, but have a setting or some affordance to show the full thing to people who do enjoy looking at it.
Also, I never click on links in e-mails directly. For something like this I'd cut and paste the address it seems Google puts another layer of redirection in Gmail to spy on you ("data-saferedirecturl", whatever that does in their JS)
I encouraged Quora to ban link shorteners. They were heavily used for spam and malware, avoiding whatever (meager) anti-spam mechanisms they were using. By "heavily" I mean "exclusively", though it's conceivable that somebody some time was using it legitimately.
They never did implement that, but it sounds like it might be a good general rule for many web sites that accept and display content from users. If you're concerned about the way long links appear you can abbreviate them on the screen (the way HN does).
An example in this paper cites the shortener used by Google Maps. The researchers were able to enumerate all the short links by brute force and join destinations from specific residential addresses. This is scary because now you've essentially created all points of interest that 1 person visits (originating from their home address).
Google's response was to expand their URL tokens from 5 characters to 12. The sparseness makes it uneconomical for someone to brute-force their way through. Microsoft OneDrive's response was... interesting.
I went to generate a QR code the other day for a URL, just went onto some random website from a quick Google search.
The generated QR code had the URL rewritten to a short URL, and buried in some small print was a limit to how many times the URL could be “scanned” before you have to pay.
I guess these sorts of sites _really_ count on people missing this and spending thousands on print before realising.
I've really grown to hate link shorteners. They get used to obfuscate the real URL, so of course my pihole and other adblocking software blocks them. But even the local gov't insists on using shorteners in the links they put in e-mail. Instead of just making the URL of the website sane. So I have to jump through hoops just to get to the site they link to.
In my company we created our own link shortener using AWS S3.
... just create an S3 bucket with a short domain, configure it for static web hosting, and upload empty files which have the "Redirect" metadata property set to the destination URL. Voila!
You won't have analytics (maybe this can be configured via AWS, but I can't say) but you don't need a server either.
I want to eventually create a friendly control panel to create and delete shortcuts using React, AWS Lambda and Cognito... but I still haven't had time... and we only need to add a handful of short links per year. This can also be scripted and done quickly through the CLI.
> ... just create an S3 bucket with a short domain, configure it for static web hosting, and upload empty files which have the "Redirect" metadata property set to the destination URL. Voila!
Heavy “Dropbox is just cvs mounted over ssh, easy!” vibes over here.
Cloudflare's Workers/KV is pretty ideal for a link shortener. There's a small bit of js to write, but the KV database is just short->long and it's cached at the edge. And it's either free (< 100,000 requests/day) or $5 for 10 million requests.
And the admin panel provides a simple way to edit the KV database, so you don't have to write a db editor.
Note that Cloudflare Workers run before the cache unless you get creative (you basically need a second Cloudflare domain configured in front of your workers). For something as simple as a URL shortener it may not be critical but it does mean that you are paying for every request which can add up for a popular link.
Ah, I was talking about the other cache...the KV cache. Meaning that the short->long mapping is cached for performance reasons, so it's an eventually consistent, distributed, link shortener.
But, yes, not free if you exceed 100k requests/day. $5 per 10 million requests beyond that.
As someone used to shorten his links a lot, that was my biggest concern. As an avid blog reader tho i tried avoid short links as much as i could, although very often to no avail.
Another problem with long lived short URLs is that the account used to generate it can be hijacked later and the short URL be pointed at a different destination with malware or other malicious intent at the end. I've seen this happen a lot in my time.
> My link shortening tool provides me with analytics
I run a link shortener site, and use it privately and don't publicly expose the API.
One thing I noticed regarding analytics, is that the click count is always skewed. When I post a shortened URL on Twitter, within seconds the click count is always `>10` views. After further investigation, it seems there are automated bots that scoop up URLs the very second they are posted.
Also Twitter runs little microbrowsers that scan the page for metadata which helps them create a 'preview' of the link.
After looking at the useragents of some requests I'm seeing generic Firefox UAs which I can only assume are random surveillants (not bots) who habitually scan Twitter for interesting or anomalous content. We truly do live in a world where nothing is left `unseen` (by bots or actual humans).
The 'Government Digital Service' (GDS, sort of 'tech company for the civil service') continually impresses me.
I've no idea who (even which party) initiated it, but it was just sort of suddenly awesome. Or maybe it just evolved rapidly under great (civil) leadership and was 'always' there just not so great.
Blog has lots of good stuff too, especially articles on accessibility often do well on HN, since that's something they 'obviously' need to worry about, and actually do & do it well.
Often in comments here too (Robin something is a username I recall) - not the sort of crusty 'what's an HN?', 'you can't do that because the Oracle database on our IBM mainframe doesn't support it' department I might've formerly imagined at all.
Sadly they are encountering resistance though. Some departments would rather spend £X bn contracts with HP, Fujitsu etc. so they can retain more control.
Also they used to publish amazing service status dashboards, showing how many transactions were published, error / success rates, etc. for every digital service.
Apparently these were all killed off recently with no replacement, and no good reason given.
If that's the US Dept of Digital Services you're talking about, I think it was created by the Obama administration to resolve the rollout of the Affordable Care Act site where people can sign up for various plans. They see to do some cool stuff
Sort of; the actual founding of USDS happened after the healthcare.gov recovery, but was directly inspired by that and included many of the same people, including the first administrator, Mikey Dickerson.
I appreciate the core message, but it's quite disappointing to see a government message exclusively talking about how Google Analytics (coupled with Twitter/FB Analytics) is the one solution. Especially as they problematize user privacy.
Given that this is a mainly message for those communicating on behalf of gov.uk, I think the best they could do is host a URL shortener for use by government communicators. It's also good advice for businesses.
> If you’re adding campaign URLs to offline materials – like posters or leaflets – and don’t want to feature a long web link, I’ve got good news for you too. GDS provides the option to you to request a shortened version of a full GOV.UK URL.
I'm disappointed that they mentioned Google Analytics. People willingly using Twitter (or Instagram) is a thing, involuntary Google tracking is another.
A massively over-engineered solution for the completely made-up problem of not being able to use the original URL. Yes, that’s perfect for blockchain technology!
Feels like a bad idea. Shortened links often don't need the level of longevity blockchains provide, nor will they be able to afford the cost if decentralized storage with high availability.
The article ironically links to LinkedIn explainer which states if the URL is > 26 characters it will be replaced with their URL, Not a hyperlink like Twitter of many other platforms which tells the reader where the link points after redirecting through the tracking URL.
I believe API's exist. You can follow redirects in most webclients pretty easily, or there's "redirect detectives" online: https://www.redirecttracker.com/
you should specify to what country the "government" refers to. Do you propose a generic thing for governments around the world, or is it specific to a particular one?
These days I never click shortened links without first verifying where it will take me. There is so much malware out there and browsers are nightmarishly insecure that a single link click could result in a getting completely pwned.
Pro-tip: append the “+” character to any bitly link to show the target link without first visiting it.
Pro-tip2: consider browsing with JavaScript disabled by default. Enable it on a per-domain basis.
I still wouldn't trust the plus character to not fail or whatever one day. I manually expand each short URL I get using various webservices. I'm sure there's an extension for that. I would still just walk to the expander website and paste it in though.
Another more technical option is to make the request using curl and printing out the “location” header. Can browser extensions make non-redirecting requests and inspect the return headers?
As the article notes, we don't want to socialize people to click on just any link.
I never click on links unless I know where it is going to lead me. Shortened links are one example. Even with an accompanying description, it raises red flags. Links to reputable image or video sharing sites without an accompanying description, is another example since you never know what is going to be on the other end.
Description: In a world dominated by bit.ly, ad.fly, and several thousand other malware cover-up tools, this list reduces the length of URLs in a much more legitimate and transparent manner. Essentially, it automatically removes unnecessary $/& values from the URLs, making them easier to copy from the URL bar and pasting elsewhere as links. Enjoy.
I assume the shorturl would be named (hypothetically gov.uk/ucas -> gov.uk/university-clearing-through-ucas). Fully established tech companies has a similar process for requesting short links, for instance Google has an internal form to request g.co/ short links.
I get the impression that they want to ensure people choose meaningful short(ish) URLs, rather than getting random alphanumeric suffixes, because they are invested in the trust placed in the gov.uk domain and it not looking like phishing bullshit. So it makes sense to me that they want to curate the namespace rather than making it either self-service or fully automated.
Meanwhile, domain registrars are still emailing customers asking them to "click this link" to verify their contact information. No URL shortening there, just a wildly irresponsible process.
Edit: I know it's required by ICANN (I read the emails) it's the "click here" action that bothers me and perpetuates dangerous behavior.
Does ICANN actually mandate that a 'click this link' be included in the email, or that an email is sent asking the user to verify data so that a 'please login to your account to verify' would suffice?
> Does ICANN actually mandate that a 'click this link' be included in the email
Probably. It's been a while since I worked in that industry, but ICANN has always been pretty picky about the exact contents of emails.
Changing the user workflow in the way you're describing is out of the question. The entire purpose of clicking a link in the email is to confirm that the email was received at the domain's WHOIS contact address. Allowing a user to log in and click "confirm" without clicking a link in the email wouldn't confirm that.
> Does ICANN actually mandate that a 'click this link' be included in the emai
This is my issue. I've spent years telling clients to not click any links in an email from your bank/insurance etc. but yet Network Solutions and others are still putting a big fat "Click here" button in the email.
reply