If you are not using Tor Browser and want to fix the security hole without disabling WebSocket completely, running the web browser in a separate network namespace is a workaround - you get a loopback interface which is independent from the main namespace, and you create a NAT interface within the network namespace to allow outgoing traffic. It's also a possibility for a website to probe other machines, such as the setting page on your router. For better protection, you should block all the local addresses defined by RFC1918 via netfilter/iptables as well.
For developers who needs less restrictive blocking for debugging, you can run multiple Firefox processes in different profiles (firefox -P --new-instance), each running in a different network namespace - to make it easy, you can code everything in a shell script and create desktop icons for them. I normally use an ad-blocked and 3rd-party-cookies-blocked profile for web browsing, but a naked Firefox profile for development.
Sorry to invoke the meme, but Opera did it first, in Opera 9.50 (2008). I don't have a good reference to hand, but  is a developer complaining about this. [Edit:  covers the feature in some detail.]
Opera also blocked access to private IP addresses (so there were three tiers: public IPs, private IPs, localhost; higher tiers could communicate with lower ones, so the block was only unidirectional).
IE10+/EdgeHTML-based-Edge (and I know there was some talk about blocking this in Chromium-based Edge) also blocks it, so that too is prior art to the Tor change.
One is clearly that you need to communicate the requesting IP deep enough into the network stack to the point where you get the DNS response (if there is one), which means there's a fair bit of work to ensure this is done everywhere;
Another is it's known to break corporate websites (https://internal.bigcorp.com/ on a public IP expecting to be able to access private IPs on their intranet), and smaller browsers are reluctant to lose corporate users by changing long-standing behaviour;
Then there's the WebRTC question: if two users on the same local network start a WebRTC call, do we want all this traffic to go to via first point where the public IP is known? For the majority of users the answer is no. And as long as that's possible, there's a (very limited) means of local communication. (But if it's limited to only things that respond to the WebRTC handshake, we're already in a much better place!)
This website used to run Java applet. This applet was signed and it could access restricted APIs to access USB device. So website talked to applet and applet talked to USB device to sign data.
After major web browsers disabled Java applets, they implemented another approach. Now user must install a software which runs a web server on 127.0.0.1. This webserver listens on a specific port and uses web socket to communicate.
So an ability for external website to connect to 127.0.0.1 actually is essential for this use-case.
My guess is that there are plenty of other websites which use local web server to interact with locally installed software. I know at least one another such a website: Blizzard website. It runs web server in its game launcher and website can communicate with it.
PS also they have to install custom trusted certificate because browser requires wss from https and there's no easy way to get a legitimate certificate for that kind of use.
That bypasses the entire certificate question and lets the website know it's communicating with exactly this app and not something happening to listen on the port (and vice versa, too).
... Or, depending on what you're doing, just use a real desktop app, perhaps with an embedded browser.
It sounds like random other websites (Ebay, etc) would be able to interact with people's USB devices this way too. Maybe without people knowing?
The browser connecting to the government website accesses two servers: the original one and the second local one you install yourself on your system. The local server runs natively and therefore can access the USB device. Like all servers it should be programmed such that misuse by hackers is prevented.
The only thing missing is a rogue website abusing it.
There's no guarantee you will never connect to any rogue website that abuses this government mandated backdoor.
Should websockets be able to connect to the local machine?
In some cases that could be useful (even if something the kinds of people visiting this site might be uncomfortable with). E.g., a local daemon that listens for a websocket connection and a public web page that's used for managing it.
I don't think it was implemented this way, but think something like Battlefield 3's matchmaking: the user visits the online service and finds a match to join, and when they want to join it passes the relevant information to the game via websocket.
I imagine stuff like this has been implemented in at least a couple cases that you'd break just wholesale blocking connections to 127/8.
As someone less tech savvy but still concerned, are there any guides available on how to do this?
I think in this case though where you're more concerned about the very real problem of websites accessing localhost, it probably outweighs the maybe of a firejail exploit.
These tools often drop privileges as soon as the program is executed, in firejail, there's also an option to disalble root entirely within a namespace.
There are some blogs that talk about how to do this:
If anything, this should require very explicit permission granting from the user. I’d prefer it be something more like an undocumented toggle accessible solely to developer types.
That's too little. All access from a different origin should be blocked by default, not only to local nets.
How shall origin be defined? I can envision the likes of Microsoft which have many, many second-level domains making calls between them.
We can’t allow the site itself to grant access. How would this be managed, other than “please stop and think what a domain name is supposed to be before spraying your product across twelve of them?”
(Which is in turn made harder by IPv6 public addressing, where you can’t just block the private IP range because you might not be behind a NAT in the first place, instead only behind a firewall. So your address A::B, can route to your Intranet peer A::C, which isn’t public-routable, but is a public address. But there’s nothing, other than the firewall, that says that that’s not a public address. It’s a hard problem!)
evil.com can set a short DNS TTL, and after you access it, it can rebind its address to 127.0.0.1. Then subsequent requests to evil.com go to localhost (e.g. fetch("evil.com", ...) on evil.com will go to 127.0.0.1 if the DNS rebound successfully).
Caching a website's IP on first use doesn't help, either, because it breaks long sessions on websites that use DNS rebinding for legitimate purposes (load balancing, fallover).
The only real way to fix this is for the local webserver to check the Host header on the HTTP request... or look at IP addresses. But building a global registry of IP addresses is hard, so we're stuck with trusting application developers (and malware writers) who run servers on localhost to use good security practices.
The process is different enough from cookies to warrant another large discussion about how to do it, with plenty of trial and errors. But the stakes are much lower, as in the worst case the user will get a dialog, instead of a site being broken.
Still, doesn’t mitigate attacks against non-HTTP speakers.
Forgoing CORS and making all inter domain requests user opt-in would make the web experience a lot worse, IMO. Making all intranet or localhost requests user opt-in seems less disruptive.
In the case of scanning, a CORS denial can still reveal information about the user's internal network, as a CORS denial is a different result than a network timeout or a TCP RST.
That a script is able to gather information about an origin that did not it in seems like a serious bug to me.
I don't think you need to overdo it in terms of making the warning red, etc. Just a popup will really discourage people from trying to use this for fingerprinting.
BTW the site says:
"Port scanning is malicious." I don't agree. There are many many things that can look like a port scan but are not malicious, most notably NAT traversal attempts by WebRTC, games, chat apps, and so on.
It's not against the law. It might be _shady_ but it's not illegal. When I'm teaching cyber intro classes, I let folks know portscanning is NOT illegal but shady. It's like going to a business after hours. It's not illegal to rattle doors and windows to see if they're locked. The police might have a different take on it, but it's not illegal.
While it can be used to get information to bad things, it itself can be used for good things too.
I’d be all for a user notification that says “fnord.com wants to access 192.168.0.10 on tcp/443, which seems to be a web server on your home/work network. Are you sure you want to allow this?” I’d want to see this for each new access request, such that port scanning would not be a use case that was supported.
Sure, have an about:config toggle to shut this off, with appropriate warnings.
I find it very surprising that now any random website can access it with no oversight.
Why worry about spectre and meltdown when such blatant backdoors are implemented in browsers?!
For instance, if I'm running a local chat application and need it to communicate with the web version, why does the website need to be able to port scan to accomplish this? I can think of other ways to accomplish this that are a lot more secure.
But I think if same-origin were enforced more strictly, they could have found another way.
Here's an interesting tangential article about how they get around obstacles with SSL certs for localhost: https://letsencrypt.org/docs/certificates-for-localhost/
Your bot is running a redis server locally, it allows local connections, because it's just a bot, boom.
Taking it a bit further, if we have really smart people involved: the timing of the attempted connections/rejections tell you something about the system that you can use to detect bots/scrapers.
Another example of this being used in the past is to scan for chrome extensions that scrape site content as well. I believe LinkedIn might have gotten hit hard for trying something similar but they were using extension URLs not localhost. Some extensions do spin up localhost services though.
A port scanner running on a webpage without the users knowlege is never legitimate.
So the question is what legitimate reason is there for a port scanner running in a web browser with the users knowledge?
I don't get upset if someone opens and closes a socket to my VPS to see if something's there. My VPS is exposed to the internet. If a socket opens, it should be secure anyways. There's the chance nginx has an unknown zero day, but if I wanted to avoid that, I'd firewall it.
Things are a little less nice if you open a socket and start sending data to see what's there, assuming the server doesn't respond with a banner.
That might be a reason for you to port scan your network.
It is not a reason for your website to port scan my network. Especially since your website running inside my browser is inside my firewall.
> and want to run a security audit
Then you use tools designed to run security audits. You don't open a huge security hole in everybody's browser just so you can use a browser to run a security audit.
There's really no legitimate reason for eBay, or any other website, to portscan your computer. There's nothing there needed for browsing their website.
That's not what's happening here.
My laptop is not exposed to the public internet because it's behind a firewall / NAT. This is like going to my house, plugging a device into an Ethernet port on my router, and scanning my internal network from inside my network.
Except instead of them planting a device, all they have to do is get you to navigate to their webpage. They're getting your laptop to do the port scanning for them, and in so doing, they get access to your internal network. The problem isn't port scanning, the problem is NAT busting.
It shouldn't matter what malware is on a client device as long as the client has authenticated; the server/company/ebay should be protecting their API from abuse at the API layer, not the client layer.
Know your network.
Though port scanning can be (and maybe even frequently is) done with malicious intent by looking for misconfigured/bugged servers, I disagree that it's inherently malicious. Port scanning is just about checking to see what services a host is offering you. It's like going to a random shop at a mall and asking what services they provide. Would asking about their services be malicious?
It feels like the reason asking about services is considered malicious is because shops frequently give out info to the public that they shouldn't have. It's like:
client: What services do you provide?
shop owner: Well, I can provide you with a list of all my clients along with their personal information they entrusted to me.
So, is the client being malicious for asking or is the shop owner the one that was in the wrong for mistakenly providing that info to the public?
I feel the only reason we don't blame the shop owner is because even though he's the one that mistakenly discloses private info, sometimes he's just following a script written by a random programmer unassociated with him. Maybe the response was a mistake on the programmers part, maybe it was a mistake in how the shop owner used the script (a configuration error). In the end, it's simpler to blame the client for asking out-of-the-box questions (after all, most clients just come in to ask if you're giving out flyers/pamphlets because that's what everybody does) and so they don't feel responsible for the response that results.
I can provide a shop that also offers things different than http(s) with open access to the public. It shouldn't be a crime/violation to ask me if I offer them.
I dislike the growing idea that HTTP is a core part of the internet, and not just the most popular part. The difference lies in if we're going to see legislation that dictates proper use of the lower networking layers like TCP/IP by stuff of the upper layers like HTTP. I'd really hate to see something along the lines of "it's illegal to use a TCP port unless it was specified as available to the public in some (possibly js-rendered) part of an HTTP response."
Message board nerds seem totally convinced of the idea that computer crime law tracks the state of the art in offensive computer security, but the two concepts aren't directly connected at all.
I speak both for myself and, I think, for a lot of security researchers both academic and professional when I say that I am very, very nervous poking at a website that hasn't given me permission to, say, check if an input that generated a crazy error is, say, letting me inject SQL, while at the same time I am never scared about port scanning things. There are companies, well-respected companies, that do nothing but port scan everything on the whole Internet.
In this case, eBay is the shop, and I'm the customer. It's like walking into eBay and when I walk in I have to empty out all of my pockets and open my phone screen to show them that no one is telling me what to shop for (VNC).
If you want to continue using that analogy, then you have to consider that everybody is blind and deaf, and checking to see what's locked is the only way to know if something is available.
But you can see what ports/doors are available. TCP doors are defined in the RFC and they are numbered 0-65535. Those are the ones available.
Port scanning still is analogous to trying all these doors and see which one are open.
Just because it is a lot of doors to choose from doesn't make it very different. That's why guests ask a host where the bathroom is.
When you visit a website, it's not very cool for that site to check which of all your TCP ports are open. It's none of their business.
> If you want to continue using that analogy, then you have to consider that everybody is blind and deaf, and checking to see what's locked is the only way to know if something is available.
> instead of waiting for the guy to tell you which one to go to?
How does that translate to TCP/IP? What is "the guy" representing? The way I see it, there is no guy.
I don't think this part of the analogy is accurate. There are no "authorized personnel only" ports
The first half of the analogy is good though.
Nobody should catalog wifi access points and their location!
Of course, now this ethical lapse is a business model and apple, google and everyone else does it. literally anyone with a smartphone is doing this to your wifi access point. And they will do the reverse to find out precise location.
> including the ones that say "authorized personnel only", "private", "do not enter"
Basically, to answer you separately, an analogy like that doesn't represent TCP accurately. In your analogy, you can
1) see from afar for visual cues indicating whether access is being given to you, and
2) try opening it.
Your argument is that doing 2 is invasive, because they can do 1.
However, in TCP, you can only try to make the connection. There is no see from afar. If I give you an IP address, there's no standard way for you to tell me whether FTP is available, without trying to connect to the port! That's your only choice!
So, yes, "I was trying to find out which [service is available]" is a very valid reason.
> Port scanning is a brute force, over-reaching probing technique.
It certainly is brute-force, and that sucks. I think there's a network service / protocol called Portmapper/rpcbind that lists the services available and port numbers they're on. I only know that NFS uses it, but nothing else. If that were standard, then I'd agree port scanning is over-reaching. As it stands, though, I don't consider it over-reaching when it's the only TCP mechanism to see what's available online.
> Though port scanning can be (and maybe even frequently is) done with malicious intent
I agree that it's wrong for eBay to be doing this. What I disagree with is specifically the statement "Port Scanning is Malicious".
Unless it asks for explicit consent for a security audit or something.
I have to wonder what value eBay would get from port scanning its customers. Is it part of an attempt to detect bots/attackers? Is malware running on their server trying to determine if the client is likely vulnerable to some propagation method?
I'd say 90% of the time the powers that be at the company had no idea someone was running that software, or that it was still running at their company, and then someone moved a firewall and the system was exposed to more than intended. Then they'd turn it of ... and find another similar tool running somewhere else.
It could be a simple as a test or security system run amok.
This demonstrates the absurdity of the CFAA more than anything else. Sorry for sounding like a broken record but the CFAA is not salvageable and MUST be repealed.
Edit: Link to the company http://www.forensicstrategy.com/
He also has a data recovery company now http://www.myharddrivedied.com/
As for the CFAA that's seemingly down to how aggressive the prosecutor is feeling about your case. I don't think it should be there's no real access happening and unless it's extremely aggressive and degrades network connectivity it's hard to argue there's any real damage done.
Criminal law is usually not this simple, as most criminal laws will take into account the mental state of the person performing the action.
You can get to the same answer axiomatically from the text and case history of CFAA (a port scan literally can't grant you the access a CFAA claim needs to prove you intended), but that's obviously treacherous for non-experts to do; instead, the empirical demonstration should be conclusive here.
I don't know why this scan is occurring, but fingerprinting is the most obvious guess, and intrusive fingerprinting performed by real companies is usually about ATO prevention, which means they're not going to tell you any more about it (ATO defense is an arms race).
From the article:
> Looking at the list of ports they are scanning, they are looking for VNC services being run on the host, which is the same thing that was reported for bank sites.
> VNC is sometimes run as part of bot nets or viruses as a way to remotely log into a users computer. There are several malware services that leverage VNC for these purposes.
That’s what these sites are doing.
That one you linked is a messed up case. There is a phenomenal podcast that interviews those guys and walks through their engagement. https://darknetdiaries.com/episode/59/
(No, seriously, I know people who believe that.)
I'll be downvoted for saying this.
Because it's against the site guidelines.
> Please don't comment about the voting on comments. It never does any good, and it makes boring reading.
Shodan is outside your network's firewall, therefore only able to access services you've exposed to the wider web.
If I understand the article, the websocket scan eBay is doing is trying to connect to local listeners on your laptop, behind your network's firewall and possibly even behind your laptop's firewall.
It does not, however, sound like an attacker can establish arbitrary TCP connections (at least using the technique from the article). Instead, the attacker can determine if something is listening on a port because it will take a different amount of time to negotiate/drop a connection to a port when there is a listener than when there is not a listener.
In other words, this sounds like a variant of a timing attack. As such, presumably, this particular avenue of attack can be mitigated by the browser vendor inserting a delay s.t. no information can be gleaned from how long it takes to negotiate/drop a websocket connection.
EDIT: I also wonder if it would be possible to do a similar port scan using the timing of XHR requests to localhost (e.g. http://localhost:[port]).
Maybe not, but what if the ports you have open actually are HTTP servers for development purposes? In that case wouldn't a website be able to crawl your unreleased work, and/or mess with what you're doing, with requests seemingly "out of nowhere"?
One time, I port scanned my public IP (of my ISP) from an EC2 box, and I got an email from EC2 saying they received an abuse complaint from the ISP for port scanning activity.
- Monitoring your own network or that of your clients for exposed ports
- Researching Internet topology, or performing aggregate queries like “how many nginx servers are connected to the Internet”
Can you use it maliciously? Yes. But, most of the time, if you have a target it would make more sense to do the port scan yourself. And if you’re just dragnet searching for vulnerabilities, most you find will probably already have been exploited. Sites like shodan are good for the overall health of the web because they force website owners to maintain security posture. If you know that foregoing a wordpress upgrade means you’re one script kiddy with a shodan account away from getting hacked, you’re going to keep your site up to date. This saves you from script kiddies, but also from the more sophisticated hackers who would run a port scan themselves anyway.
Any legitimate security service is going to be doing there own scans, surely.
Statistics, yes, but I can't see those stats being especially good. You could probably get equally good nGinx data from netcraft, who IIUC get the data from http responses banners on :80 :443.
I'm not sure I buy the "security posture" line, isn't it circular. Tools to help crack your site are good because it means to have to have counter-measures to combat tools for cracking your site?
Only legitimate use of port scanning for me has been testing access to my own/clients computers, I feel. That's not too say I've not used it for illegitimate things ...
Legitimate usage from researchers and people reading about infrastructure they have the right to do security testing on may be a larger percentage than you think.
1. External network monitoring: know what you have connected to the Internet and get notified if anything changes unexpectedly. This has actually gotten significantly more challenging with services deployed to the cloud where your IT department might not even know which IPs to keep track of.
2. 3rd-party risk assessment: understand the security exposure of your partners, vendors, supply chain or other 3rd-parties. For example, lets say you're an insurance company that wants to provide cyber insurance. Shodan data can help you understand what sort of risk you'd be taking on. The data has also been used in M&A as part of due diligence to get a metric on the security of the IT department of the company they're thinking of acquiring.
3. Market intelligence: basically Netcraft on steroids. Shodan doesn't just have web information but also for many other protocols. This information is used by hedge funds and vendors to understand which products are purchased and deployed to the Internet. The data is skewed due to the nature of public IPs but there are still things you can do.
4. Policy impact: get a measure for how policies at the country-level are impacting Internet connectivity. For example, the OECD used Shodan to get a measure of Internet-connectivity per capita.
5. Fraud detection: is your customer trying to make a purchase from a machine that's been compromised? Or running a a VPN/ proxy? Shodan is used in transactional fraud detection to flag suspicious payments.
What percentage of torrent traffic do you suppose - or better have stats for - is not copyright infringing? I'd think it's about 0%.
Would certainly be interested if you can prove that's wrong.
Not an eBay employee, but used to work in fraud detection. Two very obvious related guesses from my experience:
1. Fingerprinting a user to help identify account takeover (ATO). Open port signatures is probably a pretty good signal for that kind of thing (and it doesn't seem to be measured in https://panopticlick.eff.org/).
> However it is also a valid tool used by administrators for remote access to machines, or by some end user support software, so the presence of VNC is a poor indicator of malware.
2. In a Bayesian sense, this probably isn't right. I don't know what eBay's traffic looks like but I'm willing to bet that all other things being equal, traffic coming from a machine with an open VNC port is riskier. Fraud detection is a game of probabilities, so the existence of a valid user showing a particular characteristic doesn't mean that the characteristic isn't useful in a fraud model. The example I always give is that when I was doing this (quite some time ago), we could have had a 99% accuracy rate for a simple rule banning IPs from Turkey, Ghana, Nigeria, and Vietnam. It's not because there weren't any valid users from those countries, it's just that the fraudsters where overwhelmingly likely to be using IPs from those countries.
Some more info here: https://medium.com/@brannondorsey/attacking-private-networks...
Example code: https://github.com/brannondorsey/dns-rebind-toolkit
A malicious DNS rebind server: https://github.com/brannondorsey/whonow
Disclaimer: I performed some of this research a few years ago. So those resource suggestions are my own, but they feel very relevant here.
HOWEVER -- how the hell is localhost port scanning allowed to happen without my permission?!
This feels no different from a website trying to check the existence of named directories on my file system or something.
Does WebSockets not require permission to function at all, or shouldn't it be limited to some kind of CORS-type policy or something to connect without a permissions dialog? Or even if it's allowed to port scan the entire public internet, at least block your local machine and network without explicit permission?
There doesn't seem to be a way to access anything locally, just test for open ports. I use SSH tunneling a lot and was having a minor freak out.
(In the private/loopback IP ranges we should really just make those requests always fail, but I addressed that in another comment as to why that's not trivial.)
1) Will I ever actually use this
2) How is this gonna screw me over
WebSockets, WebBluetooth, WebAssembly, Web-You-Can-Access-my-Accelerometer-and-Battery, haven't ever wanted to use those. Ever. For anything. For any reason. (Edit 3: Oh yeah, I forgot! WebRTC!)
Edit: Fantastic. You can't disable it in Firefox. So what, does Firefox need a freaking iptables implementation now? 
1 - https://bugzilla.mozilla.org/show_bug.cgi?id=1091016
"The only theoretical reason for the WebSocket pref these days is the possibility to disable it easily in case there is a security issue found in the protocol itself or so."
The protocol itself is the security issue. ALL OF IT.
Edit 2: So I don't have the time to investigate every new fad when it comes out. I originally thought WebSockets were raw sockets, but they aren't. Firefox blocks access to port 22 -- I was hoping all privileged ports, but it seems just those. Opening a WebSocket to netcat dumps out a HTTP request, so it seems unlikely that you'd be able to talk with anything that doesn't talk HTTP and WebSockets. Firefox also seemingly blocks access to 192.168/24 and 10/8.
Edit 4: Done researching this now. I went to ebay on Firefox, and wasn't getting websocket scans. But I've got a stack of uBlock and NoScript... maybe that's interfering with it some how? Opened up a stock config for google-chrome -- that's my browser for "some dumb new web tech that isn't working in Firefox" -- not seeing any scans when I open up inspector and click "WS".
Regardless, his point still stands. You can totally use WebSockets as a port scanner for localhost, assuming the Content Security Policy allows it. Now I gotta go update my nginx configs...
You’ve never used a web-app chat client?
APIs like these don’t exist for the sake of regular unprivileged web-apps. They exist for the sake of browser extensions (or browser “apps”, or apps within a browser-projector like Electron), specifically in order to be used to add driver-like or service-like capabilities to devices like Chromebooks where the browser is the OS.
That's not really true, though: they're part of the Chrome team's belief in not limiting the capabilities of the web as a platform for app development (on the basis of "if you lack one feature the app needs, the entire app ends up native"). This is a large part of Project Fugu: https://developers.google.com/web/updates/capabilities
BOSH? Awkward, but it works without websockets.
I'm pretty surprised however, that a nearly 10 year old standard is being considered as "superflous" as newer technology like WebBluetooth and WebUSB. What we had before Websockets wasn't really long polling, it was Flash.
Yes, fundamentally, on a technical level, there’s not much difference between holding open an HTTP connection in long-polling, vs. holding open a websocket connection.
But the abstractions presented by webserver gateway interfaces (e.g. CGI, prefork process-per-connection language-module embeds), languages/web frameworks (e.g. Ruby on Rails), and platforms (e.g. Heroku) back then, just didn’t support long polling efficiently.
HTTP backends, back then, were all built on an assumption of serialized queuing of HTTP requests—with each web server/web stack/worker thread serving requests one-at-a-time, getting each request out of the way as quickly as possible. There was no concept of IO asynchrony. Web servers were request loops, not event reactors. Libuv didn’t exist yet; nginx didn’t exist yet. The standard web server was Apache, and Apache couldn’t “yield” from a request that was idle.
And, as such, providers like Heroku would queue at the load-balancer, and only proxy a single open request to your web backend at a time, under the presumption that it wouldn’t be able to handle concurrent load. So you had to pay for 2x the CPUs (e.g. Heroku “dynos”) if you wanted to be able to hold 2x the connections open!
Entire third-party businesses (e.g. https://pusher.com/) were built around the hosting of custom web servers that were written in an event-driven architecture, and so were able to host these pools of long-polling connections. But they were freakin’ expensive, because even they didn’t scale very well.
Eventually, it was realized that the just-introduced Node.js could do IO asynchrony pretty well, and people started building explicit “websocket servers” in Node, culminating in the https://socket.io/ library. Back then, you couldn’t just put your regular HTTP load-balancer in front of your websocket backend, because your regular HTTP load-balancer almost certainly didn’t support held-open connections. You needed to host socket.io on a separate host/port, directly open to the internet. (This was one of the driving forces of Node’s adoption: as long as you were putting a Node app directly on the Internet, you may as well just put the rest of your HTTP app in there as well, and make the “websocket backend” into your whole backend.)
Sure, these days, every backend, load-balancer, and NAT middlebox can handle long-polling just fine. But we got there with a decade of struggle, and “legacy”-but-not-really code that used WebSockets because they guaranteed the semantics that long-polling couldn’t.
(I should mention, though, that WebSockets still have some advantages in the modern environment; namely, idle WebSockets are known to be idle at the runtime level, and so, unlike with a long-polling HTTP request, a mobile device can relax its wakeup-timer intervals when the only network connections it’s holding open are idle WebSockets.)
Nope. Not once. And I've been using the web since Mosaic.
I see business web sites offering to chat with me all the time. I ignore them. If I want to chat, I'll let you know.
Apple's business-to-Messages thing works so well, I hope it puts the scammy webchat companies out of business.
Though, honestly, I prefer the web-chat customer service for my bank/cellphone provider/etc. to calling them on the phone. I don't want to wait an hour on hold with my phone using up both battery and minutes; I want to just leave a window open on my computer and have it ding when they're ready.
A decade ago people use comet for this, with php and apache! Every single ongoing comet connection occupy an apache process, a significant resource hog, yet people still use it because they have no other choice. These days we have websocket but I bet comet can be implemented with minimal resource penalty these days thanks to proliferation of async webserver support in modern backend stacks.
Chrome the browser is a stalking horse/test harness for ChromeOS.
Back in the 90's if you wanted an ohms law calculator you had to go download a poorly written program from some random website. Network admins started locking down what you could download, run, and install due to security problems. Flash became a hit and they started piling on features in the browser so you could run things dynamically without having to download something.
Fast forward almost 30 years and the browser has become so full featured it is practically a weak OS sandbox that allows you to run just about anything. It was originally being extended to avoid that in the first place, and here we are almost back to square one.
The big disadvantage of the browser is of course that there's huge competitive pressure, and most users prefer usability over security. Keeping things secure without asking the user about their intentions every step is a huge challenge (see also Windows UAC, which struggles with the same problem).
It's not necessarily an advantage, it's just a different threat model. An OS is protecting against an attacker already having access to the system (whether physically or over network.) The assumption is that the system is working properly and it's the operator that is malicious.
For the browser, the assumption is that the operator is working properly, but the systems they will be accessing are malicious.
The browser security measures are like a guard at the castle gate, allowing or preventing people from entering. The OS security is like locks on the doors inside the castle so that only people with the right keys can get into various protected rooms.
Both are necessary because they're preventing different things (access to the system vs. access once you're already inside the system.)
I'm no expert, but I am computer savvy. My experience is that more security ALWAYS limits functionality that I want. I get around it all with VMs and snapshots. I have totally wide open VMs that have no host access and just reset to the snapshot when I shut them down. the irony is that it is not only harder to choose functionality today, when you want it, but also harder to choose security. The choices are made for you. it is infuriating.
Also uBlock has an option "Prevent WebRTC from leaking IP adresses".
WebRTC should be disabled by default or firefox should ask
explicitly like with webcam-access.
There have already been reports where sites use your browser as a
peer in a P2P-network (without your consent). This can be really
problematic depending on where you live.
A local IP?
If you're using a workspace VM, you can run the VPN client in a pfSense VM, which is functionally equivalent.
If you're not using a workspace VM, and can't run the VPN client in a router, you can run the VPN client in a pfSense VM. You bridge the WAN interface of the pfSense VM to the host LAN adapter. So then the host can't use it. And you configure the LAN interface of the pfSense VM as host-only. So now the host machine (your workspace) can reach the Internet only through the VPN client in the pfSense VM.
Or you can just make sure that WebRTC is disabled.
But WebBluetooth, ASM, etc are all fairly insane. WebRTC feels like a massive security issue (I've seen a demo of someone using WebRTC to find computers on an internal network at a security conference years ago. Even if that hole is fixed, it's still a hacky solution to video streaming behind NAT).
I agree; most of this stuff needs to have ways to disable it, in the base configuration screen of the browser (not hidden somewhere in about:config).
Almost every site with some kind of front-end framework like React will use a bundler of some kind.
Last I checked (> 1 year ago) it was WONTFIX because of some very idiotic (IMO) reasoning. I keep it permanently disabled and have never missed it (media.peerconnection.enabled in Firefox btw).
"If you're not paying, you're the product" has its own form of economic inflation.
I'm not familiar with Mastodon or WSS. Can you describe how using WSS make the end user's experience better? What would be different if web sockets weren't used?
There are other FOSS ActiviyPub servers such as Pleroma (written in Elixir), Pixelfed (Instagram type interface) and PeerTube (distributed video). ActivityPub is a protocol (like e-mail/SMTP) for subscribing and replying to posts. ActiviyPub is just used for the backend (how servers communicate with other servers; like SMTP sending e-mail or RSS readers polling RSS).
Mastodon uses websockets to stream posts to the client/web browser. People who make mobile apps and desktop clients also tend to use WSS. It does fall back to regular HTTP polling in case WSS fails. Pleroma and others also re-implement the mastodon API. It means you can have different front-end web clients (Pleroma-fe, soapbox-fe, etc) on top of different backends (Mastodon/Pleroma).
The advantage of WSS is being able to stream new statuses with a socket, rather than polling constantly for new updates.
If you get on a Mastodon/Pleroma/Pixelfed instance (there are hundreds out there or you can setup your own), you can follow me at @email@example.com
Personally, I'm looking forward to the point that web apps are capable enough to let PWAs do absolutely anything an Android or iOS application could do.
Can anyone expand on why this technique isn’t more common? I’m so sick of seeing folks reinvent HTTP (poorly) on top of WebSockets. I get if extreme low latency is (allegedly) a requirement.
Hard answer: WS has been around for longer, and it's had more "marketing" for lack of a better word -- more people know about it and know how to use it. Retooling existing WS-using code to use HTTP/2 pipes would be a considerable effort with little to no perceived benefit to most users and teams.
Speculative answer: most web developers live closer to the application and presentation layers, and there's resistance to learning technologies that involve HTTP connection management e.g. in nginx (not to say that there are none of these, there are just fewer of us). WS was at the right place at the right time with a good high-level interface available to the client, and gained traction because of this.
Because WebSockets is 10 years old and HTTP/2 is 5 years old, and that's not including support in major frameworks for SSE.
I mean, I could half-ass my work a lot more often if they'd get rid of these burdensome restrictions on cross-origin requests. But they ain't going to.
But, at every point in the web’s evolution (including today), there was always been something that needed to live on a different host or port for some reason or another—usually because it’s too leading-edge for load-balancers to understand how to route it correctly.
TCP load balancers that can handle long-lived flows with connection stickiness et al, are a very modern invention in the web’s history; and even they still stumble when it comes to e.g. WebRTC’s UDP packets, or QUIC.
Not everything needs to run in a browser.
This wouldn’t be a meaningful security improvement for anyone.
You're imagining a world where the most popular dev environment goes away, and service providers decide to use HTML forms instead of forcing me to download an app every time I want to order a pizza. That world does not exist. The apps aren't going to go away, and your security model can't be, "people just won't install untrustworthy apps."
And put things in perspective here -- we're talking about a security vulnerability that allows port scanning primarily for fingerprinting purposes. A native app can not only port scan, it can literally just make POST requests to those open ports across separate domains. The security risks we're talking about are not even remotely equivocal.
Don't get me wrong, stuff like port-scanning should be fixed in web browsers. But even with these vulnerabilities, the web is still unquestionably the safest consumer-accessible application platform that we have today. Moving applications off of the web and back onto native platforms would be setting security back half a decade.
When someone comes to me and asks how they make their phone more secure and more private, the number one piece of advice I give them, every single time, is "avoid native apps and use websites instead. Don't install Facebook, use the website. Don't install random clicker games, browse them online instead."
The web has been a major asset in my quest to get friends and family not to install a bunch of random malware on their devices. Doubly so when you throw kids and younger users into the equation. I am eternally grateful that the web is advanced enough that people can join a Zoom meeting without installing Zoom on their computer.
Honestly though, this is just me being bitter at web developers.
For applications that want special access to my machine, there SHOULD be a barrier to entry or inconvenience like a client-side installation.
Chrome OS was far ahead of its time, both in its general user hostility and in its excessive resource consumption for seemingly mundane tasks. Like when your roommate's YouTube streaming makes your Excel-like app horribly slow, because your new spreadsheet app needs high-speed internet for no obvious reason.
No, cross-compilers to JS are and were thing too, and were used by cryptominers.
I don't need WebRTC for chat, because IRC is still working just fine.
I don't need WebRTC for video calls, because Skype is still working just fine.
I don't need WebRTC and WebAssembly for online gaming, because I have Steam to install games locally. Plus latency and performance of games emscripten-ed to wasm tends to be atrocious.
So the only uses where I have seen the new Web* APIs in my life was obnoxious ads and newspaper websites trying to shove recommendations and notifications into my face.
What would you want to cross-compile that couldn't be done much better by building a proper local app?
Just because something doesn't have value for you does not suggest it does not have value, period.
And obviously, Skype and IRC use the internet, too. So what is the benefit to me - the user - of "Skype inside web browser" over "Skype separate of web browser"?
I understand that WebRTC makes "Skype inside web browser" possible, but it also makes a lot of less desirable behavior possible. So there is be a trade-off between the advantages and the risks from adding WebRTC.
Which benefit does WebRTC generate for the user?
I.e. why is it a good thing that web browsers added this?
umatrix is the layer7 firewall you're looking for, it can block websocket connections, cross-domain ones in particular are quite easy.
But you can also operate umatrix in a blacklist-based approach, i.e. simply let most requests through except the categories you deem problematic (e.g. cross-site websockets in this case)
I'm finding this to be my main problem. I have a fairly solid computer repair/troubleshooting background but can't keep up with the layers of crap that are pasted on top of modern software.
An ongoing list of "features" being added to web browsers and how to get rid of them would be hugely helpful, but I've never found a centralized site for this topic, it's all just scattered around various tech sites.
I get that adding lots of user controls makes state management difficult, but there are tried and true ways to do this; users need the browser to work for them, not the other way around.
I mean, there's a lot in about:config that the settings panel doesn't support.
Web browsers are the new cross-platform runtime, better to get used to it.
1. Why are web browsers becoming application platforms? Web browsers are document readers, and we should treat them as such.
2. Why does Zoom (or Slack, or insert-thing-here) want me to download a native app? I should be able to do it in my web browser!
I sympathize with both philosophies, but they cannot co-exist.
Parent is asking for granular access control over these very advanced and double-edged features, which is a perfectly valid request.
Having permissions is the norm for native apps on mobile, and it's slowly becoming the norm also on desktop, finally.
Browsers are the new OS, they have to implement permissions too without anything enabled by default.
Does Zoom, Slack, or insert-thing-here need the technical ability to portscan localhost in the manner of this blog post? Well, Zoom had their vulnerability making web requests to their app bound to localhost, but did they need to be able to do it? They could get the job done without that functionality.
They already have both an app and a web-only experience. (They try to hide the latter by only presenting the option when the former has evidently failed, but it's still there.) So, it has it both ways by your 1 & 2. Almost as if users should be able to choose.
AFAIK this is only partly true, if the web server does not support the websocket protocol, you cannot connect to it .
So if I am understanding this correctly, WebSockets only support a small subset of HTTP and it should therefore not be possible to use them to connect to "classic" HTTP servers or to send GET or POST requests to it.
Binding to 127.0.0.1 is a nicely cross-platform way to do inter-process communication (I've done so in the past to mitigate JVM startup/warmup issues).
I've never written this code defensively, because if you run programs that throw random shit at locally-bound ports, that's your responsibility. The web community has decided it's a good idea to give arbitrary websites that capability. It's true that the 'random shit' may only take the form of WebSocket requests, but this is only a minor comfort.
From my perspective, this needs to be locked down.
edit: On second thought, you have always been able to trigger similar requests by e.g. just setting the src attribute of an image: Opera aside, browsers apparently never implemented proper cross-network protections. So from now on, I'll be extra careful to make sure all my servers can handle unexpected, potentially malicious HTTP requests even when bound to 127.0.0.1.
That said, I still do think this is something that needs fixing on the browser-side.
What about 172.16.0.0/12?
Chrome, OTOH, will happily open a websocket to these IP ranges. Another good reason not to use Chrome.
Why is “I can use websockets on a private network” a bad thing?
...Just in case you missed it and you’re using Linux, from the article...
“I thought it might be because I run Linux, so I created a new Windows VM and sure enough, I saw the port scan occurring in the browser tools from the ebay home page”
With some polish and maintenance it could serve as base for a simple(r) browser for the masses. It is essentially a small wrapper around WebKit, which is far from simple, but it has the benefit of great support for modern web features.
LocalStorage - about:config Dom.storage.enabled
Canvas - block JS via noscript
DRM - Preferences->Disable DRM content checkbox
WebRTC - about:config media.peerconnection.enabled;
As a side note, noscript has been an eye-opener for how many things are loaded when I open anywebsite! It's been fast, but I've been getting annoyed by guessing what to enable to move past a blank webpage.
And WTF is newrelic and why does half of my internet browsing use it?
The web is now a no-install app delivery platform, it makes sense for these things to exist.