Hacker News new | comments | show | ask | jobs | submit login
Say Cheese: a snapshot of the massive DDoS attacks coming from IoT cameras (cloudflare.com)
336 points by jgrahamc on Oct 11, 2016 | hide | past | web | favorite | 208 comments

I've been warning about this kind of thing for years and years. Both professionally and in my personal life. On forums and anywhere people will listen...

IoT is a security disaster

I hate to suggest things like this but we may need legal remedies to solve the problem before it becomes an Internet Zombie Armageddon (IZA). It's that bad.

There's going to be tens of billions of IoT devices installed at businesses and homes in just a few short years. If they're not receiving regular, timely security updates the Internet as we know it may just plain collapse from the sheer weight of all the zombies.

I propose a mandatory "nutrition label" of sorts for IoT device packaging. Something like this:

    * Updates itself regularly
    * Conforms to NIST security standards
    * Public penetration test results: https://ptr.iot.nist.gov/A2L7P9D23345
    * Shares data with 3rd party: No
    * Administration by vendor: No
    * Requires Internet connection to administer: No
    * Requires 3rd party service to function: No
    * Notifies of intrusion: Yes
    * Logs all access: Yes
    * Encrypts all network traffic: Yes
Something like a publicly-posted penetration test would be incredibly valuable. Especially if it was performed after each update of the device and at regular intervals. Even if it was an automated test (which in theory could be 'gamed') it would still be better than what we have now which is nothing!

As Dan Geer warned[1], embedded devices must not be both unfixable and immortal. Either they have some sort of remove management interface and receive regular security updates, or they must have a limited lifespan.

Creating embedded systems that won't be updated and live indefinitely is knowingly creating a hazard and nuisance to society.

[1] http://geer.tinho.net/geer.blackhat.6viii14.txt

Maybe we should leave the task of authentication, encryption and whatever phoning home these devices need to do to some kind of industry standard relaying protocol that lives on a router. You would then only need updating the router to keep any WAN activity safe for the IoT devices. Updates can also be delivered safely trough this mechanism, which lazy manufacturers would definitely still forget to do (but may be useful for keeping local network access safe , which would not rely on this mechanism).

That is, if router manufacturers could also keep their crap safe...

If you don't trust an off the shelf router for this, you could also use a computer setup as a router with such relaying software. It would be as simple as running a dhcp server or something.

Routers could also default to denying any authentication from WAN, with the user changing this policy on a per device basis or just allow eveything.

> or they must have a limited lifespan

This seems like it would give free range to sell shitty devices that are obsolete the moment they come out of packaging - or worse, that brick themselves on a timer for supposed security purposes (think HP ink cartridges.)

I agree, but the unfixable+immortal combination isn't really an acceptable situation.

The solution for most devices is to simply keep them off the internet. Being a host on the internet requires some level of care and maintenance so you don't become a hazard to others. I doubt most light-bulbs or thermostats are worth the ongoing expenses of maintaining security updates into the indefinite future. In theory, traditional goods that last more than a few years should be more successful on the market than self-destructing devices with dubious internet features.

This is fine, there is a market for that like anything else. The problem is where that is not a function the consumer is aware off at the time of purchase.

Now perhaps some good recycling laws would combat abuses of such an effect.

Tying IoT devices to cloud services almost guarantees a limited lifespan.

That only means they might eventually stop fulfilling their primary function. But they might remain plugged in and serve as bot without anyone noticing for years.

Very true. While I had assumed that most people would toss their gadgets once their functionality had been disabled, I wouldn't buy a new fridge or other appliance just because I can't load cat pics on it anymore.

Depends on the appliance. Your fridge will probably work when disconnected from the cloud. Your LED lightbulbs? I can imagine them getting bricked. I know of one laser cutter, featured some time ago on HN, that becomes a very big paperweight when disconnected from the Internet...

While this is a case of "two wrongs make a right"[0] - anything that provides even some value without the backend up will still be connected, perennially at risk of getting taken over by a bad actor.

[0] And a pretty strong argument against "forever" IoT devices.

Maybe the solution is some sort of prepaid credit for a cloud computing network? like lambda or something. That would at least shift the burden of staying running.

> As Dan Geer warned[1], embedded devices must not be both unfixable and immortal.

So when do you want me to turn off your colored LED light bulbs you bought? Yeah, thought so.

> you bought?

I assure you I have not (and never will) purchase any "internet enabled" (or "smart") device. Most devices worked just fine without a network interface.

To turn off my (standard) LED light bulbs, use the switch on the wall.

Well, probably it will be a security nightmare for a while, until the sheer weight of vulnerabilities and incidents causes vendors to radically change their security practices, regulation or not. I think a good example of what I would expect to happen is by analogy to the PC internet of the 90's, where Windows 9X viruses and worms were just a fact of life when using computers, and every email inbox was 95% spam each time you opened it. Remember that what is now considered basic security practices (UAC, https, "Are you sure you want to run this exe?") were at the time considered hardcore security measures that only enterprise systems needed to care about.

The difference is that now more of our daily lives depend on the internet being reliable, and, that some IoT device's failure mode might involve a cost in human lives. But the fact that the incidents are going to be worse will also probably put more pressure on companies to fix their security. Regrettably, I don't think this will actually happen until some startup's fancy new IoT gadget gets hacked in a way that causes it to kill a few hundred people and the resulting lawsuits end the company.

>ntil the sheer weight of vulnerabilities and incidents causes vendors to radically change their security practices

I've been in the professional IT game for 15 years. The "please run this macro in word to view content" issue still exists today and is one of the main vectors in ransomware. Heck, the "click this to view my photos" 'sexy-pics.exe' problem has not been solved in Windows either. End users don't care about UAC pop-ups, they click through them. See the recent literature on "Security Fatigue."

I think you have way too much faith in the industry to fix its own problems. Its clear that there are many classes of security problems that remain unfixed for decades. We're just seeing the newest version of the very same problems we've failed to address in the past.

I have no idea what the solution here is, but I imagine some level of certification is needed. Perhaps as simple as validating that you're forced to change the default password/keys and auto-update for security updates is on by default with some level of commitment to perform security updates for x amount of years after purchase. Toss in mandatory fail2ban-like functionality and you've patched the most common exploit use cases. I imagine that's a fairly hands-off approach here that won't interrupt business and will cost very little to implement.

On the end-user side of things, having a router that can detect when you're part of a botnet/DDOS and stop traffic would be valuable. Why aren't these $200+ Netgears and Linksys's running any sort of IPS? No IPS in this day and age is pretty crazy. An implementation of snort or similar wouldn't eat too much CPU and would only raise the cost marginally.

Its still very much the wild west out there from a security perspective.

You're suggesting to put a ("snort or similar") IPS console in front of the end-user who still clicks on the "click this to view my photos" sexy-pics.exe?

In my opinion, the end-user is not the one causing the problem. The manufacturer is causing the problem and -- just like we do things at $work -- he who causes the problems (should be the one who) suffers by being the one to also fix it.

The real solution for end users will be to add an actual physically different input path for the problem.

For example, imagine if home PCs and even laptops, had a special (macro) SD card like slot. This is the slot that the user put in their ownership card to get it to turn on at all, to log in to their OS account.

The computer would REFUSE to continue to operate with this card present. It would then enter 'normal mode'.

In normal mode no OS updates would process, no programs could be installed, no new scripts / downloads could be run. Trying to run a new thing would bring up the Authentication Required screen. They would need to insert the ownership card again. They would then need to expressly authorize what the new program could do.

That's the experience that end users for an appliance might desire; at least once they got used to it. I also strongly require that the end user is also the //owner// of the hardware. Not whoever made the OS/etc. All of the above controls would be in software, which could be replaced if they decided.

The problem is the IoT devices don't harm their owners or the company that built them but rather other targets on the internet.

It's probably not a great idea, but how about legalizing hacking for self-defense? If you get DoSed then those devices, are fair game.

That would shift incentives back to the owners to some extent, since they're basically acceptable targets as long as their devices are insecure.

This comment shows the disconnect between many comments I read on this forum[0] and the real world, frankly.

Whenever Apple or Samsung release an update for their smartphones, I have a to help family members click through dialogs to install them. How are they supposed to assess and implement security for an IP camera they bought at a department store?

How would they do it anyway? Is the expectation that they fire up Wireshark, identify traffic flows to and from their device, then configure the firewall on their consumer-grade router to limit this traffic?

[0] previously read just '... disconnect between this forum ...'

Please don't generalize from a single comment to "this forum". You're painting thousands of people with a very broad brush.

Fair point, have updated my comment.

I'm not sure how that would help the problem. When you're talking about thousands or more devices, counterattacking them individually wouldn't be feasible.

Also I imagine many botnet owners secure the devices they take over so that others can't steal them for their own botnet.

This is a Bad Idea, because you have no way of knowing whether the devices you are "counter-hacking" are simple IoT devices or if they perform a critical function of some sort. The last thing you want to do is accidentally take offline/brick an Internet-connected medical device or computer at a hospital ER room, for instance, since it might literally result in someone's death.

That sounds like a "but think of the children" argument to me. We can conjure up ways how anything can go wrong.

But should we really worry about that scenario? If you have internet-exploitable and already-exploited devices in the ER room, haven't the horses left the barn already?

Stolen IoT data is the perfect way to case a house. That's a market waiting to happen.

Do people still case houses? I know at least in my area burglaries are waaaaaay down because unloading the stuff inside is basically impossible. The few remaining burglaries are essentially smash the door run in and grab any loose cash, unlocked guns or jewelry and run out. To pull that off it seems easier to do it the old fashioned way, by ringing the door bell.

I'd be much more worried about identity theft & credit card theft with IoT devices than physical theft.

Those are the burglars who aren't able to peer inside multiple homes via xbox cameras.

Provided they continue to function as expected and aren't harming the network they're a part of. Both are big assumptions.

Yes, it really is amazing to look back at the slating UAC got when it was released under Vista.

And I _still_ have to disable it on every Windows machine so I can modify things in Program Files without jumping through hoops.

Why are you modifying things in Program Files? Are you using very old software that expects the user to modify that?

Reverse engineering, mostly.

I've been trying too. On Reddit I just get downvoted - which I actually don't understand.

I've also been warning governments about how bad things are and how bad things are going to be. I can see some movement, but it is too slow. We need more mandatory, automated penetration testing of all corporations by regulation. We also need cyber army reserves to help bring some of the knowledge that we have in the private sector to our governments without having to work in a shitty beige office downtown Ottawa.

A bunch of Teslas hacked at the same time is a weapon of mass destruction. We need to recognize that we have weapons of mass destruction online and take countermeasures now so we don't wake up one day to the news that 20 million cars accelerated to 200km/h and crashed into gas plants and government buildings all around the country. We need to regulate industry so the "move fast and break things" doesn't actually move fast and break things.

> We need more mandatory, automated penetration testing of all corporations by regulation.

No thank you, there be dragons there. Before you know it that regulation leaks to all software development that includes calls to connect() or bind().

But an Underwriters Laboratory for pen testing? Yes, please. (Or even UL themselves!) I saw an article on HN within the last 12-18 months about a similar concept/security auditing.

UL and similar agencies already check that plugging a device into your electrical system doesn't set something on fire. They have simple, yet strict standards that must be applied.

Likewise the FCC is very particular about radio emissions and will outright reject your product if it's too noisy.

We absolutely need something that can handle obvious problems like this, plus a way of coordinating with the vendors of these products should a vulnerability emerge that requires a patch or a recall.

> Likewise the FCC is very particular about radio emissions and will outright reject your product if it's too noisy.

That's only really recently true--just ask a ham radio guy about the noise floor if you want an earful. The FCC was pretty lax about this until this year when they finally quit accepting cross-certification from emissions labs from countries like China.

That's a slippery slope argument. I know regulating tech is not going to be popular here, and maybe we leave it for corporations that are valued at over a certain amount, but right now foreign governments are hacking our financial services companies, communications companies, phones, laptops, everything. 95% of the time a it's an extremely easy hack that would have been discovered by a routine scan.

There's an important distinction between "regulating" as in setting prices, and "regulating" as in setting the rules of the competitive marketplace.

For example: You can't sell phones that explode. You can sell phones that are cheaper than the competition.

Maybe IoT manufacturers or even the consumers owning the things should be liable for damages if their products participate in a DDOS. "Underwriters" labs proves you built your product to some standard, and that presumably makes you insurable because you have a first line of defense against a lawsuit.

But as opposed to EMC standards, software vulnerabilities arise periodically and standards need to be continually updated. Would devices need to be audited periodically to maintain certification?

> A bunch of Teslas hacked at the same time is a weapon of mass destruction.

Daemon[1] was a fun read, but I have no desire to see it become reality.

[1] https://www.goodreads.com/book/show/6665847-daemon

Fantastic book, great ideas.

And terrible prose :(

Among others, the right to explore and fix these devices ourselves.

That current abilities to do so, in the U.S. are arbitrated by the Library of Congress, seems entirely insufficient. Except that it perhaps gained us some temporary rights to do so that corporate lobbyists via Congress would otherwise already have taken away.

This tech has become essential to society. Intellectual property "rights" need to find their limits at a point before they put this essential function at risk. If the original manufacturers can't get this right, and where the underlying chipset technologies are from what strikes me as an oligopoly -- regardless of the number of brands and models they get stuffed into -- then the public should have every right to research, verify, and fix the products, themselves.

Ultimately and hopefully, this also works, financially. If you can't make it work properly, you lose control of your revenue stream, rather than using a government-granted monopoly (IP) and your income on lawyers and lobbyists to coerce people to suffer with your failure.

I've called for a mandatory physical "wifi off" switch for all Internet-connected devices that disconnects the antenna. Useful for the times when you want your light bulb to just be a light bulb, and not DDOS some journalist.

Some IoT and IoT-like things are simply completely non-functional without some sort of network connection. A physical switch is great for light bulbs, but it turns my child's crib-cam into a decorative art piece.

And when you would flip that switch, anyway? Wouldn't it make more sense to control this from your router? Something like changing the wifi password or network name would work even better (assuming you control the router.)

IoT should be Intranet of Things, not Internet of Things. I'd say it's simple as that. But the commercial interests are so opposed to engineering sanity that I have no idea how to tackle it.

Maybe someone should start enforcing mandatory VPNs for households? I.e. I get you want to control your LED bulbs from your phone, but that doesn't imply they should be accessible from public Internet.

Also I wonder if the whole thing isn't just a symptom of a problem that we can no longer expand connected technologies while expecting zero knowledge from end users. Maybe it's about time to require some minimum amount of computer knowledge from citizens of a technological civilization?

I think so too. Public networks are networks full of hackers and NSA agents. The VPN-like protocol should be built into device so it doesn't accepts any packets not coming from its home network. And VPN (unlike SSL) allows encrypting traffic for external obverver while keeping it plaintext for owner.

Maybe someone should start enforcing mandatory VPNs for households? I.e. I get you want to control your LED bulbs from your phone, but that doesn't imply they should be accessible from public Internet.

You don't even need a VPN, just have the device only accept connections from local IPs (and use TCP to avoid spoofed IPs).

That allows attacks comes from another compromised device or TCP SYN packet that uses unusual routing. We've already seen Javascript based attacks that use the browser as a gateway to the LAN.

You need strong authentication and secure (proven input recognizer[1]) code in anything that connects to any network, even a guaranteed LAN. Border security is only one layer of protection, and LANs are rarely an isolated network.

[1] https://media.ccc.de/v/28c3-4763-en-the_science_of_insecurit...

That doesn't work universally, for example when your device is at home and your smartphone is in another country. Using VPN allows to build a secure network over public infrastructure.

If something has a connection to the Internet--even if only through a proxy--we must treat it and all devices on the same network as if they are connected to the Internet. Because they are!

If Internet connections worked like radio broadcasts (one-way) then we'd be fine but they are inherently two-way connections. That means that for every outbound request a hole is opened up for return traffic. Even if you think, "that doesn't count" one must still account for the fact that PCs get compromised via side channels all the time and are subsequently used as pivot points for further attacks on any attached networks.

We need to stop treating intranets like they're somehow "safe" or special places apart from the Internet. They're not.

Always treat every device on any network as if it is being placed directly on the Internet because most of the time it really is. NATs and proxies aren't security tools even if they pretend to be.

I agree that for some devices it doesn't make sense. But having the power to take a device offline, and know in your bones that it is offline, is useful.

That doesn't mean that you shouldn't be able to control it all from your router too. But if your home router gets hacked, or there's a devastating crypto vulnerability, or you suspect your teakettle is ratting you out to the NSA, you should still be able to boil water with it.

Finally, a mandatory physical switch would remind people (and inform them) of just how much stuff they use is now net-connected.

I like your sentiment but merely disconnecting the antenna isn't good enough. That just results in bad reception which often results in power being boosted which can introduce a lot of unnecessary noise into the WiFi RF spectrum.

I don't see any way around implementing them as bog standard "RFKill" switches. A software switch is basically the only way to go because the WiFi chips are controlled in software.

Thanks for pointing that out. I'm happy with a software kill switch, as long as it's provably isolated from the rest of the system.

The nub of the idea is a legal requirement for a physical switch that will in all cases immediately disconnect the device from the network.

>we may need legal remedies to solve the problem

I doubt we can solve this problem legal way because all economic forces are against us.

Making a product secure costs many times more than making a product itself. "Regular security updates" is a nightmare to vendors. Nobody actually does it, especially on consumer market. Vendors react to massively exploited vulnerabilities at best. Any long-term maintenance, including security, simply don't fit into current business models.

And people want cheap products, they don't want secure products. Government contractors want super cheap products (with support but this doesn't change much). And education/propaganda won't change that until some big disaster happen.

I think there's an analogy to be made with the FCC (and non-US equivalents). IoT consumer devices are not general purpose devices. They are mass-produced and serve a specific purpose. They should be certified and accountable in a way. If not, they should get fined for being irresponsible.

If we can't force manufacturers and carriers to ship security updates to Android phones, there's zero chance of forcing it for light bulbs.

Get insurance companies interested. Force IoT vendors to buy insurance against potential damages caused by vulnerabilities in their devices.

Well, as I keep repeating...

Do not plug your "things" on the Internet unless there's a serious gain from it!

This solves most of the problem. The rest will only get solved by free software, and if the government should mandate anything, it is that the software can be managed by the user, not a laundry list of warnings that will mean nothing in practice.

(By the way, that "conforms to NIST security standards" is intended as a "yes" mean "bad" kind of warning, right?)

> Do not plug your "things" on the Internet unless there's a serious gain from it!

Why not? It's kind of a tragedy-of-the-commons situation.

I gain a fancy lightbulb that can change colors when I play with my phone. Do I, personally, really care if that lightbulb is part of a botnet that's beating someone's website into submission? Statistically, it's unlikely to affect me, so it's all gain for no cost beyond paying for the bulb.

Maybe you would care if that website was yours. These attacks do not discriminate. If your Wifi connected lightbulb (just writing that makes my eyes roll back) is part of a botnet, one day, it might just take down something that you actually need.

That said, them being connected to the internet isn't an issue itself. Make all your IoT devices connect through one central gateway (in your house) that controls all of them and suddenly instead of 20 devices broadcasting on the internet you have a single entry point. Obviously this entry point would be the target of many, many attacks, but securing this one gateway (that would be easier to update than your lightbulb.) is much less of a complicated task than securing all twenty.

But you would care if the lightbulb was caused to heat up and start a fire.

Sure, but manufacturers have adequate incentives to avoid that, because they will be held liable.

It's way more likely to affect you than that it making any relevant harm to anybody else.

Do you care about thieves checking your lights from anywhere on the city? And what about your lights launching attacks against your other computers?

> Do you care about thieves checking your lights from anywhere on the city?

Well, this article (and discussion) is mostly about these devices being sources of DDOS attacks.

> And what about your lights launching attacks against your other computers?

I thought my comment was pretty explicitly clear that most people would not care at all, even if they knew.

All very well in theory, but I'm sure it will suffer the same fate as ordinary nutrition labels - the general populace just doesn't care/isn't motivated enough to care

Bundle all of those points up into some sort of standard level of acceptance and let conforming products have a big shiny sticker saying so on the front of the package?

As a bit of an analogy, while I may enjoy poring over computer system security details, I'm pretty novice at a lot of more real-world things. Yesterday I was buying a new dryer exhaust hose. I read some reviews online, and looked at boxes in the store. I managed to learn enough to realize I wanted an all-metal construction so it wouldn't catch on fire. But I really wanted some big shiny signal to let me know "HEY YES IF YOU HAVE NO IDEA WHAT YOU'RE DOING, BUY THIS ONE!"...

The problem is going to solve itself, just like it solves itself already. Before volumetric DDOS became widespread, no ISPs cared enough to do anything about it, the risk simply didn't worth anything to put resources into dealing with it. Today it does and many deploy some measures to quickly react and at least null route targeting IPs to minimize impact on the rest of their customer. Some IXs and large networks even provide filtering capabilities for free to filter those attacks closer to the sources and avoid sending large volumes of traffic through their networks.

So, don't worry about it.

You say that like "ISPs" are some mythical non-human entity. SOMEONE has to make that decision, and the more educated on the topic of potential threats the person making the decision is, the better the reaction time and mitigation of issues will be.

The post above was publicly broadcast in hopes that the person who makes the decision reads HN, not because it is expected that the common everyday Joe reader will do something about it.

ISPs are well aware about these kinds of problems, but don't make decisions based on potential threats. It has to become a real threat first.

Correction: it has to become a real threat to their bottom-line first. Before that, there's no business incentive to care.

The European Commission is trying to push for something like that, but I don't know if it's going to be anywhere as "strict" as what you suggest. It may be just something like C, B, A, A+, A++ ratings for security, which wouldn't be too bad, unless what's behind those ratings are really some inadequate rules.

SSL Labs' HTTPS ratings work because the developer of the test is also constantly tweaking and optimizing for current "best practices", and the ratings are not too far off from what the sites should be rated at in terms of HTTPS security.


What mjg59 has been doing is utter genius. Buy something online, discover it is full of backdoors and violates the GPL, post utterly scathing review, buyers stop buying it. We need more people to do that.

While I'm not a security professional nor am I always a fan of adding bureaucracy or overhead if there is no clear benefit, I absolutely agree with you. Even if this information is not easily consumable at first to the layperson, it would be helpful to security professionals. It would also allow for easier comparison - at least from a security perspective. Let's face it, some IoT things like security systems get installed by installation companies, so I'm sure they would appreciate knowing more details about the various products, and which to recommend for their installations, etc. Furthermore, it would set a precedent, and who knows, maybe in the future the newer iterations (of these "labels/label info") could improve enough for the ease of reading/understanding by consumers.

If I put my consumer/layperson hat on for a moment...Even if there is no government mandate to add this type of "label" to these IoT products, if I see 2 comparable products side-by-side on a shelf (or ecommerce website), but one has this "label/label info", and one lacks it, I would definitely purchase the one which HAS the label.

If the traffic is encrypted how do you guarantee that the device doesn't spy on user and doesn't have backdoors preinstalled?

And why would a device need full Internet access? Shouldn't it be only accessible in home network with maybe some (user selected) servers from Internet it is allowed to connect to?

We could use some sort of VPN (from device to other owner's devices) so that even if it is connected to Internet directly it doesn't accept any packets coming not from its owner. VPN also provides encryption while allowing user to examine traffic in clear text.

I think we need some type of easyly deployed, ubiquitous, simple, snandartised VPN type protocol so that the users can build protected networks over public insecure infrastructure.

Today if you connect your device to Internet you connect it directly to NSA and chinese hackers.

And if your device is disconnected from public network hackers cannot break into it even if they have exploits.

NIST has too much influence from the NSA. There needs to be a better standards equivalent where the US doesn't wield so much influence in subverting security protocols. It may be a pipedream considering every Western country wants to be part of the five eyes program.

edit: The most democratic and open platform in the world was made by developers from all across the world vs US made operating systems like Windows and Apple which were basically trojan horses. We need an equivalent for security protocols, and protocols in general.

We can't let US stazi entrenched Silicon Valley and their faux "Startup Scene" destroy the internet as rapidly as they are today.


I agree with you. I was just using the NIST as an example because they've been in the news recently; they just released some new security standards that are actually quite good! Presumably we'd use an independent organization who's whole job is to come up with best practices for the security of IoT devices.

Semi-related aside: Microsoft Active Directory violates the NIST's new password hashing guidelines because it doesn't use a random salt when storing password hashes.

>> The most democratic and open platform in the world was made by developers from all across the world vs US made operating systems like Windows and Apple which were basically trojan horses.

By platform do you mean the Internet? The Internet was initiated by the U.S. Defense Advanced Research Projects Agency.

This will realistically not be a major problem. Most of these IoT devices operate only in a LAN or talk to a specific "cloud" service.

Thus, there's no way for an attacker to get access to services on them on a large scale basis (well, without hacking the cloud service, but hopefully someone can actively maintain that).

The cameras are only an issue because many people exposed them directly to the internet so they could access them remotely.

You will not see many people expose speakers, lightbulbs, blind controllers, thermostats, fridges, TVs, etc to the internet, so it's not exactly a major threat for most devices.

Not for now at least. If IPv6 becomes wide spread available to consumers, we're in for a whole new wave of attacks on these types of devices unless home routers are configured to block incoming traffic by default to devices behind them.

Your assumptions about the safety of LANs are wrong. LANs are not "safe" from attackers and NAT is not a security tool.

If an attacker compromises any device on your network you think they'll stop there? Hell no. They're going to pivot and compromise every other vulnerable device on the network and we're already seeing "crime kits" (aka customizable malware) with built-in exploits for things like networked printers and cameras. It won't be long before they have built-in exploits for light bulbs, refrigerators, and more.

Also, what about IPv6? To date, consumer routers that support IPv6 are configured by default to hand out global IPv6 addresses (as they should!) to connected devices via DHCPv6. What this means is that your IoT light bulb is going to get a globally-accessible IPv6 address.

You can setup the firewall to block inbound connections to your light bulbs, toasters, microwave, refrigerator, etc but who is actually going to do that? Firewalls are a huge pain in the ass to configure for zillions of little devices like that and in order for them to work properly you'll need to ensure that each device gets a reasonably static IPv6 address which means going deep into the configuration of your router.

Also consider that even if you have near perfect security for your LAN we're all constantly moving in and out of our homes with connected devices that could be compromised elsewhere. Laptops, phones, tablets, etc leave our homes, connect to who-knows-what free WiFi network and then return; possibly in a compromised state.

Never assume that because something is on a LAN it is "protected" by the mere fact that it doesn't have a direct connection to the Internet! If something else on that LAN has a connection to the Internet you must treat it as if it were exposed to the Internet because it is.

> If an attacker compromises any device on your network you think they'll stop there? Hell no. They're going to pivot and compromise every other vulnerable device on the network and we're already seeing "crime kits" (aka customizable malware) with built-in exploits for things like networked printers and cameras. It won't be long before they have built-in exploits for light bulbs, refrigerators, and more.

Yes, while it would certainly work on a targeted basis, it's impossible on a massive scale basis. You can't just go hack all smart lightbulbs because you can't just go hack all home networks. There's no large-scale threat here if a vulnerable lightbulb is behind a NAT, where it cannot be directly attacked. It might be a point where an attacker who broke in already could lurk, but it's not a new and trivial point of intrusion.

> Also, what about IPv6? To date, consumer routers that support IPv6 are configured by default to hand out global IPv6 addresses (as they should!) to connected devices via DHCPv6. What this means is that your IoT light bulb is going to get a globally-accessible IPv6 address.

I commented on this already: By _default_ routers need to be firewalling off incoming connections completely via v6 too, as they effectively do with v4, NAT is probably the only reason we're not still in the worm era. This is incredibly trivial for manufacturers to do, and I'm hoping the majority will do it. Seriously, it's like one iptables rule that needs to be in their config.

NAT _is_ a security tool, like it or not, if you're preventing your most vulnerable devices from being directly attacked, that's a damn good security tool. Not a perfect one by any means, but a very important one for the non-expert majority of home network operators.

Legislation would only be used as a vehicle to achieve monopoly control by large tech companies who would have their lobbyists write the law. The Internet would become a true oligopoly with other uses of the wire forbidden.

The only answer is to fix the Internet to make DDOS harder and less effective. Implementing standards to block IP spoofing is a start since it would make source quench actually work.

Another not mutually exclusive answer is decentralization. DDOS works because you can use it to take down all but the largest central servers or those that don't buy protection. But if apps are decentralized, an attacker can only play whack a mole with individual endpoints to far less effect. If decentralized protocols do mobility well victims could just switch connections or their ISPs could react by rotating IPs.

> Legislation would only be used as a vehicle to achieve monopoly control by large tech companies...

Food labeling requirements haven't lead to monopoly control over food. Why would IoT labeling be different?

Don't get me wrong, I'm cynical too! The lobbyists that control our legislators with purse strings will most certainly do everything in their power to ensure any such labeling only makes things better for their corporate overlords! I just think, "what's good for corporate overlords" is actually good for everyone in this case.

I would expect VC-funded businesses to reject any such, "Requires 3rd party service to function" label since they're all about locking customers in with "network effects". I would also expect established, old-school businesses like Microsoft to reject any label that would identify a product as "open source".

...but what corporate overlord is going to object to labeling like, "Regularly updates itself"? That's something that's cheap for them to do but can be expensive for upstart competitors.

There's much to be gained and almost nothing to be lost with labeling IoT devices.

Food is a commodity. It would be very very hard to accomplish the same kind of lock-in. Lock-in is easy with software because of its brittleness.

Maybe it's a sign of the times. Maybe it's a sign that I'm old and cynical. Maybe I'm just having a bad day. But what if the IZA is the kind of predictable, "9-11" scale event that "changes everything" - meaning lets the powers-that-be rewrite the rules of the game in a way that (further) empowers them and (further) disempowers the citizenry?

How surprised would we be to hear governments uniting in response to an IZA that the answer is a new, centrally controlled and managed "internet" - because terrorists, the economy, and think of the children?

The solution is going to have to come from the network side; getting the DoSing devices automatically blackholed by routers. I know there are many technical issues with implementing something like that at scale but it seems more tractable then trying to enforce security practices on $20 web cams.

Yeah, but routers are a shitshow in and of themselves, with plenty of vulnerabilities. It's like using a sieve to plug a leak.

How can it update itself regularly without an Internet connection to a third party?

Also, "logs all connections" is tricky. Why should my fridge need a hard drive? How much space must be available, and how far back should it store logs?

Will your IoT lamp stop working if the vendor goes out of business? That's what "Requires 3rd party service to function" means.

It's not contradictory with, "Updates itself regularly".

You've got me thinking about this though... If I were selling an IoT device today I would likely just outsource the majority of the self-update problem by using an open source Linux distribution as the basis of my product and merely configure it to perform automatic updates.

The product's software could be updated via a separate software repository (e.g. like a PPA). That way it would get automatically updated at a regular interval along with the OS (assuming I put out regular updates).

This would actually be an ideal setup for consumers too because if I went out of business or my software stopped getting updates the consumer could just replace the software performing the IoT functions or merely remove it and at least be left with a functional embedded computer to hack around with.

Obviously most consumers wouldn't do that but it'd be nice to give them that option.

Replying to "logs all connections" separately...

Who said anything about logs necessitating the need for permanent storage? My router doesn't have a permanent log... It just keeps a buffer of the last N log entries in memory (e.g. `logread`).

Alternatively, I can (and have) configured it to forward its logs to a central server on my home network. Syslog forwarding and collection has been around for a really long time. It may be tricky to meet the "encrypts all network traffic" certification and syslog forwarding at the same time though since encrypted syslog normally involves a PKI.

The whole Certificate Authority concept doesn't play really well with IoT, actually.

If we want non-tech friends and relatives to buy the right things, we need to boild down to traffic light label, or something like EU energy label for score out of 8 (that manufacturers dislike, and had to be recalibrated when >50 of appliances were A rated - guess it works well too).

To me, we should have IoT devices on a different protocol not everything http(s). Nothing gets out unless you got an IoT passthrough router.

An IoT hub that shows me what devices, connections, and manages what can connect to where and when. QoS and other neat features if needed.

It'll never happen now of course, but would sure beat the current fustercluck.

Good luck with that. Most Android devices stop receiving updates after one major version upgrade. It will never, ever happen for all these extra devices. Stop dreaming.

As someone novice in network security, is there any public penetration testing tool that someone can recommend, Ideally open-sourced

Kali has a boatload of open source tools:


The original Internet was a "security disaster". The solution was not to certify every device that got connected to the Internet. Instead, we invented firewalls.

Firewalls were designed to keep bad traffic from entering LANs. For the IoT era, perhaps what we need are firewalls that keep bad traffic from exiting LANs.

> Firewalls

Sigh. Firewalls worked great for Yahoo, Sony, and LinkedIn, right?

Firewalls are just a layer. They don't even do that much!

Not only that but the average consumer knows nothing of firewalls. Do we expect Joe Average to be able to login to his router and configure a firewall to allow his refrigerator access to its security update server? Or maybe we should expect Jane Average to configure her Firewall to perform MitM attacks on SSL so traffic going to/from her IoT coffee maker can be inspected?

You are seriously under-representing the impact of firewalls on the security of the Internet. Every consumer router has a built-in firewall. They don't need to know anything about it because the default configuration works for most consumers. That's what we should be aiming for as a solution for the IoT security problem. Something that just works for most cases without configuration... much less extensive labeling with commensurate demands of technical knowledge and attention on the part of every consumer.

They are not perfect (nothing is), but a Internet without firewalls would be apocalyptically worse.

Most consumer routers have a built in firewall, yes but I've never seen one outside of my own house that's actually configured to block anything. Have you? No one configures them!

No, NAT is most decidedly not a firewall.

Firewalls are a horrible non-solution. The actual solution was that MS started taking security seriously after it started hurting their reputation.

> perhaps what we need are firewalls that keep bad traffic from exiting LANs.

You mean like just about every firewall in existence today?

This as QR-code with an app that somehow helps you out. Just a thought

My initial thought was to provide the penetration test/general security info URL as a QR Code =)

    <QR Code Here>
Something like that... Easy for anyone with a phone to scan or someone at home to enter by hand if need be.

Your username is very fitting

your first and fifth points are an oxymoron. Obviously nobody wants their dishwasher, washing machine, fridge, microwave, to put up messages, "is it okay to update now?" - can you imagine dozens of devices in people's homes doing that?

so your first point, automatically updates, implies your fifth point, administered by vendor, should be "yes". (Not "no", as you've put it.)

Further, I would like you to think more broadly about infrastructure. I think every person's home should have a local network called "shitty backdoored and malicious IoT crap". The question is how to administer that network, what its topology should be, how these devices should be able to receive information from the Internet, and in general what their capabilities should be and the way in which they should be firewalled off.

Having these devices on users' home wifi via their home router (the status quo) is obviously wrong and insufficient. I'm afraid your checklist is also, therefore, wrong and insufficient.

so, think harder at the architectural level, please. the existence of IoT stuff is obviously good. (as in, obviously there can be a benefit to devices being able to be connected - in the abstract, not based on today's very flawed infrastructural and architectural choices) the infrastructure we have for it today is obviously insufficient. I don't have a comprehensive solution in mind or anything but it is worth giving serious attention to by people who have spent years on the subject.

today, they are failing. It's not that people are ignoring their advice: they just don't have any. They've thrown up their hands and given lists like yours, which are not a comprehensive solution and are not even internally consistent. They need to think outside the box and propose completely new infrastructure.

there needs to be different infrastructure in place, not an SOP for the current infrastructure. we need better security baked into the topology and architecture of how the devices are connected.

> Obviously nobody wants their microwave to put up a message, "is it okay to update now?"

Then why are you putting it on the internet?!

Either you create devices that are secure - which may include things that neither the manufacturer or the user want to do - or you shouldn't be putting it on the internet.

> I think every person's home should have a local network called "shitty backdoored and malicious IoT crap".

Relying on border security is a terrible idea. A firewall is not an excuse to ignore host security.

Right, so with the binary choices we have today you've given a good analysis. but there are other choices when people think outside the box and experts create new infrastructure. I think we're on the same page and you see some of the same issues I do, we're not going to come up with a goood solution in this comment thread.

pretty sure we can at least agree that all IoT choices available to users and manufacturers today are hugely problematic for one or both of these parties.

> Obviously nobody wants their dishwasher, washing machine, fridge, microwave, to put up messages, "is it okay to update now?"

No, I don't expect anyone to be asked to update. I expect it to work just like Chromecast devices do: It updates in the background while in operation and applies that update at the next reboot. If idle for long enough it'll reboot itself (usually in the middle of the night when no one is expected to use it).

Self-update mechanisms aren't rocket science. In fact, there's so many FREE options to choose from at this point you can just take your pick. Don't need to code anything on your own!

Sorry, that's bullshit. Automatic updates are one of the worst ideas about IoT devices, because the updates almost always remove functionality. Manufacturers have been known to remove key functionality, or just straight out brick recently sold devices when they have liability concerns or their business plan changes and they don't want people using the old devices anymore.

  - http://www.theverge.com/2016/4/4/11362928/google-nest-revolv-shutdown-smart-home-products

  - https://nest.com/support/article/Nest-Protect-Safety

I agree with you, and my point was that the list I replied to is not well put-together since we must remove the fifth requirement, that the manufacturer should not administer the device. Yes, they should. (Chromecast being an example.) so we're saying the same thing - it's just in conflict with the suggestion that manufacturers shouldn't administer devices remotely.

This doesn't solve the security issue though, it just moves it to the self update mechanism.

Lack of updates create a security issue. In fact, I'd argue that update mechanisms are the most critical security issue of our time!

By neglecting to update devices you increase risk by ensuring vulnerabilities won't get fixed. You also might not get notified in the event of product recalls (if you never registered your purchase).

I understand your argument though: By solving one security issue we're creating another: Trusting the vendor. To that I have this to say...

If you don't trust the vendor why did you buy their product in the first place!?

Updates cannot help against zero day vulnerability. A better solution would be not to allow anyone to connect to a device.

Lack of choices? Nearly all PCs and phones come with crapware I don't trust.

* Open source: No * GPL violating: Yes * Tivoised: Yes * Backdoors: Yes

There is always the risk of too many standards. People doesn't know even now what all the green labels mean on the food they buy.


> People doesn't know even now what all the green labels mean on the food they buy

You don't need to know what they all mean, you only need to know what the ones you care about mean.

Interesting. This set of attacks is coming from real IP addresses. (SYN floods and single-packet UDP-based attacks use a totally bogus source IP address.) The device can be talked to and potentially located. Somebody needs to be finding some of those devices and analyzing them. How are they controlled? IRC? Some set of known domains or IP addresses? A self-healing mesh network? All of those have been used, and most modern botnets have a quite sophisticated control network. This provides both robustness and conceals the actual controller. In modern botnets, everything is encrypted, so it's hard to take over the botnet.[1]

Attacks from real IP addresses are potentially blockable upstream. That's only a temporary fix. The device, if behind a NAT, can request a new IP address and evade the block. It may do so periodically, just to confuse the issue. At least you can block a hostile IP at a firewall, so it can't use server resources. That's what Cloudflare is doing.

On the legal front, it's worth suing makers of vulnerable devices for negligence. Their EULA will not protect them, because the victim isn't their customer and isn't a party to the EULA. A few court orders to recall a product might get the attention of the IoT industry.

[1] https://www.damballa.com/downloads/r_pubs/WP_Botnet_Communic...

On the legal front, it's worth suing makers of vulnerable devices for negligence.

Where does one sue the no-name Chinese manufacturers of these devices? I have a feeling a lawsuit won't get very far in Shenzhen Municipal Court. There's absolutely no consequences to them building these devices and you cannot expect a solution from them.

Nolo Press: "In between the manufacturer and the retailer, there may be any number of wholesalers, suppliers, distributors, or other "middlemen." Each and all are part of the chain of distribution of the defective product, are therefore potentially liable, and should be named as defendants in your defective product lawsuit."

Cloudflare can start by suing a retailer. That's likely to result in the retailer pulling all merchandise from the offending vendor, even before the lawsuit gets very far. (Otherwise, they move to a higher tier of negligence and damages for knowingly allowing the problem to persist.) That will start to get the attention of manufacturers.

See this article.[1]

[1] http://www.techrepublic.com/article/lock-it-down-ddos-attack...

Lots of this crap gets shipped straight from China by a distributor on Amazon. Try to sue them and I'm sure they'll just disappear and come back with a different name

Sue Amazon?

Sue the importers then, and/or file ITC complaints.

Who says there is an importer and not a chinese retail site or ebay seller?

To my knowledge most of the C&C protocols are unsophisticated [0].

For now there is little evidence that says the bots are behind NAT's. I'm pretty sure this will come though. In the blog post I stated that there is indication that port 23/telnet is or was in past open.

Blacklisting IP addresses is indeed effective for L7 attacks. It's harder for L3 when the source IP's could be spoofed.

[0] https://medium.com/@cjbarker/mirai-ddos-source-code-review-5...

Don't sure the makers.

Sue the people who own and use the devices.

The first few times will be ugly and unfortunate, but if it becomes clear that buying a vulnerable device and installing it in your home is a potentially life-ruining liability, people will quickly stop buying and using the devices. And without a market for insecure IoT gadgets, well, one way or another we won't have insecure IoT gadgets.

How do you decide who to sue? If your device is part of a botnet there may be thousands of owners partially responsible. Most of them are probably in different country.

All you have to do is hit a few people in the US. This is, ironically, one of the few cases where looking up IP address and ISP records works, since the issue here is not "some random person on my wifi pirated a movie", but rather "you owned this device which was part of a botnet".

Once a few Americans have been hit with ruinous lawsuits, the market for insecure devices will collapse, worldwide, overnight.

...Are you seriously advocating to cause financial ruin to many persons for buying a device they didn't know was vulnerable and being attacked by someone?

I don't even know what to say. Aside that you're clearly not targeting the right thing.

I'm saying that if someone seriously wants to end insecure IoT gadgets, there is nowhere else in the chain, other than the end consumer, to usefully target. Manufacturers are typically overseas in jurisdictions where the appropriate regulations won't reach, and retailers can be bypassed by online sales and delivery. So if you want to use the threat of lawsuit to discourage insecure devices, the end user who purchases the device is the only entity in the chain who can be targeted.

And, you have to admit, it'd be effective: the market would instantly collapse if device owners could be held liable for damage caused by their devices.

Strangely enough, I think the solution is in devices like OnHub, Google WiFi, or other third party managed routers (yes, another IOT device).

Why? When attacks like this are first detected, they can be shut down at the source. Known bad devices can be preemptively locked down behind a firewall. Traffic to specific IP addresses, or packets which spoof headers to cause amplification attacks could all be detected and dropped before they even reach the ISP.

Yes, there will be problems and bad diagnoses, no, I don't want one of these managing my home network; but it would be perfect for my mother, for my non-technical friends, and for the average Joe with a subverted IP camera.

I like this idea a lot. Perhaps every IoT device should publish a profile of its expected behavior to the router. For example, a wifi-enabled thermostat might tell the router that it only intends to connect to a particular range of servers, by DNS name. The router could then set up an automatic firewall rule that stays in sync with DNS changes. The user should be able to disable that firewall rule (to overcome technical issues), but most users will never have a reason to do so.

We could bootstrap this idea by having the router fingerprint IoT devices and apply matching firewall policies from a database. If only we could install "apps" on routers, like we do on Android and iOS devices, someone could get started with this idea right now.

The only issue is that after that point, the device can never connect anywhere else.

So allow the device vendor a way to push a configuration change?

Then the attacker would push that configuration change before DDoSing someone.

Well, I would hope that any IoT device that accepts remote configuration or software updates also checks digital signatures before applying changes. If it doesn't, then I would want the automatic firewall on the router to block all updates.

Your router would notify you of the change and ask if it was intended before applying it, maybe?

I realize how close to Vista's UAC this is getting.

Make the configuration change come from the approved endpoints.

This is actually a really neat idea.

In another comment[0] I described how I keep my cameras away from the internet. Someone like Google could easily build something into their routers and firewall them off as they're added to the network. The firewall could allow them to talk to the appropriate cloud service, and nothing else, in order to better secure them without the user even knowing it's happening.

This doesn't solve all of the IoT problems, but it does help some for the one described in the article.

[0] https://news.ycombinator.com/item?id=12685381

> Strangely enough, I think the solution is in devices like OnHub, Google WiFi, or other third party managed routers (yes, another IOT device).

Until Google decides that your Android traffic get priority over iOS traffic.

Which is a more likely problem--a big company trying to control my stuff (see the Phillips LED Light kerfuffle) or my device participating in a botnet?

I can mitigate the botnet by turning my device off. Giving a company control of my devices isn't so easily mitigated.

That's a problem that will only get worse. To give an example, last year, Incapsula recorded ~9,000 IoT cameras attacking them. A few months ago, Sucuri recorded ~25,000.

CloudFlare is seeing close to 50k. And that's the attackers just using a small portion of their real power for http floods.

Our report from a few months ago breaking down the types of cameras and networking doing the attack - very similar to what CloudFlare saw:


*I work at Sucuri.

I've got a bunch of cheap Wifi cameras and I don't worry about this kind of thing at all.

All of the cameras live on a separate Wifi network that is unable to connect to the internet. I use Zoneminder to handle all of the recording, motion detection, etc. For remote access, I've got a VPN server running. Fortunately, the cameras I use are perfectly happy to work without being able to talk to the manufacturer's servers (though they do try!).

In a nutshell, I get almost all of the "benefits" of the cloud service (the only missing thing is off-site storage, but a fix for that is in the works) with almost none of the drawbacks.

Of course, this is NOT easy for the average consumer to set up. They expect to be able to plug the camera in and have things Just Work, which is why we've got cameras that can open ports on routers/firewalls, register themselves with cloud services, etc, with minimal effort.

As for IoT in general, my future plans are to use this separate WiFi network for all IoT-like devices that don't really need to connect to the internet. The Philips Hue is a good example - it has a bunch of cloud-enabled functionality, but I don't actually need any of it.

It sounds like you actually worry about his kind of thing a great deal, considering the amount of effort you put in.

Believe it or not, it was a whole bunch of little things that added up to this state.

When I started with the cameras I was just using the motion detection and email features that were built into the firmware. I never bothered setting up the cloud stuff because I didn't need it. Slowly I transitioned to having the cameras dump images onto a local NAS via FTP. Eventually I started using the cameras for more than watching the cats, so I needed something with better motion detection and recording capabilities. I started with 'motion', then used a fork of 'motion' that seemed to work better, then finally ended up with ZoneMinder.

On the VPN side, I decided many years ago that I wanted remote access to my "stuff" - file shares, etc. This started out with the PPTP VPN built into my router, which is terrible but sort of worked. The next step was writing a script that would run SSH and set up a couple dozen tunnels, which is differently terrible but sort of worked. Eventually I got tired of maintaining that and ended up with an IPSec VPN using racoon -- split DNS and split tunneling are truly amazing.

The extra Wifi network actually started as a misguided attempt to prevent other stuff (Netflix, mainly) from interfering with the cameras. I figured that putting them on a separate Wifi network would isolate them and give them dedicated bandwidth. My first attempt involved using a Raspberry Pi as an AP. It worked, but not very well at all. The second attempt used the Virtual AP functionality on my router -- it wasn't until I set this up that I understood that Virtual APs don't quite work this way. Around this time I started to see articles about cloud-enabled cameras doing bad things and decided that this setup was worth keeping.

The camera and VPN stuff both went on over the last 5-10 years or so, with the Wifi changes being something I did within the last year.

You're right that there was a lot of effort, but as you can see it wasn't really driven by privacy/security until the end. Most of these things were driven by trying to find better ways to scratch various itches.

The next planned change is to re-address my network so that it doesn't use the range that every consumer router seems to use by default. This way I will be able to use VPN when I'm visiting friends without having to tether to my cell phone. Like everything else, it's just scratching an itch.

Similar setup. Two IP cameras, one wired with PoE and one WiFi. Both connect to my NAS via their own network that has no direct access to the WAN but can be accessed from the LAN or via VPN. I guess there could be some level of attack surface if someone is dedicated enough to gain access to my "normal" network (the one for PCs and stuff that access the WAN) and through this, gain access to the cameras.

But they don't announce themselves to the WAN and despite the fact that I may be naive, I don't think there is any reason I would be an attractive enough target for someone to try to gain roundabout access via a PC or phone on the WAN-enabled network.

Same thing with the Hues. No "cloud" stuff enabled and still work fine in the house where I need them to work.

I guess I like the idea of network-accessible devices (at least ones that make sense to control or access via the network) but there isn't much that I want accessing the WAN compared to the items that have the ability baked in.

I've annoyed several friends by pleading with them not to buy those "just connect to your wifi and view on our iPhone app!" cameras because I've seen how easy it is to find and exploit these via Shodan and friends. Most people don't ever even open the web interface to set up a password since it clearly shows up on the phone app and "it's just so easy!"

If the iot devices are on your same subnet they could easily be scanned and compromise by any compromised device with Internet access. This stuff is bundled up in toolkits that require little to no interaction by your attacker

Yeah, I guess I ought to reassess things. With the automated stuff, the "least obvious target" thing sorta loses its effectiveness.

So you keep your cameras devoid of the updates that the manufacturer might release? You also restrict your cameras from talking to other fellow cameras.

The manufacturer's update process is to download the firmware image from their site, then connect to the camera with your browser and upload the image. This process works fine, it's just not automatic. I don't think the cameras themselves have any way to automatically update anyway.

Yes, I restrict the cameras from talking to each other too. There's no need for one camera to talk to another in my setup.

A general solution to IoT security would be for all IoT devices to only communicate to your personally owned home gateway, which would run open-source drivers for each device to provide the networking/external communication functionality. The IoT device could even be assigned its own isolated network link to the router (i.e. sandboxed).

That approach seems like the only logical conclusion for the device owners. I hope this is how the field will eventually evolve.

While its against the commercial interests of "Internet data" companies (who might subsidize devices), in the long term regular consumer appliance manufacturers will implement it if theres enough market pull for it.

That is kind of what Nest's Thread protocol allows IoT devices to do. But strangely enough, Google is going the opposite direction with its devices, including coming up with such stupid ideas of controlling your (OnHub) Wi-Fi router over the Internet through an app.

Thread is just an IPv6 wireless mesh routing protocol right [1][2]?. On its own it doesn't solve 2 key security problems:

* Any device can send arbitrary data to somewhere on the internet

* Internally, any devices can send arbitrary data to any other device


Another less-critical but important usability limitation is the short sighted nature of how device logic + APIs will evolve long-term for Device-to-Device communication.

Device-to-Device implies the application protocols are running on the devices themselves, which can't be bugfixed or upgraded by the user and in the majority of cases will likely never be updated by the manufacturer after release.

In [2] they give examples of devices like fridges and thermostats talking to each other. There's 3 long-term ways I see of handling device-to-device application APIs:

* Option 1: Raw device-to-device (Short term): How the hell will those APIs be agreed upon and evolve across multiple years and device products? Bluetooth now barely handles relatively simple interop use cases (files, contacts, audio...). Will Thermostat X support Fridge Protocol 8? How will the user handle more advanced use cases?

* Option 2: Vendor managed (Currently the most likely): The devices will talk to the vendor servers which will talk to other vendor servers which will talk to your other devices. Terrible for the user for so many reasons but great for the vendor: its the simplest to implement, keeps protocols secret and locks user into giving that specific vendor endpoint all that juicy data.

* Option 3: User managed (IMHO Preferable): Computer drivers are a much better model for this. Drivers sit over devices to transform the weird device API (fixed target) into a consistent interop API (community driven moving target) that does what people want. This still supports device-to-device: device A -> API call on gateway driver A -> gw driver interop API -> API call gateway driver B -> device B. This is what Option 2 (vendor managed) will actually have to do under the hood anyway. This could be implemented in a simple way e.g. with per-device(type) containers exposing a TCP/UDP port to the device.

I predict Option 3 to become mainstream when the year of the Linux desktop lands :D


[1] https://www.threadgroup.org/portals/0/documents/resources/Th...

[2] https://www.infoq.com/articles/thread-protocol-for-home-auto...

(Thread is a poorly chosen name for a new computer protocol:P)

I have a few cheap chinese ip cameras and I noticed that they have telnet open (I subsequently rooted them to fix this) and of course have them behind my home firewall.

However this and other IoT devices got me thinking that maybe it's time to start an underground device hacking organization solely committed to not only rooting and reverse engineering these devices but also releasing new firmware for all of them with open source internals. I'm sure this would have to be "underground" due to all the legal restrictions around the embedded device drivers.

Who wants to do this? I wouldn't know where to start besides a .onion site...

>not only rooting and reverse engineering these devices but also releasing new firmware for all of them with open source internals

I had the same idea, except in my version automated scan finds exploitable IOT devices and updates them with new firmware straight from /dev/null. Problem SOLVED.

Actually it would be a nice thing to do. Wonder how long would it take manufacturers of all this crap fix their shit after massive warranty claim flood came in

People already release custom firmware for routers. I'm not sure what the manufacturers think about this but I'm sure the same community could be built for IoT devices.

The manufacturers would probably be upset they don't get my data to analyze anymore.

Does Cloudflare still host (edit: proxy) the companies that do DDoS attacks thus serving to justify the use of Cloudflare services?

We don't do hosting.

Quoting Brian Krebs:

> These two services, like most booters, are hidden behind CloudFlare, a content distribution service that lets sites obscure their true Internet address. In case anyone cares, Lizardstresser’s real Internet address currently is, at a hosting facility in Bosnia.


So they don't do hosting... What you've quoted just explains how they act as a layer in front of the actual server

Host/proxy - a non-relevant detail. I misspoke.

Either way, Cloudflare appears to both help booters and then also sells services to protect you from booters.

Cloudflare's CEO spoke about this here: https://www.youtube.com/watch?v=SWFX-zEYwN0

An almost hour long talk. TL;DR?

The talk is about how CF protects free speech by allowing anyone on their network, good or bad. Only a US court order can force them to not deliver content for a specific website.

Examples are given how Amazon removed Wikileaks from their servers without court order, how another CDN removed content because they had other clients that didn't like that content etc.

I cannot judge how honest it all is, but I'm impressed and do believe their intent. One of the scary things is what happens if Google, Microsoft or Amazon decides to buy them. Or some shady Russian or Chinese investor...

Cloudflare CEO equates protecting booters with protecting free speech.

Definitely has nothing to do with the fact that a healthy market for booters is the best salesman they could ask for.

(I work at CF)

I usually agree that spending 1h on a talk may be much. But I found this one to be really good. Matthew is both entertaining, and revealing some things that weren't public before.

I'd say the 1h is worth it.

I think the point of a TL;DR is to allow those short in time to still take some points away, or to give an overview and entice to consume the entire thing.

I'd say "Cloudflare recommends Cloudflare" is pretty uninformative while a TL;DR isn't.

Sometimes there isn't a TL;DR. Like explaining cancer or quantum physics in 30 seconds misses most of the important details.

Sure, but GP seemed to focus on "I know best how to spend your time", whether intended or not. Irksome.

Thanks! That was a very interesting video, worth the time.

The tire store both sells you new tires, and sells tires to the guy that slashed them.

Cloudflare thinks that everyone should be immune to DDoS, both good people and bad people. I agree with that sentiment.

They're not helping the booters do booting activities.

> Connection: close

well if they would have keep-alive it would be probably even more disastrous. especially if the botnet would have some intelligence that it would try to detect the keep alive timeout and send requests exactly in this time frame.


also a lot of these devices are probably built with linux. and here is something that is probably bad with linux, the lifespan of most linux iot devices are probably outspanning the official lifecycle of any distribution/kernel. this may lead to problems that the vendor will just abadonned it or doesn't care anymore, if he even provided a way to upgrade. on windows this is probably as bad as well since most vendors will probably not pay additional fee's after the next windows iot gets out of live.

it's probably time to abadon most os's for these devices and create a new open source project for these kind of devices, hardended from scratch. this would at least help with the unmaintained part of the os parts, which would make the situation at least 1% better. well it won't fix the laziness of certain vendors.. but it would be a beginning.

The OVH write-up [0] is very interesting combined with this, as they had to withstand an even bigger attack. Unfortunately, they don't publish as many statistics. But apparently their attack came mostly from Spain, so different AS than Cloudflare.

Their plans to upgrade the Ddos mitigation sound very interesting, will be interesting to see how long it takes until they need the 5Tbps:

>will be replaced by a new technology developed in-house and based on FGPA (programmable integrated circuits) and codes that have been in development for the past 18 months. This will allow us to offer up a VAC capable of withstanding DDoS attacks with peaks up to 5 Tbps without slowing down our network.

[0] https://www.ovh.com/us/news/articles/a2367.the-ddos-that-did...

Internet is decentralized so there is no authority that could stop DDoS attack. We need a protocol that would allow a host to require peer or upstream network to block/allow traffic from specific IPs or subnets. And of course the denied traffic should not be paid for.

Seriously? I get the cameras scenario keeps coming up, but most of these (if not all) are CCTV systems with a DVR with open remote management that either:

- Is turned on by default (bad)

- Is turned on without enabling even the slightest of security by the end user (stupid but still bad)

- If a or b aren't true, it has some sort of exploit that the company will never fix (if they are still in business)

The "rest" of IoT in many cases doesn't have this problem at this scale. You're either talking about proximal networks with no outside internet (z-wave, 15.4, etc) or something with secure communication with the home service (PKI for example).

I'm more and more frustrated with IoT fearmongering, if you want to talk about CCTV fearmongering? OK. But these aren't Nest cams, or Arlo cams. They're off the shelf DVR CCTV systems.

At the moment a ddos is like an open weapon that can be deployed by nearly anybody to further any agenda; governments, extortionists, and anyone with an axe to grind.

But putting the burden on all devices that connect to the network is idealistic. Sure it may work for the 99 billion devices, but what about the 1 billion devices that may not be properly secured. This wack a mole strategy rarely works.

There must be a way for edge nodes to quickly detect a ddos and shut it down.

We should be extremely wary security hysteria often used by people with agendas to gain control and lock things down. Only 'approved' devices can connect to the network or everyone on the internet must surrender their passport to the nearest authority and be surveiled 24/7 to prempt and prevent criminal operations.

The first attacks were described as only lasting minutes. Is this the perpetrators stopping? If so, why? In case people start to notice their 'smart' lightbulbs are glowing unusually bright? Otherwise what else stops the attack?

Proof of ability to potential customer? Checking that their command and control software works?

I doubt you'd notice a difference on the device unless you were doing something network/CPU intensive on it, even then, it'd just seem slow.

Ah OK that makes sense, thanks. It's handy for companies like Cloudflare to get a heads up on what's around the corner.

If you have failed to take a site down in 15 minutes, continuing for longer is not going to make a difference. Also the attacker may be renting time on the botnet.

Nobody seemed to mention it, but the large payload query is designed to also DoS the hell out of old/insecure PHP binaries that will blindly accept (and keep appending) as many items as you want. (In this case the array $a).

Similar attacks probably work against other CGI parsing things that don't expect to have hundreds of thousands of qps each with long querystrings.

The only way I see IoT security being taken serious is when someone makes a bot that hunts down insecure IoT devices and permanently disables (bricks) them. Only then will customers will force manufacturers to take the subject of security seriously.

Does anyone if there's a commonality amongst these IoT security cameras that is facilitating this? Is it just non-sensible defaults or is something more like the embedded webserver/SoC?

I think the biggest shared threat is the ability to punch holes in firewalls with upnp. That and the fact that they're running ancient unpatched software stacks.

Indeed, that gets back to not sane defaults though I suppose. The filet-of-firewall was particularly troubling, Flash and DNS rebinding is all it takes if UPnP is running on your router. This worth read if anyone is interested:


It's also unclear to me if the devices are being connected directly to the public net - I would have expected most homes and businesses to have some kind of firewall. I suppose it's possible that UPnP is exposing a web interface and/or telnet.

How about this for a fix? Just thinking out loud here...

Let's make a protocol that allows a server to specify to its upstream ISP to cut off all traffic from a particular IP address. The ISP can either soak up the traffic, or go further up to its own upstream ISP and relay the request, and so on until the source of the attack is reached. Now it's between an IoT device owner and his residential ISP, it's up to them to sort it out.

To motivate all ISPs to play along let's make a law for each ISP to be financially responsible for failure to honor such requests, enough for a victim to pay for defense plus some %% in penalties.

Sort of an automated "Ceases and Desist" letter, and a failure to abide is an automatic tort.


Let's be a hacker and abuse the protocol and shutdown the internet

BGP flowspec works kind of like that. Also routers don't have unlimited capacity to hold everyone's filtering rules, don't forget that.

Your typical large-scale attack has a few thousand IPs at most, so it's technically doable. All that's missing is the motivation for everyone to abide. With enough incentive hardware makers might accommodate, too.

But how do you know which ips? To request for in the first place without banning legitimate users?

It shouldn't be hard to tell a person from a dos robot. Person can authorize to get whitelisted, or solve captcha. Persons rate of requests is much lower than that of a bot.

The problem is what to do once abusive IPs are identified. Right now you will pay for all the bandwidth used.

Time probably time to name and shame brands. CP Plus is doing so much marketing in India with TV slots.

They should probably invest that in doing some research and patching .

If things get too bad, can cloudfare pool with google, etc. and just block the IPs of those affected? (until someone fills a captcha)

AFAIK Cloudflare only deal with HTTP and HTTPS, so nothing else goes through their proxy.

Which doesn't matter anyway because the traffic is still going through the edge networks (the ISP and a little bit upstream from there), so is still bandwidth impacting at some point.

We really need a way of getting the ISPs to talk to the providers upstream, and a way of notifying back that a particular host is causing problems. Unfortunately ISPs don't want to do this because unhappy customers = revenue. Also I don't want my traffic analysed on the way through, so there's a freedom issue on this too.

So it's not an easy problem to solve, but the ISPs could do it. But they won't.

On the flip side dealing with this sort of stuff is why i'd like to work for Cloudflare, Google, Facebook or others. It's fascinating how technical solutions can be bypassed by customer or provider inertia.

It's high time we start fixing criminal liability for companies building such poor quality software and hardware.

Seems somewhat ironic that an article about DDoD is published on CloudFlare's blog.

CLoudFare seems honestly interested in making the Internet a better place, one that does not need its current services. I guess they can pivot easily into a cheap bandwidth provider, so this doesn't hurt much.

I salute their help and everything, but it's always important to keep in mind that helpful companies are just an acquisition away from antagonizing us.

Ironic? To me, it seems quite fitting that a company who has "protecting you from DDoS attacks" as one of their major selling points would publish an article about (recent and relevant) DDoS attacks.

Ironic in the sense that they protect companies offering DDoS as a service.

I'm not so sure "ironic" means what you think it means.

Their graphs mention Mrps and Krps, any idea what the R stands for?

Requests (i.e. HTTP requests).


We detached this subthread from https://news.ycombinator.com/item?id=12684368 and marked it off-topic.

Seriously? That comment is worthy of Beavis and Butthead. And not in a good way.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact