Hacker News new | past | comments | ask | show | jobs | submit login
You Can't Trust Network Time (diode.io)
58 points by dominicl 27 days ago | hide | past | web | favorite | 54 comments

DHCP already has support for providing an NTP server IP; there has been at least one proposal[1] to also provide a DHCP option specifying the "rough current time", so that devices have at least _something_ to start with. As for trust, you're already going to be trusting DHCP to give you an IP address, you could also trust it for this (with some safeguard in place to catch known-wrong values).

This kindof reinforces how having an 'IoT Gateway' would be a good idea: The gateway can provide DHCP, and rough time, and (S)NTP, and DNS, and can also act as a proxy to the outside.

[1]: http://www.potaroo.net/ietf/idref/draft-ogud-dhc-udp-time-op...

> This kindof reinforces how having an 'IoT Gateway' would be a good idea: The gateway can provide DHCP, and rough time, and (S)NTP, and DNS

Aka a router. Am I missing something?

I doubt most routers provide DHCP. Maybe consumer ones.

What's the point of this nitpick?

Networks that configure everything statically are fairly rare, and do so intentionally. If they don't, they have a DHCP server somewhere already, and if they aren't consumer-level networks they sure as hell don't want some random "IoT gateway" box providing DHCP. So it's clear what "router" means in this context.

I have a separate dhcp server so if the router has issues machines can still attach to the network and talk to local services.

Most layer 3 switches provide DHCP, and since almost all consumer routers also provide DHCP, probably most routers do provide DHCP.

In my experience the majority (all the ones I've owned at least) of consumer grade routers have DHCP.

Yes, as routers generally only speak Ethernet and WiFi while IoT devices speak Zigbee, BT/LE, Powerline, 433 MHz custom protocols, a bunch of low power WAN protocols and for the more professional/industry settings CANbus, Profibus and more.

AFAIK Powerline is Ethernet, so that’s covered.

Zigbee, BT/LE, 433Mhz and friends don’t do IP, so they won’t need DHCP and DNS.

So again... What am I missing?

I run NTP at home, using an inexpensive USB GPS dongle as the time source.

Unfortunately it's technically challenging to create a configuration to use it in a meaningful way, especially when the system reboots.

Really what you need is a process to bring up the GPS, wait until the time is up and reliable, then set the hardware clock and launch NTP "correctly" (without falsetickers and all the other ntp convergence nonsense)

I run NTP at work, more specifically, we use GPS time sources to synchronize a wide-area paging network (by wide area I mean over an area nearly a quarter of a million km square). Timing is extremely critical for our application to work correctly over an area that size.

The only viable option for us is about 3 (or more) rackmount NTP appliances from a well known vendor which have the GPS already integrated, support for multiple time source modules and power supplies, and have integrated software to drive the whole thing. This means we need to get GPS antennas installed on the roof of the datacenter and fiber run from there to our racks.

Our remote sites (connected by 4G and VSAT) also have GPS units, but they are connected via RS-232 serial (not USB) and suffer from some of the synchronization issues you describe. They fallback to the time servers in the datacenters in the event that the local GPS time source is rejected.

Hmm, a quarter million km square really would be a wide area: 62 billion square km, or just about the surface area of Jupiter, if Jupiter had a proper surface.

A quarter million square km seem more plausible.

Well I'd like to tell you it was the former, but in truth our paging transmitters aren't powerful enough to reach Earth from Jupiter. Thank you for pointing out my typo.

If your service area ever expands, NTP has got you covered.

It's the most crufty software in the universe, but it supports interstellar time. :)

Next post: "Why You Can't Trust DHCP"

Do you trust your local McDonalds with that kind of access?

If you have no time information (i.e. no rtc), then yes

Articles that go around in circles while avoiding reflection or synthesis are infuriating. Of course relying on the PKI isn't going to magically solve every problem. But the article wasn't titled "Why you can't trust the PKI", because we're all bored of reading about that.

At a certain point, you must choose some immutable trust root. If your kink is to use public CAs (fuck me harder, (go)daddy), it implies attacks. If you chose to setup your own private signing key, it implies attacks. If you choose longest sha256 chain ("blockchain"), it implies attacks. If you simply stop trying to retain absolute control over devices after they're owned by someone else, it implies attacks.

You can hardcode further constraints as mitigations, but the fundamental drawbacks remain. If you want direct synthesis from the immediate complaint, keep a bit of a state to ensure that the date only ever moves monotonically forward. However you'll still be open to an attack if the device is off while a certificate is compromised.

This is a genuine problem across a variety of platforms, not just IoT. There really is no great way to "solve" for trust when one party might have extremely stale information. Awareness of this problem is surprisingly poor.

> In the next blog post of this series we want to introduce a new secure network time protocol based on blockchain technology and show how even the smallest devices can establish completely secure connections.

This is a pretty disappointing way to end the article. Most "solutions" end up being some form of obfuscation, I'd guess this is too.

A verifiable way to put bounds on time is a very useful thing. If you know the time within an hour or even within a day can be a big help. There are cases where I’ve used firmware build time as a lower bound for current time.

There is only a catch-22 if we need _that time source_ to verify its time.

That's not how things work, though: we have hundreds available, so there is no catch-22 whatsoever: we check multiple sources and go with the majority vote, then flag an error for any outlier(s).

If your wifi connection is MITMed, your adversary could MITM all the connections, even if you make hundreds.

If you have trouble imagining a user taking their wifi lightbulb to a coffee shop, imagine instead you're a device vendor and a customer is trying to MITM their own device to embarrass you in a Defcon presentation...

Sure, but now you've turned it from "You can't trust network time" into "you can't trust your wifi" and while 100% correct: now you're commenting on a different article altogether.

How do you know that it’s not the majority of your sources that has been mitm’ed?

Not all of these sources come through the Internet. Think GPS, GSM, radio time signals (DCF-77 or the likes).

Those solutions all require additional hardware - hardware that's far more expensive than an on-board real-time clock and battery.

If we can't get precise time securely and do not have real-time clock, it does not mean, there are no options left.

For instance, device may be booted for the first time in secure controlled environment and synchronize time fro trusted source.

Then, every N seconds (like 60 to 3600), device will write current time to local file. Most devices have some kind of writable local storage.

Any next synchronization will not be able to return time before already experienced moment. Certificate expiration will actually work, with a bit higher tolerance of a few hours.

Adam Langley's Roughtime protocol[1] largely solves some (but not all types) of time malfeasance, and a variant of it is already an IETF draft specification[2]. There is also NTS, which is secure NTP and provides some protections, but not nearly as many.

[1]: https://roughtime.googlesource.com/roughtime/

[2]: https://tools.ietf.org/html/draft-roughtime-aanchal-01

An (interesting, but impractical) solution to this problem would be to use astronomy to deduce the current date first, which should be good enough to validate the certificate.

See this QA on Astronomy StackExchange: https://astronomy.stackexchange.com/questions/12886/get-date...

One could pin a cert, hope the device is powered on in the validity period to update it? Obviously a bad idea for other reasons but maybe it's the best worst choice.

Maybe dumb, but... Why not just make ntpd actually check certificate revocation? Easier than fixing everything that uses HTTPS, and seems to fix it?

1. The device doesn't have a battery, so when it boots it thinks the time is Jan 1, 1970. X.509 wasn't even invented at that point so there are no valid certificates for that date.

2. The device is being MITMed so it receives a time of, say 2010 with a certificate that is also valid for 2010 even though that certificate is long expired in reality. The device has no way to know otherwise.

I don't know the point of this attack though.

>1. The device doesn't have a battery, so when it boots it thinks the time is Jan 1, 1970. X.509 wasn't even invented at that point so there are no valid certificates for that date.

Defaulting to date of manufacture instead of unix timestamp 0 would get you closer, although it's still not a solution.

You could also periodically store time in a non volatile memory, to be loaded on boot.

OpenWrt can use the last modified timestamp of a file, eg the most recently updated log file. It’s a decent middle ground.

Right, but Let's Encrypt certificates are only valid for 90 days. And there's loads of electronics in the world that's been powered off for more than 90 days :)

Yes, there's lots you can do to mitigate the risk but nothing perfect.

Which, realistically speaking, is quite fine.

I understand that we can't validate by time. But we can go fetch revocation lists.

Wouldn't those revocation lists be MITM'd as well?

You’d need to validate the revocation list. Now you’re right back where you started.

As mentioned downstream, a cached revocation list could be served. Additionally, certs that expire are not commonly revoked, and revoked certificates may be removed from revocation lists after they expire.

Assuming something in the trusted side of the network chain has a persistent clock or verified time then this issue could be avoided. For things that connect directly to untrusted networks e.g. public cellular internet the only solution I can see is using devices with a persistent clock themselves.

So ignore the time on the cert? Isn't that better than having no cert?

The reason I think they're effectively equivalent is say I have an IOT device that it's command and control is on example.com.

When using NTP and having stolen a certificate in the IOT devices trust store for example.com. I spoof the time and then I can spoof example.com because the devices clock is now in the validity time for the stolen certificate.

When using server time over TLS and having stolen a certificate, the device on cold boot (Or all the time depending on how well written it's implemented) will connect to the server ignoring the validity time, and again get it's clock set to a time that matches the certificate.

Either of these methods require compromising the private key for the cert. Basically the weakness is using time to limit the window for a compromised cert is a lot harder on IOT devices which routinely don't maintain their clocks.

Now, if the TLS based time service was running under a different cert, it might change the surface area a bit. Since the cert for the TLS based time service would also need to be breached, along with the example.com certificate (2 certs in theory should be harder to breach, but it depends on the org and how they leaked the secret in the first place).

This also depends on the IOT device manufacturer not signing a certificate for 25 years that will basically never expire regardless of the time (which I've seen).

I'm probably the only one who thinks it's really stupid to connect things like refrigerators and washing machines to the internet, but I'm going to say it anyway.

And it's even stupider to give the user zero local tools to interact with them short of letting them loose on the big bad internet.

I wouldn't mind if my washing machine could ding my phone when there's a load of damp clothes in there that I need to handle before they grow mold. That's cool, I'm forgetful, and I waste a lot of water being that forgetful. But it shouldn't need to talk to the internet to do that.

The problem is not smart appliances, per se, it's the centralized and vulnerable way those smarts are being implemented. Because it's in the manufacturer's interest to compromise my own security and autonomy.

Which is why I don't own any, and I'm sitting here working on my own washing machine door transmitter instead.

> The problem is not smart appliances, per se, it's the centralized and vulnerable way those smarts are being implemented. Because it's in the manufacturer's interest to compromise my own security and autonomy.

Ding ding ding. Most things could be easily implemented without adding external dependencies or impacting privacy but there's no coin in it.

Mine has an LED that indicates “clean” and slaves from the internal door switch. Handy source.

This could be eg. an SNMP metric/trap, along with other sensors in the machine, then parent could add the "smartness" via their own software.

You're not, but that's only a small piece of it. IP cameras, smart speakers, tablets, e-readers, smart watches, etc., all have this problem in some form or the other.

It's a perfectly fine thing to do if your refrigerator or washing machine has software from a company that (1) has sufficient expertise to do software well, (2) prioritizes keeping it secure, and (3) commits to supporting it for the entire usable life of the appliance.

But usually 0 out of 3 of those are true.

The devices come from companies that specialize in hardware, so the software is usually an afterthought.

They don't prioritize keeping it secure because they know customers are more interested in features like whether the refrigerator's shelf configuration or the washing machine's capacity.

And they don't commit to supporting it for the lifetime of the product because once they have your money, they have little incentive.

You are not the only one.

As a matter of fact, I run NTP at home and get the time from an inexpensive USB GPS dongle. (I actually use more than one)

Unfortunately I believe you need 4 to do it right with the way NTP seems to work

Nope, it's dumb. No white goods or appliances (including HVAC!) in my house should ever depend on an internet connection.

As always it depends what you mean, I'm all for having IoT devices on my local network, exposed in safe ways through some kind of gateway device. This models most homes in some way today anyways.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact