
The Internet of Unprofitable Things - popey
http://strugglers.net/~andy/blog/2018/12/24/the-internet-of-unprofitable-things/
======
alexandros
Device fleets are incredibly hard to get right, and if you have no
updateability, you have to nail them the first time. For someone coming from a
cloud background, it's a jungle out there, where all sorts of "easy" or
"solved" problems are nothing of the sort.

As a sort of PSA, I want to plug the work that we have done at balena.io
(formerly known as resin.io) to solve problems like these for everyone
deploying Linux IoT devices, including open source / non-paying users.

We've built the open source balenaOS[1] and the (newly) open sourced
openBalena[2][3] management server that anyone can use without paying us a
penny. If you're about to manufacture a fleet of linux devices, and are about
to put them out there, and don't want to pay for a service (so you're not
using balenaCloud) but also don't want to solve problems we've spent years
already finetuning, (such as ntp, dns, cellular modem support, read-only
filesystem, host updates, etc etc) use the above projects and save yourself a
mountain of pain in the ensuing years.

I hate being alarmist, but working with device fleets over the last 7 years,
I've seen things you wouldn't believe. I've seen devices catch on fire, drown
in storms, launch DDoS attacks, and everything in between. It's incredibly
hard to underestimate how bad things can get when you put your devices out
there with no ability to find them and update them if push comes to shove.

[1] balenaOS: [https://www.balena.io/os/](https://www.balena.io/os/) [2]
openBalena: [https://www.balena.io/open/](https://www.balena.io/open/) [3]
openBalena github: [https://github.com/balena-io/open-
balena](https://github.com/balena-io/open-balena)

~~~
zestyping
This looks really cool. I may be deploying a fleet of Raspberry Pi or similar
servers in the near future and this might save us a ton of headaches.

The target context is a medical facility in a remote location without Internet
(or extremely limited Internet), though. Is there a way to distribute updates
by USB stick?

What's the best place to read stories of people who have tried deploying with
Balena on RasPi-type servers and/or contexts without Internet access, and
learn about their successes and difficulties?

Thanks!

~~~
walrus01
_please_ do yourself a favor and don't remotely deploy rpi or anything that
uses microsd cards for OS image and boot. The write lifetime of those, even
using "industrial" cards, will mean a huge failure rate even with very light
usage. Only use some sort of real sata3 or nvme bus SSD.

~~~
sdenton4
There are ways to reduce the risk, though, depending on your workload. It is
possible to use the sd card just to boot, with all actual file systems in a
USB attached device. You can also use some of the (very limited) memory as a
ram disk for things with lots of writes...

~~~
walrus01
Yes, but even so, the write wear leveling algorithm in a $50 consumer class
SSD and dwpd (drive writes per day) are vastly better than microsd media.

~~~
milesvp
Will second this. A friend of mine had to write his own wear leveling software
to make use of SD cards in his hardware, sadly bom made anything better way
too expensive to be commercially viable.

~~~
dogecoinbase
If your friend has written about this anywhere or is able to share any code, I
would love to learn more!

------
ttflee
The way they burned the NTP addresses reminded me of Douglas Adams quotes "The
major difference between a thing that might go wrong and a thing that cannot
possibly go wrong is that when a thing that cannot possibly go wrong goes
wrong it usually turns out to be impossible to get at or repair".

~~~
Carpetsmoker
I think that a pretty neat idea to solve all these time-related issues would
be more digital wristwatches.

~~~
ttflee
But making sure that there is a wrist besides each clock is yet another
problem.

~~~
Carpetsmoker
That's what the towels are for.

------
HocusLocus
Good a place as any to mention my short story

[https://slashdot.org/comments.pl?sid=7132077&cid=49308245](https://slashdot.org/comments.pl?sid=7132077&cid=49308245)

possibly the only work of fiction ever inspired by NTP, IoT and cryptographic
protocols.

This struggle with embedded ip address seems to echo this part of my story,
"These hastily made things flooded the market and soon replaced other well-
documented things. At times something failed and its inventors could not say
why, they just assembled a new one or went bankrupt."

~~~
SuperPaintMan
I absolutely loved this, thanks for posting it.

Do you have a list of your stories avalible?

~~~
HocusLocus
NSA and the Desolation of Smaug
[https://slashdot.org/comments.pl?sid=3863455&cid=44005849](https://slashdot.org/comments.pl?sid=3863455&cid=44005849)

Paced by the Animals
[https://slashdot.org/comments.pl?sid=4352911&cid=45171567](https://slashdot.org/comments.pl?sid=4352911&cid=45171567)

The Trix Rabbit of Thorium
[https://hardware.slashdot.org/comments.pl?sid=11375261&cid=5...](https://hardware.slashdot.org/comments.pl?sid=11375261&cid=55580891)

------
Waterluvian
I got butterflies reading this because it's something I can see myself having
done early in my career.

A relative newbie finding himself responsible for nontrivial design and
implementation decisions for fleets of robots. Luckily they were always
updatable. But if you asked me to set up the NTP story for them (which they
had but people smarter than me worried about it) I would have Googled it for a
while and just hoped that I didn't miss any fundamental understanding of how
to use NTP.

p.s. this article felt like it was the _perfect_ length. It shares the perfect
amount of detail succinctly.

------
reaperducer
I like that story.

The author responded to the initial problem in the old-fashioned internet
"we're all here to make things work" kind of way, without letting himself get
taken advantage of.

And then when the problem decided not to play nicely, he increased the
pressure in a civilized way.

These days most companies would have just said, "Sucks to me you" and cut off
the dummy IoT company.

This also illustrates why big companies like Apple maintain their own NTP
services.

------
pjc50
I'm trying to coin "Postel decentralisation" for things like this: people
assume NTP is a distributed robust system, but in practice it turns out to be
run by one very overwhelmed guy in a basement somewhere.

(It _could have been_ set up properly to be distributed here, but they didn't
do it)

~~~
bcaa7f3a8bbc
NTP itself is not decentralized, but the largest NTP provider is the NTP
community pool, www.pool.ntp.org. The pool is operated by MANY PEOPLE IN
BASEMENTS AROUND THE WORLD, using DNS to distribute the traffic. It's
practically pretty robust for most purposes.

However, people just go to a list of NTP servers, then copy a few into the
code, instead of using the distributed pool. Then it's not a surprise that the
NTP in a product is going to stop working, meanwhile a one very overwhelmed
guy who happened to run one of the server is going to have serious troubles,
see
[https://news.ycombinator.com/item?id=18753835](https://news.ycombinator.com/item?id=18753835).

------
parliament32
Good on the author for playing nice with them. Hardcoding IPs for _any_
purpose is a bad idea, this is literally the reason DNS exists.

~~~
hn_throwaway_99
I don't think that was really the main problem in this case. Hardcoding any
address (DNS or IP) _that you do not control_ is always a horrible idea.

~~~
ams6110
Does anyone really control their IP addresses? I thought that ultimately those
were controlled by ARIN, RIPE, et.al.

~~~
alaties
If you have your own dedicated IP range and you are advertising routes
correctly, then you're as close to ownership of your IP addresses as Google or
Facebook are of theirs.

ARIN and RIPE control allocations, but do not control routing to allocated
ranges.

~~~
jsjohnst
I think the point GP was making is that you don’t “own” IP addresses, you
effectively “lease” them. If you don’t pay your ARIN (or others) dues, you’ll
lose them.

~~~
blfr
Sure but that is also true of a domain name.

~~~
jsjohnst
Domains are much more vocal about expiration in my experience, but maybe
that’s just the registrars I’ve used.

------
jsjohnst
It’s so simple to setup your own Stratum-1 server, I don’t get why more folks
don’t. The one I keep in my house is a 1U unit about the size of a 16 port
switch and keeps time accurate to 15ns via a small Multi-GNSS antenna (yes,
serious overkill for home needs, but I already owned it for a use case that
used to need it).

~~~
bcaa7f3a8bbc
Unfortunately, having more Stratum-1 servers does nothing to help clueless
device manufacturers who hardcoding the IP address of an individual server,
and damaging the community service. But yes, when we talk about the NTP
community in general, we certainly needs more Stratum-1 servers. Nowadays,
running a Stratum-1 is easy, the hardware is cheap, a single broad computer w/
time source from a shortwave radio or GPS. And if one loves electronics, one
can build a precision oscillator (evem a Rubidium standard can be used for 300
USD), or go straight to make a GPSDO to achieve highly-reliable timekeeping.

Nevertheless, there are two practical difficulties.

1\. Static IP addresses are unavailable in most home connections. And many
home broadbands throttle UDP traffic, dropping them when the pps rate is high.
It makes one's home unsuitable for hosting a NTP server.

2\. Unlike Stradium-2, you cannot simply use a "cloud" service for your
dedicated server. To run a NTP Stratum-1, you have to physically host your
server with the customized hardware in a datacenter, which costs 100
dollars/month in my city, not including network transit and bandwidth. I
really want to run one, but I cannot afford it.

3\. Shortwave / GPS reception is usually not available in a datacenter, and an
antenna installation is usually not allowed. You can be creative, a good way
is using the time provided by mobile basestations. But it needs experience.

~~~
Rjevski
> a good way is using the time provided by mobile basestations

That would definitely not be Stratum-1, so I wouldn’t recommend it.

~~~
bcaa7f3a8bbc
Thanks for reminding me. If a mobile basestation is not usable, it seems to me
that the best way is trying a WWVB-disciplined oscillator. Shortwave is
certainly not available indoor, but longwave reception is possible. Hopefully
it won't be killed by NIST due to the funding issues...

~~~
jsjohnst
I’ve never been successful getting long wave reception in a data center
(didn’t try running cable to an external antenna though, as then why not GNSS
if that’s doable).

------
walrus01
It's referenced in the article, but one of the earlier ntp abuse cases:

[http://pages.cs.wisc.edu/~plonka/netgear-
sntp/](http://pages.cs.wisc.edu/~plonka/netgear-sntp/)

Makes for interesting reading.

------
bcaa7f3a8bbc
In short, if you are ever going to make an embedded device, or an operating
system distribution, or anything with NTP default on, please make sure,

1\. NEVER, ever hardcode an individual NTP server (in form of a IP or domain).
DO NOT just go to a list of NTP servers, then copy a few into your code. DON'T
ping pool.ntp.org and get its IP address written down. DON'T DO ANY OF THESE!
PLEASE!

2\. DO NOT use Stratum 0 and Stratum 1 servers. Please use Tier-2 and lower.
Practically, if you follow Rule No.1, then you are always following this rule.

3\. If the scale of your system is small, in hundreds, or in a few thousands,
PLEASE USE pool.ntp.org, this is the NTP community cluster backed by DNS load
balancer. Always request the DNS, and make sure the IP is not cached locally
for too long. If you need more than one servers, use 0.pool.ntp.org,
1.pool.ntp.org, 2.pool.ntp.org, etc (3 is often enough).

4\. If the scale of your system is large, such as tens of thousand, or you are
making a new system, you SHOULD request a customized prefix from pool.ntp.org,
such as debian.pool.ntp.org, it helps the community to manage the traffic. If
your system is a large commercial one, you ARE REQUIRED to donate some servers
to the NTP Pool to compensate the community. Another option is running your
own private NTP cluster. The policy is here:
[https://www.ntppool.org/en/vendors.html](https://www.ntppool.org/en/vendors.html)

5\. If possible run a standard NTP implementation, like NTPd, chrony, or
something else as long as it's written professionally. Nowadays even
lightbulbs run Linux, then why don't you run a standard NTPd?

But If you can't, then make sure...

(a) implement NTPv4, DO NOT use NTPv1.

(b) Read the new SNTP RFC if you are implementing an SNTP client.
[http://www.faqs.org/rfc/rfc4330.txt](http://www.faqs.org/rfc/rfc4330.txt)

(c) DO NOT synchronize time on the beginning of an hour, or 00:00 UTC! Select
a minute in a hour randomly for synchronization.

(d) Use an exponentially-increase retrial interval, DO NOT keep retrying when
the server is unreachable, you are launching a DDoS attack!

(e) Support Kiss of Death packet, your client should immediately stop
requesting a server, cease and desist, once a KoD packet is received.

(f) Make sure the client will stop requesting the builtin list of servers,
once an alternative server is set by the user.

These should have been written in all textbooks related to practical
networking lectures, but apparently, there aren't. People don't even realize
that their actions are harmful, and we have all the problems...

\--------------------------------

The NTP community is a complete tragedy of the commons. Even many government
institutions cannot keep up with the abusive traffic, and stopped providing
public NTP servers.

Today, if we don't count Microsoft and Apple's NTP, almost all public NTP
servers are provided by the volunteers from
[https://www.pool.ntp.org](https://www.pool.ntp.org). By using DNS, it forms a
NTP cluster to distribute the load. These people provide time for the entire
Internet, and they are the people who withstand all the abuses day by day.

People just assume they are some random super servers that always work,
without being responsible for their actions, such as hardcoding IP addresses,
writing abusive retry code (without exponential increment of timeout), and
making a cronjob that initialize a synchronization exactly at midnight
(without randomization), effectively a DDoS.

Usually, if a device comes with hardcoded NTP addresses, it, in fact, usually
indicates their program is poorly-written, and the manufacturers are
irresponsible. Those devices have the worst homebrew NTP implementation on the
planet,

1\. They send ancient NTPv1 packets, while the latest version is NTPv4.

2\. They synchronize their time on the beginning of an hour, effectively
making a flooding attack. Another larger flooding attack starts at 00:00 UTC.

3\. They retry interval is around 3 minutes, if fails to reach the server,
make even more traffic to the server, rather than an exponentially-increase
interval.

4\. They still try to talk to the default hardcoded servers, even if an
alternative server list is set.

5\. They don't support the Kiss of Death packet, nothing can stop them if they
became wild.

Stratum 0/1 servers are the most vulnerable: they have highest accuracy, with
reference clock. Despite the acceptable usage of ST-1 is only passing time to
downstream, or for scientific purposes, since there's only a handful of them
and often listed publicly, they are often spotted by those manufacturers, and
put in their devices by default.

Stratum 0/1 are usually provided by universities, or unpaid volunteers for the
public good of the Internet. If a single server got hardcoded in those mass-
manufactured devices, serious consequences can happen, the volunteer may
literally bankrupt: your whole institute/school will be kicked out from the
Internet [0]; when you came to the manufacture asking to pay the damage they
are responsible for, you are threatened by a lawyer from California. [1] The
whole Internet community should honor the spirit of self-sacrifice of these
NTP volunteers.

The NTP community pool is Stratum 2+, suitable for general use. It has similar
issues of abuses - once you're in and became well-known on the net, there's no
way out and you keep receiving bad-traffic, because some clueless people have
hardcoded your IP address, or has put it in a cache that never expires.
Fortunately given a reasonable bandwidth, it is often a negligible issue and
safe to ignore. But there are exceptions. [2] One of my NTP server became
DDoSed one day, because an ISP cached the IP address for pool.ntp.org with a
large TTL, and the IP address happened to be mine! The traffic was 40 Mbps...

In contrast, NTPd has proper rate-limit mechanism built-in, such as KoD and
good pooling interval, blocking NTP does NOT causes more user traffic. What
increased is the abuser traffic. The damage caused by a standard NTPd and
silly sysadmin is much less significant and is negligible compared to the
Internet of Scary Things.

By the way, not only hardware devices can contains dangerous NTP code, but
also software.

As long as manufactures still write broken code and unaware of the proper way
to use NTP, nothing can be done to solve this issue. Many involved in these
misuses and abuses are totally unaware what they are doing. The proper way to
use NTP should have been written in all textbooks related to practical
networking lectures.

[0]: Flawed Routers Flood University of Wisconsin Internet Time Server
[http://pages.cs.wisc.edu/~plonka/netgear-
sntp/](http://pages.cs.wisc.edu/~plonka/netgear-sntp/)

[1]: Open Letter to D-Link about their NTP vandalism
[https://web.archive.org/web/20060423012837/http://people.fre...](https://web.archive.org/web/20060423012837/http://people.freebsd.org/~phk/dlink/)

[2]: Recent NTP pool traffic increase
[https://mailman.nanog.org/pipermail/nanog/2016-December/0895...](https://mailman.nanog.org/pipermail/nanog/2016-December/089588.html)

~~~
wpietri
On the one hand, I completely appreciate this. When I set up NTP, I'm careful
to think through consequences and tread lightly. After all, I learned my
sysadmin skills at a university before the September that Never Ended, so it
really was a collegial place back then.

On the the other hand, the internet is a much bigger place. Things are orders
of magnitude more complex. The feedback loops that made NTP work well in a
1990s university environment are mostly absent. When a problem happened then,
I'd see something in the logs or in packet captures, figure out what was
happening, and get the responsible person quickly on the phone. That's not
even hazily possible these days.

As much as I'd love to think putting a stern warning in textbooks would fix
this, I doubt that would matter at all. What we really need is a major
increase in observability or traceability. And failing that, what we'll get is
common resources getting sliced up so they fit within domains of
accountability.

~~~
bcaa7f3a8bbc
I do agree. A complete solution is designing a new generation/revision of time
protocol with accountability and anti-misuse as parts of its design, just like
how people are implementing ASLR and NX in C programs and starting to use
memory-safe languages. Or just see how TLSv1.3 removed all unsafe algorithms,
so it will be little damage even under the worse case scenario.

Also, I think NTP needs more publicity. We need people to be aware of it
before we could get feedback. The community then can have a watchdog team that
spots misuses and publishes alerts.

~~~
Animats
_designing a new generation /revision of time protocol with accountability and
anti-misuse as parts of its design_

It would be "time.google.com" and you would need a Google account.

~~~
bcaa7f3a8bbc
Or perhaps RoughTime when security is needed?

[https://roughtime.googlesource.com/roughtime](https://roughtime.googlesource.com/roughtime)

------
starbeast
Immediately thought of Bruce Sterling's article on anticonventional objects. I
guess this company goes in the buildable, but neither desirable or profitable
section. - [https://www.wired.com/2013/10/design-fiction-
anticonventiona...](https://www.wired.com/2013/10/design-fiction-
anticonventional-objects/)

------
sokoloff
What RTC drifts so much in a week (or even a year) as to throw off lighting or
HVAC controls timing?

~~~
pstrateman
An RTC chip without temperature compensation in an extremely variable
temperature environment.

~~~
sokoloff
How far would a lighting control or HVAC control have to drift before it
became impractical to use? I could imagine 45 minutes could still be “eh, who
cares?”

45 minutes in a year is under 1 part in 10000.

~~~
TylerE
I can imagine _5_ minutes being well into _I care_ territory. Could be the
difference, say, between pulling into a dark drive way or having light.

~~~
sokoloff
Program the lights to come on 30 minutes earlier then. An overcast day and a
new moon vs a day with high cirrus overhead and clear skies to the west almost
surely has more variability than 5 minutes' difference.

~~~
TylerE
If it drifts at 5 minutes per week you still have to reprogram after 6 weeks.

~~~
sokoloff
Sure, but that's ~500ppm drift, which is fairly extreme.

~~~
freeone3000
HT1380 drifts 10 seconds every 20 hours under normal conditions. I don't think
this is outside the realm of possibility if they chose a cheaper chip or had
weird temperature fluctuations.

------
Scoundreller
1\. 70 GBP/hr seemed like a low rate for a consultant (what’s an electrician
or a plumber cost on a subscription contract?). I’m glad author doubled it.

What happened to NetThing’s customers after they ceased trading? Who took over
the lighting management of the car parks etc. ?

~~~
grifferz
Author here. I'm sure you're right. I pretty certain I could have charged them
much much more and they'd still have accepted it.

In conversation with the software eng it was implied that they intended to
send someone on site to each of over 500 sites to reimage the devices. That
must have cost them way more than £70/month and the way that after ~10 months
the number of devices actually went up to over 1,000 suggests they were happy
to just keep paying.

The thing is, it was essentially no work. All I did was remove a firewall
rule. I had to run NTP anyway for my regular customers. Initially more time
was spent just in email back and forth and honestly I was enjoying that.

Because of it being basically no work, I had a moral problem with trying to
find the absolute highest amount of money they would bear.

I know that is wrong and it does me no good, but I couldn't get past it.

What did annoy me was their inability to pay bills on time, and time I spent
chasing invoices and creating custom late payment paperwork that is never
relevant for my usual customers.

That was the main impetus for doubling the rate, and despite me jokingly
suggesting that their product was not good enough to be profitable (I have no
real data on that either way) I suspect they had much bigger organisational
problems to be consistently paying late and ending up insolvent.

~~~
sleavey
I like your attitude. If you were stuck fixing the mess the software engineer
had to fix, you'd appreciate a random stranger making your life easier when
they didn't need to. Maybe they'll pay it forward. Think of the money you
didn't charge as your donation to making the world an ever so slightly nicer
place!

------
csours
I should really put some time into writing this up, but my first script that
went corporate-wide was an NTP catch up script.

Now that I think about it, I really don't know why an NTP catch up script was
needed.

Basically VMWare time was not reliable. Windows will by default not catch up
unless you get to around 5 minutes off. My script checked every day to see
what the drift was and correct it if it was more than 5 seconds of drift.

The underlying reason for concern was logging - we wanted to make sure that
our log times were comparable.

------
avip
> "their innovative reluctance to pay for anything on time"

Thanks great and educational writeup.

------
jpm_sd
It's pretty cheap to embed a tiny GPS receiver and crappy chip antenna these
days. Probably enough for time sync in many situations.

~~~
c22
Is it? The cheapest gps chips are still a couple bucks a pop (not counting the
extra costs of more complex integration.) And from my experience these cheap
gps and pcb antenna solutions are basically useless indoors. NTP is probably
still the right solution for these devices, being essentially free to add to a
device that already needs an internet connection. Care should just be taken to
implement it correctly, obviously.

~~~
reaperducer
_And from my experience these cheap gps and pcb antenna solutions are
basically useless indoors._

This is true. I know a TV station that has a tiny satellite antenna bolted to
the outside wall to run its internal NTP for all of its wired and wireless
devices because GPS simply doesn't work inside the building.

I'm not sure if that's a problem because of all the electronic equipment, or
the construction of the building, or the fact that the building sits almost
underneath a 500-foot-tall tower with several 10 to 20KW transmitting antennae
on it.

------
jiveturkey
all too common. this happens for the simple reason that such devices are
created by people that have never deployed anything to the field, and never
worked under anyone who has. that and of course the pressure to get something
to market quickly.

------
mcnichol
Man I really enjoyed your writing style.

Got a huge kick out of that.

------
robertAngst
Question for people who read the link-

Do you like the writing style and inclusion of gifs?

~~~
NikolaNovak
... There were gifs??

I guess I'm so used to useless ads and graphics in blog posts, I honestly,
scout's honour, did not notice and have no recollection whatsoever!

A damning statement on the Internet of today perhaps, but it neither enhanced
nor detrimented my reading of the article ‍️

~~~
ineedasername
same! I had to go back and look because I would have sworn there were no
images at all! Now I realize my brain had classified them as ads and simply
edited them out of my conscious awareness all together.

------
quickthrower2
lesson: politely tell shitbag customers them to go f’ themselves.

------
xte
BWAHAHAHAHA thanks both for the nice post and for politeness you prove to
have.

That's not much IMO a problem of "modern IoT" but a problem of modern
managerial-driven society that lead to a proliferation of Ford-model workers
at ANY level, not only the lowest one.

People simply can't reason autonomously anymore, at any level, can't really
understand "the big picture" of pretty anything: think only at periodic "cry"
for $FamousFreeServices down and the relative cue of polemics that follow...

------
Joboman555
Interesting subject - but I found myself pretty quickly lost in the technica
terms. I do not even know what an NTP server is.

~~~
eropple
I don't really understand comments like this on HN. I've derived a lot of
value through the years from thoughtful questions asked to dig further into an
article but I've never understood why somebody would assert "I don't even know
what X is." Search engines exist, yes?

~~~
bytematic
His argument is likely that you shouldn't have to when a simple explanation
could fit into the article.

~~~
eropple
But it's not an "article". It's a blog post. An informal post, written
informally, for an obviously technical audience.

I very, very rarely explain what "AWS" is when I'm casually writing about
cloud stuff. It's table stakes. You should know, or you aren't gonna
appreciate reading it anyway.

~~~
toomuchtodo
Logged in just to upvote your comments. This is a place of intellectual
curiosity, and I don’t understand those who expect knowledge to be spoon fed
to them. If you don’t know a term, search engine it and work your way down the
stack.

You might find yourself pleasantly surprised you’re providing an NTP server in
the NTP global DNS pool.

~~~
Drdrdrq
> search engine it...

That is, go to google.com or DuckDuckGo.com and search for it...

I mean, open the browser, click on location bar (on the top) and write...

That is, if you are on Windows, click Start menu (which is now 4
rectangles),...

Nevermind... /s

~~~
Wowfunhappy
I use Safari on macOS.

When I need to google a term, I highlight it, and then press ⌘C, ⌘T, ⌘V,
enter. (Copy the search term, open a new tab, paste into url/search bar,
search term)

I've gotten quite fast at the keyboard sequence; it takes maybe one second
total. I imagine I could make this process even faster with a plugin, But I
see no need.

I would like to think that most Windows machines would let you be similarly
performant by default. But if not, that's further evidence in my book that
Windows just sucks...

I will note that some acronyms can be annoyingly un-googlable, as the same one
stands for a wide variety of different terms. This problem does not apply to
ntp, however, which comes up right away.

~~~
sbmassey
Can't you just right click and search from the context menu, since you just
highlighted the term so the pointer ought to be in the vicinity.

~~~
Wowfunhappy
...y'know, I actually forgot that existed. IIRC, at one point, that opened a
search in the current tab rather than a new one, so I got used to my little
keyboard shortcut instead.

The behavior appears to have changed at some point, though, because it now
opens searches in a new tab. I'll probably change my behavior now. Thanks for
the reminder. ^_^

