They’re finally moving over to VOIP, but there was something romantic about this giant refrigerator sized behemoth, ready to break down at a moments notice. The battery backup system for it was homemade, about 24 car batteries in another room on a shelf, daisy chained together behind an inverted. Soon, it’ll be replaced with two pizza box sized servers.
HSC is the equivalent for lower-level electronic parts. They’ll also be closing down in January.
They've been around for a long time. Jobs and Woz apparently bought parts there for the original apple computers.
So it basically comes down to security flaws exposed to whatever interfaces it really is connected to.
> Miniaturization and flexibility come at a cost.
Companies have little to no incentive to update software for older products unless there are lucrative support contracts. This has been the case for decades.
It's like going from Hollerith card based tabulators to a System/360
But overall, I get the feeling that people today are afraid of things they can't vendor out, things they can't replace cookie-cutter style. Sometimes that concern is justified, when you think in terms of support spanning decades, but overall I see it as a vertical integration of bureaucracy all around us.
It's not just this control system, it's everything. Was it Caltrans that hit a modernization roadblock because the agency hand-built railway sensors and circuits around that same time and now can't find replacement parts because people don't build things in-shop anymore?
What remarkable faith? It's a program to start/stop AC/heat units based on temperatyre readings, not to send people to Mars.
A talented high school student can write one today, and do it with a Rasberry Pi and for much less than $1 million (more like 5-10K).
Not as a student, but as a temp in a secondary education unit, I maintained and updated the payroll system used to pay ~50 schools and over 1000 teachers every month. It was a VB/MS SQL thing written by another temp a few years earlier, but it did the job.
Yes, these things can all be done by amateurs. By tinkerers. We all here know it. We've done it. You've done it. Simple devices with telemetry and some logic. Simple programs that do complex things in the aggregate. Take it to your back yard, and either one of us or a highschool student could make a traffic light cheaper than $250,000 - $500,000 per intersection (WA state estimate). But neither one of us could build, install and maintain a quarter million of them, and I think you'd be hard-pressed to mobilize an army of nerds to make it scalable.
Yeah, for all we know a PE was involved in oversight of the project. That's an unknown that TV reporters didn't dig into and should have, because I think the story would discover a much more relaxed time.
But if one wasn't involved, I'm saying it took faith on behalf of the administration to assume responsibility for risk in backing this project, whether consciously considered or not. And there are risks with electrical projects beyond the computers, namely things catching on fire. What were the safeguards here? What if a logic error burned out a fan? What if someone hesitated to use the radios during a regular comm window out of habit and there was a medical emergency?
Faith here was well-founded because obviously nothing burned down and no one died. But who today would accept that risk without going to a licensed professional?
Well, in the real world it didn't matter at all.
The payroll sheets were then sent to the state's revenue service so that teachers would get paid. If there was an issue, the revenue service would spot it.
Occasionally there were a few slips, because this or that law regarding teacher compensation, insurance contributions etc changed 2-3 times a year. We'd just calculate the differences and issue correction invoices to compensate. At worst a teacher would get a month's salary when they shouldn't (e.g. they have stopped working, but the school didn't notify in time), and then have to give it back.
>Yes, these things can all be done by amateurs. By tinkerers. We all here know it. We've done it. You've done it. Simple devices with telemetry and some logic. Simple programs that do complex things in the aggregate. Take it to your back yard, and either one of us or a highschool student could make a traffic light cheaper than $250,000 - $500,000 per intersection (WA state estimate). But neither one of us could build, install and maintain a quarter million of them, and I think you'd be hard-pressed to mobilize an army of nerds to make it scalable.
The story is not for "a quarter million" of traffic lights, though (which would require tons of money, installations, construction work, big material orders, etc), but about an AC/heat controller for a school district.
Such things as the latter, a "tinkerer" can often do much better, faster, and more effectively than some "specialized" firm.
>But if one wasn't involved, I'm saying it took faith on behalf of the administration to assume responsibility for risk in backing this project, whether consciously considered or not. And there are risks with electrical projects beyond the computers, namely things catching on fire. What were the safeguards here? What if a logic error burned out a fan?
The student didn't install the AC/Heat units themselves.
And the school could always have a "licensed professional" come and check the student's program for correctness, for 1/10th or less of the cost of a 1.2 million dollar replacement, how about that?
Back then, you could do things more freely with less capable and available hardware. Different environment. Concept of computers was scratching the surface of what they could do for us, and people didn't have a notion of liabilities. And yes, the student didn't install the HVAC systems, but how did tele-operated harwdware interface with it? They didn't have echo dots and knockoff power plugs back then.
Today, you have all that available hardware but can't realistically do things as freely. Suffice it to say that the environment has clammed up on all fronts for an amateur to enter an industry. In some cases rightfully so (only experts for surgeries, plumbing and wiring for me please). In others, egregiously wrong barriers. For example, commonly the first thing that happens in a public institution or agency is that the losing bidder files FOIA requests trying to litigate why their solution was passed up and by whom, including copies of correspondence between them. Imagine having to deal with that dimension.
Back to traffic lights: for something so rudimentary yet so vital, it costs millions of dollars for a handful of intersections, and they're everywhere. Nearly a prime candidate for a four raspberry pi solution, right? You'd think someone would have attempted it.
I agree, and I think that's the key insight in all this. For some things the extra legislation, rules etc make sense. In many other cases, they complicate things, and can even end up delivering not just a costlier but also a less effective outcome (e.g. the 100M government website when a 2-3M one made by a small team would do better).
The rest is the actual cost of the product.
Outside of interesting questions like whether amateur tinkerers and hobbyists can be as reliable as professionals are other (also interesting) questions like "what factors drive the way institutions and corporations perceive and react to risk?"
> A talented high school student can write one today, and do it with a Rasberry Pi and for much less than $1 million (more like 5-10K).
Yes, writing a bang-bang controller would be trivial. Even some kind of PID controller wouldn't be too hard for an enterprising high schooler to try.
What makes the professional systems valuable is that they can build much more accurate models of how the building heats and cools. These models depend on the building's geometry, usage patterns, outside climate, etc., and are notoriously hard to produce. However, they can produce meaningful savings over long time periods since they ultimately use less energy due to the quality model.
Older electrical meters weren't all that accurate, especially for larger systems - that's true even today. They're analog as an only option of measurement due to incredible amounts of power going through them (too noisy for hall effect sensors). Large buildings are reactive loads to the grid, almost like an inductor. Anyhow, older meters picked up velocity for usage calculations.
Some A/C controllers carefully modeled how fast the power meters accelerated as an instrument, then used that information against them, throttling and ramping power use to slow the measurements down, basically causing under-metering.
Saved USPS millions a year in power bills.
That's one way to view it, another point of view would be that all those people whose jobs we're automating away still get to work somewhere doing something.
Read only file systems on the SD boot volume are another solution, but this can cause other headaches (obviously) or may not be appropriate for the software you intend to run.
It’s been running 24/7 in my garage since 2014. YMMV I guess?
Getting it all into a software state that can easily be transferred to "something that runs Linux and this emulator that requires next to nothing" certainly seems sensible at any rate.
Samsung EVO SD card, have multiple times rebooted by pulling the power (which is not ideal) because of unrelated issues that were due to some OS problems (SSH doesn't respond, etc) without a problem, but haven't had to do that in a long while after changing the OS and getting NIC stuff set up properly.
I have seen that the majority of SD corruption that happens tends to be in situations where Pis are being run under-powered off of a USB hub or something similar instead of a wall-plug.
I believe with a decent-quality SD card, and properly powering the Pi, the chances of SD corruption are relatively low. That said, I would prefer a USB3, SATA, NVME, or even eMMC interface, but the Pi has worked remarkably well for me. USB2 booting the Pi is always a backup option if my luck takes a turn for the worse
It is something I worry about though.
Unfortunately not all SD cards, even if advertised as the same capacity, are exactly the same size. If your replacement card is just 1 byte smaller your dd clone won’t restore - the “free” space has to fit into the target card as well.
Long story short - most popular methods of cloning microSD cards for the Pi only ensure the ability to restore to a larger SD card, not necessarily one of the same advertized size.
For sure there are workarounds, but it’s not as straightforward as you might assume. Personally I’m much more a fan of other tools such as rsync or version control etc for cloning data to and from Pi projects.
The better answer is just to USB boot the Pi anyway - they’ve supported this for a while now, no card corruption issues to mitigate.
(The Amiga does have a bunch of outdated stuff in it, but the circuit board is quite large, with mostly standard-sized ICs, and other Amigas can typically be cannibalized for components. The repair options for a modern laptop on the other hand seem to be rather limited: everything's tiny, it's machine-assembled, there's five chips on the entire board and each one has 1100 pins. If something goes wrong, you need to replace all of it.)
Nah, my ~8 year old laptop was still trying to process all the js to render their site even after I finished reading the article and hit the back button -- that's probably the standard they're holding up here...
But, really, the only problems I ever have with this old lappy is the occasional bug that slips through the Fedora testing process and a battery that really should be replaced.
8 Years ago was 2010. My replacement laptop is basically 2x the power of the old one, and that's really only for short bursts due to thermal issues. Actually the one in the garage likely is better for large parts of the year because it's cold in the garage.
SSDs changed everything. What’s to wear out other than fan bearings?
Meanwhile, your Amiga has this enormous motherboard that's full of DIP-size ICs, many of them socketed, and even with your schoolboy soldering you can extract the ones that ain't. And it almost cetainly doesn't matter if you're hoping to get their replacements from an Amiga that's completely different, because Commodore just didn't have the money to make N different versions of their chips, even assuming you're trying to replace the custom ones in the first place.
Or was it just the disk drive that fucked up, or one of the power supply capacitors, or something like that? - well, why didn't you say! No problem. Plenty of those about!
P.S.: but, all that said: what I was thinking about, when it comes to a modern desktop, is that you'd just replace parts as much as possible, repairing things as required, then when it finally dies just buy another one and restore from your last backup. But while bench repair is more of an option with a desktop than a laptop, in either case you probalby wouldn't actually bother, and you'd just throw it away and buy a new one at the first sign of trouble. So in fact, maybe it doesn't make much of a difference overall.
(First was the mighty Sinclair Spectrum+. Whywhywhywhywhywhywhy did I sell that thing.)
| “What is the difference between your laptop, model 344 revision 001, and another laptop, model 344, revision 002?”
This is one of the things I despise about the Windows world. Say you have an HP EliteBook 9460m. Sounds like quite a specific model number. You want the graphics drivers. You go to HP.com, navigate Support, plug in your model number ... and are given the option of the Intel or the NVIDIA drivers.
What the hell? Do HP not know which one they put in this machine? Was it just a crap-shoot at the factory? That’s bonkers, in my book.
I'm doing the job for 1/4 of that money, just pay me economy class trip and a motel.
Seriously, 1.5 to 2 million dollars? The Amiga, although it was an amazing piece of hardware (I owned 500, 600, 2000, and 4000), was far from being certified for any industrial use, so I wouldn't expect any extra spending in that context.
Even using the industry standard for the job (PLCs) would be cheaper let alone using smaller single board computers (no, I'm not referring to Arduino or Raspberry PI, there are rugged solutions well below $500) and doing all development in house.
Do I miss something or they overextimated that spending by at least one order of magnitude?
in austria, decades ago they invented winter holidays to save on heating for a week. then they found out that it cost them more to reheat the building after that week, so now they just keep the empty buildings heated.
Edit: And as the comment thread in the article explains they're probably renovating the whole HVAC system, not just replacing that single Amiga.
If it's not broken, don't fix it.
They are going to spend $2 millions breaking something that works. And that's if they are lucky and it doesn't go over budget.
The same reasons that something like a Nest thermostat can save you money at home.
What are the odds that the new system will save them > $1M over its life?
It has worked 30 years so Bayes says it is likely to keep working.
I’d be very impressed and delighted to see this running in 30 years time ... but realistically there’s vanishingly small chance the hardware would last that long without some sort of failure. For now they’ve said they were able to scrounge replacements for broken parts, but that cannot last
This is the best part of the article. I remember a fellow student in 1995 being asked to make the website for our whole school board. He had played around with some WYSIWYG program back then and excelled.
If you can’t figure that out for yourself, the good news is that there’s still plenty to learn about this industry (and “industry” in general), and you’re unlikely to get bored. In the meantime, something something Chesterton’s Fence.
There must be off-the-shelf components to plugin in to replace the on-site controllers that can be controlled remotely?
Also, what's the chance the modem just connects over POTS to another modem with no controls on authorisation?
You get paid based on some short future time window perception of the value your labor is adding. When really random people were coming to work some random Tuesday adding a few lines of code, documenting something so that someone else a few years later could add robustness or change a capability a few years later, etc., and it adds up to something that gets the job done for 30 years.
It’s a marvel.
I’ve worked on various projects that resembled this in some ways, primarly a defense research system for simulating aircraft properties and susceptibility to threats, built in the 80s and 90s in C, and still being constantly used for top level intelligence briefings in the 2000s.
To say that the engineers who made the foundational system to have a prayer of living for 20+ years were underpaid relative to the value they created is just a ludicrous understatement.
The reason? It was about 90°F (32°C) in the classroom.
On the previous Friday, one of the teachers turned the thermostat dial up as far as it would go. Presumably they intended to turn it down later but forgot.
Really, is reading comprehension that difficult?
> The frequency used by the archaic communication system overlaps with maintenance works’ radios, which can sometimes cause interference with the system. When that happens, everyone has to turn off their walkie talkies for 15 or 20 minutes.
It’s a school. Unfortunately it’s not particularly expensive.
Yes I know the fed reserve's been steadily devaluating the dollar (and some standard inflation), but I think it's still telling.
PS: The F-22 for example along with a huge swath of military aircraft can also fly over 65,000 feet. But, rather than just carry a camera they also need to have weapon systems etc.
I agree a 10 million seems reasonable today. That was the point.
Also, be careful comparing just the name. Today's U2 were built in the 1980s and are about 40% larger than the original design. They also include a lot more electronics where the orional needed to land and get film developed.
And then of course, the GH is an unmanned drone, with a little lower ceiling, etc. These would reduce costs. Obviously it's a far-more advanced project, but it's also had outrageous cost overruns.
I think it's crazy when you think one of our military's highest-performing planes, even if not as exotic as an SR71, has some kind of reasonable cost comparison to an air conditioning service that a high schooler built in their spare time. Remember, a lot of that inflation is devaluation (this is debated) from Fed printing money and playing games with Wall St, but not all of it.
> It’ll cost about $1.5 to 2 million to replace the Amiga...
Hope this "kid" is getting paid for his work.
They don't share boilers or anything I'm guessing.
Wow ... That is beyond irresponsible ...
Do you want to be right, or do you prefer to be happy?