We hear all the time about promising kickstarters that fail and everyone just nods their heads in agreement that "shipping hardware is hard". And surely it is. But yet, here was one guy shipping thousands of custom-designed modern-ish computers to hobbyists and businesses around the world at a very fair price for multiple decades.
I have had an APU2C4 acting as my firewall (running OpnSense) at home for the past few years. It's been pretty a pretty good little box. My singular complaint is that the CPU has always been much slower than I'd like. Standard firewall functionality works fine, but doing any admin stuff involving package management takes ages, even with an SSD.
I'm not sure what I will replace it with when the time comes. Despite talk of alternatives in these comments, there is nothing on the market that checks all of the same boxes that the APU2 does: 1) AMD64 2) fanless 3) low power 4) multiple interfaces 5) under $200.
> Despite talk of alternatives in these comments, there is nothing on the market that checks all of the same boxes that the APU2 does: 1) AMD64 2) fanless 3) low power 4) multiple interfaces 5) under $200.
ODROID-H3/ODROID-H3+, maybe? [1, 2]
1) AMD64/x86-64 architecture (Intel chip).
2) optional fan; I'm unsure if it's necessary for any particular load
3) 2W at idle up to 18W with CPU+GPU pegged
4) 2 2.5GbE NICs builtin, M.2 card available to add another 4
5) bare board is $165 direct from hardkernel (H3+) or $129 (H3). Almost certainly over $200 once you include RAM, SSD, case, power supply, and shipping, but I think the same is true for pcengines stuff too?
While a Raspberry Pi is much more powerful than your basic router, these kinds of routers are those with genuinely extra features that will strain out a RasPi (for example, multi-gigabit routing, advanced traffic shaping, and in GP's case packet-based firewall with a possibility of deep packet inspection). Until we have decent emebedded ARM-based computers, it is still likely that there would be a reliance on x86 computers simply because they exist.
* Much better options for flash storage; it's real easy to buy an extreme high endurance m.2 with the device, vs constantly rolling the dice with a microsd card on a raspberry pi. The APU also has a SATA port and an SD card slot (which they also provide very high endurance SD cards for).
* x86 + coreboot means the device is well understood and open, and will run basically anything you want unmolested. No need to deal with crazy Broadcom firmware.
* High quality Intel Gigabit NICs mean they always "just work" in Linux, and work well.
* Expandable with m.2 pcie cards instead of relying on USB...PC engines will sell you a good vetted 802.11 card that basically just drops into their existing chassis.
* I think all of the APU2s have ECC ram.
* Before the Raspberry Pi 4, the APU2 had significantly more compute power than any prior RPi. I don't own a Raspberry Pi 4, so I have no idea how they stack up.
I've had a bunch of these for years now, and they all run unmolested basic Debian, and I've never seen one of them fail (I've seen an Alix fail, but it was recent and that unit has to be very, very old at this point). Every Raspberry Pi I've ever used has felt like a gamble whether or not it was going to die (I've lost a few, not being abusive to them either). In a router, it's absolutely worth it to know that every time that device powers up, it's going to work.
> people still buy the WRT54GL in large enough numbers that Linksys continues to earn millions of dollars per year selling an 11-year-old product without ever changing its specs or design. "To be honest, it somewhat baffles my mind," Linksys Global Product Manager Vince La Duca told Ars. But production won't stop any time soon as long as Linksys' suppliers, including chipmaker Broadcom, keep selling the parts needed to build the WRT54GL. "We'll keep building it because people keep buying it," La Duca said ... The WRT54GL and other routers in the WRT54G line have sold 31 million units in their 14-year lifetime. The WRT54GL is the only one still being manufactured and sold.
A valid question. The best answer is flexibility. On AMD64, I am guaranteed to be able to run pretty much any OS I want if my needs should change. OpenBSD, FreeBSD, weird Linux distros, whatever. Comparable ARM boards, however, tend to only work with Linux and have highly variable levels of support at that. Support can wane depending on interest and level of involvement of the community.
A Raspberry Pi would be an excellent choice if they ever offered one with multiple gigabit interfaces. But it's the only ARM board I would currently consider for the role.
Oh no, I love my Apu2.D4 OPNsense router! Such a versatile, robust and low-tdp platform. What are alternatives? I use a protectli [1] at my second place, but it is more expensive and feels a little bit too bulky, compared to the APU. I also have an Odroid Cloudshell 2 with XU4 as a NAS [2], but the platform is not as widely supported, e.g. without coreboot support (I still love the Odroid). The Raspberry Pi Zero WH is difficult to get these days (and not powerful enough), and I find the Raspberry lineup a little bit too fragile to hardware issues, from my experiences. My APU with open coreboot firmware worked 7 years without a single issue.
What I would wish for would be a low-cost embedded solution (< $400) with support for ECC memory.
PC Engines APU2 offered unmatched flexibility in the sub-$200 x86 segment:
low-power, fanless
open-source coreboot firmware
Intel NICs (vs Realtek)
open schematics
USB, SD, SATA or mSATA storage
ECC RAM
GPIO
TPM and DRTM
3rd party cases for rack/wall mount
supported by Linux, Xen, OpenBSD, FreeBSD, pfSense, OPNsense
They are taking orders until June 2023, for shipment in ~6 months.
We've been developing an industrial IoT gateway at work on top of the APU2 series, and there really is nothing quite like it. I'd add to that list support for mPCIe LTE modems. My only real complaint with the APU2 was that the story around TPM support isn't great - while they do technically support a TPM, its via an additional board you plug in, and availability is more or less non-existent. Apparently the CPU has a built in TPM but PC Engines were unable to get it working, and AMD weren't engaging with them on resolving that.
Recently we analyzed the Hardkernel offer in light of OpenSecurityTraining2, and it looks pretty good but is more expensive than UP2. For example, the UP Squared Pro 7000 Series [1] is fascinating.
The pros of Hardkernel are open schematics (same as PC Engines), which is crucial if one would like to have open-source firmware like coreboot. Quite good testing documentation, transparency, and community. UP2 does not provide schematics, and their support could be better, which can be proven by reading their forums. There needs to be a sense of transparency and community building for UP2.
Of course, UP2 and Hardkernel products have other drawbacks compared to PC Engines, namely, no D-RTM, fewer NICs, higher TDP, no ECC, etc.
I moved from my APU2 router to a HUNSN Intel Celeron router, still fanless, 2.5Gb in 4 ports, still can run opnsense, tho I run OpenBSD on it. So I'm happy with it.
I'm running an NA-345 from Axiomtek, which I've been very happy with. Lanner Inc also make small embedded systems for a few hundred dollars as well. Both are viable here.
The ALIX/APU platform is sadly the only one on the market that:
- has at least three network interfaces
- is a relatively straightforward x86 system you can just slap a mainstream Linux distro on
- is small enough to be integrated into a custom enclosure
- is affordable.
It's been a reliable platform with only one broken serial port and one broken ethernet interface in some 500 deployed systems (flash endurance is another story, however).
Sadly our sourcing department might have some other hidden requirements in addition to those listed above. At a quick glance, the unit prices are still at least some 100 eurobucks higher than Pascal's :-)
In any case, due to COVID-induced shortages we already had to transition away to other hardware. This has been a huge relief as we had to ditch our custom legacy software stack built on APU+OpenWRT and came up with a generic Ansible/AWX based solution you can run on almost any platform with no tweaking.
Also some time this year a successor to the (Zen1 and very dated) Epyc 3000 series is supposed to show up. I've been running 2 of those and they were nice back when they were fresh. Much more of a small-server setup though.
Other devices can fall in that category, or be very close to that, although being much faster they obviously cost more than the ALIX/APU. See for example https://www.ipu-system.de/
That said, I have been extremely happy with my old WRAP boards; they worked for years without a hitch with PfSense (now I would use OpnSense), and didn't swap them with newer PcEngines boards just because at that time they were already a tad slow for today's Internet.
Check out the Qotom fanless mini-PCs. Mine has 5x network interfaces, and it's a straight forward x86-64 system. I run OpenWrt on mine, but it's just a PC so you can install any mainstream Linux distribution on it. It comes with an enclosure, which might not be what you're looking for.
The cost depends on how much memory / storage you need, obviously, but it's not that different from the ALIX/APU once you include things like power supplies, cases, etc.
I built a home OpenBSD router several years ago using the APU2. I haven't touched it/loggied in for several years although I keep meaning to so i can update the OS. It's been sat ticking along nicely doing DNS, DNS filtering, cross subnet routing, wireguard for remote hiccup free.
I have a Xeon D1521 board that I use which does support ECC. It was under $500 when I bought it, but can't find it now. Supermicro does make a mITX board around it with 2x10GbE for $800. Asrock (which made the board I have) sells a uATX one in mid $500 range (I think I spent $499 on mine[1]).
I'm guessing protectli might be a contender here. I think they're able to router gigabit as well. I think there are some similar variants that can be had from Aliexpress but I certainly wouldn't be running those as my router.
> Between cameras and CCTV, it now takes over 2000 cameras and 100,000 sensors to run a retail establishment. So much, so that the edge processing demands led Amazon to start its own OCP project to meet the processing and networking needs.
PC Engine complained about a lack of design support. Personally I just am not that sympathetic. There are a variety of new Ryzen embedded cores which have really great capabilities that could fit this model. We're just not in the same world where AMD is going to sit down & lead you through design, like a much much smaller world was. And board design is harder, faster systems. It's just harder to make boards.
What really sucks is that this frell over a huge market segment. Industrial embedded stuff is all crazy expensive. New embedded systems are often $700 plus.
I'm also curious to know what availability on embedded parts is. I get the impression old parts aren't necessarily easy to get, much less get with reasonable discount prices. The low-end R1505G from 2019 was $80 when announced, & my vague impression is this is still about as low as one's gonna go for getting a embedded Ryzen chip. The GX-412 chips from 2014 were probably half the price.
It's so awful that the gap between rpi & x86 grows. Sure there are excellent second hand markets, but the x86 market running upmarket is a terrible general situation. Decent embedded chips should be a very cost effective available baseline, should make lots of affordable baseline low frills systems available.
While we're having a moment of silence for PC Engines, they deserve credit for small biz survival through 2020-2022, with supply chain disruptions and grey markets for Intel NICs and other components. Now AMD is ending sales of embedded GX-412TC, with 6 month backlog and >30% drop in laptop sales.
Why can't AMD extend sales of this old SoC until they have a replacement Ryzen Embedded V3000 SKU with comparable power profile? GX-412 is an older node. Customers want to buy it. AMD has struggled to get adoption and design wins for Ryzen Embedded. Why abandon existing embedded customers?
With a decent implementation the integrated Gpu should be able to d3 cold sleep at incredibly low power usage. If this isn't the case it seems like a defect.
AMD has definitely scaled up but a configurable 10W TDP hardly seems like a showstopper. It's a little hard to know what TDP really means these days, which complicates, but it feels like some give/take in design to handle a 10W chip should be managable.
Overall good post & a great different view. I'm not convinced of either your view nor what I've said. It just doesn't feel like it should be a huge blocker as is. And it feels like there should be better than 10 year old offerings that would work.
I think we all agree that it should theoretically exist, but I've not seen any existence proof in either a paper spec announcement (many embedded boards are not sold directly to consumers) or a shipping device review.
Solidrun/Compulab in Israel announced a DIN-mountable fanless router based on Ryzen Embedded V3000, https://www.solid-run.com/fanless-computers/industrial-embed.... Premium components and proprietary cooling to support 30W+ TDP. Impressive, but probably $1K price tier.
I don't know anything about this market so I may be completely off-topic here but would RISC-V help in this case? RISC-V has a modular design where the size and complexity of you CPU can be adapted pretty dramatically (RV32I is not that much more complex than a 6502) and being "open" and free can be manufactured by anyone.
Indeed today it remains a niche but we see some reasonable momentum and it's getting traction.
Again I don't know anything about PCEngine (reading the title I though they were still making that game console from the 90s...) but I am interested to see where RISC-V could take over some x86 business and that's looks like it.
No. RISC-V is just the ISA spec. It doesn't give you an actual chip. Producing an actual reasonably fast CPU is much much more challenging. So RISC-V can be as modular as it can be, it is up to the manufacturers to create chips using it.
RISC-V doesn't give you a magic wand to create chips. It just gives manifacturers easier terms for licensing. Not you or PC Engines an easier time to design and source things. AMD also has all the licenses it wants. The advantages that RISC-V brings doesn't affect them.
Moreover any actual RISC-V chip that's worth replacing the AMD in PC Engines will be at least as proprietary as the AMD chip. It will have its own design requirements that PC Engines still needs to obtain from the whatever manufacturer.
RISC-V only makes the job of the IC manufacturers easier. For everybody else, it is yet another thing to struggle with porting.
But what RISC-V would provide is a "standard" ISA. You can avoid having a manufacturer/vendor lock-in. Anyone can manufacture a RV64EMAFDZicsr.
> Not PC Engines an easier time to design and source things
Isn't it better to rely on a "standard" technology than a proprietary one? x86 is not standard, it's widespread. The Duopoly in place can decide the price, the feature, even its client. They can decide one day to stop providing what you need and that's it. With a standard chip, you can go next door in Shenzhen and get what you need.
> Moreover any actual RISC-V chip that's worth replacing the AMD in PC Engines will be at least as proprietary as the AMD chip
Why? We are not talking about a supercomputer to train LLM here but simple network appliance than needs efficiency. The GX-412TC PC Engines was using, doesn't look that impressive. Aren't there already RISC-V chip today on par with it? Like VisionFive for example?
> But what RISC-V would provide is a "standard" ISA. You can avoid having a manufacturer/vendor lock-in. Anyone can manufacture a RV64EMAFDZicsr.
Anyone can manufacture a chip based around one of Arm's CPU core designs if they pay a licensing fee to Arm, and many different manufacturers do. If you want to, you can even make your own CPU design supporting an Arm ISA, again if you pay a licensing fee.
Likewise, you could license a RISC-V CPU design from some company… for a smaller licensing fee, but you will have RISC-V's worse software compatibility, and probably worse overall performance, efficiency, die area and quality.
The ISA is a tiny piece of the puzzle and not the most important one. RISC-V is cool because Arm won't sue you for making a hobby project that implements it in hardware, but it doesn't solve problems other than “Arm's license fees are too high”.
>and probably worse overall performance, efficiency, die area and quality.
Unlikely. E.g. SiFive offerings beat the crap of ARM equivalents, in performance, power and area.
>but you will have RISC-V's worse software compatibility,
Using the standard ISA gets you the widest possible ecosystem.
Of course, this isn't true yet, but it will be in practice, when considering the current trends and the fact hardware takes years from the time the decision is made till having hardware in hand.
>RISC-V is cool because Arm won't sue you for making a hobby project that implements it in hardware,
With 10b+ cores shipped in 2022, we're well past the "niche ISA for hobby projects" stage.
> The question on everyone’s mind is when RISC-V will come to user-facing applications. The answer is that this may be closer than people expect. There are currently 4 companies working on large RISC-V cores that compete with the biggest and fastest from Intel, AMD, Arm, Apple, etc. These 4 companies are Ventana Micro Systems, Tenstorrent, Rivos, and Akeana. Each of these companies have teams with impressive pedigrees, but that alone doesn’t guarantee success.
Yet, the most reliable services people use every day run on x86. It's not "really bad".
>due to its complexity
I don't see the complexity of the instruction set causing downtime. Compilers abstract 99% of it away.
>due to obtrusive (SMM mode), non-auditable firmware.
I've never seen SMM mode cause down time, nor have I had the non-auditability of the firmware caus me down time. These are low level things that are a part of how the CPU works. When you are working at a high level you can just ignore them for the most part. For reliability you are always going to need to be able to handle bad chips due to manufacturing defects once you are operating at scale.
> I've never seen SMM mode cause down time, nor have I had the non-auditability of the firmware caus me down time.
Hyperscalers (OCP) pushing for coreboot and LinuxBoot have probably different experience. AFAIK they hate SMM especially with SMI handlers coming form unknown source.
Not saying SMI latency is huge problem in industrial applications like CNC.
>Yet, the most reliable services people use every day run on x86. It's not "really bad".
When reliability is paramount (e.g. pacemakers), x86 is naturally avoided.
>I don't see the complexity of the instruction set causing downtime. Compilers abstract 99% of it away.
Complexity of the ISA affects the whole system. It breeds bugs not just in hardware, but also in operating systems and toolchains. These can affect reliability.
>I've never seen SMM mode cause down time
I have. With SMM, the rug is pulled under the OS's feet. Your CPU can be taken away at any time, for any unexplained reason, without prior warning, and for an undetermined amount of time.
I have seen SMM cause spikes of latency breaking pro audio pipelines, and I have seen SMM grab a CPU and not return it.
I'm not a RISC-V expert, primarily relying on open-source firmware community knowledge and the opinion of such figures as Ron Minnich, but AFAIK, most firmware for production RISC-V deployments is closed-source, so we can't say if vendors use OpenSBI implementation or some modified version for their malicious purposes. If I need to be corrected, please point me to products that state how they transparently use SBI. There is evidence of transparent use of SMM in coreboot. SBI is not the only RISC-V TEE. There are also other concepts, like MultiZone [1] and PMP-based Keystone [2], advertised as trusted execution environments
The openness of specification only matters a little here - UEFI specification about management mode is also open. But IBVs screwed us many times [3] by implementing low-quality SMIs that could elevate privileges, and the spec does not guarantee that implementers follow it. Lack of tooling makes compliance difficult - it slowly changes with various startups seeing that as an opportunity (e.g., Binarly, Eclypsium?).
My point is that SMM as TEE is not a unique x86 feature, and other significant architectures have similar mechanisms. OS has a means of figuring out that it was in SMI, so it is not entirely hidden but still has superpowers. In the CNC world SMI latency is measured from OS [4].
Trusted execution environments are hammers and can be used for a good purpose in a transparent way and for malicious purposes. Further typically depends on the trustworthiness of mechanisms implemented and used by vendors, but thus keep the fact that TEEs and peripheral MCUs are everywhere, which may lead to the extended attack surface.
Why do we see those TEEs, peripheral MCUs, everywhere? I like the explanation in this lecture [5]. No architecture can quickly fix that problem.
There's little RISC-V could do to prevent bad SMIs.
It already did well enough by standardizing SBI and by providing a high-quality open implementation of it.
This minimizes (or even removes) incentives to provide a proprietary solution.
Some vendors will of course do whatever they want, instead of just using opensbi.
But they'll be opting out of being compliant with the platform specs, and of benefiting from the support for SBI present in operating systems and embedded toolchains. Such an implementation would just make themselves and everybody else miserable.
In an ideal world, the market will avoid non-compliant SoCs. In practice, there will be some of these to point at as examples of how not to operate.
You're mixing implementations and the ISA. A RISC-V chip can be as daunting as any other x86 and I can assure you they will be.
Those modes are one of the ways to enforce export controls. Any competitive chip has something similar, say ARM TrustZone or whatever x86 manufacturers name their security mode.
If you don't put such features, the Asian manufacturers will instantly copy your IC.
R1102G has nominal 6W tdp, so its very comparable to GX-412 in that sense. It also has 2x2.5Gbe networking builtin so that should be a major win for something like pcengines
It is perhaps cheaper to use an existing design instead of designing a special one for embedded applications. In fact the GX-415GA with an iGPU looks like the exact same package as the GX-412TC, the GPU is probably just not fused off.
The cores run at significantly higher clock, up to 2.6GHz vs up to 1.4GHz. Once you factor in the dramatic IPC improvents of zen, it should be perfectly sufficient replacement.
More cores are better for security and virtualization, per Spectre/Meltdown. Why go backwards from 4 to 2 cores, when there are Intel mini PCs with 6 cores and Ryzen Embedded V3000 with 8 cores?
There were also security improvements between Ryzen Embedded V1000 and V3000.
Do embedded customers want a "perfectly sufficient" equivalent of their 8-year old device, or a flexible device with headroom for the next 8 years of sales & support?
There has been some progress since then. There's probably other fronts, but my first question to myself here was, "I wonder what P4 (network programming language) has been up to with acceleration." (https://opennetworking.org/p4/). And sure enough there are a couple hits, P4GPU (2016), APUNet (2017), P4GPU (2022). Alas none seem to be open source atm.
FPGA's are the more common target, with so called "smart nics". They're present in lots of fancy cloud hardware & could be excellent at helping us push packets at a good power point. I think if the software market were more evolved people would be clamboring for these things & they would have become more mainstream popular & in demand, & would be available at lot more price points.
AMD should be releasing cpu+fpgas (from their Xilinx acquisition) pretty soon, and if they're smart there'll be some affordable low end options - small ryzen embedded cpus and a modest chunk of network processing fpga - so folks can mess around & get good. We are seeing them start to upstream some software stuff. The actual kernel drivers for the XDMA CPU<->FPGA DMA made it into the upcoming 6.3 (https://www.phoronix.com/news/AMD-Xilinx-XDMA-Linux-6.3), there's a whole new FPGA bus "CDX" that is slated for 6.4 (https://www.phoronix.com/news/AMD-CDX-For-Linux-6.4), and there's a LLVM-based eBPF XDP C code to FPGA compiler Nanotube (https://www.phoronix.com/news/AMD-Xilinx-Nanotube-Compiler) that... well... looks like it might categorically enable these use cases with almost no effort. (After years of being thrashed by awful OpenWRT router's quirky vendor packages for network accelerating, I say: heck yes, yes pleaseeeee!)
GPUs do indeed seem >50% likely to be bypassed, that FPGA just fits better & wins faster. But they both had & have a ton of potential. Especially if there are APU style GPUs that have shared memory (where there's not the latency of sending stuff back and forth over PCIe). Think of all the years ago AMD tried to sell us "Fusion" and Heterogenous Systems Architecture (https://en.wikipedia.org/wiki/Heterogeneous_System_Architect...). Well, here we are 12 years latter & it's happening. It could already have happened, we just didn't have software to actualize it, especially in the networking domain.
To your point, I do think 2 cores can be a potential issue. Ideally the iGPU should be able to power down quite effectively, ideally d3 cold sleep. Wasting the silicon isn't ideal, but generally I think a vanishingly small iGPU has such a small penalty to include, adds so much value to a chip for so little real cost. It feels overzealous to me to exclude iGPU-having chips & I think it's based in older prejudices from times when iGPUs used to entail more real trade-offs, with more impactful power budget & chip-size hits. What's important now is whether there's enough processing (which the networking world for now continues to ask for in the form of CPU), memory bandwidth, and network bandwidth, at solid price and power points.
We have so few examples to learn from, so few embedded machines available these days at reasonable price points (<$400), that I think we just don't know how suitable what we have would be or not.
The Soekris net6501 was an Intel Atom with a Xilinx FPGA. Alas, Soekris closed for somewhat similar reasons several years ago, though a bad batch of chips from Intel which caused an RMA nightmare didn't help and may have been what pushed them over the edge.
I wonder if in pursuit of outright performance, AMD stopped optimizing for low power consumption. Desktop Zen3 Vermeer (which the newly announced 5000E series also are) with chiplets idles at around 20 watts, presumably due to having to maintain signal integrity between the chiplets. Similar story with the chiplet-based RDNA3 cards idling at 100W. Anecdotally, my Renoir laptop is also lacking in battery life.
I went once in Zurich and knocked at the door of PC Engines to buy a Wistron wifi card (it was cheaper then shipping it to Belgium), it was one guy in a house with plenty of hardware. Memories!
No surprise the official website has no info about the company (e.g. who is the CEO of this company by the way? A name at least...?). Of course there is no info, as I suspected its a single person doing everything. One man show.
Some factual mistakes I have read in this lengthy thread (not going to individually jump to each post to reply there):
APU2-3-4-5-6 all models running literally the same ~10 year old obsolete prehistoric SoC / CPU. And to be precise, its clock speed is just 1.0Ghz! Not 1.4Ghz! It makes a huge difference in this case, ~40%. The 1.4Ghz is a boost clock speed, and its only possible for a single (1!) core, and only while the other 3 cores are literally idle. The 1.2Ghz boost is an interim clock speed though. Quite unrealistic sustained workload, right? So if you want to calculate performance based on clockspeed and IPC, you'd need to assume the 1.0Ghz, and not the 1.4. By the way, this CPU boost feature was broken since inception, until 2019, so literally only 1.0Ghz was possible even for single core workloads until that time. Have to mention it was a PITA for the firmware developer subcontractor (a word about this as well below) to make it work with lot of sweat and blood and tears. Thanks to AMD being a jerk, and not providing any support document for these folks. (People assume its only Intel who the dick company is. But I can assure everyone, AMD is the same arrogant profit-oriented a..hole company, as Intel. No difference! Just because Zen is successful in the past t-6 yrs, AMD is still a difficult company to deal with, if you are an embedded system integrator partner of theirs. End of rant.)
- ECC: thats another misleading feature: its mentioned (very only in the schematics PDF, nowhere else you will find this technical info clearly written in the APU2 models page: only the 4GB RAM model has ECC. The 2GB models DO NOT HAVE ECC! As a proof, you can see it for a fraction of second when the BIOS boots, and you see the RAM information. If you bought a 2GB board, believing you'll also have ECC, you are screwed!
Intel 210 vs 211 NIC: because the quad-core CPU in this prehistoric SoC is so low powered, it DOES MATTER that the i210 supports 4TX&RX queue, but the i211 only supports 2! So if your workload is heavy packet processing, the i210 is a must! As the i211 wont be able to distribute packets evenly among all the 4 cores, only between 2, so performance will suffer.
Bonus little known fact: PPPoE protocol does not support the RSS (Receive Side Scaling) protocol, because PPPoE is NOT based on IP. So all your PPPoE traffic will hammer CPU0, and that core only. Performance will suffer. Nowhere near close to gigabit routing under such conditions buddy! Just you wont read this caveat anywhere, thanks "sponspored" reviewers, hiding this painful fact! The internet is full of people complaining about their gigabit speed not achieved, and it always turns out a PPPoE-type of subscription is in service.
Firmware: because pcengines is a 1manshow company, he (she?) outsourced writing the BIOS for APU to a polish company called 3mdeb. When you buy an APU, you rely on a 3rd party company to provide coreboot BIOS support.
As if you bought a motherboard from Asus or Gigabyte, but the UEFI comes from an "unknown" company. When they bankcrupt, and you go to Asus/Gigabyte, they will tell you: sorry, but the bios is not provided by us for your motherboard, go complain to that company who (was) responsible for it! Sounds strange, right?
So in short, there is no such thing as a default non-coreboot BIOS in the APU product. This became an issue, when people wanted to see improvements on the BIOS, and the PCENGINES github issue tracker was full of open items. And the 1manshow pcengines company replied that its 3mdeb who writes the bios, pcengines doesnt deal with such topics. Contractually the customer bought a product, that is branded as PCENGINES, not a oroduct that has the 3mdeb logo on it, I hope you see my point.
But anyway, this was sort of ok until very recently. In Aug 2022, the continuous flow of coreboot releases for APU stoppes suddenly. Feeling the coming bankruptcy of pcengines, 3mdeb backtracked from the relationship this February (the CEO of 3mdeb confirmed this on various forums). So everyone who is following the story knew, that pcengines APU is officially dead by now.
Last point: it is very disappointing, that this product will be gone. While it had its limitations and weaknesses, it was a so-so performant and reliable x86 router solution. As many pointed out, full-x86-64 compatibilits, noFAN, ECC, multi-NIC support was the pros. Cannot understand why on earth there are no other zen1-2-3based embedded routers in the 10Watt-range, that consumers could buy, without paying the artificial "embedded-so-it-will-cost-100%-more" TAX? Also I have to question PCENGINES 1manshow person as well: built a product 6 years ago (if I am not mistaken, APU2 has been released in 2016-17?), the EOL of the SoC was known on the 1st day. But did not prepare another alternative, based the whole company on literally only 1 product (apu2-3-4-5-6 are all based on the same SoC, so technically they are still 1 product, even if there are differences in the # of NICs, and the SIM slots). So when the known EOL time comes, hasnt prepared for a new product? Yes I know, AMD to blame. But seriously, throwing away this whole market segment?
This is a great loss. APU was my favourite system to bootstrap infrastructure in my home and labs because it was powered by coreboot and runs Linux out of the box, and also is one of the few low powered devices which still would outperform an RPi or clones. I have a stack of APU2 and APU4 devices and nothing but love and gratefulness for pcengines for making those guys. 100% good experience
Wow this really sucks. I've got a bunch of these...I don't think I've ever seen anything as flexible, capable and stable at this price point ever. It really sucks that AMD is really dropping the ball here.
My introduction to PC Engines was finding a strange embedded PC board in a pile of electronics scrap. (I think it would turn out to be an ALIX board.)
Connecting to its serial port brought up a Linux prompt, but I didn't know the password. I popped the compact flash card into a reader; maybe I could see it in the password file in /etc? Hmmm... seems to be encrypted. Well I have write access too; can I just... make the password empty?
I was fairly new to Linux at this time, so the fact that that worked was a huge boost to my confidence. I played around with putting different minimal Linux distros on it, and just generally having fun fooling around with a tiny x86 SBC.
When I needed a new router for my home network, there was no question but that I was going to get one of these (though I got the newer APU2 board of course).
I can't believe PC Engines is shutting down! End of an era.
Sad day. I picked up an ALIX board quite a number of years ago and it served as my first introduction to using OpenBSD for routing. I've been using the APU2 line since then, and I absolutely love the devices. Really great low-cost hardware that, for my purposes, has been rock solid for years.
Exactly. Soekris was the first thing that came to my mind after seeing this news, having used various Soekris boxes for home router for many years. Eventually got replaced by Ubiquiti EdgeRouter 4 (which are now also impossible to get via common sale channels, besides ebay) running OpenBSD.
At work we have hundreds of Alix/APUs in the field and they have been great, very low hardware failure rate (occasionally a NIC going bad). We will probably make a final order to carry us through and then it's back to the drawing board.
I also have fond memories of my personal Soekris box running m0n0wall in the early 2000s (I think back then pfSense was more bloated/slow on limited hardware). My experience with that setup was definitely what made me consider the PC Engines equipment for work.
Similarly I have also switched to Ubiquiti for home use.
I have a feeling that the MIPS-based Octeon SOCs are EOL or close to it, and Ubiquiti is allocating everything to the Unifi versions of the same hardware.
I used these for home router + wifi until last week... they were pretty good, low power and low heat / fanless, relatively normal x86 with a BIOS tailored for embedded. I can see they're a bit niche both themselves, and using a niche cpu for the vendor and arm64 wants to eat their lunch. But it was good while it lasted.
I coincidentally replaced mine just before this with 2.5G arm64 based board, also fanless.
I had mine APU2C4 for over 6 years, used as an all in one router, WiFi AP, NAS. Replaced it few weeks ago with Odroid H3+. Now I can have way more storage, 64 GB instead of 4 GB and performance while idling at same or lower power (3.5 Wh). APU2 (like the case) did indeed felt a bit more industrial grade though.
There is no storage limitation on PC Engines, you can get 1TB mSata and put inside and it should work without problems. Also apu2 and other have SATA port inside [1], there is also SD card slot as well as USB.
My apu2d4 on OpenBSD has been running perfectly for a number of years now. Sad to see this wrapping up but it was certainly a fun experience. I given Pascal a big thanks and wish him all the best!
Neat history but not all that efficient in terms of heat, instructions per cycle, or money.
While RPi prices are still stupid, there's a variety of low-spec IoT ESP32, STM32, and pocketbeagle. Bananapis are far more capable than RPis for about +50% $.
If you have more money, embedded Ryzen and EPYC such as used by Deciso OPNsense boxes (FreeBSD-based).
None of those cover the same properties, though. PC Engines did reasonably cheap full computers (embedded, or at least headless, but GBs of RAM), x86 (so you didn't need some special hacked up OS image just for your exact board), and coreboot (high quality open source firmware).
This is a sad day for me, I have been running one of these with OpenWRT for a long time and they are absolutely stellar. I was hoping for a new version at some point, perhaps with some higher clock speeds for higher peak loads, but oh well, it was a good run. Thanks, PCEngines!
Nooooo....My APU2 has been kicking butt for past 5 years. Previous to that I used their Alix platform with PFSense. Those things are stable as rock. The only issues I ever had was with CF and SD cards. Actual devices are rock solid.
Unfortunate, but at this point I'm starting to give up for doing routing/firewalling on low power consumer hardware anyway.
At 10G and above, it's just very hard for such CPUs to keep up, and reasonable power usage seems to require hardware acceleration.
I think part of the issue is the tiny ethernet frame size. Dealing with millions of packets per second is just hard on a low power CPU. And unfortunately due to the internet, jumbo frames are only a viable thing on the inside.
So now the equation swings towards purpose-built devices that can actually accelerate routing, bridging and firewalling in hardware.
Most ARM cases would mean a tremendous amount invested in reverse engineering of binary blobs required to configure network and crypto-accelerators. Also, in most cases, it means no support *BSD like OPNsense and pfSense. That leads to fully proprietary designs, lack of transparency, and limited trustworthiness.
What we were able to realize over the years of writing firmware and researching new solutions for network appliance vendors is that support for accelerators is considered a premium feature and typically is available only behind paywall corporate unobtanium. Consider very respected silicon vendors, such as Marvell [1] or NXP [2].
If you have in mind hardware (no matter if x86, Arm, RISC-V, or POWER) that would be able to mark all checkboxes that PC Engines score [3] with 10G performance, then it would be exciting competition otherwise, we are very far from community expectations, where most BSD users with open-source firmware based hardware choose PC Engines or Protectli [4].
Did I just get a message from ChatGPT? Because this comment doesn't quite make sense. I said nothing about ARM, and those links don't lead to quite where I'd expect.
No, just from not native English speaker. You mention high performance 10GbE hardware without being precise about CPU architecture or vendor/model of the firewall. Apparently high power non-consumer grade hardware, but such hardware lacks PC Engines properties. You also mention accelerators and this is what my comments is about: reality of build firewalls.
My comments is about designs that potentially could meet the requirements, but surprisingly those have different problems. How we know those designs? We consult for firewall vendors doing research. We would be glad to be wrong by being pointed to viable competing solution which can reach score close to PC Engines. Performance is not the only factor for small ISP and security-concious privacy-concerned infrastructure builders.
So it is not that trivial to purposefully build firewall these days that can compete with PC Engines low power consumer/SOHO design.
"Despite having used considerable quantities of AMD processors and Intel NICs, we don't get adequate design support for new projects. In addition, the x86 silicon currently offered is not very appealing for our niche of passively cooled boards. After about 20 years of WRAP, ALIX and APU, it is time for me to move on to different things."
If I may, my current NAS runs off a 10+ years old Atom D410 which is way slower than the GX412, still perfectly adequate for managing two ZFS pools and serving them over a local network via NFS and CIFS while running some other services.
I'm sure if you moved from firewalls to other things, you could find other niches in need of new products, especially now that the Raspberry Pi 4, which is comparable speed wise is nearly unobtanium and way overpriced.
How much would it cost to design two new boards, say one with 1 or 2 NICs and 5 SATA ports, then one with one NIC, WiFi and a good number of GPIOs? I would surely buy a few of them, and probably others would as well.
Maybe Netgate [1] for anyone using PcEngines + pfSense CE. It’s less hardware, but functionally close (for throughput) and gets you onto pfSense Plus. I know they’re not liked here, but I doubt there are going to be many (or any) sub $200 firewall appliances that I’d be willing to rely on once the PcEngines supply runs out.
I’ve configured ~150 pfSense CE setups on APU2s. It’s a sad day for small businesses. Hopefully Netgate doesn’t increase prices on that low end appliance. I’d love to be proven wrong, but it’s the only pragmatic option now for small businesses.
I have no personal experience with either of these, but I was considering them when looking for options to handle router duties for my 10 Gbit fiber link:
I've used Protecli in several small office NGFW builds (<20 people, ~5 remote VPN users, 1 dedicated VPN tunnel to home office) and they have been great over the past 4-5 years. Can recommend.
> [Ten64] platform (Layerscape) is somewhat interesting but sadly has been so slow to appear in reasonable products that it has been surpassed by basic Celeron-based systems ... seems to me that the Layerscape platform will go the same route as the Marvell (Armada 385 I think) ones did; one or two revisions and then radio silence where only highly integrated partners still keep using it ... it's not powerful enough as a CPU, and the hardware acceleration suffers from the same problems that basic Broadcom SoCs and higher-end ASIC-accelerators do: they get outdated and useless rather quickly
Ten64's have been shipping for a while now, though you are best to ask in our support forum: https://forum.traverse.com.au/ . We haven't posted much on Crowd Supply as it's a very manual process to get stuff up on there.
I'm not too familiar with TrustZone, but I'm not aware of any limitations in the secure world. I haven't tried OP-TEE or any similar secure world firmware' simply as no one has asked for it.
When we did the research for our network appliance customers, we were very interested in NXP. Have you tried to buy Application Solutions Kit from NXP? We faced the problem of not achieving spec performance using NXP reference boards. After reaching NXP, support informed us that we should buy ASK, which would solve our problems. Unfortunately, it was beyond our research budget.
No, we haven't bought any ASKs. We have worked with them on a DPDK project and they were quite helpful when it came to debugging difficult bugs with it.
Improving the routing performance in Linux is near the top of my TODO list, however. XDP is one candidate (see https://forum.traverse.com.au/t/vyos-build-my-repo/181/5 for some results) and using the LS1088's AIOP is another possibility.
Linux is important but FreeBSD is the key to heart of majority of DYI firewall builders. I'm huge fan of your project and looking for opportunities to get hands on experience with it.
I see that support for FreeBSD going in good direction:
dsl (Dmitry) is the main developer behind the DPAA2 drivers for FreeBSD and he's done a fantastic effort so far. Myself and Bjoern (bz) have also written bits of it.
The performance has improved compared to how it was a year ago (e.g struggling to get 400mbps throughput) but there are some severe issues in -CURRENT :( I believe dsl is trying to get them fixed by 14.0-RELEASE.
used office (thin-)clients make much, much beefier routers and mediacenters without using more power (thou some have fans running at idle load) and costing less.
They are indeed reasonably good for things where it doesn't really matter if it falls down, but I'd be laughed out of the room if I suggested we built a business on top of second-hand thin clients, doubly so when I turned up to a client site carrying on with two USB ethernet adapters and a USB LTE modem hanging off the back to put in a machine room.
I have had an APU2C4 acting as my firewall (running OpnSense) at home for the past few years. It's been pretty a pretty good little box. My singular complaint is that the CPU has always been much slower than I'd like. Standard firewall functionality works fine, but doing any admin stuff involving package management takes ages, even with an SSD.
I'm not sure what I will replace it with when the time comes. Despite talk of alternatives in these comments, there is nothing on the market that checks all of the same boxes that the APU2 does: 1) AMD64 2) fanless 3) low power 4) multiple interfaces 5) under $200.