Hacker News new | past | comments | ask | show | jobs | submit login
Show HN: SatCat5, the open-source FPGA Ethernet switch (github.com/the-aerospace-corporation)
190 points by ooterness 7 months ago | hide | past | favorite | 30 comments
We've just launched v2.5 of SatCat5, the open-source FPGA Ethernet switch [1]. SatCat5 contains various FPGA building blocks that let you build a custom mixed-media Ethernet switch. It was originally intended for cubesats [2] but has many other potential applications.

The headline feature for this release is support for the IEEE-1588 Precision Time Protocol (PTP). SatCat5 has demonstrated end-to-end synchronization to within 50 ps-rms, which is approaching the world-leading performance of CERN's White Rabbit Project [3]. Except we're doing time-transfer over regular, non-synchronous Ethernet. The key breakthrough is a new technology for digital timestamps that we've published in IEEE Access [4].

This project was featured on HN back in 2023 [5]. Since then, we've changed to the CERN-OHL-W v2.0 license, which has much better legal clarity for FPGA projects.

[1] https://github.com/the-aerospace-corporation/satcat5 [2] https://digitalcommons.usu.edu/smallsat/2020/all2020/174/ [3] https://white-rabbit.web.cern.ch/documents/White_Rabbit-a_PT... [4] https://ieeexplore.ieee.org/abstract/document/10367970 [5] https://news.ycombinator.com/item?id=34817550




Hi!

I'm one of the engineers working with cubesat hardware and software integration, and I've been following this project since 2023 because I agree that standardizing over Ethernet over (twisted pair, UART, single pair, etc...) will solve a lot of integration pain from all of the DIY protocols that the cubesat/smallsat community uses

Some thoughts:

- You actually have a publicly available ICD at https://github.com/the-aerospace-corporation/satcat5/tree/ma... , please publicize this more in the readme! You're a very rare exception in a world of contact us and sign an NDA first and commercial in confidence ICDs, which is great, and leads me to my next point:

- Compared to HTTP API documentation, space ICDs sucks. One of the ICDs that I have put its protocol documentation as screenshots instead of copy pasteable text. Of course ICDs sucks because you won't know its quality until you buy it... Anyway standardizing on ethernet for transport layer give you all the tooling at 99% of the world's computer use, and it might be a first step in making a OpenAPI / Swagger type tool but for space protocols

- Any plans for 10base-T1S single pair ethernet? Our 6U cubesats barely have enough power to run its payload computers, let alone a FPGA based switch. But in a way a switched star network can be more reliable than a multidrop bus, as you can isolate babbling idiot nodes, and filter out accidental/intentional MAC/IP address spoofing


Thanks for a very thoughtful reply!

Regarding the Slingshot ICD: Noted, I'll make a point to update the readme. That document is literally the payload ICD for the Slingshot-1 cubesat that launched in July 2022 [1][2]. Future missions may take things in a slightly different direction, but that ICD is a flight-proven point design that shows what's possible. The space industry's NDA-by-default attitude definitely creates a lot of unnecessary friction.

I totally agree that off-the-shelf tooling is a huge benefit of leaning into Ethernet. In particular, Wireshark has been essential in winning over hesitant developers; it's just light-years ahead of what you usually get for debugging microcontrollers.

I've been watching the various 10BASE-T1S / 100BASE-T1 / 1000BASE-T1 standards, but company interest has been limited. AFAIK none of them can be attached directly to FPGA GPIO/SERDES/etc., so you need a separate PHY chip on both ends of the link. For us, the competition is SGMII over coax (for FPGA payloads) or SPI/UART over 4 wires (for microcontroller payloads). Cubesat cable runs are so short that the ability to directly attach to GPIO typically outweighs potential size and weight savings in the cable harness, plus it's one less chip with unknown radiation tolerance, etc.

That said, the good news is that -T1 PHYs typically have one of the usual *MII options (RMII, SGMII, etc.), so many are already compatible with SatCat5. The bad news is 10BASE-T1S requires CS/MA and PLCA logic that is not yet supported. (But if anyone wants to contribute cool features to this open source project, please reach out.)

[1] https://aerospace.org/sites/default/files/2022-08/Slingshot_... [2] https://aerospace.org/article/aerospaces-slingshot-1-demonst...


Great work on this, it looks really promising! The industry is really a mess with a mix of interfaces. On one end there is stuff like RS422 which is simple but only the hardware layer is really standard and normally its not set up or tested to more than 1Mbps. On the other end there is stuff like spacewire which is a good standard but it's not something a university payload team can dive into with COTS parts.

Question: what's the power consumption like? When I was an aerospace grad student, I tried really hard to use ethernet to interface everything for a cubesat-sized spacecraft. But it turned out each port consumes about 0.25W at 10Mbps and 0.5W at 100Mbps. That's per port, so up to 1W just to connect 2 devices. That is a lot when your main cpu consumes 1W, and payloads and other devices often use much less. It would have almost tripled total power consumption.

I looked into the EEE (energy efficient ethernet) standard and it didn't seem workable for using COTS things, it only seemed workable if you had design authority over everything.

I kind of knew changing the physical layer was possible but I didn't know where to start, and anyway the microcontrollers I had picked out at the time because they "had ethernet" turned out to be very hard to use with more standard devices, defeating the point.

Another couple of questions. I like gecko connectors but why not choose the ones with mating screws? Also interested in criteria for selecting the Tyco connectors.

By the way, there is a small company that makes very tiny ethernet switches[1] for drones. There are 100Mbps and 1Gbps versions and they have nice connectors you are probably already familiar with. They are very nice for development of embedded stuff, even if you don't integrate it in your project and just use it for the sake of getting a pile of routers off of your desk.

[1] https://gadgetsmyth.com/shop/

Note: My opinions are my own and not those of my employer.


Power consumption varies a lot with the number of ports and the FPGA platform. The "ac701_proto_v1" example design for the Xilinx Artix7-200T has 4 SPI/UART ports and 4 gigabit ports (a mix of RGMII and SGMII). That design hits around 150-700 mW [1], depending on how many gigabit ports are active.

One way we've saved a lot of power is by skipping the Ethernet PHY. Last time I looked, a 100BASE-T PHY was about 130 mW each and a 1000BASE-T PHY was about 480 mW each. Double that because both ends of the link need a PHY, and that roughly lines up with your numbers.

The upside of using a PHY is that "real Ethernet" can drive hundreds of meters of cable. The PHY chip is effectively a converter from the local interface (*MII) to the long-distance interface (10/100/1000BASE-T). But on a cubesat or a small UAV, cables are short, so you can skip that conversion to save power on both ends. The downside is that the many *MII flavors are not mutually compatible.

Regarding the Gecko connectors, the very simple reason to avoid screws is floorspace. The "Handle" payload interface unit [2] has 12 Gecko connectors in a 4x3 grid. The screw mounts add enough width that it wouldn't fit in the allotted space. The retention clips are quite robust, though you do need a small tool to remove them. There's no reason you couldn't have screws on one end of the cable and clips on the other.

Regarding the Tyco connectors, it's definitely a compromise that we might revisit. Size and signal integrity were the main concerns. The space industry is quite risk-averse, so something with provable signal integrity for 1250 Mbaud SGMII signals was a necessity.

The GadgetSmyth product line is quite interesting. I had not heard of that company before. I'm surprised that their 1000BASE-T pigtails are not using twinax or STP cable, but the switch cards are definitely aligned with what we're trying to accomplish.

[1] https://digitalcommons.usu.edu/smallsat/2020/all2020/174/ [2] https://github.com/the-aerospace-corporation/satcat5/blob/ma...


Running Ethernet without (or close enough to without) PHY was a common trick for 10Gigabit (or more) inside a rack - with Direct-Attach Cables being essentially composed of minimal amount of logic to get two SFP/SFP+/QSFP interfaces on both sides to talk to each other directly.

Thick, ungainly twinax cables, but cheaper than fiber, and 10G-T was latecomer with a lot of teething problems.


Thank you for the replies!


Maybe I’m just looking in the wrong places, but open hardware at this quality level seems to be extremely rare. Does anyone have other good examples of complete, professional-quality open source HDL projects that aren’t CPUs? I’m thinking of the hardware equivalent of PostgreSQL, something truly industrial strength with readable code you can learn from.


The open-source community for gateware is not as large or well-developed as the one for software, but there's many good projects out there.

OpenCores [1] has been around a long time, and there's many high-quality FPGA IP-cores hosted there. The Open Hardware Repository [2] is another good resource. Not all OHWR projects involve FPGAs, but many have full hardware/gateware/software stacks, including the White Rabbit Project [3].

[1] https://opencores.org/ [2] https://ohwr.org/explore/projects/starred [3] https://white-rabbit.web.cern.ch/


I know about OpenCores, but it consists of, well, cores. I’m looking for fully realized projects, with controller, glue logic, external interfaces, test benches, multiple hardware realizations, all the tedious stuff you have to do to make a working version>1 device. That is, what do you do with the cores? To me it seems analogous to the difference between a B-tree library and PostgreSQL. White Rabbit looks great, thanks for the pointer!


Ah, I see. I'm coming from a background where nearly everything is a bespoke one-off design. (After all, if you're making a million of something, why use an FPGA?) My colleagues are usually looking for flexible, multipurpose building blocks that they can assemble into something unusual. That's the main intent for SatCat5, though we do include fully-realized example designs that may be closer to what you're looking for.


As the OP says, FPGAs are kind of niche and devoted to specialized applications. For example there's Corundum[0] for in-network compute, kind of exotic and maybe too pricey for a hobbyist. The biggest open-source thing out there is of course MiSTer, but it is tied to a particular old dev board and toolchain last I checked.

>what do you _do_ with the cores?

I mean, this is kind of the problem, but in a way it's a funny problem. The strength and the curse of FPGAs is that they can become almost anything you want. However getting that sea of logic into the desired form requires knowledge of the FPGA family, the tools, the hardware, and digital design in general. The "tedious stuff" unfortunately makes up a great deal of FPGA work! Many days I feel like a glorified plumber...

As far as cores go, in FPGA land, the vendor would probably already provide a core (IP, in hardware parlance) for say fooing bars, that is easier to use and probably a far sight more reliable than the project on OpenCores. (Which probably hasn't been touched in a decade, is written in VHDL-87, is using Wishbone when you want AXI, and lacks a test suite.)

As a hobbyist, you might search and find the project on OpenCores, but be scared away by the cruftiness. You then write your own foo_bars module, thereby learning a great deal, but contributing to the fragmentation problem. Plus seeding the internet with weird code with corner-case bugs for LLMs to trip over.

[0]: https://github.com/corundum/corundum


In software world, I can point someone to PostgreSQL (I’ll keep using that example, though my point is there are lots of examples!) to see a fully-realized product including all the tedious stuff. And it’s nothing to do with hobbyists, this exact codebase is the same one that keeps billion-dollar businesses running.

What are the equivalent projects of equal professionalism and completeness in open hardware world? Maybe Open Compute Project?

Or does the jump to “hobbyists” indicate there is very little like that, in other words virtually all serious hardware is not open?


Most serious hardware is not going to be open. I think this is mainly because hardware companies need higher margins, because working with physical things costs money. For most companies that use FPGAs, the custom stuff is the secret sauce, kept under lock and key. Most open source stuff out there is either hobby, or effectively serves as an advertisement for design services.

I'm not that familiar with OCP but certainly driving commodity hardware costs down is a valid object. My interest is more in specialized hardware at the board level. I believe CERN and other high-energy physics labs collaborate on designs, but I've been out of the field for a long time. Much of that sort of hardware doesn't have use in the industry at large; historically speaking, if there is demand, the model has been to spin out a small company to sell equipment to other labs.

The only category of core with what I'd call a big open-source impact is RISC-V processors, as you mentioned above. This is because a small processor is often useful on a non-SoC FPGA. But there were already free-as-in-beer microprocessors available from the FPGA vendors, so RISC-V adoption is really more of a political/philosophical choice.

When people talk about someday having an OSHW scene approaching the vibrancy of the FOSS scene, I'm not so sure there are parallels. People talk about having open tooling, but the existing tooling is already free-as-in-beer (for the device types most people could afford, anyway). The cost of the vendor IPs is already baked into the cost of the device, so you'd still be paying for it. The argument is that open source tools will be easier to use, but I haven't found that to be the case at this point in time.

I think that hardware is just a fundamentally different medium from software, and attracts different kinds of people. It's a comparatively smaller crowd, who tend to be lone wolves. Maybe someday there'll be a critical mass.


Huh, what a neat project!

Especially the Ethernet-over-UART/SPI/whatever part is quite interesting. It's incredibly obvious in hindsight, but it's of course a great way to create affordable wired IoT devices. You could just talk Ethernet over RS485, and be able to communicate to a bunch of devices with a single $20 translator and a $0.15 transceiver per device!


Yes, that's a great example of the "mixed-media" use case for SatCat5. If you have an Ethernet switch where some ports are Ethernet-over-RS485, and others are Ethernet-over-10/100/1000BASE-T, then the switch doubles as a sort of media converter. Using a common transport layer means it's easier to link sensors, actuators, and compute resources.


One of my dream projects would be to grab some off-the-shelf switch IC and build OSHW switch around it with fully free blobless Linux based software stack. Something like SparX-5 [1] could work here, and the vendor software is already Linux based (but presumably absolutely horrible). The problem is that you'd need fair chunk of cash to get anywhere, so this basically needs a kickstarter or something to get going.

[1] https://www.microchip.com/en-us/product/vsc7552


You might be interested in some of the stuff Andrew Zonenberg is working on: https://ioc.exchange/@Azonenberg. I don’t have a link to a specific post, he usually has a couple of things on the go and they all get interleaved. Pretty wild Ethernet stuff in there!


Why Ethernet and not SpaceWire? Surely you don't need multiple hundreds of mbit/sec across a cubesat?


We continue to use SpaceWire where it's appropriate, and it was one of many options considered in the initial trade-study [1] that led to the creation of SatCat5. Among other things, many Aerospace-built cubesats have a laser communication downlink [2][3] that can reach 250 Mbps and up. It's not hard for raw sensor or video data to saturate a 1 Gbps link, and future missions may even need 10 GbE.

[1] https://digitalcommons.usu.edu/smallsat/2020/all2020/174/ [2] https://www.eoportal.org/satellite-missions/aerocube-ocsd [3] https://aerospace.org/article/aerospaces-slingshot-1-demonst...


I had no idea that laser downlinks were even remotely possible from a cubesat - that's really kinda astonishing. Regarding the other high bandwidth stuff I imagined that "a communication satellite" was more or less a life support machine for a box made by Nokia.

CAN is actually kinda cool, should you ever feel like going down a rabbit hole.


In my next life I will go to college an study computer engineering. It never ceases to amaze me the cool stuff you hardware hackers can do.


Interesting. Seems like it could be useful for custom ethernet protocols, eg. USB or HDMI over ethernet


Oh, is there an open source switch that supports 10 Gbps SFP+?


I make my own open source FPGA switches, 6 and 12 port 10 Gbps SFP+ up to 100 Gbps QSFP28. Older model described in [1]. Email me if you want one.

I also sell cheap $150 10 Gbps and 25 Gbps QSFP28 switches but they are only partially open source.

They still sell some of the NetFPGA models, very pricy [2].

There are a few Chinese and Russian switches but you can't buy them anymore because of the war.

[1] https://www.researchgate.net/profile/Merik-Voswinkel/publica...

[2] https://netfpga.org


> I make my own open source FPGA switches, 6 and 12 port 10 Gbps SFP+ up to 100 Gbps QSFP28.

How big is it physically? I don't really get it from the link.


There is a picture of a FPGA switch in the pdf [1]. You usually put it in a outdoor or indoor metal case or a PC slot.


Development of SFP+/XAUI is on our wishlist for SatCat5, but not yet supported.


How do you packetise stream formats (ie UART)?


SatCat5 uses the Serial Line Internet Protocol (SLIP) [1] to provide frame delimiters. Each SLIP-delimited frame contains the normal Ethernet MAC-layer header and footer (i.e., destination MAC address, source MAC address, EtherType, user data, CRC32 frame check sequence).

[1] https://en.wikipedia.org/wiki/Serial_Line_Internet_Protocol


SLIP lives!




Consider applying for YC's W25 batch! Applications are open till Nov 12.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: