Hacker News new | past | comments | ask | show | jobs | submit login

> In the software world, Crashlytics, Sentry, and other error monitoring systems have been offering similar solutions for years. Memfault is the first such solution for firmware.

Just curious, why can't Sentry be used in a firmware? (I don't do firmware dev)

There are a few concepts that do not map neatly:

1. For firmware, each user is on their own hardware. Rather than a session you need to track a device and the state thereof. Devices exist for a longer period of time than sessions do, and you need to have a concept of "device history".

2. Sentry assumes backtraces can be generated on the client side, which is impractical for firwmare

3. Our focus on embedded allows us to run some more specific analysis. For example we automatically detect if your MPU (memory protection unit) is misconfigured on an ARM chip.

I built a translator service between my devices and Sentry to take the memory dump and a known symbol file to rebuild the context serverside and send it back to Sentry. I only used this during development and early testing, but it was super useful.

At one point I thought about building this out more, so I'm glad to see someone is taking this on more seriously.

We are taking it very seriously ;)

Happy to hear that an you thought to build this translation service even for early development! It's usually an after thought at the companies we've talked to, and an expensive one too.

> 2. Sentry assumes backtraces can be generated on the client side, which is impractical for firwmare

If you can produce a minidump you can send it to Sentry and have it stackwalked on the server. Breakpad/Crashpad can produce such dumps for you. We do not have coredumps yet but that could be added if there is demand.

Yeah, minidump and breakpad/crashpad can be great when running native code on an OS!

One challenge with embedded devices (i.e ARM Cortex-Ms) is that it is not always safe/viable to try and grab the thread contexts at the time of crash. These MCUs have none or very limited support for memory management (a MPU at best), the OS/application state usually lives in the same RAM area, and the stack dealing with exception handling is usually quite small. For these reasons, it's usually desirable to grab a very simple memory dump and offload as much of the processing as possible to the server. The "core dump" format that is used for linux is saved as an ELF with a few special PT_NOTE sections to convey thread information. This would be hard to generate on most embedded devices. The coredump we collect for embedded devices is closer to a mini-dump. It is basically just a raw memory dump and the current register state on the device. On the server side, we recover the thread contexts and backtraces by using the debug symbols, the RAM capture, and analyzers for the RTOS that was used (i.e FreeRTOS, Zephyr, ThreadX, Mbed OS)

We have customers using us for firmware errors :)

Yes, I was excited to meet some of them! It is certainly possible to use Sentry with firmware, but I'd venture to say that Memfault is an easier integration and a better experience.

Makes sense. Thanks!

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact