
Adafruit's Read-Only Raspberry Pi - tdrnd
https://www.raspberrypi.org/blog/adafruits-read-only/
======
dividuum
I'm operating a digital signage service for the Raspberry Pi ([https://info-
beamer.com/hosted](https://info-beamer.com/hosted)). When we started designing
our Linux based custom operating system we also decided to go with the read-
only approach. So far we haven't had a single device go offline due to SD
card/file system problems. It was really the only sensible way to do this as
you never know when/if devices get turned off.

We also handle system updates by using a A/B partitioning scheme, so you can
even turn off a device in the middle of an update without breaking it. The
additional benefit from that is that all system updates are atomic and self-
contained, so you always have the newest version without any potential
leftovers from the previous version (or the previous version if the update got
interrupted). Customers that have installed our first version 3 years ago now
run the exact same OS that we released today. It makes it really easy to
provide support.

For obvious reasons we also need a read/write partition to store downloaded
assets. We treat this partition as volatile and have an easy way to
automatically verify and repair content if anything gets corrupted for
whatever reason.

If you want anything running reliably on a Pi, I feel this is the only way to
do this.

~~~
megous
Have you tested unexpected powerdowns durring writes? Unless SD card is
specifically designed to survive this, I would guess you might get corruption
anywhere on the card, despite writing to specific area of the card, due to
wear leveling algorithms. It might only become an issue when the card was used
for a while, so that all flash blocks were provisioned at least once.

~~~
dividuum
Not 100% of course, but so far we haven't encountered any problem. We also
checksum both the operating system and all downloaded data periodically so we
can catch and repair any corruption. Of course this won't help if the system
doesn't boot any more. So far we haven't had this problem.

In addition to the system part being read-only, we also try to minimized
writes to the data partition. We're using some of the ext4 settings to make
sure writes happen less frequently as we're perfectly capable of recovering
from lost data after a reboot.

~~~
k_f
I actually implemented pretty much the same layout (custom kernel, A/B
partitions, but one each for the OS and the player software) for a similar use
case (background music player). We used AUFS for partition layering. In
particular, our setup was as such:

OS: SquashFS file system (ro) -> Config partition (ro/rw) -> tmpfs partition
(rw)

Player software: SquashFS file system (ro) -> Data partition (ro/rw) -> tmpfs
partition (rw)

The config partition is mounted read-only, all changes are written to the
tmpfs partition. We then use a script to bulk write desired files to the
Config partition (i.e. remount rw, copy files, remount ro). This limits the
write time to a fraction of a second (these are only small text files).

As for the data partition, we remount the partition rw when we're updating the
music files and check for filesystem consistency on bootup.

We hadn't had a single system fail because of the SD card in over 2 years.

~~~
dividuum
Neat! We're also using a SquashFS root file system. It's only 30MB total at
the moment.

Right now there isn't a way to do remote system configuration changes (through
a UI, it's doable from a command line). Most changes happen through the "apps"
deployed on those devices and those configuration is treated as volatile and
might get lost during power outages (but then repaired later). If we decide to
allow modification of the few system settings we have, I might treat them as a
new A/B cycle and recreate the complete system with the changed settings on
the other partition. That way I also get automated fallback if anything is
misconfigured and (for example) the device fails to connect to the new WiFi.

------
mschuster91
That doesn't help against corruption on the SD card itself from unexpected
power down or cheap power supplies... especially the latter has bitten me more
often than I can count.

~~~
prophesi
I'm a bit confused. The article says "They’ve shared a script which forces the
Raspberry Pi to run in read-only mode, so that powering it down via a plug
pull will not corrupt the SD card." \- Is that not the same as an unexpected
power down?

~~~
mschuster91
That's two different sources of corruption. The shared script prevents that
the SD card is being written to - which means that the filesystem won't get
corrupted due to half-written or inconsistently written data. The other source
of corruption is the SD card's microcontroller which does not know that the SD
card is being used in readonly mode and may do stuff like housekeeping,
garbage collection etc. nevertheless - which might be interrupted by a power-
out.

~~~
zaroth
If you have the OS level setup for read-only access, can't you also flip a
physical write-protect switch on the media at that point to defend against
even that?

~~~
joezydeco
The write-protect switch on SD cards is not a physical break of a write line.
It's merely a signal to the host saying not to write. A malicious or badly
written driver can disobey it.

~~~
zaroth
I would have guessed a signal to the local microprocessor to stop any writes
as well. Oh well.

So would the solution be for the host, or some SD adapter, to provide a super-
cap to ensure smooth shutdown of the SD unit as a whole?

~~~
mschuster91
Hmm. A supercap combined with a diode on 5V and a line to a GPIO from Vcc to
provide enough time for the OS to shut down... like this

    
    
        (RPi GPIO)        <-----------(resistor to 3v3)-|
        (RPi microUSB in) <-------------(diode <-)------+---- Vcc
                           | Supercap
        (RPi GND)         ----------------------------------- GND
    

When the supercap is huge enough (or, for what its worth, use a powerbank with
passthrough capability!), this should work - when the GPIO goes low, the
system can initiate a safe shutdown, and especially it can signal a "shutdown"
signal to the microSD.

------
pishpash
A proper button to trigger the 'shut-down' sequence would be better.

~~~
dividuum
You can't rely on users to actually do this every time. The more robust you
make your system the better.

~~~
pishpash
On the contrary, a sudden power loss is an exceptional case. Not having a
button is just Raspberry Pi's (generally poor) hardware design decision, along
with countless others relating to the power system and usability. Making your
system entirely read-only _for this particular purpose_ is a hack, not a
solution.

~~~
dividuum
Of course it's a solution. It solve (among other things) the problem of
unexpected power loss. You can't design your system for the happy path and
hope for the best.

~~~
megous
Do you turn off your computer by pulling the plug? Anyway, I was just reminded
of 90's. The times before ATX.

~~~
dividuum
Of course not. Because my OS isn't specifically designed for that. If it was
(like my router): Of course.

------
polpo
It's good to see a nice easy script to do this. I can confirm that mounting
the SD card as read-only works great. For a couple years, I used a Raspberry
Pi as an AirPlay receiver hooked to a set of speakers, and whenever I was done
listening, simply turned off the power. I had / mounted read-only and never
had SD card corruption.

------
codewritinfool
Reminds me of the "IPE" project, Industrial Perennial Environment. I've
successfully used that for a few things, some of which is still running
without corruption for years now.

~~~
Animats
Good recovery from power fail ought to be the normal case. It is for just
about everything else. What's wrong? Does the thing need better shutdown
management hardware? Is it powering down without going into reset before the
hardware starts malfunctioning? (The Motorola 68HC11 was notorious for that.)
Or what?

BeagleBoard has this problem, too.[1] They have a power managment IC and
shouldn't have the problem, but it's not working right.

[1]
[https://elinux.org/Beagleboard:BeagleBoneBlack#Improper_Powe...](https://elinux.org/Beagleboard:BeagleBoneBlack#Improper_Power_Down....All_Revisions)

------
soapdog
Isn't this how Alpine Linux [1] normally works?

[1]: [http://alpinelinux.org](http://alpinelinux.org)

~~~
atomi
You can write changes to the card with a special command.

~~~
soapdog
yes, but running from RAM is the default and thus safe to unplug at any time
IIRC...

------
TeMPOraL
> _The only fix? Wipe the SD card clean and start over, waving goodbye to all
> files you didn’t back up._

> _making your Pi read-only is irreversible_

I understand the article is intended for people not very experienced with
computing and hardware, but come on. The only fix? Irreversible? Scaring the
readers bit too much, IMO.

------
stcredzero
Would it be possible to have a laptop or desktop that spends most of its time
operating in such a Read-Only mode?

~~~
majewsky
From what I've seen, the systemd developers are interested in going into this
direction. They established /run/ as a tmpfs that is available very early
during boot, and built systemd such that it never needs to write to a
persistent filesystem during its operation. (It does, by default, write logs
to /var/log/journal, but you can switch it to non-persistent storage at
/run/log/journal instead.)

Many programs can be made to work the same way. There are still plenty of
programs that need to write longterm state to /var/lib, but you will probably
need a writable partition anyway for /home, so you can just reuse that
partition for /var/lib as well.

OS updates would then be handled via an A/B partitioning scheme, where you
have two copies of the system partition, and when an OS update is ready to be
installed, the inactive one is mounted read-write and the update written onto
it.

------
akerro
OpenELEC has it by default, nothing new or revolutionary.

