
Raspberry Pi – UASP, Trim, and Boot Performance via USB - geerlingguy
https://www.jeffgeerling.com/blog/2020/raspberry-pi-usb-boot-uasp-trim-and-performance
======
zaroth
Wow, the animated gif of the Defrag running sent me _way_ back.

I used to run that like it was religion, watching the blocks all move together
and change color, and knowing the machine would run just a bit faster after it
was all done was a bit of magic.

~~~
xattt
It _felt_ like you were cleaning and tidying up.

------
NortySpock
Just wanted to say thanks for doing this research on flash speeds and posting
your results for the last few years, they've helped me pick both reasonable
sd-cards and mid-cost USB thumb drives that work for my budget and RPi 4.

------
jeswin
Samsung Evo+ cards with the same speed rating (say U1) and model number can
vary in performance quite a bit. I have tried three of them on my rpi4, and
found one which was considerably faster than the others. To be sure I verified
it with crystaldiskmark and indeed, it was up to 2x faster in writes and
reads.

Samsung cards tend to outperform other brands in that price range.

------
alexellisuk
Good to see you up here on the front page Jeff, well-deserved for all your RPi
tutorials and videos.

~~~
geerlingguy
Ha, good to see you in the comments too!

------
geerlingguy
tl;dr - the Pi 4 now has USB boot enabled by default, and you can get way
better performance from a fast USB 3.0 drive than the built-in microSD.

I like the Arcanite for large files and basic storage needs, or an NVMe + USB
3.0 adapter for the best performance for a reasonable price.

UASP can speed up operations by up to 50%, and many SSDs and high end flash
drives do support it. TRIM support is less prevalent, and you often need to
manually enable it (there's a guide linked in the post).

Weirdest finding: the Arcanite flash drive firmware indicates it supports
trim, but when I tried running fstrim something blew up in the drive, and I
could no longer read/mount/initialize it from Mac/Linux/Windows (I had to buy
a replacement to finish the testing).

~~~
dvdbloc
Do you mean the Arcanite flash drive literally let the smoke out or did it
just stop working?

~~~
rwmj
Not likely to be smoke. However TRIM failures were very common with SSDs early
on and even today some distros disable or partially disable it by default (see
for example this Debian warning:
[https://wiki.debian.org/SSDOptimization#WARNING](https://wiki.debian.org/SSDOptimization#WARNING)).
Upstream Linux does not enable TRIM for MD-raid devices unless you use a
special kernel boot parameter. "Continuous TRIM" (which means setting the
discard parameter on the filesystem so that files are automatically trimmed on
deletion) is also a good way to reveal bugs and problems, some of which will
merely kill performance when you delete files, and others will nuke your data.

------
avian
Does anyone have any experience how reliable is such modern USB attached
storage when run 24/7? Some years back I made a box (x86, not RPi) with a
bunch of USB connected drives (from Western Digital, if I remember correctly).
After a few days of uptime the drives would simply drop from the bus. I was
guessing there was a bug in the drive’s USB controller.

These devices are usually meant to be used as a carrier that’s only connected
to a computer to load/off load data. Long-term operation probably isn’t a
priority when developing them.

~~~
phoboslab
Not related to USB, but flash storage quality in general varies a lot.

We have a few hundred Raspberry Pi in operation at customer sites, running
24/7\. Back when we started our business we used generic Micro SD (can't
remember the brand). They would fail after 2-6 months. This was a big hassle.
We have customers all over Germany and would need to ship them a complete RPI
as a replacement, as the customers are usually not tech-savvy enough to just
replace the SD.

We now use SanDisk's "Industrial" Micro SD and haven't had this problem since.

Also, reducing writes to the SD is certainly a good idea. Ideally - if you can
- mount the system as read only.

~~~
CraigRood
Interesting you mention "Industrial" cards as I have been looking at cards in
this range after reading quite a few issues with RPIs over the past few
months. I have a RPI3 with a SanDisk Extreme, running for 18 months 24/7 with
no issues yet. Plenty of re-imaging and high temps. RPI 4 has a SanDisk Ultra,
only been running for 3 months 24/7\. Now whilst we all should have backups
anyway, I really don't want the worry or hassle of restoring backups couple of
times a year, I would rather have a card that will last years rather than
months.

Slightly off topic but I have also been looking at using net boot instead. Not
only would it be just as fast as the Micro SD cards but much easier to perform
backups locally and offsite.

------
megous
How does this change with LUKS enabled? I don't see a point of storing data
unencrypted these days.

~~~
dijit
For normal laptops it won’t be so much difference in performance, as LUKS by
default uses AES-XT as the encryption type, and most modern CPUs have AES
accelerators in them.

For rPi though? Not sure they have an AES accelerator, and if they don’t then
it’s going to be a huge cost to performance.

Not only because USB is using a CPU core to operate (because USB uses CPU not
dedicated hardware like FireWire) but also because it has to transparently
translate and encrypt every access.

It seems like the raspberry actually has zero crypto extensions anyway:
[https://www.raspberrypi.org/forums/viewtopic.php?t=207888](https://www.raspberrypi.org/forums/viewtopic.php?t=207888)

~~~
I_Byte
>Not sure they have an AES accelerator

The ARM chips in the all of the Raspberry pis do not have AES accelerators.
Encrypting your pis drive is going to be a serious hit on performance like you
said. I don't recommend it.

If you really really really want to encrypt your Rpi then you need to use a
disk cipher like Adiantum. It is based off of ChaCha12 and was made
specifically for low end CPUs that lack AES accelerators.

------
londons_explore
These reports always comment on raw performance without checking if the drive
actually meets the storage contract with the host.

That's like having a 'hall of fame' for the fastest marathon runners, but not
having anyone keep an eye on if they took a taxi half the route...

These benchmarks should always have a section where a drive has its power
pulled repeatedly while under heavy load. If, ever, a single sector becomes
overwritten, rolled back, or unreadable, except those being written between a
pair of write barriers, then the drive has failed the test, and all other
performance results thrown out. That drive took the taxi.

------
brodouevencode
As an end user of Raspis and all their improvements over the past few years,
to anyone connected to or working on the project, thank you.

------
intrasight
I've a rpi4 that I'm setting up as a media player. Files come from a NAS. In
the past (rpi3) I've just used an SD card in the Pi. I'm curious if I'd get
better performance using USB. Also curious if I did use USB if I could setup
automatic caching to the drive.

~~~
NortySpock
I'm using an RPi 4 as a NAS/home server and the difference between SD speeds
and USB speeds was pretty noticeable (subjectively); even simple linux
commands (ls, cp -r, mv) sped up after switching to a SanDisk Ultra Flair.
Except for one USB drive...

Recently I had the misfortune of having a debian update screw up my docker
daemon and I ran over to Best Buy to grab any-old-USB drive to use as a new OS
drive. As it happens the SD copy over went fine, but as I performed basic
"setting the server up" operations like copying over the old data (50 GB of
various sizes) and firing up docker-compose scripts, performance deteriorated
rapidly to the point where a docker install of a lighttpd container took at
least 30 minutes and the system became effectively unusable.

The poor-performing drive was the PNY Elite-X Fit, USB 3.0, 128GB, purchased
for ~$22

I've switched back to a SanDisk SD card for the moment which is perfectly
adequate for the next few months but I hope to get back to running off a USB
soon when I'm not so busy.

------
destitude
Is there a reason the XPG SX8200 256GB SSD with 3500MB/s wasn't tested? Would
there be a performance boost compared to what was tested on the Pi 4?

