
How I recovered a lost email from my email client’s memory - weinzierl
https://www.ctrl.blog/entry/restore-lost-email-from-ram.html
======
cranekam
> Evolution [..] has a bad data loss bug. It sometimes deletes the email body
> text in the compose window after changing the signature [..] it has bitten
> me about twice a month for the last two years.

and

> I might have been able to partially recover the message from the Draft
> folder if I’d retained my cool and acted immediately. It had been
> overwritten by an empty message instead. I must look into versioning my
> email draft folder at a later time.

This person has a much greater tolerance for shitty software than I do. I'm
certainly not a perfectionist and appreciate that almost all software has
bugs, but come on! Arbitrarily deleting draft emails twice a month _for two
years_? Requiring convoluted versioned draft folders to work around this
glaring issue? Why are they punishing themselves like this?! They must find
something really awesome about Evolution to deal with this level of annoying.

~~~
Joeri
Maybe they have an exchange mailbox? If you want exchange support on linux
with full syncing of mail, calendar and tasks there are two options I know of:
evolution and hiri. And hiri is paid abandonware. I’ve set up both, both are
not good. Lately I’ve been using outlook web access, which is still bad, just
not as bad.

Really I guess I miss outlook for windows. There I’ve said it. Judge me if you
will. Its search feature is broken, but everything else worked well.

~~~
deadbunny
Option 3: davmail[1] acts as a translator between Microsoft's proprietary
protocols and open protocols.

1\. [http://davmail.sourceforge.net/](http://davmail.sourceforge.net/)

------
thedanbob
Does anyone know of a good Linux email client that isn’t crippled by show-
stopping bugs of this sort? I used Mailspring for a while, which has a nice
modern interface, but quit after discovering that my drafts were only saved
locally, not on the server. This has been an open bug for at least two years.

In the end I’ve always fallen back to Thunderbird as the least bad option.

~~~
petepete
Have you tried Geary? I helped a colleague get set up with ElementaryOS a
while back and it looked really nice and, compared to Evolution, was much
lighter and faster.

I haven't used it in anger though, I tend to stick to email in the browser
these days.

~~~
thedanbob
I did use Geary for a while, I can't remember now why I stopped. I appreciated
how lightweight it was but I think it was just a little _too_ lightweight,
missing one or two features that I'd rather put up with thunderbird than live
without.

------
lixtra
Of course recoverin an email is an innocent disguise.

The same approach works for recovering any secret information that people used
on a computer that an attacker can access. Of course there are plenty of
possibilities. But it’s eye opening to see them in action.

~~~
xfitm3
Yes, encryption keys can persist in memory, too. That's why many law
enforcement agencies use something like a HotPlug[1] + mouse jiggler to keep
machines powered on when executing a search warrant.

[1] [https://www.cru-
inc.com/products/wiebetech/hotplug_field_kit...](https://www.cru-
inc.com/products/wiebetech/hotplug_field_kit_product/)

~~~
WrtCdEvrydy
If you don't want to pay for HotPlug, you can also grab a full memory dump
using FTK Imager or Belkasoft...

~~~
segfaultbuserr
It's why operating systems should implement a lockdown option to restrict
users from performing arbitrary access to memory or kernel, even if the user
is root. I mentioned before that, on one of my computer, I completely disabled
dynamic kernel modules, hotpatching, /dev/mem, no ptrace() to arbitrary
process, etc., making it difficult for root to do any low-level access to
memory or kernel. I also enabled IOMMU, it isolates the address spaces of
different hardware from each other, so no external hardware cannot have
arbitrary RAM access via DMA, hardware-based memory capturer won't work. The
only way to attack is either an 0day or a cold-boot attack, the 0day threat
can be reduced by using a security-minded kernel, like PaX/grsec (not
available to the public anymore), OpenBSD, or HardenedBSD. As for cold-boot
attack, future hardware may support full memory encryption [0] at the hardware
level and fix this vulnerability. Mouse jiggler is a problem, but USB
firewalls already exist [1], if proper policies is enforced by the firewall,
unauthorized hardware cannot register as an input device.

There may be still some exploits, especially when you consider that Linux
kernel is not designed with security as its first priority, and over the last
20 years a lot of black magic has been developed to insert bad things into the
kernel, but at least doing the countermeasures I mentioned will make it
difficult. Hence, it's impossible to do any low-level changing or debugging on
the system without rebooting it - which will immediately revert the system
back to a "at rest" state, and triggers full-disk encryption. Other people may
choose to do the opposite, it's a tradeoff between uptime and security.

Unfortunately, any attempt to introduce such a lockdown will be accused of
being an evil technology that enables DRM. However, ultimately, the question
is _not_ whether a computer is locked down, but who is in control of the
computer and it's locked down to protect whom.

[0] Don't confuse "memory scrambling" and "memory encryption". The vast
majority of PCs today already use memory scrambling - the memory controller
will "scramble" the data in RAM to a seemingly-random pattern using a Linear
Feedback Shift Register, but it's done for electrical considerations - if
there are too many 1s or 0s in a row, excessive current spike (di/dt) is
produced, and it reduces signal integrity and creating excessive
electromagnetic interference - LFSR-based scrambling is not for cryptography
purposes and trivial to decode. On the other hand, memory encryption is a true
solution that provides cryptographic protection to the RAM, and many hardware
vendors have roadmap to implement it. Currently, it seems that there are two
types, the first type is a "full memory encryption" \- protecting RAM from
physical access, the second type is "per-application memory encryption", which
allows an application to request a segment of encrypted memory with an unique
key - protect sensitive data of one application from accidental access by
other programs. Both are helpful.

[1] [https://lwn.net/Articles/738306/](https://lwn.net/Articles/738306/)

~~~
xfitm3
I forgot all about grsec - sad to see its no longer publicly available. Thanks
for the tip on IOMMU and your other measures.

~~~
segfaultbuserr
Just a note for you (and other readers), as I think it needs some elaboration.

> _no ptrace() to arbitrary process_

Traditionally, ptrace() restriction is a grsec feature. But in mainline
kernels, the same feature is available in the Yama module, see [0]. Use Yama
with "kernel.yama.ptrace_scope = 3" will permanently disable ptrace() for all
users, including root, and it cannot be enabled again. Then, you should also
compile your own kernel, so you can disable /dev/kmem (CONFIG_DEVKMEM),
/dev/mem (CONFIG_DEVMEM) and /proc/kcore (CONFIG_PROC_KCORE) in the Linux
kernel. Also, I forgot to mention kexec(), which allows the attacker to
execute another kernel without rebooting, so CONFIG_KEXEC should be disabled
as well. And the list goes on and on, I think it's necessary to download an
old grsec kernel, and using the configuration section of grsec as a checklist
(and try disabling them using mainline technique if possible) if your security
is serious business.

If you do these things, it will block the technique described in the original
article.

[0]
[https://www.kernel.org/doc/Documentation/security/Yama.txt](https://www.kernel.org/doc/Documentation/security/Yama.txt)

------
weinzierl
> There are specialized tools you can use to analyze this data blob.

These are called _file carving_ tools and two better known ones are _foremost_
and it's successor _scalpel_ [1].

[1]
[https://github.com/sleuthkit/scalpel](https://github.com/sleuthkit/scalpel)

~~~
dr_zoidberg
Scalpel, as good as it was back in its time, sadly has been stalled. Carrier
and/or the folks in charge of The Sleuth Kit have taken it into their github
repo[0] but there haven't been commits for ~7 years now.

I did a thesis on file carving some 10 years ago, and scalpels ideas where
very good back then. Photorec[1], however, has been the gold standard for a
long time on (open source) file carving. It can handle text based formats way
better (scalpel is severely limited in this aspect due to the "header/footer"
paradigm), and is a wonder with stream based formats (that can have boundaries
on the bit level).

And it's not because they authors weren't good[2], I think what mainly
happened is that they didn't have the time to keep maintaining the software
they created (I know that has happened to me more than once).

There are also some commercial file carving tools, though most are aimed at
having better integration with forensics software (like Encase, FTK, Oxygen,
etc) or automate parts of the process, like document analysis. Still, if you
just want to compare them by their ability to recover files, I'm pretty sure
Photorec makes it to the top.

[0]
[https://github.com/sleuthkit/scalpel](https://github.com/sleuthkit/scalpel)

[1]
[https://www.cgsecurity.org/wiki/TestDisk_Download](https://www.cgsecurity.org/wiki/TestDisk_Download)
(PhotoRec is part of TestDisk)

[2] They're some of the best in the field of digital forensics

~~~
weinzierl
Good to know! I always had ignored Photorec because I thought it is only for
image formats.

To add to your list of options there is also YARA when used with appropriate
rules. I don't know how it stacks up against specialized tools though.

~~~
dr_zoidberg
Photorec supports a crazy ammount of file types (about 400 I think, but since
they keeep adding it may well be over). Fun thing: Diablo II savefiles (and
other games!) are carve-able with Photorec.

And it can also handle fragmentation (though I haven't tested the later
versions to see how strong that is).

------
Stierlitz
I've noticed the same with web-able apps, you spend time typing-up some
missive and then it freezes and refreshing the page loses it all. Could linux
write everything to a file every 30 seconds. A bit like a keylogger, only you
know it is there.

~~~
code_duck
There used to be a Firefox add-on called Lazarus that did this.

[https://www.pcworld.com/article/227948/Firefox.html](https://www.pcworld.com/article/227948/Firefox.html)

From Tom's Hardware:

"Lazarus: Form Recovery is a free downloadable Add-On for the Firefox web
browser that automatically saves everything you type into forms of web pages
you visit.

With Lazarus: Form Recovery, you will never lose what you write after a crash
the browser or other technical problems. In the case when a problem, simply
right click and select "recover form" to retrieve data previously typed."

However, that was for web forms, not an email client or other applications.

~~~
teddyh
I’ve found that this can replace Lazarus in modern Firefox:

[https://addons.mozilla.org/en-US/firefox/addon/form-
history-...](https://addons.mozilla.org/en-US/firefox/addon/form-history-
control/)

~~~
pabs3
Thats really buggy with Firefox ESR, Mozilla are adding more WebExt APIs to
make it work better though so hopefully it will work at some point.

------
haddr
I was using Evolution on daily basis at work (around 2015-17) but I switched
quickly to Thunderbird due to stability issues. I was using Evolution
primarily for its support for Exchange Server, but it wasn't very stable at
the time. On the other hand, the same was possible in Thunderbird through a
very solid proprietary plugin (exQuilla).

------
dredmorbius
I used to (when this was still possible) dump /proc/memory (or kmem?) to file
and rummage through it looking for partially composed website submissions when
Netscape decided to eat itself, back in the 1990s. Remarkably successsful.

~~~
mellow2020
Using Process Explorer at least, it's still just a right click on any process
away.

~~~
dredmorbius
There used to be a global system memory file in /proc, 2.0 / 2.2 kernel
series, unless my memory's playing tricks on me. Even content from dead
processes was still (briefly) available.

------
paulpauper
Yup..this is also how the fbi recovers stuff too. After Ross Ulbright was
caught they did this to his laptop.

~~~
saagarjha
I thought they just kept the computer open?

------
joyj2nd
Because of some reasons I use evolution and thunderbird. The search function
in Evolution is abysmal.

May try out InScribe

------
therealmarv
nano ftw.

Modern web email clients have this since a long time and Gmail never crashed
internally for me ;)

~~~
saagarjha
I use nano, but I have to say I am very surprised that they mentioned it as a
viewer for large binary files. It reads the entire file off of disk into
memory and then tries to split it into lines! It can't even do well on files
with large enough lines :(

~~~
d2wa
Nano reads large files in chunks. Other text editors do indeed pull in the
entire file before doing anything useful.

~~~
saagarjha
No, it reads the whole file at once:
[https://git.savannah.gnu.org/cgit/nano.git/tree/src/files.c#...](https://git.savannah.gnu.org/cgit/nano.git/tree/src/files.c#n671).

------
thulecitizen
I misread this as: "I recovered a lost email from my client’s memory". It made
me think of the Black Mirror episode 'Crocodile', and I was quite amazed. Then
I saw 'email' client... haha

~~~
d2wa
Author here. That was actually the working title up until two minutes before
publishing.

~~~
jaclaz
Yep, and the title, once re-parsed is fine, but I also had - initially - the
wrong impression, it flashed before me how you hypnotized your client
(customer)and managed to retrieve from his/her memory the contents of an
e-mail message he/she ddn't rememeber anything about.

~~~
d2wa
That’s pretty much exactly how I thought the previous title might have been
misunderstood!

