
Unfork() - salgernon
https://github.com/whitequark/unfork/blob/master/README.md
======
aloknnikhil
> Permission to read from or write to another process is governed by a ptrace
> access mode PTRACE_MODE_ATTACH_REALCREDS check

I presume that is necessary for this in addition to belonging to the same UID?

> As far as I know process_vm_readv isn't even detectable if the agent process
> is more privileged than the examinee process—so you're free to manipulate
> your private copy of the application in the comfort of your own address
> space.

Interesting. This would be really useful in debugging. Many issues don't
reproduce except for in specific configurations. Having access to the memory
dump of the live process "streamed" to the debugger would be great!

~~~
ktpsns
> Having access to the memory dump of the live process "streamed" to the
> debugger would be great!

This is also possible with standard debuggers, such as GDB: It can attach to a
running process and not only examine the memory, but also debug (stop, pause,
skip, ...) the stack trace and control flow. Usage is as simple as _gdb -p
$(pidof my_running_program)_

~~~
andrewaylett
You might also find [https://rr-project.org/](https://rr-project.org/)
interesting -- it lets you step backwards too.

~~~
Enginerrrd
Interesting. Without letting storage explode, I can't think of an easy way to
do this since computation isn't really reversible.

~~~
saagarjha
You can keep track of the before and after states whenever you do something
nonreversible, like a syscall.

~~~
Doxin
And even then you can probably just store the diff instead of a full image.
And even then if you run out of memory you can just start evicting the oldest
snapshots.

------
mmoez
I know Windows doesn't get too much love here. But we have to admit that Win32
has already this kind of feature since ages: Process access routines such as
OpenProcess() [1] coupled with ReadProcessMemory() [2] will do the job in a
clean way.

Taking a snapshot of other processes is also a basic use case of this family
of functions [3].

[1] [https://docs.microsoft.com/en-
us/windows/win32/api/processth...](https://docs.microsoft.com/en-
us/windows/win32/api/processthreadsapi/nf-processthreadsapi-openprocess)

[2] [https://docs.microsoft.com/en-
us/windows/win32/api/memoryapi...](https://docs.microsoft.com/en-
us/windows/win32/api/memoryapi/nf-memoryapi-readprocessmemory)

[3] [https://docs.microsoft.com/en-
us/windows/win32/toolhelp/taki...](https://docs.microsoft.com/en-
us/windows/win32/toolhelp/taking-a-snapshot-and-viewing-processes)

~~~
klodolph
How clean is it? Can you simply exec the result?

Glibc used to have unexec(), which is fairly old, but it was removed because
nobody used it (except Emacs, and there were better solutions to the problem
it was solving).

~~~
mmoez
> How clean is it?

It's as clean as any official Win32 API which uses their privilege system to
restrict/allow accesses to each and any bit of information on the process
state and/or memory.

> Can you simply exec the result?

This is possible using CreateThread() [1] which creates a remote thread inside
another process execution context.

[1] [https://docs.microsoft.com/en-
us/windows/win32/api/processth...](https://docs.microsoft.com/en-
us/windows/win32/api/processthreadsapi/nf-processthreadsapi-createthread)

> Glibc used to have unexec()

My understanding is that unexec() was more about making a snapshot of the
whole process state to an executable on disk.

~~~
hackworks
That is my understanding too. Solaris had a flag for dldump
([https://docs.oracle.com/cd/E19455-01/806-0627/6j9vhfmop/inde...](https://docs.oracle.com/cd/E19455-01/806-0627/6j9vhfmop/index.html)).
Emacs moved to a portable dumper (maybe inspired from XEmacs)

------
jagrsw
And for something completely different - but in the same vein - a stand-alone
'cd' binary -
[https://github.com/robertswiecki/extcd](https://github.com/robertswiecki/extcd)
\- enjoy!

~~~
Hello71
isn't this more or less the same as

    
    
        gdb -batch -n -ex 'call chdir("whatever")' -p $$

~~~
vidarh
That does seem to be conceptually pretty much what it's doing. Except your
version works on more architectures.

It's a simple example of ptrace() though.

------
euske
Totally, off topic, but it's funny to think about what kind of the feature
would be if we put "un-" on each syscall:

unseek

unselect

unpipe

unsync

etc.

~~~
geofft
Emacs somewhat famously uses "unexec" in its build process, you build a
skeletal Emacs in C (mostly the Emacs Lisp implementation), run it, load and
compile and process a bunch of Lisp that implements the editor itself, and
dump the resulting process memory back out to disk. The result of this
eldritch process is the final emacs binary. When you exec emacs, you get an
environment that consists of the editor code ready to go.

~~~
lilyball
I'm given to understand that the macOS implementation of malloc had to have
special-cased code in it just to support emacs due to this approach.

~~~
tambourine_man
Why?

~~~
jandrese
Because linkers are too easy?

It really looks like some overly clever college student's weird trick that
somehow managed to survive for decades in an established product.

~~~
jws
It comes from a time of machines executing instructions thousands of times
slower than they do now. Literally – thousands. Memory access was about as
fast as an instruction execution, so the amount of compute you can justify per
unit of data was hundreds of times less than it is now. They did however have
virtual memory systems with on demand page fetching.

Also, that machine was being time shared with a dozen or more users.

Launching emacs or TeX on this machine might take tens of seconds without
access to unexec(), but only 3 seconds for the freeze dried version.

unexec() was easier at the time. There were no shared libraries, no address
space layout randomization. One memory region grew up from the bottom, one
down from the top. There was no mmap() jamming mysterious stuff in the middle.
Just copy the bottom, copy the top, do _magic_ to adjust the stack for your
unexec() call, and write the thing out as an executable.

(Yeah, I excised unexec() from BibTeX back in the ‘80s to port it to a 68k Mac
for a coworker, then later implemented unexec() for a Motorola 88k based
multilevel secure SysV system in the early ‘90s because launching emacs was
driving me insane. I prefer our shiny new future of stupidly fast computers.)

------
apeace
> Nevertheless, I think that with some effort two allocators or even dynamic
> linkers could survive together.

Famous last words.

------
koolba
> How limited is this approach?

> A: It's true that meshing address spaces is much harder than copying them.
> ... [truncated] ... 64-bit systems with ASLR are far more forgiving.
> Nevertheless, I think that with some effort two allocators or even dynamic
> linkers could survive together.

That is a really cool side effect of ASLR!

[1]: [https://en.wikipedia.org/wiki/ASLR](https://en.wikipedia.org/wiki/ASLR)

------
avodonosov
Does it pause the process whose memory it is copying?

Freezing the process can affect its correct operation. (Sometimes when I need
a memory dump of a production java app, I can't take it because can not afford
freezing a production app)

Without the freeze, the memory copy we get can be inconsistent.

~~~
aargh_aargh
I have no idea but the FAQ says it's CoW.

~~~
Liskni_si
If I understand it correctly, it's more of a Copy-on-Read/Write, and as
opposed to fork, it's only one-sided: read/write is only detected and results
in a copy on the unfork side; if the original process changes memory, it
doesn't result in a copy as nothing is monitoring this (the userfaultfd only
monitors the unfork side).

~~~
avodonosov
The ideal approach would be if it turned the original process memory into
"copy on write", and created a paused exact copy of that process. This would
give a consistent, immutable, snapshot of the target process memory, without
freezing for the duration of actual memory copying.

One could then take a core dump, java heap dump, or similar, of the paused
copy process.

I'm curious, why does the tool try to copy the original process memory into
the memory of the tool itself, risking a collision? Is it impossible to create
a third process - an exact copy of the original process?

~~~
Liskni_si
The FAQ says:

> all while leaving no ptrace and sending no signal

If this is a design goal, I'm afraid it is indeed impossible to take a
snapshot of the original process. As far as I know (I researched the status
quo 2 years ago when I needed copy-on-write for VM cloning/forking), the only
way to make a snapshot of a process' address space is to invoke the clone
(fork) system call. If you need to take a snapshot of another process, then
you need ptrace.

But you're absolutely right that the unfork functionality itself can be
implemented more robustly by doing this ptrace/fork trick.

------
tom_mellior
I've read the quirky FAQ and would now be interested in what this really does.
The Readme mentions some demo code that's not in the repository. It also
instructs us to run this on cat and enjoy, but... _what_ would we observe and
enjoy?

------
saagarjha
I wonder if 64 (well, 48) bits of address space is enough to glom together
_every_ process on a normal Linux boot without collisions…

~~~
smallnamespace
If you assume each process needs, say, 16MB of contiguous space, then you get
48-24 bits left, which by the birthday paradox implies you can have up to
2^(24/2 = 12) ~= 4k processes before you start colliding about half the time.

~~~
saagarjha
> If you assume each process needs, say, 16MB of contiguous space

Unfortunately I’m not sure that’s a good assumption, due to the stack and heap
needing to exist even for statically-linked binaries.

~~~
smallnamespace
Yes, but simply replace with 'average number of allocations * number of
processes' and * 'average size of allocation'

------
Iv
Isn't that a bit similar to what debuggers typically do when you ask them to
attach to a given process?

~~~
saagarjha
Debuggers touch other processes from afar. This merges the debuggee into
yourself.

~~~
skrebbel
No, it merges a copy-on-write clone of the debuggee into yourself. That's
quite different and, indeed, you can do similar things with it that a debugger
could.

If I understand this right,the process being unforked into you won't notice a
thing and will happily chime on.

~~~
saagarjha
> No, it merges a copy-on-write clone of the debugger into yourself.

But you _are_ the debugger…

> If I understand this right,the process being unforked into you won't notice
> a thing and will happily chime on.

Right, whereas when running an actual debugger you need to deal with signals
and making sure you don't touch memory.

~~~
skrebbel
> But you _are_ the debugger…

Ah thanks. My autocomplete didn't like the word "debuggee". Edited!

------
friend-monoid
The inverse of fork is called join, right?

~~~
hirundo
In the git context it's a merge. But Unfork() seems more like a rebase.

~~~
louiz
In the git context, fork doesn’t really mean anything. That’s just a github
thing.

------
acoye
You could build an entire new array of malware with this :D

------
whateveracct
userfaultfd is an extremely intriguing hammer :)

~~~
fsfod
The write protected mode[1], if it ever gets merged could have some
interesting uses for GCs.

[1]
[https://lore.kernel.org/patchwork/cover/1033856/](https://lore.kernel.org/patchwork/cover/1033856/)

~~~
whateveracct
I could imagine it being interesting for VMs and DBs too. Imagine a VM whose
memory "looks" like virtual memory but under the hood is transparently
persisted between invocations.

~~~
fsfod
That's kind of one of the main uses of the API so far by QEMU for live
migration of VMs by streaming memory on demand over a network
[https://wiki.qemu.org/Features/PostCopyLiveMigration](https://wiki.qemu.org/Features/PostCopyLiveMigration)

------
khaki54
Seems like this would be useful for re-attaching to a shell or process you
have either disowned or otherwise lost control of. Would be neat to see some
common use cases on the FAQ page.

------
equalunique
Is it just me, or is this something that would make Linux even more vulnerable
to cyber attacks? What protections are there? Would OpenBSD's pledge prevent
something like this?

------
CriticalCathed
This is a hack.

I like it.

Refreshing.

~~~
keeganpoppen
if only "hackernews" had more hacks like this xd

------
pacman128
Since it brings two processes together, maybe _spoon_ would be a better name.

~~~
gjm11
Or, along the same lines, another four-letter word sharing two of its letters
with _fork_. But that might be too distracting.

~~~
deckar01
krof

~~~
keanebean86
I was thinking Fnnl (pronounced funnel)

But that's probably the name of a startup and would confuse people.

------
sitkack
Whitequark is a Wizard. They should team up with Sammy.

~~~
rurban
Whitequark is better than Sammy in SW. He's also the maintainer of SolveSpace.
[https://m-labs.hk/software/solvespace/](https://m-labs.hk/software/solvespace/)

~~~
saagarjha
According to her Twitter, Whitequark prefers feminine pronouns:
[https://twitter.com/whitequark](https://twitter.com/whitequark)

~~~
felipelemos
By the parent comment I thought Withequark was a group/team of people.

English is also not my native language.

~~~
progval
"They" can mean either a group of people, or a single person of unknown gender
(or neutral gender).

------
psaux
Sounds like alternatives for Git :) But seriously, need to read more on this.

~~~
woodrowbarlow
alternative for git? i don't follow. can you elaborate?

~~~
isatty
I think they just read the title and assumed its a repository fork? No clue
either.

~~~
psaux
Yes, apologies just having fun. At first glance, I thought it looked like
Linux Alternatives.

