
Takeover.sh – Wipe and reinstall a running Linux system via SSH without reboot - tambourine_man
https://github.com/marcan/takeover.sh
======
gizmo
Pretty cool, although I'm pretty sure I would never use something like this.

What has saved my skin on a number of occasions is the ability to boot remote
servers into rescue mode and chroot into the broken system. That way you can
use package managers, all your diagnostic tools, and everything else the boot
image doesn't provide.

Basically you just mount the different partitions and then chroot just swaps
/proc /sys /dev of the rescue image with the real ones, and BAM you're back in
business.

For details see threads like: [http://superuser.com/questions/111152/whats-
the-proper-way-t...](http://superuser.com/questions/111152/whats-the-proper-
way-to-prepare-chroot-to-recover-a-broken-linux-installation)

I know that for many of you this isn't rocket surgery, but for those who don't
know you have to google for "chroot" when you boot into a rescue image and
discover you can't do anything, you might just remember this post.

~~~
shocks
Haha yes, this is the default install procedure for Gentoo! :D

~~~
exDM69
Something like this has been my default way of installing Linux for years -
and I've installed Debian, Ubuntu (with debootstrap), Gentoo and Arch this
way. Typically I just create another partition to my LVM volume group, chroot
and install over there and then reboot without removing the old OS install and
I run the install procedure from the old OS (instead of booting from a Live CD
/ usb stick).

I do this because I know it works and there's no guesswork involved in what
the OS installer does. They aren't really intended for installing beside
another system and the default partitioning options aren't always that great
(RAID, LVM, crypto, etc).

Basically you only need 3 things to run Linux: kernel, initramfs and rootfs.

~~~
yellowapple
You really only need two (kernel and initrd/initramfs), at least to get into a
basic running state. This requires building your own initrd, of course, but
it's pretty common in non-graphical Linux installers (Slackware's install
media still does this, IIRC).

I _think_ it's even possible to embed the initrd in the kernel binary itself,
but I've never really investigated that.

~~~
exDM69
Yes, it's a fun exercise to build a tiny Linux install that's fully on the
initrd. Not that you'd want to have that kind of system in daily use outside
of special applications.

> I think it's even possible to embed the initrd in the kernel binary itself,
> but I've never really investigated that.

Yes, the kernel config has an option to embed the initrd in the kernel image.
I'm not sure if there are any advantages to this.

~~~
arca_vorago
I use the initramfs method to put an small ssh server in that I can use to
unlock full disk encrypted headless boxen, so I could see initrd in the kernel
being used in a similar way.

------
predakanga
For anyone interested in adding this to their toolkit, I would suggest reading
this StackOverflow answer:
[http://unix.stackexchange.com/a/227318/189858](http://unix.stackexchange.com/a/227318/189858)

In short, the answer details how to switch your running system to use an in-
memory only root filesystem, without restarting. This allows installing a new
OS, resizing the OS disks, etc.

It's a risky operation, but the linked answer covers many pitfalls that you
might run into - I recently used it to shrink the root partition on a remote
server, very much appreciated the detail.

~~~
aaronmdjones
I've been using this procedure to remotely replace operating systems for
years. The most common scenario is a VPS provider that doesn't give you the
choice of OS you want.

Infact, I will be using it later today, to replace a Debian system with
Gentoo, no less. The README in OPs link is spot-on here (the last few
paragraphs).

~~~
djsumdog
This is exactly what I thought when I saw this post. I really want to use Void
Linux on a provider, but no one supports it (except for those who allow custom
images like Linode). It'd be great to be able to provision a standard Ubuntu
or CentOS box and then replace it with the OS I actually want to run.

------
rdslw
Another nice trick of this family (with reboots, or without with using
systemd-nspawn) lies with clever btrfs usage. Long story short:

* use btrfs, and create your main root filesystem as a btrfs partition _subvolume_ and another btrfs subvolume for snapshots (also a sub of master btrfs partition)

* to start any experiment (e.g. installing whole gnome and 500 different packages you MIGHT WANT TO REVERT in the future) create before the risky operation snapshot (btrfs subvolume snapshot / /.snapshots/yournameofsnap) of current filesystem

* experiment in any way :)

* switch between old root (snapshot you created) or the new one with (btrfs subvoulme set-default)

* delete any of them (btrfs subvolume delete)

btrfs copy-on-write allows all of these commands to happen instantly without
(almost) any actual copying. Also booting from both volumes is possible
without any additional steps as long as master btrfs partition is the one to
be booted from UEFI.

[https://wiki.archlinux.org/index.php/Btrfs](https://wiki.archlinux.org/index.php/Btrfs)

~~~
floatboth
Congratulations, you've reinvented ZFS Boot Environments :)

~~~
LeoPanthera
Only to a very high level. btrfs snapshots are writable, but share blocks, so
you can boot directly from a snapshot. It's a better system.

------
notaplumber
This sounds similar to the more cleverly named FreeBSD Depenguinator project
which could be written over top of a remote Linux server replacing it with
FreeBSD, without console.

If you have remote console access, a similar thing can be done for OpenBSD by
dd(1)'ing a miniroot ramdisk install image.

------
camtarn
Seeing if I understand what this is doing: this keeps running the same Linux
kernel and kernel modules, but swaps out absolutely everything else up to and
including the init system - is that right?

~~~
efficax
More precisely, what it does is start a new init system on a pseudofilesystem,
starts an sshd chrooted to that new system, and lets you then login there,
where you can then, if you want, umount the original root fs, wipe it, install
a new os, then reboot the system.

~~~
camtarn
Thanks!

------
simon1573
I guess this could be really useful for installing distributions that are not
available at some VPS providers.

~~~
tyingq
Another trick to do that is to use the VPS's rescue mode, download qemu, and
start it with the remote desktop / VNC console. Then, do the install from
there. Poor man's IPMI.

------
ce4
Reminds me of Debian Takeover from more than 10 years ago :-)

[https://wiki.debian.org/DebianTakeover](https://wiki.debian.org/DebianTakeover)

------
zimbatm
[https://github.com/elitak/nixos-infect](https://github.com/elitak/nixos-
infect) is similar but doesn't require to pivot root.

------
Aissen
FYI, there's vps2arch that does the same thing with a different approach:
[https://github.com/drizzt/vps2arch](https://github.com/drizzt/vps2arch)

 _Edit_ : it doesn't really do the same thing. vps2arch could be implemented
on top of takeover.sh for better reliability.

------
nashashmi
Somebody correct if I am wrong, but this script somehow allows the session to
live in the RAM. Once the OS is running directly from the RAM, the hard drive
can be wiped and a new OS can be installed. The system is then booted to run
off of the hard drive.

~~~
VoidWhisperer
I feel like this is, on one hand, an amazing thing to be able to do because it
removes the necessity to have IPMI to remotely re-install a system.

On the other hand, this seems like it would be an incredibly easy thing to
screw up and potentially leave yourself with a corrupted or unbootable system.

~~~
ams6110
Hence the advice to only do this on machines where you have physical access,
or at least can trigger a PXE boot via IPMI.

------
dredmorbius
This is conceptually similar to the chroot installation method, which has been
a documented, if not entirely standard, method on Debian for quite some time.

[https://www.debian.org/releases/stable/amd64/apds03.html.en](https://www.debian.org/releases/stable/amd64/apds03.html.en)

[https://wiki.debian.org/chroot](https://wiki.debian.org/chroot)

~~~
yellowapple
It's also a common installation method for the more "advanced" distros. The
"SAG Trifecta" (Slackware/Arch/Gentoo) uses a lot of chrooting in its
installation procedures.

What's interesting about takeover.sh, though, is that it goes a step further
and causes the chrooted system to actually replace an existing OS without a
reboot and (theoretically) without involving additional boot media.

------
geoffmcc
I wonder if this would help me switch from Ubuntu Desktop to Ubuntu Server on
my laptop that has a broken screen.

~~~
pipo098
you can always connect an external screen and go from there

~~~
geoffmcc
I can connect HDMI and can get into UBUNTU, but I cannot get into BIOS to tell
it to boot from USB. If I could get that figured out- I think HDMI will
activate on the boot of the ISO, but Im not sure. I even tried VGA to a
monitor, but that dont show me bios either. Dont wake up till Ubuntu starts to
boot.

~~~
umanwizard
Can you physically disconnect the hard drive? Maybe you'll get lucky and USB
will be next in the boot order. Contrary to popular belief, you can USUALLY
plug the hard drive back in while the system is running from USB without any
problems.

~~~
geoffmcc
oh. You may be onto something. I am pretty sure USB is boot option 2, but if
not it would be my disc drive- but I can make that work too. It's a Lenova
(maybe acer) laptop with some wierd tool it seems to open it up. I can try to
figure it out though. I just want to use it as an inhome server so it dont
need to look pretty.

~~~
umanwizard
Keep in mind, you might get a kernel panic when you try to hot-plug the drive
back in. It's not guaranteed to work, but it's worth a try.

If that doesn't work (kernel panic or whatever) you can disconnect the drive
and connect it via a SATA-to-USB adapter in which case hot plugging should
definitely work.

------
NGTmeaty
Holy shit, that's really cool.

------
technologyvault
Wish I had known about this hack before today, even if it is just experimental
at this point.

------
dpweb
removed. No delete on HN comments? interesting

~~~
bjacobel
I think you meant to comment on the GitLab postmortem thread.

~~~
dpweb
oops, yeah..

