Hacker News new | past | comments | ask | show | jobs | submit login
Is chroot a security feature? (redhat.com)
88 points by katzeilla 39 days ago | hide | past | web | favorite | 83 comments

So the basic argument is that if you can become root you can escape the jail. So if you have access to a vulnerable program you can become root and escape.

This might be true, but it's still a good practice for defense in depth and does offer a security benefit. It helps protect you when there are non-root flaws in software. For example a web server that lets clients get to directories with '..' notation. Even with that compromised software, they wouldn't be able to get to /etc or /home if they are in a proper chroot.

There are several more ways you can escape chroot(2) through VFS attacks (nd->root protections are very minimal).

However, the more important point should be that there is a similar syscall which doesn't have the same flaws as chroot(2) because the protection is based on mount namespaces -- pivot_root(2). The problem with chroot(2) is that the mountpoints above "/" are still accessible by the VFS internally. pivot_root(2) gets rid of the mounts and so VFS won't reach them anymore.

I see there's a similar wrapper binary - but apparently you need to both pivot and chroot?


The man page is out-dated (and Michael Kerrisk has posted on LKML that he plans to rewrite it). You don't need to do chroot("/") -- that operation makes no sense because there is no semantic difference between chroot("/") and doing nothing (even the kernel can't distinguish the two cases).

What you need to do is the following:

  mkdir newroot/oldroot
  mount --rbind newroot newroot # if newroot isn't a mountpoint
  pivot_root newroot oldroot
  cd /
  # and now get rid of the old root's mounts
  mount --make-rslave oldroot
  umount -lR oldroot
At least, approximately. There is a more clever way to do it by doing "pivot_root . ."[1] but posting that here just makes it more confusing. As a complete aside, this is what container runtimes do when switching to the container rootfs -- and is the main reason why --no-pivot-root is incredibly insecure (because it uses chroot).

[1]: https://github.com/opencontainers/runc/blob/v1.0.0-rc8/libco...

So what are these mystical ‘several ways’, preferably some that do not require root and not require proc mounted in the chroot?

I can think of ptrace() attach to other processes of the same user outside of the chroot environment and inject arbitrary code.

In modern systems, ptrace requires root even if attaching to the same user. So this hole has been closed.

> pivot_root(2) gets rid of the mounts and so VFS won't reach them anymore.

So what happens to open FDs, like the FD for the binary you are executing?

I should've added "with the exception of magic-links", which effectively bypass VFS's resolution. Yes, if you get a file descriptor from a foreign namespace then you can make use of it for various escapes. Unfortunately there really aren't many protections against this type of thing.

Though, /proc/self/exe alone won't give you resolution access to the host. It takes a bit more work and more bugs.

Alright, how about a more useful example: I open a file in the users home directory and then drop my namespace to /var/void to prevent myself from opening any additional files?

This is a very common pattern, and killing off open FDs seems like a very wrong thing to do.

The correct thing to do is bind-mount the user's home directory, pivot_root(2) and then open the files you need. That way, file descriptors opened in the user's home directory aren't from a namespace where the host's root is reachable.

Introducing a foreign file descriptor (unless you really know what you're doing) is a bug, assuming your goal is to be secure against escapes. Even AppArmor would struggle to protect you from this.

Leaving the home directory accessible is the exact problem that needs to be solved. An attacker doesn't need to break out of a sandbox to access the rest of my system: they can download it from debian.org. Being able to open new files after initialization is a bug.

The solution is to initialize the application before it can touch untrusted input, and then drop any ability to do anything other than process the resources set up in the init phase. The program will ideally be running in a completely empty namespace when handling untrusted data.

You can think of FDs as capabilities. Dropping the namespace prevents the program from requesting new capabilities. It's perverse to tie owning capabilities to being able to request new ones.

If you are passing file descriptors to a sandboxed process from the host filesystem, it is very hard to defend against all attacks (especially if you pass directory fds). If you just pass a single file that can work fine, but it does make mistakes far more serious.

I apologise for misreading your original problem. But the solution is identical -- bind-mount the individual files or subdirectories that you need the process to open.

It really isn't that hard, and is exactly how privsep works on all of the OpenBSD daemons, augmented by pledge.

Your proposed alternative allows untrusted code to gain new capabilities. That's not a design I want to trust.

I am trying to outline how you would do this on Linux. I appreciate OpenBSD has different facilities, but this is like telling a Japanese chef that their technique for dry-aging fish isn't that great because you know about an amazing way to roll out croissant dough.

Yeah, it's nice that you have pledge() and unveil() but that doesn't help Linux programs.

Linux has the facilities needed for this -- they've existed in POSIX for ages. Doing a better job of it would be nice, but the suggested approach seems like a pretty big regression.

Unveil() is irrelevant to this discussion, because your program ideally has nothing in its namespace to unveil.

Pledge() is a useful additional layer of protection, dropping access to things that can't be controlled through the file system -- but Linux has seccomp-bpf that fills the same role. It's also unnecessary for privilege separation: It wasn't implemented until about 2 decades after the methods I'm talking about were in widespread use. And not just in OpenBSD -- from what I remember, Solaris had a section on it in their programming manuals.

The principle is simple: Start the program, acquire capabilities (ie, fds) needed to do the job. Then, drop the ability to request capabilities, drop root, and start processing untrusted user input.

Many comments in this thread are bashing the article for saying that chroot is useless or shouldn't be used. However the article itself takes a more nuanced approach.

From the conclusion:

>It is not hard to consider the chroot() system call a security feature. In theory, it sounds great, but if you really take the time to understand what is going on, it is not really a security feature, it is closer to what we would call a hardening feature. It might slow down an attacker, but in most situations it is not going to stop them. We are dealing with a situation where calling this a security feature is likely more damaging than not because it creates a false sense of security. It is human nature to let our guard down if we believe we are safe. Using chroot is no safer than not using a chroot. You would be far better off investing your resources into a custom SELinux policy and ensuring your system is properly hardened. Good security has no shortcuts.

I think the point is that you shouldn't consider chroot the only or best option for securing your system. It's just one tool in the toolbox and needs to be understood properly and not trusted blindly as the solution for all security vulnerabilities.

I have a hard time reconciling calling chroot a hardening feature, but then turn around and say that it's no safer than not using it.

I don’t think that’s unreasonable on its own, whether it is here or not.

For example, in a non computer scenario, wearing bicycle helmets is definitely better for you if you get into an accident as the odds of injury reduces. However, the false sense of security may also make the wearer more reckless while riding their bike and potentially lead to more accidents.

I think that’s the argument being made here. The false sense of security chroot may provide will lead to a person not hardening their systems as much if they weren’t using chroot at all, even if chroot itself does increases security.

> the article itself takes a more nuanced approach

Let's see what the article says:

> Using chroot is no safer than not using a chroot.

Not very nuanced in my opinion.

Few people argue we should replace chroot calls with SELinux these days. The article is interesting from a historical perspective but clearly needs a "(2013)" context.

The article is inconsistent about chroot, but regarding SELinux, it has a point. SELinux looks like an old and solid mechanism to isolate processes from each other. Nowadays we have lxc to do that as well, but it does not make SELinux useless.

What's the argument that 2019 people would make? Replace chroot with ______?

> It's just one tool in the toolbox and needs to be understood properly and not trusted blindly as the solution for all security vulnerabilities.

That's true of every security feature. That is security 101. Saying that is prActically saying nothing (worthwhile). And it would be better said alone. Not in context of of tool otherwise people will assume ur bashing that tool as worthless.

How can this even be a question 20 years after FreeBSD Jails ?


Nothing would prevent root from just modifying the process memory directly to change the chroot() location back to '/'.

Huh? The chroot root directory is stored in the kernel, in the task structure, not in process memory. For that matter, every process can modify its own memory, not just root.

It is in fact a bug, or at least a design flaw, that root can break out of chroot. For instance, the Unix filesystem, historically, at the VFS layer (or whatever Linux calls it), has a simple tree structure, Unix allows nested chroots, and chroot(2) doesn't chdir(2) you into the new root. If you chroot to a subdirectory of your root and don't chdir into it, Unix is too dumb to know you're "above" your current root.

Yeah, I assume the author meant to write "kernel memory" there. I think what they were getting at is that root could theoretically mknod a /dev/mem device inside the chroot jail and then modify kernel data structures through it. Or root could issue an init_module(2) syscall to install a kernel module that breaks out of the jail.

Right, but on modern Linux, you're a seccomp call away from blocking those vectors. The inadequacy of chroot was a more coherent story to tell 10 years ago, but now, it seems silly to suggest that chroot isn't a security feature; it is, it's just part of an ensemble of related features that need to be deployed in concert.

This isn't true. For example containers use pivot_root instead of chroot or MS_MOVE because there are specific security issues with keeping a reference to the parents mount table.

If you care about security there is no reason to use chroot on Linux, there are better approaches.

Is there a vulnerability pivot_root breaks that applies to unprivileged users? I wrote quickly and imprecisely and agree that in general chroot isn't especially helpful against the superuser (for a bunch of reasons).

Well, this was published in 2013. That was right around the time seccomp-bpf was introduced, but I don't think best practices around it were widely known at the time.

I can't believe this guy works for redhat.

> you are no safer inside a chroot than relying on system permissions to keep a user in check

But how may systems lock down every executable in /bin and /usr/bin?

Chroot is about denying an attacker a hackers toolkit when they get access.

It's not just about hacking the current system, but the network.

No gcc? No netcat or ssh or netstat? Or even bash? The first thing hackers look for is what tools are available on the system, and denying them any is a form of security. This had been known for decades.

chroot() started out as just a chdir() operating on root directory instead of current directory.


> Using chroot is no safer than not using a chroot.

This is terrible advice. Software should drop privileges early, and chroot is as important as it has ever been. Other useful things include seccomp-bpf and pledge.

OpenSSH and Postfix is absolutely safer because they drop privileges after reading configuration and binding ports.

The problem is that there are many well-known ways to get around chroot(2) (which don't require privileges). This includes /proc-related escapes as well as more general VFS attacks.

Really, tools should have switched to using pivot_root(2) a decade ago instead of continuing to use chroot(2). Unfortunately this article completely ignores the existence of pivot_root(2) -- which makes it read as though they are trying to convince people to stop using root-switching a security feature (which is definitely bad advice).

> This includes /proc-related escapes as well as more general VFS attacks.

Right, there are numerous ways to mess up a chroot, just like any other namespace isolation such as process and network namespaces.

Basically as soon as you bind one namespace inside another, the containing namespace will have access to the contained namespace. That much should (hopefully) be obvious. If you mount a procfs inside your application it is no longer contained from the syscalls that procfs necessitates.

But that's not typical for an application that drops privileges. Even decaces old software such as BIND, which has had numerous vulnerabilities over the years, chroots to the data directory and drops to an unprivileged user by default.

There is an important distinction here whether the application itself should drop any unnecessary privileges, as exemplified by chroot, seccomp and pledge, or whether the application should be started with as little privileges as possible, as exemplified by SELinux and Docker. In the former case, the responsibility falls on the application developer, and in the latter case on the system administrator or end user.

The article explicitly argues for SELinux instead of the chroot. That is, arguing the latter approach is superior to the former. That is experimentally verified bad advice by now.

Software such as Chrome rely on chroot-style compartmentalisation and is much better off because of it.

> tools should have switched to using pivot_root(2) a decade ago

Yet people keep using the simpler tool. The threat model is the same and pivot_root is easier to misuse. In fact, it is probably a good idea to do both.

I've never seen this advice, would you mind elaborating or citing a more elaborate source?

Since pivot_root requires the old root to be a child of the new root, traversing back to the full system seems trivial. Actually, I don't see why this syscall isn't worse than chroot due to this.

> Actually, I don't see why this syscall isn't worse than chroot due to this.

You can (and everyone does) unmount the oldroot. Then in your mount namespace the old root is not reachable through the VFS alone. You need to make use of procfs's magic-links or other such tricks.

grsec doesn't use pivot_root, you can read into that however much you'd like: https://en.wikipedia.org/wiki/Grsecurity#Chroot_restrictions

Isn't it better to fix those many well-known ways to escape chroot?

Some of them can't be reasonably fixed (doing a path_is_under() check on all ".." traversals would fix most security bugs -- but it would make all path lookups be O(n^2) which is unacceptable) and there is also the risk of yet-unknown attacks.

chroot is a layer in the security onion.

Like every other layer it adds security but isn't perfect. Security will never be 100%. If you want more security, then likely you want another layer in the onion.

Yes, but pivot_root(2) is a far more robust and secure way of doing the same thing. There really is no reason to use chroot(2) instead of pivot_root(2) (unless you are running code on initramfs -- and even then you should do the tmpfs trick that everyone else uses).

> There really is no reason to use chroot(2)

You mentioned intiramfs yourself, but there are other uses of chroot where security doesn't play a role. Deterministic build environments are an example.

This article is entirely FUD.

chroots are useful when running software which we don't necessarily know is secure. It reduces the impact of flaws in software we don't control.

The point is taken that proper permissions would effectively be no different from a chroot, but this is disingenuous, particularly if considered with the statement, "but note that setting up a chroot can be far more complex than configuring a system".

This is pure hyperbole and deserves to be called ridiculous. Compare configuring a chroot, which is easy, straightforward, common and well understood with configuring a user to exclude access to everything in a system except what the chroot would allow, and try to reasonably suggest that the chroot is more complex.

You can do lots with login.conf, but hang on - systemd does things differently. So instead of decades of well understood chroot history, now you have to figure out how login.conf works with systemd. Then you have to figure out what version of systemd it is, and whether behavior has changed, and whether documentation even matches your system's systemd...

In other words, this whole article has no purpose aside from trying to disparage chroot without any actual, meaningful information.

So how does Red Hat benefit? Of course Red Hat doesn't want people running lots of processes in a single installation. They want people to run a ton of system images in huge VM environments and have people become accustomed to needing lots of resources to do the simplest of things. This makes paying Red Hat for "support" easier to swallow. One instance with lots of tasks? Insecure, according to them. A separate VM for each program? Much better, if you believe them :P

Nonsense. Red Hat supports containerization in more ways than I can count. Also, most businesses running RHEL in a VM environment (like ESX) are using per host licensing where you can run an unlimited number of RHEL VMs on a single physical host.

If you forget to setuid after chroot it's much less secure :P I have a free online service where anyone can run any code on a fairly powerful dedicated server. And it uses chroot, setuid, iptables and Apparmor. It has not yet got pwned, knock on wood.

I don't understand why the article recommends pivot_root. Its man page says that it may or may not change the current root and the current working directory of any processes or threads which use the old root directory, which means it can change the root of other processes! Also, it says that its typical use is for switching from the boot RAM disk to the actual root filesystem. This is an entirely different use case than chroot, and doesn't seem to have any security benefits either.

That's why FreeBSD created jails, the idea was to actually make a secure chroot.

The original purpose of chroot was nothing more than control where an installer installs its files.

IMO, chroot is good for changing root :) but not for security. Example: zfs clone your current root, chroot there, experiment / test /etc.

> Putting a regular user in a chroot() will prevent them from having access to the rest of the system. This means using a chroot is not less secure, but it is not more secure either. If you have proper permissions configured on your system, you are no safer inside a chroot than relying on system permissions to keep a user in check

Lol. So, all security mitigations ever are pointless.

"it is not really a security feature, it is closer to what we would call a hardening feature. It might slow down an attacker, but in most situations it is not going to stop them."

Does his argument boil down to this? If so this seems like a silly argument over semantics. Is there anything in existence that can "stop" an attack?

Wait, are there people who really think chroot offers security?

I mean, at least read the man page before making assumptions?


> This call changes an ingredient in the pathname resolution process and does nothing else.

> It is not intended to be used for any kind of security purpose, neither to fully sandbox a process nor to restrict filesystem system calls.

> In particular, the superuser can escape from a "chroot jail" by doing: mkdir foo; chroot foo; cd ..

> However, if a folder is moved out of the chroot directory, an attacker can exploit that to get out of the chroot directory as well. The easiest way to do that is to chdir(2) to the to-be-moved directory, wait for it to be moved out, then open a path like ../../../etc/passwd.

EDIT: Fixed link to a more up-to-date man page. They explicitly call this out as not being a security feature in the updated man page.

On BSD, jail() is used as an extension to chroot() specifically to improve the security for externally accessible services. Breaking into ftpd (jail() has been around that long) would only give an attacker access to the configured subdirectory, and not /etc/passwd and shadow.

The reason chroot() isn't a security feature is because Linux only half copied/re-implemented that feature. Properly implemented, it's a decent idea.

I was left to wonder if chroot along with a well-crafted SELinux policy might get one closer to security nirvana.

i often wondered how exactly one can escape from a chroot but never really tried or read that man page until now. I tried the above and it did not work as described. I tried both changing to the parent directory directly after chroot and chrooting inside the chroot and doing it then.

Edit: Now that i really read that page i realized 2 things, first this page is about the syscall and not the command and second it says the syscall itself does not change the current directory to the provided path. I guess the chroot command does just that and this is why it did not work as described. The example given literally still does not work as advertised.

This is the more correct escape (EDIT: and works today):

  fd = open(".", O_PATH);
  fchdir(fd); // escaped to "." and chroot no longer blocks ".." past "/"
  chdir("../../../../"); // go to real "/"
  chroot("."); // broken out
The "chdir .." trick used to work a long time ago. The man page probably hasn't been updated in ~20 years.

Not really an issue. If there's an fd pointing outside of the chroot, of course it's possible to do stuff outside of it. That with a path FD a total escape is possible isn't that obvious, but not unexpected.

Well, it is an issue if you can use it to break out of containers that use --no-pivot-root (you can). fd-related escapes are possible for pivot_root(2) but are usually not as trivial as the one for chroot(2).

Also there are other attacks against chroot(2) which don't make use of fd-related escapes (nd->root protections in pathname lookup are very minimal and there are lots of ways to trick them).

This progam is bugged as you're relying on file pointer aquired before the chroot.

Imagine this program was running within a chroot(2) that it didn't set up. You can nest a chroot(2) and break out -- that was the point I was making.

No, nested chroots do not change anything here.

In your example, you would always need to acquire a file descriptor outside the current chroot first. Then such a file descriptor has to be passed over some mechanism into the process running in the chroot. For that, you would definitely need help from some other process running outside the chroot. Breaking out a chroot on Linux is not as trivial as you present it.

> In your example, you would always need to acquire a file descriptor outside the current chroot first.

The attack is the following:

  0. Process is running in a chroot.
  1. Process opens a handle to ".".
  2. Process chroot()s to a directory underneath ".".
  3. Process chdirs to handle made in (1).
  4. Process chdirs to ".." a bunch of times.
  5. Process chroot()S to "." which is the host "/".
Yes, this requires CAP_SYS_CHROOT (and this is why user namespaces don't work in a chroot -- because breakouts would be trivial). But as I mentioned above there are other VFS-related attacks which are a bit too complicated to get into in a HN comment (they involve RENAME_EXCHANGE and other such fun tricks).

But it doesn't require another process to help you. Seriously, just try it.

Again, this is an application bug. It's well known that you need to chdir after chroot and be careful with file pointers acquired before chroot. This behaviour could, arguable, be considered a feature of chroot, albeit a footshoot-y one. Either way, a program behaving in that way is flawed and should be fixed (which is usually not that hard).

Alright, I'll repeat it one more time.

The (attacker's) program is already running inside a chroot -- presumably because they got code execution from some chroot-ed process (after all, chroot is a layer of security right?).

The attacker opens a file handle to ".". They then chroot() to a subdirectory. Then they fchdir() relative to the file handle all the way to the host's root. Then they chroot() to the host root. They have now broken out of the original chroot(). I really don't think I can break it down any further.

I would not see this as breaking out of a chroot, because your process already had the file descriptor handle before the chroot system call. Being able to access files that were opened before the chroot call is an intentional feature and used to limit access to a specific set of files.

No. The file handle is created after the initial chroot. Then the process chroot again! By fchdir to the file handle, the new AND original chroot are defeated, and a follow up chdir to ../../.. will go to real root.

Just to make sure I understand what's happening here:

1) a direct chroot("../../../") would not work, as the kernel restricts ../ to / if at the chroot-root.

2) chroots are not nested.

3) By making a handle to a path, then chrooting, the security measure 1) is disabled for that handle (as it's outside of the current chroot), and the kernel doesn't keep track that it was created in an earlier chroot.

Is that right?

(1) is mostly right. The actual restriction is that ".." will be a no-op if the current directory is equal to nd->root (what your current root is).

(2) Yes, each process only has one root at a time.

(3) The restrictions don't apply to file handles at all (an fd doesn't know what chroot it was created in). Instead what happens is that the restriction in (1) is useless once the current directory of the resolution is above the nd->root (after all, a directory above nd->root is not equal to nd->root).

Ah, thanks I think I got it now. That's indeed trivial to execute as long as one still can do chroot.

to be clear, neither will work on a reasonably modern system?

No, the above escape will definitely work on a modern system.

Thanks that was quite helpful.

> Wait, are there people who really think chroot offers security?

The real answer is that this is a bad question. chroot disables some attack vectors. For example a path traversal vulnerability in a reverse http proxy is almost entirely mitigated by putting the proxy in a chroot. There may be extra details like access to cache or configuration, but chroot largely prevents harm from that issue. It does little against people with local access. So basically - it's more nuanced. "X offers security" is a trap.

But isn't chroot typically used by normal users anyway ?

Maybe were you thinking about fakeroot ?

"fakeroot runs a command in an environment wherein it appears to have root privileges for file manipulation. This is useful for allowing users to create archives (tar, ar, .deb etc.) with files in them with root permissions/ownership."


chroot(2) requires privileges (CAP_SYS_CHROOT on Linux). Otherwise you could trick setuid binaries into doing some pretty bad things.

The same that keep writing security critical code in C, without any kind of static analysers and code fuzzing as part of their build.

If an unprivileged process can gain super user privileges, you have a problem. This applies to both situations with chroot and those without chroot.

To understand just how trivial it is to escape a chroot with root, just run "nsenter -m -t $$" and your out.

That doesn't work for an unprivileged user, does it?

Is systemd-nspawn more secure?

cyphar made a lot of comments here on usage of mount namespaces and pivot_root.

Not really.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact