> Almost all popular applications on flathub come with filesystem=host, filesystem=home or device=all permissions, that is, write permissions to the user home directory (and more), this effectively means that all it takes to "escape the sandbox" is echo download_and_execute_evil >> ~/.bashrc. That's it.
> To make matters worse, the users are misled to believe the apps run sandboxed. For all these apps flatpak shows a reassuring "sandbox" icon when installing the app (things do not get much better even when installing in the command line - you need to know flatpak internals to understand the warnings).
I have not used flatpack. Is this description accurate? Also:
> Up until 0.8.7 all it took to get root on the host was to install a flatpak package that contains a suid binary (flatpaks are installed to /var/lib/flatpak on your host system). Again, could this be any easier? A high severity CVE-2017-9780 (CVSS Score 7.2) has indeed been assigned to this vulnerability. Flatpak developers consider this a minor security issue.
There was already a post on this. Basically the argument about home is true but this is because 1) apps should not use filesystem access but rather portals (if they can) 2) nothing should be executable in the home folder (nobashrc, no script, etc...)
If I remember well the second argument was about update not being frequent enough.
So nothing fundamentally about Flatpak but more about the infrastructure (lack of updates) and the use of it (we should not allow home access and use Portals or we should disable bashrc).
Says who? The purpose of a home directory to contain user-specific files including executables. Developers compile their software and write their scripts in their home directory. Even if we made the absurd decision that no file may be executed from the directory, there are many ways to cause harm by simply editing user-specific configuration files (e.g. in ~/.config).
Arguing that the problem is with executables in $HOME rather than Flatpak is incredibly delusional.
The data URI scheme is standard and widely supported, does not rely on the host bitty.site being reachable and does not need JavaScript. One can even create data URIs with a small shell script that is given a filename argument:
There exists DJB's redo approach [0], which i implemented [1], where dependencies and non-existence dependencies are only recorded after the build. A typical dofile is a shell script, so you do not need to learn another language. Targets also automatically depend on their own build rules (I have seen such a thing only in makefiles authored by DJB).
I wrote a blog post to show how to integrate dependency output for both dependency and non-existence dependency generation [2]. The game “Liberation Circuit” can be built with my redo implementation; you can output a dependency graph usable with Graphviz [4] using “redo-dot”.
There is only one other redo implementation that I would recommend, the one from Jonathan de Boyne Pollard [5], who rightly notices that compilers should output information about non-existence dependencies [6].
I would not recommend the redo implementation from Avery Pennarun [7], which is often referenced (and introduced me to the concept), mainly because it is not implemented well: It manages to be both larger and slower than my shell script implementation, yet the documentation says this about the sqlite dependency (classic case of premature optimization):
> I don't think we can reach the performance we want with dependency/build/lock information stored in plain text files
An issue I have with make is that it can not handle non-existence dependencies. DJB noted this in 2003 [1]. To quote myself on this [2]:
> Especially when using C or C++, often target files depend on nonexistent files as well, meaning that a target file should be rebuilt when a previosly nonexistent file is created: If the preprocessor includes /usr/include/stdio.h because it could not find /usr/local/include/stdio.h, the creation of the latter file should trigger a rebuild.
I did some research on the topic using the repository of the game Liberation Circuit [3] and my own redo implementation [4] … it turns out that a typical project in C or C++ has lots of non-existence dependencies. How do make users handle non-existence dependencies – except for always calling “make clean”?
Make has no memory so it can't remember things. It simply compares the dates of files. If a dependency is newer than a target the target is rebuilt.
If you want to keep some kind of memory you have to build and keep track of it yourself.
But the problem you point at is simply poor design. It is not a normal occurrence for system header files to move around like you state. If they do, a full rebuild is indeed required. It shouldn't so often that that is an issue.
If an apt-get upgrade fixed an issue in a system header or a library, but the date of the fix predates the last build (quite common; last build from yesterday, fix from two days ago but downloaded today) then make will do nothing (or any subset of the right things but not all) and make clean; make will do the right thing.
Relying on time stamps is a design decision that was good for its time, but it is no longer robust (or sane) an a constantly connected, constantly updated, everything networked world.
djb redo takes it to one logical conclusion (use cryptographic hashes to verify freshness)
There are other ways in which make is lacking: operations are non atomic (redo fixes that too), dependency granularity is file level (so dependency on compiler flags is very hard, dependency on makefile is too broad; redo fixes this too); dependency is manual (redo doesn't fix this; AFAIK the only one that properly does is tup)
> Relying on time stamps is a design decision that was good for its time, but it is no longer robust
I agree with the sentiment, but a small nitpick:
Relying on time stamps for older/newer comparisons is not robust.
Using time stamps (and perhaps file size) for equality checks is quite robust. And the combination with cryptographic hashes is even better (if a file is recreated but has the same contents afterwards, timestamp checks would trigger an unneeded rebuild, while a crypto hash check would recognize that there's nothing to rebuild).
Typically if a system header has changed or been added due to upgrading a library package you'll need to rerun any configure script anyway (since it very likely checks for header features and those decisions will change). So unless your build system magically makes configure depend on every header file used in every configure test it runs, you'll need to redo a clean build anyway, pretty much.
Make has a whole pile of issues, but this one really isn't an aggravation in practice, I find.
apt-get upgrade does not usually upgrade a package, despite the name; 99.9% of the time it applies a bug or security fix, almost never changing any functionality or interface - and would result in the same config script.
And that assumes you actually have a config script, which is also a nontrivial assumption.
Djb redo lets you track e.g. security fixes that change libc.a if you are linking statically, but that's not usually done.
The only build system I know that guaranteed a rebuild whenever and only when it is needed is tup. (Assuming you have only file system inputs)
Is this a common problem? I can't think of any project that does this, and there's a simple solution as well: don't shadow system headers. That's just asking for pain, regardless of how well make handles it.
I don't think this problem is limited to system headers. Something as innocent as #include "foo/bar.h" can be affected by this if you pass -I with at least two unique paths to the compiler.
Could you summarize how you handle this in redo? Also what about the case where a header file does exist but is out-of-date and because of that triggers an error (e.g., version compatibility check with #error) -- how do you handle that?
I, for one, handle it with a tool that mimics the compiler's preprocessing phase and emits both redo-ifchange information for all of the headers that are used, and redo-ifcreate information for all of the non-existent headers that are looked for during the process.
I also have a wrapper that takes arguments in the forms that one would invoke g++ -E and clang++ -E, tries to works out all of the platform and compiler include paths, and invokes this tool with them.
It's then a simple matter of invoking these redo-ifchange and redo-ifcreate commands from within the redo script that is invoking the compiler.
You can see this plumbed into redo in a real system in the source archives for the nosh toolset and djbwares.
I use strace(1) look for stat(2) syscalls that fail with ENOENT. An advantage of this approach is that I do not have to imitate the C preprocessor, so parser differentials can never happen. The following default.o.do file from my blog post [1] handles the case:
This approach is also used for building Liberation Circuit if strace is installed [2].
I think the compiler should output the necessary information. To quote Jonathan de Boyne Pollard [3]:
> As noted earlier, no C or C++ compiler currently generates any redo-ifcreate dependency information, only the redo-ifchange dependency information. This is a deficiency of the compilers rather than a deficiency of redo, though. That the introduction of a new higher-precedence header earlier on the include path will affect recompilation is a fact that almost all C/C++ build systems fail to account for.
> I have written, but not yet released, a C++ tool that is capable of generating both redo-ifchange information for included files and redo-ifcreate information for the places where included files were searched for but didn't exist, and thus where adding new (different) included files would change the output.
JdeBP, could you please release your tool under a free software license? I suspect it has fewer errors than the similar CMake approach [4].
Are you saying you want to be able to compile either way /usr/include/stdio.h and /usr/local/include/stdio.h, but remember what the last compilation used and know what header would be used in the next compilation, and if it's different, mark the target as stale and perform the action?
I guess you'd need to keep a log of the build and test cpp invocations for diffs.
This is where I saw the beauty of including dependencies w a project. Even on my own systems, as environments change, things break, and having a stable in-tree reference had paid off.
It's a tough situation, but I find myself leaning to @tedunangst position over the years - usually I try to adapt my machines (incl software) to my needs, but this case I need to take control/responsibility, and here be dragons. Does cmake actually solve this? Do other build systems?
I love that I get to use python to write the dependency graph, it allows for some interesting stuff.
Other than that, it's mostly the ease of use. This is enough to compile a C++ project (that has all its .c and .cpp files in the same directory as the SConstruct file), and it'll pick up on all dependencies correctly:
I'm not sure if it's hiding dependencies on system headers or not. But I can force it to show them by adding /usr/include and /usr/local/include to CPPPATH (excuse the long code block):
From what you are showing us, the answer to How does scons handle non-existence dependencies? is that it does not handle them at all.
Go and look at M. Moskopp's graph. It has a lot of dependencies for non-existent files that the compiler would have used in preference to the ones that it actually used, had they existed.
It used to be that scons really forced you to use subsidiary SConscript child files for anything more complicated than a couple files in a single directory instead of being able to lump it all into a single SConstruct.
I think that was quite long ago. Yes, `SCons` can fit into a single file if you so choose. But its behavior under recursive builds (with nested directory structure) is far more predictable than most build systems I have seen.
Almost all other systems have this flaw because they require dependencies before a build. If recording dependencies after the build, the entire problem becomes very simple. See here:
The example above was of a change to the build structure (include path, specifically) which changes the dependency tree in a way that doesn't involve changes to the files themselves and will be invisible to make or other tools that use file timestamps. I pointed out that this was just one of a whole class of intractable problems with dependency tracking that we all choose to ignore. It can't be fixed, even in principle. The only truly sure way to know you're building correctly is to build from scratch. Everything else is just a heuristic.
The following sentence about my redo implementation is wrong:
> In 2014, Nils Dagsson Moskopp re-implemented Pennarun redo, retargetting it at the Bourne Again shell and BusyBox.
I targeted the Bourne Shell (sh), not the Bourne Again Shell (bash). Also, my redo implementation contains redo-dot that paints a dependency-tree – I have not seen this otherwere.
You only put /bin/sh as the interpreter. You still used quite a number of Bourne Again shell constructs in the scripts themselves. Some are not in POSIX sh and utilities, some are not in the Bourne shell. (The Bourne shell is not conformant with POSIX sh, so targetting POSIX sh is not the same as targetting the Bourne shell.)
These include amongst others local, $(), >&-, the -a and -o operators to test, and command. (You also failed to guard against variables expanding to operators in the test command, but that is a common general error rather than a Bashism.)
Beware of testing against /bin/sh, even the BusyBox one, and assuming that that means POSIX compliance, let alone Bourne shell compatibility. Even the OpenBSD Korn shell running as /bin/sh or the Debian Almquist shell running as /bin/sh will silently admit some non-POSIX constructs.
* Add Enhances: caja, tumbler (>= 0.1.92~), nautilus, nemo
These are some of the many file managers/thumbnailer programs that support
desktop thumbnailers like exe-thumbnailer, and I have verified (at some
point) that all of these work.
This article has some good points but the reasons we add more logic to templates are:
- just pre-generating everything outside of the template can be very efficients. Especially if you language can't make everything lazy or if you have several representations for the same dataset.
- designers want a bit more freedom that just printing x. Having to go back to the dev team everytime you need a little tweak is terrible
- all templates are not HTML
- it's way easier and faster to prototype
- rendering caching != data caching
- everything is not about display. Linking and injecting resources are a big deal, and putting that outside of the template is a huge pain.
- conditional template inheritance ? includes in loop ?
- stuff like wordpress have entire business based on the fact you can switch templates on the fly without touching the blog code base or without the wp team to know what you are going to need inside the template in advance.
> you simply include everything in the markup that might need to be there, and the programmer removes whatever is not necessary
I realize you may not be cosigning on everything in the article you're quoting, but this is the author's first suggestion to an alternative to template languages.
I work on an SPA that was built like this. The index.html is over 10k lines long. It contains almost every single piece of the UI. Templating libraries and languages aren't perfect but they offer a better separation of concerns than just "dump it all in one file and write some imperative dom fiddling code to add different states"
Note that I made a different proposal above the quote.
I think SPAs are either fundamentally dishonest engineering, in the same way that a microwave wrapped in artificial wood veneer is (and a stainless steel microwave is not) or should result in such a template. If you really think that this is too much, IMO you should not make an SPA in the first place.
> I think SPAs are either fundamentally dishonest engineering, in the same way that a microwave wrapped in artificial wood veneer is (and a stainless steel microwave is not) or should result in such a template.
Could you rephrase or expound on this metaphor? I have no idea what you mean by "fundamentally dishonest engineering" or what that has to do with SPAs and templates.
> Almost all popular applications on flathub come with filesystem=host, filesystem=home or device=all permissions, that is, write permissions to the user home directory (and more), this effectively means that all it takes to "escape the sandbox" is echo download_and_execute_evil >> ~/.bashrc. That's it.
> To make matters worse, the users are misled to believe the apps run sandboxed. For all these apps flatpak shows a reassuring "sandbox" icon when installing the app (things do not get much better even when installing in the command line - you need to know flatpak internals to understand the warnings).
I have not used flatpack. Is this description accurate? Also:
> Up until 0.8.7 all it took to get root on the host was to install a flatpak package that contains a suid binary (flatpaks are installed to /var/lib/flatpak on your host system). Again, could this be any easier? A high severity CVE-2017-9780 (CVSS Score 7.2) has indeed been assigned to this vulnerability. Flatpak developers consider this a minor security issue.