There were some misunderstandings that I want to clear up (maybe I will add them in an update to the blog post):
1. Some people mentioned that this would "only affect RAR files" and it would be safe to extract 7z files with 7-Zip prior to version 18.05. This is wrong, because 7-Zip detects the file type from the magic numbers at the beginning of the file. So the exploit can be renamed to 'exploit.7z' and it works just as well.
On /r/sysadmin, someone even mentioned that a temporary solution might be to block RAR files. By the same argument, this is unlikely to be effective.
2. Almost all versions prior to 18.05 are affected. I manually checked version 15.05 and 17.01, and they are definitely affected.
3. Not only 7-Zip itself is affected, but essentially all software that uses 7z.dll as library to extract files. This includes various anti-virus software. However, exploitation may be more difficult (though not impossible) if ASLR&DEP is properly enabled (on all modules).
It's fascinating that this category of equipment, which searches for viruses by running untrusted code, is still regularly installed in all corners of valuable networks.
As far as I understand the bug, this is not about running untrusted code (but: a parsing error resulting in state corruption). Unless you refer to the 3rd party lib (7z), used by virus scanners, to analyse rar files. But typically "untrusted code" means code that was supplied "at runtime", not at compile time (like a lib), so e.g. if a virus scanner would actually execute a .exe to evaluate its effects, or run javascript found in a webpage.
To be fair to virus scanner vendors, the only way to mitigate this kind of bug is NIH: don't use 3rd party libs, implement everything yourself. But then, of course, without bugs yourself, as well :)
> To be fair to virus scanner vendors, the only way to mitigate this kind of bug is NIH: don't use 3rd party libs, implement everything yourself. But then, of course, without bugs yourself, as well :)
That is not the only way to mitigate such vulnerabilities. AV vendors have had plenty of time to work on sandboxing parts of their scan engines that have repeatedly been found to have vulnerabilities like this one. Somehow we've arrived at a point where no one would recommend using a browser that doesn't utilize sandboxing to some degree, but when it comes to security products, you can count yourself lucky if they don't just run that code as SYSTEM. That should really tell you all you need to know about the state of the AV industry.
It is a very blurry line between running strange code and running library code against strange data. Mostly though, I'm trying to say that AV boxes should be outside the firewall.
What AVs use this? I keep hearing horrible things about AVs but little proof. I know Tavis from Google finds bugs and AV in rare cases can reduce security, but that's a far cry from them embedding FOSS projects to save time. For AV companies writing a rar parser is a single-day activity, its hard to believe they need this app.
The few places I've seen 7z used is in installers where the input is known (installer archive in 7z format) and I'm assuming signed in many cases so you can't feed it random inputs. I find it hard to believe Sophos and Symantec and Trend are copying and pasting 7zip.dll into their apps.
I don't know about 7-Zip specifically, but AV vendors use plenty of FOSS code. Here's some findings just from Google's P0 showing that Symantec[1], Bitdefender[2], Microsoft[3] and Avast[4] all use unrar in their products. It wouldn't be far-fetched to assume some might use 7-Zip for other archive formats.
Maybe I'm just way overestimating how complex RAR archives are (I admit, I have not looked into this), but I think you're out of your mind if you think that someone could write a parser to analyze RAR archives in a day. That it would be better tested, debugged, and more secure than a tool that has probably seen more widespread use than any single AV.
DEP was previously disabled because Igor used to compile 7-Zip with VC6, which doesn't support the /NXCOMPAT flag. I convinced him back in January to enable it for 7-Zip 18.01. Note, however, that 64-bit versions of Windows enforce DEP even if the /NXCOMPAT flag is missing. Since Windows 10, the 32-bit version does this as well.
ASLR was primarily disabled because Igor wanted to strip the relocation from the binaries in order to save about 0.5-1% in file size. I have discussed this with him, and convinced him to enabled full ASLR for 7-Zip 18.05.
So now we have 7-Zip 18.05 with full ASLR and DEP. Stack canaries (/GS) are still disabled though.
If you can convince him to use Control Flow Guard, Stack Canaries, and HE-ALSR then you should be nominated for for whatever the security community has as an equivalent to a Nobel prize.
If you can convince him to get rid of his custom garbage Stdlib replacements and use ISO C++ then you're a hero to maintainability (and would probably improve the performance because the stdlib has move support).
HE-ASLR I am discussing with him right now, and I think we will get this.
But honestly, I don't think we will ever see a 7-Zip with /GS or CFG. Not only would this cost about 1% in binary size, it would cost an additional 1% in runtime performance loss. Additionally, it would require compiling 7-Zip with a modern compiler like VS2017. You're just asking for too much.
Like blibble says, this is an absurd tradeoff given that 7zip's primary use case is unarchiving files downloaded from the public internet. Between that, the willful ignorance and dismissiveness of security measures displayed https://sourceforge.net/p/sevenzip/feature-requests/1270/, and the apparent need for you to keep pressuring him to enable each countermeasure one-by-one, I question the prudence of using 7zip at all. He is clearly a great developer, and I have no reason to suspect ulterior motives, but his actions don't engender trust.
I know this probably isn't the answer you're looking for, but it might be worthwhile to maintain a parallel fork to enable these options for yourself/others. That way, folks that feel the same way you do can have their cake and eat it too. I realize that it's not an ideal solution though.
Why am I getting downvoted? I sincerely wonder what is so objectionable in my comment. Most uses of 7-zip are one-off events that last around a minute long. Most users aren't decompressing terabyte-range files. It makes no fracking difference to users if they have to wait 5 minutes or 1 minute. But it makes a difference if they receive 7z archive and get their bank account cleaned because the author prematurely optimized his program on a 20 year old compiler. If there was ever a reasonable situation where security flags could be disable then client-side decompression utilities aren't one. Anyone who needs that 5% performance improvement can compile the library themselves. This is what is wrong with this industry.
Consider that it was misplaced to suggest a 500% performance drop. It's used to extract large archives, backups, videos, documents. Folks don't want it to take half an hour instead of a few minutes.
Thanks for replying, this explains things. I guess I extrapolated from my needs too much, I didn't consider users who do daily 10+ GB compressed backups.
I still wonder if there's an ethical dilemma that lies underneath this - is it fair to prioritize needs of users who use your tool in a different way (for backup jobs) over users that use your tool only occasionally, perhaps not even out of their own initiative (they were sent 7z file from a ostensibly trusted source).
> Additionally, it would require compiling 7-Zip with a modern compiler like VS2017
Ahhh, but does say VS2017 produce a smaller executable file, or a faster executable?
I am pleased Igor cares about individual 1% improvements - they stack up to significant savings. However I agree for our work usage security is more important.
> does say VS2017 produce a smaller executable file, or a faster executable?
If I recall correctly, Igor once said that he tested the new VS compiler and it produced neither smaller nor faster executables. I believe there was almost no difference.
Given what I've seen of his code, I'm not surprised. When I mentioned the stdlib I wasn't joking. His code is unable to use modern compiler optimizations such as move, RVO, and constexpr because he's using idioms that prohibit it. It would be nice if he would at least release a solution that others can use to build with VS2017.
Why is that asking too much? I realize paying for software is not something everyone wants, but doesnt the free versions of VS work for compiling 7-zip?
Dunno, just as with FB/CA, users have agreed to what befell them. Relevant extract from LGPL 2.1: "THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE LIBRARY IS WITH YOU."
I agree with others here on the security vs. performance and security vs. binary size trade-offs. As a thought experiment I wondered at what point it would become a hard decision if I were in his shoes. I think a 10% performance hit would make it a harder decision for me, but at that point I would probably still err on the side of caution and prioritize security.
Of course it's possible this is exactly one of the reasons Igor owns something like 7-Zip, and I don't :-)
Developer cares about efficiency. It seems strange in today's world of Electron apps and text editors that take up a GB of ram and have hundreds of ms of latency, but that's clearly where his priorities lie and I, for one, can respect that.
If people are so up in arms about the security problems of 7-Zip, they're free to fork it.
> Some people mentioned that this would "only affect RAR files" and it would be safe to extract 7z files with 7-Zip prior to version 18.05. This is wrong, because 7-Zip detects the file type from the magic numbers at the beginning of the file. So the exploit can be renamed to 'exploit.7z' and it works just as well.
But the contents of a file is what really determines the actual file type, though.
So if you decide to block RAR files, you should do that by looking at the magic number. Then you are good to go. And it's not like you have to write a lot of code or anything to do this. The standard 'file' utility in Unix is able to determine the type of a file -- often by looking exactly at the magic number.
Just like you would inspect the actual contents of a file if you were running an image hosting site -- and not trust whatever arbitrary filename the uploader told you that the file had.
Of course, but I would still strongly advise against this.
If you really cannot avoid implementing something like this, you should inspect the 7-Zip code in order to be 100% sure that the magic number detection in your filter is identical (or matches a superset) to the one from 7-Zip.
> you should inspect the 7-Zip code in order to be 100% sure that the magic number detection in your filter is identical (or matches a superset) to the one from 7-Zip
Those are the two magic numbers for RAR archive version 1.50 onwards and RAR archive version 5.0 onwards respectively, and those are the places they are referenced in the 7-zip source code. I looked at the source archive of the 18.05 version, downloaded from https://www.7-zip.org/a/7z1805-src.7z. I guess if you wanted to be really rigorous you'd look at previous versions as well. If the 7-zip project makes use of a version control system and that vcs supports the equivalent of git blame then that should not be too difficult of a task for whoever wanted to go to that extent of investigation.
And here is where one copy of the 'file' command identifies the same magic numbers as RAR archives:
You missed my point. It is trivial to find out what the magic number is. What is more important though: How exactly is the magic number matched? From what you have written, one might be tempted to simply check whether a file begins with this magic number. And this would be wrong. If you take a look at the matching in CPP/7zip/Archive/Rar/RarHandler.cpp:
7-Zip finds the magic number if it appears within some searchHeaderSizeLimit, i.e., the file does not need to start (at offset 0) with the magic number. For example, 7-Zip will extract a RAR file which begins with [00 52 61 72 21 1A 07 00] (instead of [52 61 72 21 1A 07 00]) just fine.
7-Zip needs to start a Go Fund Me or similar for a Code Signing certificate. They're like $69-89/year, which is expensive, but for such a popular piece of software it would be a nice safety net in case of site compromise.
Too bad none of the big CAs have an Open Source/Charity program that would provide a Authenticode Certificate for use with that software.
I second it. The thing with donations is many people assume that someone is gonna donate (including me) and click "not now or later", where as in gofundme we would see how much they raised out of total goal and many people will then put the money.
IIRC 7-Zip has explicitly decided not go get signed. It doesn’t help all that much anyway, SmartScreen still catches your application and nags the user.
Unfortunately, I cannot seem to find any reference, so I might remember it wrong or it wasn’t about 7-Zip or whatever. The thing with SmartScreen is (unfortunately) still true.
I wonder what do you mean by "not useful"? They just have to participate in the reputation system, but that's an issue only when the certificate is young.
Here's an excerpt from MSDN:
> Detractors may claim that SmartScreen is “forcing” developers to spend money on
certificates. It should be stressed that EV code signing certificates are not required
to build or maintain reputation with SmartScreen. Files signed with standard code
signing certificates and even unsigned files continue to build reputation as they
have since Application Reputation was introduced in IE9 last year. However, the
presence of an EV code signing certificate is a strong indicator that the file was
signed by an entity that has passed a rigorous validation process and was signed
with hardware which allows our systems to establish reputation for that entity more
quickly than unsigned or non-EV code signed programs.
I didn’t say “not useful”. Clearly they’re useful. I said non-EV certs “aren’t as useful”. Which is just a fact (as evidenced by the Smart Screen “reputation boost” that EV certs get).
I already read that blog post. I’m person that linked to it in the forum post.
However, isn’t getting an EV certificate impossible for a natural person? You’d have to be some sort of legally recognized organization. Not exactly suitable for small-scale Open Source development.
KeePass: This isn’t an EV certificate (has only OID 2.23.140.1.4). Certum also clearly states, topmost on the description of how to get an EV Code Signing certificate:
> We do not issue EV Code Signing certificates to natural persons!
Yarn: Not an EV certificate either: "Organizationally validated certificates used to sign standard objects." (2.16.840.1.114412.3.1 in addition to 2.23.140.1.4.1).
> It doesn’t help all that much anyway, SmartScreen still catches your application and nags the user.
Windows 10, in the default configuration, won't let you install unsigned applications at all. It might "nag" early downloaders when a certificate is present but you can override that (and the nagging stops eventually due to popularity).
But more importantly, if the site were ever compromised, it makes it easier to spot if a compromised binary is posted. Since hopefully the bad guy wouldn't have the code signing certificate. Right now the official binary looks like a compromised version.
When I was just installing it now (this post reminded me that I hadn't updated it recently) I was put off by the lack of publisher in the installer and went back to double-check it was the official version. It's really off-putting seeing "publisher: unknown" when installing and my immediate reaction was, "wait, did I install it from the legit site?".
AFAIK 7-zip doesn't take donations (feel free to correct me), but you could argue that the donations were going towards the author of the software to thank them for their work and not towards funding the development, so the author has no reason to feel compelled to buy a cert if they don't want to. Of course, that would all depend on the wording of the donation page, which I don't believe exists.
I've never found (recently) how to donate to 7-zip, it seems the author removed the option to do so on the website. Maybe I'm too stupid, so please correct me if I'm wrong and you find a link.
I can picture this working, actually. Someone could put some smart contracts into the Ethereum blockchain, for example, one per piece of software, with the instruction that if the contract receives more than a certain threshold in total donations, a transaction is sent to a CA asking them to issue a code signing cert to the developer of the relevant piece of software (hardcoded into the smart contract).
The CA would have to be in on this, by having an Ethereum address to receive the crowd-funded amount at, and they would have to make contact with the developer and verify them using their normal methods, but there would be a strong financial incentive for them persuade the developer to accept their certificate. Perhaps if the developer declines, then the funds controlled by the smart contract can expire and be sent back to the unsuccessful crowd-funders.
Yeah, there is no question about it. This website clearly says that it's the legit source for 7zip. There is even a red box on the right side of the page. This needs to be taken down if this is not an official source.
The left side has a navigation to different translations of the page.
All but the English version link to the German page as well.
I'm guessing it was once part of the build pipeline but has since been abandoned.
So yeah, it is an official source. It's just outdated
Or you know, the author of 7-zip could pay for a digital certificate and sign the executable. Fake websites and "Trojans" are a known problem, with a known solution.
Unfortunately, 7-zip barely has any security involved. No digital signatures, no ASLR, no NX bit, no stack canaries, no nothing.
Hopefully these security concerns wake up Ivor. Its not the 90s anymore: developers have to participate to get a proper security posture. That's why Windows tried so hard to get everyone to use sandboxed Win10 Apps / Metro Sandbox by default, because these problems require the developers to care about security.
> So yeah, it is an official source. It's just outdated
`whois 7-zip.de` resolves to a private person in Germany. This does not look official to me. More like a crowdsourced effort of providing translated websites with a dangerous effect in case of security vulnerabilities.
The versions provided are (as of 2018-05-04T10:20:00Z): en 18.05, de 16.04, zh 16.04/18.05, eo 18.01, fr 18.01, ja 18.05, pt 18.01, es 18.01, th 18.05, vi 18.01
I just checked and I was on v9 from 8 years ago on my work pc. Why bother fixing security bugs etc if you're not going to roll them out? With other Windows software I get told about updates when I load them (winscp, Virtualbox) or they check and update themselves (Firefox).
Because there are multiple conflicting priorities here. On the one hand, it is good to keep software updated, and therefore software should check for updates. On the other hand, software should restrict itself to solving one problem domain. Interacting with the internet is something wholly distinct from decompressing files, and so the software should not branch off into a new domain. Choosing between these priorities is not necessarily straightforward.
I could also argue that automatic updates are themselves a security hole. They are a way for new code to be downloaded and run, without notifying the user. As a result, it means that your security depends on the security of a machine not under your control. Not too much of a risk for Firefox, but imagine having a program that auto-updated from SourceForge during its experimental fling as a malware distributor.
A real-life example of the risk of automatic updates: about a decade ago, the maintainers of the Shareaza multi-protocol p2p client lost control of their domain, and the new owners of the domain pushed their own (sleazy, commercial) software to unsuspecting Shareaza users via its update mechanism.
(IIRC the maintainers learned the right lesson from that, and started signing their updates so it can't happen again)
Windows is special here because Microsoft never properly solved the distribution problem for software.
If you want the typical user on Windows to run updated software, your software has to at least entice updating if not auto-update straight away.
Not solving this whole distribution mess is by far the worst downside of Windows as a platform. Not getting malware when installing software on your Windows PC is hard.
Assuming I agree with you, what's the reason for not telling me about updates when I run the app? What's the advantage of the decision they've taken which is to not announce this?
Because even determining that there is an update available requires checking against an outside source to see if an update is available. This requires internet access, which requires handling network protocols, network security and encryption, none of which have anything to do with file compression. Increasing the scope of a project introduces additional failure modes, and a larger security risk.
If a project already performs telemetry, or if they have developer announcements, then the project has already increased its scope, and checking for updates is a relatively minor addition. If it is a well-behaved stand-alone application that doesn't make unwarranted external connections, then checking for updates is a large increase of scope.
Old thread, but I'd like to point at http://scoop.sh [1] which provides a command line package manager for Windows - and allows easy update of packages.
Chocolatey is similar in some ways, but scoop works hard to isolate installed apps from each others, and from other users (which can be good and bad). It's a little like an apt-like wrapper for binary-only (x)stow.
[1] scoop is hosted on github - the download url is on proper ssl, the bare scoop.sh domain presents a github-cert.
Note that the standard 'p7zip' package from Debian/Ubuntu doesn't support RAR. However, they have an additional package 'p7zip-full' or 'p7zip-rar' for RAR support. I didn't check explicitly, but I assume these are affected.
I checked the versions in buster (p7zip-full 16.02+dfsg-6, p7zip-rar 16.02-2) and they look unaffected to me. Turns out that the Debian maintainers patch upstream sources to include hardening flags, e.g. -fstack-protector-strong -D_FORTIFY_SOURCE=2 -Wl,-z,relro. You can use hardening-check to check the binaries on your system.
Okay, so these packages come with more mitigations than 7-Zip on Windows. However, looking at the source code, I am pretty sure they are affected by the same bug.
"Unaffected" was probably the wrong word to use. What I meant is that one of the mitigations (making the executables position-independent) should prevent the bug from being exploitable for remote code execution on Debian.
That's right, they patched CVE-2017-17969, which affected ZIP decompression. Interestingly, I believe they didn't patch CVE-2018-5996 (affecting RAR), which I published [0] on January 23 together with CVE-2017-17969.
Ubuntu doesn't get these patches, unless some non-Canonical volunteer puts in the effort to prepare a package and get a sponsor from Canonical. Future version of ubuntu will eventually get the fix because Ubuntu it will be forked off from a sufficiently new version of Debian.
I have always used 7-Zip on Windows. Having done some reading now, the author's general attitude towards the tradeoff between security and executable size/speed have convinced me to try and not use it in the future. Thankfully I rarely have to use Windows these days.
Hi all,
In 18.01 Igor had fixed CVE-2018-5996 with adding some variable like _errorMode or m_TablesOK.
And in 18.05 I don't see this variables. Igor was replace it by _solidAllowed to fix CVE-2018-10115. Does it fix for both CVE-2018-5996 and CVE-2018-10115?
Thank you
I think this is correct. Since _solidAllowed is set to false at the beginning of Code(), it will remain false if an exception occurs in the middle of decoding (CVE-2018-5996). This will enforce PpmError being set to true for the next item, which in turn will enforce the (possibly broken) PPMD state to be reinitialized. In some sense, this means that the new bug fix is a generalization of the first one, fixing both CVE-2018-5996 and CVE-2018-10115.
For extraction, use bsdtar from libarchive. It supports various container and compression formats. I never looked back to GNU tar, bsdtar is vastly superior.
There are extensions for tarball and zip file formats that handles the problem, but not many software supports these extensions, whereas every software supporting 7z handles unicode filenames correctly.
Nowadays when that sort of bug is discovered, the question that naturally comes to my mind is "would that have happened if the software were implemented in (safe) Rust"? In that case it looks like the answer is no.
Of course 7-zip is much older than Rust so that's just a thought experiment.
Comments like this hurt the adoption of Rust by making the community seem hopelessly naive. It's just how people make the Go community look bad by acting like Go invented CSP.
Rust is a (very?) good language that I hope will see more adoption but it is not the first memory safe language. Garbage collected languages are perfectly appropriate in many situations. Ada is almost 40 years old.
Memes about programming languages (and programming language communities) die hard so it's probably a good idea to avoid reinforcing them when possible. I read this comment as a sort of implicit variant of "RIIR". Judging by the downvotes, I think others did the same.
Rust probably doesn't even deserve the "RIIR" meme as 1) "RIIR" seems to happen way more often on HN/similar than on mailing lists or bug trackers and 2) much of the time the person saying "RIIR" admits to not even being a Rust programmer themselves. I think it's just a side effect of Rust (justifiably) emphasizing safety, and by extension security, in its presentation, and the tendency of some people to conflate the elimination of a certain class of vulnerabilities with the elimination of all vulnerabilities. To be fair, this poster didn't make this mistake.
Edit: Another mistake that some people make after being introduced to Rust is assume that languages that don't explicitly emphasize memory safety in their presentation aren't memory safe. This poster comes across as potentially making this mistake.
Modern C++ does not mandate that every value be a valid instance of its type wherever it is theoretically accessible, so I'm not sure why you're saying modern C++ fixes this. Actually, the fact that Rust does is something people somewhat regularly complain about. It is a heavy-handed performance / code complexity vs. safety tradeoff that certainly fixes this bug, which I think is a pretty reasonable thing to point out.
If you follow C++14 core guidelines, the defect described would not have occurred. The bug at its core is usage of unsanitized input data. You don't need to "mandate that every value be a valid instance of its type wherever it is theoretically accessible", however that may differ from simple sane C++14 paradigms.
OPINION ALERT: Honestly guys get over rust, it does not offer a single advantage in real-life programming scenarios.
The C++ compiler can't enforce many of these rules (and many of the ones it does claim to enforce, it only does heuristically), and the document itself is not shy about admitting that. It also makes quite clear that it expects not just mechanical checking, but things like peer review, to take these guidelines into account--meaning yes, you are expected to know a large portion of the document. Also, a lot of the listed enforcement mechanisms are literally just flagging suspicious code--you would have to disable a tremendous number of such flags to execute many perfectly reasonable programs. Stop misleading people about the contents of this document actually being enforced by the compiler, please.
1. Some people mentioned that this would "only affect RAR files" and it would be safe to extract 7z files with 7-Zip prior to version 18.05. This is wrong, because 7-Zip detects the file type from the magic numbers at the beginning of the file. So the exploit can be renamed to 'exploit.7z' and it works just as well.
On /r/sysadmin, someone even mentioned that a temporary solution might be to block RAR files. By the same argument, this is unlikely to be effective.
2. Almost all versions prior to 18.05 are affected. I manually checked version 15.05 and 17.01, and they are definitely affected.
3. Not only 7-Zip itself is affected, but essentially all software that uses 7z.dll as library to extract files. This includes various anti-virus software. However, exploitation may be more difficult (though not impossible) if ASLR&DEP is properly enabled (on all modules).