Hacker News new | comments | show | ask | jobs | submit login
7-Zip: Multiple Memory Corruptions via RAR and ZIP (landave.io)
252 points by landave 58 days ago | hide | past | web | favorite | 101 comments

Not turning on standard mitigation techniques because of binary size is one of the strangest reasons I've heard. And then still programming in an unsafe language, quite self-confident for a "humble programmer".

https://www.cs.utexas.edu/~EWD/transcriptions/EWD03xx/EWD340... It has already taught us a few lessons, and the one I have chosen to stress in this talk is the following. We shall do a much better programming job, provided that we approach the task with a full appreciation of its tremendous difficulty, provided that we stick to modest and elegant programming languages, provided that we respect the intrinsic limitations of the human mind and approach the task as Very Humble Programmers.

He won't go https, sign his binaries, or enable mark-of-web either. It's strange to see people still playing small binary golf in 2018.

Oh wow, I always assumed he just defaulted to HTTP version of the site, but HTTPS was an option. But nope, there isn't even a version of the site served over HTTPS. In the year 2018.

Yeah, that's pretty terrible. Seems like the only secure way to get a copy of 7-zip is to install it via Chocolatey or Sourceforge.

Since they get theirs from an insecure source it really isnt.

No, you can get them from the official Sourceforge project over HTTPS: https://sourceforge.net/projects/sevenzip/

Or use a newer toolset than VC6 apparently. Honestly at this point there needs to be an intervention.

It's open source; feel free to fork.

sigh already downloaded the source, trying to convince myself I need to do this. The code is pretty much pure C++98 along with all the pain that comes with that.

"Secure 7-Zip" even sounds like a good name.


Is there a good alternative to 7-zip?

I also lost my trust into zlib recently, given that the current released version is full of bugs, and the current master is written horribly. Not as bad as openssl, but close. E.g. they are still using K&R sigs, wrong casts, and have several copy&paste bugs which you only detect with a stricter c++ compiler.

for just decompression, unarchiver has free software implementation of unrar. for compression, use command line (tar -J, zip, whatever) or file roller.

Libarchive (sometimes known as bsdtar) supports 7z with an independent implementation.

I think he means an open source unarchiver that works on windows. Obviously Linux and bad have tar/gz/bz2 etc

What are your criteria? I moved to Bandizip a few months back and it covers all of my requirements quite well.

PeaZip is pretty good and is FOSS

> PeaZip is free file archiver utility, based on Open Source technologies of 7-Zip, p7zip, FreeArc, PAQ, and PEA projects.

It seems to use 7-Zip and/or p7zip binaries.

p7zip was also patched. 18.00 doesn't have the bug, but it looks like Peazip hasn't been updated with it.

I'm not sure what other FOSS alternative to use. BandiZip is free (as in beer) but doesn't seem to have an update since Sept, so if it uses p7zip under the hood for 7z, it's also vulnerable.


Please stop with the hyperbolic strawman. None of "https, sign his binaries, or enable mark-of-web" are "latest performance killing mitigation strategies". Nobody is arguing with 7z's success, either. I don't know where you're sensing that.

These are just reasonable (or even necessary) techniques to ensure the security of the product you're shipping. We ought to know better.

> He should be using a pile of frameworks on top of frameworks on top of a managed code environment with all the latest performance killing mitigation strategies. That's how we do things in 2018: keep adding crap until the software is slow and bloated.

This attitude is much of the reason why software security is as bad as it is today. I'll take a little "bloat" over ZIP files being able to install ransomware (and security vs. performance doesn't have to be a tradeoff anyway).

Using HTTPS for downloads and signing binaries won't magically make software slower.

It's the security equivalent of washing one's hands after using the toilet...

I feel strongly that you do not understand the performance implications of the mentioned mitigation techniques.

These specific ones? Yeah, not that big a deal, but my post is about a development mindset, not the specific implementation. A few vulnerabilities show up in otherwise really great software and the idiots who browse this joint literally say the guy needs an intervention.

I don't get what you're arguing - this has nothing to do with the developer mindset. He just has to flip a compiler switch and vulnerabilities won't be trivially exploitable anymore.

It's totally irresponsible.

Well, HTTPS and high-confidence builds are popular concepts here, given that this forum is frequented by people on both side of the security fence.

At least I would assume it is. There are lots of white-hat reports, so...

No kidding. The authoritarian security cargo-culting irritates me too.

The fact that 7-zip bugs are rare enough that they make news when they are discovered already says a lot about the overall quality of the code. Many other projects with all the bloaty mitigations and other ostensibly "for security" cruft still manage to create severe bugs on a regular basis.

I read posts like this and it's just so clear to me why we're so fucked.

On the other hand, Dijkstra also has this quote which suggests he is not too fond of "mitigation" either:

"If you want more effective programmers, you will discover that they should not waste their time debugging, they should not introduce the bugs to start with."

Yes, that's why the quote was already suggesting to use better languages, but mainly it's about attitude and being aware that things go wrong, so you should turn on what you can you responsibly decided to stick to C.

Two comments:

The way its written, I first took the mention of finding this "during the analysis of a prominent antivirus product" to mean that you were reverse engineering some AV thing and found that it was scanning for this vulnerability (i.e., to protect against bad archives). After a second read, it seems like maybe not, and that the AV itself re-used parts of 7-zip for its own implementation and was therefore vulnerable itself. Still not sure, though.

The way the stylesheet makes the "rendered" form (especially section headings) resemble markdown source is pretty neat.

You are completely right with the first comment. The antivirus product itself reuses parts of 7-Zip and is vulnerable itself. I mentioned this mainly because I did not analyze the original 7-Zip software, but only discovered that it was affected as well after I had found the bug in this antivirus product.

I admit that this is confusing, so I'll probably try to rephrase this.

Seems like a possible license violation then (7-Zip is LGPL).

LGPLG is let's you use a library without distributing the source code of the whole program.

Read section 4 of https://www.gnu.org/licenses/lgpl-3.0.en.html, there are some things you still have to do.

It's common knowledge that AV programs scan files inside compressed archives. Obviously you need to run the decompression code to do that.

It's also common knowledge that the AV industry has a huge software quality and engineering problem ("let's unpack malware and emulate x86 in kernel space, because that never backfired before!").

I've actually heard many people (including one Chrome developer) that they don't even use AV anymore except Windows Defender because 99% of AV break Windows/applications by using non-standard hooks and may even introduce new vulnerabilities with their kernel drivers/etc.


Honestly, if they can't even stop viruses from infiltrating closed systems like Android and iOS, I don't see how an anti-virus suite could ever win the battle against a user intentionally installing a virus.

(Of course, desktop apps are a different ballpark than web apps/websites where you're merely connecting to a website vs installing a dedicated application with filesystem access.)

I'm one of those people. I would be really interested in seeing what percentage of exploits, malware and spyware is caught by each feature of the common AV suite. I suspect that Defender is at the sweet spot.

Additionally, I see the entire AV market as a leech that was only able to grow as it had because Microsoft was able to shirk its responsibilities with respect to security for so long. Now that Microsoft are trying to handle the problem the AV companies are crying foul because they think they deserve the right to exist, which means Microsoft shouldn't do too much to help their own customers.

AV companies are the car dealerships of the digital realm. Superficial middlemen ensconced in a bygone era leeching money from the unwary. We should be careful, lest we legally formalize that relationship as we have with the dealerships.

My experience so far (helped maintain lots of non IT people's computers) was also, that the performance gain from removing antivirus software (and just using defender) was worth the theoretically less protection.( If even so)

Besides, with the behavior of most free Antivirus I could not really distinguish from common spyware. Everything needs to call home these days ...

So when friends ask me if this computer is now virus/spyware free .. they are usually a bit disappointed when I tell them, probably not, even if we delete everything and remove windows (but it would help).

But most people, including me, need windows from time to time, so it's allways a compromise. But common (free) Antivirus is really just snake oil.

Many of my customers run the common free anti-virus programs and I can assure you that they are not snake oil. The logs and alerts from blocked infection attempts are testament to this.

I certainly would say that most of the free anti-virus is pushy, hungry and generally not a particularly great marketing exercise.

Sure they do things - but they promise real security and protection - which is a laughable claim, when you look at detection statistics. They are not much better than defender. But unlike defender, they themselves act like spy/adware sometimes. Showing advertisement on the desktop etc. and communicating with the server a lot and not for updating.

Oh and my experience with most of their logs is, that they log and alert allmost everything - even when it is trivial as a tracking cookie - but make it look like a real threat to the ordinary user to show what good work they are doing. And especially on older computer really slow things down.

So I cannot recommend them and rather stress the importance of updates and not mindlessly install/click random things. (but with updates allmost everything important does that automatically now anyway)

> The logs and alerts from blocked infection attempts are testament to this.

Infection attempts by what? Scans of phishing mail attachments they wouldn't have opened anyway? At least if they know what they are doing. In addition, many AV have this annoying habit of reporting quite a bit of false-positives based on sys-calls or some weird heuristics, this leads to the situation where even totally legit software, from a trusted source, triggers a warning.

Which then conditions people to just click past the warning, at that point you might as well not even run the AV at all and instead just scan individual files online through some virustotal-like service and teach proper user behavior.

Due to this dynamic installing a good ad-blocker will probably do more for the security of the average windows user than any AV software ever would.

"Which then conditions people to just click past the warning, at that point you might as well not even run the AV at all"

anectode: a girl in my student flat wanted to give me some file and copied it to her usb-stick. But as she plugged her stick in, a antivirus warning popped up and said very clearly INFECTION DETECTED. But she just clicked it away and said it does that all the time since weeks ...

Wait what?!?

And she was a student (for high school teacher), so supposed to be not stupid. But in this case the antivirus was actually one of the better once which mostly only said something if there was something. But to her it had the same meaning as "update me please". Not something to be bothered with her now, as her task was to copy something to the USB stick ...

So yes, definitely also bad conditioned, but also plain stupid. Or overburdened.

So for those people, antivirus (wheter from the os or third party) which really blocks stupid things and scans everything, make sense. And there are a lot of those people ... in my example it was someone who grew up with computers, but there are still many around who had to learn it much later in life. And they just click onto everything.

> Infection attempts by what? In January I've seen logs blocking drive-by malware attempts, lots of infected email attachments and an infected USB stick.

These are not false positives. They were not legitimate software from trusted sources. The logs I read were real-world true positives and they were not inconsequential trivia like tracking cookies or the like.

I don't think that in any of the cases the user would have had a warning to blindly click through.

Not entirely sure how an advert blocker can stop email or device-carrying malware.

> They were not legitimate software from trusted sources.

Infected email attachments, unless they come from a trusted sender, I consider "useless positives" because nobody, with the appropriate training, should be opening them in the first place.

Kinda along the same lines of tracking portscans and counting those as "thwarted cyber attacks", like many government agencies tend to boast about, it's nice for padding stats but is it a real security gain?

Afaik by now one of the most common successful attack vectors is drive-by kits [0], increasingly served trough advertisement channels. Ad-blockers/disabling Java minimize this risk quite a bit, with low overhead, while having the added comfort of making the web more user-friendly.

Which to me is the most sensible solution, unless one really likes opening weird email attachments and/or plugging in untrusted devices.

> I don't think that in any of the cases the user would have had a warning to blindly click through.

If the user is already careless enough to connect untrusted devices and/or opening random email attachments, then I have no trust in said user to heed any of the following warnings, as he/she already had to ignore previous best practice warnings to get there in the first place.

[0] http://www.securityweek.com/internets-big-threat-drive-attac...

> If the user is already careless enough to connect untrusted devices and/or opening random email attachments, then I have no trust in said user to heed any of the following warnings, as he/she already had to ignore previous best practice warnings to get there in the first place.

Don't blame the user, when we as computer scientists are too stupid to build secure systems. I expect from an computer system / software that I don't get infected if I plug in a USB stick or open a PDF file. The software devs of operatings systems and applications as well as hardware vendors are to blame. That's it.

If you want to protect the user from email attachments from strangers, block them all, don't base it on a scan that picks up some threats.

> nobody, with the appropriate training, should be opening them in the first place

How many users do you administer again?

A whole lot of 5 users, I realize that in bigger companies it's probably less hassle to just install AV software and "hardblock" undesired behavior.

I imagine that depending on the country you are operating in this might even be a requirement to prevent legal hassle, getting sued for "neglect" if not running AV software and something actually goes wrong but IANAL.

But let's also keep in mind that AV solutions can have the exact opposite effect of what they're supposed to do, from data leakage [0] [1] to straight up remote code executions [2]. Which isn't that surprising, considering that more complexity is usually a bad thing to add to any system, especially if it's as deep-rooted as most AV suits tend to be.

[0] https://www.directdefense.com/harvesting-cb-response-data-le...

[1] https://www.siliconrepublic.com/enterprise/kaspersky-nsa-lea...

[2] https://landave.io/2017/06/avast-antivirus-remote-stack-buff...

I guess that doesn't matter as he said "should".

But yeah, reality is different ...

I recommend to keep the microsoft scanner active (Windows Defender or how they call it by now).

It catches all the common stuff with high accuracy and only some of the exotic stuff gets through (which would probably be a pass for most other scanners too)

No no, it was original antivirus. But it's been a while(I got hesitant doing maintenance), so I can't tell for sure which ones.

I don’t think I know of windows users that use anything other than defender these days on their personal machines, but I (am forced to) use McAfee on my work machine. I suspect that the enterprise tooling is better for third party AV and that’s the only thing other than inertia keeping corporate desktops on those old AV products.

While I think that's a pretty reasonable approach to antivirus these days, Defender was one of the scanners that was doing that. https://arstechnica.com/information-technology/2017/05/windo...

Yeah, anti-virus is a crap idea.

Real security is proactive security (exploit mitigation, sandboxing, correct code, safe languages).

Reactive security kinda sucks. You have to patch known vulnerabilities, sure, but detecting exploits? Ugh. Eww. Do not like.

And indeed users mostly install malware these days, because self spreading (actual "viruses") is hard (we're not in the DOS/Win9x days anymore). So users should be proactive as in not clicking on TotallyNotMalware.exe :)

This is really just not accurate. You can say that AVs are crap, but please don't put all of your eggs in the 'proactive security' basket. At one point that was actually the prevailing attitude, and it just failed absolutely miserably.

Instead, today, we see more companies invest in what's called "incident response". Part of a healthy incidence response program is signature detection - AV plays a role in this.

If you don't have good detection capabilities you're missing a huge portion of what makes an organization secure.

Relying on proactive users is also a recipe for disaster and not a realistic goal at all, nor should it be.

Of course you should be able to respond on security incidents. If you have a security incidents it's often too late and security boundaries are already borken. In the long term we want to have secure systems which are secure by design and by default. We have to invest heavily in incident response because we have have all of these shitty and broken systems. We should already have started heavily on building secure systems and secure languages and should call on everybody to invest time to build these instead of building reactive technologies like AV.

I'm also one of those people.

When it runs code to detect if it is a virus, you have to trust the sandboxing and frankly I've yet to see a sandbox that at some point hasn't been taken control of and exploited (this goes for web browsers too). So it's better to just not execute code than to trust the AV's VM to execute the code without being compromised.

I use programs on a whitelist basis and only update for security patches. This avoids issues like what happened with Transmission.

AV also detect have pitiful detection rates - something like <50% of exploits daily. It's "something" but once you're compromised you're compromised and using an AV just gives a false sense of security.

E: I imagine the downvotes are from my claims of pitiful detection rates or claims that AV is basically security fanfare. Don't take my word for it then.

[0] https://www.theguardian.com/technology/2014/may/06/antivirus...

[1] http://www.blackhat.com/presentations/bh-europe-08/Feng-Xue/...

[2] http://www.blackhat.com/presentations/bh-usa-05/bh-us-05-whe...

[3] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1916708

I also believed this for a long time but recently I stumbled upon this https://www.malwarebytes.com/remediationmap/

Obviously this is an anti malware company PR (but please look at the MS Consumer (defender) failed detection rate). This made me research again the AV landscape and then I found this https://fatsecurity.com/tools/test-results-calculator?compan... A website that seems to aggregate a lot of AV independent tests in an easy to use UI.

MS Defender is not really good now. Even free AV are better and have lesser performance impact than Defender (but obviously come with ads, less privacy, etc.).

(I submitted recently the map to start a discussion about it but did not seem to interest anyone)

AV sucks, Defender sucks more, brain behind the keyboard is king.

Well the definition of a "virus" have expanded greatly over the years.

The original definition was a piece of code that would latch onto binaries, and be spread that way. And those had telltale signatures (a virus may have added a jump at the start of the binary to the end, where a copy of the virus code resided, and then jumping back to the beginning of the actual program code).

And i am not so sure there is much distinction between visiting a JS heavy site and running a program locally these days. After all, we are seeing the likes of Google exposing USB and Bluetooth via JS APIs now.

"because 99% of AV break Windows/applications"

This statement does not at all pan out in my experience. It is very rare for me to come across broken applications or a faulty Windows installation because of anti-virus software.

Your experience seems very limited.

Be a MSP with thousands or tens of thousands of different configurations, all the different antivirus software you can think of, and 20+ years of doing it and you'll see anti-virus break everything you can think of.

AV prevents windows from shutting down. AV prevents windows from booting up. AV prevents windows upgrades. AV prevents windows updates. AV causes blue screens. AV quarantines critical windows system files. AV blocks network resources unexpectedly. AV cripples network performance. AV breaks hardware drivers. AV causes programs to unexpectedly close or crash.

Are just a common list I have to deal with.

I'd suggest you don't know what experience I have.

I've certainly had problems with anti-virus programs, but no more than I have had with any program, operating system, driver or hardware problem really. Mostly they behave, sometimes they misbehave.

I've rarely seen anti-virus causing shutdown problems. Boot problems have almost always been faulty storage devices or operating system corruption. I've fixed more faulty WU instances that are broken because of Microsoft (recent Win7 authcab problem, IE cumulative years ago, etc). than because of anti-virus. The programs that I have come across that are crashing are rarely solved by removing anti-virus software.

If anti-virus was as widely and horrendously crap as some people keep repeating then I'd know about it because I'd be saying the same thing.

Are you saying av isn't widely crap? Because as a senior sysadmin I agree with gp very much. I don't understand why you are defending av at all honestly. It is one of the crappiest software industries in existence!


Anyone remember GStreamer being a Linux vulnerability because it had a 6502 CPU emulator to run... NES music files, and hackers managed to jump outside of the VM.


It makes me wonder how easy it is to break out of a particular anti-virus vendor's VM, in a vulnerability that wouldn't exist if you weren't even running AV (outside Defender) at all.

I don't know what this comment contributes to the conversation besides being vaguely condescending. I didn't ask why an AV product would be re-using 7-zip code.

> The way the stylesheet makes the "rendered" form (especially section headings) resemble markdown source is pretty neat.

As far as I see it's just section headers, but I agree the stylesheet is very nice. The section header prefix (###) could be an anchor to the section though (<a href="#section">).

So I just tried to compile 7-Zip with VS2017 and /DYNAMICBASE. The main binary 7z.dll is 1,569,792 bytes in total, 9344 bytes (0.595%) of which are used by the relocation table. Enabling stack canaries (/GS) gives me a 1,578,496 byte binary (including the relocation table), so another 8704 bytes more.

It would be interesting to have a comparison based on locally built binaries both with and without these features enabled.

Performing the tests on packing the actual 7-Zip source code (as shipped without extras) would be a valid reference suite.

I assume you mean a performance comparison? The runtime performance cost of ASLR on Windows is zero once a binary has been loaded, since the code is relocated at load time.

Stack canaries might cause a slight performance hit, but it is usually below one percent, since it creates only a small cost per function call for a fraction of all functions.

People not caring about load time costs are probably one of the reasons the guy still uses VC6, which starts up practically instantly.

It isn't even per function call in all implementations.

Timeline with a sane date format:

2017-12-29 - Discovery

2017-12-29 - Report

2017-12-29 - MITRE assigned CVE-2017-17969

2018-01-10 - Patched version 7-Zip 18.00 released

7-Zip 18.00 is not really "released" at this time.

18.00 is marked as "beta" in the official website, and 16.04 is still at the top of the list. An average person trying to download 7-Zip right now will most likely choose the vulnerable version.

Beta versions of 7-Zip frequently stay in that status for months, if not years. Between 9.20 and 15.12, 7-Zip produced nothing but beta versions for 5 years. I understand the project moves slowly, but this is not a release model that facilitates quick dissemination of important security patches.

> Between 9.20 and 15.12, 7-Zip produced nothing but beta versions for 5 years.

That's not all that surprising. The software was 10 years old when v9 came out and the major version number is just the year of release. There aren't 5 major releases that never got out of beta. The major version numbers in 7-Zip are misleading this way because the author doesn't really conform to standard conventions. Of course, that is pretty obvious once you use the software for awhile. It still doesn't properly support UAC.

Yes, the way 7-zip releases are done is not ideal and the versioning scheme is just weird. I wish he would make his versioning clearer.

Thanks for pointing this out. I just fixed it.

Does the most recent version on the 7-Zip website, 18.00 beta, contain the patch? It's two weeks old.

7-Zip doesn't appear to contain an auto-updater or an "update me" button.

I find it a little bit concerning that the release note of V18.00 beta does not mention any fix for a security issues. I guess it's included in the "bug fixes"...


Yes, 18.00 beta is the patched version. The current release (non-beta) version is not patched yet. Moreover, the POSIX port of p7zip is not patched yet at all.

While this analysis was done for 7zip, I would imagine that pretty much every packaged implementation on any platform would have these issues, since most people do exactly the same thing - reuse the reference implementation.

Just checked keka on macOS, and it uses the p7zip code.

>If you use Shkarin’s PPMd implementation, I would strongly recommend you to harden it by adding out of bound checks wherever possible, and to make sure the basic model invariants always hold.

Sounds like a fun project.

Why wasn't it found with afl-fuzz?

The RAR PPMd bug can only be triggered if many conditions are satisfied. For example, the RAR archive needs to be mostly correctly structured, and needs to have at least two items that are compressed with the right flags (e.g., RAR version 3, PPMd). Furthermore, the compressed streams need to be constructed such that the bugs are triggered. Hence, I believe the bug is difficult to hit with straightforward coverage-guided fuzzing.

Because AFL does not find every path-execution-based vulnerability?

MS COM C++ style coding for those that are interested and curious about all the S_FALSE and STDMETHODIMP macros.

Where do you learn about these things? This all went over my head.

I found Tobias Klein's _A Bug Hunter's Diary_ to be quite readable and enlightening even if I couldn't follow all of the code.


What do you mean exactly by "these things"?

It may be that the blog post is difficult to understand simply because I have written it poorly...

I read posts like that and marvel at how much people can understand. Well done and thanks for posting.

Happy to have switched to WinRAR years ago. At least they do QA.

What software doesn't have bugs? This was discovered and it was patched in a timely fashion.

I am sure WinRAR has no bugs whatsoever, but would still like to see some evidence to that fact.

...did you actually pay for it? You'd be part of a select group then :D


I did, indeed. Sadly, the subreddit ignored my submission.

Applications are open for YC Summer 2018

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact