- Ghidra is basically the first real competitor to IDA Pro, the extremely expensive and often pirated state-of-the-art software for reverse engineering. Nothing else has come close to IDA Pro.
- Ghidra is open-source, IDA Pro is not.
- Ghidra has a lot of really cool features that IDA Pro doesn't, such as decompiling binaries to pseudo-C code.
- It's also collaborative, which is interesting because multiple people can reverse engineer the same binary at the same time -- something IDA only got VERY recently.
Context: in IDA, certain changes you make can inadvertently wipe out a lot of work - for example, undefining a function (U) can erase all your annotations in a single keystroke; defining a return type incorrectly can completely mess up callers, sometimes to the point where they won't even decompile properly; making a typo to an array size argument can obliterate the stack and every variable annotation you made on it, etc. etc. Many of these require much more work to undo than simply reverting the change you made. So a functioning undo is a big deal
Some more comparisons:
- Ghidra's type system is nice, and in some ways nicer than IDA's. Semi-automatic struct inference rocks, and it comes with a big type library.
- Ghidra will decompile code from a dozen different architectures. IDA will only do x86, x64, ARM and AArch64 (and you pay for all of those separately). In theory it could decompile a custom architecture if you implement your disassembler backend thoroughly enough.
- Ghidra's UI is marginally worse than IDA because it's implemented in Java Swing (compared with IDA's Qt).
- Ghidra and IDA both use Python for scripting. However, Ghidra's Python is actually Jython, which gives it access to the entire state of the system (minus the decompiler, which is native code - but you can interact with all the code that drives the decompiler). This is really big - the API surface of the entirety of Ghidra is pretty massive so the scripting opportunities are similarly exciting.
- Ghidra has a (mostly functional) patching interface which understands assembly. IDA Pro, despite costing many thousands of dollars, gets confused when you try to assemble something as basic as "mov rdi, rdx" in 64-bit code. (There's an outstanding bug which breaks ELF files - but being open-source, I'm sure it will be fixed soon)
I'd like to hear the rest of this story, I was considering buying IDA Pro (though these days I'm having a lot of fun adding M68k support to Avast's Retdec)
I grew up using, ah, other methods of satisfying my need for an interactive debugger and those methods continued to be viable after giving hex-rays $1100 and getting flaked, so I wasn't materially impacted by the flakery, even though I "should" have been. Had I been materially impacted rather than merely angered and frustrated, I probably would have tried other escalation paths -- phone, twitter, maybe even snail mail -- and I suspect I eventually could have gotten through, and my expectation absent evidence to the contrary is that if I were to have gotten through they would have helped me out. My takeaway would be "their self-service site is rubbish and they force you to use a company email address (as opposed to your gmail) and then their support email servers sometimes silently drop messages from your company email address, or something," not "they're crooks." On a professional app sold at a professional price, though, that's still not a good thing, and it informed my software choices going forward.
Since I can not find your nickname in our database, I can not say more.
Just searched for your username in our chat and our email and don't see anything so I assume you've got a different email?
I do so love the shell code compiler of Binary Ninja, though. It works very well and has definitely saved me a lot of time.
As soon as the code is up I hope to submit a PR, which will be pretty easy since I already have the diff.
So you're excluding deploying/maintaining Open Source Software as a service. That basically excludes how Open Source is supposed to get monetized.
Is it? This means that incentives would be wrong, because then developers would be incentivized to produce difficult to use (but useful!) software - with various poorly documented features, multiple ways to solve the same problems, poor and inconsistent UX... Oh wait... </s>
I am sorry Redis labs got the heat for their license change and I really hope some solution crops up. I appreciate opensource, but I am getting tired of poorly implemented systems, just because there is no incentive to do it differently.
I think the solution lies in "free-to-use (but not free-to-sell), source available" licenses. I haven't seen one that would convince me yet, but I am certain that with big-tech companies behaving like they do, more and more developers will think twice before giving away their work for free just so others can make billions off it (and take away income from the very companies providing bread and butter to FOSS developers - coughAWScough).
Keybase, Inc. https://en.wikipedia.org/wiki/Keybase
The money always comes from somewhere...
The answer is, sell support.
Note that RedHat, often quoted as success story, is really a services company that just happens to write an occasional piece of software, and support them when their customers need it.
They do? Maybe. But they are walking a thin line between their short term (make it complex so we can, you know, sell support) and long term (make it simple enough that people don't jump ship) commercial interests. This is the reason I call it "unnecessarily" complex - it is in the interest of companies who sell support services that the product is not as easy to use as it could be.
Do we really need this? I think not, and I actually prefer what Redis Labs, MariaDB and others have been doing with the licenses for their modules. Sure, Business Source license and similar are not open-source (as in "freedom to take the product you have built and sell services on top of it, driving you out of similar business as collateral damage"), but at least they provide developers with the incentive to produce easy to use software, and not just because they feel like it, but because they can actually earn their living from it.
There might be some exceptions that "make it work", but this is in spite of just selling services on top of their product, not because of it. The cards are stacked against them - it is much easier (and profitable) to take other persons' product and build on it that it is to build your own.
In my experience, selling support is an acceptable answer only in the eyes of would-be competitors. Otherwise it is just plain wrong. </rant>
Check restrictions in https://docs.binary.ninja/about/license/index.html
> Restrictions. Subject to applicable copyright, trade secret and other laws, you are permitted under this License to reverse engineer or de-compile the Software but you may not alter, duplicate, modify, rent, lease, loan, sublicense, create derivative works from or provide others with the Software in whole or part, or transmit or communicate any of the Software over a network in order to share it with others.
That's pretty much the only reason one would reverse engineer it, in this context - and it's somewhat misleading to suggest otherwise.
I think it's probably pretty unique right now in that it's under an OSS license without all the source available.
To be fair, IDA Pro has a decompiler plugin to do this.
IDA has always had a weirdly low price point given the bill rates of people who use it, and it's interesting to see that price being competed all the way down to free.
In the past, the same could have been said of compilers and even web server and mail server software.
> many [most?] fields where people routinely decompile stuff are very highly compensated.
If it's more freely available, and more people have experience with it, then the compensation might go down as the supply of people with this experience goes up. I'm not sure using salary as a justification of what a tool price should be makes a whole lot of sense. to me, it just sounds like an inefficient market because there's not enough competition (justification on the ground that it does much more than any competitor and thus can command a premium does though).
I'm not arguing that a capable free alternative is a bad thing. I think there's an industry business case study in what Hex-Rays could have done to keep this from happening, though.
Is the fact that Hex-Rays is Russian one of the reasons why Ghidra exists? (Honest question.) If so, is there anything they could have done differently?
Sounds like every developer working on an open source stack.
Market forces are often reined in, eg by human rights, those things aren't part of the market operating they're mitigations of the damaging effects. In this case it's an external force (gov action), not the market, that has created the availability of the product.
Large player in widget market has low marginal cost and deep pockets, sells widgets at marginal cost its competitors can't match. Also totally normal market practice.
Competitors exit market or are relegated to minor market share, leaving de facto sole survivor. Also totally normal market practice.
We could be talking about chrome just as easily as IDA pro here.
I'm not sure the exact same thing could have been said which to me seems like a testament to how complicated software pricing can be. Web servers never really sold†, platform vendors eventually figured out it's better for them for compilers to be free (non-platform vendors still sell compilers), etc.
† back in the 90s, Netscape used to pester web companies to make their Apache installs lie that they are Netscape web servers.
Perhaps they would benefit from some type of "free/cheap for noncommercial use" license?
That's not going to prevent many people from taking the five finger discount I'm sure, since they'd rather have as many of the features as they can, but at least nobody can say HexRays isn't trying.
There are a lot of good and interesting games that were made in the DOS era for PCs which used the DOS4GW DOS extender, and their binaries come in the OS/2 executable format (LE/LX) which is unsupported in IDA's free version. A lot of good and interesting games also happen to run on game consoles which use non-x86 processors.
Ghidra probably won't have plugins to support all of these weird old legacy formats and CPUs which the full IDA package does for a while, but hopefully it'll get there eventually. If it doesn't seem too difficult, I might even try creating a LE loader for it myself.
It feels like to stay in business with software like this, it has to be lucrative, but not too lucrative, or else FAANG companies (or occasionally governments, like in this case) will either gobble up or kill the market.
That is, Hex-Rays do not want to have any business relationship with the proverbial would-be teenage hackers.
Outside of that, Hex-Rays is a small business which has probably around than 1mln eur/year of turnover and they do not want to grow it much more. It was a Basecamp-style business long before DHH made the concept of anti-growth popular.
When I went to renew my support, they grilled me again. This was just a few weeks ago. I gave up and figured Ghidra was just around the corner. Looking forward to trying it.
I emailed them and told them a) I didn't appreciate being treated like a criminal (won't get into the specifics, but one set of answers led to another set of questions, but I'm a consultant with my own company, website, physical address, company history, blog posts, etc. -- I work in security / reverse engineering of electronic devices)
I also told them I've never had to work so hard to give someone my money. Finally I gave up. Let the market speak.
What's been significantly improved in IDA over the last 10-15 years? Certainly not the x86 decompiler, which costs something like five times as much as IDA itself. The interface is still super-clunky and missing functionality like keyboard shortcuts for frequently-used functions.
I'm ecstatic that there's finally a realistic alternative.
IDA comes with amazing technical support. I've emailed complaints, then gotten a freshly-compiled build with a bug fix within a couple days. Funds are thus improving quality in ways that customers request.
In fact, the essence of decompilation is a NP-Complete problem: Graph Isomorphism.
So far, our decompilers are just greedy scheme to approximate the original expressions as best as possible by treating each instruction as a tree then as a graph, but still even a single assignment could cause the entire outcome of the code to change a lot, let alone to correctly recognizing heavily optimized procedures.
Edit: Wiki said it is NP-Complete but I was pretty sketchy about it. I think the better wording should be "at least NP"
So as one point of comparison you might look at the tools of software engineers, which are essentially all free today.
To get to hex-rays having a reasonable price you probably have to look at jobs like pipe welding where the equipment is expensive and the hourly high, but the comparison is much less direct.
There are lots of apps that make lots of assumptions about how filesystems behave, generally based on the local filesystem and maybe on one popular networked filesystem for the platform (NFS, SMB, AFP).
If one of those assumptions is violated, applications can crash or refuse to interact with you. Some just refuse to write to any networked filesystem. Some run only on whitelisted filesystems. Some will hit an error due to an unsupported operation on your filesystem, fall back to some ancient code path using long-since deprecated Carbon APIs that only work properly on 32 bit systems, and so truncate all of your data to 2 GB.
Problems like the latter are really helped by being able to do some reverse engineering of the application to figure out why the heck it just writes out the first 2 GB of the file.
Because this isn't our bread and butter but only an occasional tool in our toolbox, the licensing on IDA Pro can be rather frustrating. We use it only once every couple of years to debug some kind of compatibility issue like this, and so we usually have to dig around to figure out if we still have valid licenses, deactivate systems that we're no longer using, and so on.
They mostly hire their researchers straight out of college if they have high C proficiency and train them internally to use IDA Pro.
I know my comment isn't exactly what you asked, but I hope it clears some light.
Also partially OT, just wanted to say that I was a sort of college-roommate with one of their present-day senior security researchers in the early 2000s and to this day I remember that person as one of the most code-obsessed persons I have ever met, and I say that in a good way.
He was looking at almost every program running on our room's computer (yes, we only had one computer in our room of 4 or 5, no laptops) as a thing to be "broken apart"/analyzed/made sense of, he had a state of mind and a way of looking at things when it came to computers that I've never met since then at any other computer programmers (I've mostly met desktop, backend and front-end programmers, I'm a data-obsessed person myself). I realized in the meantime that in order to enter this "computer security" field and especially in order to be good at it you need to have a different set of skills and especially a different way of looking at things compared to other computer programmers.
All bridges should be free, The marginal cost of one more user is effectively zero.
Good comparison might be Synopsys VCS. Prices are not published but I believe they are over $30k/cpu/year and for larger designs you really want a big sim server.
This is shocking, because, in an E-mail exchange a few years ago, Ilfak wrote to me:
> [...] we at hex-rays do not have any ideas how to implement dynamic database synchronization, so it is unlikely that others will come up with a good solution.
I think Binary Ninja's enterprise version might involve clients connecting to a server that maintains the database. It would be more like Google Docs if that is the case. Actions in the GUI would request atomic transactions on the server, then display the current state.
And the decompiler is dope for IDA
If you wanna run this thing, you should probably build it from source yourself (don't trust the binaries) and even then run it in a pretty well sandboxed virtual machine. I would not be surprised at all if the NSA left some surprises in that thing.
Huh. I would be extremely surprised if the NSA were to include some kind of malicious or pseudo-malicious easter egg in the open source RE toolkit they're releasing. How dumb would they have to be to pull a move like that, and for what? The self interest just doesn't line up.
There are those who suspect Heartbleed came about this way.
- Many people don't use SELinux, especially on distros like Arch, Gentoo etc. where it (luckily) doesn't come as part of the package, SELinux is far from universal.
i would say these both tools ,as well as r2 have their own merits and weak points, and it would be good not to exclude one and take the other as better, but to have them compliment eachother in your arsenal.
in the end if you want quality, then manual work is always better than these opinionated tools, and sometimes that is required, so really the tools offer different perspectives / opinions of the same thing, and that is valuable in any case. when you run into the limit of 1 tool ,another might just fill that gap.
What do you think of BAP?
What is a really great contribution of Ghidra, to my opinion, is the detailed specification of all supported ISA in Sleigh (their terse and concrete ISA specification language). Ghidra ships with about ~200kLOC of instruction descriptions and this is the most valuable contribution to the community. We're planning to support Sleigh in the nearest future, and I believe that Sleigh might become a standard de facto for instruction semantics specification.
If this is expensive to you, then it’s not for you. This is for people who are making real money with these tools, not hobbyists dicking around.
That's an odd perspective. Imagine if this type of sentiment were applied to paint brushes. There is a lot of useful work that is not economically viable per se, and to discount that and to be pejorative feels wrong.
Not necessarily nice, but it makes sense.
I don’t see how the perspective is odd. Having tools like Core Impact and the knowledge of how to use them well can propel you to a six figure income easily. On top of that these tools are also business expenses you can use for tax write offs.
They are certainly worth the investment. The only people who see the price as steep are those who cannot see any viable way to make a decent ROI off them.
Then think about the fact that some people are poor and can't float thousands of dollars long enough to learn and get employed with tools like this.
1) No one is entitled to a career in cybersecurity or reverse engineering, no matter how poor or sad your origin story is.
2) There are always lucrative opportunities in this world that are out of reach by people who lack some resource. In this case, it's money, but it could easily just have been something like popularity, beauty, connections, location, or even plain old brains.
I always wanted to be popular and loved by many, but I came to accept long ago that it just wasn't going to happen. I'm an introvert, I keep to myself a lot, don't get much pleasure from social outings, and at the end of the day people just don't give a fuck about weird people like that. So I just try to enjoy the gifts I do have and the things that come naturally to me. We all have to accept the realities of our lives at some point, even the poor.
It's true that some pre-existing conditions can limit what options people have, but it doesn't apply to everything.
It's important to be discerning about when this effect applies and when it needn't, and work to open more opportunities to more people wherever possible.
That said I just renewed my license so I have to get some use out of it, but Ghidra does seem like it could be the real deal. Honestly, I never really expected any free/FOSS alternative to IDA to ever exist at this point, so the possibility is tantalizing.
When I use IDA, almost all of my actual work in the tool itself is very "boring" RE stuff, because it does its job. I am not constantly fighting with it to get basic things analyzed propertly, or fighting a lack of supported features that prevent it from opening something, or a bad analysis engine that misses 80% of things I later reverse by hand. You could comparatively stitch something together with the tools in Radare to patch over this for the cases it doesn't handle. You might even call those "edge cases", but reverse engineering is 90% edge cases and 10% easy stuff. I'll already be done by then.
I should also be clear that part of the issue is that reverse engineering is a money game, one where money is easy to come by if you have the clients -- and as a result and a lot of the developers of those tools have more money/labor available than the Radare developers. That also means people who need this can simply throw money at a problem, like an expensive IDA license, and move on. That doesn't mean Radare developers are incompetent. If you gave them a lot of money -- like, enough to fund 5-10 core developers for a couple years -- Radare would dramatically improve extremely quickly, I'm sure. (This is one of the reasons why I suspected a true competitor to IDA would never come around as FOSS -- it takes a shitload of money to do that, and it's also something you can make a shitload of money from.)
But I'll say this: if you put me into a situation where I had to reverse something, I'd pay for an IDA license 10/10 times even if every Radare developer was at my command, and I'd probably still get it done faster (most RE tools I know of lack even the most basic, fundamental features IDA has had for years -- such as FLIRT -- that can dramatically improve reversing speed.)
If we switch, it will be to Ghidra or to Binary Ninja.
Bonus points available for:
* "the source control is ZIP files on a network share"
* "yeah we use forced squash commits on everything to keep the Git history nice and linear"
* "it was designed by a contractor who is now uncontactable"
That includes malware analysis, vulnerability research, and emulator development.
Too often companies pay 6 digits for a feature that some supplier rips directly from an open source on the Internet (often GPL) and then sells as his own.
The concrete difference between the two is that vulnerability research is mostly focused on the technical security aspects. Eg. is there a buffer overflow here yes or no? From an efficiency perspective it makes no sense to hide the source code or even credentials from the pentesters performing this research.
An attack simulation is more holistic in nature, the question becomes "can your security team detect when we exploit this buffer overflow?". The blue team and the red team do not share details, and to give the blue team a proper exercise they are often not even informed. To do a proper red team exercise the scope must be very broad. Both technical controls as well as procedural operations are in scope. If you call application/network security research a red team exercise I think you're doing it wrong.
So a red team, in the sense of the word that I specified, does not have access source code, and most definitively sometimes needs to reverse engineer binaries.
I’m sure everything performs well on ELFs built with -O0 -g but in most real world usage, Ida is queen.
Since everything is open source, if ghidra is as good as people say it is, I’m sure people will make better guis for it (and tui) in no time.
Tax paid competition for existing commercial products. Isn't that considered evil/wrong by pure capitalists?
The most recent liberation of useful taxpayer funded software that I can think of was over ten years ago, when NIST released NFIS2 - the fingerprint software that the FBI relied on. They of course had to be crappy about it and wrap it in export controls that limited its utility, but it was interesting to see all the work that internal development had done - very polished, with man pages going back to '97. Ah the memories: software classified as munitions, the clipper chip...
Sure, it might be a great tool for free, but who knows what else might be hidden in there?
At worst they will know how to mask their real malware from analysis with their own tools.
anyone else virtualizing three layers deep to get to this?
There's zero chance there's some secret trojan, because the people who are interested in this type of software are the exact people who would be able to find it.
* As long as you have citizenship... which is the minority.
(And if not I'm sure the community will reconstitute it)
The decompiler for instance is a precompiled binary (elf64 file on linux) wrapper in some java code. The C/C++/? code is not provided.
I wonder if it will make it's way to FLARE.
2. supporting classified proprietary architectures (think missile chips or something)
3. The intermediate representation (architecture independent representation of code) can be integrated in to many other classified tools. Maybe for automated analysis for example.
But it's also possible this is just sort of a labor of love type thing.
From that perspective, the ideal is what the NSA ended up with, a codebase whose development is fully in-house. Notably, though, second best would be to just have access to the source code of an existing tool, so you can at least make your own patches if necessary, even if you’re not in control of the codebase’s overall direction. Did the NSA ever seek that in IDA’s case, and could they have obtained it if they did? I don’t know the answer to either question… but source access certainly isn’t offered to typical customers. In general I’m surprised that “paid + source access for customers” isn’t a more popular model of software development.
From my perspective, which admittedly is very different from the NSA’s, I was never very interested in low-cost IDA competitors like Hopper or Binary Ninja, but I’m very excited about Ghidra. Why? Partly because it’s a more full-fledged competitor in terms of feature set, I admit – but the competitors I mentioned are bound to narrow the gap over time. Partly because of cost: I myself am at a point where I could justify the $600/y for Binary Ninja’s commercial edition, or even the order-of-magnitude-higher cost of the Hex-Rays decompilers, without wincing too badly. but I believe that reverse engineering should be accessible to beginners and amateurs. (Piracy is a partial solution, including in IDA’s case, but some people don’t like to do that.).
But the main reason I’m excited about Ghidra is that I have the source code. As a concrete example, I’ve spent a good amount of time reverse engineering software for the Nintendo Wii and Wii U. Both consoles have a main CPU based on the PowerPC architecture, but with a custom ISA extension for an extremely barebones version of SIMD. Well, both Hex-Rays and Ghidra support PowerPC decompilation (although that’s a relatively recent development), but unsurprisingly, neither of them have full support for that ISA extension. IDA actually does have built-in support for disassembling it, but AFAIK not for decompiling; Ghidra doesn’t seem to support it at all (but I may just need to configure it properly). What can I do? Well, in practice, nothing, because I don’t care about the Wii U anymore. But if Ghidra had been released a few years ago, I’m pretty sure I would have gone and implemented support for the extension myself; I haven’t looked at Ghidra’s source yet, but since it already supports other vector ISAs, it probably wouldn’t be that hard. With IDA, I was stuck. The SDK supports adding custom instruction sets for disassembly, but the decompiler SDK is so limited that supporting them there is either impossible or at least would be a huge hack.
And that’s just one of many customizations I‘ve wanted over the years. Some of them are probably easier said than implemented, but at least now I can put that to the test!
If you need some very specific small functionality by next monday for $1million, then that development can be arranged.
However, NSA could also reasonably want that their targets (who have extensive capabilities of their own, likely including insiders in various companies) can't find out that NSA needs that very specific small functionality by next monday. They may not care if that functionality becomes available to the general public sometime in the next year (preferably in a more general manner that covers the reasonable/common usecases instead of just the one NSA had at the moment), but leaking the information that you needed (and thus probably used) X at time Y often isn't acceptable.
At the very least, they'd need every single employee who works on that feature or can see that this feature was developed to be vetted by them i.e. to have a security clearance; and that also requires the company to have the appropriate processes and infrastructure for separate, secret codebases and builds that can't be seen by uncleared people. And that is something many companies can't or don't want to provide.
Microsoft has a "shared source" agreement with governments.
Viva la open-source revolution
Ever tried to use IDA Pro on the same project with a co-worker...at the same time?
IDA Pro still doesn't support collaboration, although there are very broken hacks that attempt to add it. Binary Ninja supports collaboration if you buy the enterprise edition.