
Panopticon: A libre, cross platform disassembler for reverse engineering - adamnemecek
https://panopticon.re/
======
jbaviat
Great work. Therefore that's one more disassembler in the wide. Some of the
great open source ones, reverse oriented, that I have been able to test are:

Metasm: [https://github.com/jjyg/metasm/](https://github.com/jjyg/metasm/)

Radare: [http://radare.org/](http://radare.org/)

Capstone: [http://www.capstone-engine.org/](http://www.capstone-engine.org/)

Capstone is based on LLVM, that you cannot beat in term of architectures and
industrial quality, and has great plugins, which make it kind of my favorite.

Of course, for non-scripting needs, you need decent graphical interfaces, as
provided by these ones:

IDA: [https://www.hex-rays.com/products/ida/](https://www.hex-
rays.com/products/ida/)

Hopper: [http://www.hopperapp.com/](http://www.hopperapp.com/)

IDA is clearly the best, but Hopper is a fair choice if you need it and you
can't afford IDA license for personal use.

[Edit: add forgotten IDA and Hopper...]

~~~
kayamon
FYI you most definitely can beat LLVM in terms of quality. Its diassembler was
designed originally only to support the instructions that the compiler could
generate. While they've since filled out most of the instruction set, there
are still a whole set of subtleties it can't deal with well, simply because
the compiler part of it was never designed to output them. (e.g. see segment
prefixes)

~~~
jbaviat
It depends on what you are looking to achieve.

If you take a look to e.g. McSema
[https://github.com/trailofbits/mcsema](https://github.com/trailofbits/mcsema),
these guys have been able to use LLVM to transform assembly back to LLVM
intermediate representation (IR). The IR is an abstract language used by LLVM
during compilation to optimize the generated code. Being abstract basically
allows to use optimizers across all LLVM supported architectures.

In these terms, I think LLVM has a clear advantage over competitors.

The following blog post give a nice overview of the McSema achievements:
[http://blog.trailofbits.com/2014/06/23/a-preview-of-
mcsema/](http://blog.trailofbits.com/2014/06/23/a-preview-of-mcsema/)

~~~
munin
hi, I'm one of the mcsema authors and the original author who was responsible
for mcsema using the LLVM instruction decoder and it was a mistake. if I had a
choice between going back in time and assassinating Hitler, or convincing
myself to use XED instead of mcinst...

~~~
jbaviat
Hi Munin,

What you did with McSema was really impressive and I’m glad discussing with
one of the authors! I get the point you regret using LLVM, it seemed to me to
be the best choice since LLVM has wide instruction semantics implementations,
that more basic tools such as Xed do not offer. If I get it correctly… you
would rather re-implement such semantic yourself? Or have you got any other
tool / idea I’m not aware of?

~~~
munin
I don't regret using LLVM at all. I regret using the instruction decoding
features in LLVM. if I could do it again (and someone younger, smarter, and
better looking than me is) I would combine the LLVM IR with the XED
instruction decoder. we would still emit the semantics of the instructions as
LLVM, but we would use XED to figure out which instruction we were decoding.

------
joshuamcginnis
Let's assume you have reversed engineered some kind of boolean check and you
now want to patch it to always return true or false. What does that process
look like at a high level?

~~~
tomf64
You insert the equivalent of "mov eax, 0x1; ret" (or 0x0) in x86 for whatever
architecture you're using as the first instructions of the function.

~~~
joshuamcginnis
I mean even more high level than this. If I open a binary, can I just write
the new machine code to it directly and not be concerned with recompiling?

~~~
SCHiM
As long as all the instructions are the same size (or smaller padded with no-
operation instructions) then yes. If, however, you do change the size of the
application all relocation deltas need to be changed, and all relative jumps
and calls need to be recalculated.

~~~
georgiecasey
i was trying to edit one byte in an ELF, no change in file size and it kept
crashing. i read that each section of code is hashed and obviously my byte
edit changed the hash. i was pretty out of my depth tbh.

~~~
ndesaulniers
Sometimes code will add additional checks such as hashes to verify that parts
of the text section haven't been modified. Should be able to remove those
checks, too. Just have to track them all down.

------
simplexion
I like the About me:

 _Hey I 'm Kai Michaelis, I'm in an IT-Security Masters program in Bochum,
Germany and work part-time for people who use the term Cybersecurity
unironically._

------
dikaiosune
Requisite Rust fanboy'ing: GitHub stats say 90% Rust code.

~~~
steveklabnik
The author posted on /r/rust with some details:
[https://www.reddit.com/r/rust/comments/4ihtfa/panopticon_a_l...](https://www.reddit.com/r/rust/comments/4ihtfa/panopticon_a_libre_crossplatform_disassembler/d2yj305)

A particularly interesting comment:

    
    
      > Also I found iterator invalidation bugs simply by translating
      > C++ to Rust, thanks Borrow Checker!

~~~
pcwalton
Yeah, it's really interesting how many iterator invalidation bugs you find in
real C++ codebases. It's so easy to think that they don't exist until you have
a tool (or a human) actively searching for them…

~~~
dman
Isnt this whole category of bugs easily avoided by using

1\. _SECURE_SCL on windows

2\. _GLIBCXX_DEBUG on glibc based toolchains

~~~
Someone
Those do not remove any bugs; it just changes how they are handled (ignore and
hope for the best vs print diagnostic and terminate program, and the latter
not in all cases)

~~~
dman
Assuming that tests exist, and have good coverage, wont the checked iterator
implementations mentioned above catch most issues with invalid iterators?

~~~
pcwalton
I don't know of any large, widely used, security critical, heavily scrutinized
C++ program that uses those flags in production. I also don't know of any such
C++ program that hasn't had game-over use after free bugs. So the answer to
your question is somewhere ranging from "we don't know, but unlikely" to "no".

------
martanne
Does somebody know how it compares to radare2?

~~~
miles
From their GitHub repository[1]:

"Panopticon is under heavy development and its feature set still very basic.
It's not yet able replace programs like radare2 and IDA Pro. We are working on
it, check back regularly!"

[1] [https://github.com/das-
labor/panopticon/blob/master/doc/feat...](https://github.com/das-
labor/panopticon/blob/master/doc/feats.rst)

------
umanwizard
Anyone know how this compares to IDA Pro ?

~~~
busterarm
If you can afford an IDA Pro license and cost of plugins, does it matter?

~~~
umanwizard
Yeah, it matters. If this is better, I'll use this; if IDA is better, I'll use
IDA. I'm not 100% sure I understand your comment.

Btw, most companies can afford an IDA license. It's pretty cheap compared to
the salary of a developer, at least in the coastal US.

~~~
busterarm
If your job can justify it, great, but mine can't. I want it for personal use
and I can't justify it.

~~~
q3k
If you have more time than money, here's a possible option:

    
    
      - Buy IDA Pro using a credit card
      - Find bug or two, submit to bug bounty [1]
      - Pay back credit using bug bounty money
    

[1] - [https://www.hex-rays.com/bugbounty.shtml](https://www.hex-
rays.com/bugbounty.shtml)

~~~
lucb1e
If life were so simple, everyone would be doing it don't you think?

~~~
dfc
My friend's grandfather was an economist. When he was a little boy he saw a
$20 bill on the sidewalk on a walk with with his grandfather. He pointed to it
and said "Grandpa look someone dropped a twenty dollar bill! We will be rich!"
His grandfather dismissed his enthusiasm and explained that if there was free
money on the ground somebody would have picked it up by now.

------
aantix
On a side note; the visual callgraph is interesting.. It'd be interesting for
other IDEs to implement something similar with a historical listing of
input/output values for the corresponding displayed methods.

~~~
tptacek
What's the visual callgraph? Is it something different than the CFG graph in
the screenshot? I feel like most disassemblers do that.

~~~
aantix
Never worked with a disassembler. Didn't realize that this is commonplace.
Could be beneficial for other languages as well..

------
bogomipz
The project states:

"Panopticon is a disassembler that understands the semantics of opcodes. This
way it's able to help the user by discovering and displaying invariants that
would have to be discovered "by hand" in traditional disassemblers."

Doesn't every disassembler have to understand the opcode semantics in order to
disassemble and make sense of them or am I misinterpreting that statement?

Also can anyone explain what the "invariants" are? When I hear the word I can
only think of loop invariants and I'm guessing that is not what the author
means here.

~~~
hcs
You say "in order to disassemble and make sense of them", but the "sense" that
a disassembler makes can just be the direct conversion from machine code to
equivalent assembley. Machine code is just another syntax, directly converting
that into assembly can happen instruction-by-instruction.

Making more sensible assembly that a human would write is another thing, if
you want to follow jumps to figure out where instructions are and regenerate
labels, then that's one step towards semantics.

I don't know what kinds of invariants panopticon is looking at, but there
seems to be code for static analysis using abstract interpretation[1]. This
could do the data-flow analysis, determine types and recognize persistent
variables.

[1]
[https://en.wikipedia.org/wiki/Abstract_interpretation](https://en.wikipedia.org/wiki/Abstract_interpretation)

~~~
bogomipz
Now I see. Thanks for the explanation. this is pretty neat then. Seems like a
great learning tool

------
munin
I noticed that this only supports AVR. what is your plan to support x86? would
you think about linking against some external code that provides semantics for
x86?

~~~
flanfly
The README is a bit out of date. It support MOS 6502 and parts of x86_64 too.
I currently writing the lifting code that translates x86 opcodes to a simpler
intermediate language:
[https://github.com/flanfly/panopticon/blob/feature/rreil/lib...](https://github.com/flanfly/panopticon/blob/feature/rreil/lib/src/amd64/semantic.rs)

~~~
rvense
How do you find adding new architectures to it? I've been meaning to add some
architectures I want to Radare for a while now, but it's a bit daunting due to
lack of docs and the build system is argh. And I've also been wanting to learn
Rust, so this seems like an obvious fit for me...

~~~
flanfly
Panopticon includes a simple disassembler framework. The basic idea is that
you provide opcode patterns akin to regexp and write a function that emits
mnemonics and code describing opcode semantics. There isn't much
documentation, but you can take a look at the AVR and MOS 6502 disassemblers
in lib/src/mos resp. lib/src/avr.

------
chris_wot
What is the best guide to learning how to reverse engineer programs?

~~~
DarkLinkXXXX
I suggest "Reverse Engineering For Beginners"[1] and "Hacking the Xbox"[2].

[1]: [http://beginners.re](http://beginners.re) [2]:
[http://libgen.io/book/index.php?md5=64E13DD5E86FE633A48C2261...](http://libgen.io/book/index.php?md5=64E13DD5E86FE633A48C2261CBF505E2)

------
shabbyrobe
Slightly OT, but are there any good libre tools for reverse engineering binary
file formats rather than executables?

~~~
pfalcon
Start with binwalk.

------
teacup50
GPL means I can't use it as a library unless I also GPL.

No thanks.

~~~
chris_wot
You could ask them to make an exception:

[http://www.gnu.org/licenses/gpl-
faq.en.html#LinkingOverContr...](http://www.gnu.org/licenses/gpl-
faq.en.html#LinkingOverControlledInterface)

------
partycoder
I like "ht editor".

------
searine
>Panopticon

Good lord what a cliche name.

~~~
dang
Please don't post snarky dismissals. If you think there's an important point
about the name, you're welcome to make it substantively.

