
Hexyl: A command-line hex viewer - okanesen
https://github.com/sharkdp/hexyl
======
Twisol
I admit, my first thought on seeing the title was "What's wrong with xxd?"

This is nice! I like the colorization a lot -- it's like with source code, you
don't realize how much you rely on the colors until you get a new computer and
need to download syntax definitions for your editor again. Only feature I
_really_ think is missing is line numbers.

Nicely done!

~~~
hwj
What are line numbers in context of a binary?

~~~
twtw
Addresses.

~~~
archgoon
File offsets; not addresses :) I assume the tool isn't reading ELF format
files and working out the memory address.

------
twic
One ludicrous nitpick: the logo doesn't show anything hexyl, that's
1,2,3,5-tetramethylbenzene. If it was a functional group, it might be
3,4,5-trimethylbenzyl or something like that.

Hexyl would look like:

    
    
      R
       \/\/\/

~~~
sharkdp
The logo shows a small part of this molecule:
[https://en.wikipedia.org/wiki/Hexanitrodiphenylamine](https://en.wikipedia.org/wiki/Hexanitrodiphenylamine)

.. which is apparently also called "Hexyl". Granted, I was just looking for a
short word that starts with "Hex" and I was never good at organic chemistry
:-)

~~~
teacpde
Many of your projects have cool logos, how do you make them?

~~~
sharkdp
Thank you. Not all of them are by me, but I typically just use Inkscape.

------
hwj
While reading this my hope was it not being written in JavaScript. I'm not
disappointed ;)

~~~
pantalaimon
It's comparably slow though :p

    
    
        xxd `which hexyl` > /dev/null  0.12s user 0.09s system 99% cpu 0.219 total
        hexdump `which hexyl` > /dev/null  0.19s user 0.05s system 91% cpu 0.256 total
        hexyl `which hexyl` > /dev/null  1.69s user 0.39s system 95% cpu 2.175 total

~~~
ByronBates
The same author also wrote hyperfine, a tool to compare performance of various
program runs.

    
    
        hyperfine './target/release/hexyl ./target/release/hexyl' 'xxd ./target/release/hexyl' 'hexdump ./target/release/hexyl'
        Benchmark #1: ./target/release/hexyl ./target/release/hexyl
          Time (mean ± σ):      1.529 s ±  0.028 s    [User: 1.476 s, System: 0.050 s]
          Range (min … max):    1.491 s …  1.581 s    10 runs
    
        Benchmark #2: xxd ./target/release/hexyl
          Time (mean ± σ):      70.5 ms ±   0.5 ms    [User: 68.0 ms, System: 1.2 ms]
          Range (min … max):    69.5 ms …  72.3 ms    41 runs
    
        Benchmark #3: hexdump ./target/release/hexyl
          Time (mean ± σ):     262.4 ms ±   2.8 ms    [User: 260.1 ms, System: 1.5 ms]
          Range (min … max):   259.8 ms … 268.8 ms    11 runs
    
        Summary
          'xxd ./target/release/hexyl' ran
            3.72 ± 0.05 times faster than 'hexdump ./target/release/hexyl'
           21.70 ± 0.43 times faster than './target/release/hexyl ./target/release/hexyl'
    

Currently hexyl seems nearly 22x slower than xxd.

~~~
sharkdp
... and I have already used hyperfine to benchmark hexyl as well :-)

Yes, it's a shame. But I don't think there is too much we can do about it. We
have to print much more to the console due to the ANSI escape codes and we
also have to do some conditional checks ON EACH BYTE in order to colorize them
correctly. Surely there are some ways to speed everything up a little bit, but
in the end I don't think its a real issue. Nobody is going to look at 1MB
dumps in a console hex viewer (that's 60,000 lines of output!) without
restricting it to some region. And if somebody really wants to, he can
probably spare 1.5 seconds to wait for the output :-)

~~~
userbinator
_We have to print much more to the console due to the ANSI escape codes and we
also have to do some conditional checks ON EACH BYTE in order to colorize them
correctly._

A few extra comparisons and output for each byte shouldn't be that much
slower; fortunately the function of this program is extremely well-defined, so
we can calculate some estimates. Assuming a billion instructions per second,
taking ~1.5s to hexdump ~1 million bytes means each byte is consuming ~1500
instructions to process. In reality the time above is probably on a faster
CPU, so that number maybe 2-3x more. That is a shockingly high number just to
split a byte into two nybbles (expected to be 1-3 instructions), convert the
nybbles into ASCII (~3 instructions), and decide on the colour (let's be
_very_ generous and say ~100 instructions.)

The fact that the binary itself is >1MB is also rather surprising, especially
given that the source (not familiar with Rust, but still understandable) seems
quite small and straightforward.

~~~
steveklabnik
Rust binaries can be large because unlike C, the standard library is
statically linked, as well as jemalloc. Jemalloc will no longer be the default
as of the next release, so that will shave off ~300k...

~~~
eridius
What's replacing Jemalloc?

~~~
yoklov
The system malloc implementation. Users who want to use jemalloc have to opt
in, but doing so is relatively easy (using the jemallocator crate from
crates.io).

~~~
jabl
Why was this done?

Did rust become less dependent on allocator performance, or did system
allocators improve enough? IIRC glibc malloc has improved a lot over the last
few years, particularly for multithreaded use, but I don't know about windows
/ macOS.

~~~
steveklabnik
So, long ago, Rust actually had a large, Erlang-like runtime. So jemalloc was
used. Over time, we shed more and more of this runtime, but jemalloc stayed.
We didn't have a pluggable allocator story, and so we couldn't really remove
it without causing a regression for people who _do_ need jemalloc.
Additionally, jemalloc _was_ already removed on some platforms for a long
time; Windows has been shipping the system allocator for as long as I can
remember.

So, now that we have a stable way to let you use jemalloc, the right default
for a systems language is to use the system allocator. If jemalloc makes sense
for you, you can still use it, but if not, you save a non-significant amount
of binary size, which matters to a lot of people. See the parent I originally
replied to for an example of a very common response when looking at Rust
binary sizes.

It's really more about letting you choose the tradeoff than it is about
specific improvements between the allocators.

------
xvilka
It is possible to use also radare2[1] for this, along with radiff2. It
provides more options and has zero external dependencies.

[1] [https://github.com/radare/radare2](https://github.com/radare/radare2)

------
codewritinfool
This is great. The same author also wrote insect, which is cool, too.
[https://github.com/sharkdp/insect](https://github.com/sharkdp/insect)

~~~
EForEndeavour
Very cool! I greatly appreciate the author's inclusion of a `Pros and cons`
section in the README, particularly the cons.

------
heliostatic
For something with a little more firepower, I like:
[https://github.com/evanmiller/hecate](https://github.com/evanmiller/hecate)

------
ausjke
the author appears to be a true rust master, he wrote quite a few neat
programs in rust, very impressive and that actually got me interested in rust
lang

~~~
teacpde
Same here, I like small and neat Rust projects like this, the source code is
concise and easy to read.

He also authors
[https://github.com/sharkdp/bat](https://github.com/sharkdp/bat), which is
also worth to check out.

~~~
ausjke
bat is awesome, I'm using it daily since I found that, better that 'cat'

------
zwischenzug
I use hexer for this, which is an editor too.

[http://blog.metaclassofnil.com/?p=757](http://blog.metaclassofnil.com/?p=757)

There's also a vim mode for hex, which I use less often:

[https://vi.stackexchange.com/questions/2232/how-can-i-use-
vi...](https://vi.stackexchange.com/questions/2232/how-can-i-use-vim-as-a-hex-
editor)

~~~
globuous
There's hexl-mode for emacs that's pretty cool as well:

[https://www.gnu.org/software/emacs/manual/html_node/emacs/Ed...](https://www.gnu.org/software/emacs/manual/html_node/emacs/Editing-
Binary-Files.html)

------
iso-8859-1
Needs a diff mode so that it can also replace vbindiff.

------
neuland
The colorized output is a nice incremental improvement on xxd. And, I think
that it's nice that the README doesn't unnecessarily emphasize that it's
written in Rust.

------
JoachimS
Slightly off tangent: One tool I have used to dump data (and more) is cxmon. A
very nice monitor if one grew up using monitor cartridges on C64, Amiga etc:

[http://cxmon.cebix.net/](http://cxmon.cebix.net/)

[https://github.com/cebix/macemu/tree/master/cxmon](https://github.com/cebix/macemu/tree/master/cxmon)

------
lsllc
The colors remind me of editing MS-DOS files & drive structures with Norton
Disk Doctor! Fantastic!

------
krylon
I do not use a hex editor/viewer often. Not at all. But when I do, I am
painfully aware of how sparse my toolbelt just became.

This little program should fit nicely into that category. An area I would like
to explore, but have little-to-none incentives.

------
DarkCrusader2
The repo author has some pretty interesting rust projects. Worth checking out.

------
farmerbb
I've personally found vbindiff excellent for viewing files as hex:
[https://www.cjmweb.net/vbindiff/](https://www.cjmweb.net/vbindiff/)

------
rswail
Cool tool :) having a hex dumper that understood UTF-8 and dumped the
characters as one of the columns would be awesome too. These days, ASCII is a
small subset of strings.

~~~
sharkdp
Thank you! See:
[https://github.com/sharkdp/hexyl/issues/6](https://github.com/sharkdp/hexyl/issues/6)

~~~
laumars
Just a minor thing while you're eyeballing this thread: I noticed your fork of
lsd has the wrong build instructions.

[https://github.com/sharkdp/lsd#from-
sources](https://github.com/sharkdp/lsd#from-sources)

Good work on these tools though. You make me want to learn Rust :)

~~~
sharkdp
I forked lsd just to fix these build instructions:
[https://github.com/Peltoche/lsd/pull/33](https://github.com/Peltoche/lsd/pull/33)

I removed my fork...

~~~
laumars
Ahhh yes. Apologies.

------
xbryanx
Sorry to ask, but what is he use case for a hex viewer like this? What are the
kinds of projects you might be working on when you use something like this?

~~~
gpm
Your lab manager goes "hey, can you generate these proprietary binary files
(currently made with a horrible gui) to control this robot programmatically".

~~~
ducktective
But shouldn't one use an editor, like vim, in those cases?

~~~
gpm
Uh, vim doesn't open binary files very well, and the initial goal is to just
display them to figure out how they work not to edit them.

Afterwards you will probably want to run experiments with a hex editor, but I
found that xxd + vim was better for reading then doing so directly in a hex
editor at the time. If I did it again I'd likely start with this because of
the color.

------
EamonnMR
I've been looking for a command line hex editor too.

~~~
pwg
bpe:
[https://sourceforge.net/projects/bpe/](https://sourceforge.net/projects/bpe/)

Or the Joe editor in -hex mode: [https://joe-
editor.sourceforge.io/](https://joe-editor.sourceforge.io/)

------
RealityVoid
Would have loved being able to run it on windows. I use hex viewers for
embedded work and have been looking for a decent one for Win.

~~~
sharkdp
It works just fine on Windows (you probably need PowerShell for the ANSI
escape codes)

------
obphuscate
Ok, been a lurker on HN for awhile, but created an account to post.

This is freaking awesome. Been looking for something like this for awhile!

------
jbtule
I wish it would work with less or have less like features, taking the first n
bytes is okay, but pretty minimal.

------
nickysielicki
Is anyone aware of a React component that implements a decent hex viewer?

------
ape4
Hope this gets into distros

~~~
andrewshadura
It will.

~~~
andrewshadura
e.g. [https://tracker.debian.org/pkg/rust-
hexyl](https://tracker.debian.org/pkg/rust-hexyl)

------
IWeldMelons
I guess I am too old, but I use mcview. Or HIEW.

------
w4rh4wk5
Has anyone compared this to dhex, hexer, or ht?

------
platz
why am i seeing ugly background-colors with rxvt-unicode
[https://imgur.com/a/qkzAkqR](https://imgur.com/a/qkzAkqR) instead of the good
foreground colors I see in xfce4-terminal
[https://imgur.com/a/LY6bPn9](https://imgur.com/a/LY6bPn9)

i do seem to have 256 colors in urxvt my $TERM is rxvt-256color and i have
rxvt-unicode-256color installed

~~~
sharkdp
Please see:
[https://github.com/sharkdp/hexyl/issues/9](https://github.com/sharkdp/hexyl/issues/9)

------
sashk
I miss hiew.

------
fxfan
If I can make a suggestion - I don't mind the program being slow because of
all the logic - how would it be if you didn't give special status to ascii but
instead tried to intelligently assume a string encoding for strings and
accordingly colored "binary" vs "text" (utf-8, utf-16)

------
miki_rkt
What's wrong with hexdump?

~~~
insertcredit
It's not written in Rust.

