
Show HN: Sixel-tmux, display graphics because it does not eat escape sequences - csdvrx
https://github.com/csdvrx/sixel-tmux
======
csdvrx
Following the discussion on ET, here is my current work on a tmux fork to
focus on accurate rendering of graphics (ANSI and sixels)

My main goal is to be able to use sixel-gnuplot (a simple recompilation of
gnuplot with the sixel backend enabled: [https://github.com/csdvrx/sixel-
gnuplot](https://github.com/csdvrx/sixel-gnuplot) ) on both remote and local
hosts, in split windows.

A few issues remain to do that, but it already works in horizontal split
windows after 2 days: [https://github.com/csdvrx/sixel-tmux/blob/master/sixel-
tmux....](https://github.com/csdvrx/sixel-tmux/blob/master/sixel-tmux.jpg)

You can recompile the source within msys2 on Windows 10; I also provide a
binary on [https://github.com/csdvrx/sixel-tmux/raw/master/sixel-
tmux.e...](https://github.com/csdvrx/sixel-tmux/raw/master/sixel-tmux.exe)

~~~
abcdef123xyz
I took a look at this but at the moment you are just commenting out the bits
that stop tmux sending SIXEL to the outside terminal, so there is still a lot
of work to do for proper SIXEL support:

\- You don't check if the terminal outside even supports SIXEL. A major tmux
design feature is that it avoids sending output to a terminal if it doesn't
know it will understand it.

\- What if there are two clients attached, one in a terminal with SIXEL, one
without? Does it need to draw an ASCII placeholder?

\- The SIXEL image needs to be saved so it can be redrawn if needed (current
window is changed, detach/attach, scrolling into history etc).

\- If the SIXEL image is scrolled partly off the terminal or the window pans
so it is partly off the terminal, the image needs to be cropped to the right
size and drawn at the right place.

\- The problems with the timer and large DCS, just turning this off is
bringing back the problems with fast output it is intended to solve. A better
solution would be to only disable these while a DCS sequence is in progress.

If you get it working with these addressed I would be open to merging it into
tmux itself.

~~~
abcdef123xyz
An achievable option in the short term would be to do the following:

1) Fix the DCS issue as I suggested.

2) Check if the terminal supports SIXEL (there is a code in DA or DA2, or
perhaps we need a flag).

3) If the terminal supports SIXEL, pass it through, if it doesn't then ignore
it.

This would allow SIXEL to display on output although it would be corrupted as
soon as tmux redraws. It would be a start however and could be added to tmux
immediately.

~~~
abcdef123xyz123
You can try this:

[https://gist.githubusercontent.com/nicm/9acc00b67035b5f2b8a8...](https://gist.githubusercontent.com/nicm/9acc00b67035b5f2b8a81ec11c197b15/raw/e8f0c259f8b8146670a331f4aac544d0787e9152/gistfile1.txt)

Which does the basics of this:

\- Bump the sequence timeout to 5 seconds instead of 100 ms for more time for
SIXEL images but still not let the terminal hang forever. This could be made
more sophisticated perhaps.

\- Check the code in DA for SIXEL support and also check a terminfo(5)
extension flag Sxl.

\- Recognise SIXEL sequences and pass them through (like the 'tmux'
passthrough escape sequence) and copy them to the attached outside terminals
if they support SIXEL (DA or Sxl set).

I didn't try to disable the terminal-can't-keep-up code during DCS, that is
more complicated because some panes may be inside DCS and some may not. I only
have a small test SIXEL image so I would need a larger one that demonstrates
the problem to look at that.

I probably won't move this any further forward unless someone tests this, but
it may be a good start for further work on SIXEL.

~~~
csdvrx
Interesting, I will have a look and test it, thanks!

For images to test, on debian I recommend you install libsixel. You can then
just use img2sixel to convert a jpg to sixel. Alternatively, install gnuplot-
nox and:

export GNUTERM="sixelgd size 1280,720 truecolor font arial 16"

then you can test with various screen resolution and sizes, like:

echo plot sin(x) |gnuplot

Yes, my approach does not solves all problems. It is just a quick and dirty
fix to show sixels in the current window because I use gnuplot very
intensively on remote hosts. As soon as I switch to another pane, the output
is lost. But I use plots mostly to get a quick idea of how things are going,
every few minutes or hours.

About
[https://news.ycombinator.com/item?id=21646398](https://news.ycombinator.com/item?id=21646398)
:

1\. yes I do not check for sixel capacity. My approach is that any sequence I
do not know about should be left as is, because it may have been made for a
good reason, and there will always be newer sequences and terminals I don't
know about. For graphics such as sixels, your approach to check for sixel
capability might be better as they will need to be redrawn

2\. what to do if there are 2 terminals attached is linked to #1. I would by
default output the same sequence, because I do not touch what I do not know!
If you are checking capabilities, it may be best to show an empty block of the
same size at the place where the sixel output would be (a placeholder).
Another option would be to use a converter like libcaca to turn the sixel into
a rough ascii rendition. It would add a dependency to tmux, but also add a
feature for terminals that can't keep up, here because they don't know how to
interpret sixel

3\. and 4. saving the sixel image is what mlterm does. You could save the .six
files in ~/tmux/%PID%WIN%PANE

If you want to be consistent with text lines, you could keep all images until
the text line after the sixel image is thrown out of the history.

~~~
abcdef123xyz
You can try the sixel branch in tmux - the top of the branch does some
conversion and scaling (currently fits image to the pane and scales to the
terminal xpixel/ypixel) so it might interfere too much but if you just took
b34111b3 and d2e3f3c1 you would have plain passthrough (I might make a
separate branch for that actually).

There is still a lot of work to do, I haven't decided if the best option would
be to store all images in a big list and draw them separately when needed, or
whether to mark the cells they cover and draw the SIXEL when the cell is
drawn. SIXEL seems to have some strange behaviours and xterm seems to have a
few bugs so I'm not sure how close to emulate everything - it might be better
just to do it a sensible way. I don't think many applications will rely on
being able to draw on top of SIXELs or have partial characters appear
underneath.

I couldn't get img2sixel to generate the right colours, but convert from
ImageMagick worked.

