
Show HN: Murex – non-Posix shell designed for productivity - hnlmorg
https://murex.rocks/
======
zokier
Both the colon and the arrow are weird. In what sort of situations there is
ambiguity in the direction of a pipe or between command and its arguments that
this would solve? The colon would make more sense if redirection specifiers
would be written before the colon, but alas they are not.

How does the magic with JSON etc work? The docs do mention something about
mime types but that is never really explained properly. If I make my own
command that outputs JSON, can I make Murex understand that without explicit
casts?

Overall the whole thing feels bit quirky to me, but there is cool stuff in
there too.

~~~
hnlmorg
Thank you for your feedback. The arrow token is something I've gotten used to
typing with the likes of C++ and Perl but the rational of adding it to murex
was so that each command was more visually distinctive which isn't always the
case with the pipe character. However I did expect it would polarise opinions
so it's completely optional (ie you can use the regular pipe token if you
prefer).

The colon I'm a little on the fence about myself. It's a throwback to an
earlier iteration of the syntax and it does add a little "code smell" to the
interpreter and code behind the command prompt. In fact I did nearly remove it
from murex just last week. However it is also an optional token so you're free
to ignore it if you'd prefer.

The JSON magic is just Go's standard library unmarshalling JSON into a slices
and maps and then remarshalling it back to JSON at the end. This design means
I can swap out a JSON interface with a YAML, TOML or most other serialisation
formats at runtime. For example you can have JSON coming in then output it as
YAML (or visa versa). Tabulated data such as CSV is also supported but I had
to be a little more creative in how are interchanged with formats like JSON
since they represent different ideas of structured data -- sadly converting
between CSV and other formats is still rather buggy but I'd definitely on my
road map to fix.

In terms of how a file is recognised as JSON (or any other format), there is
two methods: you use the `open` builtin which is a bit like `cat` but works
with HTTP URLs as well as file paths. If it's a HTTP address it uses the
Content-Type header to get the MIME (just like your desktop browser would do)
and passes that type information down the pipeline. If it's a local path then
it checks the extension. You can see the supported extensions and MIME types
by typing:

    
    
        config get shell mime-types | pretty
        config get shell extensions | pretty
    

The issue is when using a classic UNIX tool like `cat` which wouldn't know a
JSON from a CSV. So to support them you can also cast if you want to. for
example the following two examples do the same thing:

    
    
        open file.json | ...do something
        cat file.json | cast json | ...do something
    

I completely agree there's a lot of quirkiness to the shell. Some stuff I
might even do differently if I were to start again from scratch and definitely
some POC code in there I really need to rewrite. This shell was born out of a
desire to solve some personal annoyances with Bash and I think it's fair to
say I overindulged myself in places. But I figured if I design it for myself
then at least I'm guaranteed one person will use it :) Of course if anyone
else likes the project and finds themselves wanting to use it then I will
absolutely take their feedback on board too.

edit: also thanks for pointing out the areas where the documentation was
unclear or lacking entirely. Documenting this thing has proven to be more time
consuming (and less enjoyable) than writing the code! But I'll work on that.

~~~
zokier
> and passes that type information down the pipeline

That's the big question mark to me, how is the type information passed along.
If I do

    
    
        open foo.json | bar
    

then how will bar know (or will it?) that its input is json? Or is the type
information only internal to the shell and not exposed to the individual
pipeline parts?

~~~
hnlmorg
It's internal to the shell however so is the individual pipeline parts to some
extent.

The long answer is that all commands in the pipeline are treated as either a
murex shell script or a builtin function. Any external commands are actually
the builtin function called "exec" which is calling an external command as its
parameters. This means I could effectively write my own pipeline with richer
APIs beyond just reading []bytes of data. This doesn't come without
significant drawbacks as well (which I can also talk about if your
interesting) but I'm working to mitigate them.

So the following two lines of code are equivalent:

    
    
        open foo.json | grep foobar
        open foo.json | exec grep foobar
    

So if you then were to pipe that into another builtin you'd need to cast it
again. eg

    
    
        open foo.json | grep foobar | cast json | ...do something
    

This is just illustrative though. Technically this would break the JSON file
because "grep" just works on dumb lines and JSON isn't a linefeed document.
This is why murex comes with such an extensive number of builtins, for example
if you wanted to do a grep-like regex match against elements in a JSON array
and output that as valid JSON you could the following pipeline:

    
    
        open foo.json | regexp m/foobar/
    

However going back to your original question, if someone wanted to write a
shell script or any other utility that support murex data-type but wasn't part
of the murex shell binary, then there's no way murex could pass that type
information to it.

This is a problem I've been mulling over for a few years now and I've
considered a few options in that time - none of them are without their
problems. At one point I even laid down some POC code but that has since been
deleted because pragmatically it seems a premature problem to solve while I'm
the only person developing and using the shell.

edit:

It's also worth noting that there is a default data-type that is assumed even
when no data-type is specified. That type is a loosely tabulated "catch all"
type format for your standard unix tools. That means output from commands like
`ls -l`, `ps`, and other classic utilities can be parsed and fields selected
intelligently. eg if you were to type the following partial command line then
hit tab:

    
    
        ps -fe | [ <tab>
    

You'd see the headings from `ps` in your autocompletion suggestions:

    
    
         UID     PID     PPID    C       STIME   TTY     TIME    CMD
    

So you could select one or more of them:

    
    
        ps -fe | [ UID PID CMD ]
    

...and murex would filter the output of ps to only those 3 columns.

It's not perfect by any means - since there are a thousand different ways
developers can output data to the terminal - but it does seem to be an asset
more than a hinderance for my particular use cases. Of course YMMV and I'd
welcome any feedback if that were the case :)

------
CaptainZapp
Not to be a spoilsport. But you may want to rethink the name.

There's a French financial software company called Murex.

Since a shell by the same name is very much computer related and I assume that
they have a trademark on the name you may be in for a world of hurt, legally
speaking.

~~~
hnlmorg
Good point. I did spend a fair amount of time deciding on a name - it's
surprising just how few good names aren't already being used in OSS. So I'd
love some suggestions if anyone has any.

~~~
CaptainZapp
I really hate the to be the bearer of bad news, but it would really suck to
invest a lot of work and when your product gets popular (and visible) to get
sued.

Naming is hard, but how about something like _MXShell_ ?

That's, of course, if it makes sense.

Edited to add: There seems to be an instagram account by that name and a
Twitch streamer. Else then that I can't find much with a cursery Google
search.

~~~
hnlmorg
I genuinely appreciate the feedback. I was hoping there was enough distance
between this project and the French firm but I'd stupidly not considered that
they'd trademark their name.

~~~
CaptainZapp
It's not a household name, but it's a fairly large, privately held, company
with a significant reference list.

They have a web site, if you want to check them out.

------
giancarlostoro
Wow this is really nice, I use Fish cause it's nice out of the box and not for
scripting. I think that's what my sweet spot is, I use Bash for shell
scripting, cause it's almost always installed or something compatible is
installed in most Distros, but for my terminal emulator's shell I don't care
what I use as long as it is indeed intuitive. I'll be sure to give Murex a
shot.

------
burlesona
Looks very cool. Has some of the same ideas as Nushell. Seems like a lot of
these “better shells” are coming out lately, I wonder why that’s in vogue at
the moment?

~~~
em-bee
possibly the younger generation of developers is less dogmatic about POSIX and
'portability'.

another theory is that people realized that you can use a different shell for
interactive use than for scripting.

the popularity of python, perl and other languages for scripting may have
helped with that.

i am looking forward to see more. fish only scratched the surface.

~~~
hnlmorg
I don't agree with this. Or at least it's not applicable for me. I've been
managing UNIX and Linux systems for around 2 decades so sadly I'm now far too
old to be included as part of the younger generation any more :(

murex came about more by accident than intention. It was a personal project to
"solve" a real world problem I was having at work but it kind of morphed into
a shell. Even the shell aspect was very different in the beginning to what it
is now. In fact almost every aspect of the shell has changed in the last 3
years.

The project is older than the Github repo because it was a private project
hosted on a personal git server for the early stages of its life and I
squashed the commits when uploading to Github to hide the early code (read:
"mistakes") but once the project did start morphing into a shell the intention
was always to make scripting a first class citizen because I specifically
didn't want to use a different shell for interactive use than scripting. In
fact if anything, I wanted to bring scripting paradigms to the REPL shell but
without creating a command line that was too verbose for everyday use.

~~~
em-bee
i see your 2 decades and i raise you 3 :-)

for a while i have been dreaming of the idea of having a real programming
language for use on the commandline and i always get stuck at the verbosity of
the syntax.

i didn't want to invent a new language however but possibly reuse an existing
one. just that none of the languages i came across really fit.

for starters it would have to be a language that uses spaces to separate
arguments, like lisp, smalltalk, red, forth and a few others. but there are
more potential requirements that most languages don't really support, like
being able to specify string values without quotes...

~~~
hnlmorg
I'm with you on that one. The original murex syntax was very Javascript
inspired. In fact it was even object oriented and originally called "jsh". The
ironic thing is I hate JS as a language - though that's less about the syntax
and more about other specific grips.

Anyhow, first I made quoting strings optional, because mandatory quotes were
_fucking annoying_ for regular use. Then the parentheses disappeared. Then the
object orientated model was dropped.

Murex is a very different language now to what it was when it started; though
there are still some legacy features in there which are throwbacks to early
designed (eg the default data-type is called "generic", which made sense when
it was OO as generic was meant to handle any content but makes less sense now
that it's a first party type in its own right).

There's definitely a lot wrong with murex though. It's a long way off being a
proper language. But for shell work, it has been my default daily shell for a
few years now. I also use it for writing any shell scripts albeit just stuff
for personal use.

From what you've posted, I'd be very interested if you ever designed a shell
language. It sounds like you and I have similar desires for what the next
iteration of UNIX shells should look like.

------
boygobbo
A couple of name suggestions from Latin:

\- cortex (means husk or shell)

\- nux (nut, i.e. a fruit with a shell)

'Murex' by the way means the sea snail that Tyrian purple dye was made from.

------
rcarmo
I like the data access bits, but find the arrow “operator” a bit weird. I keep
wondering if a simple wrapper around jq and the like wouldn’t be enough.

~~~
hnlmorg
Thank you for the feedback. Yes, the arrow token is definitely a polarising
one. I personally like it because it's more visually distinctive but even I
find myself using the pipe character accidentally on occasions due to muscle
memory. So I completely understand why many others wouldn't want to use it
too. This is why I made the arrow token is completely optional. You can use
the traditional pipe character if you prefer. eg the following two generate
the exact same ASTs:

    
    
        open example.json -> foreach { -> regexp s/foo/bar/ }
        open example.json | foreach { | regexp s/foo/bar/ }
    

You could even use both tokens in the same pipeline if you wanted:

    
    
        command -> command | command -> command | command
    

...though I'd urge against doing that - it's better to pick one token and use
that throughout your pipeline.

re jq: that certainly covers some use cases but it doesn't cover them all.

For example murex has a hint text - which is a line below the interactive
command prompt - and it works a little like the tool tip text in GUIs. It's
context sensitive and will give you hints about the command line you're typing
such as

\+ full path to an executable (including resolving symlinks) so you can be
confident you're about to execute the program you think you are

\+ it expands variables if any $VARs are used to give you a clue about how the
final command line will be interpreted

\+ it gives you descriptions about the application - parsed from man pages or
the source code (if available)

\+ it will display the code behind an alias so

The idea is to inform the user of any hidden traps behind shells that often
lead to mistakes.

That's just the hint text. There's been some improvements in autocomplete
suggestions that aren't available in most shells eg:

\+ man page parsing,

\+ suggestion descriptions,

\+ the ability to use regexp to jump areas in a nested path (similar to fzf)

\+ and it can also manages timeouts in dynamic suggestions better than bash.

I've tried to model the REPL environment of murex more on what tooling I'd
expect from an IDE rather than what tooling I'd expect from a POSIX shell.
That's not to say I've created a polished UI/UX nor that there isn't some
Bash/zsh features missing from murex. There's still someway to go before murex
will be out of it's beta stage. But that's the design inspiration behind the
shell.

------
uncletaco
What is that typeface? Every vowel looks like an "o" that was cut up to look
like another vowel, its absolutely hard to read. It's also ironic that there's
(allegedly) a focus on readability for the language but absolutely none of
that concern found its way to the website.

I do like the piping syntax.

~~~
hnlmorg
Thanks for the feedback. I'm dyslexic so I did actually spend a long time
picking a readable typeface for the website since I often struggle with some
of the artistic choices other designers inflict. I found this typeface (and
the other stylings of the site) to be what was easiest for me to read.

However I do acknowledge that I haven't conducted any form of research into
other people's opinions and nor have I tested the site on many other platforms
aside Firefox and Safari. So what works for me might not work for other
people.

To that end, if others agree it's not the most readable typeface (or any other
issues with the site are pointed out) then I will definitely change it. As you
rightly say, being readable is the most important concern for that site.

~~~
uncletaco
I wouldn't say I'm dyslexic, but I do struggle with typeface that uses a lot
of "o" in them, I'm not quite sure what the word is for it.

Unironically the most readable typeface I've ever used that comes standard on
all computers is Comic Sans or a variant of it.

~~~
hnlmorg
Thanks for the feedback everyone.

I've changed the typeface now, however you may have to hit refresh on your
browser because I've used some pretty aggressive cache-control headers.

It's not a hugely massive change in typeface style because I really can't
stand sherif when reading large chunks of text however I have also included
some tweaks in the spacing which might also help.

Let me know if that has improved things or if I need a more dramatic change in
typeface.

~~~
uncletaco
I know it can be frustrating to present work and criticism veers off to focus
on other things, but thank you for being receptive. Its much better now :-)

~~~
hnlmorg
I don't mind. I'm not a designer so I welcome any feedback on that front. I
was more surprised at how bad others found it because I usually struggle with
printed text.

Thanks again everyone for the feedback

------
kseistrup
The link should have been
[https://github.com/lmorg/murex](https://github.com/lmorg/murex) or
[https://murex.rocks/](https://murex.rocks/)

~~~
dang
Thanks! We've changed to the latter from
[https://github.com/lmorg](https://github.com/lmorg).

