
Show HN: Purely Functional Scripting - continuational
https://github.com/topshell-language/topshell
======
continuational
Hi all, TopShell is a reactive scripting environment that I've been working on
for a while. It's purely functional with type inference that runs as you type,
as well as a bit of autocompletion.

You can work with SSH, files and HTTP requests, and there's a pure DOM library
for scraping. You can draw with SVG or HTML.

I'm looking forward to your feedback!

~~~
maddyboo
TopShell looks very interesting, thank you for sharing!

This is a problem domain which I've been thinking a lot about recently. I
write _tons_ of shell (bash) scripts. I hate bash and its peculiarities, but
it remains one of the quickest ways for me to get work done; I've tried to use
many other languages to fill the gap of writing scripts that start as quick
one-offs but have the potential to evolve into a more refined piece of
software over time, but so far haven't had much luck.

Most languages force me to go to great lengths for everything I want to do in
this domain:

\- implementing command-line arguments/flags/subcommands: either overly-
simplistic (getopts) or overly complex (cobra)

\- force me set up error handling for everything that could possibly go wrong

\- aren't flexible enough to transform 'soupy' data between various
representations without basically writing a full-blown custom parser

\- can sometimes be difficult to distribute a minimal-dependency release to
remote machines or other users

Functional languages are awesome for many of these things, but can be awkward
when dealing with the side-effects that we normally _want_ when dealing with
scripting tasks.

So far, what I gather from your readme/wiki is that TopShell is:

a) A functional language seemingly inspired by ML, Haskell, et al.

b) A reactive development environment for the language, à la Jupyter

I love the code examples in the readme; I think this is something most people
want to see but a lot of projects overlook. However, I think the readme is a
bit barren on the philosophical front. You have a great wiki page [0] for that
- I think it could be condensed and integrated into your readme so new
visitors have a good idea of your vision.

As someone who would consider using TopShell, my concerns are:

\- I can't find documentation on how I can use my own editor/environment if I
want, rather than the web interface (which is nice for its notebook-like
experience, but I don't see it fitting into my normal workflow)

\- I want to continue to integrate with programs which adhere to the unix
philosophy; how does TopShell fit into this?

\- How dedicated are you to the development/maintenance of the project?

\- Do you forsee growing a community of contributors (and if so, how?), or do
you want to keep this as a personal pet project?

\- Do you think the project will be alive in 1, 3, 5 years?

[0]: [https://github.com/topshell-
language/topshell/wiki/TopShell:...](https://github.com/topshell-
language/topshell/wiki/TopShell:-Reimagined-Terminal-and-Shell)

~~~
continuational
Thank you for the long and thoughtful comment. I think you bring up a lot of
valid points, and I don't have complete answers to all of them yet, but I'll
do my best to address them here.

> Functional languages are awesome for many of these things, but can be
> awkward when dealing with the side-effects that we normally want when
> dealing with scripting tasks.

I think that's a common sentiment, but I have a different take on it - purely
functional programming is actually great at expressing _intended effects_ \-
as _opposed_ to side-effects!

With the right standard library, I think it might be possible to make I/O at
least as easy as in imperative scripting languages like Python. As a small
example, compare reading a file as a string in Python and TopShell:

Python:

    
    
        file = open(“testfile.text”, “r”) 
        text = file.read() 
    

TopShell:

    
    
        text <- File.readText "testfile.text"
    

Does that translate to bigger examples? I think it does, but a more serious
efford to compare the two would be required for a convincing argument.

> I can't find documentation on how I can use my own editor/environment if I
> want, rather than the web interface

This is not yet possible. What I have in mind for this is moving the code
execution to the backend and having some way for an editor to communicate with
the TopShell server - perhaps through the Language Server Protocol.

> I want to continue to integrate with programs which adhere to the unix
> philosophy; how does TopShell fit into this?

TopShell can _use_ such programs, but TopShell scripts can't currently
_become_ such programs. I'm still looking for the right way to do this - one
way would be to have a `main : System -> Stream Bytes -> Stream Bytes` where
`System` contains command line arguments and so on, and `Stream Bytes ->
Stream Bytes` is stdin to stdout.

> How dedicated are you to the development/maintenance of the project?

I'm not sure what an answer to that looks like - though what I can say is that
I use TopShell for all my own ad-hoc scripting needs.

> Do you forsee growing a community of contributors (and if so, how?), or do
> you want to keep this as a personal pet project?

I hope for a community of contributors, but I have no experience in community
building per se. There are many low-barrier ways to contribute though - for
example, by extending the standard library. Any tips?

> Do you think the project will be alive in 1, 3, 5 years?

I do think so, but as it is, there's a "bus factor" of 1. Please don't bet
your business on it right now unless you're prepared to take over maintenece
on a moments notice.

> Can sometimes be difficult to distribute a minimal-dependency release to
> remote machines or other users

To me, a script is a single file I can send to a friend (or put on a machine),
and it will just run. Most scripting languages don't really do this, because
they assume that package management is handled somewhere else, and implicitly
depend on an environment. When TopShell eventually gets a `main` function, the
goal will be that all you need to run a script is the script file and
TopShell.

~~~
maddyboo
> TopShell can use such programs, but TopShell scripts can't currently become
> such programs. I'm still looking for the right way to do this - one way
> would be to have a `main : System -> Stream Bytes -> Stream Bytes` where
> `System` contains command line arguments and so on, and `Stream Bytes ->
> Stream Bytes` is stdin to stdout.

That actually sounds perfect. If you were able to implement this, I could see
this tool coming in very handy for tons of things.

I’ll be keeping a close eye on the project, thank you for your thorough
response!

------
yoz
This looks great, and I'm eager for more!

I was initially put off by going straight to the repo's declared homepage (the
Playground) and just seeing two empty text areas. The repo README isn't quite
enough on its own - as you say, the interactive scripting experience is a huge
part of the value here.

I strongly recommend doing what you can to get visitors to that experience as
directly as possible. For example: pre-filling the Playground with a short,
good example (e.g. the JSON-to-table one). Many people won't get as far as
pasting that into the empty Playground. Once I tried it, everything made
considerably more sense to me. (I'm not a regular FP coder; I've done some
Elm, some Miranda in college, that's it.)

There are plenty of other things you can do (embed a couple of GIFs in the
README; offer a choice of examples in the Playground) but I'd focus on (a)
getting people to the Playground (b) prefilling the Playground with an example
(c) add a few explanatory comments to the example.

Again, this looks great, and I'm eager to use it. Congrats!

~~~
continuational
That's a great idea - I will look into that. Perhaps even a little menu with a
few different examples. One challenge is that the Playground is sandboxed, so
the more interesting examples involving eg. SSH or scraping are mostly off
limits.

Thanks for the feedback, and be sure to leave an experience report if you do
try it! TopShell is very young, and its future evolution will be guided by
concrete use cases.

------
lidHanteyk
Looks like Elm.

Why did you add magic imports? It seems strange to go for purely functional
behavior and then add magic side effects.

~~~
continuational
It is indeed in the Haskell family of languages, just like Elm and PureScript.

TopShell is all about the out-of-box experience - not just a language, but an
environment that emphasizes an interactive scripting experience, where you can
see data as you fetch and manipulate it.

------
spleeder
Seeing how closely TopShell resembles Haskell I am curious why you chose to
implement it in Scala rather than Haskell or Purescript.

~~~
continuational
The reason is that I do Scala at my day job, and thus I'm simply more
experienced in that than Haskell.

While I've written a compiler in Haskell in the past, and prefer the language
in many ways, I don't know how to do front end programming in Haskell, compile
Haskell to efficient JS or debug performance problems. I also don't know if
there's a good Haskell IDE these days. PureScript would have been risky for
me, since I haven't done anything in it before.

------
gitgud
The first example is a HTTP request, aren't network requests inherently
_unpure_? as they rely on state outside of the program and could cause side
effects...

~~~
continuational
Purely functional programming languages can't have _side effects_. But they
can have _intended effects_.

Like Haskell, TopShell uses monads to represent effects, and eg. Http.fetch
returns a Task, which is similar to the IO monad in Haskell.

The returned Task is a pure _description_ of how to perform the HTTP request.
Every time you apply Http.fetch to the same arguments, you get exactly the
same description back, thus maintaining referential transparency.

Only the top level knows how to interpret such a description and perform the
effects (eg. open a HTTP connection etc). Thus you compose Tasks into bigger
tasks and then let the top level run them.

------
iamwil
to OP: How long have you been working on it? What was the biggest challenge
you ran into when working on it?

How does piping work? Can you split and join the streams?

~~~
continuational
I've been working on it for a little over a year. The biggest challenge so far
has been designing and implementing a type system with the flexibility
required for ad-hoc scripting. Building a standard library that can tackle
dirty real world problems as nicely as possible is an ongoing challenge too.

In TopShell, Streams are analogous to pipes. The primary way you use TopShell
Streams is by combining and transforming them into new streams. The interface
for streams is very similar to what you might find in a Functional Reactive
Programming library, and indeed it can be used to eg. animate SVG based on
data that's streamed from somewhere.

~~~
iamwil
Why would ad-hoc scripting require more flexibility with a type system? What
kind of flexibility? Is it that you need more types? Or more union types to
handle all the different variations that are the same, but all slightly
different?

~~~
continuational
Anonymous records and variants are convenient in a scripting language because
you don't have to declare them. This in turn leads to anonymous record types
and anonymous sum types, which require something beyond HM for type inference,
and I wanted to avoid introducing row variables.

TopShell uses type constraints for this, and I've never implemented a type
constraint solver before TopShell. There are also some extra things to do
around checking explicit type annotations, when the user provides them.

