
Big Programming, Small Programming – Glow, the language - daivd
http://fendrich.se/blog/2013/09/03/big-programming/
======
jamii
This idea - having locally unrestricted components connected by a restricted,
declarative system - seems to keep popping up and never quite taking off.
Hopefully sooner or later someone will spin it just right.

Some related projects:

Opis -
[http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.167....](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.167.5260)
\- pure functions connected by dataflow combinators (with time travel
debugging and model checking at the dataflow level)

Bloom - [http://www.bloom-lang.net/](http://www.bloom-lang.net/) \- temporal
logic for coordination, message passing for communication with stateful
endpoints

Flask -
[http://dash.harvard.edu/handle/1/2797447](http://dash.harvard.edu/handle/1/2797447)
\- a c-like language for local sensor computations with a haskell dsl for
specifying communication patterns across the sensor network

Then of course there is all the FRP work (eg
[http://www.haskell.org/haskellwiki/Functional_Reactive_Progr...](http://www.haskell.org/haskellwiki/Functional_Reactive_Programming)
[http://docs.racket-lang.org/frtime/](http://docs.racket-lang.org/frtime/))

Despite all the interest, overlog (the predecessor to bloom) is the only
language I've seen so far that has any convincing demos (hdfs and hadoop in
8kloc). I'd love to know if anyone is actually using this sort of stuff in
production or, if not, why not? What goes wrong when you try to scale this up?

EDIT: I forgot to mention Mozart/Oz, one of the most interesting dataflow
languages and one that explicitly talks about layering progressively less and
less deterministic systems of concurrency - [http://www.mozart-
oz.org/features.html](http://www.mozart-oz.org/features.html)

~~~
robertfw
There has been some interesting work from Johnathon Edwards @
[http://www.subtext-lang.org/](http://www.subtext-lang.org/)

~~~
jamii
I've had his blog bookmarked for a while but never got around to actually
reading it. I'll have to take a look, thanks.

------
beagle3
It's a nice idea, but the run-length example is kind-of-misguided, even though
that's how everyone would implement it on first try.

A language should push you towards a more robust/faster/simpler solution by
making it the simple, elegant thing to do. In K, a run length encoder would be
something like:

    
    
        {(x@&~m),'(+/m)@&~m:~':x}
    

Which looks like line noise, but basically encodes the following operations:

x is the input

m gets 1 where a position in x is the same as the previous position, 0 where
it isn't. (apply ': "each-pair" to ~ "match")

Then take the sum of match (+/m) at where (@&) there is no match (~m).

Finally, add that to x where there is no match (x@&~m, literally, "x at where
no m"), and zip it (,' meaning concat each). And the curly brances make it
into a function of x.

Now, I understand most people would think that this is unreasonable. However,
in K it is easier to produce a vectorized solution than an itertive one - and
as an added bonus, the bugs tend to be either of the "wrong spec" or the "off
by one" error, eliminating whole classes of potential bugs.

And .. the result is usually quick, thanks to vectorizing (and can go on a GPU
easily).

A language that promotes item-at-a-time processing is missing a lot.

~~~
d0m
As intriguing as it looks, I certainly wouldn't want to look at that kind of
code all they long.

~~~
beagle3
I can understand the sentiment, and the only thing I can say is:

There's a learning curve; it's steep, but it is well worth it - much like Math
notation (give me "x^n" any day over "repeat multiplying x by itself n times")
or Music notation, once you're used to it, any notation that is less concise
seems arbitrarily and needlessly verbose.

(That's actually a reason not to learn K / APL: When you actually grok it,
it's hard to take modern software engineering seriously. A reason to grok them
is that -- for some problems -- you're going to become much more productive).

~~~
d0m
Well, after writing my comment, I went online looking for some quick
introduction to the language. I'll admit that there are certainly very
interesting facets to that language.

What I can say however is that there's a difference between conciseness in
writing and productivity in reading.

For instance,

    
    
        (!R)@&{&/x!/:2_!x}'!R
    

It's very concise but very hard to grasp quickly. The equivalent, in even 10
lines of code, might be way easier /and/ faster to read and understand.

That being said, I really like the functional aspect of it where most of
common operations on list were shortened. (I.e. map, reduce, etc.)

Personally, I think the Arc language strikes a really fine balance of verbose
versus concise. It's just unfortunate that there's not a more active community
and that it lacks the battery to be productive with it.

~~~
ajanuary
My understanding of the idea behind most APL style languages is that it's an
expert language, in that you need to take the time to invest in it just like a
musician would musical notation.

Once you've done that, the theory is that it is actually quicker and easier to
grasp than reading "verbose" lines of code.

That said, I've never taken the time to invest in an APL style language, so I
can't speak to whether it is actually true or not.

~~~
d0m
Makes sense. If you spend enough time it can become second nature, like
musical sheets. Still, I find that musical notations is way more limited, so
it makes sense to have it concise. But let say people were naming some parts
of the songs, and referring to it, and you had to jump and "get into the mind"
of the person who wrote it.. than maybe it'd be a different story and more
verbosity would be better.

~~~
evincarofautumn
APL/J/K code is really like mathematics, something that most programmers
aren’t exposed to in the slightest. If you don’t grok it immediately, then you
can actually just sit down and equationally work out the author’s exact
thinking, in precise terms—and you level up as a side effect, so you don’t
need to work out the same patterns every time you see them. That’s nigh
impossible in C++!

Take it from someone who has seen both sides: the concision is a good, good
thing.

------
shailesh
Sounds pretty cool!

Since you're pretty much at design stage, two ideas are worth exploring:

1\. Uniqueness typing (re: linear typing for mutable data) is a neat addition.
The paper by John van Groningen et al seems the most appropriate: "Exchanging
Sources Between Clean and Haskell: A Double-Edged Front End for the Clean
Compiler."

2\. The Shen lanaguage [http://shenlanguage.org/](http://shenlanguage.org/)

Clearly, these may not map directly to the features in the final design, but
the ideas they cross-pollinate will be very useful.

~~~
daivd
1\. Yes, the linear typing for mutable data is inspired by Clean. 2\. I look
at Shen from time to time. It looks nice, but I have never gotten around to
play with it.

------
MichaelAza
The moment I see an open source repository for this I'm going to contribute
the hell out of it.

This need to exist. This has to exist.

~~~
daivd
If Internet comments were always this encouraging, the world would be a better
place :).

------
mercurial
This looks absolutely fantastic. Well, the whole visual programming thing
leaves me cold, but the rest looks great. I'd be curious to see how data
structures are supposed to look like. I assume they're some kind of structs
(hopefully not featuring ridiculous name collisions like Haskell).

~~~
daivd
Visual programming used to leave me completely cold, but with the emergence of
tablets and the promise not to make _everything_ visual, I hope to convince
people otherwise. If you have not seen Bret Victor's brilliant "Future of
programming" [http://vimeo.com/71278954](http://vimeo.com/71278954) and
"Inventing on principle"
[http://vimeo.com/36579366](http://vimeo.com/36579366), I highly recommend it.

I promise not to make the Haskell struct mistake.

~~~
mercurial
I'll try and find time for it. Thanks for the recommendation.

------
drothlis

        In the big picture, visual programming is very natural. It is, in fact,
        so natural that we often make flowcharts of how systems work. It would
        be better if these flowcharts would actually be the system, rather than
        an incomplete, possibly outdated description.
    
        In the details, visual programming is just an inconvenient gimmick.
        IMHO. It may change one day.
    

Not exactly the same, but it reminds me of Smalltalk environments, which give
you a "browser" to view classes, inheritance relationships, etc etc. while
giving you a code editor for the actual method implementations (example:
[https://vimeo.com/27850933](https://vimeo.com/27850933) ).

    
    
        We use two types of files to program Glow. The detail files, which can
        contain anything, and the overview files, which may only contain
        constants and how components are connected to each other. An overview
        file may not use macros.
        
        Both are editable with any text editor, but the overview file can also
        be edited visually as a vectorized flowchart (vectorized to zoom in on
        details)
    

"Both are editable with any text editor" sounds much better (to me with my
admittedly limited experience with Smalltalk) than Smalltalk's file-less model
where everything is a change to the whole system "image".

------
swift
This looks great, and I encourage you to keep working on it and make this a
reality. I've thought along the same lines and I think a language like this
has real potential.

I'm pretty surprised I haven't seen J. Paul Morrison's "Flow-Based
Programming" mentioned anywhere in the post or these comments. I first read
this book years ago as a undergrad and it made a big impact on me. It
discusses ideas very similar to the ones proposed for Glow. You can read it
free online here:

[http://www.jpaulmorrison.com/fbp/](http://www.jpaulmorrison.com/fbp/)

You can find links to several existing implementations of flow-based
programming on Wikipedia:

[http://en.wikipedia.org/wiki/Flow-
based_programming](http://en.wikipedia.org/wiki/Flow-based_programming)

------
tinco
The syntax looks very pretty, but why the colons at the end of block header
statements? Python does this too, but I see no reason for it. In my opinion
that makes it look a bit cluttered, contrast it for example with the
(excellent) coffeescript syntax.

~~~
dscrd
Colon + indentation is actually the most natural way to signify a block that I
have ever seen. Pretty, non-cluttering and sort of light for the eyes. And
it's not without precedent: it resembles how colon is used in text.

------
fitchb
I'm surprised Noflo and Subtext haven't been mentioned. They seem like they
have very similar goals.

------
acjohnson55
If the syntax ends up being a great match for your ideas, I've got a lot to
look forward to! Syntax is a big aspect of how much or little I enjoy working
with a language. I'm hoping for something Python-ish in terms of simplicity
and readability, but I think there's room to improve upon even that.

I'm very happy to not see much redundant punctuation where formatting makes
meaning clear, and to not see keywords where punctuation makes meaning clear.
But since mutability is being deemphasized, maybe we can even get a single
equals for equality, and give assignment the longer symbol?

~~~
daivd
I agree that syntax is more important than us "rational" programmers like to
let on. The syntax will be similar to Python, but nothing fantastic.

I think switching the usual meaning of = and == will be confusing. O'Caml uses
= and == for deep and shallow comparison respectively and <\- for assignment.
That is a possibility.

------
michaelsbradley
The concurrency features strike me as being influenced strongly by Hoare's
CSP[1]. The flowchart like interface seems akin to Harel's reactive
diagramming formalisms[2].

[1] [http://www.usingcsp.com/](http://www.usingcsp.com/)

[2] [http://www.scribd.com/doc/167971960/Modeling-Reactive-
System...](http://www.scribd.com/doc/167971960/Modeling-Reactive-Systems-With-
Statecharts)

~~~
daivd
Yep, it is influenced by CSP. It is implicit in the text, since the primary
influences (Pipes, Go, etc) are directly influenced by CSP.

------
educating
Going to try to address the whole problem here, not just the trying to invent
a new language:

from: [http://fendrich.se/blog/2013/01/24/pre-programming-mental-
si...](http://fendrich.se/blog/2013/01/24/pre-programming-mental-silence-
meditation-with-entrainment/)

> I am, in fact, a grumpy, skeptical, philosophically materialist atheist.

from: [http://fendrich.se/blog/2013/04/09/a-better-soylent-
good/](http://fendrich.se/blog/2013/04/09/a-better-soylent-good/)

> Adding nootropics and stimulants, like the original Soylent guy did, is also
> interesting. However, to avoid complicating things, I will add this later. I
> don’t really know which substances, but at least ginkgo biloba.

I think you are taking on too much risk in order to get attention. Your
grumpyness might be a result of some other health condition like sleep apnea
that you need to take care of before taking on a radical diet that will leave
you sick and more overweight. Take the time you would have spent working on
Glow instead working on an existing up-and-coming language like Go. And, good
luck. I know what its like to feel like you will try anything to fix your life
and get relevant. But inventing a new language is not where its at. Tom Hickey
who is one of the smartest people I know and one of the best presenters I've
seen can't even convince most to use Clojure. It isn't easy.

~~~
TheZenPsycho
What's wrong with inventing a new language? Nobody _has_ to use it. Do they?
Or is he staking some vast amount of his personal reputation on it? some how?

~~~
educating
I'm suggesting the combination of possible health problems combined with a
pattern of trying to reinvent everything. I'm familiar with this pattern
myself. I'm suggesting not to go down this road, and instead focus those
creative efforts on something with more promise.

I also think that there are problems with all other languages we use, but like
I said, this is a very difficult road.

~~~
TheZenPsycho
I'm confused. Are you his mom or what? Why shouldn't he go down this road?

~~~
educating
If you want me to be blunt, I think he is incapable of it, and he is wasting
all of our time with a "what if" post about a language that will never exist.
Happy now?

~~~
6d0debc071
That's already a sunk cost to everyone in the thread. Further criticism - that
doesn't address the reasons people are enthused, serves as a time-sink for no
return in value.

i.e. if you want it to go away from altruistic concern, you're probably better
off addressing the flaws in the concept or ignoring it. ^_^;

------
z3phyr
As far as I know, a language called GlowScript (compiles to js) exists.

The language author might consider changing its name to something unique, like
'StarLight' or something so as to avoid name controversy, and achieve maximum
popularity :)

BTW, this project looks great. I'll keep an eye on it... :)

~~~
daivd
Dammit.. I checked to see that Glow was not taken :). Almost every name, that
is not an outright racial slur is already used. Still, Java and Javascript are
completely different languages and that has never lead to much confusion..

~~~
quarterto

      Still, Java and Javascript are completely different languages and that has never lead to much confusion..
    

Not _within_ the tech world, sure, but ask say a recruiter and it's a coin
toss.

------
mej10
This looks awesome! I will be interested to see how well you can make the
features fit together. I have been really interested in languages that give
you tools for reasoning at multiple levels of abstraction.

Have you tried seeing how a full dependent type system could be used in your
design?

~~~
daivd
I plan to sneak in something similar with a capable type system and contracts
for stuff that you can't express in the types. That way you could "one day"
use a solver like Z3 and start proving or disproving some of the tractable
contracts, without having to even change the programs. A bit like
LiquidHaskell.

Not exactly dependent types, but practical, I think.

------
mwcampbell
Why not use an existing imperative language to implement the components?

~~~
andrewflnr
That's what he was going on about with the C FFI. You can use C to implement
components.

------
CmonDev
So yet another flow-based language/framework? NoFlo is another recent one.
Component-based is a very common modern game programming approach BTW. Google
for "component entity system". It's a very cool hobby project but will not get
wide traction I'm afraid...

