
Streem – a new programming language from Matz - tree_of_item
http://github.com/matz/streem
======
c3RlcGhlbnI_
I am glad that someone is working on a new stream processing language, it is a
very interesting paradigm. However I hope that they provide some very robust
tools for controlling input splitting. As I have spent too much time fighting
with awk and wishing it was more flexible(it is frustrating always having
exactly two levels of splitting with only matchers on the first and only
inverse splitting on the second).

As is you would have to put in filters to resplit input into lines and that is
very messy for something that you will need/want to do very often.

For example if you wanted to parse by character it would be wonderful to be
able to do the following:

    
    
      STDIN | /./{|c|
        # stuff
      }
    

Even better would be if you took it a step further and offered something like
regex pattern matching for the block input. e.g.

    
    
      STDIN | /\w+/{|word|
        /house/ {
          # when word is house
        }
        /car/ {
          # when word is car
        }
        {
          # default case
        }
      }

~~~
bglazer
I'd like to recommend a python command line utility called pyp. It allows you
to do python string manipulation on text streams using the standard pipe
operator.

[https://code.google.com/p/pyp/](https://code.google.com/p/pyp/)

~~~
vram22
Yes, pyp is interesting. So are some other roughly similar Python tools for
doing pipe-like stuff. I had blogged about some of them here, a while ago,
including pyp, osh, and pipe (not the Unix pipe system call, but a Python
module):

Some ways of doing UNIX-style pipes in Python:

[http://jugad2.blogspot.in/2011/09/some-ways-of-doing-unix-
st...](http://jugad2.blogspot.in/2011/09/some-ways-of-doing-unix-style-pipes-
in.html)

And that later inspired me to create an experimental tool called
pipe_controller:

Swapping pipe components at runtime with pipe_controller:

[http://jugad2.blogspot.in/2012/10/swapping-pipe-
components-a...](http://jugad2.blogspot.in/2012/10/swapping-pipe-components-
at-runtime.html)

The above post links (recursively) to a few others, also by me, describing a
couple of other ways of using pipe_controller, including this one:

Using PipeController to run a pipe incrementally:

[http://jugad2.blogspot.in/2012/09/using-pipecontroller-to-
ru...](http://jugad2.blogspot.in/2012/09/using-pipecontroller-to-run-
pipe.html)

and also includes the link for it on my Bitbucket account.

There's also plumbum, another such Python module.

------
Argorak
If you ever meet Matz, talk to him about programming languages. While he gets
some flak for all the problems of his original hobby project (Ruby), he
obviously loves programming languages and gives things more thought then
people give him credit for. I had the chance to talk to him while I was still
a student and full of ideas how the language could be made "better" and he
shot them all down. For good reasons, as I know nowadays. So I always love
seeing him building languages.

I shared a small story about him and languages quite a while ago, I guess it
fits here as well:
[https://news.ycombinator.com/item?id=6562979](https://news.ycombinator.com/item?id=6562979)

~~~
danso
Does he really get a "lot of flak" for Ruby? While I know not everyone loves
Ruby, it seems crazy to me that people would denigrate Matz on a personal
level...to me (admittedly, a novice in designing languages), Ruby always
seemed well-thought out...that is, the trade-offs do not seem out of line
given the philosophical benefits, and not everyone can make claim to turning a
personal project into a worldwide language.

Also, he seems like a nice guy, not the type to be drawn into the kind of
flareups in which he would draw flak.

~~~
gaius
I don't think so many people have a problem with Ruby the language so much as
the culture around Rails.

~~~
slapresta
What would be the point of blaming Matz for that?

~~~
gaius
Well, quite.

------
pluma
*Yukihiro Matsumoto, creator of Ruby

I'm not sure everyone is familiar enough with Ruby to know who Matz is.

~~~
chc
If they don't know who Matz is, are they likely to know who Yukihiro Matsumoto
is?

EDIT: Interesting fact: I believe this is now my most downvoted comment in
five years on HN, at effectively -7. Never would have guessed. I'm not exactly
sure what it says, but I thought it was an interesting data point.

~~~
kaeluka
I, for one, remember the full name but not "Matz"

~~~
snlacks
That's where I am, I didn't realize it until I got down to the copyright.

------
vessenes
Why do most implementations of FizzBuzz special case % 15? I haven't ever
really understood this. Maybe it's just my math-y background, but it always
seemed to me you should just check mod 3 and mod 5 without an else between
them, concatenating Fizz and Buzz.

Can anyone else comment on this? Most canonical FizzBuzz programs special case
15, and I don't get it.

~~~
masukomi
just because 15 HAPPENS to be a concatenation of the output of 3 and the
output of 5 today doesn't mean it will be tomorrow. If I said "say Fizz for
multiples of 3, Buzz for multiples of 5, and 'this is a silly coding problem'
for multiples of 3 and 5 then you'd have to rewrite your code.

Some of us know that clients ALWAYS change their minds, specs are rarely
equivalent to the end result, and code against future changes that are trivial
to account for in advance.

~~~
Doji
For any implementation you can manufacture an example which will require a
person to re-structure their code.

Additionally, using %15 is not DRY. If the spec changes from saying "Fizz" on
multiples of 3 to saying "Fizz" on multiples of 4, then you will have to also
update 15->20\. If you forget to do this, you have a bug.

The correct implementation is dependent on the problem's context, and such
context is not available with the FizzBuzz problem.

~~~
masukomi
Part of my point (which i obviosuly didn't communicate well) is that Fizz
concatenated with Buzz is a premature optimization. It's the developer taking
advantage of a linguistic coincience. The instructions are to output 3
different strings based on %3, %5, or %3 and %5. I have never seen a set of
fizzbuzz instructions that actually specified that the last option should be a
concatenation of the 1st two. It's always specified as a 3rd string that
people independently notice is a concatenation of the two.

------
apoorvai
I'm not really a programming language expert, but it seems to me that having
an implementation being the spec wouldn't be a good idea. If the Streem
implementation has a bug, then the bug becomes the authoritative behavior. Any
platform specific quirks would also make it difficult to have defined
behavior.

~~~
ianlevesque
Yep, welcome to Ruby!

To be fair, a spec with tests was reversed out of the ruby implementation[1],
so things have improved a bit.

1\. [http://rubyspec.org/](http://rubyspec.org/)

------
weeksie
At the very least, it's going to be amazing to watch a master language
designer build a new language from the ground up.

That said, I'm incredibly optimistic about a new Matz language. If I was going
to guess, the syntax will be much lighter and the semantics will make VM
optimization much easier than in Ruby.

~~~
alphonse23
dude, the man publishes 3 files, claims he's going to make a new programming
language, and it hits the front page of hackernews -- Matz has some power
there.

~~~
chrisduesing
It's not power, it's credibility. If someone who has never done anything says
they are going to accomplish something large people are skeptical. If someone
who already created a successful and highly adopted programming language says
they are making a new one people are rightly intrigued.

------
samuell
Would be cool if it would incorporate some thinking and concepts from Flow-
based programming [1], as that is AFAIK the most comprehensive architecture
covering all the aspects of asyncronous concurrent processing that one might
run into (multiple in/out-ports, channels with bounded buffers, sub-stream
support, etc etc).

[1] [http://www.jpaulmorrison.com/fbp/](http://www.jpaulmorrison.com/fbp/)

[2] [http://en.wikipedia.org/wiki/Flow-
based_programming](http://en.wikipedia.org/wiki/Flow-based_programming)

------
skrebbel
Reminds me a lot of Elixir's |> operator, which does the exact same thing.
Nice! Curious how it'll turn out to compare with Elixir on other areas.

~~~
mjt0229
I've never heard of Elixir, but I always assumed that the |> originated in F#.
Ocaml has it pretty much standard, too, although you can define it in one line
in Ocaml:

let (|>) x f = f x

~~~
reirob
Also in Haskell [0]:

    
    
        (|>) :: Seq a -> a -> Seq a
        O(1). Add an element to the right end of a sequence. Mnemonic: a triangle with the single element at the pointy end. 
    

It seems to be in the containers package since 2005: [1]

[0]:
[http://hackage.haskell.org/package/containers-0.5.5.1/docs/D...](http://hackage.haskell.org/package/containers-0.5.5.1/docs/Data-
Sequence.html#v:-124--62-)

[1]:
[https://github.com/haskell/containers/commit/1e61853dbd4b9fc...](https://github.com/haskell/containers/commit/1e61853dbd4b9fc6ecc58ad655990ea5e971fa13#diff-a4090fc3521182bd8d1ac467d170c6aaR40)

~~~
platz
Which has a totally different meaning from the F# usage.

'|>' from F# is the same as 'flip $' in haskell, or (&) imported from the lens
library

------
lgleason
I love Ruby and I love Matz. With that being said there are some things that
Ruby struggles with. I know that there have been some conversations among the
core on bringing in more functional concepts to Ruby....at least since April.
To me this says that Matz is coming to the conclusion that we may need a new
language to get functional right.

While I am sad to see that Ruby may be superseded by a new language I'm really
happy to see Matz leading the way with one of the solutions. In the Ruby
community we have a expression "Matz is nice and therefore we are nice". That
has set the tone for the community in ways that have never been the same in
some of the others.

As someone who has had the opportunity to talk with Matz on multiple occasions
and work with the Ruby community it would be great to see this as a natural
evolution of Ruby and the people who love it... As I have started to move on
to working with more functional languages etc. I have started to move away
from doing Ruby, but if the community can continue on and evolve with a new
language that would be awesome!

~~~
emmanueloga_
> "Matz is nice and therefore we are nice"

This non sequitur annoys me. Deconstructing it:

* A: "Matz is nice": Let's say we all agree this is true.

* B: "we are nice": i.e., the ruby community is nice.

* P(A -> B): (A therefore B) is a slogan, so I assume the proposition P is believed to be true. Is it?

In order for P to be true, the only option is for B to be true as long as Matz
keeps being nice. Assuming Matz is still good-natured, is not hard to find
counterexamples for B (every big community have some less-than-nice people).
So the facts tell us that P is false.

Alternatively, if you assume that Matz is _not_ nice, then, without mattering
if "we" are nice or not, the slogan holds true (modus ponens [1]).

Anyway, my point is that the slogan is as silly as this rant :p.

1:
[http://en.wikipedia.org/wiki/Modus_ponens#Justification_via_...](http://en.wikipedia.org/wiki/Modus_ponens#Justification_via_truth_table)

~~~
MarkSweep
I don't understand the original quote to be a proposition. I think what it's
trying to say is "Matz is nice so we too aspire to be nice". It's not
descriptive, it's perscriptive.

------
dwash
"Copyright (c) 2015 Yukihiro Matsumoto" \- bit early, ey? ;)

~~~
michaelmior
Why? I think it's pretty normal to drop this in as boilerplate on new
projects.

~~~
callahad
Note the year ;)

~~~
michaelmior
Ha! You got me there :)

------
mostafah
And... here’s an implementation:
[https://github.com/mattn/streeem](https://github.com/mattn/streeem)

Well, that was quick.

~~~
rurounijones
Heh, got to love the repo message "Sorry, Sorry"

------
luckydude
We did a similar thing to the awk source. Made awk scripts first class, you
could pipe them to each other.

------
mijoharas
Can someone help explain what's going on here: \"([^\\\\\"]|\\\\.) _\ " [seen
here in
context]([https://github.com/matz/streem/blob/master/src/lex.l#L49](https://github.com/matz/streem/blob/master/src/lex.l#L49)).

Now it seems to be finding literal strings (so "strings" e.t.c.). That would
explain the literal double quotes on either side. so without that we get:
([^\\\\\"]|\\\\.)_ so zero or more repeating versions of [^\\\\\"]|\\\\.

What I don't understand is why there is the explicit or \\\\. construct there,
as this seems unnecessary. Am I missing something? also, why does it seem that
strings cannot have either literal \ or literal " in them?

~~~
zb
Because if there's a \ you want to skip over the next character, even if it's
a ", but if it's not escaped then you want " to be the end of the string.

~~~
mijoharas
You're completely right, I think I had read it as \\. rather than \\\\. and
thus didn't understand it. Thanks for helping me get it straight in my head.

------
programminggeek
I like the idea of dataflow or stream processing ideas. I would love if you
could make the connector pieces smarter so that you were enforcing a contract
between the piping mechanisms. I believe you could build some very interesting
systems with that approach.

------
chmartin
Awesome I hacked something like this together a couple years ago using ruby
and gnu parallel

[https://github.com/charlesmartin14/gnu_parallel](https://github.com/charlesmartin14/gnu_parallel)

it is badly needed

------
hdmoore
This is an excellent example of using a parser as a language. Whether it has
any legs depends on whether it beats existing tools on some front (sed, perl,
ruby, etc). The concurrent angle is interesting, but I have found a multi-
process approach to stream data to be more efficient than most concurrent
single-process implementations. For example, with DAP
([https://github.com/rapid7/dap](https://github.com/rapid7/dap)), we found
that GNU Parallel + Ruby MRI was more effective than a concurrent language
such as Go.

------
panic
Tab ([https://bitbucket.org/tkatchev/tab](https://bitbucket.org/tkatchev/tab))
is another interesting recent text processing language.

------
zvrba
Take a look at dss from AT&T AST tools:

[http://www2.research.att.com/~astopen/dss/dss.html](http://www2.research.att.com/~astopen/dss/dss.html)

[http://www2.research.att.com/~astopen/publications/dss-2004....](http://www2.research.att.com/~astopen/publications/dss-2004.pdf)

------
bnegreve
It's definitely a good idea. Pipes are both very powerful and very simple to
use and debug, yet they are not very common in general purpose programming
languages (examples?). I'm not surprised that someone is trying to build a
language around them. I'll follow that, but for now it's a bit too early to
judge.

~~~
eggie
See node.js streams.

------
golemotron
Great idea, but I'm a bit disappointed in the syntax. Composed chains in Ruby
are much nicer to look at.

------
nchuhoai
For people interested, you can do something like this in Ruby right now that
has nearly the same syntax:

[http://pragdave.me/blog/2007/12/30/pipelines-using-fibers-
in...](http://pragdave.me/blog/2007/12/30/pipelines-using-fibers-in-ruby-19/)

------
eccstartup
I hope it will support really big integers as in Haskell and better floating
point number calculation, although I cannot foresee the power right now.
Because it is concurrent, it may be very useful in scientific computing,
whether in small or large scale.

------
ksherlock

        true{TRAIL}	return keyword_false;
    

Well, this will be fun to debug

~~~
falcor84
This is indeed one of the nicest typos I've seen, but note that there's
already a pull request. I suppose bugs can become shallow when your 3 file
repo reaches the first page of hn.
[https://github.com/matz/streem/pull/4/files](https://github.com/matz/streem/pull/4/files)

------
kricstta31
-> in clojure or hy deals with that nicely:

(-> (read) (eval) (print) (loop))

using python-sh in hy this is possible:

(-> (cat "/usr/share/dict/words") (grep "-E" "^hy") (wc "-l"))

------
1qaz2wsx3edc
Matz, now get started on implementing transducers!
[https://www.youtube.com/watch?v=6mTbuzafcII](https://www.youtube.com/watch?v=6mTbuzafcII)

------
grandalf
Stream processing languages typically derive from SQL (streamSQL) or prolog
(Rapide) ... This one doesn't seem anywhere near as powerful but who knows.

------
vlucas
Interesting that he chose a C-like syntax after going the complete opposite
direction with Ruby.

~~~
coldtea
Huh? Ruby also has a C-like (algol derived) syntax.

The replacement of "}" with "end" etc, is a trivial replacement, not a
different type of syntax.

Lisp, Prolog, etc, would be a non-C syntax.

~~~
stormbrew
Obviously you can flatten this inheritance tree any way you like, but while
all C-likes are obviously algol-likes I do think it's useful to consider
C-like a distinct branch within that.

That said I think you probably want more than just curly braces to define
C-like even then.

------
NanoWar
Now it might be time to have a look into YACC, LEX and maybe BISON and follow
Matz' repo :)

------
pmalynin
Looking to hire a Streem professional, must have at least 5 years experience
with Streem.

~~~
enry_straker
Great. I have been streem-ing for more than 15 years now. :-)

------
10098
I see this as a very good alternative to traditional shell scripting
languages.

~~~
Dirlewanger
Anything that beats Bash's ugliness I'm all for.

~~~
10098
it would also be nice if it had a static type system, but given that it's
Matz, it's unlikely...

------
leke
What kind of areas could Streem be used in?

------
randyrand
Why is Stream spelled incorrectly?

~~~
faitswulff
Easier to search for, at least.

------
patrick73
Sorry, but if you know nothing about Ruby this is pure hype.

------
retr0h
Not a fan of the closing braces.

~~~
k__
Didn't Matz mention that he wasn't either, when explaining Rubys syntax?

I'm a fan of indentation based blocks, so this seems like a step back for me
:\

~~~
randyrand
What does it matter?

------
faragon
"Then said Jesus unto him, Put up again thy sword into his place: for all they
that take the sword shall perish with the sword" (Matthew 26:52, King James
Version)

Sony played hard many times, e.g. closing Linux on Playstation 3 devices. I do
not condone the attack, although I have no sympathy at all for such kind of
companies.

------
glibgil
Whatever,
[https://github.com/ekmett/machines](https://github.com/ekmett/machines)
[https://github.com/scalaz/scalaz-stream](https://github.com/scalaz/scalaz-
stream)

~~~
vlunkr
This is newsworthy because Matz is developing a new language, not because such
a thing doesn't already exist.

