Hacker News new | comments | show | ask | jobs | submit login

For a while, I've been pondering how to solve the same problem. I also reached similar conclusions, that parallelism needs to be implicit at the language level. I've even been working on an implementation that will run on the JVM (it's all of a week old for writing code for it, but I've been documenting the how it works for months in my notebooks). The language is based around the idea of stream processing. By using type signatures, function calls can be parallelized based on how the are called.

As an example, given two functions, A(Item) and B(List[Item]), if you call them like:

    B << A << List[Item](x,y,z)
the A should be called once for each item in the list of x,y,z, the results of which are collected into a list and passed to B. If you add in a function C(Item) and call it like:

    B << C << A << List[Item](x,y,z)
then at any given point, A can be processing a list item, C can be processing a list item and the total results of C are being collected. Which leads into, what if it is order dependent?

    B << serial { A << List[Item](x,y,z) }
will guarantee that A is called serially and the order of the List (and results) is preserved.

I really shouldn't advertise the link to the language at all here, considering I have zero documentation and there's almost no comments in the current code, but I'm going to anyways, https://bitbucket.org/ismarc/muster . It currently will parse a single file, generate the AST and generate the symbol table. I'm working on a non-parallel interpreter right now, with the intention of using the interpreter to write the compiler so the language is fully exercised to work out any potential issues with it. And on a final note, I'm targeting the JVM mainly for existing libraries, but I'm open to discussions about it.

I've been thinking about a similar thing, attaching Storm-like parallel semantics to Pipes (probably Gabriel Gonzalez's version) in the Par monad.

Some good ideas -- have you looked at Dryad?

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact