

Open Sourcing super expensive proprietary software - mqsiuser
http://www.usethetree.com

======
mqsiuser
So I have been open sourcing the heart of a super expensive IBM software
product, which IBM currently/today sells regularly for 100 tsd (Dollar or
Euro, doesn't matter much) per CPU core (discounts of 40% to 95% usually
apply). This product is IBM WebSphere Message Broker or now IBM Integration
Bus (IIB) and was thought to be IBM's "SOA" offering. Let's say mildly that
SOA failed and unfortunately the remaining parts are not "a lot" (also not
worth that much). I was in fact able to understand them and it's been possible
for me to open source the very most important parts in a POC. So the very most
important part is the message processing and transformation language "ESQL".
IBM implemented it in c and I implemented it in Java. I called the project
"use-the-tree" and open sourced it on github. A deployed version is online at
www.use-the-tree.com.

Actually it turned out that I added a couple of improvements which makes my
implementation superior to the one IBM has.

There are open source ESB's and Broker's, but I did not find (and I think it
is unlikely that there are) comparable proper implementations.

I managed to work in this SOA trouble and Integration world for about 8 years
and I am the contributor of important components to IIB which I open sourced
as "GenericESQLUtilities". My contributions are confirmed (by IBM Hursley) and
by the community (on www.mqseries.net) and are actually used. Basically I
proved that xslt is a performance disaster and on the other hand I showed how
to properly do message transformation and processing. I also just implemented
stuff that IBM didn't know how to (implement), since they bought the product
from another company (NEON) and certainly had some brain drain during that
process.

Main areas of future work are (also documented in the git hub wiki):

    
    
      - Add support for JSON
      - Add all quirks that XML has
      - Investigate if JavaScript can be used (instead of Java) (and implement it)
      - Sorting (super cleverly with Tree Map ofc)
      - On-Demand Parser (would be awesome, probably to hard for 3 months)
      - Investigate on open source solutions for parsing flat files standards like EDI/EDIFACT or HL7/ORM formats and it's usability together with use-the-tree
    

The project is very new (a couple of weeks old), it's a POC and there is
nothing in production. Though it is confirmed to hit the point.

So... I have already successfully contributed to open source (in this area)
and there is constant change in the IT world (I see a clear decline of SOA and
a push towards open source), so that I think that this project can have a
significant success in the future. Again ... "use-the-tree" is the only thing
which is left of SOA if you remove all the bad parts or the ones that have
just failed... it is "SOA - the good parts" :). It provides strong guidance on
how to do message transformation (to end up with the most efficient
transformation code) as opposed to IBM's approach, which is "we provide you
with every possibility and you better choose wisely".

Xslt would look like it, if xslt would be done properly.

~~~
smoyer
SOA as a buzz-word has fallen out of favor, but SOA as a means of segmenting a
large problem to later solve it in "chunks" has been an accepted practice
since the dawn of engineering. I'll agree that Microsoft, IBM and Oracle
(Tibco, etc) would prefer that you were passing SOAP/XML over enterprise
busses, but our layer's of RESTful services are still a "Service Oriented
Architecture" ... the loss of statefullness was a big win.

~~~
mqsiuser
> I'll agree that Microsoft, IBM and Oracle (Tibco, etc) would prefer that you
> were passing SOAP/XML over enterprise busses

They better say bye to this idea... it's hard to see why curly braces are good
and angle brackets are evil... they get used differently.

> Enterprise (Service) Bus

Buzz word, snake oil

There is no problem, there never was a problem: Build on existing (de-facto)
standards and just don't reinvent the world (with WS-*). And do not use the
beginning of the payload (like SOAP does). The solution was there already and
the solution is HTTP and the 2048 characters of the URL (to encode a function
call) and yes a machine readable data format, but NOT XML and yes, JSON is
likely/probably good enought.

You usually pay the bill for what you are doing

Open Source is free (as in freedom) but also free of charge :)

~~~
AdrianRossouw
> yes a machine readable data format, but NOT XML and yes, JSON is
> likely/probably good enough

I thought so too, until I realized json is terrible for binary. on it's own it
can't reproduce the full capabilities of REST ie: http multi-part uploads.

You need a performant streamable binary format too, and my vote for that is
protobuf.

