
Simplicity and Utility, or Why SOAP Lost - keithba
http://keithba.net/simplicity-and-utility-or-why-soap-lost
======
bradleybuda
I think this is largely accurate, but one thing the article misses is that
SOAP was never as interoperable as it was claimed to be. If you had a
Microsoft client and a Microsoft server, things would work great, but as soon
as you started mixing vendor implementations, you quickly ended up in
configuration hell. For simple SOAP messaging you could usually get it to work
with enough elbow grease, but for any of the WS-* protocols you might as well
be doomed. I lost days, if not weeks, of work to failed interop of WS-Security
across two different vendors (both Java, and both with the code under my
control).

~~~
silverbax88
I spent several years integrating Java/Linux/Unix/Microsoft/etc and I never
ceased to be amazed at how many companies/programmers thought it required
different code to make them interoperable. I had very few issues with SOAP
interaction between systems because they are just text files.

For all intents and purposes, SOAP is a formatted pipe connection. The only
issues I ever ran into were related to things like Java incorrectly
implementing RSA encryption or Microsoft adding so much complexity that nobody
used (and did not require).

There are still several major banks/insurance companies out there whose
payroll transaction systems run on the SOAP implementations I put in place 10
or more years ago.

Now I see people claiming REST is better (and in many ways, they are correct),
but I also see that large numbers of programmers don't really grasp REST any
better than they did SOAP.

~~~
twic
> I had very few issues with SOAP interaction between systems because they are
> just text files.

Did you write parsers for the messages by hand every time?

I did a bit of work with SOAP, and found interoperability to be a problem.
Encapsulation and signatures both spring to mind. But then, i was using
tooling on both sides. I shudder to think what dealing with SOAP messages
manually would have been like.

~~~
silverbax88
Why would I write manual parsing code every time? There already are/were XML
parsers in every language I needed to integrate. SOAP is just XML. XML is
easy. I've moved to REST implementations but I have just quite got why anyone
thought SOAP itself was anything complicated.

Microsoft's 'WS' implementations were a mess of SOAP, but that's a whole
different story.

------
Eric_WVGG
The S Stands for Simple —
[https://web.archive.org/web/20140710041718/http://wanderingb...](https://web.archive.org/web/20140710041718/http://wanderingbarque.com/nonintersecting/2006/11/15/the-
s-stands-for-simple/)

a web classic, can never be linked enough times

~~~
larrys
"can never be linked enough times"

Off topic question here. Does an archive.org article get cached so that when
it's linked to it comes up faster? Or does it display at the same speed of a
random archive.org page?

That page came up pretty fast for me don't know if that is random or because
other people are hitting the link from HN.

~~~
burnte
Archive.org does caching. A page is pulled from long term slow storage, and
cached.

------
nolok
The entire thing hide itself behind a "don't worry about how that works, the
tools and libs are supposed to do that for you" philosophy, and then those
would fail and break so often it was almost comical. You would then end up
debugging the entrails of a SOAP exchange, which is one of the most terrible
things I've seen.

------
tfederman
I feel like I spend a troubling amount of time as a software developer dealing
with and/or avoiding solutions that are much more complicated than the
problems they're trying to solve.

~~~
blt
tell me about it... I've been searching for a C++ library that will encode x86
instructions from syntax that looks moderately similar to assembler code. Not
a disassembler, not a JIT toolkit, not a special cross-platform IR that gets
compiled to real instructions. Just an instruction encoder with a moderately
pretty syntax. No luck so far.

~~~
s_kanev
Have you seen xed [1]? It seems to fit the bill nicely from what I understand
about your requirements.

[1]
[https://software.intel.com/sites/landingpage/pintool/docs/67...](https://software.intel.com/sites/landingpage/pintool/docs/67254/Xed/html/main.html)

~~~
blt
yes, I don't need its decoding capability and I don't think its syntax is
pretty but if I ever decide to write my own library I'll probably wrap xed in
some C++ sugar.

~~~
_dps
I'm not sure I understand your requirements but might DynASM be useful? It's
one component of the JIT library behind LuaJIT but many people use it for run-
time code generation completely outside a JIT setting.

[http://luajit.org/dynasm.html](http://luajit.org/dynasm.html)

~~~
blt
DynASM looks cool but its preprocessing step and fancy C integration
definitely place it outside the description "Just an instruction encoder with
a moderately pretty syntax."

I guess it's not fair to say that XED and DynASM are "much more complicated
than the problems they're trying to solve." They are much more complicated
than the problem _I 'm_ trying to solve. But I am surprised that there is no
minimal X86 encoder with nice C++ syntax out there.

------
ownedthx
The fact that SOAP is not reasonably supported in a browser was a huge reason
it fell down. When SOAP first came on the scene, complicated AJAX based
applications were not typical. But as more and more JSON/HTTP APIs emerged in
conjunction with browsers becoming more powerful and rich apps build built in
them, the meaninglessness & complexity of SOAP grew.

Also, another major issue with SOAP is that most all of the popular tools
would generate classes based on a WSDL. This creates a toolchain issue that is
readily solved for someone experienced, but can really suck if you are new to
the idea of generated code working it's way into your project.

Much worse was the scenario of WSDL versioning in conjunction with these tools
that generated classes. If the API never broke backwards compatibility, you
should be OK and can just use the newer class representations, counting on the
service and tools to deal with null fields appropriately (not always true
unfortunately). But if version bumps of an API/WSDL broke backwards
compatibility... then you had to have separate class hierarchies for different
versions of the WSDL; what an intense headache. Contrast that to web REST
APIs; without a formal schema and a lack of class-centric tooling, client
libraries would often let the author stuff in the params themselves and let
the serializer stuff in the body based on the params; so yes your API is not
as formal, but this isn't a big issue but offers infinite flexibility in
dealing with one-off versioning issues or interop issues.

As someone else mentioned, some platforms couldn't interop with others
(differences as it related to simply stuff like nullable fields or primitives
existed all over; total nightmare), and some features of WSDLs didn't
translate well to certain languages.

A great WSDL author would know to make their WSDL as simple as possible,
because they had spent time working with various language toolchains and new
the limitations out there, but that's asking way too much and not realistic
for someone to have to spend all their time trying to understand all the ways
someone in language XYZ might use their WSDL.

~~~
twic
My observation of the rise of JSON/HTTP was that it was a thing happening in
the dynamic language communities, and that only came into the Java world when
the battle was already over.

I suspect the dynamic language communities preferred JSON/HTTP so strongly
because it was so much easier to use: they didn't have big corporate backers
releasing SOAP tooling, but it's trivial to parse JSON straight into
structures that are more or less native to the language (arrays of objects in
JavaScript, lists of dicts in Python, arrays of hashes in Ruby).

Java did have heavy-duty SOAP tooling, and didn't (and doesn't) have a natural
way to represent JSON's objects, so it was much less of a win there.

~~~
hayksaakian
Not sure what you mean. What about

[https://github.com/douglascrockford/JSON-
java](https://github.com/douglascrockford/JSON-java) ?

~~~
twic
Given the JSON string:

    
    
        {"cities": [{"name": "London"}, {"name": "New York"}]}
    

Parsed into a variable called 'root' with either 'new JSONObject(...)' or
'json.loads(...)', compare:

    
    
        root.getJSONArray("cities").getJSONObject(0).getString("name");
    

To:

    
    
        root['cities'][0]['name']
    

The Python version is the sort of thing you find yourself writing in Python
anyway (at least, if you write tabloid-level Python the way i do); it's
reasonably idiomatic. The Java version is cumbersome and feels very un-
Javaish.

Bear in mind that with SOAP, in Java, by that point you would have mangled the
data into realistic-looking objects, and would write something like:

    
    
        root.getCities().get(0).getName();

------
treve
I think SOAP has lost in a sense that it's no longer very popular around many
web developers.

But has it really lost? I still see it used a lot in large corporate and high
tech environments.

I was never a big fan of SOAP myself, but that's also partly because when I
started, the tooling wasn't great and I got mostly a negative user-experience.
But this is many years ago.

My understand is that these days it's quite good, and it has a pretty good
user-experience. Furthermore, everything is backed by xml schemas, which makes
it easier to create and require strictly valid xml being sent back and
forward.

While some equivalents exist in REST-ish API's, it's arguably not very common
and with SOAP it's a pretty much a free and built-in feature that everybody
uses.

Modern web API design is great for getting things up and running fast, but
it's not amazing for rigorously robust design.

I don't really like the "SOAP is bloated and over-engineered" narrative. It is
uninteresting now, and was uninteresting 10 years ago.

I think it's much more interesting to learn about the things that SOAP did get
right, and see how we can apply some of these learnings in our API's.

~~~
delluminatus
I work on an SOA platform for a very large software company. We offer both
REST and SOAP versions of our RPCs. In my experience, SOAP services are okay
plug-and-play if you own both the provider and the consumer. But consuming
external APIs can be nightmarish, especially with SOAP 1.2. SOAP exists in our
organization for legacy reasons but we recommend all new consumers use RESTful
services.

Our mobile clients like REST because it's faster and easier to consume from
outside the .NET platform than SOAP. Even within the .NET platform, a client
can get up and running consuming a "RESTful" service in fewer lines of code
than a SOAPy one. Not only SOAP tooling, but HTTP client tooling in general,
is a lot easier to use these days.

WSDLs are cool because they are a fairly standard metadata format. Our clients
like them because they can perform data validation with proxy/firewall systems
before the request ever enters the corporate network. I've yet to see a
concrete benefit from that kind of strictness, though. IMO, SOAP schema
validation is so brittle that it causes more problems than it fixes.

Of course, you can also use non-SOAP document description formats like RAML
for describing your RESTful services. It's not like SOAP offers anything you
can't get for REST. But once you start adding all that on top of your REST API
stack, you might as well use SOAP 1.1. (Never use SOAP 1.2).

I don't think the main problem with SOAP is that it's over-engineered,
although it is. The issue is that the supposed benefit of "interoperability"
was never really realized. It's supposedly protocol-agnostic, but nobody
cares. It's supposedly interoperable, but everyone implements it differently.

~~~
treve
Very interesting! My knowledge was mostly based on hear-say, so it's great to
get some details from someone with actual experience with SOAP.

What's wrong with SOAP 1.2?

~~~
delluminatus
It's too confusing, I think. The effort it takes to try to understand what's
going on is too great. You can explain our REST service infrastructure to a
developer in a few hours. I don't know how long it would take to explain the
SOAP infrastructure, because I don't think anybody understands the whole
thing.

If you are trying to communicate with a vendor that uses SOAP 1.2, it almost
always seems to come down to guesswork. What set of properties do I need to
specify before they accept my request? The WSDL gives you an object schema but
it's not sufficient for the masses of WS-* extensions that you might have to
support. Meanwhile, the vendor just exposes the WSDL and assumes that's
sufficient "documentation" for clients and you don't need any examples or
explanations or other info. I find the best way to approach this is to open
Soap UI and start messing with settings until requests are accepted. There's
no point asking the vendor for documentation because they probably don't know
themselves.

Also, speaking from the perspective of a framework developer, I mainly start
to pay attention when things stop working or clients are having problems. SOAP
1.2 doesn't have any more problems, but the problems are proportionally harder
to solve. No matter how good your tooling is, it's not perfect, and it's a lot
easier to look "under the covers" for RESTful services than for SOAP services.
Of course that could also be attributed to the hellish nature of WCF, and not
SOAP 1.2 per se.

~~~
dotemacs
This. So true!

    
    
        ...it almost always seems to come down to guesswork.
    
        Meanwhile, the vendor just exposes the WSDL and assumes
        that's sufficient "documentation" for clients and you 
        don't need any examples or explanations or other info. I 
        find the best way to approach this is to open Soap UI and 
        start messing with settings until requests are accepted. 
        There's no point asking the vendor for documentation 
        because they probably don't know themselves.
    

Having just dealt SOAP API last week, I can relate to this so much. I'm glad
that I'm not the only one feeling the pain here.

------
seanalltogether
My guess is that more and more people are coming to the realization that xml
is great for fluid text and document formatting, but too ambiguous to serve as
a data structure format.

XML nodes and children are just too slippery. I've actually argued with
enterprise developers about why serving the following format was a terrible
idea.

    
    
      <user>
    	<id>123456</id>
    	<name>Joe</name>
    	<account>Account 1</account>
    	<account>Account 2</account>
      </user>

~~~
Igglyboo
Is there a hard and fast rule for deciding when something is an attribute and
when it is a child?

I would have personally done

    
    
        <user id="123456" name="Joe">
    	<account>Account 1</account>
    	<account>Account 2</account>
        </user>
    

but I've never actually produced XML, only consumed.

~~~
seanalltogether
Both ways still leave the structure ambiguous. Should <user> be mapped to an
associative array of mixed objects, or to a fixed object using a custom mapper
to shove the account objects into their own sub array.

~~~
treve
If you're looking for a generalized way to map an XML structure to primitive
types, you're going to have a bad time.

------
mpweiher
Not sure why the article equates XML with SOAP.

Request:

    
    
       GET /customers/43456 HTTP/1.1
       Host: www.example.org
    

Response:

    
    
       HTTP/1.1 200 OK
       Content-Type: text/xml; charset=utf-8
    
       <customer>Foobar Quux, inc</customer>
    
    

Seems fine to me. In fact, having named elements without the requirement for a
dictionary makes parsing straight to an object-representation (without an
intermediate property-list) much easier.

~~~
delluminatus
It never equates XML with SOAP. The point is that even while SOAP was being
pushed by Microsoft, JSON was gaining mindshare among API developers, and API
design was moving away from complex XML documents.

Using a simpler format for object serialization into XML is definitely an
option, and it's a perfectly fine middle-ground between SOAPy verbosity and
JSON compactness, but IMO it doesn't really have a lot to offer over a JSON
version of the same API. Many API providers support both formats using
content-type detection.

------
binarymax
In my opinion, WS-* lost not because of the format or verbosity. WS-* lost
because the concept is about the continuation of RPC, where you expose custom
methods that can be invoked from a client. When state is altered in custom
ways, complexity goes through the roof - to the point where you need a DSL
(WSDL) to define these methods and how to interact with them.

REST simplified things because the concept is about limiting the method
invocation to the bare minimum and always have an explicit understanding of
the state that is being changed or communicated.

The XML/JSON question is not as important as the method/resource question.

~~~
troels
The whole baked-in type system also adds a lot of complexity. On top of that
you get the complexity that is xml it self (namespaces, while theoretically a
good idea, are pure insanity).

------
Pxtl
“The essence of XML is this: the problem it solves is not hard, and it does
not solve the problem well.” – Phil Wadler, POPL 2003

SOAP takes XML's problems to the next level.

------
ChuckMcM
Hold on to this, _" tldr: Simplicity and utility trump large corporate
backing."_

When your manager tells you that your API needs to handle something "just in
case", when your customer asks, "But what if I want to use this API with
mumblefratz?", and when you look at your code and say, "Gee, if I make this a
variable I could handle any special edge case by encapsulating its specialness
in the variable ..."

------
lkrubner
Pete Lacey's parody of SOAP is both hilarious and also very accurate:

[http://harmful.cat-v.org/software/xml/soap/simple](http://harmful.cat-v.org/software/xml/soap/simple)

------
jcfrei
I've had the "fortune" to work with SOAP as well (java implementation) and I
never could shake the feeling that it was designed with the intention of being
obscure. It seems you could employ an entire batch of consultants just to deal
with defining, implementing and changing SOAP APIs (kind of a misnomer I
know). The whole change from SOAP 1.1 using an ActionHeader to SOAP 1.2 using
an action parameter seems downright misleading. Or downloading WSDL files to
automatically create Java classes sounds great in theory but now your program
has additional obscure classes with _very_ weird syntax and you have to read
through numerous reference guides just to know how you can modify the header
of an outgoing request. Such a waste of time.

~~~
ebbv
I also get to deal with SOAP occasionally. I recently got the joy of dealing
with it for a CA's API.

It's clearly the result of developers and managers more interested in what the
technology CAN do than what it should do.

~~~
ownedthx
In my experience, it wasn't that. Larger companies put value in supporting
standards, because it is an important checkbox in their marketing, and SOAP is
definitely a standard.

~~~
ebbv
I was talking about how SOAP itself was designed.

~~~
ownedthx
... oh :)

------
userbinator
One nice example of how MS really made things more complex with SOAP is the
MSN Messenger Protocol (MSNP), which started out quite simple and sane, then
got extremely SOAPy in the later versions; compare protocol interactions pre-
SOAP:

[http://msnpiki.msnfanatic.com/index.php/MSNP8:Getting_Detail...](http://msnpiki.msnfanatic.com/index.php/MSNP8:Getting_Details#Example_SYN_Responses)

and doing the same thing with the SOAP'd version:

[http://msnpiki.msnfanatic.com/index.php/MSNP13:Contact_List](http://msnpiki.msnfanatic.com/index.php/MSNP13:Contact_List)

~~~
im2w1l
I think this is a great example of SOAP's failing. While it _is_ simple, it
_looks_ difficult. As you scroll through all that XML your eyes begin to glaze
and you don't even notice the small section at the bottom. Despite being the
most important, the section is at the bottom, has weird grammar, and feels
like an afterthought.

>Here the WSDL and XML schemas for the web service descripted here, you can
use them to generate a web service binding for your programming language.

>MSN Addressbook/Sharing Service WSDL & XSD files

This is how simple it actually is:

[http://www.diveintopython.net/soap_web_services/index.html](http://www.diveintopython.net/soap_web_services/index.html)

~~~
userbinator
I wouldn't consider requiring a ton of mostly redundant and useless
information in messages (each namespace needs _a whole damn URL_ every time
it's referenced? Does the parser use it, or even care that the entire URL is
correct?) and XML parsing (which isn't _so_ bad, in comparison to all that
redundantly redundant redundancy) to be "simple" at all. WSDL/XSD is yet
_another_ layer on top of this mess of complexity and I suppose you're
referring to the fact that it _looks_ simple if everything goes according to
plan, but when it doesn't it's anything but simple. Trying to get any kind of
performance out of it is another counterpoint to "simplicity"... This is all
from the experience of someone who has worked with XMLA-based OLAP systems.

That link you gave has a summary part (
[http://www.diveintopython.net/soap_web_services/summary.html](http://www.diveintopython.net/soap_web_services/summary.html)
) that says "SOAP web services are very complicated."

------
Animats
It's all about who has the power.

With JSON, if the data returned by the server doesn't exactly match the
documentation, it's the client's problem.

With SOAP, if the data returned by the server doesn't exactly match the
documentation, the server is broken. The client, reading the WDSL file which
documents the API, will report an error and refuse to continue. This means bug
reports the server provider cannot make go away by the usual defect-denial
techniques.

------
debacle
[http://www.google.com/trends/explore#q=wsdl%2C%20restful&cmp...](http://www.google.com/trends/explore#q=wsdl%2C%20restful&cmpt=q)

SOAP is past its peak but far from dead. In the Microsoft ecosystem, SOAP is
nearly transparent, except for a bit of fiddling - you publish your WSDL from
your definitions, I consume it to generate an API class on my end, and after
maybe ten minutes on either side, we don't even think about SOAP again.

------
rayiner
It's funny that the joke starts with a dig at CORBA. When I was worked on a
CORBA-based system a decade ago, I thought I'd never see something more over-
designed. But this whole document-oriented web services thing makes me feel
like: [https://screen.yahoo.com/unfrozen-cave-man-
lawyer-1-22341242...](https://screen.yahoo.com/unfrozen-cave-man-
lawyer-1-223412426.html).

------
ultramancool
Sure, SOAP's complexity is a problem, but the issue with "RESTful" APIs is
that they're so loosely defined, almost all the way to the other extreme.
Being able to generate code and have a type-checked API, which is as easy to
use as any other local API was a very nice feature we've now lost. I wish
there were a better compromise.

~~~
Blackthorn
There is a better compromise (in fact, several):

* Originally, Google Protocol Buffers. These are still in wide use; unfortunately, they never published/blessed an official RPC stack.

* Apache Thrift, aka Facebook's answer to Protocol Buffers.

* Capn Proto, by one of the Protocol Buffers authors, now has an RPC standard!

After having used Protocol Buffers extensively, I could never go back to
untyped APIs. I'd at least use Apache Thrift for everything. Hopefully Capn
Proto gets more languages supported for its RPC soon.

~~~
blt
Protocol Buffers is great. Thrift generates much nicer C++ code but performs
slower and wants you to do everything inside their networking world. I'm
excited for Cap'n Proto. It's a good time to be doing binary IPC.

I'm still uncertain about melding the message format and RPC mechanics though.
Part of me feels like network communications is just too big of a deal to
abstract away into something that looks like a normal procedure call. But hey,
even if that's always true for big systems, it probably won't be for smaller
apps.

------
corbinpage
SOAP Web Services around still frequently used in a lot of large corporations,
where technology evolves a little slower and Microsoft products are still very
popular.

Being a Rails fan myself, I have always enjoyed using HTTP Web Services and
interacting with tech companies' Restful APIs so easily.

This post provided a clear, side-by-side comparison of the technologies
including the historical context of SOAP, which I found really useful and
enlightening. If I could give you some Bitcoin for writing it, I would. Have a
new Twitter follower instead.

It's hard to find succinct, objective technology articles in this day and age.
Most reviews I find are either unconstructively biased ("dynamic typing
sucks!") or too long and convoluted to be effective.

------
mrfusion
I always liked XMLRPC. It seemed really simple. At least in Python they simply
mapped to function calls and all of the XML stuff was handled for you. It took
minutes to get an API set up with that from any program.

I never understood why it's out of favor?

------
Yhippa
I still have to use SOAP more often than I'd like to these days. It's a pain
testing and dealing with all the different tools you have to use just to stand
up a basic request.

A lot of vendors I've worked with will publish REST web services for public
data but as soon as any private information needs to be accessed they use
SOAP. I don't know if they just don't trust the tools to make REST secure or
what but it's disappointing for sure.

Also the people who wrote the SOAP code are long gone which makes support for
them even more difficult when I'm trying to get bugs fixed. REST server code
has not been anywhere as bad to deal with.

------
markolschesky
SOAP is still common in healthcare as many IHE profiles rely on SOAP for
Cross-community document/data exchange. They are a pain to work with if you
don't have a toolset that handles it well.

------
bch
The article lists

1) JSON coming up to replace XML as a serialization technology, and

2) and Ruby on Rails (etc)

I remember even serious pressure from w/i the XML community in the sense of
XML-RPC, and the joke that the _entire_ _spec_ for XML-RPC was smaller than
the Table of Contents for the SOAP spec. SOAP was tough. It was confusing to
use (fwiw, I had the best luck w/ Perls SOAP::Lite) and difficult enough to
implement that devs I knew simply abandoned it.

------
kakoni
New Finnish gov digital infrastructure is going to built using Estonian X-road
(well next year). And yes, its only SOAP, so I guess either Finns are pretty
crazy or SOAP is making a comeback. Go figure.

------
jprince
The first thing I did as a developer was to write a SOAP wrapper gem for
Propertyware's API. It was here that I learned the concept of Pain.

------
fit2rule
Why I never used SOAP: Endian issues.

------
cratermoon
Let's hope SAML follows soon.

------
qwerta
Apples,oranges and orangutans.

Doing transactions and data format evolution in REST is hard, in SOAP it is
simple.

------
cowardlydragon
You can't argue that PUT, DELETE, and POST methods are usable in a browser,
and once a rest api starts going hogwild with headers, the GET really doesn't
work either.

The fact REST didn't provide some way for all methods to be invoked in a
simple browser was to me a step towards the inaccessibility of SOAP.

~~~
twic
Rails addresses this by interpreting a special _method parameter:

[http://api.rubyonrails.org/classes/ActionView/Helpers/FormTa...](http://api.rubyonrails.org/classes/ActionView/Helpers/FormTagHelper.html#method-
i-form_tag)

That is, a POST containing _method=delete will be interpreted as a DELETE.

It seems faintly silly; if you're exposing an API to the browser, why use
methods that you can't use directly and then have to go to the effort of
emulating them, when you could just use supported methods directly? Is the use
of the proper methods really that important?

But it does work.

------
entelechy0
Heh...we're using SOAP and WSDL for this current project...

~~~
Pxtl
You have my sympathy.

~~~
entelechy0
Thank you.

And believe me - it was certainly not my choice.

