
Amazon States Language – A JSON-based language to describe state machines - another
https://states-language.net/spec.html
======
termie
"The Step Functions free tier includes 4,000 free state transitions per
month." .. "$0.025 per 1,000 state transitions thereafter"

So Amazon has created a new calling convention, where conditional logic now
also requires a context switch and JSON serialization. Then they charge you
for the call .. $tdcall?

This will certainly have some useful applications, maybe someone will build an
inexpensive data processing pipeline on top of it. Having seen many visual
workflow tools through the years, most are simplifying complex underlying
process, but these step functions and state transitions are modeling basic
internal control flow with complex abstractions and little benefit other than
retries with backoff.

The pure functional immutable nature of the data flow is ideally nice but
tainted by the JSON. The parallelism is interesting, but it seems bolted on
instead of a more powerful central part of the design.

~~~
mSparks
wait, what..!!

is that entire system state to state or per state?

i.e. if i have 100 states each transitioning 365 times bootstrapped a thousand
times

they'll charge nearly a million dollars or nearly a thousand?

noting that takes like a minute on my desktop.

------
DigitalJack
Why do something like this in JSON? I don't work in javascript, but surely it
would have been far simpler to use. Restrict it to a subset if you have to,
but this is absurd:

    
    
      "ChoiceStateX": {
      "Type" : "Choice",
      "Choices": [
        {
            "Not": {
              "Variable": "$.type",
              "StringEquals": "Private"
            },
            "Next": "Public"
        },
        {
          "And": [
            {
              "Variable": "$.value",
              "NumericGreaterThanEquals": 20
            },
            {
              "Variable": "$.value",
              "NumericLessThan": 30
            }
          ],
          "Next": "ValueInTwenties"
        }
      ],
      "Default": "DefaultState"
      }

~~~
murukesh_s
It looks like they want to make a User interface on top of it. It is very
difficult to make a user interface that reads and writes from a regular
programming language as there are multiple ways to write the same expression
(with space, without space, with new line etc).

~~~
mSparks
18 years late.

[https://en.m.wikipedia.org/wiki/Dia_(software)](https://en.m.wikipedia.org/wiki/Dia_\(software\))

~~~
murukesh_s
Not at all related. Amazon step functions are executable state machines, while
Dia is just a general-purpose diagramming tool.

~~~
mSparks
dia creates xml almost identical in nature to this json spec.

and if you need json, plenty of xml to json convertors knocking around.

------
ChrisRus
I have been working on a moon-shot project for more than a decade that seeks
to model distributed systems as if they were digital logic circuits. One of
the natural areas of interest is of course FSM's - in particular declarative
specifications of FSM's. I continue to believe (although it's a wildly
unpopular notion among some of my machine learning friends) that this
obsession with the mechanics of declaring design intent with models and then
synthesizing runtime code will fundamentally transform software engineering
over time. But, declarative FSM's aren't enough to make it practical IMHO.

I wrote a short essay about this work in which I argue that the software
engineering community needs to embrace the design methodologies and rigour of
hardware designers: [https://medium.com/@alpinelakes/on-monday-i-learned-i-
got-ac...](https://medium.com/@alpinelakes/on-monday-i-learned-i-got-accepted-
to-the-founder-institute-991cdd88275e#.20kj8ipp5)

See also: [http://blog.encapsule.org/early-encapsule-project-
history/20...](http://blog.encapsule.org/early-encapsule-project-
history/2004-encapsule-prototype-screen-shots/) (old code but same ideas as
what I'm building now in Node.js/HTML5 @Encapsule).

Several things I believe are actually essential to make use of any of these
ideas at scale:

\- There needs to be an ad-hoc extensible standard for notating serialized
data with markers, tags, semantics, metadata (whatever you care to call it).
It is not practical to do unsupervised feature extraction on internal message
streams. And, it's _insanity_ to write/test/maintain custom
validation/normalization logic.

\- Given the above, FSM declarations must be encoded with labels (as above) so
that generic code can easily affect interop.

\- Small FSM's are reasonably easy to comprehend. But, very few systems can be
modeled with simple FSM's. Rather, real systems can be modeled as complex
directed graphs where edges represent the flow of observable state from one
FSM to another (vertices represent individual FSM).

\- Given that real systems can be modeled using non-trivial graph models of
FSM (as above), building reusable components by splicing and dicing the graph
up is logically possible. But, this is not something that mortals can do by
hand. Considerable tooling is required to make it practical to design systems
like this.

If you're interested in these topics, and want to help, look me up @Encapsule.

~~~
jnwatson
You're about 30 years late. The programming with visual state machines started
with Harel and Booch and led to UML (see
[https://en.wikipedia.org/wiki/Shlaer%E2%80%93Mellor_method#/...](https://en.wikipedia.org/wiki/Shlaer%E2%80%93Mellor_method#/media/File:OO_Modeling_languages_history.jpg)).
Executable UML (xUML) is a thing. It just never caught on.

~~~
parasubvert
UML Statecharts are definitely an interesting model , being HFSMs, and thus
modular/extensible. But I think there is a lot to be said about context and
application - this kind of thing needs to have a revolutionary 'right place,
right time' opportunity.

Kind of like Hypertext and Hypermedia, which languished as an academic pipe
dream for decades with occasional commercial moments of brilliance
(Hypercard), until Tim Berners Lee figured out the right mix.

------
lacampbell
A new market to disrupt - undergraduate computer science projects as a
service.

Wait until you guys check out my hashtable implementation _in the cloud_.

~~~
automatwon
Is a KeyValue database not a hashtable in the cloud?

~~~
dom0
Hashtables are by far not the only way to associate a number of keys with
respective values. They have some nice properties and some rather ugly
properties (memory usage, growing/shrinking, iteration, seeding / table
poisoning). Most RDBMS (for example) tend to use trees instead for indices.
And then there's the whole category of tries.

~~~
novembermike
Yeah, but most of the large key-value stores are distributed hash tables.

------
michaelsbradley
See also SCXML and SCION:

[https://www.w3.org/TR/scxml/](https://www.w3.org/TR/scxml/)

[https://github.com/jbeard4/SCION](https://github.com/jbeard4/SCION)

And for background, see the pioneering work of Dr. David Harel:

 _A list of all of his papers_

[http://www.wisdom.weizmann.ac.il/~harel/papers.html](http://www.wisdom.weizmann.ac.il/~harel/papers.html)

 _A few on Statecharts_

[http://www.wisdom.weizmann.ac.il/~harel/SCANNED.PAPERS/Seman...](http://www.wisdom.weizmann.ac.il/~harel/SCANNED.PAPERS/SemanticsofStatecharts.pdf)

[http://www.wisdom.weizmann.ac.il/~harel/reactive_systems.htm...](http://www.wisdom.weizmann.ac.il/~harel/reactive_systems.html)

[http://www.wisdom.weizmann.ac.il/~harel/papers/RhapsodySeman...](http://www.wisdom.weizmann.ac.il/~harel/papers/RhapsodySemantics.pdf)

[http://www.wisdom.weizmann.ac.il/~harel/papers/Statecharts.H...](http://www.wisdom.weizmann.ac.il/~harel/papers/Statecharts.History.CACM.pdf)

[http://www.wisdom.weizmann.ac.il/~harel/papers/Statecharts.H...](http://www.wisdom.weizmann.ac.il/~harel/papers/Statecharts.History.pdf)

 _Prof. Harel the dreamer..._

[http://www.wisdom.weizmann.ac.il/~harel/papers/LiberatingPro...](http://www.wisdom.weizmann.ac.il/~harel/papers/LiberatingProgramming.pdf)

------
lobster_johnson
Interesting that they're using JSONPath, which isn't even specified formally
anywhere. The only other major implementor that I know about is Kubernetes,
which has some odd extensions for templating. (JSONPath itself, of course,
isn't very well designed in the first place.)

We're working on a new variant of JSONPath that we're hoping to publish as a
formal, comprehensive specification. It's essentially a superset of JSONPath
with some syntax warts fixed (like the need to start with $). I wrote a little
about it on HN a week ago [1].

[1]
[https://news.ycombinator.com/item?id=13032391](https://news.ycombinator.com/item?id=13032391)

~~~
devj
IBM JSONata is another open source alternative. Check it out:
[https://developer.ibm.com/open/jsonata/](https://developer.ibm.com/open/jsonata/)

~~~
murukesh_s
JSONata - weird name but seems like a good alternative to XPath.

------
Marazan
Remember when we used XML, and then people started making DSLs in XML and XML
was the worst and too 'heavy' and having to write schemas was enterprise and
awful and we are totes using lightweight schema free sexy JSON now?

I wonder what the new thing to replace enterprise JSON will be.

~~~
aindhaden
It's still going to be XML. JSON is fairly limited, it's literally the
serialization format for JavaScript variables, and that one size does not fit
all.

~~~
Marazan
I'm totally fine with XML. I'm just amazed at the cognitive dissonance anti-
Xml pro-Json peeps are starting to display.

------
tonylucas
This is interesting, has a lot of similarities (not surprisingly) with how the
state machine we build workflows on our bot platform, although of course ours
is specifically fixed around chat/messages as a key interaction point.

Major differences that I can see are we enable multiple functions to be sent
per state, and that the output data from any state is referenceable by any
other state, not just passing it down in turn through the states.

We support fallback states but in a different way, and don't support the retry
concept directly within the state language itself, has to be built as a set of
states to perform a loop to attempt a retry.

We don't support parallel stages, but do support branches, and remerging of
those branches.

Probably the final difference I can see, is one of our options when running a
function allows you to actually append additional states to the machine during
the runtime process.

------
Swizec
I did it better 5 years ago. [https://swizec.com/blog/a-turing-machine-
in-133-bytes-of-jav...](https://swizec.com/blog/a-turing-machine-in-133-bytes-
of-javascript/swizec/3069)

133 byte interpreter in JavaScript. Input is JSON specifying state name,
write, move direction, and next state. Turing machines, basically.

Mine was for fun, but why is Amazon doing this?

~~~
slowmovintarget
People needed a state engine for sewing Lambda functions together. For
example, try doing retry with exponential back-off in Lambda. You quickly run
into a number of problems that are difficult without an execution context
outside the Lambda itself.

Step Functions give you this external context for doing retry, conditional
trigger of downstream functions, parallel trigger of additional functions and
more. Execution time of a state machine can last for up to a year, so this
also gives you a way to do more than 5 minutes of work at a time.

~~~
Swizec
That's cool.

But is there no existing language that can be used to describe state machines?

~~~
slowmovintarget
:)

They needed a syntax that was easy to transform into usage of other Amazon
resources. I'm guessing JSON was by far the most straightforward for them, not
to mention that they've been using the system themselves for quite a while.
But I'm just guessing.

------
nighthawk454
Some more info on Tim Bray's site:
[https://www.tbray.org/ongoing/When/201x/2016/12/01/J2119-Val...](https://www.tbray.org/ongoing/When/201x/2016/12/01/J2119-Validator)

------
timbray
Glad to hear that so many people already did this (only better), years ago.
Nice to have company.

The only thing new or interesting about States is it has a product behind it
that implements it at scale, available now; give it a try.

I think it's quite likely that this syntax is state-machine assembler, and
smart people will find nicer expressions of this and compile them down.

In particular, some people prefer dependency graphs to explicit state machines
for this sort of thing.

------
trickyager
Amazon has also created a ruby gem to lint JSON state machines.
[https://github.com/awslabs/statelint](https://github.com/awslabs/statelint)

------
TeMPOraL
Wait, did they just made a Lisp in JSON?

~~~
c3534l
You wait, Google is going to create one in XML, then Apple is going to invent
a cool minimalistic pseudo-Lisp that doesn't require any brackets or colons.
Then Windows is going to try to create one that only runs on .NET. Finally,
someone will create a format that to be read by all of them, and someone is
going to implement another LISP in that format.

~~~
seanp2k2
The Apple one attempts to use natural language, the Google one gets
discontinued after it becomes somewhat well-known with a cult following, and
Microsoft tries to push a competitor until after the Google one fails, at
which point they adopt the standard even though their platform store is filled
with junkware and no one cares anymore.

~~~
grogenaut
Meanwhile the aws service is still there plugging away, like simple db,
because core infra in aws was built on it and they don't want to anger
customers.

------
hacker_9
_" This document describes a JSON-based language used to describe state
machines declaratively. The state machines thus defined may be executed by
software. In this document, the software is referred to as “the
interpreter”."_

Oh no.

~~~
snoman
Everything old is new again.

So, now that it is happening to me, am I allowed to apologize to the numerous
old-coders that tried to tell me this when I was coming up?

~~~
jessep
If you've seen this before, I'd love to know what came next last time. How did
this evolve and what were the issues with it?

~~~
jfoutz
Tooling. IDEs and special languages to generate the files for the state
transitions, and the body of the lambdas themselves. (reading this kind of
made me want to go write a little haskell dsl)

If they're a little undisciplined, they'll probably add stuff to implement
counting and comparison directly, to put a hard limit on loops.

I'd also guess an addition of a couple special tasks, perhaps append to log in
s3 bucket and continue, that perhaps come with a discount.

There are about 3 million old flowchart tools out there, any feature you see
tacked on is a candidate.

Tangentially, other organizations will be inspired by this, and implement
their own language in json, but this time they'll do it "the right way" then
you'll get a working group to try to reconcile all the competing standards.

Or maybe not. kind of what happened with XML though.

 _edit_

ah, here you go. [https://www.w3.org/TR/scxml/](https://www.w3.org/TR/scxml/)
that stuff.

------
devj
Wondering why they aren't using their own data format(Amazon Ion -
[https://amznlabs.github.io/ion-
docs/spec.html](https://amznlabs.github.io/ion-docs/spec.html)) instead of
JSON.

~~~
rurounijones
Nothing here require ION as far as I can see

------
flaviuspopan
This strikes me as the underlying spec for the new Batch service mentioned at
re:invent.

~~~
slowmovintarget
It's the underlying spec for the JSON description of state machines in AWS
Step Functions.

[https://aws.amazon.com/step-functions/](https://aws.amazon.com/step-
functions/)

~~~
flaviuspopan
.....and just when I thought I've seen all the new services, haha. Thank you.

------
neurostimulant
This could be useful. Currently there is no easy way to add delay when
invoking lambdas from an event. Perhaps this could be used to create a delayed
execution triggers (e.g. Invoking a lambda 10 minutes after a dynamodb entries
updated).

------
avodonosov
JSON? Somebody forbade s-expressions?

~~~
TeMPOraL
For some weird reason people these days believe curly brackets are the the
best brackets.

~~~
junke
It seems some people would rather die than use parenthesis and acknowledge
that _those damn lispers_ had a good idea, after all. When curly braces fall
out of fashion, what will come next?

I vote for "263D first quarter moon" and "263E last quarter moon", which
cannot be displayed here.

[https://en.wikibooks.org/wiki/Unicode/List_of_useful_symbols...](https://en.wikibooks.org/wiki/Unicode/List_of_useful_symbols#Science)

~~~
avodonosov
For me it's not curly braces vs parenthesis, but about unnecessary commas and
quotes in JSON. I would rather use EDN than JSON ([https://github.com/edn-
format/edn](https://github.com/edn-format/edn)) - no unnecessary noise. I
regard EDN as a kind of S-expressions.

------
rhizome
JATEOAS

~~~
bunderbunder
Context:
[https://spring.io/understanding/HATEOAS](https://spring.io/understanding/HATEOAS)

------
swehner
Doesn't look too pretty, ugly in parts. Something strange about it. The ARN
business looks very suspicious.

Maybe this is supposed to be pre-alpha?

Consistent with the Tim Bray blog article saying the validator's
"implementation is kind of gross."

But then again: stay out of trouble, avoid Amazon.

------
shoefly
Looks like any other brittle FSM. It's missing an important layer of logic.
And yeah, I'm bias because I'm finishing up an FSM that crushes that
brittleness.

~~~
rspeer
That doesn't tell us very much, and the word you mean is "biased".

------
elcct
Language? That's bold

------
bbcbasic
Bah! Why JSON not XML?

~~~
danielpatrick
... what is a single reason to use XML over JSON beyond legacy purposes?

~~~
bbcbasic
Schema. Validation.

~~~
jack9
JSON Schema validation exists (e.g. the IAB standards).

