
Supdate – A Clojure library for transforming nested data structures - tosh
https://vvvvalvalval.github.io/supdate/
======
lvh
This is neat! When I first started reading and it described itself as
"transforming nested data structures" I immediately thought of specter, which
we use quite a bit at Latacora. We've published a library, eidolon[0], with
common specter navigators.

To get a feel for the difference I translated a few of the examples to what
they would look like in specter.

A few things I learned:

\- supdate is very useful if the data you have is already regular and not too
convoluted. By comparison, most of the time the data I pass to specter is a
giant (>100MB serialized and compressed) rat's nest (data from every AWS
service under the sun).

\- supdate is very good at this "implicit projection/selection" (in the
relational algebra sense). It feels a lot more like Datomic pull syntax that
way than specter. By comparison, the equivalent specter thing made me write a
pretty complex multi-path with submap to get the same behavior. By comparison,
specter feels better at the logic programming equivalent of this: in specter,
I express the data I want to address and specter figures out where it lives;
in supdate, I tell you exactly where it lives, structurally.

\- Implicit selection (per previous point) feels like something that's maybe
most valuable for data representation, so, UIs? To quote Perlis, "It is better
to have 100 functions operate on one data structure than 10 functions on 10
data structures.", so something I have done a few times with specter is write
a navigator for commonly accessed things (like the actions in an AWS IAM
statement) and then reuse that all over the place. It's harder for me to see
how to do that in supdate, but I guess it's all data so it can't be that hard
:)

In conclusion: I think it does what it says on the tin. There are things that
it can't express (or at least I can't figure out how), but it's very clear at
expressing the things it can. I probably won't end up using it, but that's not
an indictment of the tool as much as it tells you what we use specter for :-)

[0]:
[https://github.com/latacora/eidolon](https://github.com/latacora/eidolon)

------
vnorilo
Libraries like this are an argument for dynamic types. In Clojure, all
libraries work with associative or sequential collections, which is why you
can compose supdate with literally anything.

We do lose the embedded proofs that statically typed APIs give. We regain some
of the confidence with clojure.spec'ing them.

I wonder if there's a way to make a compile time spec - enforce contracts at
API boundaries, but let stuff like supdate work in a highly generic way in
between. C++ templates and Rust traits could work, but everyone would need to
agree on a core set to use.

~~~
jdc
Regarding compile-time spec checking — Spectrum is working toward just that.
[https://github.com/arohner/spectrum](https://github.com/arohner/spectrum)

~~~
kimi
Spectrum would be great, but I think it's still way off from being usable.
Taking a line from Specter's page, static analysis that catches idiotic bugs
(es mis-typed keys) would really be "Clojure's missing piece". I have been
trying it on and off for a bit of time now, but it still breaks on anything
but the simplest cases.

------
disalvjn
This looks like a fantastic library for updating nested data structures --
something that's definitely been a pain point in writing Clojure.

I wrote a library that complements this nicely -- it's for transforming one
deeply nested data structure into one that has a different shape, for example:

(f/transform {:a [1 2 3] :b [2 3 4] :c [5 6]} {k [vs]} {vs #{k}}) => {1 #{:a},
2 #{:a, :b}, 3 #{:a, :b}, 4 #{:b}, 5 #{:c}, 6 #{:c}}

[https://github.com/disalvjn/faconne](https://github.com/disalvjn/faconne)

------
nisa
Love it! This looks like it solves a problem I have for an community project
(parsing a 2mb convoluted json with status data for 100+ devices) in an
elegant way!

What skills does one need to have to be able to write such libraries for
yourself? I'm just feeling like a mediocre copy&understand&adapt&paste
programmer but rarely I'm able to create unique concepts/software.

~~~
vnorilo
Exposure to several different ways of solving similar problem sets could help
you think outside of the box (or in many boxes at once, at least). Clojure is
a great choice for one box. Maybe look into what Haskell or ML do with type
systems, or what Smalltalk does with message passing. I picked these to
complement Clojure's strenghts.

------
GorgeRonde
What a coincidence. I've been working on a macro to do exactly this yesterday
and think I might have a working version tonight or tomorrow. In Clojure too.

The only difference is in the way I decided to treat arrays

    
    
        (xxx {:a [1 2 3]}
             {:a [_ _ inc]})
        ;; {:a [1 2 4]}
    

This is because I intend to use it on arg vectors rather than proper data. So
I'm fine with having to access everything step by step from the left and
having a one to one relation between leafs in the data tree and leafs in the
transformer tree. Actually I wrote a cowalk function (as well as coprewalk,
copostwalk, etc), to allow for traversal and editing of multiple data
structures at the same time provided they have the same shape.

~~~
valw
Interesting! In supdate the transform would be:

{:a {2 inc}}

------
positr0n
This looks like it enables functionality very similar to specter.
[https://github.com/nathanmarz/specter](https://github.com/nathanmarz/specter)

Except

1\. Much easier to grok syntax/easier learning curve

2\. Not as powerful/generic

~~~
valw
As the author, I fully agree with this description!

The cost of abstraction should not be ignored, and that's why a lib like
supdate sometimes makes sense before reaching out to Specter, lenses etc.
(although I love those too!)

------
neilv
Some related work to look at, if you're interested in approaches like this,
are the pattern-based syntax transformers from Scheme (e.g., early `syntax-
rules`) and Racket, and the XML transformation languages.

------
Syzygies

        (require '[clojure.string :as str])
    

Infix, really? I love both Lisp and Haskell. Code like this makes me wonder
why I ever strayed from Haskell.

~~~
fiddlerwoaroof
It’s not infix, but keyword args which have been part of lisps since at least
the cltl2

