
Specter – Clojure API for immutable programming (2017) - amgreg
http://nathanmarz.com/blog/clojures-missing-piece.html
======
tekacs
Specter is tremendously useful and rarely lets on a case where it can’t
perform a transformation. For ensuring that you’re getting the most efficient
form of a transformation [0] it’s incredible —- and it is somewhat data-
oriented, allowing easy saving and reuse of navigators.

Meander [1] is also wonderful. Its unification-based approach makes for nice
expressions of intent. Its strategy-based approach to rewriting [2] is more
flexible than most common walker libraries, too.

[0]:
[https://github.com/redplanetlabs/specter/wiki/Specter's-inli...](https://github.com/redplanetlabs/specter/wiki/Specter's-inline-
caching-implementation)

[1]:
[https://github.com/noprompt/meander](https://github.com/noprompt/meander)

[2]:
[https://github.com/noprompt/meander/blob/epsilon/doc/strateg...](https://github.com/noprompt/meander/blob/epsilon/doc/strategies.md)

~~~
involans
So specter is lens and meander is syb/generics. Nice

~~~
devin
I know the author of meander, and I kind of doubt they'd describe it this way.
I will ask them, though.

------
john-shaffer
Specter is fantastic. It makes transformations on deeply nested data
structures easy. Want to transform all the "datetimes" from JSON data into
_actual_ datetimes (or vice versa)? It takes about five minutes if you have to
look everything up. Works with ClojureScript too.

Some production code:

    
    
        (defn transform-tagged-values [x]
          (sp/transform (sp/walker tr/tagged-value?) tagged-value->cljs x))
    

Give it a predicate for which values to transform, and a function to do the
transformation. Couldn't be easier.

~~~
narwally
It really does remove multiple layers of complexity that usually stand between
your mental model of what your transformation function should do and the code
that actually implements it. With Specter if I can clearly articulate the
transformation I want to do on a data structure, then I can almost always
translate that one-to-one into the code that does the transformation.

When I started using it in one of my existing projects I was able to
completely eliminate a few needlessly long and complex functions that were
littered with comments and calls to helper functions. In their place are
functions that are about 5 lines of code that barely even need a docstring
because it's readily apparent what they do just by looking at the source.

------
boxed
I invented instar
([https://github.com/boxed/Instar](https://github.com/boxed/Instar)) which is
quite similar to specter (imo a little bit nicer) because I also thought the
built in API was weirdly clunky in these respects.

~~~
didibus
What's the performance vis-a-vis Specter?

~~~
boxed
I have never run a benchmark and the specter guy really cares about it so
instar is probably way worse :)

I was more trying to show that the API could be better and hoping this would
put pressure on core devs to do it properly.

------
setzer22
Specter is always on my clojure toolbox. Not only it makes transforming
arbitrarily nested data painless, it also is usually more efficient than the
average hand-rolled code!

A great example of why macros in Lisp are powerful, and how they empower you
as a user even if you never write one.

------
jdc
Previous discussion of Specter:
[https://news.ycombinator.com/item?id=10295060](https://news.ycombinator.com/item?id=10295060)

also cf

Glom for Python:
[https://sedimental.org/glom_restructured_data.html](https://sedimental.org/glom_restructured_data.html)

Hashdig in Ruby's standard library: [https://ruby-
doc.org/core-2.7.1/Hash.html#method-i-dig](https://ruby-
doc.org/core-2.7.1/Hash.html#method-i-dig)

------
cutler
So it turns out that Clojure is the ultimate data-wrangling tool yet there's
no chance of earning a living with it. So much for innovation. Technology is a
strange industry where conservative Java bondage and a 2-bit browser scripting
extension with no stdlib reign supreme.

~~~
hota_mazi
Clojure is wholly inadequate to meeting challenges of the industry today. Not
just because it's dynamically typed (which means tooling for it is close to
nonexistent and it has terrible performance), but also because its ecosystem
is a ghost town.

It takes more than just a language to create a productive development
environment in 2020, especially one which is so flawed by designed to the
point of not even being statically typed.

~~~
dunefox
Yeah, just like Common Lisp and Julia have terrible performance. Extremely
slow, can't do anything with them.

------
mrbonner
Darn! This is sleek!

I have been learning Clojure off and on for almost a year now. I gotta say
that it has changed my mind to a "Aha, I got lisp now" moment. Working with it
is a joy, especially it integrates so well with the JVM and all available Java
libraries out there.

I can't believe I have missed Specter during my learning journey. How could I
not find out about this soon enough. I mostly deal with static typed languages
and seeing the primary use of map and keywords as main data structure scares
me. Rich Hicky said not to put too much emphasis in what in the map but the
data/keywords you need, kind of like you do not care what in the UPS truck as
long as it could deliver your package, right? But, navigating maps and nested
data structures around is the pain. Specter definitely easies that pain a lot
now that I know it!

This is why I am grateful for hackernews. Once in awhile, browsing the site
and discover something like this makes me rethink the ways I do stuff. It also
reminds me that I could be missing a lots of similar goodies on my way
learning Clojure. I am prepared to have me mind blown again!!!

------
slifin
There's a lot of alternatives in this space

Meander was the last one I can remember:

[https://www.youtube.com/watch?v=9fhnJpCgtUw](https://www.youtube.com/watch?v=9fhnJpCgtUw)

------
sova
Like SQL for nested maps in Clojureland. Neat. The DSL [ALL :a even?] does
feel a tiny bit un-clojurey to me as well, maybe making it look like [:all :a
even?] would make it clearer.

~~~
tekacs
Well there’s a reason it’s like that — ALL is actually a value, not just a
sentinel or keyword. It’s found at com.rpl.specter/ALL.

Keywords don’t form a DSL in Specter, they just do keyword navigation into
maps, the same way ‘0’ navigates into the first element of a list.

This touches upon a really nice element of Specter, which is its pattern for
reuse. Here’s a very real but deliberately simple example of using it:

    
    
      (def chat-threads
        {[“sova” “tekacs”]
         [{:text [“hey” “you there?”]}
          {:text [“line 1” “line 2”]}]})
      
      (def ALL-TEXT
        [sr/ALL :text sr/ALL])
      
      (defn text-for [chat]
        (sr/select [chat ALL-TEXT]))
    
      (def ALL-CHAT-TEXT
        [sr/MAP-VALS ALL-TEXT])
      
      (text-for [“sova” “tekacs”]) ;=> [...]
    

Here the navigator into a list, then the :text field and then the list is
bottled simply by putting several navigators in a vector.

Then, it can be used alongside other navigators in a first class fashion —
[chat ALL-TEXT] is just a nested vector. :)

------
dimovich
Specter is really handy for manipulating data structures, and have been
happily using it.

I wonder, is it possible to select some subset of a map? Say you have these
paths:

(def paths [[:some :path] [:another :deeper :path] [:yet :another :one] [:yet
:another :one :deeper]])

And select from a map these paths. You could do it with get-in, but maybe
Specter can do it faster? I couldn't find a way to do it with Specter...

------
dwohnitmok
I hope that more and more lens implementations should catch on in languages
with immutable data structures.

They really are the key to make immutable data structures as easy to update
and work with as mutable data structures.

------
kazinator
TXR Lisp:

    
    
      This is the TXR Lisp interactive listener of TXR 243.
      Quit with :quit or Ctrl-D on an empty line. Ctrl-X ? for cheatsheet.
    

Give me all the .c files:

    
    
      1> (glob "*.c")
      ("abcd.c" "args.c" "arith.c" "buf.c" "cadr.c" "chksum.c" "combi.c"
       "crazyffi.c" "debug.c" "dict.c" "eval.c" "ffi.c" "filter.c" "ftw.c"
     "gc.c" "glob.c" "halfsip.c" "hash.c" "highest-bit-main.c" "highest-bit.c"
       "highest-test.c" "isqrt.c" "itypes.c" "lex.yy.c" "lib.c" "lisplib.c"
       "match.c" "parser.c" "protsym.c" "query.c" "rand.c" "regex.c"
       "signal.c" "socket.c" "stream.c" "struct.c" "strudel.c" "sysif.c"
       "syslog.c" "termios.c" "test.c" "tree.c" "txr.c" "unwind.c" "utf8.c"
       "vecho.c" "vm.c" "y.tab.c")
    

OK, which ones are over 100000 bytes:

    
    
      2> (keep-if [chain stat .size (op < 100000)] (glob "*.c"))
      ("arith.c" "eval.c" "ffi.c" "lex.yy.c" "lib.c" "match.c" "stream.c"
       "y.tab.c")

~~~
jonahbenton
Clojure is already good at that- at conditional pipe / filter / aggregate
semantics, bc it is a lisp. Specter is for a different problem. When you have
a data structure that has some semantics and is deeply nested (and you aren't
discovering the structure, like getting size from a node via a function call)
Specter has a declarative set of operations to navigate and extract on that
data structure, extremely efficiently.

~~~
kazinator

      1> (defstruct node ()
           item
           next)
      #<struct-type node>
      2> (new node next (new node next (new node item 42)))
      #S(node item nil next #S(node item nil next #S(node item 42 next nil)))
      3> (call .next.next.item *2)
      42

~~~
reitzensteinm
I don't usually have a problem with you posting TXR comparisons in every lisp
thread, but if you're not going to take the time to understand what the
library does, this is just spam.

