Hacker News new | past | comments | ask | show | jobs | submit login
Common Lisp Object System (CLOS) (hescaide.me)
113 points by memorable on Nov 23, 2022 | hide | past | favorite | 86 comments



I've been exploring CLOS for the past couple of weeks.

From what I can tell, it seems especially nice for a system that needs extreme flexibility. I like to program MUDs and rougelikes as side projects, and after initial exploration, CLOS seems perfect for that - it can enable certain interactions that I would otherwise have to spend a lot of work and design achieving in other languages. It turns what feels like a chore in other languages into something fun.

I would imagine that how much this is a pro or con depends upon the work you do. Some might see such potential flexibility as error prone and overly complex because the domains they work in don't require such complexity. But others might see it as the building blocks for a domain with complex interactions that they have to build regardless of the language they use.


CLOS extends the Common Lisp philosophy of emphasizing developer flexibility, possibly at the expense of execution efficiency (not trying to open a can of worms here).

For any application of substantial size, I can't imagine not using CLOS classes and generic dispatch. It may be overkill for small projects, but as the code complexity grows the abstractions of CLOS make their worth known. That's been my experience most recently with my 3D graphics system.


I once made a rules engine for Magic the Gathering in python. As a design problem it is quite interesting since the key thing about MtG is that the pieces of the game can override the game rules. After reading about CLOS a few years later it seems I reinvented some of the main ideas (like cleanly re-entrant method combinators, cleanly overriding functionality of other objects in a way that can be reverted) using python's dynamiticity (especially descriptors which let you override what obj.[accesor] means).


Alan Kay’s immortal quote about OOP:

> OOP to me means only messaging, local retention and protection and hiding of state-process, and extreme late-binding of all things. It can be done in Smalltalk and in LISP. There are possibly other systems in which this is possible, but I'm not aware of them.

[http://userpage.fu-berlin.de/~ram/pub/pub_jf47ht81Ht/doc_kay...]

His reference to ‘LISP’ is, of course, referring to CLOS.


> His reference to ‘LISP’ is, of course, referring to CLOS.

Definitely not. The all-caps "LISP" should be the strongest hint, but he name drops McCarthy and Carl Hewitt. McCarthy's LISP didn't have closures, so I'm thinking Alan is referring to something closer to Scheme. Which might make sense as Scheme is based on ideas from Hewitt (actors, message passing, etc.). But Alan isn't being terribly specific. You have to understand there are thousands of different "LISP" systems in the world. Many of which are just in some thesis or white paper.


Hmm, this is interesting. Rereading that post, it looks to me like McCarthy and Hewitt are mentioned as influences rather than a manifestation of ‘OOP’ as such, and I can’t quite think of any OOP system in Lisp he could be referring to other than CLOS, but then again I don’t know sufficiently much about this area to be sure either way.


That reminded me of Peter Norvig's comment:

> Depending on your definition, CLOS is or is not object-oriented. It doesn't support encapsulation.

Lack of encapsulation would preclude the protection and hiding of state-process.


It depends on what you mean by encapsulation. CLOS classes don't own their methods, which makes CLOS much more flexible than other OOP systems. But it also means there are no "private" methods, or slots for that matter.

One way to "privatize" slots or methods is to put them in a special package, which serves as a private namespace. This doesn't prevent programmers from seeing them but it can flag any use of them as "odd" style in the source code.


And the message idea live on objective c and lisp, but not the other c++ system.


don’t forget erlang !


Maybe, though Alan Kay is certainly familiar with other object-oriented programming systems for Lisp that predate the design of CLOS.


I bet Erlang/OTP also fits (which the added niceties of potentially distributing the object graph.)


Erlang is AFAIK the only production programming language that is OO by this definition.


  I don’t know the exact implementation details of CLOS, but I feel that it can be easily replicated in C by adding the needed metadata in the structure and doing the needed type checking in the functions.
And thus Objective-C was reborn


Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.

https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule



isnt this just a small journal entry saying, "i started to learn clos and i think i like it". i dont see any siginificant information about why clos is great here. doing general oop in lisp is powerful from the get-go because you can update classes and instances interactively. you will use clos for this basic functionality. however clos also allows you to extend the oop ideas in directions that are just not available in other languages


Meh.. I code in Common Lisp at home and Java professionally and I've never understood why people praise the CLOS so much. Like the author says, 99% of the time you don't have a reason to use OOP, and the few times you do you'd prefer to use as little as possible to keep everything sane. It's also one of the few parts of CL which are not optimized well by default, using structs or other basic data structures instead of objects has visible performance benefits in SBCL if you care about that for some reason.

Yes I'm aware of the things you get in CL you don't get in Java: multiple inheritance, defmethod works on arbitrary types by default rather than having to create interfaces, better introspection.. but I don't care. If you're using these features so much there's probably something overly complex with your code to need to do so. Or you're working in such a complex domain you need to, in which case go ahead..


I praise CLOS because I've used it quite a bit and I like it.

CLOS is an object-system construction kit as much as it is an object system. I generally like to solve problems by building a language in which the solution is simple and straightforward to express, and CLOS provides handy tools for defining data structures and suites of operations on them that facilitate that approach.

I also like to work by livecoding, and that sometimes means redefining types that have live instances. If my types are CLOS classes, the Lisp will automatically catch changed definitions and will update existing instances to conform to the new definition. That's awfully helpful in a livecoding context.

CLOS has also come in handy when I needed to build novel systems of data types and function dispatch. For example, Ive worked on several knowledge-representation systems over the years, and it's been useful to have CLOS features for constructing novel inheritance schemes, access controls, truth-maintenance, persistence mechanisms, and so on.

So I've used CLOS quite a bit over the years. Some tools vex me more every time I have to use them. Others are a greater pleasure to work with each time. In my experience, CLOS has been a tool of the second kind.


CLOS is currently slower, but there are some cool ideas floating around to greatly speed it up. The roadblock has been CLOS's dynamism: everything needs to be redefinable at run time, even the classes. But it's possible for an implementation to get high efficiency even with that, using separate code snippets that are reached by a single branch (although I don't know if any CL implementation does this yet):

http://metamodular.com/SICL/call-site-optimization.pdf

(I think a modern version of Common Lisp would take this idea and run with it, making most or all built-in functions generic.)

CLOS really shines when it comes to problems that benefit from method combination, which allows one to decompose computations into reusable nuggets that can be reassembled using multiple inheritance.


> rather than having to create interfaces

I don’t know about that. To me, interfaces is the best thing since sliced bread, “OOP” or not. In particular, they bring some sanity to the design in the modern C++, where you would use templates in the lower level parts of the application, for performance, and interfaces at the higher level, for sanity.


Personally, I use CLOS all the time and think it's great. Even without defining my own classes, I use multiple dispatch and methods on arbitrary types quite a bit. Admittedly, I don't really use the advanced features like MOP, but I appreciate that they're there and that I can manipulate how the object system works if I ever need to.

As far as performance goes, it just hasn't been a problem for me, even when doing graphics and image processing. A pattern that's worked really well is to use OOP to structure and organize everything, and then have the implementation details flushed out with functional or imperative code. It's the best of all worlds.

And it's actually a good pattern in most OOP languages because it slows things down in most of them. The vtable in C++, extra GC in Java, object attributes vs. locals in Python, etc. Avoiding objects in performance critical code is just generally a good idea.


I think you have to take the praise in perspective? Go into it with the idea that you want to do OOP, and CLOS is pretty amazing.

I agree with you, though, OOP should be something you reach for purposely, not as a default. And, unfortunately, the waters have been so muddied, that I couldn't tell you the best purposes to reach for it. About the best I can see is that it does lend itself well to metaphor, and people communicate almost exclusively in metaphors, when it comes to programming?


Agreed. I program in CL and other languages too. I find Java developers tend to have a stronger understanding of OOP and that might be behind your insight. For all its benefits, it feels like at times many CL users miss the point of OOP.

CLOS itself is great, however the decoupling of data and methods is something at times I find leads to harder to understand code.


> CLOS itself is great, however the decoupling of data and methods is something at times I find leads to harder to understand code.

I think that might be familiarity rather than anything fundamental. I often find the coupling of data and methods to lead to harder to understand code; if I want to define a new method on an object in Java, I have to either subclass it or modify the class itself. Even if the method is ancillary to the core functionality of the class. That seems strange to me.


  > I want to define a new method on an object in Java, I have to either subclass it or modify the class itself.
maybe what you are looking for is extension methods?

objective-c/swift/kotlin/c# all support them to varying degrees (but sadly not java afaik)


Yes, those are a huge improvement over the Java/C++ way. I cant tell if in a typical implementation all extension methods share the same namespace though (i.e. if the language has the concept of a "namespace" can one safely define two different extension methods on the same object in two different namespaces; this is trivial in CLOS since methods are not namesepaced to objects in any manner).

A quick read of the wikipedia article makes it look like in C# you can do this, but not in Ruby.


> using structs or other basic data structures instead of objects has visible performance benefits in SBCL if you care about that for some reason.

Sure, but recall that CLOS really consists of two distinct components. The class/object system, and generic functions.

Since CLOS is baked into the CL type system, generic functions will happily dispatch against DEFSTRUCT types (as well as all of the other types, not just objects). Add the ability to :include other DEFSTRUCTS, and the idea that when DEFSTRUCT B :includes DEFSTRUCT A, then the type-of of B is both as a B struct and an A struct, then you get back alley inheritance using structures.

So, now (in theory) you get the efficiency of DEFSTRUCT, but all of the dynamic dispatch giddy goodness of DEFMETHOD, when and if you want to use it.

It can be a really nice compromise.


Generic functions in Common Lisp dispatch against "all the other classes", not "all the other types". Every class in CL is a type, but not every type is a class.


Same. I have quite a bit of OOP background, and I use CLOS, but it's clunky enough (compared to Ruby e.g.) to be a tool of last resort.

That said, I mostly work on CL code by myself, maybe CLOS is more useful to large teams, maybe that's where it really shines and why people praise it.


From what I know about CLOS, it's a great tool if you have a tight-knit team of seasoned devs, but it's a giant sack of footguns otherwise. You can redefine so much about the object system that your codebase will instantly become unmaintainable.


Redefining things about the object system sounds more like MOP than CLOS. The MOP is an extension of CLOS, but it's not part of the Common Lisp standard like CLOS is.

If you meant one part of the system redefining things defined in another part, this can be addressed with package locking, if your CL implementation supports that (sbcl does). This is a general problem of lisps, not something specific to CLOS.


You cannot use Common Lisp without the CLOS. The fact that you can delude yourself into thinking that most of the time you don't have a reason to use it is what makes it so great.


You absolutely can use it without CLOS. It was an afterthought poorly bolted on that was worse than its predecessor in most ways as to be portable. It didn't exist when CLtL1 was written and barely existed when CLtL2 was released.


Ok, I will try to use Common Lisp without the CLOS:

  * 123
  123
That looks good, let's see...

  * (describe *)
  123
    [fixnum]
Huh? What's a FIXNUM?

  * (describe 'fixnum)
  COMMON-LISP:FIXNUM
    [symbol]

  FIXNUM names the built-in-class #<BUILT-IN-CLASS COMMON-LISP:FIXNUM>:
    Class precedence-list: FIXNUM, INTEGER, RATIONAL, REAL, NUMBER, T
    Direct superclasses: INTEGER
    No subclasses.
    Sealed.
    No direct slots.
Uh-oh.

Anyway, you see what I mean now by not being able to use Common Lisp without the CLOS. Unless by using the CLOS you mean having to explicitly opt-in by using classes, multiple dispatch, multiple inheritance, generic functions, the MOP and all that weird stuff.


> Unless by using the CLOS you mean having to explicitly opt-in by using classes, multiple dispatch, multiple inheritance, generic functions, the MOP and all that weird stuff.

Yes, that's what I meant. The fact that the CL type system maps into the class system doesn't mean we all code OOP. If you want to argue semantics, sure..

SBCL will generate _very_ different code for fixnums vs general objects, so much so that claiming that (+ 1 2) is using CLOS makes no sense.


It makes sense to me. The same as claiming that doing `1 + 2` in OCaml is making use of the type system despite me not writing any type information and the language performing type erasure.


This "afterthought ... poorly bolted ... worse than its predecessor" is a type of HN comment that's too common. Not being a CLOS or Lisp user, I have no idea why you say this and can learn nothing from it.

HN's really starting to put me off with suchlike rife blanket-damning posts that are uninformative and usually come from people who have little experience.


Indeed, this type of threads always remind me that HN is not the same as it used to be. If you wish to learn about something (including about its worth), it's not the right place. Read books; read and write code; form your own opinions.


Replying to self because HN doesn't show me reply buttons to this comment's children.

This account is 52 days old, but the user behind this has been reading and posting on HN for a bit longer (let's say since PG decided to put HN online).

For the other comment, my point is that "genuinely wise" is much rarer nowadays, and instead we're drowning in "genuinely clueless". This is why you shouldn't learn about something's worth by reading HN comments nowadays. Instead you can put a limited amount of time actually studying the subject and then decide if it's worthy of more of your time or not.


Perhaps.. but clueless comments are usually evident by the very lack of solid info. So why even post?

I get further pissed off by having people argue back when I say something from my small island of expertise (SQL and a few other things) they clearly know less than me - I can't learn from them and they won't learn from me. I don't mind n00bs, we all were once, but to willingly remain n00bs by rejecting information, well I can't comprehend it.

I miss posts from the likes of BeeOnRope. The really good people get driven away.


Your account is 52 days old, which I will admit is a poor way to estimate how long someone's been around. Look at any old thread around Arc's release and you'll see the comment quality was even lower than here.

https://news.ycombinator.com/item?id=106398


Yes and no, books are fab but words from the genuinely wise can save you a lot of otherwise wasted time so I always appreciate for knowedgeable advice as well.


It wasn't elaborated on because it wasn't part of the core point of the comment, which was someone claiming falsely that to write Common Lisp you needed to use CLOS, which you absolutely don't.

You wouldn't go into why JavaScript is bad to point out that you don't need to write JavaScript to make a web page. You would just note that JavaScript was a late addition to the web and wasn't the first language usable on it.

Plenty of people were writing CL before CLOS existed.


Then "You absolutely can use it without CLOS" would have sufficed.

You opined in addition on the quality of clos. That could have been very useful if you'd explained it.


I don't know what their own thoughts on it are, but I personally think it's poorly bolted on because it's not actually used by the rest of the spec. There's a massive proliferation of functions acting on data structures, and none of them are generic even if they do the same thing. Poorly bolted on indeed.


CL's functionality is an amalgamation of the prior Lisps (thus "Common") which didn't have CLOS. But it was CLOS-ifying a variety of things by the time of the 1994 standard. You just don't see more because the immediate goal was a large degree of compatibility with those prior Lisps. If a second standard had developed a larger portion of the system would probably have been brought under CLOS.


This is a valid criticism. Many of the (possibly) non-generic functions in Common Lisp could be made generic. That they weren't was a bow to existing implementations of Lisp that wouldn't have supported it. The stakeholders engaging in the standardization process didn't want extensions that would be excessively (at the time) costly for them to implement, especially in a way that wouldn't have a large runtime performance impact.


The CL error system is built with CLOS. If you want to customize error handling you'll basically be writing CLOS.

But I agree about general data structures. Some of the sequence functions are generic but not enough of them. I presume this happened because circa 1990 generic dispatch was too slow to handle high speed data traversal. That's no longer true, and many libraries exist to "generify" more of Common Lisp.


...and that's the sort of critique I'm interested in and which should have been given initially.


> I don’t know the exact implementation details of CLOS, but I feel that it can be easily replicated in C

This makes me think he's barely scratched the surface of what CLOS has to offer.


"Easily" is a stretch, for sure, but John Wainwright developed a framework that he called "oic", for "Objects in C", that was a fairly substantial subset of CLOS.

I met him at OOPSLA ‘89 (or it might have been at AAAI) and introduced him to a couple of Apple developer guys, because I thought oic was cool. Whether because of that introduction or other reasons I’m not aware of, John ended up selling oic to Apple for use as the basis of the ScriptX object system, which was the programming language of the Kaleida Media Player (https://en.wikipedia.org/wiki/Kaleida_Labs).


The class and instance evolution protocols for starters seem not all that easy.


Exactly. Under the sheets you need class wrappers at a minimum to support these things.


“Easily” is a stretch, but with a few Lisp->C idiom translations, you could follow along straightforwardly with “The Art of the Metaobject Protocol”[1].

[1] https://mitpress.mit.edu/9780262610742/the-art-of-the-metaob...


I've read the MOP too. I seriously doubt that this is the case. There is a lot just below the surface in CLOS that a casual encounter would not reveal. This post [1] by Joe Marshall gives a taste of how these features can go unnoticed until you need them, then you appreciate CLOS for these things.

[1] CLOS!? https://www.ca.crh.com/host-http-funcall.blogspot.be/2013/07...


Though possible. See for example https://github.com/blakemcbride/Dynace


The definitive book on implementing CLOS (not how to use it) is https://en.wikipedia.org/wiki/The_Art_of_the_Metaobject_Prot...



Thank you. I should have included this.


If you like CLOS, be sure to look up "The Art of the Meta-Object Protocol." It Was written by people who were at PARC back in the day so were on the ground when Smalltalkish concepts were being developed and extended. But... it's focused on Lisp.

It describes the thinking behind an implementation of OOP for Lisp, and along the process reveals parts of the mental journey to both Smalltalk and CLOS.

https://en.m.wikipedia.org/wiki/The_Art_of_the_Metaobject_Pr...


AspectJ was created by one of the authors and introduced some of CLOS ideas neatly into Java. Most of the time I find that closures are simpler than OOP to capture state and logic together…


OOP implementation in Next Generation Shell was inspired by CLOS but it's way simpler.

Defining a type: type MyType.

At any point one can declare method: F myMethod(..., mt:MyType, ...) some_code()

Anything defined by F is automatically a multimethod. When invoked, all methods in the multimethod are searched bottom up. The first one where the call arguments match the parameters (by type, with inheritance) is invoked.

That's roughly it.


One thing bad is the word 'class'. What it is supposed to even mean? It was some fancy concept by stupid Normen when defining Simula. My Simula teacher said that you could just translate it as "design" as in "design of car and its implementation".

Anyways Python's "class" is even worse:

    class c: a=10
    print c.a
    c.a=13
    print c.a
I cannot think any English word to replace the "class". The word I am thinking of could be translated as "box". "Box of things and improved versions of it".


From https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&d...: ‘We chose the terms “class” and “objects” of classes for our new Simula. The notion of subclass was especially appealing to us, since we had seen many cases of objects belonging to different classes having common properties. It would be useful to collect the common properties in a separate class, say C to be specialised differently for different purposes, possibly in different programs.

It is about classifying objects. You want to have a shared definition for objects that share a common set of properties. Those objects constitute a certain class of objects, defined by their commonality. To specify such a class of objects, you specify those shared properties. This is what classes do in programming languages, they provide a specification of a certain class of objects.

If you know that a given object belongs to a certain class, you then know that is has the properties specified by the class. For example, if you know that an object belongs to the class of String objects, then you know that it has a length property (or whatever).

“Class” is very similar to “type”, except that “type” is usually more general. But in languages where everything (every value) is a object, “class” and “type” can be virtually the same.


yes. and in addition the word existed before, with the morning of "kind, category, group" of whatever.

https://www.thesaurus.com/browse/class

naming is hard and class is a good name for it's meaning in software.


This is why I find prototype-based object systems so much more natural. Inevitably any dynamic object system ends up having to represent classes as objects, but that retroactively opens up a lot of mind-bending complexity. Prototypes on the other hand make far more sense in a dynamic context. You just create objects that you can clone and extend. Classes just become a design pattern. It also highlights that the interfaces of an object are more important than the class.


> Classes just become a design pattern

You haven't really got rid of classes though, you've just moved them from being explicit (something the language itself knows about) to implicit (something the developer keeps in their head). You still have some objects which play the role of "classes" (they are used as prototypes for other objects) and others which play the role of "instances" (no other object will use them as a prototype).

How much of an improvement is there really if the concept still exists, but the language itself is ignorant of it?


Even C++ does not need ‘class,’ as it already has ‘struct’ that it inherited from C. (A saner language would just use the word ‘type.’)


Or as it's lived in all of java culture (and beyond) that isn't infested by that very special thought universe that exists around the landmarks of j2ee, "beans" and the spring framework, "classes" are just namespaces for organising your code that may or may not also happen to identify the type of a struct. A good term for that mindset would be "post OOP". As someone from that group, all I could think reading that CLOS post was "cute, the author really believes in all those early OOP promises? Must be a time traveler from the 20th century!" Actually checked the date, but then on the other hand they didn't have blogs yet when OOP was the future.


"mold", but the mold is also an instance of a mold.

    i = c()
    print(i.a)
    i.a = 42
    print(i.a)


"Object"? Though that creates ambiguity between definition and instances of the definition.


You just nailed it :)

Maybe instead of "Class" and "Object" we should use "Definition" and "Instance".


I dunno, replacing all instances of defclass with defdefinition makes programs seem a bit silly.


Can I humbly suggest that it's definitions and instances 'all the way down'


In CLOS, all classes are instances but not all instances are classes.

It just so happens that the object that describes a class's slots, inheritance, etc is ... an instance. Of what? A metaclass. Which is also an instance.

It's confusing at first but eventually it all gels.


i think of "blueprint" as an analogy for what "class" is supposed to describe.


Earlier lisps called it „flavour“

Defflavor instead of defclass


That makes sense only when multiple inheritance is supported though.


Flavours was single dispatch and the term for subclass was „dependent flavor“


Single dispatch has nothing to do with multiple inheritance.

“Many versions of Lisp added OOP through the addition of an object system called Flavors, which went beyond Smalltalk and included multiple inheritance and method combination.”

http://www.paulgraham.com/chameleon.html

“The power of flavors (and the name flavors) comes from the ability to mix several flavors and get a new flavor.”

https://franz.com/support/documentation/current/doc/flavors....


maybe superset or subset works as a translation? (in the math sense superset).


“In the math sense” a better term would be ‘category’ :)


touche


!00% agree at the basic level. The approach here with multiple dispactch to me feels so much more natural than the Smalltalk/Java type of OO.


> Instead of instructions like in imperative programming (and modularizing it with functions), your program is a set of entities talking to each other and collectively achieving the required goal.

So it looks like the author discovered a key characteristic of OOP which many languages including Java, support.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: