
The Case for Message Passing in Swift - ingve
http://www.buckleyisms.com/home/2014/6/16/the-case-for-message-passing-in-swift.html
======
josephlord
> I have filed a radar (Open Radar) asking for an eval() function in Swift.

Does anyone else thing that is a really bad idea in general? To do that not
only would type safety go straight out of the window but so would a great many
of the compile time optimisations that are currently possible (I don't know
how many are being done at this time).

Within pure swift the type information can be statically checked at compile
time and then the type information can essentially be discarded. To offer eval
I don't think that would be possible. It is also probably why there isn't the
level of reflection that the OP is wishing for.

~~~
Buckleyisms
I described that radar rather poorly. If you look at what I described in the
radar, I basically want them to ship the swift compiler as part of the core
OS, and an eval function would compile its argument and load it into the
current process using the dynamic linker. I definitely do not want interpreted
Swift.

Swift is certainly a language that relies on compiler optimizations. I don't
want to change that.

~~~
josephlord
But it would be unable to interact in a symbolic way with the calling app. It
might be able to access things in frameworks but it might as well be a
separate app that you compile with external tools at that point.

As to why it couldn't interact with the running app others have given some
examples but it couldn't even reliably call functions as they may be inlined
or generics may only be compiled into their required type specific versions.

I do expect[0] Apple to open source Swift when it is finally released and it
to be part of the LLVM project so you could embed whatever parts make sense in
your own applications but it wouldn't be like an eval function as I understand
it although you could probably use it to make your own playground system.
Basically it would be an LLVM feature/library not a Swift language feature.

[0] Guess. I have no inside information and may end up disappointed.

------
gress
This seems more like a case _against_ message passing.

The reason iOS is smooth and fast is that a lot of the heavy lifting is done
by optimized pure C code called from objective-C.

A major goal for Swift is to let people achieve that kind of performance
without having to write the high performance stuff in C. C is replaced with
swift's functional core. Switching all dispatch to message passing would
destroy this benefit and push more development back into C.

The case of the Latin Conjugation, and the IRC client are terrible reasons for
using message passing. Both are trivially solved with data structures and
closures. It seems that the author has little experience with functional
programming, and just wants to stick with Objective-C.

I'm not against message passing - I just think that Swift gets the balance
right, and it would take a much stronger argument than this to justify message
passing everywhere.

~~~
Buckleyisms
I suppose it depends on what you intend to use Swift for.

My argument is that Swift will be primarily used to write high-level apps. For
these apps, message passing is almost never a bottleneck. Swift is not going
to replace the optimized C and C++ code, especially for the really low-level
stuff.

But even if Apple does decide to rewrite all the Core* frameworks et. al. with
Swift, there's still an argument that Swift should use message passing by
default, and a way to opt in to vtable and direct dispatch for performance-
critical code. For higher-level apps, message-passing is a lot less useful if
it's opt-in.

As for the IRC and Latin conjugation, yes, those problems can be solved in
other ways, but those other ways involve more (mostly glue) code. Trading more
code for execution speed is the wrong tradeoff when performance is good
enough.

You're correct I want to stick with Objective-C, but not that I have little
experience with functional programming. I love functional programming. I love
most of Swift. But for the specific problem domain of Mac and iOS apps, I love
Objective-C. It makes working with GUIs much easier than anything else out
there, and the ability to method swizzle Mac apps is great for third-party add
ons.

~~~
michaeljbishop
Amen! (Minus the sizzling)

------
bigdubs
Honestly a lot of the cases presented here would be accomplished by a robust
reflection API and some form of dynamic method compilation.

These are both things I haven't seen in Swift yet and will miss dearly from
.net/C#.

~~~
pohl
Along that line of thought, it looks like there are beginnings of a mirror-
based reflection implementation already in there. It appears to only be there
for Playground at the moment.

[http://stackoverflow.com/questions/24060667/does-swift-
suppo...](http://stackoverflow.com/questions/24060667/does-swift-support-
reflection)

~~~
pstop
Yea, I noticed this in the undocumented API's, but haven't played much with
it. It's definitely cleaner than C#'s reflection implementation, but more
limited as well. Though for most uses, knowing the limitations solves most of
the issues the OP has.

~~~
astigsen
This presentation covers some of the reflection API in Swift:
[http://www.realm.io/news/swift-unchartered-territory-
swift-i...](http://www.realm.io/news/swift-unchartered-territory-swift-intro-
and-internals/)

While it is clearly still somewhat limited, it does seem like there is
potential for it to evolve into a really robust reflection API.

~~~
Glide
I disagree. Right now it doesn't seem like a lot of things aren't possible in
swift unless there is a LOT of undocumented API.

Dynamic method compilation that you would need to make an equivalent of
DynamicProxy just doesn't seem possible. What's funny is that the #1 thing I
would state is not possible in Swift is Core Data.

------
hamstergene
What this post calls "message passing" should actually be called "dynamic
dispatch"
([http://en.wikipedia.org/wiki/Dynamic_dispatch](http://en.wikipedia.org/wiki/Dynamic_dispatch)).
The term "message passing" is currently more often used as a certain way to do
concurrency, and this post is not about concurrency at all.

The post is basically criticizing Swift for being static language, and not
dynamic like Obj-C.

~~~
Buckleyisms
From the linked Wikipedia article on Dynamic Dispatch:

> C++ compilers typically implement dynamic dispatch with a data structure
> called a virtual table that defines the message to method mapping for a
> given class (C++ as such has no notion of a vtable).

In my article, I'm specifically talking about message passing as implemented
in Objective-C and Smalltalk.
[http://en.wikipedia.org/wiki/Message_passing](http://en.wikipedia.org/wiki/Message_passing)

Both the term "dynamic dispatch" and the term "message passing" are
overloaded, but I used the term "message passing" to differentiate it from
vtable dispatch, which I talk about in the article.

I'm not criticizing Swift for being a statically-typed language. I'm only
criticizing the way it calls methods.

~~~
Glide
Your definition were clear from people coming from an Objective-C background
and after understanding where Swift is coming from after the WWDC talks.

But you do realize that the way is calls methods is exactly what makes
Swift... well Swift.

I sympathize though. Swift feels like it's several yards forward when compared
to Objective-C when looking at syntax and language features. It's a couple
steps back when looking at runtime hackery (and "power" I guess). Basically
something like Core Data is impossible in Swift.

~~~
coldtea
> _Basically something like Core Data is impossible in Swift_

Huh? Is it not already available for Swift actually?

Or the problem lies in re-implementing it in Swift?

~~~
Glide
The latter because of the runtime magic that it uses in NSManagedObject.
Watching the WWDC 2014 talk states that the Swift team added @NSManaged so
that it's possible to define models in Swift to be used in Core Data.

I also tried wiring up something really quickly in Typhoon (objc-ioc
container) and it didn't work with a Swift class. I did do @objc on the class.
Something about how properties are defined in Swift. It's pretty much expected
because the runtime system is different.

Don't get me wrong, I'm not going to go "omg everything needs to be rewritten
in swift!!!" but after trying a toy project or two in Swift, dealing with all
of the implicitly unwrapped optionals from objective c code gets a wee
tedious. It's a strange feeling. Like all the swift code is safe and the objc
code is dangerous somehow. I can quickly see why people would want everything
to be done in Swift.

------
andybak
I wonder if the maxim about premature optimization needs to be considered more
in language design.

For example - I want to write idiomatic Python with all the benefits of
flexibility and readability. I then want the possibility of optimizing some
parts to the level of a static language - but only those parts that need it.

The javascript performance battle is very revealing here though - it's
surprising that asm.js hasn't got quite the clear lead one might expect.

~~~
yawaramin
> For example - I want to write idiomatic Python with all the benefits of
> flexibility and readability. I then want the possibility of optimizing some
> parts to the level of a static language - but only those parts that need it.

Then you want to check out Numba--it compiles explicitly marked Python
functions to LLVM IR.

------
mahyarm
Message swizzling/monkey patching is a hack that tries to deal with the
reality of not being able to modify base classes you don't have the source
code for and creates all sorts of surprise bugs.

If objective-c had proper support to do it via categories that can 'over-ride'
methods it would be a lot cleaner in general.

~~~
emp
But it does allow for programmers to add their own ideas. I have a traits
implementation where I can inject entire behaviours in a line of code. I am
not sure if this could be done in Swift (haven't had time to research
properly).

~~~
mahyarm
That's what I mean by implemented better as something that is officially
supported. In objective-c it's a hack, it can lead to hidden surprises since
code usually doesn't make it obvious that a specific method has been swizzled.
And you have to deal with hidden expectations in code that you have no control
over.

~~~
emp
True, though then we are waiting and hoping for features. Another example of
some runtime fun: my debug build swizzles interceptors for common UI lifecycle
and action methods (viewDidLoad, didMoveToParentViewController:,
sendAction:to:forEvent:, etc). When I am working on an app with many views, a
simple glance at the console shows me what view controller displayed, what
action a button fired and so on. Great for getting to relevant code quickly. I
don't think I could hope for this as a standard feature of a language.

------
eridius
Here's a very quick summary of the reasons cited in this article:

1\. Mock objects

There's two different types of mocking. The first is where you know ahead of
time what you need to mock, and this type is largely equivalent to just
subclassing the desired class (except any methods called directly or
indirectly by the class itself will bypass the mock). This could be achieved
by implementing basically implicit delegation, where the invocation of any
methods that an object does not respond to will be replaced by the same
invocation on a designated property of the object. This could in fact be
achieved today with conversion routines (which are undocumented and
technically unsupported, so file a radar if you want to keep them), except
when testing the object for its class or protocol support. So the language
would have to add special support for this in order to have a true proxy
object that pretends to be an instance of the target class.

The second is dynamic mocking, where you add behavior to the mock object at
runtime. This I believe is what OCMock is doing. It's effectively equivalent
to implementing every single method of the target class, and looking up any
invocations of the method in an internal table of closures, falling back to
calling the target class's implementation if there is no internal closure.
Again, this would need support for class/protocol testing. And again, the
language could certainly add support for this without message passing, by
basically generating the code for what I just described.

It seems reasonable to me to request support for mock objects itself, without
requiring message passing.

2\. Swizzling

I'm really _really_ glad Swift's class model does _not_ allow for this.
Granted, you can still monkeypatch it at runtime the same way you can patch C
code, but it's hard enough and esoteric enough that people will generally not
do it.

Swizzling is a really bad idea. Every example given for why the author wants
Swizzling is an example of something that you should not do. Changing
Frameworks behavior, or injecting into apps and changing app behavior, is a
Bad Thing™. When allowed, it's a major source for instability and crashes, and
other bad behavior. For example, recently there's been a big resurgence in
interest in unauthorized Xcode plugins, but Xcode doesn't have plugin support
for a reason. Xcode plugins are a great way to make Xcode behave badly or
crash. For example, Spark Inspector[1] installs an Xcode plugin on first
launch (without asking!) that adds a menuitem to Xcode. And when Xcode 5.0
came out, it started crashing on launch for people who had the Spark Inspector
plugin installed. And I guarantee you most people who experienced the crash
did not know it was Spark Inspector's fault and blamed Xcode.

[1]: [http://sparkinspector.com](http://sparkinspector.com)

Similarly, GPGMail is a Mail.app plugin that adds GPG support. Mail actually
has official plugin support, although I believe it's pretty limited, and
GPGMail ends up swizzling a bunch of stuff in the Compose window. Right now on
OS X 10.10 Yosemite, GPGMail causes Mail.app to crash because of the swizzles
(yes it's a beta OS, but the crash is entirely the fault of GPGMail, not the
beta OS).

Simply put, swizzling is a horrible runtime hack that is possible in Obj-C,
but is abused to do things that it shouldn't, and can cause all sorts of
problems. It's definitely not something I want Swift to support.

3\. Configuration

This doesn't require message-passing. In fact, the only language support one
might want is a way to easily build a table that maps from variable name to
getter/setter closure, although doing it in a type-safe fashion seems
relatively problematic (the trivial solution is to use Any in the setter
closure and fail if the input value is the wrong type, and the getter closure
would of course just return Any as well). But even that's just a bit of a
nice-to-have and relatively esoteric; you can certainly build such a table by
hand.

And of course there are better ways to design this once you've actually
defined how you expect your configuration to work (this article does not go
into detail). One possible suggestion would be something like a JSON blob that
defines key/value pairings that a given object should be updated with. This
could in fact be implemented as a protocol on configurable objects, something
like

    
    
        protocol Configurable {
            func configure_with(json: Dictionary<String,JSON>)
        }
    

where we're assuming there's already a JSON library that has a datatype called
JSON that represents the parsed JSON data. The object itself would then be
responsible for checking if each of its properties exists in the dictionary
and updating with the mapped-to JSON value (and asserting or ignoring if the
type is wrong, depending on preference). This would actually be a lot more
flexible than converting strings to methods, because it would allow for
defining custom keys that do various transformations on the value, or that
update multiple properties from one key, etc.

The one language feature that would help here would be support for `deriving`,
which is to say, a way to derive the implementation of a protocol on a given
type. Haskell and Rust both support this (I would assume other languages do
too but that's what I know of off the top of my head). Of course, in Rust,
only traits (protocols) known to the compiler can be derived. I am unclear how
it works in Haskell. Ideally if Swift were to gain deriving support there
would be some way of extending support to your own protocols, but right now
it's very unclear how to properly handle metaprogramming in Swift. Regardless,
this seems like a post-1.0 feature, and is not necessary for doing
configuration.

4\. Remote Methods

I am unclear what the author is asking for Swift to support here. Some kind of
built-in distributed objects, like NSDistributedObject and the in/out/inout
annotations in Cocoa, is a bad idea, as we have learned from
NSDistributedObjects. The author's example of mapping IRC commands to actions
seems to be completely unrelated to any sort of language support for this. For
that example I'd just have e.g. an IRCEndpoint object that I would register
closures with to handle various messages, and the closures can do whatever
they want.

5\. Runtime Metaprogramming

This is a dangerous topic. Runtime metaprogramming can do interesting stuff,
but it's also usually considered to be rather hacky, potentially dangerous,
and almost always a bad idea. Among Obj-C programmers that I respect, the
general opinion is that if your code includes

    
    
      #import <objc/anything.h>
    

then you've done something wrong, with the only exception being using
associated objects (and even there, associated objects are typically
considered a solution of last resort).

The author provides a link to a youtube video as an example of something cool
you can do with runtime metaprogramming. I am not willing to sit through a 49
minute video, but skimming through it suggests that the idea is to have some
sort of class that represents a word and autogenerate methods to conjugate the
words. I think. In any case, my impression here is that the runtime
metaprogramming is actually a poor man's compile-time metaprogramming.
Compile-time metaprogramming is very powerful, but also hard to get right.
It's something I think Swift needs, but it's also something that will take a
while to design correctly, and is definitely a post-1.0 feature. As an example
of how this is powerful, Rust provides compile-time metaprogramming via
"syntax extensions", which are dylibs compiled from specially-annotated Rust
libraries that are loaded directly into the compiler (which is written in
Rust) and given full access to the AST. With this functionality, one of the
libraries distributed with Rust is libregex, an re2 regular expression
library, and this library provides the regex!() syntax extension that compiles
the regex at compile-time and generates native code for matching (as opposed
to the normal instruction-based virtual machine for runtime-parsed regexes).

\---

Turns out I'm out of time. I think I covered basically everything in the
article though.

tl;dr: Swift doesn't need message passing.

~~~
leonatan
Or you are just too narrow minded. The fact is, various frameworks "abuse"
these runtime features to give you features such as kvo, kvc and Core Data,
which you use every day. I guess you next "point" would be that only Apple
should be able to "abuse" their own runtime to provide such frameworks, while
programmers you "respect" must not. Please. Just because you and "programmers
you respect" "don't want to see" legitimate featues, does not mean these
features should not be there. Don't want them, don't use them.

"Proud" includer of <objc/runtime.h>.

------
nielsbot
Isn't it true that some messages are sent dynamically in Swift? (For example,
if you are messaging an Obj-C instance)

~~~
bjustin
If a Swift class is made available to Objective-C code (using the @objc
annotation, or subclassing an Objective-C class), that class will use message
sends for calling methods. Using an Objective-C class, or a @objc Swift class,
from Swift will also use message sends for method calls.

------
lttlrck
Why is there any doubt over the meaning of 'binary compatibility'?

~~~
seandougall
As I understand it, it can mean one of two things:

\- Applications compiled in 10.10 will run in 10.11

\- Third-party frameworks compiled in 10.10 can be used to compile
applications in 10.11

Those are different concerns, and (at least from what I've seen), it seems
that Apple has promised the first but not the second.

------
drewcrawford
So I've actually been doing a lot of thinking about this, because I have a use
case that doesn't work as well as in Swift.

You have some problems. The first problem you have, is that static compilation
provides so many routes for cross-applicaton optimization that this is one of
the major ways that Swift can be faster than its predecessors. For example, if
you have this

    
    
        func min<T: Comparable>(x: T, y: T) -> T {  //Generic functions
            if x < y {
                return y
            
            return x
        }
    
        min(3,5)
    
        min(5.0,2.0)
    

The compiler has a lot of possibilities. For example it could translate this
function very literally, producing the generic min function that you wrote. It
could realize that you only use this for floats and ints, and emit 2
particular specializations of the min function. It could specialize the float
function and use the generic function for ints, or vice versa. Next, it could
inline any of these, none of them, or some of them. Finally it could do three
different things depending on if you're x86-64, armv7s, or arm64.

I have no idea what the right thing to do is in this situation, but having all
those possibilities gives the compiler a better chance of producing faster
code. When you introduce message passing, you basically forbid all of those
optimizations, because now the function might be swizzled, go pass a "minX:Y:"
string to objcMsgSend to find out what pointer we should jump to today.

Swift already does have a way to opt-in to message passing, it's called the
@objc flag (and performSelector:). It works fine, but it's extra work to opt
into it and use it. But, I don't think that's bad given that it disables so
many useful optimizations.

What may be bad is that the name "@objc" is a dead giveaway that this is a
feature with a deprecation date. So I think we need some other, reflectionish
system going forward.

For mocks one idea would be to selectively turn on message-passing as the
dispatch method at the compiler level. So you could compile your application
for unit testing with a "-fmessage-passing" like we have "-fobjc-no-arc" and
get all the dynamism in debug builds and turn them off for release builds (or
not, as the case may be).

For everyday power programming I think the answer is a lot simpler: functions
are first-class objects in Swift, so just take your functions, put them in a
dictionary somewhere and implement your own run-time dispatch. Having done
this, I can say with certainty that there are some ways the language could
improve this use case but it is possible, and if you are Chris Espinosa
wrestling a bit with the type system is not a problem.

Things are tricky when you don't have the source code, but here again I see a
reflection-ish API as being the key. I expect at some point we will be able to
enumerate other classes' public methods, and dynamically dispatch into them,
and that is certainly a missing feature with valid use cases.

Finally, I fundamentally and strongly disagree with Michael about method
swizzling and OSX extensibility. Adding PGP to Mail via hooking functions
sounds like an incredibly deranged idea. The suggestion that programs should
cooperate with a badly-misguided reverse engineering effort by disabling
optimizations that make the program faster for the other 99% is just insane.
The way to approach this is to build new kinds of extensions that provide
known, safe, documented extension entrypoints for hooking arbitrary code. Yes
that is more work, and yes it requires actually talking with people and
convincing them of your point of view, but so do most things.

~~~
Buckleyisms
> I have no idea what the right thing to do is in this situation, but having
> all those possibilities gives the compiler a better chance of producing
> faster code. When you introduce message passing, you basically forbid all of
> those optimizations, because now the function might be swizzled, go pass a
> "minX:Y:" string to objcMsgSend to find out what pointer we should jump to
> today.

> Swift already does have a way to opt-in to message passing, it's called the
> @objc flag (and performSelector:). It works fine, but it's extra work to opt
> into it and use it. But, I don't think that's bad given that it disables so
> many useful optimizations.

You could keep some of these optimizations if Swift's message-passing
mechanism preserved type information. This would make it slower than
objc_msgSend, but you could still keep a lot of cool optimizations. Some
optimizations would still not be possible, and extensive use of function
pointers does tend to frustrate branch prediction, but I feel that for most
apps, this would be fast enough.

> Finally, I fundamentally and strongly disagree with Michael about method
> swizzling and OSX extensibility. Adding PGP to Mail via hooking functions
> sounds like an incredibly deranged idea. The suggestion that programs should
> cooperate with a badly-misguided reverse engineering effort by disabling
> optimizations that make the program faster for the other 99% is just insane.
> The way to approach this is to build new kinds of extensions that provide
> known, safe, documented extension entrypoints for hooking arbitrary code.
> Yes that is more work, and yes it requires actually talking with people and
> convincing them of your point of view, but so do most things.

I think this is a great argument, but unfortunately it's not where OS X is
today. As I mentioned, app extensions may be a step towards a more workable
solution, but today, the ability to add PGP to mail using method swizzling is
important to thousands of Mail.app users. If 10.11 comes out and Mail.app has
been rewritten in Swift before we have a good alternative to method swizzling,
we'll have lost a lot of tools that make the Mac a powerful platform.

~~~
drewcrawford
> You could keep some of these optimizations if Swift's message-passing
> mechanism preserved type information.

Type information isn't what makes this fast. What makes it fast is the very
guarantee that dynamic dispatch breaks: knowing, for a certainty, where the
function will land.

> As I mentioned, app extensions may be a step towards a more workable
> solution, but today, the ability to add PGP to mail using method swizzling
> is important to thousands of Mail.app users.

A fast Mail app is important to _millions_ of Mail users.

If PGP is important then one interim possibility is to write your own mail
client. That requires extra work, but one of the hallmarks of important
problems is that we are willing to do extra work to solve them.

