
Seemingly Impossible Swift Programs - wool_gather
https://www.fewbutripe.com/2018/12/05/seemingly-impossible.html
======
kccqzy
I have translated the pretty long Swift code belabored with manual laziness
handling into Haskell where lists are inherently lazy:

    
    
        type BitSequence = [Bool]
        
        allSatisfy :: (BitSequence -> Bool) -> Bool
        allSatisfy p = not (anySatisfy (not . p))
        
        anySatisfy :: (BitSequence -> Bool) -> Bool
        anySatisfy p = p (find p)
        
        find :: (BitSequence -> Bool) -> BitSequence
        find p =
          if anySatisfy (\s -> p (False : s))
            then False : find (\s -> p (False : s))
            else True : find (\s -> p (True : s))
    

The code still works the same. But now, it is clearer what actually happens.
The lexical call chain goes like this:

    
    
        anySatisfy p
        ===>
        find p
        ===>
        anySatisfy (\s -> p (False : s))
        ===>
        find (\s -> p (False : s))
        ===>
        anySatisfy (\s -> (\s -> p (False : s)) (False : s))
        ===>
        find (\s -> (\s -> p (False : s)) (False : s))
        === (by alpha conversion)
        find (\s -> (\s1 -> p (False : s1)) (False : s))
        === (by beta reduction)
        find (\s -> p (False : False : s))
    

where I used the long arrow to mean a deeper lexical call chain, and triple-
equal to mean a single step of reduction and/or .

In other words, as long as the given predicate p consumes a finite prefix of
the infinite bit sequence to produce a result, this will successfully return a
result. Which is quite intuitive because the function is essentially doing an
exhaustive search. It has nothing to do with topology or continuous functions
or compactness, IMO. It also has nothing to do with Curry–Howard
correspondence.

Here's an example when the function fails to terminate:

    
    
        anySatisfy and
    

Here's another example:

    
    
        let parity (False : s) = parity s; parity (True : s) = not (parity s) in anySatisfy parity

~~~
repsilat
> _Here 's an example when the function fails to terminate:_

I figured this seemed too good to be true. In fact, isn't there a simple
reduction to the halting problem? Make a "sequence"

    
    
      i -> M halts in i steps
    

for an arbitrary Turing Machine `M`. You can implement the "bit lookups" by
simulating `M`. The author says their Swift code can run `anySatisfy` on this
"sequence" in finite time, right?

EDIT: quoted at the top, but afaict not mentioned later, the article says,

>> _It is well known that it is impossible to define equality between
arbitrary functions. However, there is a large class of functions for which we
can determine equality_

It would have been nice if the author had been explicit about the class of
functions for which their program terminates.

~~~
wz1000
It works for any decidable/recursive predicate on bit sequences, i.e. one that
is guaranteed to halt for any input.

`and` on an infinite bit sequence is not guaranteed to halt because it is co-
recursively enumerable, i.e. it will reject bit sequences that contain
`False`, but it will not halt on bit sequences that contain only `True`, since
to verify that a bit sequence only contains `True`, you have to check all
elements of the sequence.

In ghci,

    
    
        > and $ repeat True
        <doesn't terminate>
    
        > and $ (replicate 10000 True) ++ (False: repeat True)
        False

~~~
repsilat
I noticed my post missed the point a bit... I was thinking about functions
that check properties of bit sequences, not functions that check properties of
predicates on bit sequences.

I get how it works now, thanks. Wish I could edit/delete my old post :-/.

Just to check my understanding: a terminating program that decides bit
sequences will have

\- a maximum number of bit checks it will make regardless of input, and

\- a maximum bit index it will access regardless of input, and even

\- a maximum running time (measured however).

------
mrmr1993
> It’s going to seem incredible, almost magical, but be assured you there are
> no tricks involved.

The trick involved is: BitSequence.find is a naïve brute-force search, but the
predicates only check n bits (for some n), and laziness ensures that at most n
bits are generated.

We run into the expected problems if n is large, or there is no such n. For
example,

    
    
        func evenNumberOfOnesAtStart(_ s : BitSequence) {
          return (s.atIndex(0) == .zero ||
            (s.atIndex(1) == .one &&
            evenNumberOfOnes(BitSequence { s.atIndex($0 + 2) })))
        }
        
        evenNumberOfOnesAtStart == evenNumberOfOnesAtStart
    

will never terminate. (Disclaimer: I don't write Swift.)

While it's nice to try smuggling in some mathematics at the end, I think a
better conclusion for the article to draw is: if you ask a computer to do a
brute-force search on some fixed number of bits, it will do it, even where has
to compute this number of bits itself along the way.

~~~
Spivak
I think there might be a clearer explanation.

If a predicate only looks a finite number, n, of bits of a bitsequence then
you can brute-force search the space containing the (2^n) sequences shorter
than n. And a predicate can't possibly look at infinitely many bits of a
bitsequence as the function which resolves that predicate would never
terminate.

~~~
mrmr1993
Agreed.

(Although it's (2^m) sequences, where m is the largest index of the n.)

------
svat
See also “Seemingly impossible functional programs” by Martin Escardo, from
2007 ([1], also at [2]).

It covers similar ground and is referenced at the bottom of the article (along
with three other references also by Martín Escardó), but I thought it worth
mentioning explicitly because I remember reading it many years ago and finding
it really fun.

[1]: [http://math.andrej.com/2007/09/28/seemingly-impossible-
funct...](http://math.andrej.com/2007/09/28/seemingly-impossible-functional-
programs/)

[2]: [http://www.cs.bham.ac.uk/~mhe/papers/seemingly-
impossible.ht...](http://www.cs.bham.ac.uk/~mhe/papers/seemingly-
impossible.html)

------
kccqzy
Nitpicking but I don't think this is quite right:

> The BitSequence type holds infinitely many values. In fact, it holds an
> unconscionable number of values. It has more values than String does.

Assuming UInt is infinite precision and Strings can be infinitely long, there
is an easy injection from BitSequence to String, therefore String is at least
as equinumerous as BitSequence.

If we bend the definition a bit and allow UInt to be infinite precision (which
it is not), there's no reason not to allow String to have infinite length as
well.

Also, the article mentioned multiple times how awesomely large the cardinality
of BitSequence is. But in fact, the cardinality is the same as that of the
real numbers—both have cardinality $2^{\aleph_0}$. So its size is rather
intuitively grasped.

~~~
DougBTX
> Assuming UInt is infinite precision and Strings can be infinitely long,
> there is an easy injection from BitSequence to String, therefore String is
> at least as equinumerous as BitSequence.

You’re right that a particular series of bits can be mapped to a string, but
here the article is talking about the function that produces the bit sequence
not the bit sequence itself. There are an infinite number of functions that
can create a particular bit sequence, and each bit sequence can be mapped to a
single string, so there must be more functions than strings.

~~~
saagarjha
> There are an infinite number of functions that can create a particular bit
> sequence, and each bit sequence can be mapped to a single string, so there
> must be more functions than strings.

I don't think this logic works. For example, there are an infinite number of
"rational numbers" of the form np/p where n, p ∈ ℤ, and each of these can be
mapped to a single integer (namely, n). But there are not more these "rational
numbers" than integers.

~~~
kccqzy
When you have an injection from A to B, you only know that B is at least as
equinumerous as A. Maybe their cardinality is the same, and maybe the
cardinality of B is strictly bigger.

For your example, we can construct an injection from rational numbers to
integers, as well as an injection from integers to rational numbers.
Therefore, by the Schröder–Bernstein theorem they are equinumerous.

------
Skeime
Here is a more imperative version (using exceptions):

The assumption is (as in the article) that each predicate terminates for every
bit sequence and that the predicates are actual functions, i.e. they have no
state. Also, predicates don’t catch exceptions.

We can implement find as follows: Start with n = 0 and generate the bit
sequences with all prefixes of length n. Have them throw an exception if
somebody tries to access an index past that prefix. This way, you get finitely
many partial bit sequences. Apply the predicate to all these sequences. There
are three cases:

\- The predicate returns true for one of the partial sequences. Because it
doesn’t catch exceptions, it only looked at values within the prefix. Hence,
the result of the predicate doesn’t change if we change the rest of the
sequence—for example, we can set the rest of the sequence to 0 and we have
found a sequence that satisfies the predicate.

\- The predicte returns false for all our partial sequences. Similarly to the
case above, the result does not depend on bits beyond the prefix, so no matter
what we do, we will not find a sequence satisfying the predicate and we can
return any sequence (by the definition of find, it returns any sequence if the
predicate is unsatisfiable).

\- For some of our partial sequences, we get an exception. Then we increase n
and try again.

This algorithm has to terminate: if it doesn’t, we get the third case each
time. But taking the bit sequence that we get by always extending our prefix
on a path that keeps throwing exceptions, we get a bit sequence on which our
predicate does not terminate, contrary to our assumption that it always does.

------
xondono
Noob here woth a doubt:

“Incredible! We are exhaustively searching an uncountably infinite space in
finite time”

Isn’t this like a long stretch? Once lazy evaluation is introduced he is
pretty much checking until a first hit happens, isn’t he?

What would happen if I call the function with predicate ‘x == one AND x ==
zero’?

~~~
jozefg
This algorithm is only total if the predicates supplied are total. Since `x ==
one AND x == zero` is not total the algorithm will diverge. It will not return
the incorrect answer though.

------
ken
He lost me when he started talking about the Halting Problem.

How is checking all Swift.String or Swift.Int values equivalent to that? There
are a lot of Swift.Int values to check (2^64, on a 64-bit platform), and even
more Swift.String values (every combination of characters up to that length),
but simply being infeasibly slow to run on your Pentium "in a reasonable
amount of time" doesn't mean it's an "impossible function to implement, and
also equivalent to the halting problem". We could easily write a function to
enumerate all Swift.Int values (though it might take 100 years to run).

He throws out the parenthetical remark "it’s best to think of Int as modeling
the infinite set of all integers" without explanation. When you're discussing
whether something is theoretically impossible or not, thinking of a finite set
as equivalent to an infinite set is brushing over a pretty major distinction.

~~~
joshuamorton
With Int you may be right.

But there are an infinite number of strings (and indeed every Turing machine
may be represented as a String in swift), so it is indeed related to the
halting problem.

~~~
ken
Swift.String is composed of a finite set of symbols, with a finite maximum
length. How can you create an infinite number of these?

The article ignores the distinction in many places, but "String" in a Swift
function means "the Swift.String type", not "an abstract string type like
those used to represent Turing machine state".

~~~
timjver
You're not wrong, but the author's code does in no way rely on the fact that
you could theoretically iterate over all the valid Int values in Swift, so it
would have made no difference if the Int type was somehow truly infinite.

And the author clearly acknowledges this:

>it’s best to think of Int as modeling the infinite set of all integers

------
n4r9
Hmm, some questions from someone who isn't familiar with Swift or much
functional programming:

\- What does this have to do with Swift? Other commenters seem to agree that
Haskell would be more appropriate.

\- What does this have to do with functional programming? As far as I can
tell, an algorithm about bitstrings has been wrapped up and reframed in terms
of types. Could you not do this in C without too much headache?

\- The recursion aspect confuses me a bit; is it possible to "find" a
predicate like "all indices of the bitstring are 1"? Or is that not an allowed
type of predicate?

\- Again with the recursion, how does the "find" procedure know to terminate
if no viable bit string exists?

EDIT: I should have read kccqzy's post more carefully. I believe it answers
the third and fourth questions.

~~~
saagarjha
For the first two: not much, and somewhat. The commenters are correct that
this logic is much more easily expressed in Haskell, since it has better
support for this kind of programming. Swift is able to do this because it has
lazy evaluation as a opt-in feature as well as the parts of the type system
that matter; I don’t think you would be able to port this to C easily.

------
aaaaaaaaaab
I don't get it. What prevents us from doing the exact same approach for the
natural numbers? I.e. represent the naturals lazily via the successor function
and use the same exhaustive search as the OP for BitSequence. Of course if the
predicate doesn't stop evaluating successor(x) for any finite x then we're out
of luck and the computation doesn't halt, but this caveat also holds for OP's
BitSequence, as others have pointed out in this thread...

~~~
jozefg
So this article shows how to check equality on `(nat -> bool) -> bool`. Your
question, I think, is to figure out how to check `nat -> bool` for equality.

The issue is that there are more operations we can do with `nat`, more
properties we can check, than there are with `nat -> bool`. With `nat -> bool`
we can basically check `i` indices and so the behavior of any predicate `(nat
-> bool) -> bool` is basically determined by the first `i` indices. And that's
finite. This algorithm is basically implicitly finding the indices considered
by the supplied predicate and then brute-forcing all those entries.

With `nat` there's no such finite cut off point. It's never the case that it
suffices to test a predicate `nat -> bool` on finitely many entries to fully
determine it. So we cannot do the same brute force search.

~~~
aaaaaaaaaab
But a natural number can be viewed as a function from the natural numbers to
{0, 1} via its binary expansion.

There's a trivial isomorphism between the natural numbers and binary sequences
with finitely many ones, so a predicate of type `nat -> bool` can be viewed as
a predicate of type `(nat -> bool) -> bool`.

~~~
jozefg
It's not an isomorphism though, it misses the always true map. This means that
many predicates on nat do not terminate on nat -> bool.

------
riskable
This article does a great job at pointing out just how needlessly difficult
computer science makes itself when it doesn't have to. Example:

    
    
        !(a || b) == (!a && !b)
    

`a` and `b`. They're meant to represent, well _anything_ really but to the
average person (or weirdos like me) it would be infinitely easier to
understand if the author wrote it like this:

    
    
        !(Bob || Sally) == (!Bob && !Sally)
    

I don't know why but whenever I encounter single-characters-as-math-variables
in computer science I have to spend _far_ more time unpacking such statements
into things my brain can understand. It is _so much easier_ if such statements
are written as _nouns_ that describe actual _things_.

I remember having a hard time working with C for loops until a friend of mine
wrote one like this:

    
    
        for (int number = 0; number < 10; number = number + 1)
    

Suddenly it all made sense! I am not alone in this! I have taught programming
to kids and others who have struggled to learn programming and this seems to
be how _normal_ people think.

Note: We, as a geeky community are _not_ normal. I just have a strange
exception in that at least one little part of me thinks like a normal person
and that is with "what we name things" :)

~~~
chrisseaton
I can't understand any point to writing 'a' rather than writing 'Sally'. What
practical difference does it make?

~~~
cecilpl2
It makes it WAY easier to understand exactly why !(a || b) == !a && !b.

Bob and Sally are things that people already understand and relate to. a and b
are not, so there is an extra layer of indirection there when attempting to
reason about them.

!(Milk || Sugar) == (!Milk && !Sugar), duh.

~~~
chrisseaton
Sorry I still don't get it.

> Bob and Sally are things that people already understand and relate to.

People understand they're people's names. But that's not a relevant detail for
the equation, so why add that to the problem? It complicates, rather than
simplifies, my understanding of it.

> !(Milk || Sugar) == (!Milk && !Sugar), duh.

I don't get what the 'duh' is. You've added nothing to the equation! They're
just different names!

~~~
uryga
_`with_milk`_ and _`with_sugar`_ are booleans that people have (lots of)
experience with. this seems to help us think about this stuff – we've seen how
milk/sugar "work" many times, and we've already internalized some model of
that situation – we already "get it" on an intuitive level. hence, reusing
that model for general Boolean variables reduces the cognitive effort
required[0].

for a similar example, see [1]. summary: a study presented people with a
logical problem. they failed miserably if the question was about abstract
math-y stuff, but got it right most of the time if the variables were like _`X
is underage`_ and _`X can buy beer`_ (explained in the "Policing social rules"
section)

\---

[0] as a bonus, we can quickly sanity-check if our reasoning is correct – if
there's _`no (milk or sugar)`_ in my coffee, there's obviously _`(no milk) and
(no sugar)`_ in it!

[1]
[https://en.m.wikipedia.org/wiki/Wason_selection_task](https://en.m.wikipedia.org/wiki/Wason_selection_task)

------
tylerhou
> It’s so large that it can hold an infinite number of disjoint copies of the
> natural numbers inside it!

Nit: you can embed infinite copies of the natural numbers inside the natural
numbers as well, with room to spare. Consider the primes raised to integer
powers.

------
srikz
Off topic: DeMorgan's law

> the negation of a disjunction is the conjunction of the negations

As a non-English speaker I hated this definition until someone told me to
remember

> 'break the line, change the sign'.

This is of course referring the representation in boolean algebra, where,
NOT(A AND B) is represented with a single long horizontal line on top of (A
AND B). This becomes NOT(A) OR NOT(B), i.e., Ā | B̄ according to DeMorgan's
law.

~~~
giornogiovanna
De Morgan's law is just "not all true <=> at least one's false". I'm not sure
why people complicate it beyond that.

~~~
blattimwind
In university I had a course (curse?) called "logic in formal systems" (or
something like that); I doubt many using only the course's materials truly
understood the logic principles demonstrated there. Coincidentally, the course
insisted on using ∧, ∨, ¬ etc., while another course also dabbling in a lot of
logic using natural symbols was far better in terms of presentation and
explanations. (To this day I have to look ∧, ∨, con- and disjunction up to be
sure which is which).

~~~
Izkata
> To this day I have to look ∧, ∨, con- and disjunction up to be sure which is
> which

Tip: ∧ is shaped like the "A" in "And".

Hopefully others know tricks for the other symbols.

~~~
blattimwind
> Tip: ∧ is shaped like the "A" in "And".

Meanwhile ∨ is shaped like the "U" in "Und" ...

------
SeanLuke
Apple's been in this game a long time. NewtonScript had a powerful function
called "IsHalting". See Page 23-84 of
[http://www.newted.org/download/manuals/NewtonProgrammerRef20...](http://www.newted.org/download/manuals/NewtonProgrammerRef20.pdf)

------
nathan_f77
That was really interesting!

> This is completely impossible to do with Int’s and String’s, but here we
> have done it for BitSequence

I'm still confused by this part. Can't you convert any Int or String value
into a BitSequence?

~~~
fmap
He means that it's impossible to write such a function from the natural
numbers. It's possible for all finite types such as UInt in Swift.

In general, you can do this exhaustive search with any "compact" type and
there are a lot of compact types. In particular, the (total continuous)
functions from a discrete (i.e. a type with decidable equality) into a compact
type are compact. And as the article shows, the (total continuous) functions
from a compact into a discrete type are discrete. Together with the
observation that the type of natural numbers is discrete and that every finite
type is compact and discrete already gives you infinitely many compact types
to play with.

One caveat with this whole work (which goes back to Martin Escardo by the way)
is that this doesn't work with general recursion. E.g. in a language with
general recursion you can write a program

    
    
      kleene : (nat -> bool) -> bool
    

which computes the paths in the Kleene tree, where roughly "all total
computable paths are terminating, but all uncomputable paths diverge".
However, if you have a (total) language with, e.g., only structural recursion,
everything works out and you can apply this epsilon operator to arbitrary
programs.

~~~
nathan_f77
Thanks for your reply! I think I understood some of your comment, but got very
lost at the end. I don't know anything about: Kleene trees [1], "total"
languages [2], general recursion vs structural recursion [3], or epsilon
operators [4]. (But I've looked those up and provided some links.)

I think I understood some of the first paragraph. I'll have to do some
mathematics and computer science courses on Khan Academy.

[1]
[https://en.wikipedia.org/wiki/Kleene%E2%80%93Brouwer_order](https://en.wikipedia.org/wiki/Kleene%E2%80%93Brouwer_order)

[2]
[https://en.wikipedia.org/wiki/Total_functional_programming](https://en.wikipedia.org/wiki/Total_functional_programming)

[3] [https://stackoverflow.com/questions/14268749/how-does-
struct...](https://stackoverflow.com/questions/14268749/how-does-structural-
recursion-differ-from-generative-recursion)

[4]
[https://en.wikipedia.org/wiki/Epsilon_calculus](https://en.wikipedia.org/wiki/Epsilon_calculus)

This seems like a very advanced topic!

------
keithalewis
Balderdash, as others have more graciously pointed out. How wrong do you have
to be before having your nonsense deleted from HN?

~~~
wool_gather
Even if it is balderdash, isn't there value in analyzing(/refuting) it? I
submitted the post (it's not mine) because I was interested in hearing people
with more maths than I have expand/comment on it -- perhaps even people who
had read the original papers.

If there's something particular in the post that you find problematic, I
personally am all ears.

~~~
keithalewis
The author is correct that checking if two functions having infinite domains
are equal is often impossible. He also has correct statements of some
mathematical theorems. But people in this thread have provided counterexamples
to his claim. That means he needs to fix the mistakes he made. If you are
interested in what serious mathematicians have to say on this topic see
[https://homotopytypetheory.org/](https://homotopytypetheory.org/).

~~~
jozefg
It is possible in certain cases, that's the point of the article. One of the
contributors to HoTT is the author of the paper introducing these algorithms
:)

