
Sometimes all functions are continuous (2006) - lelf
http://math.andrej.com/2006/03/27/sometimes-all-functions-are-continuous/
======
bordercases
The article is a slow burn with a hilariously clickbait title but damn, those
are some fascinating and significant consequences.

> An interesting question comes to mind: which programming features, apart
> from the ones we mentioned above, allow us to program [computably continuous
> integer streams]? In terms of Haskell, the same question is which monads
> allow us to program [these streams]?

> Can we do it without any extra features and use just “pure” functional
> programming?

> If by pure functional programming we understand a functional programming
> language with natural numbers, booleans and recusive definitions, also known
> as PCF, the answer is _no_. The proof of this uses denotational semantics
> and domain theory, and I will not go into it now. You may entertain yourself
> by trying and failing to define [computably continuous integer streams] in
> pure Haskell or ML, i.e., no side effects, no exceptions, no mutable store,
> no parallelism, no monads.

> ... The lesson is for those “experts” who “know” that all reasonable models
> of computation are equivalent to Turing machines. This is true if one looks
> just at functions from N to N. However, at higher types, such as the type of
> our function m, questions of representation become important, and it does
> matter which model of computation is used.

It takes a hit on the trend that was in vogue in early 2010s to do AI and
metaphysics on top of Kolmogorov Complexity, Solmonoff Induction, information
physics... people took the Turing Equivalence as a carte blanche to say
whatever willy-nilly thing you want about the (potentially) computable nature
of reality without being informed by more naturalistic methods.

This is harshed even further given that these information-theoretic arguments
often used non-computable notions.

------
alain94040
I'm not making much sense of that article. It's rejecting both the
mathematical definition of continuity, and also wants to compute with infinite
real numbers, which is just not realistic to computers and languages. So the
author ends up with a lot of absurd results.

~~~
throwawaymath
That's because the author is using constructive mathematics (i.e. computable
analysis instead of standard real analysis). It is true that all _computable_
functions are continuous. Likewise discontinuous real functions aren't
computable.

The continuum hypothesis is tangentially related to this topic and makes for
fun reading.

~~~
jayd16
>In fact, no finite amount of computation will guarantee that we will be able
to tell whether x=0 or x>0

Hmm, I'm still stuck at this assertion. Why can we not assume the number is
finite?

If we assume an infinitely long number takes an infinite amount of time to
read, can we not also assume it must take an infinite amount of time to write?
If we only have finite time, can we not assume all numbers given to the
function in that time are finite?

~~~
afc
Yeah, I'm also stuck here.

The article seems to say that, because you can't produce an upper bound to the
amount of time the sgn function will take to run (for all possible inputs),
sgn isn't a function. But then... isn't it the same for every single other
function?

I think the article is conflating "given a fixed amount of time, one can find
an input for which the function will take longer to run" with "the function
takes infinite time". The later isn't true: for _any_ given input, no matter
how big, one can compute a time such that the function _will_ finish in that
time; in other words, the function always finishes, in finite time, for every
possible input, no matter how large.

It's possible we're both confused, I suppose. :-)

~~~
Ericson2314
Here's the thing: the "discrete vs continuous" most people were taught is
wrong. Discrete is [can always be construed as] continuous. Only infinite
things can be discontinuous.

State is finite but time is countably infinite for our purposes, so we model
infinite/unbounded things with programs that can run arbitrary long.

Finally, this abstract math stuff is in fact a really good UI point that most
programmers miss. In non "real time" applications, you should aim to be able
to dynamically tradeoff tardiness and richness; e.g. a fancy diagram that is
rendered at low res and then higher res. Likewise all your caches should be
evictable under memory pressure. Computing should feel fluid.

It's a pity most people only paleolithic state machine math or terminating
thing math. This falsely implies that "real world programs" which hardly ever
terminate are beyond theory, or that the smartypants thing to do is break them
down into little terminating programs and some big spooky event loop
whateverthefuck (browser, apache, framework du jour, etc etc.). Build codata
out of codata!

~~~
Retric
How would you turn f(x != 1) = 0, f(1) = (has no value) to be continuous?

Because it sounds like you’re simply redefining the terms. At which point you
might as well be using “Spork”. Because, defining a new system has zero impact
on a different system.

~~~
GolDDranks
I believe that the function you just defined is already continuous. It's not
defined at 1 so checking its continuity doesn't make sense there, and at every
other point it's clearly continuous because it's constant. Points that are
arbitrarily close to 1 are still continuous because the intervals around 1 are
open.

~~~
Retric
In calculus f(x) = x/x - 1 is not continuous at 1 based on the initial
definitions. Moving to fixed-point arithmetic the idea of limits gets odd, but
the function still needs a definition for that input.

~~~
GolDDranks
I would say the continuity isn't just defined at that point... I very much
agree that one can't affirmatively say that the function is continuous at that
point. Btw. I guess you meant to say f(x) = (x - 1)/(x - 1).

However, let's ignore that; the problem disappears if we define f(x ) = 1 at x
= 1, and f(x) = 0 elsewhere.

In standard analysis that function would be discontinuous at that point.

I'm not sure about implications in the OP's kind of analysis. Would the
existence of m(1) imply that there exist some smallest input that is larger
than 1, that would make difference in output? Same for the largest input that
is smaller than 1.

------
edflsafoiewq
A related fact: all functions are defined on closed sets. You cannot "tell"
whether a number is in a set or is a limit point of a set. Since all functions
are continuous, they work at limit points too.

As a "practical" consequence: the common design of a random() function
returning values in [0, 1) is wrong. Any function you write cannot "tell"
whether it got a 1 or not.

These are still very good guidelines even for floating-point code, as long as
you, like most everyone else, treat floats as a blackbox abstraction for
"real" real numbers.

~~~
poizan42
Well, random() functions that are normally used are defined on the [0, 1)
interval of 32-bit or 64-bit IEEE 754 floating point numbers, for which it is
very much computable to determine whether they are exactly 1 (there's nothing
"real number" about them though, they all belong to a small finite subset of
the rations). Of course that isn't really an open interval in the topological
sense either.

~~~
edflsafoiewq
But that's why it's often a bad idea to test exact equality of floats: the
equality test isn't continuous.

In fact, for float computations it's usually even more important to use
continuous (ie. error tolerant) functions, since floating-point operations can
introduce errors.

~~~
saagarjha
Comparing IEEE 754 floating point number is well defined and computable
_because_ the domain is discontinuous. It’d be stupid to use anything else…

------
svat
More precisely, all well-defined computable functions are continuous.

For another exposition of the same (or similar) results, see also Dan Piponi's
post from 2008 "What does topology have to do with computability?" at
[http://blog.sigfpe.com/2008/01/what-does-topology-have-to-
do...](http://blog.sigfpe.com/2008/01/what-does-topology-have-to-do-with.html)

Also, as a follow-up, you may be interested in Martin Escardo's 2007 post on
the same blog: [http://math.andrej.com/2007/09/28/seemingly-impossible-
funct...](http://math.andrej.com/2007/09/28/seemingly-impossible-functional-
programs/)

------
skybrian
I'm not a mathematician, but it sounds like what this is saying is that as
soon as you allow a number to contain an infinite number of digits, you can
get an infinite loop when you try to do a simple comparison. (For example, try
to compare pi to an number that's not-quite pi. It might be a little more or a
little less, but you can't know this without looking at all the digits, which
is impossible.)

As a consequence, there are only two boolean-valued total functions: always
return true, or always return false.

I guess that's why we don't use infinite streams of numbers for computation.

~~~
lonelappde
You don't need to look at all the digits, but there is no finite stopping
point you can choose in advance.

It's similar to the problem of comparing two natural numbers given in least
significant bit order.

~~~
skybrian
Given a comparison function, an adversary can construct streams of digits that
will make it run forever. This would be true either way. The problem seems to
be with allowing algorithmically generated streams as input?

Maybe our domain should be limited to numbers generated by algorithms that are
known to terminate? But despite eliminating irrational numbers, this guarantee
doesn't seem like enough for practical computing. Even arbitrary precision
algorithms require their inputs not only to be computable, but actually
computed in advance, showing at least that they fit in memory.

------
tel
The punchline at the end is worth calling out: Turing Equivalence applies at
first-order computations like `Nat -> Nat`, but fails at higher-order
computation like `(Nat -> Nat) -> {0, 1}`.

------
improv32
For a very nice into to constructive math, see this[0] link as well as some
good HN discussion here[1].

[0]:
[https://home.sandiego.edu/~shulman/papers/rabbithole.pdf](https://home.sandiego.edu/~shulman/papers/rabbithole.pdf)
[1]:
[https://news.ycombinator.com/item?id=18411935](https://news.ycombinator.com/item?id=18411935)

------
carapace
Greg Buchholz in the site's comments linked to Gregory Chaitin's "How real are
real numbers?"[1] and asks,

> Isn't it a little bit crazy to talk about computations on real numbers, when
> most of them are uncomputable/unnameable?

[1]
[https://arxiv.org/pdf/math/0411418.pdf](https://arxiv.org/pdf/math/0411418.pdf)

> Experimental physicists know how difficult accurate measurements are. No
> physical quantity has ever been measured with more than 15 or so digits of
> accuracy. Mathematicians, however, freely fantasize with infinite-precision
> real numbers. Nevertheless within pure math the notion of a real number is
> extremely problematic. We’ll compare and contrast two parallel historical
> episodes:

> 1\. the diagonal and probabilistic proofs that reals are uncountable, and

> 2\. the diagonal and probabilistic proofs that there are uncomputable reals.

> Both case histories open chasms beneath the feet of mathematicians.

------
Nevermark
I don't understand the point of arguments about constructible functions on
unconstructable reals. I.e. reals with infinite digits and no finite symbolic
representation.

For anything actually computable (in finite time), on computable (in finite
time) values, you will always get discrete (not continuous) functions on
discrete (not continuous) values.

~~~
vilhelm_s
The idea of computable reals go back to at least Turing. The idea is that a
real number (or an operation on real numbers) is computable if we can compute
the decimal digits, one by one. This way, if the user wants to know it to any
desired precision, they just have to wait until the computer has printed that
many decimals. You can equivalently treat them as terminating computable
functions of type (nat -> bool), which computes the n:th binary decimal.

This is actually a quite constructive notion! At least, it works well with
intuinistic mathematics, which is why Andrej Bauer is writing about it.

~~~
Nevermark
Ok, that makes sense. Even when restricting numbers to reals with finite
symbolic/program definition, comparing two such reals requires an unknown and
unbounded number of their digits to be computed. If two symbolic
representations define the same number, but it isn't obvious, a complete digit
comparison will never end. Got it.

Thank you!

------
loicd
The result presented in TFA is that all _computable_ functions _over the real
numbers_ are continuous. To generalize that to 'all functions are continuous',
you not only need to believe that all functions should be computable, which
sounds reasonable for a constructivist, but also that all numbers should be
real, which is not. Back when I was in academia, I remember a talk by a
mathematician who thought that we should all be doing analysis with rational
numbers rather than real numbers. With such a perspective, the sign function
defined over rational numbers (e.g. presented as a pair numerator,
denominator) is both discontinuous at 0 and computable.

------
mianos
This reminds me of the coastline paradox:
[https://en.wikipedia.org/wiki/Coastline_paradox](https://en.wikipedia.org/wiki/Coastline_paradox)

Specifically, due to the fractal nature of a coastline the length diverges to
infinity. The theoretical length at the infinite scale is infinity.

------
Gehinnn
How do you implement algorithms for real numbers if there uncountable many of
them? There is always a real number your algorithm cannot read/process.

------
mturmon
Note that the comments thread below the article addresses some of the natural
objections to the novel results exhibited in the post.

~~~
quotemstr
In particular,

> the real number is not given by an infinite sequence that is actually
> written down anywhere, but rather as an algorithm, or a black box, which
> accepts a number n and outputs the n-th digit. At no point do I manipulate
> an infinite amount of information, yet I can look at any digit I want. Of
> course, I cannot look at all digits in a finite time, which is sort of the
> point of the post.

