
On Two Views of Computation in Computer Science - pron
https://pressron.wordpress.com/2016/08/30/what-we-talk-about-when-we-talk-about-computation/
======
Double_Cast
> _You’re free to think about computation as acting on bits… but for those
> bits to do us any good, they have to actually represent something (e.g.,
> data structures)._

I suspect this is a case of confusing epistemology with ontology.

Galaxies are ontologically composite in that they're the highest holonym in
the supervenience hierarchy [0]. Atoms are ontologically fundamental in that
they're the the lowest meronym in the supervenience heirarchy.

Galaxies are epistemically abstract in that the notion of galaxy is far
removed from everyday experience. Atoms are _also_ epistemically abstract in
that the notion of atoms are also far removed from everyday experience. What's
epistemically fundamental are the immediate objects of our everyday
experience, like chairs and food. (My impression of Zen is that it advises to
focus more on things like chairs and food [1].)

[https://www.xkcd.com/435/](https://www.xkcd.com/435/)

In the xkcd, which is more "fundamental", Math or Economics? Well, it depends.
Likewise, what's more "fundamental", bits or ADT's? Well, it depends.

[0] powers of 10:
[https://www.youtube.com/watch?v=0fKBhvDjuy0](https://www.youtube.com/watch?v=0fKBhvDjuy0)

[1] [http://www.itsokblog.com/2011/01/wash-your-bowl-zen-of-
mothe...](http://www.itsokblog.com/2011/01/wash-your-bowl-zen-of-
mothering_3089.html)

~~~
pron
Well, in this case all models are quite abstract, and there's a very real
computational complexity difference between them. The key, if I were to use
your way of presenting the issue, is the recognition that some epistemological
abstractions are more computationally costly than others. So it's true that
you can choose either representation as a foundation, but the representations
objectively and radically differ in their computational complexity.

~~~
Double_Cast
The key is to recognize how it's not just that TOP and TOC both measure X with
different models. It's that TOP measures X while TOC measures Y.

It's as if we claimed Spell Checking were as complex as Natural Language
Processing because they were both called "linguistics". Yes, they both
abstract over language. But the primitives of Spell Checking are _glyphs_
while the primitives of NLP are _morphemes_. Glyphs don't exhibit the same
patterns as morphemes. Which is why Spell Checking uses a Trie while NLP uses
a Matrix. Debating whether glyphs or morphemes are "more fundamental" is a
distraction.

Likewise, bits don't follow the same patterns as ADT's. Calling both _the
study of bits_ and _the study of ADT 's_ "Complexity Analysis" is misleading,
which I believe is what Pressler was getting at. But we've put bits and ADT's
in the same bucket for the entire history of computing, because the Church
Turing thesis (i.e. anything a TM can do, lambdas can do too) lead us to
believe Turing and Church were studying the same primitives.

~~~
pron
What I was getting at in the article is that:

1\. TOC and TOP ask different questions.

2\. The disparity of the computational complexity involved in the two classes
of models is so great, that it is objective proof that they represent two
distinctly different things, and therefore comparing them directly is
meaningless. That a jet and a bicycle require vastly different amounts of
energy to power is conclusive and objective proof that completely different
tradeoffs must be involved in choosing them.

------
mathgenius
I think mathematicians would call this "representation theory." This is the
whole question of do we think of a calculation as syntax, or can we represent
this syntax as a "dynamic", or action; something that is being moved around or
transformed.

Some people really are driven to just calculate stuff (pushing language itself
around), and others seem to want to know "what does it mean?" Ie. what are the
representations of this language. Logicians call this semantics: it's a model
for the syntax.

I'm a theoretical physics grad student, and see plenty of this. Most
physicists seem to just want to calculate stuff, and don't care where the
symbols "live". For me, a calculation is rarely enlightening. I want the
context, the type theory that tells me how the pieces talk to each other.

~~~
Sniffnoy
I think you've got an interesting point, and I'd like to add to it a bit and
refine/correct it a bit.

First off, minor thing, but as Jeremy has pointed out, your terminology is
wrong; "representation theory" means something else. You seem to mean
something more like "model theory".

But anyway -- I'm a mathematician, and I've noticed something like this too
when I've dealt with physicists or their writings. But before I discuss
physicists, let's discuss mathematicians, because I think you've
mischaracterized them a little.

(Note, all claims here are just based on my experience.)

You talk about considering syntax vs. considering the different models of the
syntax. But most mathematicians, unless they are logicians or set theorists or
something, don't take either of those views. They take the Platonic view --
they're discussing a world of mathematical objects; the statements just
describe it. They're not considering the statements as primary and then
considering multiple models of it, they're considering the one true world of
mathematical objects and describing it with statements (which might have other
models, but who cares). This point of view breaks down somewhat when you have
to deal with things like the continuum hypothesis and such, but that sort of
thing doesn't really come up in ordinary mathematics.

(Note though that taking the Platonic view does not necessarily mean taking
the "theory of programming" perspective the article discusses. I mean, I
wouldn't take Aaronson's view, that computation is prior to everything else;
that's pretty incompatible with it. But to my mind, computation is
fundamentally about finite strings over a finite alphabet. Talking about
computation with higher types -- where the Church-Turing thesis can fail -- as
though it were, well, _computation_ , feels really weird to me.)

Also, another terminological correction: Most mathematicans don't care about
type theory. They care about _types_ , at least implicitly, though they might
not use that term; but they are not going to consider types as an object of
study in and of themself. But yes -- when you want to understand a
mathematical object, one of the first questions is, "What sort of object is
that?" or "What space does that live in?"

So anyway -- yeah, physicists. I've noticed this too, and I don't know what to
make of it. Where physicists don't tend to fully specify their, let's call it
an ontology, and have a hard time answering questions like "What sort of
object is that?" or "So that's a function from what to what?" Where they don't
seem to properly distinguish between a mathematical model or description, and
a method of calculation that's useful when working with that model or
description. (I have so often failed to get an answer to, "So are virtual
particles an actual physical phenomenon predicted by the quantum field theory,
or are they just part of a useful method of calculation for working with
quantum field theory?") Where they seem to care primarily about methods for
getting numbers out, rather than building full, coherent models.

It's all weirdly instrumentalist, which to me seems backwards from what you'd
expect -- like, mathematical Platonism is a philosophical question, but the
physical world is definitely real! And sure, maybe the bits we can't measure
are more of a philosophical question too -- especially since one can quite
possibly come up with multiple isomorphic usable models for the laws of
physics, whatever they turn out to be -- but still, mathematicians have no
trouble just treating their mathematical objects as real things to be reasoned
about, so this kind of instrumentalist view among physicists seems weird to
me.

I hadn't even considered the possibility that they were thinking "syntax-
first", so to speak! But I guess that does kind of sum it up pretty nicely.
Bugs the hell out of me, though. Makes it pretty difficult to talk about the
math of their theories with them, too.

~~~
mathgenius
> your terminology is wrong; "representation theory" means something else. You
> seem to mean something more like "model theory".

I really do mean representation theory, in the widest sense of the word. Wide
because I am trying to make connections here, not distinctions.

So for instance, of course group representations. On the one hand there is an
abstract group multiplication, and on the other side we are representing this
as a geometric action.

The same thing goes for "bigger" gadgets, lie algebras, hall algebras, etc.
All kind of algebraic objects that we can define abstractly (syntactically)
but then lo! it is actually represented by some kind of (geometric?) action.

And then in a more logical vein you have the lattice type theories. Boolean
algebras, heyting algebras. These things have topological representations (or
is it functions on topological spaces). And then to denotational semantics of
lambda calculus. And it's all kind of a syntax on one side, and a semantics on
the other.

I don't know much about models of set theory, but I'm assuming these also
involve a kind of "active" representation of syntax. I would also include
things like algebraic topology, topological quantum field theory. These are
all "functors" which act to represent an algebraic gadget in a more dynamic
way (i'm possibly over-generalizing here.)

Anyway, there's no concrete definition for what I mean by a "representation"
so that's why i gave some examples. It's quite mysterious imho.

> So anyway -- yeah, physicists.

Well, it's easy to criticize physicists, but I also think mathematicians are
guilty of this. Ok, so they define their calculations rigorously (not always),
but then it's gloves off and away we go! I've come to realize that the whole
program of categorification is an attempt to bring more context to these
calculations (this is what I would call type theory for mathematics btw.) And
most mathematics is written down as calculations, without this context.
Probably because it's much harder to categorify everything. The simplest
example of what I mean is this: "why keep track of a (finite) set when you can
just say how many elements are in the set?" And so on. I really do think that
alot of mathematicians are just as guilty as physicists of the "shut up and
calculate" attitude. But, they are certainly much more humble :-)

~~~
j2kun
What you're describing sounds more like homology theory, where there is an
abstract set of concepts (short exact sequences, etc.) which become different
theorems in different contexts. But, unless I'm mistaken, most of the time
these are closer to unifying foundations, or starting points, from which the
special features of each model allow you to say more beyond the abstract
framework. So sure you can get some general category-theoretic theorems, but
saying the specific model is useless (as the Language folks say about Turing
machines) is ludicrous.

------
LolWolf
This is an interesting article, but I'm still fairly confused about the notion
that is made

>Harper’s attack on the utility of machine models and lack of modularity is
tantamount to an architect saying to a chemist, “both of our disciplines
concern the arrangement of molecules, yet my discipline is superior, as yours
doesn’t even have the modular notion of a room!”

But wouldn't the whole point be that a chemist, seeing the room as nothing
less than an abstraction, can also understand the room through their given
framework? Much in the same way that, for any computational model, we can
think of a TM with an Oracle tape, or some similar IO model, that has the same
computational power as any defined language?

I'm afraid that I thought I understood the distinction at first: where those
who support CT view computation as a fundamental case (e.g. every other
mathematical 'primitive' can be derived from a computation with the
appropriate oracles[1]) and and the TOP view which sees mathematical
foundations as the main case, and then sees computers as being an abstraction
of functions in some sense, such that one can construct a calculus of these
functions which constitutes a language.

Did I get this all wrong? I'm afraid I still don't understand the main
distinction to be honest. It seems that both parties are going around in
circles on exactly the same thing, but assuming different axiomatic systems
and showing that each is equivalent to the other.

[1] For example, with no oracles, we receive all of constructive mathematics,
etc.

~~~
pron
You can translate and transform many things to many others. The important
question is what is the computational complexity of the transformation? If it
is large, then the two things are very different.

As Scott Aaronson once said, waterfalls could be viewed as playing chess, but
as the complexity of the translation from the "waterfall language" to a chess
language understandable by humans is as great as the complexity of an actual
chess-playing program, all the work is done in the translation.

So the question you should be asking is where the complexity is being spent
and on what.

~~~
LolWolf
Wait, but I thought the question being posed wasn't about the computational
complexity of any model but about the power of a given computational model
relative to another (e.g. the subject of the CT thesis).

Even if we're talking about just computational complexity, if we allow Turing
machines with arbitrary oracles, or (equivalently) we allow arbitrary
mathematical constructions in languages, I'm still unsure as to why the
complexity matters.

W.r.t the waterfall case. Sure, I agree with that statement, but I'm unsure as
to how it plays into the final case---the question isn't about a specific
model performing a specific task, but about a model being able to completely
replicate another in terms of computational power. Anything I can do in a
language (with some functions) I can do in a Turing machine with the
appropriate oracles; and conversely, anything I can do in a Turing machine
with some given oracles, I can do in a language.

~~~
pron
Both models are equally powerful in the things they compute in terms of the
Church-Turing Thesis. The TOP view also has other consideration. My point in
the article was not to settle the debate on Aaronson's blog but to show that
language models and machine models are objectively and inherently so different
from one another that comparing them in terms of programming concepts (like
types) is meaningless.

~~~
AnimalMuppet
And it seems that most of the "my way is superior to yours" is based on
setting up strawmen of "yours" to make it be addressing the same questions as
"mine".

~~~
pron
Yes, but it goes a little further. I can't say my way is superior to yours if
my way means solving a harder task _and_ paying market price for it.

I got the leather seats because I chose to pay full price for them. That they
come as a standard option with the car because the car is more expensive is
not an argument. You can choose to pay the same difference in price and have
them installed in your car, too.

Whether those leather seats matter or not is no longer an objective question
but a question of values, and here is where caring about completely different
values is the point.

------
pklausler
The author contends in the article's abstract that machine models are "self-
contained" whereas "language models" (not clearly defined, but meant to
encompass at least an untyped lambda calculus) are not. But both can be viewed
as a state plus a well-defined set of rewriting rules, so I suspect that the
article is struggling to make what is really a distinction without a
difference.

~~~
pron
Author here.

> But both can be viewed as a state plus a well-defined set of rewriting
> rules, so I suspect that the article is struggling to make what is really a
> distinction without a difference.

1\. I state that the untyped lambda calculus is, indeed, a borderline case,
and is a simple rewriting system which, in turn, is a special case of a
nondeterministic abstract state machine.

2\. That many things are abstract state machines does not mean that the
computational complexity required to validate whether what you have is a well-
formed description is similar.

3\. That complexity difference is absolutely huge -- zero for the machine
models vs. exponential to undecidable for the typed language models. It is
hardly a struggle to draw the conclusion that the two are objectively
completely different classes.

Also, I gave a list of machine and language models. The latter group consists
of the process calculi and the lambda calculi.

~~~
amatus
Could you encode all untyped lambda calculus expressions by just numbering
them and writing the number as the string of bits?

Maybe this enumeration is the "external collaborator" in this case, but it's a
very simple one.

~~~
catnaroek
Inside the lambda calculus, the only thing you can do with a lambda
abstraction is apply it.

And one thing you can't do using your numbering scheme (which is basically a
Gödel numbering) is establish whether two syntactically different functions
are extensionally equal (under some fixed reduction strategy).

------
Kenji
_It’s really weird that the Church-Turing thesis, which is ridiculously robust
at first order, falls apart so comprehensively at higher type._

A statement like that makes me feel like Neel Krishnaswami does not understand
the Church-Turing thesis _at all_ (what does the Church-Turing thesis even
mean at a "higher type"), but considering that this person has significant
experience in this field, much more than I do, I am just going to assume that
it is me who does not understand until I read up on this exactly.

~~~
catnaroek
The Church-Turing thesis is a statement about which functions of type `nat ->
nat` are computable, either in the lambda calculus or using Turing machines.
This completely disregards:

(0) Computational complexity. Neither the lambda calculus nor Turing machines
were invented to answer questions of computational complexity.

(1) Functions at types other than `nat -> nat`. Curiously enough, programming
in the real world is seldom about evaluating functions of type `nat -> nat`.

~~~
pron
> The Church-Turing thesis is a statement about which functions of type `nat
> -> nat` are computable, either in the lambda calculus or using Turing
> machines.

This is how Krishnaswami frames things. From the TOC perspective, there is no
such thing as functions at all. Computations consume and produce strings. Both
Church and Turing explain how those strings can encode the natural numbers
(there was no talk of types by either). Interpreting a computation as one
computing functions on the natural numbers or any other set is an entirely
interpretive activity that is external to the computation itself.

~~~
tome
How then do _you_ state the Church-Turing thesis? I note that the Wikipedia
page claims that Church and Turing themselves used the word "function":

[https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis#S...](https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis#Statement_in_Church.27s_and_Turing.27s_words)

~~~
pron
They do use the word function, just not in terms of the language itself, but
in the same sense of saying that a falling apple integrates the function from
its mass to acceleration; it's a human interpretation of the encoding. I would
state the Church-Turing thesis like so: a universal computation model can
compute anything any physical machine can. As Hodges says in one of the links
in the article, this is how Church and Turing themselves understood it. How
would I define "anything"? As a function on any countable set -- the natural
numbers, would do -- and I would explain what it mean to compute a function:
pick a function, pick an input element, pick a finite representation for the
input and output; if a physical machine can compute the output representation
from the input representation, so can a universal computation model.

I could choose a function between _any_ two countable sets, e.g. (Nat -> Nat)
-> (Nat -> Nat): pick a representation for the input and output and all models
can compute the same functions, and those are the functions that machines can
compute. They differ in abilities regarding representation _within_ the model,
but as the language models' representation require very significant
computational work, I'd expect their representation to do more: they're paying
for it.

A C compiler works hard, so you'd expect it to provide you with a
representation more suitable to your wishes than machine code. It's just that
as a compiler like C translates a "meaningful" language to a machine encoding,
everybody notices the work. The language "computation" models work in a
vacuum, so I just pointed out that even though they don't translate anything
from one representation to another, they still have to pay for their natural
representation.

~~~
kd0amg
_a function between any two countable sets, e.g. (Nat - > Nat) -> (Nat ->
Nat): pick a representation for the input and output_

There isn't a TM-suitable representation for things of type (Nat -> Nat). No
matter how you _try_ to encode a Nat->Nat as a Nat (or a bitstring, if you
prefer), you're going to screw up equality (which means you're actually
encoding _some other type_ ). You'll map two unequal functions to the same
Nat, or (more likely) map one function to multiple unequal Nats. This is the
difference between how lambda calculi and Turing machines handle higher-order
things: A Turing machine can always inspect the "internals," so you need an
actual _bijection_ from functions to strings, whereas lambda calculus hides
the internals, so it's safe to map one function to multiple syntactic
representations.

~~~
pron
If you take the normal mathematical definition of a function as a mapping from
one set to another, pick two sets, pick a representation and all models can
compute the same mapped elements. Tell me what your (higher-order) function is
— i.e., what output element it maps to what input element — and a TM would
compute the exact same outputs. That it can’t _internally_ “reason” about
those functions is obvious, but, as I showed, that ability comes at a cost.
You could build, say, a Haskell compiler that would pay that cost and
translate the language to a machine representation (or, speaking more
abstractly, a TM could simulate, say, System Fω — at a computational cost), or
you can use an abstract language model without any underlying machine, but the
same cost must be paid regardless.

This shows that the concept of a type requires actual computational _work_
rather than just a matter of “free” perspective. Your mere presentation of a
function as an inhabitant of a type creates a computationally harder problem
than presenting a function as a mapping between sets. Put differently, when a
typed language model computes a function (Nat -> Nat) -> Nat, it does more
work, and solves a harder problem than necessary to compute that function in
the normal set-theory sense. You can choose to solve that problem on a TM by
simulating type checking if you like, but you can’t compare “compute a
function in the set-theory sense” and “compute a function in the type theory
sense”, because those are two different problems with two radically different
computational complexity costs. You can decide to do that extra work or not
(you may want to because you gain benefit from it), but you can’t compare a
model that always does — and pays handsomely for it — with one that doesn’t.

Otherwise, I could create a language model that solves all NP problems in
polynomial time by defining my language as requiring a proof certificate with
every input problem, which my model would then check in polynomial time. No
one would believe my language actually really solves a computational problem:
it simply pushes the difficulty to the collaborator.

Equality is also an internal question. Define any relation between two sets
(your definition of equality), and all models would be equally able (or
unable) to decide it.

~~~
catnaroek
> and solves a harder problem than necessary to compute that function in the
> normal set-theory sense

People who use set theory as their foundations impose on themselves the even
harder task of making sure their mappings are coherent w.r.t. whatever notion
of structure they want to work with, i.e., homomorphisms in a category of
interest.

> Otherwise, I could create a language model that solves all NP problems in
> polynomial time by defining my language as requiring a proof certificate
> with every input problem

No, this is wrong. Changing the input type means that you are _not_ , in fact,
solving the original problem.

~~~
pron
> People who use set theory as their foundations...

Let's not get into this old debate because we'll never get out of it. Suffice
it to say that it that there is no doubt that Turing and Church (and Gödel and
von Neumann) used set theory as their foundation, and when they said
"function" they meant a set-theoretic function, namely a mapping from a domain
(set) to a range/co-domain (set). If by "function" you mean a type-theoretic
function then you're talking about a computationally harder problem, as you
need to compute the underlying set-theoretic function _and_ check a proof. You
can certainly ask a TM to simulate a type checker and compute a type-theoretic
function. There's actual work involved, and someone must do it. You cannot say
that your formalism gives it to you for free, or you'll have a formalism that
proves things for free, which is impossible.

> Changing the input type means that you are not, in fact, solving the
> original problem.

Exactly! But that is precisely what you do when you interpret "function" to
mean a "type-theoretic function", and solve a different problem. Any work that
needs to be done under the normal meaning of "function" also needs to be done
in the typed formalism. If you need to ensure/prove something, there is work
and computational complexity involved. Pushing it to a collaborator doesn't
make it disappear (let's call this the law of preservation of computational
complexity). Someone has to do it, and in no model does it suddenly become
free. It's perfectly OK to say that you start counting _after_ much of the
hard work is done, but in that case you need to say that your model requires
upfront work, and therefore cannot be compared to models that don't. That's my
whole point.

~~~
catnaroek
Of course, proving things, that is, doing mathematics requires work. But at
least it's work that produces a useful result, unlike carrying out
computations without paying attention to their meaning. (By the way, this
meaning _always_ exists, even if you don't particularly care about it.) And
the best moment to study the meaning of a computation, in the sense of
minimizing the amount of long-term work, is as early as possible, which is
exactly what types force you to do.

~~~
pron
But this is now a programming discussion, not a computation discussion. Also,
I don't think it's right to declare things "useful" only if they're useful to
the particular problems you're trying to solve.

BTW, any logical formalism forces you to pay attention to meaning; choosing to
encode the logic in types has advantages and disadvantages, but it is not at
all the only way to logically describe a program.

~~~
catnaroek
> Also, I don't think it's right to declare things "useful" only if they're
> useful to the particular problems you're trying to solve.

Pray tell, what's the use of a computation without paying attention to its
meaning? My use case is simply “knowing the meaning as early as possible”, and
there ought to be no discussion that this should be everyone else's use case
as well. Nobody performs a computation without some sort of expectation about
the relation between the inputs and the outputs.

> BTW, any logical formalism forces you to pay attention to meaning

I never said types are the only way. For instance, you could use Hoare logic,
but it's notoriously more difficult to use, precisely because it gives
programs no a priori meaning.

~~~
pron
> Pray tell, what's the use of a computation without paying attention to its
> meaning?

First, I don't think that the only way to assign meaning to computation is by
_formally_ assigning meaning to the syntactic components that comprise its
program, as written in some human-readable language. Most of the relevant
meaning is in human interpretation of the input and output or the sensors and
actuators. Components' meaning may be stated informally. I am not aware that
formal meaning is the only valid meaning. People wrote the software that
landed men on the moon in assembly; that was pretty meaningful. I believe this
is still the prevailing view in computer science.

Second, you keep identifying the notion of computation with the notion of a
program written by a person. The theory of computation, from its earliest days
-- by Turing himself -- has studied computational phenomena in neural
networks, genetics (Turing was a pioneer of both NNs and genetic algorithms)
and quantum phenomena. Some of the greatest achievements in CS are not
specifically related to software (I count at least 20% of all Turing awards).

> For instance, you could use Hoare logic, but it's notoriously more difficult
> to use, precisely because it gives programs no a priori meaning.

Or TLA+, which makes types notoriously more difficult to use by comparison.

~~~
catnaroek
> First, I don't think that the only way to assign meaning to computation is
> by formally assigning meaning to the syntactic components that comprise its
> program, as written in some human-readable language.

Please suggest alternatives - that actually work.

> Most of the relevant meaning is in human interpretation of the input and
> output or the sensors and actuators.

The _relation_ between the input and the output. This can only be stated
formally.

> Components' meaning may be stated informally.

That leads to disaster very quickly.

> I am not aware that formal meaning is the only valid meaning.

Well, I am. Too much nonsense has been said as a result of not formalizing
things.

> Second, you keep identifying the notion of computation with the notion of a
> program written by a person.

No. Even if a computation “arises in nature” (which is just your way of
looking at things, actual physical phenomena is what you can measure, plain
and simple), the only way to make sense of it is to write a program that
reproduces the computation. This is how scientific theories work.

> Or TLA+, which makes types notoriously more difficult to use by comparison.

Some type systems are more difficult to use than others, and the type theory
community is definitely guilty of not paying enough attention to usability,
but I don't see how model checking has a higher power/cost ratio than sensibly
designed type systems. Which tool satisfies the following criteria?

(0) Compositionality: you can understand a large system by studying its parts
in isolation.

(1) Handles higher-order constructs (first-class functions, objects)
gracefully.

(2) Stops you from saying nonsense as early as possible.

I rest my case.

~~~
pron
> Please suggest alternatives - that actually work.

AFAIK, the only languages that fully express the computational meaning
formally with types are the dependently typed ones. AFAIK, there has been
exactly one non-trivial real-world program written in such a language, and its
author wasn't too pleased with the process (I'm referring to CompCert and
Xavier Leroy). _Every_ other program in every other languages has most of its
meaning expressed informally, in the mind of the programmer. Unless you
believe that no programs work, then nearly all programs that do express most
of their meaning informally.

> Too much nonsense has been said as a result of not formalizing things.

Formalizing things doesn't imbue them with good sense. You can state plenty of
nonsense formally. The only thing you're guaranteed is that it's _consistent_
nonsense, and, as someone said on one of my favorite TV shows, "The sole
virtue of the ineffectual is consistency".

> Which tool satisfies the following criteria? ...

TLA+, and far more elegantly and simply than any typed language you've seen.

~~~
catnaroek
> AFAIK, the only languages that fully express the com meaning formally with
> types are the dependently typed ones.

I said “formally”, not “with types”. I firmly believe in using the right tool
for the job, and some things are best handled with manual proofs. As helpful
as types might be for computer-generated proofs (type inference), I'd rather
carry out my manual proofs using good old-fashioned predicate logic. Some
proof techniques still need to be backed up by type structure, though (e.g.,
induction over datatypes).

> _Every_ other program in every other languages has most of its meaning
> expressed informally, in the mind of the programmer.

Unfortunately, I can't read minds. I can only read proofs that have been
explicitly written down.

> Unless you believe that no programs work, then nearly all programs that do
> express most of their meaning informally.

My definition of “work” is “works in all cases”. No bugs. No unforeseen cases.

~~~
pron
So you're specifying an empty set. There has never been a large program that's
been completely formally proven -- not a single one -- and there are therefore
no methods that work. The small programs that _have_ been mostly proven, took
such a great effort, that their creators wouldn't say that the approach
"works" in any widely applicable way.

~~~
catnaroek
Yep, that's the state of the art. It means that more human work has to be done
on making it easier to write programs that work.

------
readams
I'm not really sure I completely understand where the conflict is here. Even
in the "machine model" the church-turing thesis "fails spectacularly" once you
start adding in unreasonable things like halting oracles. TM+Halting oracle >
TM. It's just that the TM+halting oracle doesn't exist in our universe, even
though you can define it mathematically.

When building a language, it's similarly easy to introduce "unreasonable"
things that either require huge complexity to translate to the machine
representation or might even be completely unrealizable in the real world!
Even ignoring higher types, compiling a C++ program requires a potentially
unbounded amount of work because templates are a turing-complete functional
programming language. As stated in the article, just determining if a C++
program is valid is undecidable!

Nothing stops you from designing these languages, but you can't say that
Church-Turing fails until you consider also the systems required to actually
execute these languages in the real world. If you can't, then your programming
language is perhaps an interesting mathematical abstraction, but just as
useful ultimately as that TM+halting oracle model of computation.

~~~
j2kun
> TM+Halting oracle > TM

Why is this a failure of the Church-Turing thesis? The Church-Turing thesis
says, "Anything that can be computed in our reality (i.e. by a human, ignoring
resource constraints) can be computed by a TM." TM+Halting oracle doesn't
satisfy the hypothesis.

~~~
readams
It's not a failure of Church-Turing. The whole point of my comment is that you
can easy construct more powerful models that are "unreasonable" because they
don't exist in our universe. It's the same for programming language models.

------
ankurdhama
The article refers to one quora answer where the author says - "The machine-
based models offer no notion of composition of programs from parts". The
instructions are the parts of the so called machine model, isn't sequencing of
those instructions and control flow instructions are nothing but composition?

~~~
kd0amg
It's a very impoverished form of composition if the only things you can really
compose are the absolute primitives, rather than allowing you to compose
things that are themselves composed. When you "program" with Turing machines,
you don't have a way to take a section of "code" and paste it into another,
larger program. You'll have to deal with two pieces numbering their states the
same, overwriting each other's tape data, etc. If you have two Turing machines
that solve separate pieces of the problem, you know there exists a single
machine that does both things, but nobody actually goes and constructs that
machine. Instead, they just mention its existence as justification for writing
"do this; do that;" in a higher-level machine description.

~~~
ankurdhama
I think the problem can be defined as "Composition without isolation", each
composed sequence of instruction has this same global context (all of memory)
to work with which can lead to problems.

------
joe_the_user
Honestly,

I think Aaronson is clearly and transparently correct about Church-Turing and
higher types. Computation on higher types is just a higher-order construct
having no more to do with computation as we understand it than other higher
order constructs like Hilbert Spaces.

------
Ericson2314
Academia is feudal and thus no answer to this is about to become widely
accepted. :(

~~~
ittekimasu
Actually it's a bit more like 42.

~~~
pron
How is showing that two representations are fundamentally different because
they objectively and radically differ in computational complexity open to
interpretation?

~~~
Ericson2314
It shouldn't be.

I'm skeptical of your parsing argument because the normal way to construct
type theory terms is inductively in their "native" tree form, which is cheap.
Looking at them as a formal language isn't so convenient.

As a PLer, I hope dependently typed langauges see wide use not because they
are the one true foundation of everything, but because they are the richest
lingua franca for every academic discipline to represent their ideas—I'm
personally enamored of such theories as a foundation for math, but I don't
know much about the theory of computation and I'm fine if others just see it
as a tool.

Anyways once that happens and fields have distilled their intentions into
precise models in the lingua franca, perhaps the walls between fields can be
broken down and we finally grok each other's concepts.

~~~
pron
> their "native" tree form, which is cheap

It is _not_ cheap at all, as type checking can be arbitrarily hard, depending
on the type system, and perform arbitrary computation at the "validation"
stage. This is real, significant (and useful, depending on circumstance)
computational work, that no formalism or representation can reduce (only hide,
by pushing it over to the collaborator).

> but because they are the richest lingua franca for every academic discipline
> to represent their ideas

I disagree. I strongly recommend you (and everyone) read this terrific essay
by Lamport on a lingua franca for algorithms:
[http://research.microsoft.com/en-
us/um/people/lamport/pubs/s...](http://research.microsoft.com/en-
us/um/people/lamport/pubs/state-machine.pdf)

Not only is TLA just as powerful as dependent types, it is _far_ simpler[1].
Lamport says that the reason PLers aren't interested in TLA+ is because it's
so simple, so there's not much to write about it. You see plenty of papers
about embedding all kinds of things (separation logic, cost models,
concurrency etc. etc.) with dependent types, things that are so trivial in
TLA+ that they're barely worth a mention. Ideas from Hoare logic, behavioral
refinement, differences between notions of program equivalence from process
calculi (trace equivalence vs. bisimulation) simply collapse into simple,
familiar logic in TLA (the only thing missing is probabilistic reasoning).

Lamport wrote about his experience trying to publish a paper showing that
specifying real-time systems with TLA is trivial (it's a matter of defining a
time variable): [http://research.microsoft.com/en-
us/um/people/lamport/pubs/p...](http://research.microsoft.com/en-
us/um/people/lamport/pubs/pubs.html#real-simple) You can only write a few
papers on a simple approach that solves many problems, but lots and lots of
papers that show how to use complicated approaches to solve them.

A lingua-franca must be simple. TLA+ uses notation and concepts that are
familiar to all mathematicians and all computer scientists, plus a couple of
new concepts that can be learned in a few days. Dependent types (and
intuitionistic logic) are almost as arcane today as they were decades ago,
virtually unknown outside the circles of logic and PLT, each of them is a
particularly isolated sub-discipline within math and computer science. That so
many papers are written about dependent types is strong evidence that they
_cannot_ serve as the lingua franca, and pretty conclusive proof that they
cannot serve as the lingua franca just yet.

That PLers, as Lamport writes in his comment on my post, fail to see that
Plotkin's SOS is an abstract state machine, and that some fail to see that
computing a set-theory function and a type-theory function are two different
computational problems with radically different computational complexity is
further evidence that language models obscure rather than reveal.

Of course, as Lamport also says, real-world programming is very complicated,
and so _programming_ languages are justifiably complex (and whether dependent
types can help with that remains to be seen[2]). But that complexity is
absolutely not required for the purpose of a lingua franca with clear and
simple semantics (TLA's semantics are far simpler than _any_ programming
language) for the purpose of specifying and analyzing algorithms.

Disclosure: I've contributed some code to the TLA+ project.

[1]: Not only do properties ("types") and algorithms share the same terms in
TLA+, they are the same objects. This confuses some people to believe that
TLA+ doesn't support higher-order algorithms, when, in fact, it becomes a non-
issue. Instead of a parameter of a certain type, you have a concrete program
that _is_ the type (e.g., the program that nondeterministically returns, say,
any even integer).

[2]: I'm skeptical because I believe we're close to the "Brooks limit":
[http://blog.paralleluniverse.co/2016/07/23/correctness-
and-c...](http://blog.paralleluniverse.co/2016/07/23/correctness-and-
complexity/)

~~~
Ericson2314
I am curious what you think of math.andrej.com/2016/08/30/formal-proofs-are-
not-just-deduction-steps/

~~~
pron
He mostly talks about proof assistants for general mathematical theorems,
something I don't use, but in TLA+, proofs are of logical formulas, the steps
are logical formulas (unlike in dependently typed provers), and a computation
is just a (temporal) logical formula, so TLA+ already lets you incorporate
computations into proofs, but the process isn't automatic. Then again, TLA+
isn't really designed nor used as a general theorem prover (though it
certainly can prove general mathematical theorems) but as a tool to specify
and verify algorithms.

