
Why is Idris 2 so much faster than Idris 1? - culturedsystems
https://www.type-driven.org.uk/edwinb/why-is-idris-2-so-much-faster-than-idris-1.html
======
ngngngng
I bought the book "Type Driven Development in Idris" and got several chapters
in before getting distracted by whatever else. It was enough that I'm now
annoyed relatively often that I can't pass types as parameters to be altered
during execution. I really think they're onto something brilliant. I need to
pick it back up and finish the book.

~~~
cflewis
Hmm. My interest is piqued but this book is from 2017 and I am wondering if it
is not horribly out of date at this point. WDYT?

~~~
mx00s
There are some minor changes, but they are documented in detail [1]. The
Idris2 repository maintains tests to detect any further divergence from the
book [2].

I _highly_ recommend the book. Typing in examples and getting experience
collaborating with the compiler on edits is worthwhile. Even if you never use
Idris(2) again after reading it, the demos on creating state machine DSLs with
contextual invariants will leave a lasting impression.

[1]
[https://idris2.readthedocs.io/en/latest/typedd/typedd.html](https://idris2.readthedocs.io/en/latest/typedd/typedd.html)

[2] [https://github.com/idris-
lang/Idris2/tree/master/tests/typed...](https://github.com/idris-
lang/Idris2/tree/master/tests/typedd-book)

~~~
schr0dinger
Well minor including the switch to use of so called “quantitative type theory”
where you can include use counts for variables that inform the compiler how
long variables will stick around. Also the compiler switch to Chez Scheme
yields a pretty big performance boost.

~~~
zelphirkalt
The Idris 2 compiler is build with Chez Scheme? That's pretty cool in my
opinion, didn't know that.

~~~
mjepronk
It compiles to Chez Scheme. It's written in Idris 2 (self-hosted).

------
papaf
Chez Scheme has an interesting compiler [1] and has also been adopted by
Racket [2].

It really cool to see a little used lisp have such a great impact.

[1]
[https://news.ycombinator.com/item?id=15156027](https://news.ycombinator.com/item?id=15156027)

[2] [https://blog.racket-lang.org/2020/02/racket-on-chez-
status.h...](https://blog.racket-lang.org/2020/02/racket-on-chez-status.html)

~~~
bjoli
Andy Keep has given many talks on the nanopass compiler, and searching for him
on YouTube will give you lots of interesting talks!

Chez, together with the commercial lisps, is among the best dynamic language
systems there are. Apart from the unboxed floats me ruined by another sibling
posts, it produces really good code.

In my tests it has been ever so slightly faster than sbcl when the code does
not depend excessively on spending time in the standard library. SBCL's
standard library has had a lot more optimization work done on the individual
function level (I'm looking at you, isqrt). Most of the difference in my case
can be erased by the new SBCL block optimization though, which chez does on
the module level.

------
nilkn
How practical is Idris (or Idris 2) for real-world production programming
right now? It seems to be a pretty natural evolution of Haskell with dependent
types built in from the ground up instead of being sort of achievable with a
patchwork of language extensions. Moreover, making laziness optional is a big
deal for real-world industrial applications. (Note that even the author of
this post indicates at the end significant difficulty with space leaks in the
original Haskell compiler for Idris.)

Or, said differently, what's missing in the language or ecosystem to make it
more immediately practical? Could Idris be the successor to Haskell with
appropriate support?

~~~
Legogris
My experience is from 2015 but from a language-perspective the only things I
personally saw lacking for prime-time use would be performance and the
compiler/type-checker not always giving accurate error messages - but overall,
nothing considerable when comparing to Haskell. I don't know how much those
areas have been improving since. Overall I was very excited about Idris and
found the syntax a lot more approachable than Haskell.

If the meager library of third-party and community packages is an important
factor or not depends a lot on the domain.

I really wish some powerful corp would pick up Idris and put contributions
and/or funding to it.

~~~
tzjmetron
Precisely my experience as well! I also wish Edwin would get some sort of
corporate backing so that Idris could be funded to get into production-ready
mode. Idris2 is much faster (both compilation wise as well as execution wise)
than Idris1, but the standard library is not done (as are parts of the
language itself).

I would love to be able to write production code in Idris rather than patching
together a million Haskell extensions.

------
toastal
I find this slightly depressing that using global mutable state had such a
significant performance given the functional programming (encompassing Idris)
is very much in favor of immutable data structures and purity. I know it's
best to explicitly switch to mutability iff a bottleneck is found (like how
`mut` is explicit and not default in Rust), but I want to _believe_ that Idris
can live in a purity. This makes me think my opinions are not far off from
religion.

~~~
Kutta
Immutability is an implementation detail with no inherent value. What we
actually care about is referential transparency, encapsulation & tracking of
effects, performance, abstraction, thread/type/memory safety, etc. The State
monad in Haskell is implemented without mutation, but presents an API with
mutation. The ST monad is implemented with mutation but presents a pure and
referentially transparent external API.

It is fair to say that functional programming is primrarily in favor of
immutable data structures because it is simpler to implement them efficiently
in a setting where referential transparency is the default. Mutation (meaning
referential intrasparency) is just more complicated, and we need heavier
machinery to structure it nicely. We may use linear/substructural types,
uniqueness types, regions, etc for this purpose, but here are a number of
choices, and the picture is not as clear-cut as with plain pure lambda
calculi. Just remember that mutation is _not_ bad by itself, many functional
programmers are often happy to speed up production code with mutation, it's
just that it is usually more complicated to structure nicely.

~~~
lmm
> Immutability is an implementation detail with no inherent value. What we
> actually care about is referential transparency, encapsulation & tracking of
> effects, performance, abstraction, thread/type/memory safety, etc. The State
> monad in Haskell is implemented without mutation, but presents an API with
> mutation. The ST monad is implemented with mutation but presents a pure and
> referentially transparent external API.

Both State- and ST-using code is harder to understand than code that doesn't
use them. Immutable semantics are easier to reason about. Of course the
implementation of those semantics may involve mutation at some lower level.
But ideally we would have a runtime system that could implement immutable
semantics with no loss of efficiency compared to implementing those lower-
level mutations explicitly in our code.

------
nikivi
What language features are needed in order to implement dependent types? Can
you implement it on top of existing features of say Rust or Swift?

~~~
dan-robertson
A first feature is the ability to define “interesting” types. By interesting I
mean “generic” types which do things depending on the types of their
arguments. In other words, a way to have functions at the type level. In
Haskell you can do this with type families. I don’t know if you can do
something similar in rust (have a type in a trait and then implement that
trait in certain cases to get your function from input type to the type in the
trait implementation for the input type)

A second feature that I think is needed is a way to go between types and
values which I don’t think rust or swift can do.

Two fundamental dependent types are the dependent sum and dependent product.
The dependent product is the type of a function whose output type depends on
its input value. This is a generalisation of the product type: the product
type (a,b) is like the dependent product type (x:{1,2}) -> T x where T 1 = a
and T 2 = b.

The dependent sum type consists of a tag value of some type and a value whose
type depends on the tag. For example a vector of arbitrary length Of floats
could be written with a type something like:

    
    
      Sum(n:Nat)(Vec n Float)
    

And a value like

    
    
      (2,[1,2])
    

Sometimes you see online that every pair can be represented as a dependent sum
and this is true but completely misses the point because it implies that you
should think of it like a generalised product which you should not do.

~~~
pittma
You can, in fact, use traits to do type-level programming in Rust[1], but this
is type-level programming; it isn't /dependent/ types. The biggest "blocker"
for using dependent types is that programs must be /total/, because that
program has to be evaluated for it to typecheck! If a program is not total,
the typechecker has the potential of getting stuck. The two dependently-typed
languages I've programmed in, Mostly Agda and a tiny bit of Idris, have
totality checkers to aid in this. Generally this means that recursion is only
allowed if its structural, and there can be no run-forever loops, i.e., no
servers.

I think instead, for one of these languages, the ideal would be opt in should
you need it. Dependent types are, by their nature, proof carrying[2], and
there are times when you want this, but for a general-purpose programming
language, also times that you do not. You can't be total over IO (or any
effect for that matter), so, yeah, skip it when arg-parsing, but then opt-in
when you're processing your financial transactions or actuating your robot.

[1] [https://dpitt.me/talks/tt-for-rust/](https://dpitt.me/talks/tt-for-rust/)

[2]
[http://www.cs.nott.ac.uk/~psztxa/publ/ydtm.pdf](http://www.cs.nott.ac.uk/~psztxa/publ/ydtm.pdf)

~~~
oisdk
> The biggest "blocker" for using dependent types is that programs must be
> /total/, because that program has to be evaluated for it to typecheck!

Totality is orthogonal to dependent types. You can absolutely have non-total
programs at the type level: Rust has such programs _today_ in fact!

> Generally this means that recursion is only allowed if its structural, and
> there can be no run-forever loops, i.e., no servers.

You can have an infinite loop in a total language like Agda or Idris:
coinduction is the mechanism.

> Dependent types are, by their nature, proof carrying[2], and there are times
> when you want this, but for a general-purpose programming language, also
> times that you do not.

This doesn't make a whole lot of sense to me. Dependent types are "proof
carrying" in the sense that any program of the type:

    
    
        Int -> Int
    

Is a proof that there exists a function of type `Int -> Int`, nothing more. I
don't know what "opting out" of the "carrying" there would mean.

> You can't be total over IO (or any effect for that matter), so, yeah, skip
> it when arg-parsing, but then opt-in when you're processing your financial
> transactions or actuating your robot.

You can be total over IO, and effects do not imply non-totality either.

~~~
pittma
> Totality is orthogonal to dependent types. You can absolutely have non-total
> programs at the type level: Rust has such programs today in fact!

Absolutely, the talk I linked to gets at this to some extent. In rust,
partiality causes type checking to fail. You could use it on purpose!

> You can have an infinite loop in a total language like Agda or Idris:
> coinduction is the mechanism.

For sure, but the totality checker is appeased by sized-types, or at least
that's how I know to do it in Agda. I've heard it can be done without them,
but I'm not familiar with the approach. This is what I intended w/r/t
structural recursion.

> This doesn't make a whole lot of sense to me.

This is because, as the authors state, to type check the program is to
evaluate it, so before it can run you have proof that it is correct.

By opting out I'm more talking about the converse, to opt in when you need it.
Kind of the opposite of Idris's %partial directive.

> You can be total over IO, and effects do not imply non-totality either.

That's fair, you're right. This is poorly stated.

------
ncmncm
The usual answer for such a question about X2 and X1 is that X1 was really
slow. It is usually correct, with the proviso that X2 might _still_ be really
slow, only less so.

With most computer artifacts, it is hard to tell, _prima facie_ , that they
are slow, until somebody makes a faster one. Then the original comes to be
recognized as slow (although somebody will still insist it's not _that_ slow).
The faster one might still be slow, but that fact remains unknown.

The fact is that almost everything is slow, in the sense that somebody
smarter, more experienced, and more diligent could make it faster, often by
doing things that the person who wrote the slow version would have found
distasteful.

~~~
sfj
Compared to Haskell, it was slow indeed.

------
xuejie
The author first goes with:

> Idris 1 is implemented in Haskell, but that has little (if anything) to do
> with the difference.

But latter they also go on to say:

> Idris 2 benefits from a robust, well-engineered and optimised run time
> system, by compiling to Chez Scheme.

I must say I'm slightly confused here. Yes a rewrite might also enable to
avoid all the legacy part that might slow down the code, but what is also
possible, is that a new language and a new runtime could enable new
optimizations that are not possible before. The author did mention Chez's
profiling tools help a lot in the rewrite. So I was curious: is it really
true, that we cannot attribute some part of the speedup to language
differences?

Also I was interested in the rationale behind using Scheme to replace Haskell,
but I failed to find some reasoning behind this, anyone can shed some light on
this?

~~~
kccqzy
That's not relevant at all. I think you have misunderstood the situation here.
In Idris 1 the compiler and code generator is in Haskell, but the runtime is
in C. So Scheme isn't exactly replacing Haskell but replacing C, for the
runtime. (What replaced Haskell was Idris itself, by virtue of being self-
hosting.)

The author even says that it's difficult to write C that deals well with the
functional style of lots of little functions, and this is a problem Scheme
also has and has solved. That's enough rationale to switch to Scheme:

> Generating good C code corresponding to a functional program which might
> have lots of small (and higher order) definitions is extremely challenging,
> so it is better to take advantage of someone else's work here.

------
PunksATawnyFill
What is Idris?

Critical info missing.

~~~
thechao
Your question is fair, and it's unfair you're being downvoted. Idris is a
dependently typed programming language.

If you imagine a "cube" of _fundamental_ language features, on one corner is
"the simply typed lambda calculus" which you can think of as old-school C.
Along each of three edges we add a fundamental language feature: parametric
polymorphism (values that depend on their type; think 'C# generics'), type
constructors (types that depend on types; something akin to C++ template types
like `vector` but not like `std::sort`), and dependent types (types that
depend on _values_ ; but definitely _not restricted_ like `std::array`;
something more like MacroC or `enable_if`).

There's not a lot of good analogies for this stuff in commonly used
programming languages because the big languages are _really_ basic (not
simple!) compared to the advanced stuff the programming language theorists
have gotten up to.

~~~
tluyben2
> Your question is fair, and it's unfair you're being downvoted.

I agree partly; I didn't downvote but I think in this case the downvotes are
not that weird; it's quite well known what Idris is in HN circles, but that's
not the point; the article has links that tell you (pointing to idris-lang)
and google idris edwin brady (which is on top of the article) will get you
more than you want to know most likely but at least enough to know what it is.
So it is the absolute bare minimum to open an article and pop in; 'what is
this?' if it's not in the first line of an article which I think is not great.
A little bit more I do expect from this place; like clicking on the article,
_immediately_ recognising it as a language (that's clear within seconds of
scrolling the article I would say) and then maybe come back with 'what is
dependently typed?' or something. But going from opening the article to
hitting wikipedia explaining dependently typed languages is ... well, not
_much_ more work than doing nothing and just popping 'huh?' in here imho.

Edit: actually, just google 'idris2' and you have in the first line exactly
what, who, why etc.

