
Nim for scientific computing: Back to the future - nkurz
http://rnduja.github.io/2015/10/21/scientific-nim/
======
jph
Nim looks great! I especially appreciate the types and validations.

You mention Nim for scripting - can I pique your curiousity to add a Nim
script example to the Num statistics project?
[http://github.com/numcommand/num](http://github.com/numcommand/num)

~~~
pathsjs
I will have a look, but I cannot promise anything. There are a lot of things
that I want to do and only so much time... :-)

------
antman
Would be nice if NIM could be called from pyton passing numpy arrays. That
would speed up adaptation until the NIM ecosystem grows.

~~~
jboy
For the last 9 months I've been working (part-time) on a project that does
exactly that. The startup where I work makes extensive use of this project to
speed up our Python.

The basic infrastructure was in place after ~2 months. Since then, our team
has been using it extensively, and I've been filling out the functionality &
evolving the API (for example, testing different array iteration & access
interfaces, to see what feels awkward vs convenient, balancing the trade-off
between Nim-ness vs Numpy-ness vs Pythonic-ness, etc.).

You write your Nim procs, then annotate them with a special annotation (that's
valid Nim, but inert most of the time). When you want to auto-generate Python
wrappers around your Nim procs, there's a Python script you run that creates &
compiles Python C-API code that exposes the Nim procs in Python as a shared
library. This auto-generated code also translates Nim exceptions (including
back-traces) to Python exceptions, calls the Nim GC before you return to
Python runtime, etc. The auto-generated code also includes auto-generated
Python docstrings, extracted from the Nim procs.

There's a Nim type that provides a nice Nim interface to Numpy arrays, so you
can pass Numpy arrays into your Nim procs and access them natively. We've also
recently added support for a few cute features, like the tuple return types &
default parameter values. :)

The API is not yet solidified, so I'd call the project "alpha" or "beta", but
it definitely works! The intention is to open-source it soon (once we're happy
with the API).

Would you be interested in using something like this? What features would you
need? Even better, are you a Python+Nim coder who could contribute?

Feel free to reply here or contact me via email (contact info in profile).

~~~
pathsjs
I certainly would be happy to know about it when it comes out!

I would rather avoid the double ecosystem in the long run, but it would be a
great thing to have until the Nim ecosystem grows.

~~~
jboy
I assume you're the article author? If so, I know who you are on the Nim
forums, so I'll certainly keep you informed. :)

In terms of ecosystems, I think that there is deployed Python code in the
world that will never be re-written in Nim. I would prefer that Nim can be
used to write drop-in extension modules for this Python code, rather than not
at all.

I also think that there are situations where Python is a better fit for a
problem than Nim. For example, handling JSON files in Nim (or any statically-
typed language) will always be awkward (you have the choice between a double-
dispatch Visitor Pattern or accessor methods that cast), but in Python,
handling JSON is the easiest thing in the world. Likewise for any
heterogeneous container.

By the way: Great article about Nim, as always. :)

------
vegabook
Can we realistically expect a competent, officially supported REPL for Nim
over time? I would be immediately sold if this otherwise very promising
looking language could remove this obstacle.

~~~
TylerE
What's wrong with `nim i`? That's a repl, and it's hardly possible to get more
official than that

~~~
perturbation
It's been removed* - provided a bad UX for people coming into Nim since it
doesn't support C extensions / importing of modules that _use_ C extensions,
which can be a fair bit. If you try to use them regardless, it's easy to crash
the REPL.

* It's not entirely removed, just obfuscated so that if needed it can be still used as long as you "know what you're doing" and are willing to put up with it not being a general purpose REPL. Use

nim secret

to run it.

------
vezzy-fnord
Not sure why Nim keeps getting compared to Python. It's quite evidently
influenced by the Wirthian languages, though I suppose contemporary
programmers aren't really versed into those.

~~~
TylerE
GC, indentation-significant syntax?

~~~
vezzy-fnord
Whitespace significance is practically the only resemblance, then. Wirth's
languages obviously were garbage collected.

~~~
beagle3
Algol and Oberon were garbage collected, but Pascal and Modula-2 weren't. Not
sure about others (Euler? Modula-3? Oberon-2?)

Nim feels a lot more like Python than one would think just reading the
descriptions. I can't put my finger on why, exactly - maybe the type inference
that gives an almost duck-type experience, maybe the incredibly helpful
dogfooding community; I'm not sure.

------
ericfrederich
You have plenty of options to interface C and Python. You can use Cython to
create Python bindings to C libraries or even better you can use cffi which
works with PyPy as well

~~~
jboy
My issue with Cython is that (at least as I've understood it from the
tutorials I've read) it's a sub-language within Python, where you add Cython
elements iteratively+incrementally (diverging from Python in the process)
until the code runs "fast enough".

I'd rather work directly in a full-featured, internally-self-consistent
language (and Nim is a very nice language) from the start. Hence -- for me, at
least: Nim. :)

~~~
malmaud
Have you looked at Julia ([http://julialang.org/](http://julialang.org/))?

~~~
jboy
Yes, I have, and Julia does look nice. It clearly has some very good libraries
for scientific computing, and its syntax looks (to me) like an interesting
combination of Python (which I like) and Matlab (which I don't like).

Aside from an entirely-subjective preference that caused me to fall in love
with Nim at first sight, there are practical reasons for choosing Nim over
Julia.

The overarching reason is that there is much more Python code in the world
than Nim or Julia put together, and I think it will remain this way for a long
time. (My startup previously worked almost completely in Python, for example,
aside from some C or C++ extension modules and the usual Bash scripts for
sysadmin.) Python also has some solid webserver frameworks, and is great for a
variety of general-purpose tasks.

So, I wanted a solution that would enable us to incrementally rewrite code as
necessary, without needing to rewrite everything from Python all at once.
Also, I have no desire to rewrite our webserver code from Python. Flask &
Django are just fine.

As I understand it (and please correct me if I'm wrong), Julia doesn't compile
to C. So I can't compile snippets of Julia code and call them from my Python
main-loop.

In contrast, not only is this possible in Nim, but Nim's macro system & well-
developed AST spec make it very easy to traverse my Nim procs during
compilation, to auto-generate per-proc wrapper code in C.

~~~
boyd
Hey -- working on similar things here. Would love to follow up but don't see
your email in your profile. Mind adding it or sending me a note? (mine is in
my profile)

Cheers!

~~~
jboy
Great! Email sent to the address in your profile.

------
ilaksh
If only the Nim project had access to a few of the tens of millions being
spent by Yahoo in order to more effectively deliver clickbait about the
Kardashians and the like.

I mean, if the money actually came from a different organization than Yahoo.

Its good they do have support from 3DICC though.

------
ricklamers
What about Numba for Python in order to get the speed? Or is that a stupid
suggestion for your requirements?

~~~
pathsjs
Numba is a great technology, but I find it still a little disappointing.

The way I see it is this. Python is not a particularly fast language, but has
evolved a big ecosystem of scientific libraries. Now people are running into
the language limitations, hence the development of various tools to compile
parts of Python - either AOT or JIT - with PyPy, Numba, Cython, Numexpr and so
on.

The problem is that all these tools (save from PyPy, which is just a different
interpreter) introduce their ow sublanguage, slightly incompatible from one
another. For instance, both Numba and Cython have to infer types, but their
type systems are not equal.

Nim does not have the reach of Python ecosystem, but it is a solid language,
that manages to combine well the speed of C and the simplicity and flexibility
of Python.

I would rather start from a solid foundation and add libraries on top of it
gradually, rather than start from a developed ecosystem and add a host of
tools that try to fix the inherent limitations of the platform.

Hence, in the short term it is great that the projects created by Continuum
exist, but eventually I would like to have something a little more consistent

------
dvdplm
Lovely write up, thanks! The syntax is so much nicer than Rusts is. :)

------
progman
Nim's garbage collector is indeed very fast. I wrote an application which
makes heavy use of string concatenation and a lot of regexp matches in native
Perl syntax. I am amazed how performant the code is. Also compilation works
pretty fast.

However I didn't know yet that the Nim language is _that_ powerful.

------
nanny
> there is no FFI to C

What? Nim's homepage has the phrase "C FFI is easy in Nim.." along with some
example code.

~~~
dom96
I think this is referring to the lack of FFI support in NimScript
([http://nim-lang.org/0.11.3/nims.html](http://nim-
lang.org/0.11.3/nims.html)).

------
approbatory
In the comparison with torch you are translating x:dot(y) as x * y, but it
should be x.t() * y

~~~
pathsjs
No, it is right like that. In linalg, * on vectors is the scalar product. I
have designed linalg to agree with mathematical conventions, rather than numpy
conventions.

