

Julia, Python and Cython - synparb
http://groups.google.com/group/julia-dev/browse_thread/thread/61fb4e3847dcc2b9

======
ajdecon
I've been playing with Julia a bit lately, but I haven't had any good reasons
to write real code in it yet. I've got to say, a useful Python compatibility
layer between the two languages would be _awesome_. Being able to call SciPy
libraries from Julia would make it a _lot_ more interesting to start writing
Julia code.

~~~
fuzzythinker
Looks like Stefan thinks this is viable:
[https://groups.google.com/group/julia-
dev/msg/1530301a13a0f9...](https://groups.google.com/group/julia-
dev/msg/1530301a13a0f9ba)

~~~
StefanKarpinski
Heh. That was the easy part. Being able to get a PyObject back from the
embedded Python interpreter and manipulate it in useful ways is the real
trick. But at least it's a start. And it was pleasantly easy, due to two
things: how easy it is to call C functions from Julia, and the fact that
Python has made it trivial to be called from C code.

(Imagine using a C++ library from any other language like this. Wait, you
can't.)

------
synparb
I don't mind writing cython to judiciously optimize my scientific python code
when needed, but efforts along these lines, as well as Travis Oliphant's Numba
project (<https://github.com/ContinuumIO/numba>) seem like promising
alternatives. It's great to see some productive conversations between the
Julia and Scipy-dev folks.

------
chimeracoder
> It's a common theme when we scientific Python users talk that we don't
> really use Python for the _language_. We use it for the community and the
> libraries.

This statement doesn't make sense to me. First, I use Python for
scientific/numerical work, and I definitely use it "for the language" - at
least, certain aspects of it that are rare elsewhere.

Second, the language, community, and libraries are all interdependent. The
design of a language _strongly_ influences the tone of its community, as well
as the libraries available (both the number and the types).

~~~
ajdecon
> _The design of a language strongly influences the tone of its community, as
> well as the libraries available (both the number and the types)._

I certainly think this is true, and I think that Dag's underestimating the
importance of the underlying Python language here.

At the same time, a lot of scientists I know are used to just using whatever
language has the right library in it (either for functionality, or to
duplicate another paper's results). I know that in grad school I used Python,
Matlab, IDL, Fortran, and Java, all on the same _scientific_ project (if not
the same _software_ project). I used each language less because I liked it,
but because it had this one important piece that I really wanted to use.

And while that was a Frankenstein of a project, it wasn't all that unusual
compared to the other researchers I knew. We weren't full-time software
developers, we wanted to avoid re-implementing anything we could... so the
libraries won, all the time.

~~~
StefanKarpinski
This kind of frankensteinism was one of the motivating reasons for creating
Julia (see this InfoWorld interview: [http://www.infoworld.com/d/application-
development/new-julia...](http://www.infoworld.com/d/application-
development/new-julia-language-seeks-be-the-c-scientists-190818), please
ignore the misleading title). Python, although it can get you closer to
single-language scientific computing, can't quite get you there because of the
split language model: at the very least, libraries have to be written in C for
adequate performance; often complex projects end up requiring parts written in
C.

Doing practical scientific computing only using code implemented in Julia is
completely unrealistic. At the very least, a lot of fast kernels (BLAS,
LAPACK, etc.) are going to be implemented in C and/or Fortran for the
foreseeable future. What Julia brings to the table is the ability to call
C/Fortran libraries _without writing anything but Julia code_ — you can just
load a .so file and call functions without compiling anything (see
<http://julialang.org/manual/calling-c-and-fortran-code/>). If we could have
the same level of trivial interoperability with Python, then you could
potentially use scientific libraries written in Python, while still only ever
writing Julia code. That helps alleviate the "frankenstein project" problem
without throwing out all the excellent work that's been done by the SciPy
community.

In the other direction, there are people who already have projects that are
all or mostly Python. Forcing them to have some random piece in Julia just to
use a nice bit of Julia functionality is making the frankenstein problem
worse. If they can trivially call Julia code (perhaps via a .so, as it were
just a library written in C), then that's one less frankenstein project in the
world.

~~~
bugsbunnyak
I think it's hard to overstate how important this is.

Consider the number of bolt-in solutions there are to get decent performance
in python: numexpr, weave, pysco (dead), the emerging island of NumPyPy,
theano, the brilliant/crazy chimera language cython, and now perhaps numba.

The multi-modal nature of development affects libraries (too few devs needing
too many hammers) and even the very sustainability of the ecosystem -- as
evidenced by myriad numpy-discussion threads in the past year [0][0.1] about
core implementation issues with 100-150+ messages, but about three people [1]
of those who have the requisite interest, C chops, Python and CPython API
understanding, and time to do work in the NumPy core. As a result the core has
been mostly stuck in a local maximum for years now.

Contrast Julia: Arrays are written in the language itself [2]. Folks
practically off-the-street with single-digit weeks of Jl experience are
implementing non-trivial array datatypes [3][4]. Julia appears to have the
flexibility to allow the kind of fine-tuned optimizations that are seen in the
guts of a BLAS library [5] while simultaneously facilitating carefree, throw-
away data exploration scripts by MATLAB refugees.

If anything, I fear that Julia may be susceptible to the LISP curse if extreme
care is not taken to ensure the interoperability that is needed to build an
ecosystem. (The Python straitjacket is a blessing in this respect). On the
other hand, maybe this time github will help balance the equation...

[0]
[http://thread.gmane.org/gmane.comp.python.numeric.general/44...](http://thread.gmane.org/gmane.comp.python.numeric.general/44820)

[0.1]
[http://thread.gmane.org/gmane.comp.python.numeric.general/48...](http://thread.gmane.org/gmane.comp.python.numeric.general/48227)

[1]
[http://thread.gmane.org/gmane.comp.python.numeric.general/49...](http://thread.gmane.org/gmane.comp.python.numeric.general/49486/focus=49487)

[2] <https://github.com/JuliaLang/julia/blob/master/base/array.jl>

[3] [https://groups.google.com/d/topic/julia-
dev/YYRa6Iveevg/disc...](https://groups.google.com/d/topic/julia-
dev/YYRa6Iveevg/discussion)

[4] [https://groups.google.com/d/topic/julia-
dev/x3xFSa8iCog/disc...](https://groups.google.com/d/topic/julia-
dev/x3xFSa8iCog/discussion)

[5] [https://groups.google.com/d/msg/julia-
dev/vETgqnpesDk/OuZaB7...](https://groups.google.com/d/msg/julia-
dev/vETgqnpesDk/OuZaB7DBqngJ)

------
joelthelion
There is also a project to port parts of Numpy/Scipy to pypy, which would
probably give interesting results.

The following years are going to be exciting for open-source numerical work.
But I fear fragmentation might ruin it for everybody.

------
yyyt
Can someene of the team or early adaptors explain to me: WHY USE "END"
statement?

~~~
anateus
The syntax is inspired by/based on that of MATLAB, likely so as to be as
familiar as possible.

~~~
synparb
I don't think one should do something just because it is familiar
syntactically. Having left MATLAB for python, there were enough compelling
reasons to ditch the familiar.

~~~
fuzzythinker
Agree. The extra "end"'s makes the code look hideous.

