Symbolics.jl is the new step to a universal CAS tool in Julia to bridge the gap between symbolic manipulation and numerical computation
It is especially useful as a foundation for equation-based simulation package ModelingToolkit.jl. In the foreseeable future, I expect ModelingToolkit.jl can be comparable with the Modelica ecosystem to provide fast, accurate modeling and simulation capability and easy integration with machine learning methods for Julia ecosystem, which is crucial for engineering application and scientific research.
Could you elaborate on the design choice to model components as functions as opposed to objects ? functions seem compose well from your example but what about initial conditions and type safety ? In Modelica each component is an object where you can specify initial conditions inside each of them with constraints and can eventually make a big system, and if it compiles you can be reasonably confident you aren't mismatching units, missing outputs etc.
Do you have any plans for graphical representation ? Some of the systems I work on in Motorsports are absolutely massive with 150k equations, and having a diagram to see the connections are really helpful. An auto generated one from code would be more than good enough.
How do you handle FFI and interaction with other Julia objects inside ModelingToolkit.jl since it requires symbolic reduction ?
The FMI standard is a very popular export standard for these models. Any plans to support it here ?
I understand these are early days and I am very excited to know there's more on the pipeline. Thanks for your contribution.
>Could you elaborate on the design choice to model components as functions as opposed to objects ? functions seem compose well from your example but what about initial conditions and type safety ? In Modelica each component is an object where you can specify initial conditions inside each of them with constraints and can eventually make a big system, and if it compiles you can be reasonably confident you aren't mismatching units, missing outputs etc.
There's things for type verification and all of that. The decision comes from how it interacts with the compiler. You can easily redefine functions and create them on the fly, less so for structs. That turns out to make it easier to do features like inheritance easily in a functional workflow. Symbolic computing seems to work very well in this setup.
>Do you have any plans for graphical representation ? Some of the systems I work on in Motorsports are absolutely massive with 150k equations, and having a diagram to see the connections are really helpful. An auto generated one from code would be more than good enough.
Somewhat. Auto-generated ones from code already exist in the Catalyst.jl extension library (https://catalyst.sciml.ai/dev/). That kind of graphing will get added to MTK: we just added the dependency graph tooling to allow it to happen, so it's just waiting for someone to care.
A full GUI? There's stuff we're thinking about. More at JuliaCon.
>How do you handle FFI and interaction with other Julia objects inside ModelingToolkit.jl since it requires symbolic reduction ?
All of the functions are Julia functions, and you can easily extend the system by registering new Julia functions to be "nodes" that are not traced. See https://symbolics.juliasymbolics.org/dev/manual/functions/#R... . So if you do `f(x,y) = 2x^2 + y`, then it will eagerly expand f in your equations. If you do `@register f` (and optionally add derivative rules), then it'll keep it as a node in the graph and put its function calls into the generated code. This is how we're doing FFI for media libraries in a new HVAC model we're building.
>The FMI standard is a very popular export standard for these models. Any plans to support it here ?
There is a way to read in FMI that will be explained much more at JuliaCon, with some details probably shared earlier. It's somewhat complex so I'll just wait till the demos are together, but yes we have examples with FMU inputs already working.
The new Modelica-like usage of ModelingToolkit.jl is really a game changer for acasual DAE system simulation in Julia. It makes composable hierarchical component-based simulation of large and complex system possible with pure Julia.
Based on full featured dynamic simulation capability and machine learning ecosystem (Flux.jl etc), Julia will be a perfect choice for reinforcenment learning research, where simulation and training process can all be implemented in pure Julia with promising performance. It delivers the promise of "solve two language problem" of Julia language.
I’ve looked briefly into Julia in the past, but if stuff like this becomes pretty standard (in the way numpy is for Python), I think I could become pretty comfortable in Julia.
I prefer Sympy.
It’s very exciting to see more work in CASes being done, but I worry that “starting a CAS from scratch” isn’t the right approach. The Axiom [0, 1] project rightly identified that building a general-purpose CAS for working practitioners of computational mathematics is an effort requiring nearly generational timespans , and that you must have the right language to describe mathematical objects and their relationships. They had a literate programming policy, where all math code must be accompanied by publication-quality documentation, precisely because it’s so hard to build and maintain these systems. Some of the greatest computational discoveries and expositions came out of the development of Axiom, like the richest and most complete implementation of the renowned Risch algorithm for doing symbolic integrals.
Axiom fell into disuse for a variety of reasons, but from my perspective, found new life in a fork called FriCAS [3, 4], which is actively developed and allows a more “software engineer friendly” approach to the development of the system. The code they have is enormously complex and has mountains of knowledge from foremost experts in computer algebra.
I really wish new computer algebra initiatives attempted in earnest to make use of and extend Axiom/FriCAS so that we could continue to build up our knowledge of this exceedingly delicate and tricky subject without constantly starting from zero. Axiom has a manual that is over 1,000 pages of dense mathematics and that’s really hard to rebuild correctly.
(The only project I know who honestly tried to build upon and subsequently extend CAS functionality is Sage , which builds upon a plethora of existing open source general-purpose and specialized computational math systems.)
 Quote from Axiom manual (http://fricas.sourceforge.net/doc/book.pdf):
> With that in mind I’ve introduced the theme of the “30 year horizon”. We must invent the tools that support the Computational Mathematician working 30 years from now. How will research be done when every bit of mathematical knowledge is online and instantly available? What happens when we scale Axiom by a factor of 100, giving us 1.1 million domains? How can we integrate theory with code? How will we integrate theorems and proofs of the mathematics with space-time complexity proofs and running code? What visualization tools are needed? How do we support the conceptual structures and semantics of mathematics in effective ways? How do we support results from the sciences? How do we teach the next generation to be effective Computational Mathematicians? The “30 year horizon” is much nearer than it appears.
"Additionally, FriCAS algebra library is written in a high
level strongly typed language (Spad), which allows natural expression of mathematical algorithms."
One could argue that Julia also allows natural expression of mathematical algorithms. Coupled with Julia features like multiple dispatch, high performance (due to Julia's LLVM backend) and growing ecosystem of AD and ML libraries, it seems like Julia is probably the more “software engineer friendly” approach at this point. It doesn't seem odd that the Julia folk would want to implement their CAS in Julia. That's not to say that maybe bridges from Julia to FriCAS couldn't be built as has been done with both R and Python.
For a trivial example, the integers under addition form a group. But they also form a ring, which is that group plus a different monoid structure (and extra rules), and the real numbers form a field (two connected groups with extra structure) which meets with the ring structure of the integers. If I write an algorithm about groups, I want a way to be able to apply it to the additive or multiplicative groups of the reals. You need the ability to easily talk about forgetting extra structure but then taking results back from the simpler structure.
Another example is dimensions. It is, as far as I know, basically impossible to encode something like “force (ie mass times distance per time squared) divided by time is commensurate with current times voltage” in a type system like that in ocaml/haskell/rust (you can maybe do it with c++ but the compiler can’t check that your template is correct)
The next problem with general algorithms is that there are isomorphisms and then there are isomorphisms so things that work in mathematics (this group is cyclic and g is a generator so for my element h, there is some integer a such that g^a=h) are sometimes infeasible on computers (the process described above is trivial in integers under addition, hard in integers mod p under multiplication, and very hard in elliptic curve groups (of rank 1), even though they are all cyclic).
I don’t think these are reasons not to try and I think Julia feels like a good language for a combination of performance and interface ergonomics, as well as the library being useful in the ecosystem of other scientific or mathematical libraries.
Here are some examples in some languages.
The author of Libra (which I /think/ was the first to use this technique) gave a good talk  on the mechanism.
 https://to-ithaca.github.io/libra/ (Scala)
 https://hackage.haskell.org/package/dimensional (Haskell)
 https://docs.rs/dimensioned/0.7.0/dimensioned/ (Rust)
I don’t think it is sustainable to need to jam that kind of trickery into every step.
You don't need to use a fixed-length array either - you can used a recursive linked list at the type-level for an unbounded encoding . The Scala library is an example of that; the Github page even has an example of encoding arbitrary units like sheep and wheat.
The encodings used in the haskell library were not a church encoding (they could only express integers between about minus 9 and 9) though to be fair they claimed to be only there until better type level literals. I don’t know about the rust implementation.
Coq and Idris are quite weird languages. The reason they use Peano arithmetic at the type level is that it is easy for them to reason about. But in dependently typed languages where the type and value levels aren’t really separate, if you want to prove things about your programs, they also need to use these numbers which means you either need special language support for making them fast or you have slow programs, which is not suitable for the topic of efficient systems for computer mathematics. The other problem is that the computer can’t help much with the types and they require a lot of manually specifying things which many practitioners would consider trivial enough to be tacit. This is not good for having an expressive system for computer mathematics.
Another example of a system which tried to offer better mathematical foundations is fortress. They started out as something like “fortran but better at mathematics and without so much history” and ended up with something a lot more like haskell with a typeclass-like mechanism and mostly function language. They had a lot of issues with making the type system work with mathematics (one case is that it is hard to specify that something can be a (eg) a monoid. In mathematics, it’s often obvious or said very simply, but computers aren’t so good at knowing what is obvious. It can be complicated to say eg “this is a monoid with identity 1 and operation x,” or “this is a monoid with identity 0 and operation +” because your type system needs a way to disambiguate between the two monoids (or the language will need some bad ergonomics where it is very manually expressed). If your type system ends up needing to know about the names of things and having eg type classes dispatch on the operation or something, it could become tricky.
On HN every time the subject of language wars or platform wars or browser wars comes up, the idea also comes up that "everything is about the number of users". The self-reinforcing cycle is that the language/platform/browser with the most users also has the most development time put into it, which then attracts more users, etc.
Fine, I don't deny that the phenomenon exists. But I think that it's often overlooked that it's not just about the number of users. It's also about the quality of the users. If we could quantify "an excellent developer", I wouldn't claim that Julia has more of those than Python. But I'm convinced that Julia has more "excellent developers who are also excellent at numerics" than Python. I think the idea that the productivity as function of "developer excellence percentile" is a power-law applies even more strongly in multi-domain expertise situations, like numerical computing. So forget about 100x coders. The contributions of some people like Chris et al are closer to 10_000x as significant as that of an ok contributor.
It's not just about the quantity, it's also about quality.
I think the more important point is that Julia has attracted enough first-rate people to self-sustainably build out an ecosystem — and even more keep joining. Several aspects of Julia’s design and core tooling interact to provide compounding leverage to this group. I think it’s a similar situation to the development of the NumPy ecosystem where standardizing on a common array data structure and API led to an explosion of interoperable libraries. Julia arguably takes that a step further by allowing any code to interoperate with high performance and fluent APIs. Julia’s performance characteristics also reduce the barrier to entry because people can make deep contributions throughout the ecosystem without needing to develop low-level programming expertise on top of their domain-specific knowledge.
I feel I need to mention "Structure and Interpretation of Classical Mechanics" by Wisdom and Sussman, and the accompanying "scmutils" library, which first implemented in Scheme many of the same features, although this Julia library seems to be more complete.
There's also a great one-to-one port to Clojure by Colin Smith  in case you want to use it on a more production-friendly environment. The Strange Loop talk  is a good showcase of the power and simplicity of these kind of systems.
One beautiful thing about a Clojure computer algebra system is that it can run completely in the browser. This includes automatic differentiation, numerical integration, all of the hardcore Lagrangian and Hamiltonian mechanics work, differential geometry... it is startling stuff.
For example, here's a(n interactive!) derivation of Kepler's Third Law in the browser (thanks to Nextjournal's lovely integration), if anyone wants to play: https://nextjournal.com/try/sicm/ex-1-11
Many more exercises live here: https://nextjournal.com/sicm/
Thanks for the ideas!
He took Sussman's course and became interested in the field from that! Great work.
I'm hoping to introduce Clojure's "spec" or "schema" libraries so that the types at play can at least be inspectable inside the system. In a fully typed language, I'd implement the extensible generics as typeclasses.
I suspect it would make it quite a bit tougher (at least in the approach I'm imagining) for folks to write new generic functions, due to many type constructors...
On the other hand, the complexity is there, even if you don't write it down!
It would be a big project, and a worthy effort, to write down types for everything in SICM.
Render (pt_BR): https://github.com/hcarvalhoalves/math-fin-training/blob/mas...
I would love to see how you're using org-mode for that if possible :)
I come across this book every so often and find it really tough to read due to the complete lack of "types" in any of the code. Math, especially physics, relies heavily on units and function signatures to be understood.
1. custom units
2. custom GPU kernels
3. Custom array types
4. custom bayesian priors.
5. AD through custom types
6. Task based parallelism
7. symbolic gradients with modeling toolkit
8. Agent based modeling
9. physics informed neural networks
10. abstract tables types
or various combinations of the above?
You can write them in Julia, whereas in Python you have to write them in C/C++ and then use the FFI to call them.
Is that more in the nuances or is there a fundamental difference between these two (referring to the terms and not their specific implementations in Symbolics.jl and Coq respectively)?
Or is this question unreasonable to ask in the first place?
Proof assistants aim to allow one to write down a mathematics assertion in a precise manner, and to help the user write a formally verifiable proof for that theorem.
Extremely crudely, a CAS is like a super-powered calculator, while a proof assistant is like a super-powered unit test framework.
A rough way to think about a tool like Coq or Isabelle is that it provides a very simple core foundation built out of some mathematical logic and mechanisms for manipulating statements in that logic ("if P implies Q, then X"). Proofs end up being sequences of applications of rules that manipulate the statements until you reach some conclusion. People build up theories on top of this core (and other theories), which introduce new mathematical constructs and theorems/lemmas that represent their properties.
A computer algebra system (CAS) tends to have a more complex core because instead of only having some logic as its basis, it might know about higher level constructs like polynomials and such. This allows it to more easily operate on those constructs directly via specialized algorithms and heuristics without having to build up a huge foundation of theories that they live on top of.
That said, you could implement lots of things a CAS does in a theorem prover - it just would probably be pretty awkward to work with, and possibly quite slow. Similarly, lots of CAS tools (like Mathematica and Maple) provide features for doing proofs that are similar to theorem provers. One place I would expect those to differ though is that the small, simple core of a theorem prover allows people to careful verify it such that the theories atop it inherit the verification evidence of the core. I do not know if any such verification evidence exists when it comes to the large kernels that make up tools like Mathematica/Maple.
Term rewriting systems based on rules capture most mathematical manipulation far better than ones based on functions and classes.
Differentiation was a breeze to solve when I sat down and figured out how to formulate it so everything had an understandable normal form. Then I could feed the output of that into another algorithm that brought simple algebraic equations into a normal form.
It's not a way I've seen much programming done outside of niches in academia, but it was extremely powerful.
If I give you a term
x * y - y * x
The rulesets that apply to a term depend on the types of the elements of that term. You could bundle this information into an `Assumptions` object that carries around some rules like
commutes_under_multiplication(x) => false
commutes_under_multiplication(y) => false
The smarts in mathematics is based in the rewrite rules, not the objects they apply to. Since you do not specify what rules apply that is just a term that is already in its normal form since we have no other rules we can apply.
That you assume we are dealing with real numbers merely stems from the fact that books are too lazy to state the rules explicitly in every calculation (and with good reason) and expressions with real numbers are the most common ones that people have exposure to in the average undergraduate course.
Can't it be argued that types are simply a way to associate a considerable amount of rewrite rules and constraints with expressions and therefore spare you the trouble of repeatedly stating them again and again ?
If the type system is rich enough that custom types encode rules just as rich, powerful, concise, etc... as native types while being interoperable with them, I would say this is much, much better than a blind rewriter who either does nothing until you provide it with a rulebook or does everything it can unless you explicitly forbid it. You can just say that that every object/expression has a rulebook attached to it instead and add meta-rules to specify how rulebooks are defined, used and extended.
There is a reason why the internals of all major cas systems use rewrite rules instead of objects with types.
As a toy test try and implement a system that simplifies an expressions of a given group to normal form for that group. Using rewrite rules you end up with a program that is dozen of lines long and even better is provably correct by hand. Using types and functions on types you end up with a monstrosity that is 10 times as long and just feels clunky. Worse even something as simple as adding a super group of which your example is a sub group requires a rewrite of the program. Using rewrite rules on the other hand merely adds a few more if/else clauses in the main program.
Types are a fetish of the programming community and are used in places where they are actively detrimental. This stems from the fact the majority of programmers do not understand the difference between types and constraint rules.
I think the only way you could get something like that to work would be a type system so aggressive that it would turn off most mathematicians, who tend to have a narrow understanding of type theory.
The dominating CAS paradigm for most popular CASes is to assume everything is a real number and see how far you get. Later many major CASes bolted on features to say “well, except, this might be a complex number” and the like. This of course flies right in the face of the kind of thing you want to do. Axiom didn’t take that approach and instead created a robust system for specifying and implementing mathematical objects.
I tried doing quarternion algebra in Maxima many moons ago and it was painful. They’ve since added packages for doing Clifford algebra but it’s not exactly well integrated in the rest of the machinery.
Just overload those on your type, and voila it works with the CAS.
There's help: https://github.com/JuliaPy/PyCall.jl https://github.com/JuliaInterop/RCall.jl
Works like a charm.
Additionally, I think you also have to think about things like Documentation and Tooling.
A lot of the "scientific" Python packages (NumPy, SciPy, etc.) actually just call out to C libraries for the majority of their computation. I imagine Julia can do that already, or it can call out to something similar in the stdlib, so it doesn't really need integration with these Python packages other than just API familiarity. But is that worth the cost in performance?
Julia at this point covers everything in NumPy, SciPy and much more. For optimization, bayesian stuff, scientific, and the convergence of the above with ML, it's far ahead- https://sciml.ai/
Even has relatively mature web frameworks (https://github.com/GenieFramework/Genie.jl)
I'm very certain that you can just import the underlying packages directly but this way is easiest, especially since I'm not familiar with R.
In my view, it's not multiple dispatch per se that is the bigger departure from traditional OOP, it's the fact that methods are no longer contained in the classes. Julia draws a separation between data (structs: the nouns) and behaviors (functions: the verbs). Traditional OOP never really made sense to me; why should each class define and own its own methods? It feels far more sensible to me to just have global behaviors that are well defined. Those verbs can sometimes apply to _anything_ (leaning on more rudimentary operations that you need to define; duck-typing), and sometimes they require you to explicitly define your behavior, and sometimes you just want to add an additional optimization that's available in your particular situation.
Once you have that mental model down, multiple dispatch is just how Julia chooses which method to call... and it's really not much different from single-dispatch.
State mutations. That's it. By ensuring that your data can only be mutated by a your API, it can never get "corrupted".
Bag of functions isn't a bad way to think about it, even for Julia. In some languages, only the name of the function determines which function is called. In others, the number of arguments is used too, so eg foo/1 and foo/2 may be different functions. In Julia, the types matter too, so foo(::Int) and foo(::String) are different functions ("methods" in Julia terminology), and which is used is based on the type of the argument, rather than the number.
That's where the magic in Julia happens, as if you define a function foo(x), without specifying any types, then the specific functions that foo call will only be determined once the type of the arguments to foo are known. But once they are known, that type information can ripple all the way down, picking up the specific implementation for each function depending on the actual types used.
The point is that simple concepts are nice to explain for a beginner, but what actually built your intuition in how to use objects is the years and years learning and experiencing it's benefits and pitfalls. With multiple dispatch it's the same, but since few languages use it (and even fewer, if any, pushes it everywhere like Julia does) most people didn't experience this process.
For me when I'm using a function I just consider them as self-contained abstractions over the arguments. For example there are hundreds of implementations of sum (+), which in practice I ignore and only think about the concept of addition no matter what arguments I give and I trust the compiler/library to find the optimal implementation of the concept or fail (meaning I have to write one myself). If I'm writing a method (or function) I consider arguments as whatever acts the way I need so that I can implement the the concept on them (for example if I'm writing a tensor sum I just consider arguments as n-dimensional iterable arrays and implement assuming that - and declare for the compiler when my method is applicable, without having to care about all other implementations of sum - if anyone needs a scalar sum them that person can implement it and through collaboration we all expand the concept of sum).
And the fact that whoever uses a function can abstract away the implementation, and whoever writes a function can abstract away the whole extension of the arguments (through both duck typing and the fact that the compiler will deal with choosing the correct implementation of the concept) means everything plays along fine without having to deal with details of each side.
Once that's understood, multiple dispatch is simply using all of a function's arguments inferred types to choose which method should be invoked.
I think it's a good mental model to constantly keep the types in mind, but I have made the experience that people working exclusively in dynamically typed languages, i.e. the majority of data scientists, don't share that mental model.
Maxima is an open source variant, based on an earlier version derived from the original Macsyma code base at MIT.