People are being downvoted to oblivion for mentioning this, but for some of us coming from different languages, it definitely takes some time to get used to things.
I'm new to julia, so am probably more ignorant about this than most... but importing two packages such as DifferentialEquations and Plots, on my very modern machine takes ~30seconds (this is after they've been "precompiled").
I'm curious, assuming I haven't installed anything new, or haven't changed my installation, why does julia not cache this complied binary somewhere? It seems like this long step has to happen on each new-kernel import, but perhaps it could be avoided?
Having an option flag that would cause julia to redo the compilation (because the user has, say, changed something about their installtion), seems like a simple solution.
> but importing two packages such as DifferentialEquations and Plots, on my very modern machine takes ~30seconds (this is after they've been "precompiled").
FWIW, it is quite significantly improved in the upcoming 1.6 release.
1.5:
julia> @time using Plots
8.859293 seconds (15.99 M allocations: 913.783 MiB, 3.86% gc time)
julia> @time using DifferentialEquations
25.394033 seconds (58.78 M allocations: 3.222 GiB, 3.96% gc time)
latest master:
julia> @time using Plots
3.957724 seconds (7.67 M allocations: 537.424 MiB, 5.50% gc time)
julia> @time using DifferentialEquations
9.708388 seconds (23.08 M allocations: 1.537 GiB, 6.20% gc time)
There will be more caching in the future, but until then it sounds like you'd like PackageCompiler.jl (https://github.com/JuliaLang/PackageCompiler.jl) to build DiffEq and Plots into your sysimage.
As an alternative, don't import all of DifferentialEquations, just import the sublibraries you need. For ODEs usually just DiffEqBase and OrdinaryDiffEq are sufficient.
I see noone has answered your technical question yet, so I'll give (one) reason why this is difficult in the land of Julia.
A fundamental idea in Julia is Multiple Dispatch; methods are extended many times with different types, and common "verbs" (such as `open()`, `write()`, etc...) are used to operate upon a wide variety of objects (files, sockets, plot handles, sound devices, etc...). Exactly which pieces of code gets called when you call `open(foo)` depends not only on the type of `foo`, but also on what methods have been defined for `open()`.
As a concrete example, the deep learning library `Flux.jl` has code within it that checks to see if CUDA packages are already loaded or not within it. If they have been loaded, then `Flux.jl` will load its GPU support libraries as well. This means that:
```
using CUDAnative
using Flux
```
and
```
using Flux
using CUDAnative
```
Result in different environments, and hence, different code being run. It is unfeasible to precache all packages (you would, in the worst case, end up with a combinatorial explosion of packages that is `N!` in number of distinct package-package version tuples) and intelligent ways of thinning that down would quickly become untenable due to the highly dynamic nature of Julia.
Really what this is working around is the lack of support in the Julia package ecosystem for a way for packages to coordinate behavior; with stronger systems in place for a package environment to say "Hey, this is a GPU-using environment, and all packages should turn on GPU support!" there wouldn't be a need for this kind of dynamic behavior.
This is an example of something that happens all the time in the Julia world; when we lack a proper structure, users hack their way out of it with crazy dynamic code that does things that the language supports. We don't want to take that dynamic power away because that's where our edge over many other languages comes from, but it does make it challenging for the compiler to support everything in as low-latency a manner as a less-dynamic language would allow.
There are plenty of possible workflow improvements, of course, such as tighter PackageCompiler.jl and Revise.jl integration that would allow you to bake system images per-environment while still picking up some code changes, etc... but the penalty paid is pretty high; right now, building a system image takes many minutes, and it would be done for any kind of code change. There is compiler work underway to allow for more incremental system image compilation as well, but as with many things on our todo list, it must find its way into the compiler team's pipeline alongside all of the other cool things that are being worked on.
As mentioned in other comments, even without changing the workflow of Julia projects at all, incredible strides have been made over the last few releases, but I wanted to give you an idea of why certain things are harder than they might seem, when coming from other languages. As an aside, `Plots.jl` and `DifferentialEquations.jl` are two notoriously slow-to-load packages, beaten out only by other behemoth packages such as `Flux.jl` or `Turing.jl`.
What am I missing?