Starlark, a variant of Python, can be thought of as semi dynamic: all mutation in each file happens once, single threaded, and then that file and all its data structures are frozen so downstream files can use it in parallel
A lot of "staged" programs can be thought of as semi dynamic as well, even things like C++ template expansion or Zig comptime: run some logic up front, freeze it, then run the rest of the application later
Google’s build system uses Starlark definition files for this reason. Very easy to write flexible configurations for each project and module but building is of course very parallel.
> So it's clear that test run time is not a problem that justifies throwing complexity to solve it.
There's a lot I can respond to in this post, but I think the bottom line is: if you have not experienced the problem, count your blessings.
Lots of places do face these problems, with test suites that take hours or days to run if not parallelized. And while parallelization reduces latency, it does not reduce costs, and test suites taking 10, 20, or 50USD every time you update a pull request are not uncommon either
If you never hit these scenarios, just know that many others are not so lucky
> There's a lot I can respond to in this post, but I think the bottom line is: if you have not experienced the problem, count your blessings.
You don't even specify what problem is it. Again, this is a solution searching for a problem, and one you can't even describe.
> Lots of places do face these problems, with test suites that take hours or days to run if not parallelized.
What do you mean "if not parallelized"? Are you running tests sequentially and then complaining about how long they take to run?
I won't even touch on the red flag which is the apparent lack of modularization.
> And while parallelization reduces latency, it does not reduce costs, and test suites taking 10, 20, or 50USD every time you update a pull request are not uncommon either
Please explain exactly how you managed to put together a test suite that costs up to 50€ to run.
I assure you the list of problems and red flags you state along the way will never even feature selective testing as a factor or as a solution.
> What do you mean "if not parallelized"? Are you running tests sequentially and then complaining about how long they take to run?
Some CI runners are single core unless you pay more, some tests are very hard to parallelise (integration, E2E), some test code bases are large and unwieldy and would require a lot of cost to improve to allow parallelisation.
> Please explain exactly how you managed to put together a test suite that costs up to 50€ to run.
If you need to run a hefty windows runner on GitHub it’s $0.54 per minute, so a 90 minute full suite will cost $50.
You could be better with different tests running at PR/Nightly but if quality is a known issue some orgs will push to run total test coverage each PR (saw this at a large finance company).
Notably Github Actions runners are about 4x more expensive than AWS on demand, and windows is also more expensive than Linux.
50USD gets you about 960 linux core-hours on AWS. A lot more than 90 minutes on 1 box on Windows/GHA, but not so much that someone running a lot of unit/integration/e2e tests in a large codebase won't be able to use
> Some CI runners are single core unless you pay more
I don't know which hypothetical CI runner you have in mind. Back in reality, most CICD services bill you on time spent running tests, not "core". Even core count is reflected as runtime multpliers. Moreover, services like GitHub not only offer a baseline of minutes per month, and on top of that provide support for self-hosted runners that cost you nothing.
> some tests are very hard to parallelise (integration, E2E)
No, that's simply false and fundamentally wrong. You can run any test on any scenario independently of any other test from another scenario. You need to go way out of your way to screw up a test suite so badly you have dependencies between tests covering different sets of features.
> If you need to run a hefty windows runner on GitHub it’s $0.54 per minute, so a 90 minute full suite will cost $50.
This argument just proves you're desperately grasping at straws.
The only Windows test runner from GitHub that costs that is their largest, most expensive runner: Windows 64-core runner. Do you need a Threadripper to use the app you're testing?
Even so, that hypothetical cost is only factored in if you run out of minutes from your plan, and you certainly do not order Windows 64-core runners from your free tier plan.
Even in your hypothetical cost-conscious scenario, GitHub actions do support self-hosted runners, which cost zero per minute.
> You could be better with different tests running at PR/Nightly (...)
None of the hypothetical scenarios you fabricated pose a concern. Even assuming you need a Windows machine with 64 processor cores to do an E2E test run of your app, the most basic intro to automated testing tutorial and test pyramids mentions how these tests can and often run on deployments to pre-prod and prod stages. This means that anyone following the most basic intro tutorial on the topic will manage to sidestep any of your hypothetical scenarios by gating auto promotions to prod.
> I don't know which hypothetical CI runner you have in mind. Back in reality, most CICD services bill you on time spent running tests, not "core". Even core count is reflected as runtime multpliers.
If you’re charging a runtime multiplier per core then there’s a cost per core. Included minutes on most runners are limited to basic versions with limited cores. Try xdist pytest on a default GitHub runner and get any speed up…
> Moreover, services like GitHub not only offer a baseline of minutes per month, and on top of that provide support for self-hosted runners that cost you nothing.
Except the cost of the hardware, sysadmin time for setting up and supporting ephemeral runners, monitoring, etc…
> No, that's simply false and fundamentally wrong. You can run any test on any scenario independently of any other test from another scenario.
In end to end system tests if you have 100 hitting at the same time how do you guarantee the underlying state is the same so the tests are idempotent?
Also another example, I set up testing pipelines for an OS that ran in an FPGA in a HIL CI test. I had three of these due to operating costs. How could I parallelise tests that required flashing firmware AND have the most pipelines running as possible?
I can see tests running 50$ if you are running them on Github compute which is x10-x15 times more expensive than AWS. Still, even at 2-3$ that's still quite expensive. Maybe if you have a really large low-level build. I have a really small one and it consumes roughly 10 minutes on Github.
> Please explain exactly how you managed to put together a test suite that costs up to 50€ to run.
I'm not OP but have worked with them: have you considered a repo that might have tents of thousands of committers over decades? It's very easy to just have an insane amount of tests.
Even if you break up a codebase, you need selective testing.
Let's say you have 100 small repos, and make a change to one. How confident are you in the changed repo's test suite that you can guarantee there are no bugs that will affect other related repos? If not, which other repos do you test with your change to get that confidence?
Splitting the code into 10s or 100s repositories is a cancer. It's almost as if you're intentionally trying to make your dev life miserable by pretending that you speeded up the dev turnaround times by not running the full test circle.
> Note that while Mill's first target is Scala builds (as an alternative to SBT) there isn't much reason why Mill shouldn't be usable for other JVM/non-JVM languages too.
Now 7 years later, Mill indeed has first-class support for Java (https://mill-build.org/mill/javalib/intro.html) and Kotlin-JVM (https://mill-build.org/mill/kotlinlib/intro.html). There's experimental work ongoing to expand that support to Kotlin-Multiplatform, Android, Python, and Javascript/Typescript, with ~20,000USD of bounties up for grabs (and a similar amount already paid out):
Just wanted to say that trying (and subsequently adopting) mill this year was one of the most fun/rewarding things I picked up in scala world this year :)
In contrast, my Metascala JVM (https://github.com/lihaoyi/Metascala) is indeed metacircular, and can interpret itself 1 and 2 layers deep (each layer 100x slower than the layer before!)
Not quite 500$/month, but my book https://www.handsonscala.com/ is still making 300-400/month 4.5 years after releasing it. Not a lot of money compared to silicon valley FAANG salaries, especially given the amount of effort that went in, but it's a nice feeling to see the dollars trickling after so long
Li Haoyi! Thank you for developing your scala ecosystem and Hands On Scala. I just started using Scala again after a few years' break, using Mill, thanks to your recent blog post on the last 12 years. I forgot how easy Scala can be if you don't make it hard, and how just darn pleasant it is to code in. Would love to see Scala rise from the ashes of the FP flame wars and become Python devs' second language. Or even their first. Thanks for leading the charge!
Bazel does a lot of non-obvious things that other build tools don't, that become important in larger codebases (100-10,000 active developers):
- Automatic caching of everything.
In most build tools, caching is opt in, so the core build/compile steps usually end up getting cached but everything else is not and ends up being wastefully recomputed all the time. In Bazel, caching is the default so everything is cached. Even tests are cached so if you run a test twice on the same code and inputs (transitively), the second time it is skipped
- Dependency based test selection: You can use bazel query to determine the possible targets and tests affected by a code change, allowing you to trivially set up CI to only run tests downstream of a PR diff and skip unrelated ones. Any large codebase that doesn't use Bazel ends up re-inventing this manually, e.g. consider this code in apache/spark that re-implements this in a Python script that wraps `mvn` or `sbt` (which are build tools that do not provide this functionality) https://github.com/apache/spark/blob/290b4b31bae2e02b648d2c5...
- Automatic sandboxing of your build steps in CGroup/NS containers, to ensure your build steps do not make use of un-declared files. In most build tools, this kind of mistake results in confusing nondeterministic parallelism and cache invalidation problems down the road, where e.g. your build step may rely on a file on disk but not realize it needs to re-compute when the file changes. In Bazel, these mis-configurations result in a deterministic error up front
- At my last job we extended these cgroups to limit CPU/Memory usage as well, which eliminates the noisy neighbour problem and ensures a build step or test gets the same compute footprint whether run alone during development or 96x parallel on a CI worker (https://github.com/bazelbuild/bazel/pull/21322). Otherwise it's common for tests to pass when run during manual development then timeout or OOM when run in CI under resource pressure due to other tests hogging the CPU or RAM
- Built in support for seamless shared caches: e.g. I compile something on my laptop, you download it to your laptop for usage. This also applies to tests: e.g. if TestFoo was run in CI on master, if i pull master and run all tests without changing the Foo code, TestFoo is skipped and uses the CI result
- Built in support for shared compute clusters: e.g. I compile stuff and it automatically happens in the cloud on 96 core machines, or i run a lot of tests (e.g. after a big refactor) on my laptop and it automatically gets farmed out to run 1024x parallel on the cluster (which despite running faster shouldn't cost any more than running 1x parallel, since it costs 1024x as much per-second but should finish in 1024x fewer seconds!)
- Support for deep integration between multiple languages: e.g. building a Go binary and Rust library which are both used in a Python executable, which is then tested using a Bash script and deployed as part of a Java backend server. Most build tools support one language well and others not at all, Bazel supports them all (not perfect but adequately)
If you never hit these needs, you don't need Bazel, and you probably shouldn't use it because it is ferociously complicated. But if you do hit these needs, most other build tools simply do not cut it at all.
We're trying to support some of these use cases and provide a simpler alternative in https://mill-build.org, but Bazel really is a high bar to reach in terms of features that support large monorepos
Woah!! You say "If you never hit these needs, you don't need Bazel", but even for a small-ish project, I would like to have Automatic caching of everything, Dependency based test selection, Automatic sandboxing of build steps, or seamless shared caches!
I guess for a small team (say less than 10 people) the complexity of Bazel is too much to make it worth it, but those features you mention seem like universally useful for any project. I bet there are lots and lots of reinvented wheels out there trying to do just a subset of all these.
Empirically speaking, unless you are already a Bazel expert or are lucky enough to hire one, most orgs I've seen need at least 3-5 full-time engineers to become experts in Bazel and about ~1 person decade of engineering to roll it out (i.e. it takes those 3-5 guys 2-3 years). Adopting Bazel is not something you do over a weekend hackathon; the cost starts becoming reasonable once you have 100-200 engineers who can benefit from the 3-5 guys maintaining their build tool
For your small-ish project I'm guessing you have wants, but if the project grows at some point those will transition into needs and then you may have to make hard choices. That's my experience rolling it out Bazel myself and maintaining it for a growing organization and codebase over 7 years
I'm hoping the Mill build tool I'm working on can become an easier/friendlier alternative that provides many of the same benefits, even if it can't support the extreme Google scale that Bazel does. So if you think these things sound cool, but you don't want to spend 1 person decade on your build tool, you should check out Mill!
You make this sound considerably more involved than it actually is from my experience observing/supporting Bazel infra at three companies ranging from 10 to 1K engineers. Adopting Bazel for C, C++, Java and Go has been pretty straight-forward for years now unless you want power features like remote execution, custom rules/macros, etc
I don't doubt your experience, I can only provide mine. I've seen one rollout from the inside (~1000 engineers), performed one rollout myself (100-1000 engineers), and talked to ~10 other companies trying to do their own rollouts (100-1000 engineers). Everyone has their own unique circumstances, and I can only speak for what I have seen myself (first or second hand)
I agree. We moved to Bazel at my last job and it probably took about 6 person months. I was most of that person and it includes other tooling not related to Bazel. Some extremely competent engineers also moved over our frontend code (including stuff like Cypress) and Python code (which needs to run against 3 different versions of Python). They had no Bazel experience beforehand and asked me maybe a couple hours worth of questions and just got it done. So I don't think you need to be a Bazel genius to get this done, but it helps to have someone with a Vision, which was me in this case. All in all, I'd do it again. I'm in the process of moving all my open source code to a monorepo (jrockway/monorepo, which really should have been called jrockway/jrockway) because the development experience is so much better.
The biggest goal for me in doing the project at work was because new employees couldn't run the code they were working on. We hired people. They tried. They weren't very productive. That's my fault, and I wanted to fix that while supporting our policy of "you can use any Linux distribution you want, or you can use an arm64 Mac". Many people suggested things like "force everyone to use NixOS" which I would be in favor of, but it wasn't the solution that won. (I honestly prefer Debian myself and didn't think that my preference should dictate how the team works. The fact that I disagreed with the proposed solution is a good indicator that people would be unhappy with anything I declared by fiat.) Rather, using Bazel to provide a framework for retrieving third-party tooling and also building our code was a comfortable compromise.
A secondary goal was test caching. If you edit README.md, CI doesn't need to rerun the Cypress tests. (As a corollary, if you edit "metadata_server.go", the "pfs_server.go" tests don't need to run, as the PFS server does not depend on the Metadata server.)
The biggest piece of slowness and complexity in the workflow would be building our code into a container to run in k8s. We used goreleaser and that involved building the code twice, one for each architecture, to build into an OCI image index which was our main release artifact. The usual shell scripts for local development just reused this and it was terribly slow. Throw in docker to do the builds, which deletes the Go build cache after every build, and you have a recipe for not getting anything done. Bazel is a much better way to build containers. Containers are just some JSON files and tar files. Bazel (rules_oci) just assembles your build artifacts into the necessary JSON files and tar files. To build a multi-architecture image index, you build twice and add a JSON file. Bazel handles this with platform transitions; you make a rule to build for the host architecture (technically transitioned to linux on a macos host), and then the image index rule just builds that target with two configurations (cross-compiled, not emulated with binfmt_misc like "docker buildx") and assembles the two artifacts into the desired multi-arch image. When running locally, you skip 2 of the 3 steps, just build for the host machine. Combined with proper build caching (thanks BuildBuddy!), this means that making an image to run in k8s takes about 10 seconds instead of 6 minutes. With the previous system you could try your code 80 times a day. With the new system, you could try it 2880 times ;) This increased productivity.
I also wrote a bunch of tools to make setting up k8s easier, which would have been perfectly possible without Bazel, but it helped. (Before, everyone pushed their built image to DockerHub and then reconfigured k8s to pull that. Now we have a local registry, and if two people do a build at the same time, you always get yours and not theirs. I did not design this previous system, I merely set out to fix it because it's Wrong.) Bazel makes vendoring tools pretty easy. For our product we needed things like kubectl, kind, skopeo, postgres, etc. These are all in //tools/whatever and can be run for your host machine with `bazel run //tools/whatever`. So once you ran my program to create and update your environment, you automatically had the right version of the tool to interact with it. We upgraded k8s regularly, nobody noticed. They would just get the new tool the next time they tried to run it. (A centrally managed linux distribution would do the same thing, but it couldn't revert you to an old tool when you checked out an old version to debug. A README with versions would work, but I learned that nobody really reads the READMEs until you ask them to. "How do I do X" "See this section of the README" "Oh damn I wish I thought of that" "Me too." ;)
The biggest problem I had with Bazel in the past was dealing with generated files. Editor support, "go get <our thing>", etc. I got by when I used Blaze at Google, but realistically, there was no editor support for Go at that time, so I didn't notice how badly it worked. There is now GOPACKAGESDRIVER, which technically helps, but it didn't work well for me and I wasn't going to inflict it upon my team. I punted this time and continued to check in generated files. We have a target //:make_proto that rebuilds the proto files, and a test that checks that you did it. You check in the generated protos when you change the protos. It works and I have a general rule for all generations like this. (We also generate a bunch of JSON from Jsonnet files; this mechanism helps with that.)
All in all, you can get a fresh machine, install git and Bazelisk, check out our repo, and get "bazel test ..." to succeed. That, to me, is the minimum dev experience that your employer owes you, and if you joined my team, you'd get it. That made me happy and it wouldn't be as good without Bazel. I'd do it again!
Just as an aside, after the Bazel conversion, I did a really complicated change, and Bazel didn't make it any harder. We made our main product depend on pg_dump, and adjusting the container building rules from "distroless plus a go binary" to "debian plus postgres plus a go binary" was pretty easy. rules_debian is very nifty, and it gives me a sense of reproducibility that I never got from "FROM debian:stable@sha256...; RUN apt-get update && apt-get install ....". Indeed, the reproducibility is there. You can take any release tag, "bazel build //oci:whatever" and see that the resulting multi-arch image has the same sha256 of what's on DockerHub. I couldn't have done that without Bazel, or at least not without writing a lot of code.
I don't work there anymore but I'm really happy about the project. I don't even do Build & Release stuff. I just add features to the product. But this needed to be done and I was happy to wear the hat.
As someone who is somewhat experienced with build systems in general (though not with Bazel) and has had to solve a lot of the issues you mentioned in different ways (i.e. without Bazel), I have been interested in learning Bazel for a long time as its building principles seem very sound to me. However, the few times I looked into it I found it rather impenetrable. In particular, defining build steps "declaratively" in Starlark to me just seemed to be a slightly less bad way of writing magic incantations in YAML. In other words, you still had to understand what exactly every magic encantation did under the hood and how to configure it, and documentation generally didn't seem great.
Is there some resource (blog/book/…) you can recommend for learning Bazel?
I feel like I got the basics from using Blaze for years at Google. Things like "oh yeah, buildifier will autoformat my BUILD files" and the basic flow of how a build system is supposed to work.
Figuring out how to complete a large project with Bazel involved a few skills that one should be ready to employ.
1) Programming. The stuff out there can't do things exactly the way you want. I wanted to use a bunch of golangci-lint checks with "nogo", so I opened up the golangci-lint source code and copy-pasted their code into my project to adapt the checks to how nogo works. People have tried fixing this problem generically before, but their solutions ended up not working and there are just a bunch of half-abandoned git repositories floating around that don't work. Write it yourself. (I had to write a lot of code for this project; compiling protos the way we want, producing reproducible tar files with more complex edits than I wanted to do with mtree -> awk -> bsd tar, installing built binaries, building "integration test" go coverage binaries, etc. Lots of code.)
2) Debugging. A lot happens behind the scenes and you always need to be situationally aware of what's being done for you. For example, I was pretty sure our containers would be "reproducible" i.e. have the same sha256 no matter the configuration of the build machine. That was ... not true. I tested it and it wasn't happening. So I had to dive into the depths of the outputs and see which bytes were "wrong" in which place, and then debug the code involved to fix the problem. (It was a success, and oddly I sent the PR to fix it about 5 seconds before someone else sent the exact same PR.)
3) Depth. There probably isn't a way to be functional where you pick something out of your search results, follow the quickstart, and then happily enjoy the results. Rather you should expect to read all of the documentation, then read most of the code, then check out the code and add a bunch of print statements... with each level of this involving some recursion to the same step for a sub-dependency. For example, I never really knew how "go build" worked, but needed to learn when I suspected linking time was too high. (Is it the same for 'go build'? Yes. Why? It's spending all of its time in 'gold'. What's gold, the go linker? No, it's the random thing Debian installed with gcc. Is there an alternative? Yes, lld and mold. Are those faster? Yes. How do I use one of those with Bazel? I'll add some print statements to rules_go and use that copy instead of the upstream one.)
With all that in mind, I never figured out "everything". There is a lot of stuff I took at face value, like configuration transitions for multi-arch builds. The build happens 3 times but we only build for 2 platforms (the third platform is the host machine). I don't know why or how to prevent the host build. (I did figure out how to do this for some platform-independent outputs, though, like generating static content with Hugo.) I also wrote a bunch of toolchains but never used Bazel's toolchain stuff. I had my works-with-5-lines-of-code way of running vendored tools for the host machine and never saw the need to type in 50 lines of boilerplate to do things the "right" way. I'm sure this will burn someone someday.
In the end, I guess motivation was the key. People on my team couldn't get their work done, and CI was so slow that people spent half their day in that cycle "I'm going to go read Reddit until CI is done". Hacks had been attempted in the past, and had a lot of effort put into them, and they still didn't work. So we had to rebuild the Universe from first principles, doing things the "right" way. The results were good.
I will always prefer this approach to the simpler ones. For one thing, Bazel always gives the "right answer" when it's set up correctly. It doesn't rely on developers to be experts at managing their dev machines; you include all the tools that they need and you can update them whenever you want a new feature, and they get it for free. That's the big selling point for me. I also can't deal with stuff that is obviously unnecessary, like how Dockerfile-based container builds require an ARM64 emulator to run "mkdir" in a Dockerfile. You're just generating a stack of tar files and some JSON. Let me just tell you where the tar files and the JSON is. We do not need a virtual machine here.
I migrated my team of ~20 developers to Bazel, and I have to say I don’t think it is that complex. Not only don’t we have 3 full time engineers devoted to Bazel, as I do everything myself, but in fact it’s not even my main role to maintain the build system.
We do not use some of the more complex features like remote execution, but we do enjoy all the other features including remote caching. We reduced build times by 92% after the migration.
So my advice would be, try it and see for yourself if you think it’s worth the hassle. For my team, it definitely has been.
It is really not that hard to grok and use. I think cmake is actually a bit harder. It doesn’t hurt that bazel builds are implemented in starlark instead of a zany state manipulation language-that-isn’t-a-language.
Starlark is definitely a mixed experience IMO, from my 7 years working with it in Bazel
On one hand, having a "hermetic" subset of Python is nice. You can be sure your Bazel starlark codebase isn't going to be making network calls or reading files or shelling out to subprocesses and all that. The fact that it is hermetic does help make things reproducible and deterministic, and enables paralleization and caching and other things. Everyone already knows Python syntax, and its certainly nicer than the templated-bash-in-templated-yaml files common elsewhere in the build tooling/infra space
On the other hand, a large Starlark codebase is a large Python codebase, and large Python codebases are imperative, untyped, and can get messy even without all the things mentioned above. Even though your Starlark is pure and deterministic, it still easily ends up a rats nest of sphagetti. Starlark goes the extra mile to be non-turing-complete, but that doesn't mean it's performant or easy to understand. And starlark's minimalism is also a curse as it lacks many features that help you manage large Python codebases such as PEP484 type annotations, which also means IDEs also cannot provide much help since they rely on types to understand the code
For https://mill-build.org we went the opposite route: not enforcing purity, but using a language with strong types and a strong functional bent to it. So far it's been working out OK, but it remains to be seen how well it scales to ever larger and more complex build setups
I have been building an internal tools development and deployment platform [1]. It is built in Go, using Starlark for configuration and API business logic.
Starlark has been great to build with. You get the readability of having a simple subset of python, without python's dependency management challenges. It is easily extensible with plugin APIs. Concurrent API performance is great without the python async challenges.
One challenge wrt using Starlark as an general purpose embedded scripting language is that it does not support usual error handling features. There are no exceptions and no multi value return for error values, all errors result in an abort. This works for a config language, where a fail-fast behavior is good. But for a general purpose script, you need more fine grained error handling. Since I am using Starlark for API logic only, I came up with a user definable error handling behavior. This uses thread locals to keep track of error state [2], which might not work for more general purpose scripting use cases.
A lot of the complaints about Starlark as a programming language, and the proposed alternatives, seem to me to miss out on the UX advantages of having pythonic scripting (which so many folks who have taken a random "coding" class understand intuitively) whereas, e.g., using a lisp or lua would not. Further, having a language and runtime designed for safe use is absolutely critical, and trying to embed another runtime (js/wasm) and manage to lock it down successfully, is a much larger undertaking than I think folks realize.
> Further, having a language and runtime designed for safe use is absolutely critical, and trying to embed another runtime (js/wasm) and manage to lock it down successfully, is a much larger undertaking than I think folks realize.
For what it’s worth both Deno and workerd from Cloudflare give you starting points to run JS in a very locked down sandbox.
This is also one of my major complaints at this point. I'm building a developer tool with starklark resting at its core and I had already came across your work.
I wish the starklark team had addressed it at this point.
> a large Starlark codebase is a large Python codebase, and large Python codebases are imperative, untyped, and can get messy even without all the things mentioned above. Even though your Starlark is pure and deterministic, it still easily ends up a rats nest of sphagetti
This brings it to the point. I'm still wondering why the achievements of software engineering of the past fifty years, like modularization and static type checking had apparently so little influence on build systems. I implemented https://github.com/rochus-keller/BUSY for this reason, but it seems to require a greater cultural shift than most developers are already willing to make.
It did have influence. Take a look at Gradle, which is widely used in the JVM space. It uses a general, strongly typed language (Kotlin) to configure it and it has a very sophisticated plugin and modules system for the build system itself, not just for the apps it's building.
Gradle has its problems, and I often curse it for various reasons, but I'm pretty glad it uses regular languages that I can reuse in non-build system contexts. And the fact that it just bites the bullet and treats build systems as special programs with all the same support that it gives to the programs it's building does have its advantages, even if the results can get quite complex.
Interesting. Using a regular language has some advantages, but also many disadvantages. One of the intentions of BUSY was - similar to e.g. Meson - to avoid a fully Turing complete language, because then people start to implement complex things, thus leaving the declarative character of a build specification, which again makes the build more difficult to understand and maintain.
The basic assumption behind Gradle, I think, is that people usually implement complex things in build systems because their needs are genuinely complex. Build systems are at heart parallel task execution and caching engines that auto-generate a CLI based on script-like programs, and that's a very useful thing. No surprise people use them to automate all kinds of things. You can lean into that or you can try to stop people using them in that way. Gradle leans in to it and then tries to make the resulting mess somewhat optimizable and tractable.
You can of course get people who are just bad at software and make things over-complex for no reason, but if you have such people on a team then the actual software you're building will be your primary problem, not your build system.
>if you have such people on a team then the actual software you're building will be your primary problem, not your build system.
But if you can't get such people fired, the more tasks you can safely assign to such people the better, thus the advantage of a build system without a Turing-complete language :)
Build systems are sort of like type expressions, templates, or constant expressions in a programming language. Either a program compiles or it doesn’t. What might happen when you change the code in some unlikely way isn’t immediately relevant to whether the program works now, so it’s easy to skimp on that kind of checking until things get out of hand due to greater scale.
Also, in Starlark, any runtime check you write is a build-time check and calling fail reports a build-time error, which is good enough for users, but not for understanding how to use Starlark functions within Starlark code.
There is also the need to understand a build and to navigate a build system. Try e.g. to understand how the Chromium build works, and which options are enabled in which case. I even built a tool (see https://github.com/rochus-keller/GnTools) to analyze it (and some other large GN projects) but even so reached the limits of a dynamic specification language pretty quickly. This won't happen in BUSY.
Yes, gn is less good than bazel for a variety of reasons, not the least of which is tooling like `blaze query --output=build` and the more restricted evaluation model in starlark which is easier to evaluate.
Since starlark and bazel restrict the amount of "weird" things you can do, type-inference is pretty straightforward (moreso than in regular python), since almost everything is either a struct or a basic type and there isn't any of the common magic.
I think it's a cultural thing. People like to think of a language for the build system as a little language that somehow doesn't "deserve" a type system. And even they do think a type system is necessary, they think such a language doesn't "deserve" a complicated type system (say Java-like with subtyping and generics) which makes that type system less useful.
I'm curious, what kind of type system does BUSY use?
Which tool did you use to create that busy_spec.html file? They remind me of Engelbart's blue numbering system for documents, if I remember the name correctly.
It's https://github.com/rochus-keller/crossline/, a tool which I implemented and used for many years in my projects. It's inspired by Netmanage Ecco and implements features which can also be found in Ted Nelson's Xanadu or in Ivar Jacobson's Objectory.
I wonder why people love to create languages to be embed in applications when there are plenty of languages that are already useful and well known.
So in the end, you have to fight the half-assed small language that was created and have to find a way to connect to some real language to get things done.
Small languages, if they are suitably constrained, offer far more reasoning power and optimisation potential. This is why we need more small languages, not less. Python aims for maximum flexibility and maximum ease of use. This comes with real and serious trade offs. Python programs are very very difficult to reason about, for both people and machines.
A textbook example for you are (proper) regular expressions. This little language guarantees O(n) matching. The Python and Perl communities added backtracking without truly understanding why backtracking was missing in the first place. Now their misnamed "regular expressions" cause security issues for their users.
Even Thompson didn't use the linear time algorithm that's named after him in Ed and Grep. The Python and Perl implementations were inspired by Henry Spencer's regex, which was in turn reimplementing Thompson's backtracking implementations.
Are you sure it was Thompson who added backtracking to grep? Note also that the POSIX standard intentionally omits backtracking regex. It is a shame that others have not deprecated them.
We're talking about backtracking and not back references, right? The original implementations of Ed and Grep in Unix did backtracking, not back references.
I don't think it's purely cultural. Starlark is interpreted, which presents some challenges to type checking. You either need to make the interpreter more complex, or have an out-of-band type checking step.
The sort of "untyped" that your last sentence is referring to is a dead term, though. The only "untyped" language still in common use is assembler, and that's not commonly written by hand anymore (and when it is, it's primarily running on numbers, not complex structs and complex values). There aren't any extant languages anymore that just accept numbers in RAM and just treat them as whatever.
So increasingly, this objection is meaningless, because nobody is using "untyped" that way anymore. The way in which people do use the term, Python is only "optionally" typed, and a lot of real-world Python code is "untyped".
I think the objection is to the conflation of strong/weak with dynamic/static and it being unclear exactly what typed/untyped means, since it can refer to either. Python has always been strongly typed at runtime (dynamic), vs say JavaScript which is relatively weakly typed at runtime.
Obviously lihaoyi was referring to static/dynamic when they wrote untyped (as made clear by the reference to type annotations) but kstrauser is objecting to using the term "untyped" since that can be interpreted to mean weak typing as well, which Python is not.
Strong/weak is a meaningless dichotomy that could be replaced by nice/icky while conveying the same meaning. It just distinguishes whether I, personally, believe a given language has sufficient protections against dumb programmer errors. What counts as strong or weak depends entirely on who's talking. Some will say that everything from C on is strong, others draw the line at Java, still others aren't comfortable until you get to Haskell, and then there are some who want to go even further before it's truly "strong".
Typed versus untyped is, on the other hand, a rigorously defined academic distinction, and one that very clearly places pre-type-hints Python in the untyped category. That's not a bad thing—untyped isn't inherently a derogatory term—but because untyped languages have fallen out of vogue there's a huge effort to rebrand them.
Untyped computation in the academic sense you refer to is untyped in the sense of Forth and assembler. The untyped lambda calculus doesn't even have numbers. Pragmatically, a language in which type errors occur is a typed language.
Nor does it make sense to conflate "typed and untyped" with "statically typed and dynamically typed". These are simply very different things. Julia is an example of a dynamically typed language with a quite sophisticated type system and pervasive use of type annotations, it would be insane to call it untyped. Typescript is an example of a dynamic language which is nonetheless statically typed: because type errors in Typescript prevent the program from compiling, they're part of the static analysis of the program, not part of its dynamic runtime.
The fact that it's uncommon to use untyped languages now is not a good reason to start describing certain type systems as 'untyped'! A good term for a language like annotation-free Python is unityped: it definitely has a (dynamic) type system, but the type of all variables and parameters is "any". Using this term involves typing one extra letter, and the payoff is you get to make a correct statement rather than one which is wrong. I think that's a worthwhile tradeoff.
From Benjamin Pierce's Types and Programming Languages, which is basically the definitive work on types:
> A type system is a tractable syntactic method for proving the absence of certain program behaviors by classifying phrases according to the kinds of values they compute.
And later on:
> A type system can be regarded as calculating a kind of static approximation to the runtime behaviors of the terms in a program. ... Terms like "dynamically typed" are arguably misnomers and should probably be replaced by "dynamically checked," but the usage is standard.
The definitions you're using are the ones that he identifies as "arguably misnomers" but "standard". That is, they're fine as colloquial definitions but they are not the ones used in academic works. Academically speaking, a type system is a method of statically approximating the behavior of a computer program in order to rule out certain classes of behavior. Dynamic checks do not count.
As I've said elsewhere, I don't have a problem with people using the colloquial definitions. I do have a problem with people actively correcting someone who's using the more correct academic definitions. We should have both sets in our lexicons and be understanding when someone uses one versus the other.
Benjamin Pierce is entitled to his opinion, which you have posted over and over as though it were carved upon stone tablets. "One guy says that dynamic typing is arguably a misnomer" is not a strong case.
The Wikipedia article on type systems is well sourced and fairly well written:
> Dynamic type checking is the process of verifying the type safety of a program at runtime. Implementations of dynamically type-checked languages generally associate each runtime object with a type tag (i.e., a reference to a type) containing its type information.
This accords with ordinary usage in the profession, agreeing with the person you're disagreeing with. You should read it. It provides no support at all for the argument that Python is untyped in any sense. It quotes Pierce several times, including the first citation, so it isn't ignorance on the editor's part.
You are advancing the argument that type theory, as you understand it (there being many paper-publishing respectable academics who do not agree with you on this), trumps type systems in practice, but other than insistence you give no reason why anyone should agree with this premise. Computer scientists aren't discovering laws of nature, nor are they writing proscriptive regulations as to how engineering should be accomplished or discussed. They have their own world and vocabulary to go along with it, but they do not have standing (as you do not) to dictate how terms should be used by others.
I don't accept that argument. If someone wants to make a narrower statement like "according to Pierce you could call Python untyped" I might question its relevance but I'd let it pass. Without such qualification, if someone says Python is untyped I will laugh at them and say "what's it doing when it throws a TypeError then, offering an opinion?". Or on a message board just reply that, no, it's dynamically typed, or unityped if you must, but untyped means something else. Which, indeed, it does.
After taking a programming languages course I came away with the impression that Python is untyped. Type annotations are not required or enforced, and expression types are not evaluated prior to execution. So here’s another vote for the academics.
Strong/weak typing is very specific thing. It refers to the ability to create invalid types within a language. In strongly typed languages it is hard to defeat the type system. In weakly typed languages it is easy to defeat the type system.
Python is strongly typed (hard to escape the bounds of the type system) but (traditionally) dynamically typed (types are checked at runtime).
C is weakly typed (easy to escape the type system), but statically typed (types are checked at compile time).
That is a possible definition for strongly typed, yes. It is not widespread or generally agreed upon—you'll see plenty of people use them in ways that contradict your definitions, and you won't see any serious work attempting to define them at all. Even Wikipedia doesn't [0]:
> However, there is no precise technical definition of what the terms mean and different authors disagree about the implied meaning of the terms and the relative rankings of the "strength" of the type systems of mainstream programming languages. For this reason, writers who wish to write unambiguously about type systems often eschew the terms "strong typing" and "weak typing" in favor of specific expressions such as "type safety".
Strong/weak is not a dichotomy. It's a spectrum. That's why folks argue over where a language lands in the spectrum. OTOH, static (compile-time) vs dynamic (run-time) is a dichotomy. There's not really any in between. It's clear when and where typing occurs. So there's nothing to argue over.
> Typed versus untyped is, on the other hand, a rigorously defined academic distinction
A typed language is one that has a type system. Python has a type system. It's typed.
Academically, no, a type system is by definition static. See the definition Benjamin Pierce gives in TAPL that I've placed in many comments in this subthread [0] and won't repeat here.
Colloquially, yes, python has a type system. All I'm saying is it's unhelpful to correct someone for using the more correct definition rather than the colloquial one. Both definitions are valid, but if we're going to be pedantic we should at least use the academic definition for our pedantry.
And you're correct, I should have said spectrum, but the point is still the same: even Wikipedia refuses to define "strongly" or "weakly" typed, suggesting people use terminology that isn't hopelessly muddled.
...but Python is obviously typed. It has types. In fact everything has a type, and even the types are of "type" type. It has type errors. Saying it's "untyped" invokes a wrong impression. Your usage is very non-standard in programmer circles.
What's wrong with universally understood and well defined concepts of "statically" and "dynamically" typed languages?
As I said in another comment [0], it depends on what definition of types we're using. But if we're going to pedantically jump down someone's throat correcting their usage (in this case OP's usage of "untyped"), we should at least use the most pedantically correct definition, which is the one used by academics who study type systems and which pointedly excludes dynamic checks.
I have no problem with people using the other terminology in casual usage—I do so myself more often than not. I do have a problem with people pedantically correcting usage that is actually more correct than their preferred usage. I dislike pedantry in general, but I especially dislike incorrect pedantry.
It's not "obviously" typed. Values in python have (runtime) types, sure. But contrast that with a statically typed language in which expressions (and functions) have types. Expressions in python do not have types at all (at least before annotations were added).
There's lots of programming languages still around with untyped elements to them. Javascript is one of them, with its string/number conversions and the way arrays are defined. Then there's all the stringly typed stuff. Make, CMake, Excel, TCL, bash. You're probably right that the original use of the term came from assembly vs. high level, but that objection is meaningless, because nobody is using "untyped" that way anymore....
What makes changing the meaning of "untyped" extra confusing is that dynamically typed programming languages often have types as 1st class objects, and they get used all the time for practical everyday programming. Calling these languages "untyped" is just wrong on the face of it -- they're full of types.
> changing the meaning of "untyped" extra confusing is that dynamically typed programming languages often have types as 1st class objects, and they get used all the time for practical everyday programming. Calling these languages "untyped" is just wrong on the face of it -- they're full of types.
Just to be clear, it's the dynamically typed languages that changed the meaning of untyped. OP's usage is closer to the original and to the current usage of the terminology in the study of programming languages.
Types and Programming Languages, one of the best regarded texts on types, has this helpful explanation:
> A type system can be regarded as calculating a kind of static approximation to the runtime behaviors of the terms in a program. ... Terms like "dynamically typed" are arguably misnomers and should probably be replaced by "dynamically checked," but the usage is standard.
In other words both are standard, but that's because the meaning of "types" has changed over time from its original sense and when it comes to the formal study of programming languages we still use the original terminology.
In the time of the original use, there were only static types. Languages had very little in terms of UDT's. Even a struct in C was barely a type of its own. I don't recall the details, but there was something about struct member names not being local to the struct. Interpreted languages didn't have records or classes at all(*), and certainly not types as first class objects.
We cannot really talk about how dynamically typed languages with rich type systems were originally labelled, back when they didn't exist at all.
(*) I'm looking forward to someone pointing out an interesting counterexample.
I could even argue that Asm is to some extent typed. Depends on the processor, but some cisc have operations for different types.
But also the comment is correct: Python is strongly, dynamic typed.
I guess the typing would be for the size of the integer that you work with. For example, x86_64 assembly has different prefixes to indicate what part of a larger register you are using: 8 (lower), 8 (upper), 16 bit, 32 bit, and 64 bit itself.
There are other "typed" operations, such as branching for unsigned vs. signed integers (think JA vs JG), or SAR vs SHR (signed arithmetic shift vs. unsigned arithmetic shift—one preserved the division logic of shifting for signed integers by repeating the MSB instead of adding zeroes when shifting).
While I'm not too familiar with them (but have been meaning to learn more for years!!), SIMD instructions probably also have similar ideas of having different types for sizes of arrays.
This "strong typing" message from the Python community has always sounded like propaganda to me - designed to confuse management. Strong typing is about machine checked proofs of invariants, not whether you avoid a few daft built-in coercions.
There is a static vs dynamic distinction and strong vs weak typing
There is also a semi humorously named "stringly typed" which means weakly typed in such a way that incompatible types are promoted to strings before being operated on.
I'm not aware of any static weakly typed language, but it's logically possible to have one
It depends on what definition of "type system" you're using. Colloquially many programmers use it to refer to any system that checks whether objects have specific shapes. Academics, on the other hand, have a very specific definition of a type system that excludes dynamic detect languages. From TAPL (one of the authoritative works on the subject):
> A type system is a tractable syntactic method for proving the absence of certain program behaviors by classifying phrases according to the kinds of values they compute.
And later on:
> A type system can be regarded as calculating a kind of static approximation to the runtime behaviors of the terms in a program. ... Terms like "dynamically typed" are arguably misnomers and should probably be replaced by "dynamically checked," but the usage is standard.
In other words, you're both correct in your definitions depending on who you're talking to, but if we're going to get pedantic (which you seem to be) OP is slightly more correct.
Personally, it feels like dynamically typed language advocates have been getting more and more vocal about their language of choice being "typed" as static typing has grown in popularity in recent years. This seems like misdirected energy—static typing advocates know what they're advocating for and know that dynamically typed languages don't fill their need. You're not accomplishing much by trying to force them to use inclusive language.
Rather than trying to push Python as a typed language it seems like it would be more effective to show why dynamic checks have value.
It was an old discussion before then, even. It has nothing to do with advocacy and it's certainly not recent. It's about accuracy so that people stop hearing and then repeating the same incorrect ideas. There's no common definition of types by which Python is untyped, as though it doesn't have types at all when in fact every Python object has a type.
> There's no common definition of types by which Python is untyped
You mean besides the one used by every programming languages researcher and hobbyist? Sure, you can define "common" to exclude them, but I would give at least some credence to the definitions put forward by the teams of people who invent type theory.
As I've said here and elsewhere, I have no problem with people casually using "dynamically typed" as a term—I do so as well. But there's no cause to correct someone for using the more correct terminology.
If hearing it makes you feel defensive of python, that implies that you perceive "untyped" as a pejorative that needs defending against. In that case, your efforts would be better spent correcting the evolving consensus that (statically) typed is better than they would be spent trying to shout people down for using the academic definitions of typed and untyped.
That's funny: I've been using bazel (and previously blaze) for well over a decade, but it has never once occurred to me to think of starlark as having anything at all to do with Python! I can't see anything about it which is distinctively pythonic.
You've probably looked only at the Bazel BUILD files. They are indeed quite declarative (as the syntax is restricted even more).
If you open other Starlark files that have functions (in Bazel, that would be in .bzl files), you should recognize the Python syntax (e.g. `def` statements + space indentation).
Erm what? It's very very obviously based on Python. The docs even explicitly say that. This is the example they give:
def fizz_buzz(n):
"""Print Fizz Buzz numbers from 1 to n."""
for i in range(1, n + 1):
s = ""
if i % 3 == 0:
s += "Fizz"
if i % 5 == 0:
s += "Buzz"
print(s if s else i)
fizz_buzz(20)
The fundamental magic of react is that it lets you write code that renders O(n) UI states in a functional manner (i.e. by returning the HTML you want) rather than O(n^2) UI state transitions in a mutable manner (by mutating the page via nested callbacks), and somehow makes it kind of fast-ish
People love complaining about React, but I don't know if they remember the pre-React world where you had to write the code to manage O(n^2) UI state transitions via recursive mutating callbacks. Simple things like "have a the list of photos shown always kept in sync with a label showning count of photos" was incredibly error prone, and the complexity of implementation went up quadratically with the complexity of your app (due to the n^2 state transitions you needed to worry about!)
Today React is not the only one to do this O(^2) to O(n) complexity reduction, but it was the first to go mainstream, and very much deserves its success in spite of its various shortcomings
You're completely right, but I think it's worth adding that the O(n^2) to O(n) change isn't specific to React or UI. That same improvement is often seen when migrating other code from a mutating style to a pure-functional style.
"A pure function which transforms the entire input into the entire output" is obviously the simplest possible architecture for many programs, but people hesitate to use that architecture because of performance concerns. In practice, the baseline performance is often faster than they expect, and it can be made much, much faster using strategies like memoisation and fine-grained reactivity.
> "A pure function which transforms the entire input into the entire output" is obviously the simplest possible architecture for many programs, but people hesitate to use that architecture because of performance concerns. In practice, the baseline performance is often faster than they expect, and it can be made much, much faster using strategies like memoisation and fine-grained reactivity.
But before React came along, you just couldn't do this without major UX breaking bugs, because of how the DOM worked.
Say you have a form that you want to change depending on the state of the app. If the user is typing in a form field while an update to the app state comes, and a pure function that transforms (app state -> DOM/HTML output) resets the form (meaning removing the old out of state DOM and replacing it with the new DOM), the user loses focus on the form. So you have to add some kind of logic where the app remembers what form input the user was focused on, where in the field the focus was, etc. The more complex your app is, the more complex the DOM reset logic became, and you cannot abstract your way out of it with pure functions, because the DOM that relies on mutation slowly creeps into your pure functions anyway.
React changed this, because it gives you a pure function interface, but resets the DOM using mutation functions i.e. native DOM methods, surgically. This is achieved with the VDOM (Virtual DOM), by diffing VDOM states and then reflecting that to the actual DOM. This means when the DOM resets, there's no problem with elements getting removed and added back in, and the focus states etc. don't get thrown away with the DOM. Before React, nothing like this existed.
The problem described by antris is that, if a developer were to naively tear down and rebuild the entire DOM tree on each state change, the browser would discard some important state which belongs to the old DOM nodes. This state includes the text selection, keyboard focus, mouse capture, scroll position, and the progress of CSS transitions and animations.
React solves this problem by building a virtual DOM, and then conservatively updating the actual DOM (sometimes just mutating individual HTML attributes!) so that this state is preserved.
React doesn't have this "pure function" feature what-so-ever. hooks are magic stateful functions, not pure functions. They behave differently the second time called and therefore have all kinds of side-effects and restrictions on when, where, and how they can be used.
Accumulative recursive functions also behave differently the second time they are called...
Hooks are a declarative DSL for accumulator arguments to the recursive function (the component).
If you would rather rewrite the render loop in continuation passing style, and have a 46 line recursive call expression which conditionally changes like 15 immutable parameters to set up the next render iteration for your component, I'd like to see the code when you are done.
We've been doing the O(n) thing forever with PHP and CGI since 1995, the issue is that it required constant whole-page reloads and was a poor experience that got worse the more interactive your webapps got. React let you do the same thing but made it fast(ish) with local updates
We actually see similar innovations on the FP side as well, with persistent data structures that allow structural sharing so your "copy and update" operations can be fast enough to be usable (even if still not nearly as fast as in-place mutation)
React isn’t that special. You can do a react-like design in many frameworks, and in fact you can even do it with vanilla web components. The code ends up a bit more verbose and ugly, but it still has that o(n) core design. To give an idea, here’s a video I made where I port react’s tic tac toe example to vanilla web code.
I never want to go back to a non-declarative rendering framework.
Does anyone remember a blog post from maybe 2016-ish which described the process of "building your own" react-like rendering loop? I remember it being a fantastic explanation of what the virtual dom is and does, and why the "UI is a function of state" paradigm is great. I remember it having a live example to the right side, with prose and code on the left. I can't for the life of me find it now :(
To be fair it predates 2016 when the term “immediate mode ui” was “invented”. uis as a function of state were and still are the norm on game development. React introduced this concept on top of “retained mode ui” which is the dom. Never want to go back too despite people spreading so much hate which is not in the library itself I think but on the ecosystem. Maybe react become synonymous of too many companion libraries/bloated frameworks?
Sadly not. I did find this article too while searching for what I remember. It is a close match in so many ways, but also not quite.
I remember the article very much taking the approach that you've never heard of React before, and walking through a series of problems and discoveries like "oh, what if we just render the state to the DOM every frame? let's see how that works".
Almost like a "you could have invented React" kind of vibe. I think there was a button-counter demo app.
Not a live preview and not a button-counter demo at the end, but the overall structure and the whole O(n) vs O(n^2) argumentation was in this blog post from a now defunct domain. Maybe this is what you were looking for:
This definitely looks different, and I don't recall seeing this before. But this is a great post! It communicates some of the very important ideas in a straightforward way. Thanks!
Another one that seems so close that it makes me think I imagined the one I'm looking for, and this is actually what I read. Thanks for trying - it's not causing me to say eureka! but it is close.
The basic idea is if you have n ways your page can display, with the old PHP/CGI whole page refresh model you needed to write n endpoints, but every interaction required a slow and disruptive whole page refresh
With pre-react single page apps, if you had O(n) ways your webpage could display, you needed to write code to account for n^2 ways that each of those n states could transition into another one of those n states
With react, you write code to render n pages, and get the fine grained interactivity for free with good (enough) performance. It basically removed a tradeoff/dilemma that had been facing web developers since time immemorial, so no surprise it was a big hit
I don't know if I'm convinced. I have several react projects stuck in dependency hell. If I upgrade from build in node 18+ (from 16) I start getting errors. If I try to upgrade react then 1 of the 3 external widgets I'm using barfs out. If I do manage to get upgrade I get complaints about all the best practices from react 16 that have been deprecated. If I'd written my own all of these issue would disappear because I wouldn't have made breaking changes to my own framework.
Note: you might say node 16 is old and I agree. I'd much rather be on 22. But since I can't get out of this dependency hell without days or weeks of work the projects have been stuck for years.
Then do it.. you are free to experience it. Especially if it’s just you on the team you might be totally fine.
I will just say that any project that lasts that long will require maintenance work now and then. The issues you describe seem quite minor and are probably relatively easy to patch up. React provides code mods that do a lot of the heavy lifting, your external widgets have updates, or alternatives.
I would also add that you don’t have to use dependencies with react. It’s always good to be mindful of dependencies and limit them.
It’s the same thing, you choose dependencies to save you time, or you do it yourself.
My main point is that with your own framework you will also run into limitations, causing big rewrites taking days/weeks/months, and/or lots of upfront cost..
ScalaRx and Scalatags actually works a lot like React.js, in user-facing experience even if not in implementation. Unfortunately I was just a college kid and couldn't push it the same way the pros at Facebook could push react!
> These benchmarks were run ad-hoc on my laptop, an M1 10-core Macbook Pro, with OpenJDK Corretto 17.0.6. The numbers would differ on different Java versions, hardware, operating systems, and filesystems. Nevertheless, the overall trend is strong enough that you should be able to reproduce the results despite variations in the benchmarking environment.
> Again, to pick a somewhat easily-reproducible benchmark, we want a decently-sized module that’s relatively standalone within the project
> "typical" Java code should compile at ~100,000 lines/second on a single thread
> We explore the comparison between Gradle vs Mill or Maven vs Mill in more detail on their own dedicated pages
None of your "answers" you quote answers anything to me.
> Again, to pick a somewhat easily-reproducible benchmark, we want a decently-sized module that’s relatively standalone within the project
Why not take netty src? Why is a module more easily reproduceable? We chose a decently sized module because we want a decently sized module does not explain anything.
What about IO? Why lines with comments and empty lines? Or do you even? There is no mentioning in the text as far as I can see, "lines/second" implies this, but then you say "source lines per second", does this include empty lines? I think any compiler can compile a >1.000.000.000/second of empty lines in one file that is already paged into memory.
A lot of "staged" programs can be thought of as semi dynamic as well, even things like C++ template expansion or Zig comptime: run some logic up front, freeze it, then run the rest of the application later
reply