You are right, but this can be make more precise: pipelining is a specific form of parallelism. After all the different stages of the pipeline are executing in parallel.
You use "parallelism" with a different meaning than me.
I use "parallel" with its original meaning "one besides the other", i.e. for spatial parallelism.
You use "parallel" with the meaning "simultaneous in time", because only with this meaning you can call pipelining as a form of parallelism.
You are not the only one who uses "parallel" for "simultaneous in time", but in my opinion this is a usage that must be discouraged, because it is not useful.
If you use "parallel" with your meaning, you must find new words to distinguish parallelism in space from parallelism in time. There are no such words in widespread use, so the best that you can do is to say "parallel in space" and "parallel in time", which is too cumbersome.
It is much more convenient to use "parallel" only with its original meaning, for parallelism in space, which in the case of parallel execution requires multiple equivalent execution units (unlike pipelined execution, which in most cases uses multiple non-equivalent execution units).
When "parallel" is restricted to parallelism in space, pipelining is not a form of parallelism. Both for pipelining and for parallelism there are multiple execution units that work simultaneously in time, but the stream of data passes in parallel through the parallel execution units and in series through the pipelined execution units.
With this meaning of "parallel", one can speak about "parallel execution" and "pipelined execution" without any ambiguity. It is extremely frequent to have the need to discuss about both "parallel execution" and "pipelined execution" in the same context or even in the same sentence, because these 2 techniques are normally combined in various ways.
When "parallel" is used for simultaneity in time it becomes hard to distinguish parallel in space execution from pipelined execution.
The pipeline stages (say: fetch, decode, execute, memory access, register write back), are organised "parallel in space" as transistors on chip. The point of having a pipeline is so the stages can execute "parallel in time".
More generally, parallel in space is interesting because it is a necessary precondition for parallel in time.
In its original meaning, which is still the meaning used in mathematics and physics, "parallel" provides more information than just saying that the parallel things are located in different places in space. Such an information can be provided by other words.
One should not say that the pipeline stages are parallel, when the intended meaning is that they are separate or distinct, which is the correct precondition for their ability to work simultaneous in time.
"Parallel" says about two things that they are located side-by-side, with their front-to-back axes aligned, which is true for parallel execution units where the executions of multiple operations are initiated simultaneously in all subunits, but it is false for pipelined execution units, where the executions of multiple operations are initiated sequentially, both in the first stage and in all following stages, but the initiation of an execution is done before the completion of the previous execution, leading to executions that are overlapped in time.
The difference between parallel execution and pipelined execution is the same as between parallel connections and series connections in any kind of networks, e.g. electrical circuits or networks describing fluid flow.
Therefore it is better if the terms used in computing remain consistent with the terms used in mathematics, physics and engineering, which have already been used for centuries before the creation of the computing terminology.
I agree that the RISC-V ecosystem has some issues that I hope will be sorted out. But not all RISC-V cores are "baby cores". I imagine that XiangShan
[1], albeit student work, will spur work towards bigger, more performant OOO cores.
The 5th edition is seriously out of date. The last 15 years or so have seen massive changes in computer architecture, in particular the explosion of all manner of hardware accelerators, including modern GPUs.
The 7th edition is supposed to come out on 1 January 2025.
It's a gift horse, and you say it needs dental work? ;-)
I think most of the essentials remain true today. As for hardware accelerators, they're all some variation of SIMD or an ASIC built to run a specific algorithm.
Von Neumann stuck us with the program counter, and all the waiting that comes along with it. It's almost impossible to walk away from his legacy.
As opposed to what? A stack of program counters? A set of registers containing program counters? Von Neumann seems related to this as much as the fact that you can only beat one person with a single stick.
It's actually bad news for students when expensive textbooks get updated who go to classes with assignments based on a specific edition and their indices get reshuffled and the old editions become useless even though the core stuff which are unchanged are really the only relevant subset to the syllabus.
How seriously though? Also, the priniples behind GPUs and their use - unlike maybe their physical realization - have not changed much since 2011. (See Chapter 4.)
> The last 15 years or so have seen massive changes in computer architecture, in particular the explosion of all manner of hardware accelerators, including modern GPUs.
What mechanics would you like the eighth edition to introduce?
For what it's worth, Scala is a multi-paradigm language and can be used as a pure OO language. This is a great way to start for programmers with an OO background
I'm not sure it's a good idea though. Scala failed (mainly to Kotlin) as a "better Java". If you aren't committed to Scala's unique features, I would avoid it. It doesn't mean you need to go all in though, Scala can be simple, simpler than Java in fact.
SMT solvers such as Z3 and CVC5 tend to be good at theory solving (the T in SMT) but not so good at handling quantification. OTOH, the ATPs (= automatic theorem provers) used in 'hammers, like E, Spass, Vampire, have the opposite strengths and weaknesses: they are good at handling quantification, but not so good at theory solving. Moreover, all widely used ATPs tend to be for first-order classical logic, while Lean is based on higher-orders constructive logic. So there is still a gap to be bridged.
It is great to see that people work on this problem.
Google stopped publishing interesting AI work since they had their AI lead taken away by OpenAI, and mostly with tech that was pioneered, but not monetised by Google like transformers.
I imagine they are under pressure not to make this mistake again.
We know that any theorem that is provable at all (in the chosen foundation of mathematics) can be found by patiently enumerating all possible proofs. So, in order to evaluate AlphaProof's achievements, we'd need to know how much of a shortcut AlphaProof achieved. A good proxy for that would be the total energy usage for training and running AlphaProof. A moderate proxy for that would be the number of GPUs / TPUs that were run for 3 days. If it's somebody's laptop, it would be super impressive. If it's 1000s of TPUs, then less so.
> We know that any theorem that is provable at all (in the chosen foundation of mathematics) can be found by patiently enumerating all possible proofs.
It's a direct consequence of the format of a proof. They're finite-length sequences of a finite alphabet of symbols, so of course they're enumerable (the same algorithm you use to count works to enumerate them).
If you want a computer to be able to tell that it found a correct proof once it enumerates it, you need a bit more than that (really just the existence of automated proof checkers is enough for that).
I guess it is tautological from the definition of "provable". A theorem is provable by definition if there is a finite well-formulated formula that has the theorem as consequence (https://en.wikipedia.org/wiki/Theorem paragraph theorem in logic)
Not sure it’s a tautology. It’s not obvious that a recursively enumerable procedure exists for arbitrary formal systems that will eventually reach all theorems derivable via the axioms and transformation rules. For example, if you perform depth-first traversal, you will not reach all theorems.
Hilbert’s program was a (failed) attempt to determine, loosely speaking, whether there was a process or procedure that could discover all mathematical truths. Any theorem depends on the formal system you start with, but the deeper implicit question is: where do the axioms come from and can we discover all of them (answer: “unknown” and “no”)?
It's "obvious" in the sense that it's a trivial corollary of the completeness theorem (so it wouldn't be true for second order logic, for example).
Hilbert's program failed in no contradiction to what GP wrote because the language of FOL theorems is only recursively enumerable and not decidable. It's obvious that something is true if you've found a proof, but if you haven't found a proof yet, is the theorem wrong or do you simply have to wait a little longer?
The shortcut vs enumeration is definitely enormous right? just take average shannon entropy to the exponent of the length for example will be probably > heat death (or whatever death) of universe on all of human compute (I'm assuming I didn't bother to check but it seems intuitively true by a margin)
VHDL and Verilog got this decades later than Ada. For hardware descriptions these distinctions are even more important than for software.
Among other useful innovations introduced by Ada, has been the use of underscore not only for making identifiers more readable (a use introduced by PL/I, 15 years earlier than Ada), but also for making the numeric constants easier to read.
Many other languages have copied this, but also decades after Ada. C++ has introduced this feature only in 2014, 35 years after Ada. I am quite annoyed by the fact that for this purpose C++ has replaced the underscore used by Ada and by most other languages that have added this feature with the apostrophe. There has existed no justification good enough for this choice, it was just a stupid decision.
You are right, but this can be make more precise: pipelining is a specific form of parallelism. After all the different stages of the pipeline are executing in parallel.