Annoyingly this chip lacks the Compressed (RVC) extension, making it incompatible with all existing Linux distros. These will either have to be recompiled without any Compressed instructions (which increases I-cache pressure on other CPUs that do support it), or we'll need to ship two versions of everything. From discussions I believe they've been recompiling Fedora & Debian from sources without RVC.
The shakti effort was started in 2014. As a member of the group I can say that, then RVC was only seen at as optional extension. We got the tapeout opportunity an year ago. Later the riscv community has taken the decision to make RVC extension mandatory for distros without involving most of its committee members. Had we been told earlier would have given the support.
This is true and very unfortunate how it was handled. However I do think the decision to standardize on RVC for Unix-like systems / servers was the right one, no matter how it was arrived at.
I think the politics of how the decision was made are more important than the technical merits of the decision. Given that, apparently, some community members decided to make RVC mandatory without telling the rest of the committee, I think the only right thing to do is to recompile distros without RVC and live with it. Then, the large binaries and icache pressure will stand as a permanent reminder of how not to make a decision.
Also, is it possible to use only a handful of RVC-enabled binaries on a system that otherwise doesn't use RVC? If so, it might not be so bad.
This seems not correct to me. No one has made RVC "mandatory". For a start no one had any authority to do any such thing!
Some people building Linux-capable processors decided to include RVC.
Some people with Linux distributions (Fedora, Debian), seeing that all known current efforts to build linux capable processors were supporting RVC, and seeing the large benefits it provides, decided to build their distros using RVC.
If someone wants a Linux distro that doesn't use RVC .. it's not a problem. The instructions supported are exactly the same. There are no portability problems. If a package will build successfully with RVC then it will compile without it, no problem, with no additional human work. Just change one configure flag when you build the compiler (to default to rv64g instead of rv64gc). The only difference is the binaries will all be 30% to 50% bigger.
Some more build machines will be required, and a few GB of disk space. Theses are trivial things.
It's absolutely nothing like the difference between, say, armeabi and armhf or the differences between armv4, armv4t, armv5, armv6, armv7.
Is this really such a big issue? Currently there are already multiple arm builds from some distros to deal with similar problems: https://archlinuxarm.org/about/downloads
>> It's exactly the fragmentation of ARM that I'd like to avoid.
RISC-V has an extensible ISA so it's likely to be a mess by design. I'm huge fan of risc-v so I really feel bad saying this. It's telling that RV-IMAFDC is the base for unix systems. It was originally IMAFD = G, but now GC is the standard. Then there's the V extension which I think will be very important and it already sounds like it may be fragmented (graphics and AI may be extra beyond vectors). But then there is space in the instruction encodings explicitly for custom extensions.
These variations are important, and the ability to customize in a standard way is too. I think it's going to be an amazing decade or two in the CPU space because of risc-v but it's also going to be a lot of churn IMHO.
Once open designs catch up to the state of the art and a real base ISA solidifies I hope a RISC-6 comes along with a more optimal encoding and things become really stable for a long time. But that's just my optimism getting out of hand.
Why do you talk about 'the real base ISA' solidifies? The 'real base ISA' is solidified and its designed to let you run a lot of software at minimal complexity.
The problem is that there is NEVER one optimal ISA for all applications. It makes absolutely no sense to have V, T, J extensions in the new 'real base' because there are tons of systems that don't ever need it and it would be a waste.
That's why the RISC-V foundation defines profiles for certain application domains. RV64GC is the logic one for Linux for the moment.
Also you ignore that very often an ISA is not used as a mass market product, but rather in a smaller more specialized space. The costume extensions are very important for people like that. In other cases, the potential AI extensions for vector for example, will probably never gone show up in standard profile for Linux.
I think what we are like gone see is that RV64GC will remain the base level for the Linux profile and eventually more advanced profile with V and potentially others will emerge.
The whole point is that no long build everything around ISAs that is never optimal but rather around profiles.
>> Why do you talk about 'the real base ISA' solidifies? The 'real base ISA' is solidified and its designed to let you run a lot of software at minimal complexity.
The "real base ISA" will be whatever set of options becomes widespread 10-15 years from now. In my mind it's entirely feasible (though not as likely) that the approach of using hundreds of minion cores (see Esperanto) to do graphics will become a common thing (since there is no free GPU and none around the corner). If that is the case, the V extension will become part of the "real base ISA" because everyone will want it. So far the A and C extension went from optional to necessary - it would be naive to think it will stop there. Also, as implementations improve someone may come up with a low cost (low area) way to implement DSP instructions. Maybe that becomes common because the low additional cost is a "why not?". That's part of my point too, the flexibility will bring innovation and that may lead to certain things naturally becoming part of most peoples expectations.
It looks like OpenGL has done something similar. They had a few specifications (ES in particular) for different classes of hardware, but as it became clear that certain features could be handled by the lesser hardware variants those became part of the standard.
Again, I don't see how even in that case, the V extension would be part of 'the real base ISA'.
You seem to again ignore that there are many, many applications that don't need or want a graphics card or SIMD of any kind.
What you are talking about is not the 'real base ISA' but the 'real os application profile'. By not hard coding these choices into the ISA you have much more freedom to create new profiles for future needs without changing anything about the ISA.
> So far the A and C extension went from optional to necessary
That is not the case. Its simple false. There are tons of RISC-V chips that don't have either, some of them are even in production.
Before the Linux RV64GC profile was released there simply was no defined official standard for OS application like Linux.
> Also, as implementations improve someone may come up with a low cost (low area) way to implement DSP instructions
That already exists, its the P working group.
> That's part of my point too, the flexibility will bring innovation and that may lead to certain things naturally becoming part of most peoples expectations.
What you are missing is that 'most people expectations' are limited to a specific program domain. Nobody will ever expect graphics on a IoT edge processor. So the default profile for IoT will not include that.
I guess I'm talking about the base ISA for unix systems. This whole discussion started because someone designed a processor without the C extension and now they have to rebuild every Linux package without that. The base ISA for unix like systems was originally G (IMAFD) without C and it changed. So the base for unix systems will be whatever the major distributions decide it will be regardless of what the RISC-V foundation says. So far they are in alignment. If the P extension ends up costing little area beyond GC and a bunch of developers want it because of audio/video applications they're going to want to rebuild everything with it by default - because server chips should just include it anyway.
I'm not saying this exact scenario or the V on will come to pass, just that I expect the standard set of options widely in use will continue to change like it already has.
G was just a shortform to bundle commen extentions, it was never defined by the foundation as 'the base ISA for linux'. That was just what the first released version of RISC-V spec had in it.
I would assume that the V,B,J and possibly P extentions will all end up in the most used linux profile.
However I think the RV64GC will still be the relevant fallback that most distros will over to cover a very large range of chips.
RISC-V is designed to be universal. That means you have to support everything that is now dominated by MIPS, ARM, x86 and so on. There are lots of markets and even today we use lots of ISA.
Lets remember that there BILLIONS of chips that are internal to companies doing other hardware and all of those should be RISC-V as well.
So if you have that ambition you need to have a way to balance standardization and fragmentation. Profiles are way to find standards for specific use type primary in order to do software standardization on these profiles.
If the only issue is the kernel (which I don't know it is) ... I'm not that sure there is such a big problem. If it is essentially different architectures which requires all binaries to be compiled differently then the problem is much more dire.
I think your points in the video is important - or rather, should be important - but from the fragmentation of ARM it seems they are not as important to chip makers as they should be. However ARM's fragmentation seems to have not been a serious impediment to it's adoption.
I think though going forward the solution may be to target higher level machines (i.e. JVM). This obviously is not realistic for the kernel - but for software that most people want to run on RHEL it would most likely be fine.
For RVC the problem is everywhere. However I argue in the video that even just different kernels, bootloaders or out-of-tree drivers are a real problem when you're deploying servers at scale.
ARM has distinctly not been popular in the data center (despite years of work). That's for many reasons but one is surely the fragmentation of the platform.
> ARM has distinctly not been popular in the data center (despite years of work). That's for many reasons but one is surely the fragmentation of the platform.
At least for the server realm, ARM defined the "Server Base System Architecture" standard
Fascinating talk. Feels like every line has been earned with tears and sweat ;)
Can i ask off-topic about the build farm you mention (approx 12:00). Is build performance an issue ? How long do typical builds take ? Would a x5 performance improvement become a must-have feature ?
Actually the current build farm with 3 x HiFive Unleashed boards and a bunch of VMs is fine (perhaps less so if we had to build everything twice). There are two bottlenecks now: lack of SATA disks on two of the boards, instead we're using NBD; and the Koji build software itself which introduces a lot of overhead by creating from scratch fresh buildroots for every build.
> Annoyingly this chip lacks the Compressed (RVC) extension, making it incompatible with all existing Linux distros. These will either have to be recompiled without any Compressed instructions (which increases I-cache pressure on other CPUs that do support it), or we'll need to ship two versions of everything.
Perhaps ARM actually did have a point with their "FUD" campaign against RISC-V:
Not really, this is simply a case where the Shakti people were so early that the standard simply wasted defined yet, it was not even clear when how things would be standardized.
Why did they choose to not have RVC? Also, what happens if one of these groups working on RISC-V decides they need to add a couple of more instructions in there? Is there a group that you can send your patches for the review like Linux kernel? I do not understand the compatibility of the risc-v cpus!
I was discussing this with someone involved with the project and it comes down to wanting to handle only 32 bit instructions (RVC instructions are 16 bit; other RISC-V extensions can have > 32 bit instructions, although of course if you don't support any of those extensions then you don't have to deal with it). This is a simplifying assumption in the decoder element in their pipeline.
RISC-V does allow you to define other extensions in a methodical and detectable way, and we intend to detect those at runtime (as you would, for example, with something like AES-NI or AVX on Intel). However RVC impacts every bit of compiled code so there's no way to deal with it at runtime (something like that would make all binaries much bigger and would greatly complicate the toolchain).
i like it how they just want to process 1 length instructions (32bit). that's so much easier to build for debuggers/assemblers etc. i think it's a good move to allow for people to write custom OS onto it. As the more complicated aspects of generating machine code have gone, it's easier to write targeted compilers, assemblers and debuggers etc. i find this very interesting choice!
It's probably more about the hardware's instruction decoder. The RVC code has unaligned 32-bit instructions, and that greatly increases the complexity at instruction decode time. Now you have to worry about crazy stuff like an instruction that straddles two different pages.
I'm surprised to hear this, I would have expected a 16bit noop being necessary to align 32 bit instructions..
Is-there a summary of the discussion about this choice?
> The standard compressed ISA extension described in Chapter 14 reduces code size by providing compressed 16-bit instructions and relaxes the alignment constraints to allow all instructions (16 bit and 32 bit) to be aligned on any 16-bit boundary to improve code density.
The only solution I see is to launch a JIT translator (cough QEMU cough) whenever an illegal instruction exception pops, and hope for the best. Certainly not the best choice not to support RVC when everybody else does.
In a sibling comment "neuron_clash" states that when work on shakti started RVC was considered an optional extension, and by the time the community decided RVC should be mandatory it was too late for shakti to support it.
What does that have to do with anything? AIUI they have compiled their own distro (which is not Ubuntu). It leaves the rest of us with the problem of how (or even if) we support this chip. Do we offer two versions of Fedora, with twice the storage, twice the build time, and toolchain changes to support these two architectures? Or do we compile Fedora without RVC, impacting performance on chips which do support RVC? This is an unfortunate (but long-predicted) side-effect of not making RVC mandatory for Unix-like RISC-V systems.
If you are referring to that screenshot in the article, that is just the PC it's connected to. If you look closer at the terminal you can see it's running Buildroot.
