
PCI-SIG Releases PCIe 4.0 Specs - jonbaer
http://www.tomshardware.com/news/pci-sig-releases-pcie-4-specs,35767.html
======
userbinator
_PCI-SIG members may download the PCIe 4.0 specification for free at the
Specification Library. Non-members may purchase a hard copy of the
specification for $4,500._

Insane. PCIe 4.0 v0.3 and 0.9 have leaked already; I wonder how long until
v1.0 is available at the usual sites. Looking at the history of the PC, it's
sad that key specifications in its design have become so closed.

------
cjensen
It's very weird that after waiting 7 years since 4.0, they plan on releasing
5.0 with twice the speed just 2 years later. Is that wild optimism?

------
ramshanker
Now let's see who supports it first. Intel or Amd?

~~~
CaliforniaKarl
The answer is likely going to be IBM. See the end of
[https://www.theregister.co.uk/2017/10/26/fore_pci_express_40...](https://www.theregister.co.uk/2017/10/26/fore_pci_express_40_finally_lands/),
as well as [https://www.ibm.com/blogs/systems/power-systems-openpower-
en...](https://www.ibm.com/blogs/systems/power-systems-openpower-enable-
acceleration/)

~~~
MrRadar
The Talos II POWER9 workstation has been advertising PCI-E 4.0 support since
Raptor announced it:
[https://raptorcs.com/TALOSII/](https://raptorcs.com/TALOSII/)

------
Keyframe
So, at 64-bit bus width, we're looking at 1 and 2 Tbps for PCIe 4 and 5,
respectively. Looking forward to THAT!

~~~
cjensen
What do you mean by "64-bit bus width"? The most lanes you can get in PCIe is
16.

~~~
spamizbad
Maybe a traditional "PCI-E card" formfactor, but I don't think there's
anything in the specification preventing you from building a 32 or 64-lane
device.

~~~
kabdib
PCIe 3.x supports a power-of-two number of lanes up to 16; it handles all the
gory stuff needed to aggregate those lanes. Going to more than 16 lanes means
having multiple channels, and you'll have to worry about clock skew, delayed
(retried) and failed transactions and other wonderful things you get with
multiple independent pipes.

I've never done it, but I'm interested in how hard it is. I know a guy who's
done detailed design of PCIe hardware; I'll have to ask him.

~~~
mechagodzilla
I’ve done a lot of PCIe hardware development. I think there was some notion of
a 32x connection early on, spec wise, but x16 is as wide as it’s ever gone in
practice. All high performance PCIe traffic is controlled by DMA hardware on
the endpoint, so having an endpoint with multiple logical x16 connections back
to the host cpu would be quite straightforward to implement, and you wouldn’t
need to worry about any of the gory details you listed. Some FPGAs are
available with multiple endpoints embedded in them, for instance.

~~~
luckydude
Could you explain it like I'm five (or really like I'm a software geek and
don't know this stuff) how the signals in the multiple lanes are synchronized?
Is there some centralized clock that everyone is using?

~~~
crzwdjk
From what I understand, at the physical level PCI-E uses an 8b10b encoding,
which allows the clock to be recovered from the data. Each lane's clock might
still be skewed with respect to other lanes, but there's some deskew logic
that detects and corrects for this skew. I'm not 100% sure when the skew
detection happens. It might be negotiated as part of link negotiation, or
possibly it gets detected at the start of a packet, which has a special 8b10b
symbol that's not used for data so it's easy to spot.

~~~
kabdib
PCIe 3.0 actually uses a 128b to 130b encoding (I guess clock recovery got a
lot better, but they also use different whitening patterns). Data is
distributed across the lanes, bit by bit, and reassembled by the receiver.
There's a _bunch_ of training and link configuration stuff going on.

Once you have multiple channels of these (because 16 lanes wasn't enough
bandwidth) you have the possibility that one channel gets transaction errors
while the other doesn't, resulting in skewed arrival or even just missing
data. So you need a software layer to deal with this, it's no longer "plug
something in and you get bus level reliability," instead it's more like
networking, where you have to do some reassembly as data arrives, and probably
embed sequence number or other information in the data so you can identify bad
and missing transfers. (Or you could say, "screw it, it's reliable _enough_ "
and wait for customers to complain; if they never do, you either made the
right engineering decision or your business isn't big enough yet :-) ).

