Von Neumann's architecture (i.e. report on the EDVAC) didn't focus on random-access memory, and in fact his whole discussion on memory is rather rambling. The EDVAC itself, which the report is nominally describing, used serial-access mercury delay line memory. What really matters from an architectural POV is memory address-ability, not speed of access (random vs serial).
The whole document is, as the title promises, really more of an engineering one - a report on the EDVAC - rather than some clean abstract treatise on computer architecture.
When the report was written there wasn't any random access memory. The only memory suited to computers in 1946 was the ultrasonic delay line as implemented by Eckert. What makes it not-random access is that all the bits in the store are being constantly shifted and counted in a loop (about 1Mhz?), so you have to wait for the word you want to read/write to fly past and the controller grabs it then.
The document written by Atanasoff in 1940, which I have mentioned in another comment, analyzed all the known devices suitable to be used as memories and it proposed the first kind of DRAM, made with discrete capacitors, as the best combination of low price and high speed.
However, during the war this document was known by few.
Delay lines were already used as analog memories in radars and from there Eckert and Mauchly took them and modified them to be used as digital memories.
Von Neumann learned about the delay line memories from the ENIAC team, but he did not like the idea, because serial access seemed too slow, so he proposed to use instead of delay lines the second kind of DRAM, with a cathode-ray tube instead of discrete capacitors. Von Neumann took the idea of using a CRT as a memory from the tubes used in video cameras for television, which used an analog memory to store the image after exposure to light.
So when the report was written, probably von Neumann was not aware of the earlier RAM used by Atanasoff in his computer, but he also proposed a RAM as being better suited for computers than delay lines. Von Neumann was proven right a few years later, when at Manchester Williams made such DRAMs with cathode-ray tubes, which were later used in many commercial computers, e.g. in the first generation of IBM computers, until the development of magnetic core memories made obsolete both the delay lines and the Williams tubes (until the magnetic cores were made themselves obsolete 20 years later, during the seventies, by the third kind of DRAM, with MOS integrated circuits).
The most important users of delay line memories were the early UNIVAC computers, because they had bought the failed company of Eckert and Mauchly, for their IP.
It's interesting to read Von Neumann's perspective on CRT ("iconoscope") memory - saying how important it was not to lose the serial aspect of it since of course(!) you would not want to specify the address for every access, with most memory accesses being sequential (presumably meaning each bit of a 44-bit word), but it was advantageous to gain the speed of occasionally switching read location, which would be slower and more costly to do with delay lines (more, shorter, delay lines).
Note that Atanasoff's capacitor-based memory was based on rows of capacitors arranged around a mechanical rotating drum (a bit reminiscent of the later EDSAC's "initial orders" module!), so it wasn't true random access - there would be a rotational delay to get access to a given row. In a way it was as much a predecessor of rotating mass storage devices as it was of DRAM.
I just missed the core memory era myself, at least as a user, learning to program (while in highschool) on an IBM mainframe in the mid-late 70's, which was already using semiconductor memory. Still, years ago I bought a core memory module and a (very similar size) etched 6" silicon wafer, intending to frame them side-by-side as the defining story of my era - the switch from physical to solid state integrated electronics.
The whole document is, as the title promises, really more of an engineering one - a report on the EDVAC - rather than some clean abstract treatise on computer architecture.
https://web.archive.org/web/20130314123032/http://qss.stanfo...