The next step for the project is to improve graph layout (see Glance issue here ), which likely means moving away from Graphviz.
What tools to use for interactivity or an editor is still up in the air.
GraphViz's "dot" algorithm (i.e., Sugiyama-style graph drawing algorithm)  should give a fairly compact representation that is organized into layers, and avoids crossing edges in at least simple cases, but rotating the nodes would again "spread out" the layout by forcing increasing height of each layer.
Under "Possible solutions" you mention "create a better graph layout algorithm" - that sounds quite ambitious, wouldn't this be a PhD-thesis-level research task in itself?
The only graph drawing library I'm aware of that might be competitive with GraphViz's algorithms is MSAGL  but that's a .NET library.
However, it does seem that vertical and angled text is harder to read than horizontal text, so there is room to improve here.