Certainly when you get up into the sort of speeds things become very tricky and unpractical. The problem right now though isn't really that we aren't all driving at 100+, but that we still have all of those straight as an arrow 55/65mph roads criss-crossing fly-over country in the US. Those may have made sense at some point, but modern cars can easily do 100mph on those roads with no particularly unusual danger. Pretty much everywhere in this country speed limits are artificially low though. Most country/wooded area roads don't make any sense in cars with ABS or traction control; my limiting factor on most Pennsylvanian roads is the danger of deer, not road geometry or car capabilities.
I think with the spread of autonomous driving technology we'll be able to see even more proper, and reasonably safe, speed from cars.