It raises the bar considerably when exploiting a remote system. Without ASLR, DEP is worthless, since reliable tools exists to produce ROP chains. I think I remember seeing a fully fledged compiler somewhere, that can take high level C code and 'compile' it into offsets on the stack given a target program.
Just ASLR for executable memory is not enough. In a local EOP situation it's very likely an attacker can find where specific modules are loaded in memory. On Windows this isn't even a secret, since modules are loaded in the same memory range for performance reasons.
Here you really also need heap layout randomization, the latest version of which shipped in windows 10, and hasn't been successfully attacked as far as I know. That security mitigation has prevented me from building a stable exploit primitive in the past. And reduced the risk rating of a buffer overflow that I found from trivially exploitable, to not exploitable.
These mitigations are worth it, IMO.