For a base web server included in OpenBSD by default, I don't think dynamic loading is actually a huge win. Auditing and hardening a single, statically-compiled binary is easier than doing the same thing for one that loads shared objects at runtime.
Since adding and removing Apache modules already requires a restart of the httpd process, a good source-based build system like ports can make adding and removing modules via build flags nearly as straightforward as dynamic loading.
Not for OpenBSD, maybe, but for other operating systems based on precompiled binary packages. Nginx uses a forked worker model that probably would allow adding/removing modules without restarting the entire server.
Considering that you can upgrade the binary without any downtime then yeah, it probably would. I personally think there are bigger issues that needs to be covered first, such as support for persistent connections to backends and HTTP/1.1 support when reverse proxying. (Was just added in 1.1.4, now it just needs testing!)
Since adding and removing Apache modules already requires a restart of the httpd process, a good source-based build system like ports can make adding and removing modules via build flags nearly as straightforward as dynamic loading.