2000 tests in 4 minutes actually sounds really slow. We use postres and our Django tests run at roughly 9000/min on 3 year old laptops.
Start your pg container once and use Django's -k flag when running tests to avoid start up times. We also take care to separate DB access from business logic and to unit test those separately.
Sure, the main reason for this is that nearly all of our tests touch the database and operate on a comparatively high level, we have way more things in a fuzzy category between functional and integration tests than actual unit tests. This is of course a flaw in our test writing, but it's what we have right now ;)
Top down integration tests might be slow, but they're a really top notch way to actually know your code works in dynamic language, mvc-style webapps. Can't blame you there.
Start your pg container once and use Django's -k flag when running tests to avoid start up times. We also take care to separate DB access from business logic and to unit test those separately.