I love SQLAlchemy. Maybe things have changed (I'm a jaded Django 0.96 refugee), but as soon as you take away the Django ORM, you lose the admin and then all your advantages just sort of fall away. But really, that's my personal preference with a lot of bias from being there in the early Django days. I think the Django Rest Framework is absolutely awesome, and it almost pulled me back to Django but it just took my first morning of screwing around with some config to remind me of why I prefer the simple frameworks.
And actually, I could already feel with feature creep there. I wanted to have a remote filestore, so there's a Django plugin for doing it. But a) why is my framework in charge of how I store the files for my application (that means I need Django loaded in ipython!) and b) it didn't work quite how I needed and I could see storm clouds brewing.
In terms of general structure, if you know you're building something bigger, I'd suggest at least splitting the web bit and everything else as soon as possible. For me, leaving the web app to handling routing / validation / marshalling / sessions / templating etc feels like the right abstraction.
I look back on any apps I've built / worked on that a) started off as a CRUD system and b) are still going and most of them are a mess. Even if they're clean, the core business logic is mixed up with the standard framework structure. If you really work at it you can separate things out in the code, but my feeling is that by starting with an app that doesn't know about the web, you'll make it much easier to discover where that abstraction lies.
TLDR; use Django if it works for you. Whatever you choose, try to keep the framework out of your application.
A simple (and I think common) situation may illustrate what I took away from your comments:
- You create a CRUD app in Django with two types of users: staff (who use the admin interface to modify anything) and regular (can create/modify their own data, but only read other people's data)
- The staff make a mess via the admin interface, so you decide to add some type of undo thing. You look around for an existing addon that does model versioning, but people still spend time reverting changes back.
- You decide you need some business logic (changes going through some review/approval process) and are scratching your head about how to do that in Django.
I can see then why "I look back on any apps I've built / worked on that a) started off as a CRUD system and b) are still going and most of them are a mess."