We have it set up to only run after you've opened a PR. You shouldn't run into the issue often that way because if you're opening a PR then all your big immediate changes are already committed so you won't be pushing another commit until someone reviews which will be after black has already finished.
Sometimes I still sneak in minor changes after I opened a PR. Sometimes I open PR early because CI has integration/e2e tests that are hard to run locally. Sometimes I want feedback on certain parts early on and PR is the easiest way to show something.
There can be workflows in which pushing through CI works fine, but as a general advice it's not great because there are many edge cases.
