But then you've got the issue that anybody interacting with your project on GitHub (eg sending GH pull requests) will see them go unanswered and lose interest in contributing to your project. This is pretty much the exact rant Linus Torvolds had a few months ago.
Yes you can put "I DON'T DO PULL REQUESTS, SEND ME AN AM" in your readme but anybody can agree that that's seriously deficient compared to a GH Pull Request feature that's compatible with AM. (Not that I think GH are evil for not implementing that, just that yes, it would certainly be better if it was implemented).
"Don't implement pull requests because it is much nicer than Linus' solution, so if you implement it then everyone's going to want to use it and then when you insist on not using it no one will want to play with you. :("
Seems a bit like a flawed argument to me.
Or how about:
Progressively enhance the AM protocol so you can receive both basic AM requests from non-GitHub users OR produce a great UX (as it currently is) if both users are on GH.