Indeed and I've considered it a bunch of times but I have a fast android tablet and a nice laptop/desktop setup running Xubuntu.
$200 would be ideal, I don't need a tablet just a high quality portable digitizer rather than another tablet.
Next tablet upgrade might well be a surface pro though even running Windows (which I haven't done for a decade) as I'd really like instant access to all of my sketches going back months or years.
EDIT: It would be incredibly useful to pull up a sketch of an architecture against a git timestamp so I could figure out WTF I was thinking at the time ;).
I've got a workflow that gets close to that - I use a "smart pen" for architecture diagrams, you have to write on special paper but in exchange you get a digital version of everything you write/draw, indexed by time (and, if you want, you can record audio as you're writing/drawing and later replay the audio it recorded by pointing at some part of the text)
It's quite useful for architecture diagrams, and for capturing requirements in a meeting (because you've got full audio of what the client said).
The main drawback of the pen I use is it's quite bulky (I use a pen from livescribe), although I suspect there will be better-proportioned pens nowadays...