Check out http://en.wikipedia.org/wiki/Bishop_Castle - this guy lives out in the middle of nowhere in Colorado (where he doesn't have to worry much about inspection, although he has had people raise a bit of a fuss from time to time), but he's certainly not an engineer by any means and he's built an entire, stable castle that's supported hundreds of tourists every single day for years without any incident. It turns out that most humans have a pretty good intuition for what will stand and what won't.
Also, if you're ever in CO, it's worth the visit. Really.
Also, if you're ever in CO, it's worth the visit. Really.