While I've been a Prime subscriber for a few years and do order plenty of things from Amazon, I'm still pretty picky about what I buy from them.
Almost universally, items under $25 that ship free with Prime are more expensive than on other websites and in local stores. Especially as you get into sub-$10 items I'd normally get at a grocery store or Wal-Mart/Target/etc. The shipping cost is being built into the price.
Stuff that's sub-$10 is either not offered with Prime shipping at all, or is only sold in non-discounted multi-packs. For example, say you need some deodorant. Amazon will gladly sell you some Old Spice Classic... but only in a pack of 6 for $18. I can buy just one at a store for $2.50 or so.
In our household, my wife and I are professionals, no kids, quite a large percentage of disposable income and not such a large percentage of disposable time. We have a line item on the budget specifically to outsource things that make sense to outsource. Housekeeping, dog walking, uber cabs, taskrabbits, etc.
That's the value of Prime for us. When we need a pack of double-A batteries. Or Listerine. Or razor blades. Whatever it may be. It's a matter of supreme convenience and time-save for us to be able to just click, click, click, done. Especially so because our car sits in the garage most of the time. We can't just pop into target on our way home from work on the train. It would be a separate, special trip just for whatever little thing we need.
Even for some goods which are more expensive, once you add in Sales Tax (although that'll be solved soon enough) and gas for the car, and especially a quick time is money calculation, I would bet Amazon is cheaper overall every time.