check out newegg. They come from Supermicro, Gigabyte, and I think Asus does a line too. They're pretty new so I think only Supermicro is currently listed but they come from at least 4 manufacturers now in Mini ITX form - so fitting hundreds of consumer-class (and often awesome - this is the sweet spot for gamers) small form factor cases. Google "Xeon D Mini ITX". Make sure you get the 8C/16T version not the 4/8 version. You want the 1540 or 1541 versions, not 152x. Supermicro is proper server-class stuff - no messin' around - the others will be slightly cheaper but they're more sporadically available at mo. Also check if you need 10GbE - that costs about 100 dollars more. Still, all will work perfectly as a desktop in a mini-ITX case, and that is a nice small form factor. We're not talking about a huge ugly hunk of box here.
This is going to be "build your own" btw but anybody who's used a philips screwdriver can do this. Here in UK I have the total mentioned above at just over 2200 GBP including PSU and a cute Corsair case (http://is.gd/0pfKhg) so we're talking 3300ish dollars leaving 200 left for a pro mech KB and mouse. For the ram I had Crucial supply me for 780 british pounds (1200 USD ish) for 128 GB of 2400Mhz DDR4 ECC DIMMs (4 x 32 - there are only 4 slots so don't do 8 x 16), which works with these boards. As I said total including RAM about 2200 pounds so 3300 dollars. Or if you don't need 128GB start off with 64, say, but stick with the 32GB DIMMs so you can upgrade later.
Ahh, so they're soldered on to the motherboard. I was just looking for that sort of form factor over the weekend (as a way of getting the low-power 6th-gen i7s... but didn't think to look for Xeon).
I will have to consider this. I run a Mini-ITX board in a Micro-ATX case anyway. Perfect form factor.