Vagrant manages virtual machines. Docker uses linux containers, which is essentially "chroot on steroids" [1]. A linux container is just an isolated process (from which you run your code) on the host machine, like a FreeBSD jail. The host machine can peer into the container, but the container cannot see the host machine. Docker just helps to automate and manages the linux container life-cycle.
p.s. I'm planning on doing a couple screencast episodes about linux containers and docker on my website, which should be online in the next week or so @ http://sysadmincasts.com/
They're actually quite similar. Docker is to lightweight containers as Vagrant is to virtual machines (more or less anyway; Vagrant lets you configure a lot more at this point). The nice thing is that the Dockerfile makes the whole thing a lot more transparent; VMs are sort of nebulous black boxes, whereas you can follow the construction of the container through the Dockerfile.
I'll be interested in seeing how the ecosystem evolves when Vagrant becomes better able to manage containers.
The typical use case is to use Vagrant to spin up the machine which then runs docker. I wouldn't recommend managing Docker containers with Vagrant unless you're really heavily invested in a pure vagrant workflow and for some reason cannot afford to call docker commands directly.
Fundamentally, Vagrant builds VirtualBox VMs, while Docker builds containers (e.g. LXC containers). Other than that, feature sets and communities are different.
