Maybe it's because I'm a tub of noob (EE here, I only write code when my hardware demands it) but setting up crosscompiling the Android kernel was physically painful, and it's one of the best-documented things out there.
Cross compiling the Android kernel is fairly straightforward. Download the whole Android source tree, source build/envsetup.sh, lunch the target you want, export ARCH=arm, SUBARCH=arm, CROSS_COMPILE=arm-eabi- and then build away in the kernel repository. Honestly, the easiest cross-compiling I've had to do this side of Go.
Sure, but most things are easy because someone else has done the heavy lifting. I was just pointing out how easy building the Android kernel is, something he described as "physically painful".
An alternative to a cross compiler is to just use distributed compilation and set the current box to 0 so all jobs get distributed to other faster x86 boxes. In the past I regularly used this to build arm on a x86 and more fun building for OSX ppc on x86. ./configure and basic make stuff occur on the "slow" arm/ppc, but the actual build jobs are sent out to the farm for building.
Building a cross-compiling toolchain is hard. Totally understand that. Android [0] and Linaro [1] both provide pre-made pre-tested builds you can just pull and use. I recommend using them.
Building the kernel itself. As mdwrigh2 pointed out, this is actually pretty easy. Also the kernelnewbies community is here to help!
A cross compiling toolchain is fairly easy. You should try plan9's. What is hard is coping with the layer upon layer of the Linux toolchains where the solution is considered by many to be worse than the original problem.