EXIF isn't the only metadata in jpeg files. IPTC and other forms exist, so if you want to really make them clean make sure you get everything.
exiv2[0] can be used to do that. I have a script I use to remove all metadata from sensitive images I collect for research (UAV aerial photography) before posting).
jpegtran has always given me good results in this area. It strips all metadata, not just EXIF, and it also losslessly optimises the compression to further reduce the filesize. Obviously this latter operation exacts a fair toll on the CPU. It can also do other wacky stuff like lossless scaling, cropping and rotation.
Unless I intend on writing my own software to strip EXIF and geolocation data from JPEGs, at some point in the process I have to trust a third party. May as well be one under a lot of scrutiny!
Given that he uses apt-get to install libimage-exiftool-perl I think the title is fine. It's not likely that Matt Cutts requires link bait titles to get clicks or votes anyway.
Last time I looked into this, mogrify will re-encode the image, resulting in a small loss of quality. jpegtrans and other tools will only remove the metadata without altering the image data.
Like others have noted, there is a lot more "extra" in the JPEG file format than just the EXIF segment. Colloquially, these segments are referred to as "APP" segments; EXIF being just one out of sixteen total APP segments, of which others include f.e. JPS data, thumbnail data and ICC color profile data, to name a few. I've been using this one for a few years: http://stolendata.net/~djinn/code/appexifstrip.c
EXIF isn't the only metadata in jpeg files. IPTC and other forms exist, so if you want to really make them clean make sure you get everything.
exiv2[0] can be used to do that. I have a script I use to remove all metadata from sensitive images I collect for research (UAV aerial photography) before posting).
[0]http://www.exiv2.org/