Adding sawdust to nutmeg gave nutmeg dealers a poor reputation resulting in an insult in the 1860 election of Stephen Douglas calling Abraham Lincoln a nutmeg dealer (he threw in some other insults in the same line).
horrid looking wretch, sooty and scoundrelly (sic) in aspect, a cross between the nutmeg dealer, the horse swapper and the night man
horrid looking wretch, sooty and scoundrelly (sic) in aspect, a cross between the nutmeg dealer, the horse swapper and the night man