A woman with just one breast over the breast plate would also be bilaterally symmetric. Although, the heart, the lungs, the liver, the stomach, the intestines, and probably more, are all bilaterally asymmetric, so it would seem there are no rules that demand symmetry.
If we all had one nipple, we would have had to have evolved from a species with an even number nipples, which likely means one nipple migrating to the center like a founder's eyes. All of those things we have one of we've had one of forever. Going from an even number of things to an odd number would be a strange trick seldom practiced.
