Swords and breastplate weigh far more than a "few pounds." Breastplate is close to 20lbs, a two-handed sword up to 10lbs. And that's just a small amount of the armor a knight would have (ignoring the fact that most warriors of the era wore chainmail, not plate).