I don't think it is fair to toss all infrastructure in the same bucket but for infrastructure projects where there are US federal dollars involved, the Davis-Bacon act is one reason why costs are high.
TL;DR Labor rates are above market rates due to the legislative requirements.