Only it wouldn't, because the Earth's shadow doesn't extend that far out. Earth-Sun L2 is a million miles from Earth; Earth's shadow tapers to a point well before that.
Better than nothing I guess? Probably not by much though.
Actually, I was somewhat pessimistic:
Angle subtended by Earth at a million miles: (8 thousand)/(1 million) = 0.008
Angle subtended by Sun at 94 million miles: (864 thousand)/(94 million) = 0.0092
Area of Sun covered by Earth: (0.008 squared)/(0.0092 squared) = 0.76
So about 76 percent of the Sun's visible area would be covered by the Earth. The Sun would be about 1/4 as bright from L2 as it is at noon on Earth. That's still very, very bright, but there is some effect due to the Earth.
Also, it's worth noting that the most dangerous radiation in interplanetary space (solar wind and cosmic rays) is not "shadowed" by the Earth in quite the same way visible light from the Sun is. The Earth's magnetosphere does have a "tail" that extends behind the Earth, but there is radiation inside it.