Though in many non-techies things, like daily sales transactions it is a way to go.
Ad 1. pandas/numpy would put it on par with 2.
Ad 2. Would disagree. I know data scientist using Spark. Mostly they like Scala API.
In general, everyone got their favorite weapon of choice and what they feel comfortable. The point is that simpler solutions sometimes are just enough do their job.
Renting r3.4xlarge on AWS for an hour and play with your favorite tool may be an orders of magnitude easier/cheaper/faster than using big data solution.