
Mathematicians becoming data scientists - heinrichf
https://quomodocumque.wordpress.com/2017/02/26/mathematicians-becoming-data-scientists-should-you-how-to/
======
ykler
I was a PhD student in mathematical logic, and I know a couple people who got
PhDs in this area and became data scientists. One thing about this field (and
other areas of math) is that almost everyone in it feels mediocre. It's like
10% of the people are 10x better than the other 90%, 1% are 10x better than
the next 9%, .1% are 10x better than the next .9%, etc. The top two people in
the field seem to be notably better than everyone else. Another thing is that
the problems are incredibly arcane. They have no connection to practical
concerns or even often other parts of math, and also you can't really explain
what you are working on to someone who works in a different area of
mathematical logic (unless maybe the person is extremely good). So you have a
lot of people getting PhDs who are very, very smart by any normal measure but
who would be doing mediocre and arcane work in mathematical logic. Part of the
appeal of working in data science or working for the NSA or something is that
it is somewhat down-to-earth and people will think you are really smart even
if you are at the bottom of the top .01%. (And you can do really great work at
this level -- I don't mean to say it is all about caring what other people
think.) It is a little weird that the blog post is so negative in a way.

~~~
george_ciobanu
A lot of these "arcane" topics have long-term horizons. Agree with everything
else but the whole point of research is to probe the space of possibilities
(theoretical and practical) to find major ideas. Most of them fail but that's
part of the exploratory process - it's expected.

~~~
glial
It's expected, but as someone doing research it's super discouraging. I can
certainly see the appeal of moving away of basic research with a very long
term time horizon (fine for science, but frustrating for individuals) and
towards a field with the potential of more immediate impact.

------
nerdherfer
One side of this that bugs the crap out of me: I have a PhD in math and, after
getting sick of teaching mediocre students with a palpable aversion to
mathematics, I spent a couple of years applying for data science jobs and
getting very few responses. The few interviews I got seemed to go well, but I
had no offers and gave up on leaving academics. I have experience in software
development (C++), I have a portfolio of statistics/data analysis projects in
R/RStudio, but what I think is toxic is having spent so many years in
academics; I finished my Phd in 2003. Being an over-40 academic feels like an
insurmountable liability, even though I have many of the skills that people
complain about being in short supply. And another thing: get off my lawn, you
rowdy kids! :|

~~~
ordinaryperson
It's never too late to switch.

I'd suggest figuring out which niche of data science most aligns with your
personal interests and start there. Hiring managers don't discriminate based
on "too many years in academia", it's more what's the reason they should hire
YOU over candidate y?

Diehard animal lover? I bet the Sierra Club would love a statistician who
could help them quantify how many elephants are being poached in Africa per
year.

Interested in sports? Strava gets millions of data points a day, I'm sure
they're looking for help finding signals in the noise.

Can't get enough of financial markets? Finance jobs all over the place for
quants.

If your profile is too generic they'll see no good reason to hire you, but if
you're passionate about X it can really work in your favor.

The job search is depressing but never give up hope.

~~~
theparanoid
As an over 40yr old, yes it's probably too late to switch. After 30 it was a
shock to me getting turned down for jobs when I aced the technical interviews.

~~~
alexpatin
You might be more expensive than some of the youngins too. Just a possibility.
I'm sure there could have been something else but new kids on the block often
take less than they can.

------
stared
I had two blog posts exactly on that topic, math/phys -> data science:

\- why: [http://p.migdal.pl/2015/12/14/sci-to-data-
sci.html](http://p.migdal.pl/2015/12/14/sci-to-data-sci.html) (on academia vs
industry)

\- how: [http://p.migdal.pl/2016/03/15/data-science-intro-for-math-
ph...](http://p.migdal.pl/2016/03/15/data-science-intro-for-math-phys-
background.html) (also got reprinted at KDnuggets)

~~~
heinrichf
These posts were insightful, thank you for writing them.

------
olivierva
Slightly off topic: They still can't build proper software. Academics
(including mathematicians) are notoriously bad at writhing production grade
software. This leads to handovers of 'proof of concepts' to seasoned software
developer team who than struggle with the (often complex) mathematics/science
behind it. Imho universities should give a bit more attention on how to write
quality software; a bit of test driven development and continuous integration
is not that hard and would massively improve the quality of the software
written by scientists.

~~~
emmanuel_1234
This is precisely why I dread hiring academic-only profile. Note: I am myself
from academia, but I was lucky enough to specialized in CS-related field
(NLP). When I wanted to leave academia, Data Science was not a thing where I
live and I had to start again at a junior dev position (i.e.: it was that,
starving or staying in academia).

Now, I work with a lot of people way smarter than I am, who are mostly useless
because they can hardly prototype their stuff in Python or run an SQL query.
And they'd expect to only work on the best, cleaned and formatted dataset and
only do high-end maths on those. Reality hits hard, we're losing money paying
them and they're wasting their time not doing what they like. Add to that the
frustration / jealousy that this creates.

In that regard, I like that famous definition for Data Scientist: "A
programmer that know more about statistics than most programmers, or a
statistician that knows more about programming than most statisticians".

And don't get me started on the general repulsion for understanding the basics
of how a business runs from academia. Data Science is all about application.

~~~
jensv
[https://medium.com/incerto/surgeons-should-notlook-like-
surg...](https://medium.com/incerto/surgeons-should-notlook-like-
surgeons-23b0e2cf6d52#.9nwspr92j)

------
uptownfunk
I interview probably two to three people a week for a DS job. It's a really
difficult role to hire for. I'm basically looking for mathematicians with some
knowledge of stats who also enjoy coding and follow best practices in coding
(i.e. They didn't just pick it up and hack something that works but care
enough to document and structure so others can follow it at a minimum) but we
also need the MBA component as well, a large part of this industry is how do
you take what a DS does and deploy it in a business and ensure the business
derives value from it. But I think coming at it from a mathematical background
puts you in a much stronger footing than the cs guys who try to learn the math
later. I rarely find strong candidates without a formal mathematical
background.

~~~
kriro
Interesting, thanks. It obviously depends a lot on the job but I would have
thought that a good math background (which typically CS grads should have)
would be enough for many applications. Especially since you mention the
business problem aspect. I think that you can actually get quite far in DS (at
least in deep learning) with rather "basic math". Unless you're innovating at
the bleeding edge it's usually enough to reproduce existing ideas and applying
them to interesting problems is the real key. So my hiring would have focused
on CS grads with a natural interest in math, ideally very rigorous/meticulous.

~~~
laichzeit0
It makes me almost feel sad for the people that followed the "why bother doing
a CS degree, you can learn to code without one" trend that probably peaked and
is now maybe beginning to die a slow death as the market is flooded with
people who can write code.

If they ever want to transition into DS they're going to have to skill up and
do the equivalent of undergrad CS math curriculum (discrete, calculus, linear
algebra, math stats, etc.), which let's be honest, you cannot pick this up in
any meaningful way in a "few months" of after hours/weekend study like you can
when learning to program or learning some new "framework"; either that or just
remain another dime-a-dozen code-cutter. It's sad because if you just did the
4 years of computer science and all the math and the stats that go with it
you'd be so close to pivoting right now if you wanted to.

~~~
Almaviva
> you cannot pick this up in any meaningful way in a "few months" of after
> hours/weekend study

You can't pick up coding like this either. See Peter Norvig's famous "Teach
yourself programming in 10 years" article. The delta in the wisdom you obtain,
between a few side projects over months and battle hardened experience with
real products and code bases over years, is immense.

------
uihsdfjkh
What about the other way around?

I'm from an engineering background, going in the direction of data scientist.
Sometimes I find that my math skills could be stronger, and I try to read up
on things when I encounter them, but still it sometimes feels like there is an
infinite amount to learn. Maybe I could use some more systematic approach to
it. Anyone else who has walked this path, and could come with some useful
advice/resources?

~~~
tmccrmck
Are you talking about self-studying mathematics or going back into higher
education?

I can give recommendations on books for the former but not the latter.

~~~
praveenster
Yes, I am also interested in going the self study route with MOOCs but there
is way too much information and resources that it is hard to figure out
how/where to start. It would definitely be helpful if you could provide
recommendations on books.

------
shas3
An earnest question I've heard from people trying to switch is: what's a good
benchmark for testing where you stand in terms of coding skills? How do you
calibrate and measure your production-quality coding skills in Scala, C, etc.
when you have spent all your life in academia?

I feel Kaggle is sufficient for the exploratory part of data science. But
Kaggle's relevance to testing and honing "production quality" code-writing
skills is sometimes minimal.

In the larger context of programming and software engineering, scientific
programming is fairly easy to code-up (though they are harder to conceptualize
and understand mathematically). The coding part of non-CS/non-CE/non-some-
parts-of-EE academia is pretty much mostly scientific programming. Yet,
production quality code is seldom just scientific programming.

------
heinrichf
Note: this is the blog of Jordan Ellenberg
([https://en.wikipedia.org/wiki/Jordan_Ellenberg](https://en.wikipedia.org/wiki/Jordan_Ellenberg)).

~~~
curiousgal
I've just started reading _How Not to Be Wrong: The Power of Mathematical
Thinking_ by him. I enjoy these synchronicities.

------
wolfkill
This post mostly describes me. I am a PhD student in applied mathematics and
returned school to get my PhD precisely because I wanted to get a job outside
of academics, most likely as a data scientist or scientific programmer. 2.5
years into my program I still struggle to see beyond the degree but am
currently looking at non-academic internship opportunities as a way to start
to untangle myself from academics. I also spend a significant amount of my
time trying to teach my self good programming practices that I don't get from
mathematics (how to write clean, adaptable code, choosing appropriate design
patterns and writing generic code without over-abstracting, writing readable
documentation and using version control, etc.) To those of you who have left
academics do you have any other suggestions? How about for marketing myself as
a programming mathematician when competing against CS and engineering students
for internships/jobs?

~~~
IndianAstronaut
Take a look into data engineering. A lot of data engineering jobs require you
to implement mathematical models at scale.

------
visarga
> Do you like modeling complex problems mathematically?

Do you enjoy kicking the ball towards the gate and scoring? Then you might be
good for Football.

------
11thEarlOfMar
I've been encouraging my daughter, a statistics major, to pursue data science
by including Python/R in her studies and then possibly heading back for an
MBA. But not sure if an MBA would be a benefit. Thoughts from actual data
scientists?

~~~
nrjames
I'm a data scientist with a stats background. I'm also a self-taught
programmer (going on 15 yrs of programming in various languages). There are
very few data science jobs that I am aware of where somebody will hand you a
tidy data set and ask you to start creating models. For those jobs, it's great
to go straight from stats/math to data science, as long as you learn a bit of
R and some SAS along the way. Most data science jobs that I know of involve a
lot of data engineering. That is to say, you need to understand where the data
originates, how to get it through an API or via other scripts, how to design
and implement a reasonable (often relational) model of it and put it into a
database, how to clean it, how to query it into nice and clean output files,
and then how to model it. You also need to know how to visualize it
effectively and use it to tell a business story.

I would not be able to do my job at all if I didn't know Python, R, and
JavaScript well (and know my way around various Linux flavors). The modeling
is fun, but it comes at the end of a long pipeline requiring a lot of skills
that are more engineering oriented.

Rarely do my tasks sound like "model this weekly and give me the result."

Often, they sound like, "I need you to pull together data from these 5
sources, model it, and produce a weekly report showing these derived KPIs. And
I need to be able to access it in a web browser so that I can send links to
colleagues. And it needs to be secure. And generating a report across an
arbitrary date range needs to take less than a minute."

By the way, that is a request that I've gotten at three different jobs. To
give you a sense of what this looks like: most recently, I wrote Python
scripts to harvest and ETL the data into a Postgres database (running on
Google Cloud) and a BigQuery table, then wrote a Flask app to accept the
arbitrary report requests and query the database, run models, store computed
results in a local SQLite database for fast future retrieval... finally
producing dynamic Reveal.js slide decks available through the Flask app.

That's a long-winded way of saying that I strongly suggest that your daughter
get some practical experience with the data engineering side of data science,
preferably using Python for fetching, manipulating, storing, cleaning, and
preparing data. It's the most flexible tool for the job and it easily the most
important tool in my data science toolkit.

~~~
11thEarlOfMar
Thank you!

As mentioned separately, I suggested a data science boot camp after getting
the Stats BS, work a couple of years as a data scientist, then go back for the
MBA. My thinking RE the MBA was to give a sense of the business value of the
analyses she's performing, and, to make it easier to promote her to executive
positions.

Maybe that's old school thinking, I know that the MBA in general gets a mixed
reception these days, but we can look more closely at it after she's out of
school.

~~~
nrjames
To me, an MBA is more about business strategy. Knowing the relevance of data
to the operations of a business is vital to being an effective data scientist.
Knowing when to make the call to lay off 500 people is the job of somebody
with an MBA. Data scientists prepare and present the evidence in the language
of business. There is crossover, of course.

What many data scientists (myself included) find is that they often are
excluded from meetings and conversations that provide the context for the
analysis they are doing. I always tell my supervisors that it is very helpful
for me to sit in on as many business strategy meetings as possible, just to
listen, because it builds context around the work that I do and helps keep me
properly focused.

On the flip side, many involved in business strategy do not understand the
nuances of analysis performed to support their business questions. There can
be many reasons for that, from not being directly involved in the analysis to
being excluded from data science meetings. Effective business analysis and
data science requires trust between the players and that trust is built
through showing an ability to deliver focused, relevant, and accurate results
that support decision making.

As somebody who likes to write code and dislikes sitting in tons of meetings,
I tend to avoid climbing the career ladder to management positions. I'm happy
in a Senior Data Scientist position. At my last job, I was being groomed for
management and I never got to do any actual data science work. It was boring
as hell. I made a lateral move to a different company so that I could be more
hands on and work in an industry that is more fun. Now, I get to write code
and build and run models every day. I also get to present the results and have
a trusting relationship with the managers.

As your daughter finishes school and gets some job experience, don't be too
quick to suggest routes leading to business strategy and management. While
it's the "top of the career ladder" at many companies, so to speak, it's not
for everybody.

------
xamuel
As a mathematician who successfully made the leap, I think the biggest
requirement is ability to swallow your pride and "lower" yourself to seriously
study professional coding.

There's too much temptation to hack something in Python or whatever, google as
you go, etc. What I did was sit down and practically memorize entire
programming manuals. Many academics would refuse to do something so plebeian.

------
mrcactu5
I remember asking why i was rejected from Palantir and vaguely recall the
phrase "too mathematical".

After years of trying I have been unable to make the transition from
Mathematics into Data Science and have since shifted my attention to other
things.

------
happy-go-lucky
If you see programming as a tool for solving domain-specific problems, then
domain experts who can exploit the tool obviously have an edge over
programmers who rely on inputs from these experts when developing domain-
specific applications.

------
bayesian_horse
I guess it is easier to get into data science as a math graduate. I'm trying
without a degree, and haven't succeeded yet.

------
godmodus
A few fellow students graduated with a BA and had their aplications rejected
by most companies due to a lack of experience.

Theory is nice. But Experience trumps all. Esp when you think it mostly boils
down to PCA/DBSCAN, regression and scatter matrices.

A month of Python/R/D3 on some initial data like pubmed/twitter or any toy
dataset would go a looong way.

~~~
kriro
I think doing well on Kaggle competitions would already be a good enough
indicator for smart companies so it might be a good idea to have student teams
do those on the side. Might be a decent idea to inform students that these
exist. I feel like this applies even more for other degrees like
econ(ometrics) (since I feel like a good math grad should not have trouble
getting any data science job).

------
diimdeep
>. A big thing about the transition to tech is that you possibly start
communicating with people who don’t really know what a vector space is. Be
ready to have those conversations. Honestly ask yourself if you’re OK with
having those conversations.

This is so snobbish. What here is even think about ?

~~~
cottonseed
This is the wrong comparison. Read the second footnote. An intern is a
developer in training. The post is talking about mathematicians (who spent a
decade or more learning a highly technical field) going to work in an
environment where, largely, their peers, boss and consumers of their work do
not know, have no interest in and and will never learn the most basic concepts
of their field.

~~~
pavlov
I suspect the majority of programmers work in that kind of environments...?

The startups and Googles of this world are the exception when it comes to
employer tech-savviness.

~~~
Stasis5001
When you get a PhD, you mainly only take classes in your department for the
whole time, whereas in undergrad you still have classes in other fields. This
usually extends beyond coworkers into friend circles too. This isn't math
specific, but more generally to grad school.

Second, programmers at companies almost never work in isolation from other
programmers, and in most open office environments are close enough they could
touch another one from their desk. On the other hand, a startup with five to
ten engineers may be hiring you as their first data scientist, and bigger
companies may be putting you on an embedded team, with the nearest data
scientist a hallway or floor away. And this isn't a big deal in terms of
teamwork, but most data scientists don't have a PhD in mathematics, so if you
find higher level mathematical ideas and notation to be the most efficient way
for you to think about a problem, your colleagues may not. That's also not a
math specific thing though -- an economist, statistician, mathematician,
computer scientist, physicist, etc. are all going to have slightly different
ways they think about things internally.

------
happy-go-lucky
There is positive correlation between applied math and programming. I see
expertise in these areas coupled with a knack of getting a point across as a
potential career opportunity. No wonder academics in some parts of Asia have,
for generations, seen math as “evergreen.”

------
denzil_correa
A lot of this can also generally apply to a Computer Science Ph.D graduates
who would like to understand what working in the industry entails.

------
known
I think MBA adds more value to your career in globalization

------
M_Grey
_•Can you walk away from a problem when the solution is “good enough,” are you
able to switch between tasks or problems with relative ease? Are you OK with
simple solutions to problems that could have more complicated solutions, but
only with rapidly diminishing returns?_

Translation: Can you be a good assembly line worker, devoid of pride or
curiosity and just do the damned job? If so, great, we can use you! If not,
please return to academia.

Lovely stuff.

~~~
jknoepfler
I'm sorry, what? Being a good engineer in the real world means building
quality solutions within time and cost constraints. It's a craft. One can be a
good engineer and have both pride and curiosity. You also need humility and
the ability to put your customer's needs ahead of your ego.

I mean, by all means open up a little boutique data science consultancy that
charges three orders of magnitude more for a solution that can't be updated by
anyone without a ph.d and only beats an out of the box svm by .1%, where you
put your pride and curiosity (your vanity) first. You might even find some
suckers to patronize you, but man... I must be misunderstanding you because
that is the most clueless and entitled thing I've read in days.

98% of the time data science customers actually just want some basic
statistical insights into their data to make better informed decisions. If you
aren't willing to help with that, and also can't find a research group to take
you in (to work on _their_ projects) and also can't bootstrap your own
startup, then yes, stay in academia.

~~~
M_Grey
We're... not talking about engineers though.

~~~
jknoepfler
A data scientist is an engineer.

~~~
pryelluw
Can you elaborate as to why and how? Im genuinely curious. :)

~~~
stale2002
An engineer is a person who creates real world solutions to real world
problems.

What matters is not how elegant your solution is, what matters is solving the
problem. And doing it under cost and time constraints.

An engineer is first and foremost, a person focussed on the end goal, above
all else.

~~~
throw_away_777
This definition of engineer is so broad as to be meaningless. There is no
"fake world" with "fake world" problems, pretty much everyone is trying to
create real world solutions to real world problems. Similarly most jobs are
focused on the end goal. A data scientist job usually involves a mix of
science and engineering.

~~~
stale2002
My definition of "real world" problems are ones that people are willing to pay
for. If nobody is willing to give you money for whatever it is that you are
doing, then it probably isn't solving someone's actual problem.

~~~
throw_away_777
The problem with this approach is that people pay much more money for limited
gains they can exploit than for general gains that benefit everyone equally.
Advancing academic knowledge provides gains distributed over a very large
number of people, but the current system often does not efficiently allocate
resources to this problem.

