This is my attempt to avoid far-mode abstract thinking when talking about this, and reflect on the mistakes inherent into how I chose my specialization.
I work in machine learning, especially for “creative” leaning technology companies. I have significant experience in quant finance and education technology as well.
I chose my specialization because mathematics and statistics were my strongest skill areas in my undergrad education. I won some math competitions and had fun in team oriented math modeling competitions.
I debated going to grad school or working right out of college, ultimately choosing to work at a defense research lab. I didn’t like it very much so I went for my PhD a little leas than 2 years later.
I bounced around several jobs in the ubiquitous 2 year stints for a while, finding startups to be deplorable with low pay, bad equity deals and poor work/life benefits. I enjoyed quant finance a little but ultimately it was too boring.
I finally started working in ML in ecommerce companies, and have about 12 years experience in 2 large ones, moving from engineer to architect to manager.
I really like product development in this area. There are so many ways machine learning adds tangible value for customers especially when the underlying products involve creative aspects, like merchants selling and marketing crafts, or people designing copy text or imagery. Meanwhile there are also lots of internal use cases like demand forecasting, improvements in help center search experiences, detecting abusive content or fraud, etc.
Nonetheless, it’s a real challenge to work on ML professionally. It receives very undue skepticism and fights a huge uphill battle on every project because other stakeholders don’t invest a lot of time to understand it. Somehow a string of objective successes never sticks in their mind and every time a new company leader is brought in, it’s a whole new game of avoiding their biases and chipping away at unfounded skepticism.
On the engineering side, most of the work is unglamorous data cleaning & pipeline maintenance, constant turf wars with operations and infra teams that don’t want the responsibility of dealing with deployment constraints of ML systems that break cookie cutter patterns for other classes of systems.
It’s also exceedingly political. I chose statistics and ML in part because at some level it should supersede subjective opinions about any possible domain, bordering on being a root subject of all philosophy and epistemology, and so surely it supersedes politics when real money is on the line in businesses, right?
Alas, not true. Machine learning engineers who display a naive sense of intellectually honest curiosity applied to business decisions will get a rude wake up slap by all sorts of executives and managers who don’t give a shit and have ulterior motives.
See [0] for a good take on it that I really wish I understood before choosing ML as my specialty.
I’ve hired and lost so much great ML engineering talent over the years, and it takes a toll seeing these brilliant people not have their specialty or their comparative advantage respected, to the point that someone who is a world expert on computer vision is off trying to debug some inane kubernetes error, and management can only see “fairness” arguments in favor of this instead of realizing they’re flushing this person’s salary down the toilet by asking them to do something many other people can do at a direct opportunity cost of them using their specialization for things nobody else can do.
My advice for choosing a specialty is to interview lots and lots of people who have jobs all across the career ladder for that specialty - interns, new grads, mid-level employees, senior employees, managers and directors. Try to get a holistic sense of what they like and dislike and what negative realities you’re going to need to live with if you choose it.
I work in machine learning, especially for “creative” leaning technology companies. I have significant experience in quant finance and education technology as well.
I chose my specialization because mathematics and statistics were my strongest skill areas in my undergrad education. I won some math competitions and had fun in team oriented math modeling competitions.
I debated going to grad school or working right out of college, ultimately choosing to work at a defense research lab. I didn’t like it very much so I went for my PhD a little leas than 2 years later.
I bounced around several jobs in the ubiquitous 2 year stints for a while, finding startups to be deplorable with low pay, bad equity deals and poor work/life benefits. I enjoyed quant finance a little but ultimately it was too boring.
I finally started working in ML in ecommerce companies, and have about 12 years experience in 2 large ones, moving from engineer to architect to manager.
I really like product development in this area. There are so many ways machine learning adds tangible value for customers especially when the underlying products involve creative aspects, like merchants selling and marketing crafts, or people designing copy text or imagery. Meanwhile there are also lots of internal use cases like demand forecasting, improvements in help center search experiences, detecting abusive content or fraud, etc.
Nonetheless, it’s a real challenge to work on ML professionally. It receives very undue skepticism and fights a huge uphill battle on every project because other stakeholders don’t invest a lot of time to understand it. Somehow a string of objective successes never sticks in their mind and every time a new company leader is brought in, it’s a whole new game of avoiding their biases and chipping away at unfounded skepticism.
On the engineering side, most of the work is unglamorous data cleaning & pipeline maintenance, constant turf wars with operations and infra teams that don’t want the responsibility of dealing with deployment constraints of ML systems that break cookie cutter patterns for other classes of systems.
It’s also exceedingly political. I chose statistics and ML in part because at some level it should supersede subjective opinions about any possible domain, bordering on being a root subject of all philosophy and epistemology, and so surely it supersedes politics when real money is on the line in businesses, right?
Alas, not true. Machine learning engineers who display a naive sense of intellectually honest curiosity applied to business decisions will get a rude wake up slap by all sorts of executives and managers who don’t give a shit and have ulterior motives.
See [0] for a good take on it that I really wish I understood before choosing ML as my specialty.
I’ve hired and lost so much great ML engineering talent over the years, and it takes a toll seeing these brilliant people not have their specialty or their comparative advantage respected, to the point that someone who is a world expert on computer vision is off trying to debug some inane kubernetes error, and management can only see “fairness” arguments in favor of this instead of realizing they’re flushing this person’s salary down the toilet by asking them to do something many other people can do at a direct opportunity cost of them using their specialization for things nobody else can do.
My advice for choosing a specialty is to interview lots and lots of people who have jobs all across the career ladder for that specialty - interns, new grads, mid-level employees, senior employees, managers and directors. Try to get a holistic sense of what they like and dislike and what negative realities you’re going to need to live with if you choose it.
[0]: https://www.cato-unbound.org/2011/07/13/robin-hanson/who-car...