
Where are the “big problem” jobs? - joeyespo
http://lemire.me/blog/archives/2013/03/04/where-are-the-big-problem-jobs/
======
nnq
I wonder if the "basic income guarantee"
(<http://en.wikipedia.org/wiki/Basic_income_guarantee>) applied at something
like country-level wouldn't dramatically increase the number of people working
on "big problems" and of entrepreneurs trying "big problem businesses" - with
no personal risk of failure, people would actually start chasing the "big
problems" simply because they are FUN, and with no adrenaline from our little
economic rat-race, this could actually become the only source of "fun"
(besides hard-drugs for now and maybe some virtual-reality based
entertainment) for creative and problem solving types. I don't advocate this -
it would probably fail miserably and be "globally unfair" if applied at the
level of a wealthy nation, but I always ponder on the consequences of such a
thing. We're so "knife to neck" in this race for profit that only very few
lucky and smart people have the opportunity to ponder the big problems without
any other worries - maybe quantity (as in number of brains involved and
pumping up ideas good and bad) as opposed to quality (very few very smart and
well funded "geniuses") would yield better results on these so called "big"
problems (I think of things like "general AI" or "cure for cancer" kind of
problems). And the funny thing is that such an "arrangement" has never been
tried in the history of mankind, so it might as well "work" (in the sense of
"accelerate" the solving if "big problems").

...and traditional market forces have never been known to catalyze work on
"big problems". The only good catalyst I can think of is _war_ (think the
nuclear weapons problem or the race to the moon fueled by the cold-war).

~~~
cpeterso
Access to socialized healthcare is a similar factor. If a potential
entrepreneur in the US relies on health insurance from their current employer,
they may be reluctant to start their own business without a "social safety
net".

~~~
JanezStupar
Few people care about health care until they really need it. Thus I would
argue that it really is not such a significant factor.

Other things are much more important. In a sense all Americans are
businessmen. There are few countries that value business, entrepreneurship,
business people and property as highly as Americans.

~~~
Gustomaximus
I feel you have different perspective, maybe that of a younger man. IMO people
who are 35+ and support families will be more conscious about not having
healthcare that a younger person. And the image of the entrepreneur
predominately being a young man dropping out of college to form a company is
not accurate. The reality is a large proportion of successful entrepreneurs
are people exiting the corporate world mid career. And I would argue for them
healthcare is a significant consideration.

As for America valuing businessmen/entrepreneurs/property more than other
countries, how do you get to this conclusion? Have you worked with people from
abroad before? From Western Europe to India, China or Africa I have seen
businessmen/entrepreneurs/property being held in very high esteem. I'm not
sure how I would index how society values them but I would be hesitant to say
the US values theirs more than other countries.

------
epenn
_There are corporate exceptions. For example, Google seems to be working on a
few big problems, such as the self-driving car and Google Glass._

I'm not entirely sure I would classify Google Glass as a "big problem." Self-
driving cars can address driver safety, fuel efficiency and route efficiency,
among other things and therefore certainly rank up there. Google Glass,
although I like the concept and very well might even purchase a unit when they
become widely available, seems to address the same areas already covered by
smart phones, just with a new interface. I think there are huge strides that
can be made in the area of pervasive computing, but I'm not sure Google Glass
is what will propel that.

(I would love to be proved wrong though.)

~~~
epoxyhockey
I think SpaceX and Tesla Motors (Elon Musk) are better big-problem solving
examples than Google Glass.

~~~
bostonpete
I get why you'd say that about SpaceX, but why is Tesla in that category?

~~~
artmageddon
Consumers have been very reluctant to drive electric vehicles / hybrids
instead of regular combustion-engine vehicles. It has huge implications for
the petroleum business as well as the future of transportation.

~~~
felipe
Tesla is focused on a niche: Luxury / high-performance cars, which is a tiny
fraction of the market in units, and even if they were wildly successful they
would not even make a blip in the carbon emission statistics.

~~~
rtrunck
It is for now, but see <http://news.ycombinator.com/item?id=5310911> for
discussion from the weekend about Elon's post from a few years ago about his
plans to enter the mainstream with Tesla.

------
ChuckMcM
Perhaps there are still too many "small problem" jobs that are profitable to
solve so the capital is not out looking for the 'big' ones.

Of course the imprecision of 'big' and 'small' in defining problems is also an
issue. You can have 'big' problems which exist not because of the science, but
because of the structure. Today we were reminded that its a 'big' problem to
build a house in South Florida that won't be eaten by a sink hole. Earlier we
were reminded about building on eroding beaches. But those are actually "easy"
in the sense that you could push piles into bedrock, 75' below the house. But
we don't.

Bill Gates has made it a priority to solve some 'big' problems, sanitation,
malaria, to name a couple. Other foundations have similar goals. So they are
out there but as the other has seen working on them when they aren't self
funded is hard to achieve.

------
ambiate
'Research is not profitable. Sure, we could put 250MM a year into big data and
helping people, but our revenue from small data keeps the business solvent. We
grow, we scale, we profit. Problems are to be solved by entrepreneurs with few
resources and great intentions. There are no good intentions in business, only
profit.' -- Ex-Boss.

(The site was down for me, but I assume this is the direction of the article,
and this quote comes to mind).

~~~
vellum
Cached copy:
[http://webcache.googleusercontent.com/search?q=cache:http://...](http://webcache.googleusercontent.com/search?q=cache:http://lemire.me/blog/archives/2013/03/04/where-
are-the-big-problem-jobs/&hl=en&strip=1)

------
dj-wonk
What one considers a "big problem" depends on your values (i.e. where you want
society to go and how you want it to get there). From my perspective, many of
the "big" problems are just that because they are the result of complex
dynamics and structural challenges that cannot easily be solved by one actor
(organization / business / government).

Many of these problems are so-called collective action problems. If you are
able to get enough funding and a strong team, that may not be enough: you need
more than technical chops to move the system enough to make a difference!

P.S. I'm glad this topic is getting some air-time!

------
ihnorton
tl;dr: Every time this comes up I wish the writer would start with a personal
"taxonomy of big problems" to better frame the discussion.

At very least, I wish these writers would preface with a few examples of what
kind of "big problem" they think corporate R&D should be working on, but
isn't. OP's example falls flat to me:

> Private businesses seem remarkably uninterested in tackling serious problems
> such as energy despite soaring prices and evident problems.

They may not be interested in solving energy problems in the way OP wants them
to, but there is certainly massive industry R&D in the energy space (Siemens
and Schlumberger come to mind, many others). Just look at the ever-increasing
yields from fracking and (ever deeper) deep-sea drilling. That requires huge
R&D investments (down-bore sonar and NMR, anyone?)

If you consider the sorts of problems that are presently somewhat tractable
_modulo engineering_ {to pick a few: energy, cancer, food, genetics} I would
argue that there is probably a lot more industry R&D work than the author
realizes. It may not happen in the software industry, but realistically, for
many "big problems", software is auxiliary at-best.

If you are considering the sorts of problems that are presently only tractable
_modulo magic_ {fusion, life extension, interstellar travel} then of course
corporate R&D is not working on those.

------
soup10
There's not that many because most people don't care about doing big things.
Most of the wealthy are more concerned with protecting their wealth, status
and success than risking looking like a fool trying to change the world.

~~~
obviouslygreen
I agree with the wealthy point; it even applies to the not-so-wealthy who are
just trying to make their way and support their family and/or employees.

However, I do think a lot of people do care. It just doesn't end up mattering,
because the people who have the means have been through so much red tape and
related crap -- and usually have so many shareholders to report to -- that
it's no longer a matter of their own priorities, it's a matter of perpetuating
a business on which even _more_ people depend.

This isn't meant as an excuse, but a reason. We're not working on "big things"
because as a species we have totally failed to prioritize our own global well-
being and that of our world.

------
patmcguire
Every time someone complains about tech businesses not taking on the big
problems, I'm reminded of the early days of computer graphics when there were
all sorts of complicated algorithms to figure out which parts of a complex
arrangement of polygons would be visible on the screen.
<http://en.wikipedia.org/wiki/Visibility_problem> It turned out that the best
way is generally the comparatively crude z-buffer. (I'm not super educated on
this, this was just a story my prof told me).

The point is that the big technical problems that need the most research
muscle aren't necessarily that important, and you can't really know what the
future needs ahead of time.

------
calinet6
A simple answer: too much risk!

There is a threshold to capitalism. The problem is that no one has yet figured
out how to hedge the bet, attempt to solve extremely risky borderline
impossible problems, and guarantee a profit. It's much _much_ safer to go
after an obvious, well-described, minor problem that over 50% of the
population experiences and would pay a minor sum to improve. It's a safe bet.
And there are a lot of those!

The other problem is surely capital and education. They've diverged and are
being pulled in opposite directions. Higher education _saddles you with debt_
and makes you _extremely risk averse._ You can't ever get access to capital
because you're always playing it safe. So the system has been inadvertently
designed to lock out these "big problem" jobs, because, again, they are too
risky for people to undertake.

And then the rich are also risk-averse these days. The focus is on ROI and
increasing wealth—look at Warren Buffet, one of the world's richest men,
entirely rewarded for being risk-averse and investing in solid, tried-and-true
companies. And you're surprised that the big problems aren't being solved?
People like Buffet aren't investing in them, at least not at a large scale.
And they're rewarded for that behavior. The system is set up to reward this
sort of behavior. The idea is that once you get rich, you invest "prudently"
and grow your wealth. You don't bet it all on the big problem that's probably
impossible.

Except sometimes you do. Some people are solving the big problems. Elon Musk
is up there doing it (in space and on the ground). Bill Gates and Buffet too
are putting up a lot to fight seemingly impossible big health problems. This
is all excellent, and not to be ignored.

I think the incentives are not yet aligned correctly to get the result you're
looking for. What you want is to reward risk taking, and capitalism just
doesn't do that in the right way. There are too many hurdles, too many
punishments for failure. And the right people to solve the problems aren't the
same ones who have the capital to get the solving done, for so many reasons.

There's your big problem to solve—fix that system. Align the incentives. It's
the meta-problem, and if you solved that, the world would probably be a better
place. (Hint: there are already some good solutions, but the America won't
vote for them)

~~~
AznHisoka
Yup, don't blame the people, blame the system. That's exactly it. It's like
asking why aren't all the hot girls pursuing the nice guys.

------
cpeterso
In Robert Heinlein's novel _Time for the Stars_ , a non-profit organization
called the Long Range Foundation ("LRF") funds expensive, long-term projects
for the benefit of mankind that won't show results for decades or centuries.
Some of these projects turn out to be hugely profitable, but because profit
isn’t in the organization’s charter, all this money has to be funneled into
other longer-term and even more outrageous projects.

------
kyllo
Because the biggest problems are the most expensive problems, and the most
expensive problems require the deepest pockets, which belong to the
government.

~~~
fennecfoxen
And when the government opens up its deep pockets, even on reasonable problems
(like "low-pollution energy") you get results like Solyndra... or worse,
because it's more about chasing government money than coming up with any
results.

~~~
bearmf
But you also get Sputnik, Gagarin, Apollo, Voyager. You get microprocessors
and Internet. You get nuclear energy (though you will be right to mention
industrial fusion reactors which are always 20 years or so away :))

~~~
fennecfoxen
One difference between solar power and Apollo is that the people in the
government who did Apollo were worried about the Soviet Union handing our
collective rear ends to us, possibly on a radioactive platter. Key
stakeholders knew there were consequences for getting it wrong, e.g. their
homes and families potentially getting vaporized.

The subsidies for the Solyndra debacle are still viewed by many partisans as a
reasonable price for a laudable attempt to do the right thing, instead of a
massive waste of capital. Failure? What's that?

~~~
bearmf
I don't know much about Solyndra. But you are right that people getting the
money for research need to be honest and motivated. In my opinion, this
applies to most people working in science. It is just that research is hard
(and possibly getting even harder [1],[2]) and motivation alone is not enough.
If you have a "serious problem" to solve, you will most probably need to throw
tons of money and people at it.

[1] [http://blogs.wsj.com/ideas-market/2011/02/07/the-
difficulty-...](http://blogs.wsj.com/ideas-market/2011/02/07/the-difficulty-
of-making-new-discoveries/#)

[2] [http://www.telegraph.co.uk/science/6694932/Have-we-
discovere...](http://www.telegraph.co.uk/science/6694932/Have-we-discovered-
it-all.html)

------
carterschonwald
Those big problem attacking businesses are out there, they're just often too
busy to write blog posts about it! :)

That said, I'm one of those grad school dropouts who's stumbled into the whole
trying to bootstrap a hard problems only business.

Its a lot of work! I'm lucky that my family is based in NYC so that I can
reduce my living costs plus do hard problems only consulting part time while
spending the rest of my time working on building my tech products @Wellposed.

There are uncountably many high value problems you can try to build an
interesting business around. Just look at any Specialized / niche high value
industry and ask "can we build better tools and how valuable would those be?".
You should choose a domain where you'd enjoy talking with/getting to know
potential customers personally. if there are other folks attacking the same
question domain+combo, ideally they should be smart folks you respect who are
solving the problem with a different combination of tradeoffs so that the
products are complementary rather than substitutes. Oh, and you should be OK
with building a satisfactory product taking at least a year of hard work,
constant learning , and feeling stupid because you keep on seeing better ways
to do every thing.

Or at least that's sort of the meta pattern behind how I'm operating at
Wellposed. Its hard work, I get to feel stupid in the always learning more
sort of way, and its the most fun I've ever had.

I'll find out towards the middle / end of this year if my ideas and process
bear out on my first hard problem domain. I'm terrified and excited. Either
I'll quickly be trying to figure out a sane hiring process, or I'll be job
hunting with an interesting engineering and consulting portfolio accumulated
over the preceding year, and I don't know which outcome is more likely!

That said, entrepreneurship is hard work and gets in the way of spending
focused time on engineering. Good thing I'm a tad extroverted.

~~~
zanny
> Good thing I'm a tad extroverted.

Lucky bastard. I'm always jealous, being a deep introvert, of how others can
engage with others so openly.

~~~
carterschonwald
Its really hard though! The thing I've got to remind myself that when theres a
negative backlash, then I've just passively filtered people.

It takes work, I have my insanely self conscious days/months/years. Baby
steps!

------
betterunix
Solving big problems means taking big risks. Most businesses would rather do
less risky things that bring in the revenue sooner than risk spending large
sums of money on projects that could only bring in money years down the road.
Exceptions exist (Microsoft Research comes to mind) but for the most part,
businesses focus on their short-term profits, and leave research to
universities and governments.

This is only a problem if the government is failing to fund research
adequately. The point of government grants is to pay for riskier research.
There are limits to what governments will fund, and unfortunately there are
"research topics du jour," but I do not really expect companies to go wild
with shareholder money. The free market is not going to cure cancer, and that
is fine because the free market is not the only way to pay for research.

------
AznHisoka
Solving big problems doesn't necessarily = profit, especially in the short-
term, which is what drives most companies and businesses. Unless you got
billions of disposable income (ala Google), you just don't have the luxury to
delve into "big problems".

We live in a modern world with many advanced technologies. But in a way, we're
not unlike the hunter gatherers who had to spend their hours worrying about
survival. We spend the vast majority of our 24 hours commuting, and working
for The Man instead, worrying about paying our bills, loans, mortgages,
putting food in the table, college tuition, etc.

------
dj-wonk
Debating which particular technologies are "big" and "small" is rather silly
IMHO. No offense to armchair commenters on Hacker News, but this is not a new
topic where we are breaking fresh ground. For example, there are numerous
articles on Innovation Policy, such as
[http://www.itif.org/publications/global-innovation-policy-
in...](http://www.itif.org/publications/global-innovation-policy-index)

------
Schwolop
Well I for one would love to work on a big problem. The reason I'm not is
because the smaller problems pay, and the bigger ones (generally) don't. I
tried running my own business aimed at a big problem (admittedly not very well
or for very long) and constantly hit upon the difficulties of getting anyone
to pay for it.

~~~
nbroyal
The openings also aren't very visible (or I'm not looking in the right places,
which is entirely possible). I'm surveying the job market after shutting down
my own business, and I've seen very few openings from companies that are
working on what I would call "big problems." It's a bit disappointing.

------
rhizome
Aren't they at the companies who ask Big-O and LSP questions in interviews?

------
graycat
This article is not nearly the first time I thought about such things. His
reason for getting a technical Ph.D. was essentially the same as mine. And I
concluded the same as he did, that to use such education in business,
basically have to start a business and own it.

He omitted my basic rule: For getting venture funding for such a company, the
rule is that some successful research promising as the core 'secret sauce' of
a new business together with a dime won't cover a 10 cent cup of coffee, that
some solid research for a great technological advantage with some traction is
significantly less good than just the traction.

For more, such discussions miss some big points likely especially relevant
now.

First, let's consider the 'big problem': Since we've already decided that we
have to be in business, the 'big' we might as well take as promising to make a
lot of money, e.g., build a company worth $100 million times some factors of
10.

Second, for building this company, we come up with two lists: The list in
column A is for technology we have or might invent. Column B is for problems
people have and are or at least will be quite willing to pay to get solutions
(if there is a big question about 'product/market fit', then leave the problem
on the cutting room floor -- we want no questions about eager customers). Yes,
it can be that we deliver the solution for free on a Web site and then get
paid by advertisers eager for the eyeballs of the users -- that's a small
detail we all understand. We want no technology sold before its time, no newly
invented technology looking for suitable real problems, no products without
eager customers.

Then we pick a 'crucial core' technology from column A and a corresponding
problem we will solve from column B.

Third, we already know how we will deliver the solution -- over the Internet
-- and we already know how the solution will be generated for the
users/customers -- via software on computers. Why do we know these things?
Because of the astounding, historic improvements in price/performance of
bandwidth and computing, and platform software, the fact that these
improvements have not yet been fully exploited, and the fact that in the
current economy information generated from software can be among the most
valuable, and most profitable, results to bring to the market. Biotech? Have
to wrestle with the FDA, at $100 million or so a round. Energy? It's an old
field, has been slow to exploit nuclear energy which was and remains one of
the greatest opportunities in the ascent of man, and, thus, really is stuck in
the mud. Materials science? A sheet of carbon atoms is not yet an actual
product that can be sold to solve a real problem. Flying cars? I won't
comment!

Fourth, here's a secret (make a billion): When we look at column A of the
technology, note carefully the phrase "or might invent". No joke, guys. It's
too easy and common to laugh at original research. Most of the people laughing
have done no original research. And it's too easy to notice that the guys in
universities that do original research usually get maybe papers, professional
reputation, and tenure but no business revenue. Why? Because heavily that
research is funded for US national security or US health, and in either case
the problem sponsors in the funding agencies don't want just 'research' to
make good products to solve real problems but another E = mc^2 or actual good
progress on a cure for a major disease, especially cancer. Considering that
they are spending my tax money, I'm essentially pleased. NSF/NIH money is too
short now to fund the best, good, far out, fundamental research; there's no
hope for it funding good original research to make money. But such research
can be done. Trust me!

Then with the research, old or original, from column A and problem from column
B where the research provides a good solution to the problem, convert the
research to the corresponding software, load it onto a server, put a Web site
or an app in front of the server software, please users/customers, and make
money.

In this, research just such as I described is crucial for much of the progress
people are not seeing but have been expecting.

Silicon Valley VCs, listen up: You guys will have to start taking research,
original and/or old, seriously or risk funding just more social, mobile,
locational, sharing apps or trying to buy tickets on airplanes that have
already left the ground and are not taking on any more passengers.

Entrepreneurs, listen up: Pick a pair from columns A and B, write the
software, get the server up, and go live. The software? Mostly the time is
just 'overhead' that you have already covered as part of your day job. The
server might cost you less than $1000. The bandwidth might cost you less than
$100 a month. Or just put your software in the cloud (at maybe some risk of
your crucial, core research).

That's the opportunity.

