Of course it's awesome to work on those problems. The point I was trying to make is that in this case the company has to choose between innovating for the long term, and solving urgent problems that mean lots of $ in the short term. Internal venture projects tend to be put on the back burner when your best engineers could save 50M by optimizing the hell out of your backend.
I'm continually reminded of Neal Stephensons remark "I saw the best minds of my generation... writing spam filters." He talks a lot lately about how we've stopped trying to solve the really hard problems and instead are continually trying to sell digital "sugar water."
Truly defeating spam in all its guises (SEO, commercials, daily offers, junk mail) is a pretty difficult problem. How do you recognize cleverly concealed data? Spammers constantly adapt to spam filters and new/pristine communications channels which catch on precisely because they are relatively spam free. Of course, spam filters also evolve, this kind of runaway adversarial co-evolution is what leads to systems which can effectively lie/manipulate perception, or detect such.
1. Needing much effort or skill to accomplish, deal with, or understand.
2. Characterized by or causing hardships or problems.
I don't see lack of spaceships causing hardship, so that's really more of a definition #1 type of thing. Society built spaceships in the 60's, contrast that against the effectiveness of a spam filtering system built with hardware, software, algorithms and statistical inference techniques available in the 1960's.
While I appreciate and admire the work put into into accomplishing space fight, the existence of manned spaceships really doesn't contribute much to my existence, whereas spam sucks away a bit of my life, every day.
OK how about I rephrase: I meant "OMG-big-deal-game-changing problems." Difficult is just a proxy for that.
To Stephensons point, which he makes in the video in the link, he really means dreaming up and solving big problems. The example he gave was building the worlds tallest building. I'm paraphrasing him here: <paraphrase> Typically accomplishing the goal of building the worlds tallest building is measured only in contrast to the current record holder. So when people aim to set this record, it's only incrementally bigger. If you actually do the research you'll find that it's possible to build a building 20x taller than the tallest existing building. Why aren't we thinking on that scale?</paraphrase>
How would you get in and out of the upper (or even middle) floors of a 3000 story building? Atmospherics would be a serious concern at 10 miles up. Sounds like the kind of project the North Koreans would start and then abandon when they ran out of money.
In the '60s, the best and the brightest were being lured by NASA, to put a man on the Moon. 50 years later, the best and the brightest are being lured by the Silicon Valley, to sell ads on social media. Biggest anticlimax ever.
That might sound bad, but selling adds on social media is actually more useful for society as a hole and takes a lower percent of our GDP than the moon missions. As to cool tech, we are actually discovering earth sized planets orbiting other stars, that's IMO far more interesting that having people play golf on the moon and a lot harder. We are also getting ever closer to building things at the atomic scale. As well as consumer robots be they roomba's or self driving cars.