The blue ocean idea for Pavlok[1] (a wearable device habit/behavior trainer) wearable we developed came from a similar thought process. We wrote it up here: https://old.pavlok.com/forgotten
Did you read the forgotten link I posted? The whole point is that we used already discovered concepts, with modern technology, to solve any bad habit. The market was and is wide open.
It's connecting multiple different technologies to attack a wide open market space. Think: smoking, drinking, gambling, oversleeping, getting over your ex, overspending, unhealthy eating, etc.
If that isn't blue ocean strategy then I don't know what is.
I disagree, there is good (well-known) tools for business strategy thinking there. But I agree with you on point that one rarely just comes up with great Blue Ocean Ideas by rigorously applying thinking tools, but they can help one to shape and evaluate ideas that you come with observing and intuition.
(Edit because of downvotes: I was serious, this is a blue ocean idea. For example, imagine children with their own ontology of language and experience (not culturally acquired). That's about as blue ocean as you can get. Neural networks are pretty close. Neural news at nine.)
There are whole fields of anthropology, psychology, etc. debating how much of language and experience is learned versus innate.
As for neural networks, generative models, genetic algorithms, etc., they can help us optimize on known search spaces but that's about all they can do. It's a big leap from designing simulation-optimized bicycle parts to generating truly new ideas.
Other good sources are The Master Algorithm (ML) or The Brain That Changes Itself (neuroscience), the former surveying general capabilities and limitations of various approaches and the later surveying findings from medicine on just how plastic our sensory inputs and interpretive capacities are through recent research.
First of all, Karpathy is a heck of a lot more than the "Tesla AI guy".
Second, he's describing a pretty radical implementation of what amounts to a new form of fuzzy constraint programming using exactly the technique I described: brute-force optimization on a very sophisticated objective function. It's fundamentally statistical, and it's still fundamentally an optimization problem. Neural networks happen to be modular compared to other optimization techniques, and they happen to do a miraculous job at capturing latent higher-order structure in these optimization problems.
The real innovation he's describing is that we now have powerful enough optimizers that it's cheaper, easier, faster, and sometimes more reliable to just fuzzy-optimize problems instead of coding bespoke solutions for them. That's pretty amazing, but suggests absolutely nothing.
Now, a self-modifying neural network might be interesting, that can monitor its own performance and decide when to retrain part of itself, mutate its own architecture, or request new training data. But we definitely are not there yet.
With all that said, there's a much more down-to-earth version of your idea that neural networks can have their own ontology and experiences: that's just an anthropomorphization the latent structure that they capture. This isn't unique to NNs -- it's something researchers have been exploiting for decades. It's only a small stretch to argue that that's what fitting a decision tree or a principal components model is. This is reflected directly in the jargon that these models "learn" a representation of the data. You can extract new representations of reality right now by running HDBSCAN on a data set of your choice. Neural networks just let us scale that up to richer, higher-dimensional problems.
Maybe that's all "experience" really is, consuming data from the world and encoding various representations of that data. But when it comes to generating truly novel ideas, instead of incremental improvements on existing ideas, I'm bearish on how far our current neural network technology can go. Maybe AlphaGo, which apparently invented new Go strategies that no human player had ever thought of, is a counterexample. Or it's the exception that proves the rule. I guess we'll see as computation power continues to improve.
I challenge anyone who's read this to propose a single blue ocean idea here