If you find a novel problem, there are two possibilities. The first is that people have been living without a solution to that problem for 100,000 years and never noticed. It doesn't matter how life changing your solution might be, you will need to convince people that their lives need this particular form of improving. The other possibility is that people are well aware of the problem and everyone before you that has tried to solve it has failed. You will need to convince people with money and talent that you can succeed at this herculean feat, and then you actually need to do so.
On the other hand, if you find a problem that people are already spending money to deal with, and you can offer a solution that's better (faster, cheaper, more flexible, more convenient, etc.), then you're in the money. You don't need to be particularly brilliant or creative, just bold enough to try something different.
There will be a few that go for the exceptional route, but the generic startup is going to go for the generic product, for exactly the reason that it became a generic product in the first place.
There are at least two other possibilities. First, the problem may only have started to exist (or to become a real problem) in the relatively recent past. Carbon capture technology is a good example - 1000 years ago we didn't need to capture carbon. Maybe 100 years ago it would've been a good thing to get started on, but we didn't know the underlying problem was a problem yet.
Second, the problem may have existed for some period of time, but only recently has the technology developed to solve it. I'm sure when people had horses and buggies, some of them thought that it would be nice if they could just tell the horses where to go and then go to sleep in the back. That wasn't possible, and it similarly wasn't possible for early cars to have any kind of self-driving technology. Now we're at the point in terms of hardware and software that it at least appears to be a tractable problem.
Your first option is part of my first option - the problem developing recently is a possible explanation for why people lived without a solution in the past, but you still need to convince people its now a problem they need to spend their money on solving. As we can see specifically with handling climate change, that's easier said than done.
Your second option is part of my second option - some new technological development may allow you to do what no one has done before, but you must still convince people that you have this capability. As we see with self driving cars, many have claimed that hardware had reached the point it was achievable only to realize the problem is harder than they thought.
Did Otto Benz or Henry Ford need to 'convince' any horse and buggy owners or riders?
I can't find evidence that they spent any more than a small effort, less than a quarter of a year of full time marketing, before they got serious traction.
They simply built and offered a better and compelling enough product that little to no convincing was needed.
It often comes down to risk aversion and the pursuit of proven business models for investors. However, there are startups out there focusing on novel problems, but they might face more challenges in securing funding. To enhance your resume, showcasing how your outsourcing project for a multinational corporation tackles unique challenges or innovates within their industry could set you apart and demonstrate your ability to address novel problems effectively.
I could say that not everybody is enough of a problem-solving leader, but realistically it's almost nobody who is enough of a problem-solving leader.
And those that are, they have been systematically blocked by an institutional structure overwhelmingly built & dominated by those having much less leadership ability, with most of the upper echelon having merely prevailed during occasions of good fortune through reactive financial manipulation as opposed to gifted visionary problem-solving.
Seems like it's now a majority around the world who are settling for a declining caliber of leader, lower than would have been acceptable when inherent greatness was more often a characteristic of top decision-makers.
Those hollow suits have really proliferated in recent decades, odds are getting worse that those who aspire strongest to organizational leadership are usually the least suitable for the job. Especially when it comes to problem-solving of any kind, much less groundbreaking progress like was seen so much more in the past.
And you would like to see novel problems being solved?
Major problems? Lol.
The system would have to be reversed dramatically just to overcome some well-recognized nominal snafus.
One issue is the lack of domain expertise. To really understand a problem you need to be a domain expert. Young people, who are a large part of the tech world, don't really have a lot of that.
You can also learn what problems are worth solving by talking to people (ie: customers). How many customers have you talked to today?
That's why so many startup books talk about customer focus, customer acquisition, erc. Someone needs to buy it. Who is that someone?
Most things in general don’t do novel things. Its easier in terms of investing and time spent thinking to merely iterate on an idea thats already proved itself in the marketplace.
In the UK, the situation is more extreme, where every so-called 'startup' I find is either working in the payments industry (... truly a new furrow to plough /s) or the coin industry.
It has always been like that and people still sought to solve their problems. The problem with bigtech is that it pays too good, so people are incentivized to do nothing
Conceptual impedance matching is the biggest problem. To use a personal example, I've got this silly idea, the BitGrid, which might (but probably not) prove to be the optimum way to get petaflop computing to the masses. It's based on some obviously bad design choices (when viewed from the FPGA world, for example), so that someone familiar with that world would quickly come to the conclusion that it's just not worth consideration.
Programming the thing requires a completely new set of tools, as there's no Von Neuman architecture to be found. It breaks my brain a little bit, every time I work on it. Completely writing new software isn't something we've had to do since the 1960s. We've always been able to bootstrap off of old ideas, and old code. Effectively, none of the tools normally used to develop software help.
The "impedance mismatch" of this thing is just too big for anyone to even think about making a startup around it.
Something that was a small improvement on FPGAs would be far easier to sell, but there's not much investment money in hardware, let alone ASICs.
A similar example would be the Jet Engine... nobody thought it could work, and it required a completely new way of doing things. Thus, nobody wanted to try to develop it.
Big changes require stable supplies of long term research and development funding. That's something the market just isn't equipped to handle, especially when driven to meet the quarter's numbers.
Many novel problems that would be worthy to solve don’t have a clear path to monetization. And novel problems are inherent risky since no one has implemented a successful solution before.
I mean, what do you expect them to be working on, Nuclear Fusion? "Novel problems" make up the tiny minority of all work. The really hard problems are either already solved or lie beyond the reach of the average javascripter. And there's more, easier money to be made in the banal.
I guess that the difficult part when trying something genuinely novel is finding funding. If you can't point at competitors, because there aren't any that proves that either you have a brilliant idea and you're the first. Or it's complete rubbish and there simply isn't a market.
If you want to have a focus on solving novel problems then the business world isn't the best place in general as companies are too growth oriented to take the necessary risks. I would investigate the sources of the research papers startups base their products on.
Because a somewhat novel solution to a well-understood problem is easier to find a market for (both for the product, and for the company, which has to sell itself to VC's.)
The problem is that a purely innovative technological change is usually tiny and incremental. Most existing technology is more than good enough for most people most of the time.
The really important problems that hold back progress, or oppress people's lives, cannot be solved by a quick-fix startup.
There is bureaucratic bloat, regulatory capture, crony oligopolies, over-regulation, vested interests, rent seekers, oppression by crisis, cover-your-ass management, negative sum legal action, nimbyism, planning quagmires, hyperactive enviromental zealots, bad zoning laws, silent majorities, corruption, luddite unions, private equity rape & pillage, dysfunctional politics at every level, propaganda, people that fervently believe propaganda, arrogant elites, useful idiots, incompetence as a badge of honor, capture of education/media, unaccountable globalism, sociopaths in power, aspirational victimhood, hypocracy, too much debt, economic obfuscation, government intervention, lack of common sense, too much individuality, lack of community, family breakdown, sustained irrationality, consumerism, distortion of the property markey, printing money, government setting interest rates, too many laws, too much surveillance, too many taxes, executive orders, two-party systems, deep state, military-industrial complex, nanny state, poverty trap, atomization of society, drug addiction, opiate deaths, over-incarceration, prison-industrial complex, war on drugs, espionage, excessive copyright protection, intellectual propety theft, nepotism, extrovert dominance, urban decay, litter, noise pollution, light pollution, modern architecture, processed food, particulate pollution, crumbling infrastructure, advertising billboards, did I mention too much debt? ...
... profit over truth, efficiency over beauty, feelings over facts, posture over competence, cheap talk over real commitment, do-what-I-say over watch-what-I-do, abstraction over practicality, focus groups over philosophical coherence, partisanship over leadership, and championing failure over success.
More fundamentally, there is the doom loop of democracies voting themselves more printed money for social welfare, while distracted by hedonistic bread and voyeuristic digital circuses.
I'd say it usually comes from the difficulty in hiring. Extremely difficult to convince someone to leave FAANG / quant firms to work in a no name startup with very high chance of failures, and therefore usually you need a good GTM strategy first (which often requires you have acquired customers / revenue). Often in startups, the priority is figuring GTM strategy > acquiring customers > ... > hiring.
What novel problem would you want startup founders to solve? Are those problems solvable with current technologies? Is there a distribution model that'll work for that solution? Will the unit economics work? How big is the market for this novel problem? Is it worth spending time and money on this problem instead of a "generic product" that seems to be making more money?
Some academic startups do come up with novel ideas, but the novelty typically lies in the technology rather than the marketing standpoint. For example, Anyscale offers a novel technology, but from a business perspective, it is a generic scalable training framework.
Technical companies tend to be more modestly valued than consumer oriented companies.
Why? Maybe part of it is because of the inflation of tech stocks. Maybe because if you are providing a new service or product for a highly technical market, the total possible market capture and subsequent payout for venture capitalists is much more easy to calculate. Maybe because scale is limited. Maybe because the pace of adoption will be slower.
Most founders of startups only want money. They do not care if they make other people's lives better or worse.
This started with ethics laws. Ethics only means following the law and making money.
Seeking the greatest good for the greatest number was aborted when ethics became mandatory.
Ethics literally turns people into sociopaths. They are legally prevented from considering the welfare of others if corporate profits are on the line.
I am NOT an ethical person. I am a moral person. I work for the greatest good for the greatest number.
It is not a lack of diversity. A homogenous group without women or older workers is still capable of trying to improve the world for everyone.
Having said that, cutting costs is a moral thing to do. That eventually improves everyone's standard of living. Capitalism is better at cutting costs than it is at anything else, and capitalism has pulled more billions out of poverty than any other economic system ever.
It is perfectly moral to disrupt an industry by introducing a better product at a lower price and driving the incumbents to innovate or go out of business.
On the other hand, if you find a problem that people are already spending money to deal with, and you can offer a solution that's better (faster, cheaper, more flexible, more convenient, etc.), then you're in the money. You don't need to be particularly brilliant or creative, just bold enough to try something different.
There will be a few that go for the exceptional route, but the generic startup is going to go for the generic product, for exactly the reason that it became a generic product in the first place.