I remember before OpenAI no big company wanted to host a chatbot or provide a service around them after people made the all the bots say few racist lines. All the customer service bots were basic matching and answering questions from fixed database. Even if you even see 1-2 year back, companies were so focused on refusals, early Google's and Anthropic's model refused lot more things than now.
Now it's the same with image generation or voice model. They refuse everything but it takes one player to decide the market is worth capturing and everyone else will follow the money. As long as people can't generate nudes of any given person and there is a toggle(on by default) to block accidental nudes, I would say it's fine.
I don't think most major players will want to get into the pornography game. It's far, far too fraught with ethical concerns to make it worth it. Musk has no such concerns, though. And I believe that also extends to revenge porn, which will be one of the biggest use cases for this tech at scale.
There are different stages though. Just because revenge porn exists doesn't mean all sexual output is unethical. If a company proves that generating animated hentai is say worth $10b in revenue, I am sure others will follow suit in dialling down the restrictions slowly.
And AI just reduces the budget and time, it can't output anything other that what could be created by say Adobe suite.
Now it's the same with image generation or voice model. They refuse everything but it takes one player to decide the market is worth capturing and everyone else will follow the money. As long as people can't generate nudes of any given person and there is a toggle(on by default) to block accidental nudes, I would say it's fine.