I think this a role tools like ChatGPT will fill: spit out a crappy version of something that humans can edit later. It gets you over the hump of looking at a blank page, but isn't itself a deliverable.
I plugged the elevator pitch of "my novel" and asked it to write the plot of that novel (I will never write it, I just like to think about the elements of it and have some notes and snippets) and it spat out almost exactly what I had written out as the longer form of the outline.
I laughed, because I knew exactly how original my thinking was going in. It was really funny (and depressing if I had a big ego) how close it was to my idea.
However, doing so allowed me to iterate a bit and I took a minor aspect of the plot (people don't always have enough to eat) and magnified it (the government is the only source of food and they use this to control people). It has made the story much more interesting (still not super original!).
Last week I was experimenting with ChatGPT for some of the sorts of trade pub columns I sometimes write. I wouldn't have given anything it created straight to an editor--though I've seen worse. But it gave me some bulleted paragraphs that I mostly agreed with and could certainly serve as a starting point for revising and fleshing out with more detail, maybe some numbers, quotes, links, etc.
While it can head you off down dead-ends, having some structure and words to start with tends to be easier than a blank sheet of paper.
