It's a bit like modern pop music. We are well beyond the point where software can not only write the music, but even take the once human voice and recreate it to sing the song. Of course it's not great, if one were to really listen - but it is passable, and it makes money. So it drowns out the remaining bits of real human creativity.
Add some Grover-like text for the lyrics (https://grover.allenai.org/), and you're most of the way there. Then use a deepfake singer ( https://youtu.be/k3qJOPrJOQ4 ), and you're most of the way to done.
Depending on the audience, particularly their level of attention or focus on the music, you can probably get away with playing a completely un-human pop song that people will like.
I find the possibilities here very interesting, but I also greatly fear that this will become the norm the same way that nearly every modern song uses autotune.