One thing that we did with distributing certain copyright-protected textual material was to scramble them at the paragraph level.
If you take every paragraph in the Harry Potter saga and sort the paragraphs in alphabetical order, it's just as good for training short-context-window models, but not a "harm to the market" leading to a lost sale for anyone who wants to read the books.
The resulting model doesn't have access to the information about what follows what, so it can recreate paragraphs but can't recreate their proper order for a chapter or book. Well, it can try to guess..
Totally get that but the law doesn't care about that as I understand it. For the four-fold test it matters whether the use is going to affect the market for the original work (not the technicalities of how the model works). If people generate pseudo-Harry Potter via a model that was trained on Harry Potter then the court may well decide that the market for real Harry Potter is affected. That doesn't seem an unreasonable conclusion to me.
I'm pretty sure that's what the lawyers will argue in the Silverman case for example. It's going to be interesting to see how the courts decide.
If you take every paragraph in the Harry Potter saga and sort the paragraphs in alphabetical order, it's just as good for training short-context-window models, but not a "harm to the market" leading to a lost sale for anyone who wants to read the books.