Weavers weren't concerned about "passion" for cloth, they were concerned about their economic ability to compete with machines that required huge capital outlays and centralized profit at the expense of the middle and lower class workers.
Sounds like we're repeating history then. If it were just about passion then artists wouldn't worry about AI because it doesn't deprive them of their hobby. It's only a problem because it takes away from their income.