When you're designing a codec or some image processing pipeline, having a human in the loop is great. Designing codecs to have human-acceptable artefacts is as much art as science. Otherwise you end up like Xerox that had scanners which tended to compress "6" as "8", since they looked close-enough to their compression algorithm[1]
That is also how JPEG XL quality was decided. I viewed every quality affecting change manually and if it didn't pass, it didn't pass - no matter what objective metrics said.
The final stage of their Image Quality QC was always "The Old Men With Good Eyes."
There were some employees that were professional "eyballers." They had full veto power.
I'm not sure if they still do that, but it would not surprise me, if they did.