The article says that I like to find ideas by “playtesting” then goes on to describe playtesting as a form of user testing.
Whereas there is some definition of the word playtesting that might be applicable to the way I design puzzles, it is not remotely the one in the article. In fact I think too much “playtesting” as described here will make your game boring, and will degrade your skill as a designer, and over time make you a boring designer. It is important to avoid this.
If you playtest too much it means you don’t really know what you are making or don’t have confidence to determine what playing it can be like. Even if you don’t have these skills or this confidence yet, you should be working on building them. Certain high skill levels of design will not be accessible otherwise.
The amount of playtesting we did on The Witness was very small — about 5 or 6 days over the course of 6.5 years, on a game that takes 50-100 hours to play through.
Whereas I do sometimes get puzzle ideas from bottom-up exploration of a space by myself (which is the definition of “playtesting” that would make sense here), in fact the vast majority of my puzzle ideas just spring to mind after I have built a sufficient understanding of the possibility space. So these are more like two distinct phases of design, and you might switch between them when designing one game. But I think most of the actual puzzle ideas have to come at least semi-intentionally, or the game will not feel very strong.
You have to know where you are trying to drive the truck.
I stepped away from all this for awhile now to actually create instead of just doing research and found that many of my initial ideas were flawed. What you mentioned is incredibly insightful and I hope you'd allow me to directly quote you and edit my post so that others who visit will be able to better understand the whole scary but wonderful process of puzzle game design.
Goes without saying, but I am a huge fan of your work and you as a person so I'm incredibly honored by your response. That said, I hope that I have not mistakenly misdirected the audience by putting words into your mouth in the form of advice. Please let me know if there are other portions of this that you find incorrect.
Either way, thank you for all you've done!
A bit offtopic, but could you remove the fixed widths of your #content (.baby-container) at screen width below 767px ? 300 pixels is just too narrow for the content. Maybe set width to auto and set margin to 15px for a better responsive experience? :)
Compare to giant companies like Twitter, where I have heard engineering & product managers say "Every idea has been thought of, it's only a matter of testing them one after another." What an intellectually bankrupt idea.
It is the most experienced product designers who can just look at something and tell you if it is good or bad, before trying it, that big corp product undermines the most.
that's because for every product designer that can tell good from bad, there are 10's of pretenders who dont know what they're talking about.
The stakeholders don't have the time or expertise to vet (otherwise, they'd be product designers themselves!). Thus, they need objective verification. This is what turns into the 'matter of testing the idea'.
But to get beyond that - and getting beyond playtesting, data-collection and other brute force feedback mechanisms entails doing so - one has to systematically devise original feedback signals - pass-fail checklists, writers' rubrics, dependency graphs, workout logs and so on. Once you have the framework in hand and have validated its philosophical premise to some degree, you can feed your work and ideas through it and it automatically corrects you and guides the ideas away from major fallacies.
Of course, some amount of experimental data helps, but the huge amounts we have grown accustomed to have a habit of producing a lot of busywork - a product tuned and tweaked and prodded to improve this or that short-term KPI or audience appeal metric - without striking at the critical issues that one can probe with a bit of followthrough on self-feedback.
Audience maximization is almost literally regression towards the mean to maximize profit.
It implies cutting corners ; but some of these corners are actually making your game stronger, they're giving it its personality.
Think of "Dwarf Fortress", "brogue", or even "Minecraft". What kind of feedback do you expect to get from most players? Better graphics? Real-time play? Would it make the game stronger?
For Dwarf Fortress, I believe most players would want a more understandable user interface and a tutorial, and I honestly believe that would make it a better game.
One general result from designing games is that players tend to be good at knowing what needs improving, but very bad at knowing how to improve it, so it's a good idea to see where they get stuck or frustrated, but then ignore their advice on how to fix it.
The idea you're exploring might be more akin to the Mario Cram School which involved getting staff across all of Nintendo, not just designers, creatively exploring Mario development. But straight up assuming that Nintendo's approach towards design makes it a thumbs-up dismisses notions people have that newer Nintendo games overuse tutorials to the point of infantilising the audience.
Although their games are mass mass market, they still feel opinionated. Their designs have a certain Nintendo-ness to them that's persisted for decades in almost all their games. Even when they partner up with a third party their design shows through.
If they had extensive playtesting early on I would think their uniqueness would be eroded. It obviously hasn't. Most recently Nintendo set the bar for what an open world game should be like with their take. Not just in quality but by applying their method they elevated the genre.
Authors also have editors and 'test readers' for their novels. That doesn't erode uniqueness. Why should this happen for games?
I also don't agree that "Indie" = "better" or "unique". AAA companies, especially Sony and Nintendo have released amazingly creative games. Just a few from Nintendo include Pikman, Animal Crossing, 1-2 Switch (cow milking), Cubivore, and there are several others. Meanwhile 99% of "indie" games are skinned versions of old games who's best examples are often Nintendo's. How many "Metroid"vanias are there? How many Smash Bros clones?
As one more final point, the games made at big companies are made by actual people who poor their creativity into the games they make. Indies don't have a monopoly on creativity. Being AAA != can't be creative nor does it mean can't take risks on new ideas. There are plenty of examples of new ideas from these companies.
No spoilers, but almost every set of 523 main puzzles in The Witness has a twist that requires a full 3D environment in really interesting ways.
The +135 +6 bonus puzzles require a 3D to 2D transition, but only ironically.
Aren't almost all the puzzles 2D graph paper mazes?
But I strongly disagree that the most creative or the best game will ever be mainstream. When is the last time a AAA game came out like Baba Is You? Celeste? Hollow Knight? Those are games I play and replay because they're so enjoyable and memorable. Nintendo makes great games but I don't feel any of them are quite the same experience as the best indie games I've played.
I mean, what does that even mean? If we use the analogy of movies, where the medium as an art form is more established: Pixar movies are creative, but they can still be bland (most of them are anyway). Yes, some Nintendo games pushed games as a medium forward, but I don't remember any Nintendo or Sony game that pushed in the direction Braid or The Witness did. Very few games do actually. "Games" games are more like toys. I enjoy playing with them, but I don't step away from them and think deeply about the experience I had. And I wish more games would strive for that kind of experience.
>Meanwhile 99% of "indie" games are skinned versions of old games who's best examples are often Nintendo's. How many "Metroid"vanias are there? How many Smash Bros clones?
Compare that to what the parent comment said
>but they're not nearly as unique an experience as you'll find from some indie developers.
By a matter of definition, I would expect any creative media to not be bland. Even if every other part of it is boring the exploration of the creative ideas in it should be stimulating.
> I don't remember any Nintendo or Sony game that pushed in the direction Braid or The Witness did
I'm not sure what direction you are referring to. I don't think that either has really influenced gaming as a medium. The closest game I can think of as a comparison for Braid would be Portal and I think it also didn't really push any boundaries for games.
The Witness is still a little young, but I don't think its likely to lead the way for other games either.
On the other hand, I think the original Metroid and Legend of Zelda radically pushed forward what games can do. I think both of them would make you think deeply about the experience of playing them if you played them in their time.
That does not make it worse than other puzzle games, just annoying in parts. Especially its huge reliance on perception as opposed to actually thinking and figuring things out, but then it's not quite a puzzle game. It's more of a big escape room game which does have puzzles in it. There are others on the market, few are well known.
(Even something like Antichamber is more of a puzzle game than Witness, yet it is still an escape room rather than a pure puzzle game.)
A relevant contrasting comparison would be to something like The Talos Principle which was heavily playtested including some puzzle redesigns, reordering, some having been dropped. Puzzles were also automatically checked to be doable in desired and a few atypical ways.
The other design principles apply though the same - minimalism, tutorial principles and difficulty scaling.
Antichamber has the guns and blocks and physics, Talos has the few tools and pseudophysics, Witness has varying puzzle mechanics but one input with tons of output.
The difference is the degree of things being hidden. Puzzles with hidden pieces tend to become adventure games. It does not matter if you have to carry the pieces or if they're already in place.
I don't understand this attitude for a game that is called The Witness.
This right here is a really underrated statement. It's why I think the designers behind WPC-style paper puzzles (Toketa, gmpuzzles, etc.) consistently produce works of higher quality than most levels in most puzzle video games. Working on paper forces you to design with intention rather than design by playing around with objects on a grid.
Intentionality is one of the most important aspects of good puzzle design. Reminds me of a 2001 quote from Nikoli editor Nobuhiko Kanamoto: Computer-generated Sudoku puzzles are lacking a vital ingredient that makes puzzles enjoyable - the sense of communication between solver and author.
Quick question - how do you gain this understanding of the possibility space without bottom-up exploration? Is it a matter of having good influences (and who might those be)? Or some other reason?
That said, you can always be surprised, and it’s delightful when this happens, so you don’t want to get on an egotistical high horse and decide you really know the system without playing it. The actual experience of the actual system is your reality check.
But, the more experience you have with that system, the more you can see where things might go (unless it’s intensely chaotic, thus unplayable).
You are a genius and a god among us. Thank you for making the deepest and most beautiful games I have ever enjoyed.
Your tech talks are also awesome.
For those that like reading more than listening ars summarizes it
HoneyComb hotel being my favorite!
The games have been around for a long time, and seem to be updated for modern platforms. But the website is still a blast from the 90s, and the graphics are joyfully 90s amateurish :)
One of the interesting things he did was invent his own language, and virtual machine for doing his games.... http://www.kaser.com/kint.html
Any others of his that you would recommend?
The controls took a few minutes to get used to, but the included documentation was superb.
(update: Non paywalled version https://pdfs.semanticscholar.org/e401/51cbec7bdcc65b59fff53c...)
Cameron Browne (the author), suggest many way to evaluate your puzzle, and this was used in his further research as a way to automatically generate fun and interesting puzzle games.
Portal 2 is a great brain-teaser if anyone is looking for a constructive use of their time tonight. I recommend the Nyskrte NYS maps if you want a real head challenge.
I didn't like Portal 2's official single player campaign as much as the first Portal, but the two-player coop mode is excellent, and a rare example of multiplayer puzzling.