The secret to Mario games is a new gimmick introduced every level. They've gotten better at this over the years, and Mario 1 + Lost Levels is the worst example of it. That makes it a great comp for GPT-3 which can churn out an endless supply of flavorless brick & pipe levels and still feel vaguely Mario 1. Were this tool to live up to the hype of "indies punching above their weight", it would need to design novel platformer mechanics. The work it's doing isn't the hard part of platformer level design.
For another example of why this isn't commercially viable, look at what happened with Super Mario Maker. In that game _humans_ are given a fixed set of Mario doodads with which to build levels. But Nintendo kept the secret sauce for themselves - the ability to create new doodads. What follows is millions of derivative Mario levels unworthy of their own game. Even if you trained MarioGPT on the rich set of level data available in Mario Maker, you would not have an algorithm that makes commercially viable Mario levels.
> What follows is millions of derivative Mario levels unworthy of their own game.
It doesn't refute your point, but what actually happened was that brilliant tinkerers found ingenious ways to combine the basic tools to create whole new classes of advanced gadgets that enable styles of gameplay not intended by Nintendo. Here's a playlist with some examples and tutorials: https://youtube.com/playlist?list=PLekbcfvMB1gYieKXixxXVBTYC...
AI can be leveraged by an human designer to do that with some effort. Like, humans may have good taste in level design and AI may explore the concrete possibilities.
AI might be able to do this in the future by itself
AI models might be able to do this in the future by themselves, though with current paradigms AI will barely generate copies of existing levels with little creativity. Sure, a composition of existing level pieces could lead to an interesting level design, though it would be more by accident than by design. Models do not maximize player enjoyment, there is no metric for that. Maybe engagement metrics could be used, but I don't think players would stick around long enough playing bad levels to reach a viable model.
New models and paradigms will come up, but until then I'd say anything AI-generated will feel pretty vanilla and somewhat incoherent.
My point isn't that the Super Mario Maker players weren't having fun flexing their game design muscles. My point is that a million makers on a million joycons couldn't generate enough commercially viable content for a single game. So what hope does GPT have? Both situations have similar design constraints, which I'm arguing is missing the critical design component necessary to make commercially viable platformers.
The reason why commercial viability is of interest is because the article claims this tool will be valuable to game developers and I don't think it will be because it doesn't solve for any problems in the business of making games. Nobody is stuck deciding where the pipes and bricks go.
To end on a positive note, lots of open world games use terrain generators as a first pass. AI might have better luck in that domain.
> My point is that a million makers on a million joycons couldn't generate enough commercially viable content for a single game.
How does this criticism follow after seeing a playlist full of creative uses of the limited systems available?
What do you expect, these individual makers using a proprietary tool somehow actually making a commercially viable game out of their levels that they can't even export and are entirely based on the closed source engine powering SMM? That never would have happened because of the nature of the platform, not the content being made.
Nintendo released SMM knowing it wouldn't be a direct threat to the Mario franchise. They were able to guarantee this because they know a Mario game with 80 levels needs 80 things we've never seen before. SMM ships with 200 things we've definitely seen before.
It's possible that some combination of those things is new, and good for a level worth of content. But there isn't 80 of them. The playlist has stuff like invisible pipes, lag spike inducers, soft lock strategies, etc. This style of troll design is popular(?) within the SMM community but you wouldn't sell a million copies of it in its own game.
> Nintendo released SMM knowing it wouldn't be a direct threat to the Mario franchise.
Sure, that would certainly be a good pro albeit hardly the sole reason as you're making it out to be.
> They were able to guarantee this because they know a Mario game with 80 levels needs 80 things we've never seen before. SMM ships with 200 things we've definitely seen before.
That just plain does not follow. You can do millions of things with those 200 things, and combinations of those 200 things have certainly never been seen in a Mario game.
The playlist has 117 videos. This only scratches the surface of what is possible in SMM.
They are insanely challenging and certainly introduce new mechanics, which is your entire premise, that levels need gimmicks. There are levels that literally take hours due to how complex and new they are. They do things that Nintendo never intended you to do and could easily fill multiple games' worth.
Have you actually tried it out? Played some of these creative levels? Because everything you say makes me believe you have never even opened SMM let alone looked at the levels out there. There are literally tens of thousands of videos showing off thousands of new mechanics built on top of those standard widgets, in tens of thousands of hard levels. Your description of what is out there is just, plain and simple, not reality.
Just because it's not physically possible to make a commercially viable game due to the nature of the platform does not mean that there isn't enough content (and then some) out there made by a million joycons. If a person can make it, an AI can eventually figure it out.
I've played it and also watch it often on Twitch. You can go hours without seeing a decent level. What the SMM community thinks is playable content is very far off what commercially viable content would be. Shell bounces, abusing physics with bouncy blocks, leaps of faith, "be small Mario on purpose"; are all unshippable content, fun only to folks who have accepted the constraints of SMM.
The latest Mario lets you throw your hat to control other characters and there are dozens of mechanics born from that interaction alone, each one unlike the other. SMM has no hope of ever competing with that.
Comparing a 2D platforme to a 3D one is like apples and oranges.
They serve entirely different purposes and have different constraints, the biggest one being an entire dimension. Odyssey's mechanic would be extremely dull in a 2D platformer.
Your opinion is also just that, an opinion. Of one person. Just because you don't find it entertaining or think it's worthy of a game doesn't make it fact.
It being relatively popular on Twitch, enough for you even to watch it, is testament in itself that there is clearly interest in the levels that are being created and playing them, which in itself proves the viability of the levels. If they were anywhere near as dull, repetitive and lacking due to the constraints of SMM as you claim them to be, people wouldn't be streaming or watching them.
The 2D/3D aspect is so irrelevant. I could have chosen New Super Mario Bros U and talked about riding on the back of dragons, micro mushrooms, ice flowers. SMM2 could add these mechanics but they by definition aren't novel and combining them will not generate novel content likely to be in the next ~two dimensional~~ Mario game. That game will have a bunch of stuff never seen in any game before.
SMM is not a flop and I never claimed it is. But if the levels are so good, why doesn't anyone sell them and make Mario brother money? Because SMM levels are the 2nd slice of cake.
I mean your entire premise is that some flashy new mechanic is what's holding SMM back, and you chose a shit example and I called you out on it. That's hardly irrelevant. Inconvenient for you, maybe. Next time, don't choose such a bad example as your only point. You've done it twice already in this thread, first with SMM itself.
> But if the levels are so good, why doesn't anyone sell them and make Mario brother money? Because SMM levels are the 2nd slice of cake.
No, it's literally because there is absolutely no platform for exporting or selling the levels. They are levels based entirely on the engine of SMM. They can only be played within SMM. Even if you were to take the elements and reproduce them in another engine, it won't work the same because it's based on the timings of SMM.
I already said as much. There not being a platform to sell these things does not mean they don't have value. If it were possible, the best levels would most certainly be good enough for a full game.
Just because there's no riding on the back of a dinosaur does not mean the combination of mechanics in new ways isn't itself a novel mechanic. You're discarding it because it isn't some flashy first party thing. People have done crazy things within the confines of SMM and throwing it all away because Mario isn't throwing his hat onto an enemy or having a new power up available is absurd.
You will just have to move on and find someone else on the internet to argue with.
I think you are not trying very hard to understand my points because they aren't that complicated and you continually misconstrue them. You quicky destroy the nuance of the conversation and then argue with your absurd black & white interpretations. Any attempt to clarify is met with increasing hostility.
>My point is that a million makers on a million joycons couldn't generate enough commercially viable content for a single game. So what hope does GPT have?
Your criticism is that the AI doesn't create new game functionality, even though it doesn't have access to create new game functionality?
That's an artificially impossible bar you're setting for the AI. Maybe if it did have access to create new functionality it would be able to?
That's exactly my point. The path to new game content can't be reduced to putting the blocks in the right place. You also have to come up with new mechanics out of thin air, consider the educational burden of your mechanics, the emotional tempo, how the level plays for different player types (speedsters, young players), how the mechanics reinforce the theme of the zone you're in, and more.
What I am pushing back against is the idea that since GPT can assemble blocks, that it's somehow approaching game design.
> My point is that a million makers on a million joycons couldn't generate enough commercially viable content for a single game.
The toolset is limited, so you end up with Mario levels of LittleBigPlanet.
If you provide a fuller toolset (like UnrealEd or the ability to mod), then you absolutely have viable content, enough for (in the case of CS) the original publishers of the base game acquiring your commercially viable content.
Among the many things I've wished I had time to do, I've wanted to sit down and build a procedural level generator in which the 'tricks' you refer to are first-class citizens, meaning both that they could be incorporated into the levels consistently by the generator and that we could control their introduction into the player's vocabulary consistently.
This approach could be a viable approach to that, but it may need some tuning. It is possible that the problem in this case is less GPT and more the training set; the examples given imply that the levels were characterized as a whole by some very superficial criteria, so it isn't necessarily a surprise that the resulting levels are equally superficial. The system was never trained on "shell jump" (not that that appears in Mario 1 AFAIK, it's just the first Mario term that came to mind), so it never produces them. I would want to look at training on a screen-by-screen basis, with some overlap, rather than levels, and more richly categorizing the input data.
If I were designing a new Indie game, I'd be feeding it some hand-crafted level snippets. However, in terms of getting it out, it would be hard to know whether I can feed the GPT system enough input with enough categorizations to know whether it would just be more cost-effective to design the levels directly. At the moment it is not obvious to me how to convince GPT to understand the concept of level flow, or even something as simply as "this pipe is physically impossible to jump over".
It is also possible there just isn't enough input data to really make this slick. There aren't that many publicly-available Mario levels.
While obviously it'd be more freedom to be allowed to do just whatever you want, artistically it helps to have some sort of constraint. "I can do anything" is a bit... vague. OK, but what should I do? If you give me six specific lego bricks I'm constrained, but immediately I have ideas, and if you think "Six lego bricks means maximum 6 factorial ideas" you're badly mistaken.
If you train MarioGPT on MM2 levels, the reason you don't get "commercially viable Mario levels" is that's not what the community ever wanted to build, it's like training a model on abstract portraiture and then complaining this doesn't produce saleable landscape paintings. Mario Maker has multiple communities, let's look at two of them, in both cases they are not "commercially viable" for whatever that's worth.
Kaizo. Kaizo means roughly "re-arrange" in Japanese but eventually Kaizo Mario is a style in which tremendous skill is needed to navigate the course. Basic Kaizo techniques include the "Shell jump" in which Mario throws a shell, it bounces off a wall or other surface, and Mario jumps off the shell he threw. Mario can of course arrange to throw, jump off, and catch shells more than once, and he can cause Yoshi to swallow and then spit out a shell, jump off that shell, and catch it. Good Kaizo players think nothing of a multi shell jump to climb a wall, they'll assume that if there's a shell and a wall that's what is intended.
Kaizo Mario is far too difficult to be commercially successful. Most people could learn, if they're got good hand-eye co-ordination, but it's not easy and most people would only ever be passably good at it, so that hard Kaizo levels might be impossible either because they didn't figure out the technique or because their skills are inadequate, very frustrating.
"Chocolate" Kaizo (which is Kaizo where you also change the game's rules) isn't possible with Mario Maker, but even if an AI were able to make the best Chocolate Kaizo levels, they're not commercial, the best Chocolate Kaizo today is probably something like "Grand Pooh World 2" but there are maybe a few hundred people in the world who have fun playing something like that, so where's the money?
OK, next community, Troll. Troll Mario subverts the assumptions about the central concept of Mario. The idea is to surprise and perhaps frustrate the player, unlike Kaizo great skill is not mandatory, but patience is, and you need to be able to accept that you were wrong and learn from mistakes which many people struggle to do. A Troll level might present Mario with two apparent routes forward, a mushroom power up with a door, or a fire flower and a pipe. Except nope, those are both instant death, the correct solution is to jump into the obviously deadly pit, it wasn't really deadly and Mario gets a different mushroom then is pushed into a one-shot teleport.
A common Troll trope is the "anti-softlock" complete with use of the "Slide theme" music. Nintendo's levels are designed so that either Mario can win or you will be put out of your misery quickly to try again. Where it's possible to instead get stuck, unable to die, that's called a "Soft lock" - as opposed to a hard lock where the game just freezes. The anti-softlock then is the art of a Troll level making it possible but very difficult to die, even though Mario can't win. Fashion changes, sometimes it's popular to have actual softlocks, sometimes fake ones, where Mario will die after say 15 seconds somehow, but often especially later in a course, you have complex puzzles in which the only benefit of the solution is Mario dies and you can start over from the checkpoint you reached.
This is neat and all, but procedural 2D level generation can be done really well just with simple heuristics, see Spelunky from 2008. And that can be built into the game and computed efficiently on the fly, not requiring an internet connection.
I assume this is why Angry Birds 2 is such a terrible game compared to previous incarnations. All the levels get a base then a random generation of enemies. I wonder if it's actually possible to beat all incarnations, as many times I will get to a level, lose all my hearts a few hours or days in a row then easily complete the level without really learning any tricks or strategy.
It has served as a good example to teach kids in my life about the scam of digital artificial scarcity employed by the game by making you wait for hearts or pay.
Came here to post the same thing. Procedural level generation has been there for a while and it does not need advanced AI. Probably if the same thing was done for games like Call of Duty or Medal of Honour type games, it would be more impressive.
Agreed, my argument is only on "sufficiently advanced". Mario is super simple to create a AI for this purpose. And the levels look very unhuman generated. :)
A lot of roguelikes (e.g. Dcss) do this as well. I'd be interested to see a game with truly random procedurally generated levels, though the levels may end up being repetitive and mundane.
I don’t believe that’s true. DCSS is mostly true procedural generation from a seed against constraints. There however some special fixed layouts for final floors, and small predefined vault rooms.
I've played an absolute fuckton of DCSS and it is true. There are small "blocks" that you'll see over and over that are exactly the same. The best example I can give is the room Crazy Yiuf hangs out in with the tree hallway leading to it.
Yes, you do. Each piece has randomness within the piece too, so even then it's not all the same.
Being able to learn how the level generation works as a player is part of the experience of playing a roguelike game, so I don't think that's a bad thing though! Games with too much randomness and not enough structure can feel a bit samey
Absolutely, but I can imagine an AI like ChatGPT, which is able to write stories that at least feel "creative"—might be able to generate levels that feel hand-crafted but are in fact entirely original.
While it does arrange prebuilt blocks it also does more, it will punch holes in the walls as needed and does procedural population of what's in each block. The second one takes things further.
Effectively, yes. Although a lot of additional things are also randomized like item/enemy spawns.
Then for Spelunky 2 there's the randomizer mod which randomizes almost everything. It pretty much never ceases to surprise you. Look up spelunky 2 randomizer on Youtube to see for yourself.
The first game I played and was aware that it was procedurally generated, though I would not have known that term as a child, was the first Diablo, and a quick search shows that it was already a well-established concept by then, going back to the late 70's/early 80's.
I'm surprised there haven't been more games like Cloudberry Kingdom. It has fantastic level generation with a bunch of settings to play with. Players can even have various movement abilities which the level generation considers. If you got stuck it provided an AI to follow.
I remember having this one on the Wii U... Each level wasn't as much a level but more a short "trick" the player had to complete. It seemed to choose a path first, then place obstacles just narrowly avoiding the chosen path.
Hey! One of the authors of the paper here. Actually, many of our generated levels are playable because our model actually predicts the path of a search algorithm (A* agent) that was able to solve a ton of levels. MarioGPT is not always perfect though, as it sometimes predicts impossible jumps lol
It's interesting because this is using GPT-2 (https://huggingface.co/distilgpt2 specifically) which you can just fine on a reasonable GPU.
But I'm not convinced the results are any smarter than a randomized procedural generation (I'm sure using it for text generation instead will yield sub-par results).
For another example of why this isn't commercially viable, look at what happened with Super Mario Maker. In that game _humans_ are given a fixed set of Mario doodads with which to build levels. But Nintendo kept the secret sauce for themselves - the ability to create new doodads. What follows is millions of derivative Mario levels unworthy of their own game. Even if you trained MarioGPT on the rich set of level data available in Mario Maker, you would not have an algorithm that makes commercially viable Mario levels.