Super Mario 3D World's architecture goes back to Super Mario Sunshine. Some parts go back all the way to Super Mario 64, but not the object / actor management. The ring buffer isn't really emulating OAM, either.
You can trace the evolution of "LiveActor" all the way through until it ends up in Super Mario Odyssey.
This architecture was so successful it ended up as the basis for all new Nintendo game development, so Breath of the Wild, Pikmin 3 and Splatoon, Mario Maker all use this new "Actor Library", or "al".
I have not looked at NSMBU, but NSMBWii uses a different core structure originally developed (as far as I know) by the Zelda team. I think it's mostly phased out these days, as is the set of "egg" libraries developed by the Mario Kart: Wii and Wii Sports teams.
> The ring buffer isn't really emulating OAM, either.
I mean, you're right, it's not a literal implementation of OAM in the sense of controlling the same things OAM controls. I was speaking kinda metaphorically.
NES/SNES OAM was useful for reading back entity physics data (because it gave objects X/Y position registers) which meant that developers (incl. Nintendo themselves) often chose to rely on the OAM-object "components" of a entity as the canonical handle for tracking the entity in the game physics (Rather than having a table somewhere in work-RAM of separate "physical" components for entities.) Games like SMW literally just index a table of actor behaviors off the OAM-object's name-table data; what an entity "is" from the game's perspective, is determined by what it currently looks like!
Since the OAM had a finite size, this reliance on OAM for tracking entities forced games into a structure where entities' lifetimes are coupled to the lifetime of their OAM-object representations. Which meant that every NES/SNES game relying on OAM to track entities needed an algorithm for dynamically allocating OAM-object slots to entities; and so, for evicting entities if OAM was exhausted. (Level design was done with a hard eye for avoiding OAM "thrashing" by keeping entities spaced apart, but the system still needed to be able to handle the case where mobile entities ended up following you and piling up.) Which brought into existence the common OAM LRU cache-eviction algorithm—i.e., the practice of "despawning" the oldest off-screen entities when new on-screen entities need OAM slots.
This determined a lot about the design of these NES/SNES games. It made mobs in these games into things that would lose their state whenever they were scrolled "far enough" off the screen; which in turn forced a design where—rather than a level just running a "start script" that would spawn entities at initial positions, tracking them in RAM from then on—you instead had adopt a hybrid approach where entities had both an OAM-object representation, and also an associated "spawner" (usually existing just as static level-data in ROM, though sometimes coupled to a bitflag tracking destroyed spawns) that would trigger [re]spawning for the entity.
SM64 is essentially "emulating OAM" in the sense that it assigns entities handles in a fixed-sized buffer, and then uses a very OAM-like logic (basically, "memory pressure" on this buffer) to decide when entities should be de-spawned; and then uses spawners to recreate entities that have been de-spawned due to this memory pressure (meaning that most entities don't "exist" until you get close enough to them.)
SM64 didn't need to do things this way; the N64 has enough RAM to track all the entities in every SM64 map at once, IIRC. They chose to impose this constraint artificially, in order to continue to build SM64 levels according to the design philosophy they had "discovered" due to the original constraints of the OAM system.
Later games in the Mario series, if-and-when they choose to have this de-spawn/re-spawn tracking feature†, are essentially "pretending to have OAM", but not really emulating it the way SM64 does. For example, Mario Maker de-spawns entities when they're scrolled sufficiently far off the screen, in a way that mimics OAM sufficiently well that re-spawning and enemy spawner semantics still work—but which isn't really an OAM-like system, in that there's no static buffer with memory-pressure causing de-spawning (and in fact, as long as the entities are willing to squeeze into one visual screen, existing entities will never be forced to de-spawn.)
† You could get a very interesting analysis of the way Nintendo probably internally divides/project-manages the Mario games, by just determining which titles "emulate" OAM the way SM64 does; which titles loosely mimic OAM, like Mario Maker; and which titles don't even bother with de-spawn/re-spawn tracking at all, but instead have persistent physical entities that just "go quiescent" when they're out of sight. (IIRC there's no Mario title that uses the fourth option—pure view-frustum culling of distant models that continue to "tick" while culled.)
> developers (incl. Nintendo themselves) often chose to rely on the OAM-object "components" of a entity as the canonical handle for tracking the entity in the game physics
I don't know how the SNES worked, but AFAIK most NES games did not track objects in this way. Instead, the game engine maintained its own buffers containing object state and copied necessary information to OAM every frame.
OAM only stored graphics state for the rendering hardware, which is not a convenient form for the game engine for a number of reasons. For instance, objects are nearly always composed of several OAM sprites placed next to each other, objects that are not visible during a given frame are not present in OAM, and a single animated object can switch between so many different graphical forms that it would be complicated to identify which object corresponds to a graphics tile from OAM. Additionally, OAM doesn't have extra room for non-graphical object state (like behavior timers or velocity information).
Semi-offtopic, but you've clearly spent a lot of time studying Nintendo's code, from a range of eras... I'd be curious to hear, if you had to make a very broad assessment, how would you rate the quality of Nintendo's programming?
Nintendo is quite clearly second-to-none on the design/creative end, how much does that translate to the technical aspect of game development? Speaking purely in terms of software.
I find this particularly interesting in the context of a company that appears to retain many of the same programmers today as they did 30 years ago, when software development was obviously much different.
You can trace the evolution of "LiveActor" all the way through until it ends up in Super Mario Odyssey.
Sunshine - https://github.com/shibbo/Corona/blob/master/include/actor/T...
Galaxy 1 - https://github.com/shibbo/Petari/blob/master/include/Actor/L...
Odyssey - https://github.com/shibbo/OdysseyReversed/blob/master/includ...
This architecture was so successful it ended up as the basis for all new Nintendo game development, so Breath of the Wild, Pikmin 3 and Splatoon, Mario Maker all use this new "Actor Library", or "al".
I have not looked at NSMBU, but NSMBWii uses a different core structure originally developed (as far as I know) by the Zelda team. I think it's mostly phased out these days, as is the set of "egg" libraries developed by the Mario Kart: Wii and Wii Sports teams.