This looks absolutely fantastic. If you're interested in the technical details currently available, check out the about page[0] on the official MoonRay site, or this presentation from SIGGRAPH 2017[1]. I'm particularly excited for the HeatMap render pass, seems like a really practical way to identify what's bogging your scene down. Not to mention the gorgeous volumetric rendering this engine is capable of (see How To Train Your Dragon: The Hidden Realm[2]).
Really impressed that Dreamworks is making this move. They've made some of my favorite films of all time (the Dragons franchise genuinely changed my life and is a major part of why I love filmmaking, film scores, and 3D rendering) and I'm glad to see they're doing this. I can't wait to try MoonRay in Blender once a community integration exists!
I suspect that every production renderer must eventually have something like a heat map.
Back when I worked at Pixar on the RenderMan team, I was the one who implemented the cpuTime and sampleCount AOVs (i.e., passes) [0] in RenderMan RIS as well as some of the other diagnostic AOVs. And SPI's Arnold also acquired a pair of equivalent AOVs at some point [1]. The cpuTime AOV wasn't exactly novel then either, as I recall; I think there'd been something like it back in the old REYES days.
I'd also contributed a bit to the render time heatmap and added the sample count heatmap in the XML stats files [2]. Those are more thumbnail resolution than the AOVs, but on the other hand they were always-on for all generated stats files. The also displayed with nice little histograms.
One of the things that I learned in my years at Pixar was that rich diagnostics, statistics, or metrics are one of those key things that separates production renderers from other renderers. Pipeline folks, render wranglers, and TDs thrive on them.
Back in the day, I was head developer of Rendition, a "real time" preview renderer that was compatible with Mental Ray in Maya. We implemented a heat map, and it was often shocking how much a seemingly innocuous aspect of a material could impact ray counts. Of course this was pre-pathtracing, so it showed up differently.
> We implemented a heat map, and it was often shocking how much a seemingly innocuous aspect of a material could impact ray counts.
Definitely. I think that "you don't know until you profile" applies just as much to optimizing renders as it does to optimizing code.
> Of course this was pre-pathtracing, so it showed up differently.
Yep, I remember those days of distribution ray tracing. There were all sorts of tricks to keep the ray trees under control, like lowering the number of glossy or shadow samples the deeper you were in the tree. I don't miss them. Pathtracing made things so much easier to tune.
Oh definitely! So much easier now. After Rendition we did another version called Spraytrace which tried to force mental ray into more of a pathtracing type mode of operation, (eliminating ray branching, rather than light transport) by faking the mental ray api return values, and overlaying our own sampling. It worked, but was rather a hack. God knows why we didn't just see the writing on the wall, and write a path tracing renderer. :-)
Oh shoot, you're totally right -- doing a quick search reveals that Blender's Cycles engine has totally had this feature for a while and I was just completely unaware of it despite using it frequently! Although even without the heatmap pass being unique/new I'm still super excited for the rest of MoonRay, especially those delicious volumetrics!
Oh yes, I'm not knocking the release of MoonRay as a whole. I was at HPG '17 when the paper that erichocean mentioned was present. I'm definitely excited to get a peek at it.
I also wonder now how much work it was to disentangle it from their infrastructure? A lot of internal studio tools and libraries tend not to work outside of their studio's intranet -- not so much from direct copy protection, but more just because of heavy assumptions about filesystems and servers.
I'm guessing it's a combination of genuine goodwill (Dreamworks has a history of open-sourcing stuff) and wanting to be able to hire people who already have experience with their renderer instead of having to train every new hire. Not to mention the improvements that will surely be made to it once anyone can contribute a pull request. Whatever their reasoning, I'm hyped because it seems like an unconditional win-win -- artists get a great new render engine, Dreamworks gets kudos + free development + market/mind share.
I know a guy who hacked on Apple's compiler fixing bugs. It turned into a job offer, but he didn't like the interplay of compensation and relocation. Now would have been a good time for him to check back on their relocation policy but in the interim it would be a large demotion instead of a small one.
I used to joke that F5 owes me a job and 2 month's salary for all the free QA we did for them back when their session affinity code was just a checkbox on the PR material. If they were more open that might have played out a bit differently.
LLVM and Clang (I’m assuming you’re referring to these based on timing) are not Apple’s. Both have been under the LLVM Developer Group from inception, and LLVM existed for many years before Apple hired one of the lead devs.
While it’s true that Lattner worked for Apple while he created Clang and that his personal contributions were probably biased, it’s always been an outside of Apple project with independent contributors.
LLVM used GCC C as a front end. Clang was an Apple project, since GCC wasn't designed to integrate into an IDE like Xcode.
>One of Clang's main goals is to provide a library-based architecture, so that the compiler could interoperate with other tools that interact with source code, such as integrated development environments (IDE). In contrast, GCC works in a compile-link-debug workflow; integrating it with other tools is not always easy. For instance, GCC uses a step called fold that is key to the overall compile process, which has the side effect of translating the code tree into a form that looks unlike the original source code. If an error is found during or after the fold step, it can be difficult to translate that back into one location in the original source.
I said it was created by him while he was at Apple:
> While it’s true that Lattner worked for Apple while he created Clang and that his personal contributions were probably biased
But it wasn’t silo’d into Apple and then released to the open source field, it was created as an open source project under the LLVM Developer Group from it’s inception.
So yes, his contributions and goals were to make it targeted for Apple’s needs; but there were plenty of independent contributors that were focused on just making it a good C/C++ compiler front-end for LLVM and were more focused on making it a general competitor to GCC.
It would be more accurate to refer to is as an Apple sponsored project.
I could explain it but I don't have the energy to respond to a gotcha today. This has nothing to do with the conversation in the grandparent or the whole thread really.
It sure is. Especially when you nerdsnipe yourself and try to drag a conversation with you.
> Just don’t spread your mistruths.
You are very invested in pretty much nothing at all. What mistruths am I spreading? That Apple appreciated people fixing bugs that affected them? Oh. No.
In general: the digital arts industry is heavily, heavily dominated by the tools artists can get access to when they're learning digital art. Artists want to create, and (as a general rule / stereotype) they have a bit less patience with the same-purpose-but-different-interface redundancy that the software industry is extremely comfortable with.
A company that doesn't make their digital art tools available to artists learning to be artists risks stalling the recruitment pipeline because all other things being equal, artists will prefer to work at a house already using tooling they know rather than be forced to learn some other thing that does something they can already do with another toolchain.
> Really impressed that Dreamworks is making this move.
It's bone-headed. They could have partnered with a cloud vendor and maintained control and a large percent of the revenue. Instead, Amazon can now turn this into an AWS offering, drive mindshare to the AWS offering, and then bleed DreamWorks of whatever competitive advantage this offered them. AWS makes money, DreamWorks loses talent and leadership that moves over to work at AWS.
Open sourcing big software with an enterprise market and giving it a permissive license is stupid, because you can't compete. Look at pretty much every permissively licensed database vendor. Elastic's recent fight comes to mind.
In ten years, when this stuff becomes easy enough for small studios to run (something my startup is working on), the need for big studios like DreamWorks to pursue projects funded by institutional capital evaporates. Everyone will be making real time 3D animation. If you look carefully, you can see the genesis of that trend now.
If I were in a leadership position at DreamWorks, I would be staffing up how to make these offerings broadly available. How to make tools that put more people in the director seat. That sounds like the mission of a company named "Dream Works". Let more people dream. I guess this does it, just not in a way that they'll be using your company to do so.
Big disruptions are coming to the creative fields. This kind of move is like giving away the keys to Amazon for free. (I mean, they are buying Hollywood studios left and right. If you don't see it, you're asleep at the wheel.)
I disagree. Dreamworks doesn't make their money because of their rendering engine, they make it because of their movies. The engine is a necessary and integral tool for making those movies, but the engine alone is not the beating heart of Dreamworks. Open sourcing MoonRay is a smart move because it means they can start hiring people who already have prior experience with it, as well as benefit from the free optimizations and improvements the OSS community will surely make!
Having a rendering engine that's really good is useless unless you have the talent to use it to make a movie, and Dreamworks is going to maintain their branding and reputation for a long time regardless of whether their tech is open source or not.
They aren't "giving away the key" because the render engine is not the key. It's a complement, and they're simply commoditizing their complement.
While I agree that high quality art is going to become completely commonplace over the next decade or two, I really don't think it's tied in any way to one company releasing their internal render engine to the public.
Yes, I have to second this. While there is some edge to technology in the render sphere, no one went to see Moana or Sea Beasts due to the amazing water effects. They went because of word of mouth or advertising and were subsequently awed by said effects.
You can have the most advanced visual effects and still make a terrible movie. And you can make a great movie with slightly outdated effects and still wow people (Mitchels vs The Machines is a great example).
Dreamwork, being in the business of movie-making and not technology as a service, only benefits from an open renderer atmosphere. Pretrained talent, open source collaboration and contributions to improving their software and PR wins.
Similarly, video games that focus solely on amazing graphics are often incredibly boring to play, while there are plenty of low budget games that are very creative and fun to play, despite not having amazing graphics. Another example is Anime - very simple and dated graphics, yet fun characters and story lines. The renderer is far from the most important aspect of making a great movie or game, although it can certainly make it more enjoyable.
I'm genuinely curious, what about The Mitchells vs. The Machines graphics felt slightly outdated? It felt like they pushed the boundaries of the medium to me, including doing a lot of custom rendering work to make their final frames match their concept art as closely as possible, right down the the stroke textures and light-responsive character outlines. Or were you just referring to them not making any significant advances in physically-based rendering, because in that case I see what you're saying.
Side note, that movie was spectacular. I have poster for it on my apartment wall!
In the coming years, all sorts of people will be making films, and they'll be empowered by a new generation of editing, compositing, and rendering tools. They won't need large sums of capital or large teams to accomplish what the Pixar of yesteryear did.
There are so many people that want to direct, yet the current Hollywood regime only has so much room at the top to grow into. That's going to change.
And if you doubt that people want to be creative, just look at TikTok. Really, YouTube should have been our first evidence of this.
> Dreamworks doesn't make their money because of their rendering engine, they make it because of their movies.
For now.
It's their job to know where the market is heading, and the world of 3D animation is about to be blown wide open for smaller teams to accomplish what the big incumbents have done for decades.
Tooling will be one of the areas where money is made, and they just gave it away for free.
> Dreamworks is going to maintain their branding and reputation for a long time regardless of whether their tech is open source or not.
This is where we fundamentally disagree. Films currently require institutional capital and large teams to make, and that's about to change in a big way. The film industry is about to undergo something vaguely resembling what the news and journalism industry underwent as access to the tools of creation and distribution become more widespread and accessible.
The studio system exists because making movies is hard. It only needs to exist while that remains the case.
The year is 2043, and I’m in the mood for a “Heat bank robbery movie with an Alien twist”.
Soon enough, convergent-rendered Val Kilmer is stumble-crawling through an extensive duct system containing numerous big city bank branches, relentlessly harried by a xenomorph Al Pacino.
Is it access to high end rendering software and the hardware to run it on that is what is currently stopping small studios without a ton of money from making the kind of movies that DreamWorks and the other big studios make?
"How to Train Your Dragon: The Hidden World" had around a couple dozen people in the art department, around 3 dozen in the sound department, over 150 in visual effects, around 100 in animation, around 150 in music, and another 150 additional crew.
I'd expect paying so many people is what makes these movies expensive and beyond the reach of smaller less well funded studios.
DreamWorks Animation (aka DWA) is a huge advocate of open source.
Over the years they've bankrolled a lot of development around Linux on the desktop and have one of the most astonishing NFS infrastructures I've ever directly layed hands on.
They've talked about their love of open source over the years, especially at Red Hat summit.
(I spent quite a bit of time working with them and their former CTO is currently a teammate of mine).
Fortunately, Mike (Cutler) & Sean (Kamath) documented many things in the USENIX presentation cited by newman314 that I'm still under NDA about. I worked with both of them and they're extremely knowledgeable folks.
Now, switching gears to talk about that presentation. ;)
I'd call out a few things.
1) Note on slide 17 the call out to _multiple sites_. Yup. You got it. There is a single view of the file store. At a practical level to the end user the sync time between sites is trivial to non-existent.
2) automount, automount, automount (Slide 25). The automount configuration is stored in LDAP providing a convenient way to correlate and sync data which is already integrated with the name service switch/NSS (man 5 nss).
3) Caching (Slides 22-24). This is where a lot of the magic happens. It's this cache tier which _presents_ as if it's a single file store.
One conceit I will make (which is glossed over a bit) is that the use of DNS and suffix searching is critical in a situation like this. It allows each site to use configuration management to push out relevant config files (e.g. /etc/resolv.conf) which can then present a search order like "site1.us.example.com us.example.com example.com". Thus, when you do a query for the unqualified name "ldap" (or even better "IN SRV _ldap._tcp") NSS will hand that off to the resolver which will dutifully check for the following (in order) until it gets a positive match or fails: ldap.site1.us.example.com, ldap.us.example.com ldap.example.com. This allows for a DNS based fall back as well as the added benefit of always hitting the local system (instead of a remote site).
Also, make sure you check out the stats on Slide 28.... 4 PB... in 2008... before they had to render a movie for each eye... (3D animation!)
There is a lot of staffing churn in the film world and many films are built by farming out work to dozens of separate VFX studios.
I speculate that having this open source increases its popularity, which makes more likely that any given potential hire or VFX company will have experience with it, which makes it easier for DreamWorks to ramp up new hires or contract out to third party companies.
When I was at EA, we switched from a proprietary UI tool (which was quite well suited for consoles) to Flash (which wasn't) entirely because it eased staffing problems even though the technology itself was a worse fit. The best tool is often the one that people you can hire already know.
god, Flash wasn't suited for anything and yet somehow it got used for everything until Apple managed to kill it by keeping it the hell off the iPhone. It was kind of okay for so many things and actually great at none of them.
It might even still be in use at Cartoon Network, it's been a while since I last heard from my former co-worker in the dialup-delivered Flash cartoon mines who ended up as the animation director for Foster's and Titans but he sure was still using an old version of Flash/Animate long after the browser plugin was retired.
There's probably a lot of lessons in Flash's status as a tool that did a ton of different things half-assedly.
It was like the JavaScript of UX and 2D games for a while. It was trash as a technology, but it was a technology that millions of UI designers and indie game developers cut their teeth on and already knew deeply, so it stuck around much longer than it should have. Like grown-ups trying to ride tricycles to work.
I remember someone at EA talked to Adobe about getting access to the Flash (.fla) file format so that our UI tools could read it directly to export into the in game format instead of reading the already compiled and compressed .swf format. And they were like, "Uh... you really don't want to do that. It's like a memory dump of the entire application state. It's a giant mess. We're really sorry."
By all accounts, it seemed like the Flash codebase was a dumpster fire. But so many amazing little games and UIs were built using it.
The .fla format was a OLE compound file. It wasn't that bad -- I reverse engineered how it worked in the Flash MX 2004 days, and it was mostly editor data structures formatted like how you'd expect.
Even small companies can create competitive high-end renderers today so it's not much of a differentiator. Filmmaking is about human talent mostly, not technology. (Same thing happened in live-action filmmaking, anyone can afford the equipment today.)
But there's a huge advantage to being the first high-end renderer to go open source, so lots of small studios will adopt it if it's as solid as it looks.
It'll become available in Blender.
Maintenance will then get extended to the community, including (most likely) support for alternate shading schemes (MaterialX/Lama has decent momentum right now, converging with Sony's OpenShadingLanguage).
Apache 2.0 is an ideal license for this kind of project.
Yeah. They've open sourced OpenVDB and other smaller things before, and have contributed things to Embree and OpenSubDiv, but those were libraries/storage formats, not entire production-capable renderers.
At the same time however, does their renderer give them that many advantages? As someone who works on a (sort of) competing proprietary renderer, it's a lot of work and effort to do it and support it, and maybe they want to build a community around that from smaller studios and compete with Renderman a bit for mindshare?
OpenVDB is a really cool tool and a nice C++ codebase. During my PhD I adapted it to manage volumetric data extracted from LiDAR measurements and found it to be much more efficient than more popular alternatives in the robotics world, such as Octomap (which is also a great tool in its own right!).
Students get used to your pipeline before you hire them. The technology is nothing groundbreaking but the best that is available for free. Most students will take that opportunity.
This is huge and people underestimate how valuable it can be for a company. DreamWorks Animation's product is not MoonRay, it is movies and they're commodizing their complement.
If every animation studio started using MoonRay because of this, they'd have won.
Good point in your second sentence, but hard see where you're coming from in the last one.
Surely the win for DreamWorks is the competitive advantage of having aspiring creators be familiar with their pipeline. If all animation studios use MoonRay, then that advantage gets erased, and the ultimate impact is null. (The only way this isn't true is if you subscribe to the belief that more MoonRay use → more MoonRay experts → more DreamWorks movies → more profit. That's probably not sound. This probably won't impact how many movies they put out, just how good they look (after benefiting from open source work) and how easy it is to hire for the movies that they put out.)
The estimate would be that their tooling investment is no longer a major competitive advantage, compared to other studios. But, if other studios or new studios switch to their tooling or start using it, then functionally they're subsidizing skilling up animators who they can later hire as contractors or full-time onto their projects.
They're in the movie business after all - speed of production saves a lot of money.
> If all animation studios use MoonRay, then that advantage gets erased, and the ultimate impact is null.
Other studios could also contribute improvements, which the original studio could benefit from. That's one of my motivations for open-source: I'm selfish and I enjoy the benefits from the efforts of others making my software better, should they find it useful.
And it would mean that they're all on a level playing field - everyone would be using MoonRay so there wouldn't be an obvious technological advantage.
(Note that this could mean that if you thought your software was putting you at an advantage, you might not want to open-source it, but you could also believe that even if it is better, your competitors won't / can't switch to it).
> And it would mean that they're all on a level playing field - everyone would be using MoonRay so there wouldn't be an obvious technological advantage.
But if the improvements to MoonRay drive down production costs for movies, the entire industry wins.
This won't be their entire pipeline though. People will have to use the open source version in vanilla DCCs without all of Dreamworks' integrations, so will only be exposed to a bit of it in terms of configuring the renderer's options.
Can students realistically run this kind of software on the hardware they have? I would guess DreamWorks only runs it on fairly expensive networks, and would not have prioritized keeping it usable on single-machine setups.
I disagree -- a large part of the lookdev process at animation studios involves single artists working on single workstations with single GPUs while they work on their part of the pipeline. I'd wager that Dreamworks almost certainly does big distributed renders for the final frames of the movie, but a large number of the in-between steps and still-frame test renders probably happen in individual employee's machines -- not to mention that having an engine that can preview the render in near-realtime is a staple of modern 3D pipelines now that we have such beefy graphics cards.
If we're speculating, Pixar is also announcing it's Renderman 25 XPU cloud native render tech. If gpu cluster & virtualization continues to explode and transform the effects industry. I imagine all the major studios would seek ancillary revenue, farming out their cloud investments during idle hours, to third parties in gov research and academics to use for large scale data viz ;)
When I was on the design team for the Lucas Presidio Campus, we were collapsing all the various divisions to the new campus, aside from Skywalker...
I was the LV/Datacenter/network designer for the project.
One of the early design requirements was to 1) have fiber to the desktop and 2) have every machine on an interconnect matrix which brought the edge desktop connections as close to the core as possible such that every workstation would become a part of the ender cluster when idle/at night/on-demand etc...
This didnt happen but it was a goal back in ~2004
--
I recall them telling us when Jobs came to visit and they had the "Death Star" <-- render cluster they were working on... Lucas apparently saw no future and sold it to Jobs for ~$10mm?? and thats where Pixar got it start.
That was an amazing fun project to work on.
---
Oh - and during the design sessions, Cisco borked the design review for their (joke of a response) to the RFP, the then CIO for LucasArts in this design meeting with ILM, Lucas, Cisco others -- he says to the entire team in dead seriousness
"When can I have power-over-fiber to the desktop" with a look like "Ha! GotchA!
My experience from when I was in the industry was the opposite. Studios would generally maintain a solid baseline renderfarm and then send overflow to the major cloud providers.
They also weren't really set up for the kind of strict isolation and security that offering resources to external parties would require.
Interesting, seeing as at least on Trolls they were using AWS for some of their compute for rendering. Some Dreamworks employees on the tech side left for AWS as well.
>We expect to see the code base grow stronger with community involvement as DreamWorks continues to demonstrate our commitment to open source
Translation: we think making this open source will bring in significant outside contribution so we can get a better renderer with less investment ourselves while not having to pay external providers
Open source is a good strategy for the runners up trying to compete, you get the good will from giving something away in exchange for other people improving your product for free.
Why are you framing this in such a negative light?
They open something up, let people use it, and hope that some people might contribute back. Cool, sounds great to me. They aren't forcing me to contribute back to the project. Seems pretty win-win.
Not necessarily: it would depend on what Dreamworks' requirements have been up to now as to the current state: obviously, they use volumetrics, but almost all their stuff is CG and not necessarily photo real, being more artistic-driven.
That fact might have allowed them to take short-cuts.
I'd guess that the volume rendering in MoonRay is pretty top notch considering Dreamworks was also responsible for creating OpenVDB. Furthermore, while their characters and environments are often stylized, watch a few minutes of How To Train Your Dragon 3 and you'll see their volumetric rendering is stunningly believable.
We added MoonRay today to the ASWF Landscape, which is a great tool for seeing all the open source in the visual and special effects industry. Check it out at https://l.aswf.io
Blender integration using Hydra. "By adapting Hydra as a render add-on to Blender, any renderer that supports Hydra [ie MoonRay] can be connected to Blender easily, by plugging-in to the Hydra add-on." - https://gpuopen.com/learn/amd-usd-hydra-blender/
It's good to see companies freeing important software like this. That shows other companies and everyone else that you can do that without too much risk and that it might even give you benefits. I think many companies are afraid of doing it and I think this is how we can show them that there is no problem with doing it (and if there is a problem, we will learn that; we learn something either way).
Since free software is so important to many people and one of the biggest questions about how the digital society should work, it's crucial that we explore the possibilities of releasing free software. I'm tired of the current situation where on one side we have a ton of software and on the other side people who can't use that software because of how unethical or impractical it is. We need to find something that works for everyone and this is a step in that direction.
Individual creators are now building scenes in Blender and Unreal Engine. The floodgates are about to open for individual creators doing animation. An easy to use render farm is a good startup idea.
It's really just some shoestring and bubblegum ontop of amazon / google gpu instances. As commodity AND niche as a renderfarm is, it seems like if it was to run, it would have to be via a foundation like ummm the blender foundation.
We are in a world, where left pad was opensource, question must always be why is it closed source. I mean not to say closed is bad, but a question is good.
Really impressed that Dreamworks is making this move. They've made some of my favorite films of all time (the Dragons franchise genuinely changed my life and is a major part of why I love filmmaking, film scores, and 3D rendering) and I'm glad to see they're doing this. I can't wait to try MoonRay in Blender once a community integration exists!
[0]https://openmoonray.org/about
[1]https://jo.dreggn.org/path-tracing-in-production/2017/Moonra...
[2]https://www.awn.com/animationworld/how-moonray-became-hidden...
EDIT: I've also come across this excellent blog post written by an engineer who worked on MoonRay for 4 years: http://rendering-memo.blogspot.com/2019/
And this paper recommended by erichocean in their reply below: http://www.tabellion.org/et/paper17/MoonRay.pdf