AI and the Future of 3D Filmmaking

What’s in store for the animation industry, now that we have AI?

Made with DALL-E

I. The Rise of AI

Every day is a new headline spelling doomsday for us in the film industry. The culprit of this prophesied calamity is almost always (or always, when traced back far enough) AI.

With all of the leaps and bounds made with generative AI tooling in recent years, the arts have emerged as the first target to be AI’d into obsoletion. Lacking the powerful army of legal and cash flooded soldiers to fight by their side, artists are left with their life’s work being reduced to a mere AI prompt.

And as much as we try to put a halt to AI’s potential by citing every moral dilemma, legal injustice, and well-founded fear of dystopian realities coming to fruition, there is simply no stopping it — unless our profoundly capitalistic society chooses to collectively shun the most significant “value add” in all of human history, for all time.

What does this mean for me, as a 3D animated filmmaker at Pixar Animation Studios, a studio best known for the painstaking human effort behind each and every meticulously crafted detail on screen? Do we bow in resigned defeat rendered obsolete by our new AI overlords? Are we to be among the first casualties in this new age?

II. Evolution of the Industry

I liken this period of history to when hand held cameras, film editing tools like Adobe Premiere Pro, and platforms like YouTube made the process of capturing, editing, and distributing film accessible to the masses, effectively democratizing access to live action filmmaking in the early 2000s.

Animation is a ruthlessly time and labor-intensive affair. It’s still the case that only the rarest of individuals with truly colossal resources and near-impossible-to-attain levels of skill could hope to create an studio-quality 3D animated film, end-to-end, entirely on their own.

The immensely complex 3D filmmaking pipeline consists of screenwriting, editing, concept art, storyboarding, modeling, rigging, shading, characters, sets, layout, animation, simulation, effects, crowds work, lighting, compositing, and rendering. Crafting animated feature films takes years upon years of time with mass amounts of compute and hundreds of highly specialized technicians and artists.

I remain more optimistic than ever about the future of 3D filmmaking. With the recent advancements in generative AI, I do acknowledge that animation is one of the best-positioned industries to be disrupted by AI technologies in the near future. Much like hand held cameras and editing software did for live action content, AI, generative and otherwise, in tandem with advances in real-time rendering and distributed computing, will democratize the ability for a single person to make high quality 3D animated films from end-to-end on their own.

Technically, I make my living in service of the Great Mouse. While I unfortunately have no useful insider information on the matter, I was intrigued to hear about Disney’s recent massive investment in Epic Games in February 2024, giving them a 9% stake in the company.

While the public angle was “bringing Disney into the metaverse,” sending the company’s most cherished animated personalities to wander around and interact with us commoners in a game-type virtual environment, it’s hard to ignore how Epic Games has also developed one of the most powerful setups for democratized animated filmmaking — Unreal Engine.

Self-advertised as “The most powerful real-time 3D creation tool,” Unreal Engine comes with comprehensive tools for animated filmmaking, complete with vast libraries of free assets and a robust real-time rendering engine which allows for rapid iteration and clean final renders.

Unreal starkly contrasts with Disney’s tooling, which is advanced by all means, but is largely inaccessible to the masses and is not meant for use cases any smaller than a well-funded production house.

Disney’s partnership with Unreal Engine is an acknowledgement of the impending 3D revolution from the industry giants. With this forged alliance, they have strategically positioned themselves to weather the chaos of AI and emerge as frontrunners when the landscape of 3D completely changes.

III. The Future of 3D Filmmaking

Generative AI is a small hint of what AI can truly change in the animation industry. While generative AI video generators like Sora, Pika, and Runway lack the controllability and reproducibility to create high quality films off-the-bat which can stand beside the likes of Pixar, Disney, and Dreamworks, the innovation of text-to-video also enables text-to-3D (as depth information can be pulled from videos), making AI-asset generation a possibility. These AI generated 3D assets can be cleaned and refined to perfection. Similarly, there are several companies and research projects focused on infusing specific domains in the 3D pipeline with AI, such as AI-based animation autocomplete and AI-based rigging.

As this trend advances, AI will increasingly facilitate every piece of the 3D pipeline with humans providing initial guidance via prompting and refining the final output to perfection. These innovations democratize filmmaking such that one does not necessarily need to have highly specialized knowledge about highly specialized tooling in a highly specific domain to be able to participate in 3D filmmaking.

Instead, it’ll be possible to create an animated film end-to-end as an individual by having a breadth of knowledge about the basics across all domains within the 3D pipeline, coupling that with a strong sense of artistic taste to understand what makes an output genuinely good, and being able to refine that output to perfection.

In this era of democratized 3D filmmaking, storytellers can make 3D animated films as easily as a teenager wielding a smartphone camera can produce short-form video content. In this world, all of us who joined the animation industry with the desire to tell great stories are no longer limited to a life of pushing a handful of pixels for large corporations (which prefer to fund stories with mass appeal in order to turn astronomical profits). We will no longer have to claw our way forward for the opportunity to be a creative and share our stories with the world.

So, are we filmmakers to be amongst the first to take the hit in this new era of AI?

Perhaps, initially.

Initially, many of our skill sets with specific tooling and the mechanics of filmmaking will be made obsolete and we will have to upskill on the next generation of tools built with AI. This was expected — we signed ourselves up for being career learners the moment we chose to work in an innovative industry marked by ever-changing technology. However, our cultivated passion and taste for making great art is a thriving quality in a world where AI has shifted the balance of things.

In this new era where execution is made easy, creativity reigns as king.

I’m Shubha. I write about computer graphics, tech, life, and anything else I find interesting! Feel free to check out my website for more odds and ends — shubhaj.com