For 50 years, Industrial Light & Magic (ILM) has been pushing the boundaries of cinematic innovation. ILM’s enduring “invent-whatever-the-story-needs” philosophy continues to drive its work in groundbreaking visual effects, constantly challenging audience perceptions of reality.

As ILM marks its 50th anniversary, it’s a moment to not only celebrate its cinematic achievements but also acknowledge the collaborations that fueled its journey. Among these, ILM’s long-standing work with NVIDIA stands out as a testament to how deep technological collaborations can reshape an industry.

Putting the story first

Currently NVIDIA’s VP of media and entertainment, Richard Kerris previously served as Lucasfilm’s CTO where he led research and development, IT and information services across ILM, Skywalker Sound, LucasArts and more. During his time there, the core philosophy remained the same: technology serves the story, never the other way around. Kerris recalls, “We pushed boundaries with real-time performance capture and cutting-edge visual effects pipelines, all in service of better narrative experiences. Little did I know during my Lucasfilm presentation at NVIDIA’s very first GTC in 2009.”

ILM’s collaboration with NVIDIA began in the early 2000s with ILM’s release of OpenEXR, a 16-bit “half-float” pixel format that serendipitously mapped one-to-one onto the FP16 math already exposed in NVIDIA graphic processing units (GPUs).

This bridge between NVIDIA’s technology and ILM’s artistic vision would transform how visual effects were conceived and executed, accelerating time-consuming processes and enabling new levels of creative freedom.

A prime example of this was GPU simulation. During the production of The Last Airbender, ILM’s proprietary Plume solver, which renders realistic smoke and fire, saw a dramatic 10-15x speedup when run on NVIDIA GPUs. This also proved that the NVIDIA CUDA parallel computing platform could handle “hero shots” – the most visually demanding sequences – rather than just previews, in a significant leap forward for production pipelines.

NVIDIA technology also helped revolutionize interactive lighting. By 2011, every artist working on the acclaimed animated feature Rango was equipped with an NVIDIA GPU-powered workstation – enabling them to iterate on lighting designs at “the speed of thought,” a paradigm shift from traditional, laborious render-and-review cycles.

Reflecting on the Road to Virtual Production

Another turn came with the advent of real-time rendering. At GDC 2018, ILM’s immersive team (then ILMxLAB), working with Epic Games and NVIDIA, showcased Reflections – a Star Wars short film entirely rendered in real time using ray tracing, a feat previously considered impossible.

The demo, powered by an NVIDIA DGX Station with four NVIDIA GPUs, sparked what would become today’s ubiquitous virtual production workflows. The impact was almost immediate: less than six months later, at SIGGRAPH 2018, NVIDIA announced its groundbreaking NVIDIA RTX technology, which democratized real-time ray tracing by making it accessible to anyone with a standard PC and a single NVIDIA RTX PRO GPU.

StageCraft, Helios and the NVIDIA Stack

These graphics technology breakthroughs found their ultimate expression in ILM’s StageCraft, the pioneering virtual production platform used for The Mandalorian. When ILM scaled StageCraft for the show’s second season, the company introduced Helios, a proprietary, real-time ray tracer underpinned by NVIDIA Vulkan ray-tracing drivers and RT Cores, specialized hardware units within NVIDIA GPUs designed to accelerate ray tracing.

Furthermore, NVIDIA OptiX, an AI denoising technology, played a critical role in achieving cinematic quality at real-time speeds. More recently, NVIDIA DLSS 4 – a technology that uses AI to render frames at a lower resolution and then reconstruct them to a much higher resolution – has been integrated into StageCraft, enabling superior visual fidelity at full frame rates.

Today’s StageCraft volumes are powered by racks of NVIDIA RTX PRO and NVIDIA Blackwell GPUs, which deliver 2.5 times the pixel throughput compared with ILM’s first-generation volumes, all while nearly halving power consumption.

Looking Ahead: NVIDIA’s Media2 Initiative

As the industry continues to evolve, initiatives like NVIDIA Media2 are gaining traction. Unveiled at CES 2025, Media2 is a comprehensive, AI-powered framework designed to revolutionize content creation, streaming and live media experiences. It seamlessly integrates NVIDIA RTX PRO GPUs, advanced AI models, NVIDIA NIM microservices and real-time video processing capabilities with the NVIDIA Holoscan for Media platform.

Additionally, Media2 provides pre-built NVIDIA AI Blueprints, which empower media companies to create personalized content, enhance live broadcasts with real-time intelligence and interactive elements and streamline their entire production pipelines.

Media2 is already being adopted by major industry players, demonstrating its potential to deliver smarter, faster and more immersive media experiences – building on a legacy of media innovation.