What Is Neural Rendering? How Tiny AI Networks Inside Your GPU Draw Frames, Shrink Textures by Up to 85%, and Why the RAM Shortage Is Pushing Xbox's Next Console Toward It

Xbox's Project Helix lead says the memory shortage is forcing console makers to rethink how frames are drawn, and neural rendering is his answer. Here's what neural rendering actually is, step by step, from upscaling and frame generation to neural textures that use a fraction of the memory, why RAM prices make it matter, where PS5 Pro and PS6 fit in, and the catches.

Zerocade · 6 min read
ShareWhatsAppX
Official AMD FidelityFX Super Resolution teaser image
Image: AMD

The short version

  • Neural rendering means running small AI networks inside a game's rendering pipeline, on the GPU's AI cores, to do jobs that used to be calculated the slow way.
  • You already use it if you've turned on DLSS, PSSR or FSR upscaling or frame generation. Newer forms rebuild lighting, materials and even textures.
  • It saves memory. Nvidia showed a scene's textures shrinking from 6.5GB to 970MB with neural compression, about 85% less.
  • Xbox's Jason Ronald said on November 16 that the RAM shortage makes neural rendering central to Project Helix. RAM prices are why consoles cost more this year.

Why this came up now

Jason Ronald leads development of Project Helix, Xbox's next console. In an interview with GamesRadar published on November 16, he called this a "challenging time" for hardware because of the global RAM shortage. He also called it one of the most exciting, because "we have to invent entirely new technology".

The technology he pointed to is neural rendering, which he called "an entirely new way to generate a frame". GamesRadar summarised his view as producing frames without needing much more raw power or memory. That matters when memory is the expensive part.

Official Nvidia diagram showing how DLSS 5 neural rendering works

Image: Nvidia

Step 1: the old way

A traditional renderer calculates everything. For each pixel, it works out which surface is visible, what texture sits on it, how light hits it and how that light bounces. More pixels, more frames or more realistic lighting all mean more calculation and more memory for textures and buffers.

That's why games run at lower resolutions or 30fps on consoles. Our CPU-bound vs GPU-bound guide covers where those limits come from.

Step 2: let a network guess the rest

A neural network is a set of maths trained on huge amounts of example data until it can predict an answer. Neural rendering puts small ones at specific steps inside the frame. Microsoft's DirectX team defines it as the suite of techniques that use AI and machine learning in rendering. Its "cooperative vectors" feature, in DirectX Shader Model 6.9, lets a small network run inside a pixel shader without taking over the whole GPU.

The jobs those networks do so far:

TechniqueWhat the network doesExamples
UpscalingTurns a low-resolution frame into a sharp high-resolution oneDLSS, PSSR, FSR Upscaling
Frame generationCreates extra frames between real onesDLSS 4 adds up to three per rendered frame
Ray reconstructionRebuilds clean ray-traced lighting from a few noisy samplesDLSS Ray Reconstruction, AMD's ML ray regeneration
Radiance cacheLearns how light bounces around a scene and predicts itNvidia Neural Radiance Cache, AMD FSR Radiance Caching
Neural materialsReplaces complex material maths with a small networkNvidia claims up to 5x faster
Neural texturesStores textures as a compact network instead of an imageNvidia RTX Neural Texture Compression
Neural lightingAdds photoreal lighting and materials to the finished frameDLSS 5

Each one swaps expensive, exact calculation for a fast, trained guess. The GPU's tensor or matrix cores, built for AI maths, do the guessing.

Official Nvidia diagram of RTX Neural Shaders being trained on game data and shader code

Image: Nvidia

Step 3: why it saves memory

Textures are one of the biggest users of graphics memory in a game. Today they're stored with block compression, which shrinks them by a fixed amount. Neural texture compression instead trains a tiny network per material to rebuild the texture when it's needed.

Zerocade chart of neural texture compression memory savings: Nvidia's GTC 2026 demo cut 6.5GB of block-compressed textures to 970MB, and an independent beta test cut 272MB to 98MB in transcode mode and to 11.37MB in inference mode, alongside 2026 console price rises caused by memory costs

Image: Zerocade

  • Nvidia's own demo at GTC 2026 cut a scene's textures from 6.5GB to 970MB. That's one scene, chosen by Nvidia.
  • An independent test of the beta SDK on an RTX 4090 cut 272MB of textures to 98MB in its safer mode, which turns them back into normal compressed textures on load. Its full mode, which rebuilds texture data as it's drawn, got it down to 11.37MB, at some cost in frame rate.
  • Microsoft says Helix supports neural texture compression, according to Tom's Hardware's report on its March slides.

Upscaling saves memory too. A game rendering at 1080p and upscaling to 4K needs smaller buffers than one rendering at native 4K.

Why RAM prices make it urgent

Memory makers have moved factory capacity to AI servers. TrendForce expects regular DRAM contract prices to rise another 10-15% this quarter, with graphics memory prices still climbing. Sony and Microsoft have both raised console prices this year and pointed to component costs. In the US:

  • PS5 went from $549.99 to $649.99 in April, and the PS5 Pro from $749.99 to $899.99.
  • Xbox Series X went from $649.99 to $799.99 in August.

Sony CEO Hiroki Totoki said in May that memory costs will be "very high" in its next financial year, and that Sony hasn't decided when to launch its next console. We covered how this hits PC builders in our DDR4 vs DDR5 guide. On paper, a console that needs less memory per frame is cheaper to build, which is the bet Ronald is describing.

Where PlayStation stands

Sony is on the same path. The PS5 Pro's upgraded PSSR, which launched with Resident Evil Requiem in February, uses a network from Project Amethyst, Sony's partnership with AMD. Mark Cerny has described Amethyst hardware for a "future console", including Neural Arrays for AI work and Universal Compression to save bandwidth. That's widely assumed to mean PS6, but Sony hasn't confirmed it.

For Helix, Microsoft has confirmed a custom AMD chip built for next-gen DirectX and FSR, with ML upscaling, ML ray regeneration and multi-frame generation. Developers are due to get early hardware in 2027. A $1,800 price and a late-2028 launch have been reported, but both are rumours. We covered the earlier Helix price report in October.

Official Nvidia comparison image from Starfield in its DLSS 5 announcement

Image: Nvidia

The catches

  • It's a guess. Networks can smear fine detail, ghost behind moving objects or invent things that weren't there.
  • Frame generation adds lag. Made-up frames look smooth, but your inputs only register on real ones.
  • Art direction. When DLSS 5 was shown in March, one developer called it "just a garbage AI filter", and others said it airbrushed faces. Nvidia says studios can tune the effect. Our Hogwarts Legacy DLSS 5 report covers how it landed.
  • It needs support. Each game has to build these features in, and neural texture compression is still a beta SDK.

What to watch

Watch for the first games to ship neural texture compression, and for Microsoft's next Helix details. In India, the easiest way to see neural rendering today is DLSS 5 in Hogwarts Legacy or NBA 2K27, which needs an RTX 50 card. Price trackers put the RTX 5060 Ti 16GB at around ₹45,000 to ₹46,500.

Found this useful?
Send it to a friend who's been asking.
ShareWhatsAppX
Whodunit
Who did it? Place the suspects and catch the murderer.
Solve today's case →

Keep reading

Official AMD EPYC 9006 banner, AMD's first Zen 6 processors
Up next

Is AMD's Zen 6 Desktop Ryzen Really Delayed? A Well-Known Ryzen Tool Developer Denies the N2X and Summer 2027 Claims: Who Said What, What TSMC's N2, N2P and N2X Mean, What AMD Has Confirmed, and What Ryzen 9000 Costs in India Now

Two days after a Chiphell post claimed AMD's desktop Zen 6 chips had moved to TSMC's N2X process and slipped past Computex 2027, Ryzen tuning-tool developer 1usmus called both claims inaccurate. Here's what each side said, how much to trust them, what TSMC's 2nm variants actually are, what AMD has confirmed about Zen 6, its CES keynote on January 5, and what Ryzen 9000 chips cost in India if you're buying now.

Official screenshot from Devcats' earlier hidden-object game A Planet Full of Cats

Bethesda Confirms 'A Wasteland Full of Cats', a Fallout-Inspired Cat Hunt From Devcats, After an Australian Rating Leaked It: What the Listing Says, Who Is Making It, and What's Still Unknown

22 Nov 2026
Official Intel Core Ultra 9 badge

Intel Nova Lake Leaks Again: a 28-Core Desktop Chip at 5.2GHz, a 16-Core Laptop Chip With DDR5-8000 in a Partner Datasheet, and a SilverStone PSU With Three CPU Cables 'for Nova Lake-S'

22 Nov 2026
Official Nvidia App artwork

Nvidia Fixes Its SSD-Writing Driver Bug With Hotfix 621.35: What Changed, Who Needs It, How Much Was Being Written, How to Check Your PC, and Why a Hotfix Isn't the Same as a Normal Driver

22 Nov 2026
Official Deltarune key art from the game's website

Deltarune Chapter 6 Is Now Playable From Start to Finish, Toby Fox Says, and Chapter 7 Work Has Begun: What's Left, Why There's Still No Date, and What It Costs in India

21 Nov 2026
GPD Win Mini Ultra pricing key visual

GPD Win Mini Ultra Gets a $1,699 Price: Intel's Arc G3 Extreme, 32GB of RAM, an 8-Inch 120Hz Screen and a Keyboard, Plus GPD's Own Frame Rates and How It Compares With the MSI Claw and ROG Xbox Ally X

21 Nov 2026