Video: Here's How Auto HDR Enhances Games On Xbox Series X|S

Like I said those extra CUs WILL ABSOLUTELY MATTER.

This is one of the most hype features for me. Being able to go back and play my old games, or even just the old games I am playing now, with HDR? Yes please.

@Senua Even though auto HDR isn't taking up any of the system resources to add it in. There is no 'extra' cost to using it. Its built into the Video out section - the part that sends frames to your TV and uses an Algorithm to assess the image and add HDR to it. Because its an Algorithm and looks at the completed frame - it doesn't really know what everything should be - like a matte dull white or a sun bright white so some games, the algorithm doesn't work well so its 'disabled'.

This video is HDR-enabled. I watched it on my LG C9 and the effect is way better than I expected. It makes me want to check all my favourite games again.

@BAMozzy It doesn’t work that way. There’s no static algorithm available to generate heat maps dynamically or atleast it won’t work as good as a stochastic solution. You need to use a ML inference model using DirectML which will ofcourse benefit from extra GPU headroom. When this feature matures further and becomes generalised like DLSS 2.0 for ALL games, along with other things like AI super sampling, AI inference for character behaviour, DXR etc. these will be greatly benefitted from additional headroom.

@Senua It doesn't mean that its using the 'GPU' for the processing and AI algorithm. That can still be built into another area - like the Multi-Media portion of the APU that also deals with Audio. Part of that 'video' out section is likely to be where the Auto-HDR is being applied and not part of the 52CU's of the GPU. Its also responsible for encoding the video inc encoding for HDR so a HDR enabled TV will automatically switch over to HDR modes.

I would be surprised if MS are using a part of the GPU to do this - its much more likely to be in the Multi-media block so therefore the 'extra' CU's in the GPU are not being utilised to add HDR.

As you can see at the top in the centre, there is a sizeable portion named as Multi-media HW Accel - which is where I expect Auto-HDR is being applied...

Incidentally, my TV has an 'auto HDR' option too which isn't 'too bad' - although purists will hate it because its 'not' what the content was mastered to look like and it generally does well - although sometimes you do get the 'whiter is brighter' (as seen in Banjo Kazooie with eyes) but with using an AI method, they can tweak and teach the system to be more accurate. It still doesn't work well across all games - Fallout is one such game and there are others too that the 'algorithm' doesn't work well.

@Senua Definitely, Machine Learning or some kind of RDNA2 AI is involved. This is not a simple "whiter is brighter". The games really look HDR (I am lucky to have checked this HDR-enabled video on my C9).

@BAMozzy I might be wrong but that custom H/W acceleration block is designed only for things like FFT, video encoding, spatial audio processing etc. Unlike Nvidia Tensor cores, these next-gen consoles don’t have separate custom hardware for matmul ops and other similar things. Instead they have powerful CUs which can do low precision calculations in parallel and thus aiming for “economical upgrade to traditional rendering, not a complete replacement” in things like RT.

https://www.youtube.com/watch?v=IRUEhmNRHUs&ab_channel=HDTVTest

@BlueOcean
Did your LG C9 register it as HDR, like it would with standard normal HDR game or film etc?
Normally on the LG it comes up in the top right hand corner with HDR when it goes into HDR mode?

@BlueOcean
Actually sorry that was a pointless question as it was a video you watched.
I guess until you try it for real on the console we won’t know.

@BAMozzy Like I said, this feature is absolutely leveraging machine learning. This has also been confirmed by Digitalfoundry earlier this year.

https://www.eurogamer.net/articles/digitalfoundry-2020-inside-xbox-series-x-full-specs

I guess they have also made a video explanation about this. Also you might wanna checkout the earlier Xbox Wire articles. And yes I had watched the HDTVTest video when it came out.

I've heard that MS is done adding additional OG Xbox and 360 games. There is a licensing process for that with Publishers and it's apparently been shutdown for good. So while Auto-HDR is nice, we won't be getting anymore old games added. Which is a shame...

You are also mixing up the fact that the Algorithm was devised by feeding images over and over and over again and then using Machine Learning to create and refine the Algorithm that can be applied in the 'Multi-media' section with using ML to analyse every frame and decide how it should look if HDR had been applied at the very start.

The Coalition used the AI Algorithm on making Gears of War and then 'refined' it by hand to ensure its most 'natural' - things like the whites of eyes not being 'super bright'. The fact that this step happens 'early' in the video aspect means they can apply the auto HDR at the best point. You don't want to apply it after its at the point 'normally' of going to the TV - something a TV has no option to use for its 'auto HDR' Algorithm. Its still an Algorithm which is applied across the board but because the Algorithm was created by using Machine Learning AND applied at the optimum place, it gives incredibly good results - although NOT perfect and why some games will not work well with that Algorithm.

The Algorithm has been created using Machine Learning, understanding what tone mapping difference there is between an SDR and a professionally mastered HDR image would be. This helped create and fine tune the Algorithm that Xbox Series X use.

From the Coalition who used the Machine Learning Algorithm to get a 'basic' idea of what their game would look like in HDR - and how that Algorithm works - but then had to go in and 'fine tune' everything because its still just an Algorithm applied...

Gears 5 is being praised as arguably the best game to take advantage of HDR. Can you elaborate on how the studio leveraged machine learning to pull this off?

Penty: We used a machine learning algorithm that was developed by Redmond's ATG group to train an inverse tone-mapper for color-space conversion. We then blended this 50 percent with a Reinhard buffer to allow maximum control.

The technology behind the HDR is only part of the equation though. There was a lot of tuning needed on the content side to ensure muzzle flashes and skyboxes were in the proper luminosity band for example. Also, there is a lot of tuning from myself, working with the art director and engineers, on exposed parameters to ensure the HDR looks proper. Sometimes you can tune it to look "great" for one level but then the next level looks awful with those tuning values, so you need to be very careful to have global HDR values setup that still look great but also can handle a wide range of scenarios and maintain the artist’s intent.

Finally, we spent a good amount of time ensuring we had a robust calibration screen that represented a range of lighting scenarios. We implemented the ability to hide the calibration image while tuning the HDR sliders and also exposed a curve to represent your HDR output. Finally, adding controls like an HDR contrast and max brightness really helped empower the player to get the look they wanted.

@Dezzy70 Yes, the HDR logo popped up when I watched the video on the YouTube app and the HDR was noticeable.

I can't believe I gave nintendo money for their Mario collection

@GamerDad66

Heard from whom? Proof?

Microsoft - upgrades are free on our console.

Sony - we want you to give us $ for “remasters” of 2 yr old games even though our buffoon CEO said “nobody plays old games”. 😂

@KelticDevil dude, i laughed OUT LOUD when i saw Sony was selling a Spider Man remaster less than two years after it launched. like, really? it's going to sell 5 million copies because people are that dumb, but still lol

meanwhile, Microsoft is like, oh hey Panzer Dragoon Orta has HDR now, enjoy.

As far as BC goes, MS said it's basically over for OG Xbox and 360:

www.engadget.com/amp/2019-06-10-xbox-backward-compatibility-scarlett.html

"We have no plans to add additional Original Xbox or Xbox 360 titles to the catalog on Xbox One, but we're excited to continue our work on Xbox compatibility across platforms and devices, which remains a top priority."

They could restart BC just on Series S/X, but that would leave out a huge number of other Xbox owners.

@GamerDad66

They were talking about the Xbox One back then. That was 2019. I’m sure it’ll re-start on the Xbox Series X/S.

@KelticDevil @armondo36 But it's a very impressive remaster worth paying for again. Wait, it isn't. Meanwhile, free remasters on Series X/S.

@GamerDad66 Okay I see what you mean. They stopped because they started working on Series X backwards compatibility with Xbox One and backwards compatible games. They later said that they will welcome more backwards compatible games but it's up to the publishers and rights holders.

@BlueOcean

Haha! Exactly.

Oh and let’s not forget the remaster of a PS3 game that all the PlayStation fans act like is some brand new, next-gen exclusive.

And don’t they always say Microsoft just puts out a bunch of sequels? 😂🙄

@KelticDevil I don't know what they're going to do with the "Xbox has no exclusives because they're on PC" (apparently, everyone has a $2,500 PC) mantra now that PS5 games are coming to PC. Or the "Xbox One will hold Series X back" and "We believe in generations" motto now that PS5 games are cross-gen games including Miles Morales and Horizon 2. Demon's Souls is a PS3 remake, I love remakes but don't say Series X has no new launch games if PS5 hasn't new games either! 😅