dxt5nm Format bug?

[chris]

Chris wrote:

Fridger has tested dxt5nm support and was getting very nice results; I'm pretty sure that the feature works correctly if you get the channels right.

In case you overlooked this I would like to point out that after I remapped Seldens nm.png as
black => dxt red
red png => dxt green
blue png => dxt blue
green png => dxt alpha
Then compressing as dxt5, I got the same depressions Selden was getting with his dds normalmap.

It wasn't till after I normalized only with the Gimps normalmap plugin and then remapped as I shown above, that I got the correct results as shown here.

I don't understand at all . . . I looked at the shader for compressed normal textures, and it looks exactly how it should. What do you mean that you normalized with the Gimp's normalmap plugin? I need to know what transformation this is applying.

--Chris

[cartrite]

I'm not sure what the gimp's plugin is actually doing but I got the same result on the un-normalized one by uncommenting the line in shadermanager.cpp

This was the line 1334. I uncommented
Quote:
source += "vec3 n = normalize(texture2D(normTex, " + normTexCoord + ".st).xyz * 2.0 - vec3(1.0, 1.0, 1.0));\n";
and commented out 1335
Quote:
//source += "vec3 n = texture2D(normTex, " + normTexCoord + ".st).xyz * 2.0 - vec3(1.0, 1.0, 1.0);\n";

I rechecked the mipmap normalmap and it worked. So uncommenting the normalization line is what did it. It seems that this problem is ready to be dealt with.

http://www.celestiaproject.net/forum/viewtopic ... 9462#79462

Here is a link to my site. If you navigate to the bug page both of the normalmaps are there. The normalized one and the unnormalized one.

cartrite

[chris]

I'm not sure what the gimp's plugin is actually doing but I got the same result on the un-normalized one by uncommenting the line in shadermanager.cpp

This was the line 1334. I uncommented
Quote:
source += "vec3 n = normalize(texture2D(normTex, " + normTexCoord + ".st).xyz * 2.0 - vec3(1.0, 1.0, 1.0));\n";
and commented out 1335
Quote:
//source += "vec3 n = texture2D(normTex, " + normTexCoord + ".st).xyz * 2.0 - vec3(1.0, 1.0, 1.0);\n";

I rechecked the mipmap normalmap and it worked. So uncommenting the normalization line is what did it. It seems that this problem is ready to be dealt with.

That's very strange . . . The line that you commented out is only used for normal maps that *aren't* compressed. It shouldn't have any effect at all on the appearance the appearance of dxt5nm maps.

--Chris

[cartrite]

I don't know. All I know is that I was experimenting with putting different configurations of ( ) around the line

Code: Select all

source += "n.z = sqrt(1.0 - n.x * n.x - n.y * n.y);\n";

to see if something was being done out of place and nothing changed. I gave up and I figured that I'd try to uncomment that line to see if there was a quality difference with the png maps. Then I noticed that the dds map didn't look depressed anymore. So I normalized it and ran it on the unchanged code and that fixed it too.

Those files where compressed with the Gimp as dxt5 and not nvdxt.

[cartrite]

At first I thought it was Seldens nm.dds that was mapped wrong. When I remapped it everything worked. Before I remapped it, I was experimenting to see if something simple could fix it in the code. After I got it working with the remapped version of the file, I noticed that I was using the .exe that I modified so I ran it on the original and I THOUGHT THAT IT DIDN'T WORK anymore. I triple checked or I thought I did. I must have been using Seldens file though. Right now it doesn't matter if I normalize or not, as long as the file is remapped, it works. Sorry
cartrite

[chris]

At first I thought it was Seldens nm.dds that was mapped wrong. When I remapped it everything worked. Before I remapped it, I was experimenting to see if something simple could fix it in the code. After I got it working with the remapped version of the file, I noticed that I was using the .exe that I modified so I ran it on the original and I THOUGHT THAT IT DIDN'T WORK anymore. I triple checked or I thought I did. I must have been using Seldens file though. Right now it doesn't matter if I normalize or not, as long as the file is remapped, it works. Sorry
cartrite

Well, I did learn one interesting thing . . . It turns out that the NVIDIA Photoshop plugin uses alpha for the x component of the normal and green for the y component. Ugh. This is also what the NVIDIA white paper suggests. It's arbitrary whether to put x in alpha and y in green or the other way around. But, it appears that x in alpha is more the convention, so I guess I should change the shaders in Celestia. It's just a batter of swapping two characters in shadermanager.cpp.

Fridger: you've done more work with this than anyone, I think. What have you found the conventions to be?

--Chris

[cartrite]

So then remapping would look like this ?
black => dxt red -discarded
red png => dxt alpha
blue png => dxt blue -disarded
green png => dxt green

[chris]

So then remapping would look like this ?
black => dxt red -discarded
red png => dxt alpha
blue png => dxt blue -disarded
green png => dxt green

Yes. Although I think it's clearer to write it like this:

dxt red <= zero
dxt green <= png green (y component of normal)
dxt blue <= zero
dxt alpha <= png red (x component of normal)

But, it's exactly what you described . . .

--Chris

[cartrite]

Sorry, I was a little dazed and confused there for a second.
So basicly the only thing your doing is swapping red and alpha. The gimps dds plugin has an option to do this. All one would have to do is add an alpha channel to the png normalmap and while saving as dds choose (check)swap red and alpha and set the compression to dxt5.

cartrite