No standard normal maps

[jim]

Hi Chris,

I'd thougt all problems with bumpmapping are soleved but there is a little yet important bug with normal maps. Celestia can't use standard normal maps. That means all normal maps build with tools from Nvidea or ATI don't work correct. I'd not used these tools so far therefore the bug was not discoverd.
To get a working normal map in Celestia the red channel must be inverted. That's all but i think it's better Cellestia works with standard normal maps. I'm not a programmer but i think the bug concern Celestias internal normal map builder and the shader too. Probably it's only sign bug but this should fixed before 1.3 release.

I'd build two test normal maps from my test bump map (round holes) for you :

http://www.assimilated-empire.com/celes ... al_map.jpg
http://www.assimilated-empire.com/celes ... al_map.jpg

If you use this you will see an other problem. I saved this JPGs with high quality but you can see still ugly artifacts. That's the reason why i want an addidional grafic file format.

Bye Jens

ps. I hope your trubble last weekend will take a good end.

[chris]

Hi Chris,

I'd thougt all problems with bumpmapping are soleved but there is a little yet important bug with normal maps. Celestia can't use standard normal maps. That means all normal maps build with tools from Nvidea or ATI don't work correct. I'd not used these tools so far therefore the bug was not discoverd.

To get a working normal map in Celestia the red channel must be inverted. That's all but i think it's better Cellestia works with standard normal maps. I'm not a programmer but i think the bug concern Celestias internal normal map builder and the shader too. Probably it's only sign bug but this should fixed before 1.3 release.

Thanks for bringing this up . . . I'll modify Celestia to use 'standard' normal maps. That sign discrepancy has been there for a while, but was never of consequence when Celestia always generated its own normal maps from height maps.

If you use this you will see an other problem. I saved this JPGs with high quality but you can see still ugly artifacts. That's the reason why i want an addidional grafic file format.

Why not just use PNGs to avoid the artifacts? Size?

ps. I hope your trubble last weekend will take a good end.

All taken care of . . . I didn't get hurt in the accident, and I got a new car out of it all!

--Chris

[jim]

Why not just use PNGs to avoid the artifacts? Size?

I use PNGs or very high quality JPGs and yes the problem is the size. This should be only a demonstration how sencitve are normal maps for compressed images formates (JPG, DXT1, DXT3, DXT5). The file format i suggested (palleted 8 bit DDS) has no artifacts and is 4 times smaler then a PNG (in the memory). High quality 8k normal map could be build with this.

Bye Jens

[jim]

Hi Chris,

I checked in the fix last night, so it will appear in 1.3.0. The only thing I didn't do was make the fix for non-vertex shader render paths. Does anyone still use the NVIDIA combiner render path without either NV or ARB vertex programs? I'm considering removing it . . .

--Chris

Sorry Chris the bug is not fixed in pre6. I'm so angry about that i've not checked this sooner. Now it's probably to late for Celestia 1.3.0. I don't know what you have modified. But i can say this about the render pathes:

1. Basic: no shadder
2. Multitexture: no shadder
3. NVidea combiners: orientation of bumbmapping is 90 degree off on equator
4. OpenGL vertex program: bump effect on the night site but with the old normal map correct orientated
5. OpenGL vertex program/NVidea combiners: work with the old normal map correct but not with a standard normal map.

I would say the problem is still the same.

I've uploaded my test normal maps with round holes again. I hope the new links work.

http://home.arcor.de/jimpage/files/stan ... al_map.jpg
http://home.arcor.de/jimpage/files/cele ... al_map.jpg

I agree with you that we don't need all the render path. I would say render path 1 and 5 are enougth. But it should be possible that the bump effect, the specular effect and the haze effect can be separate switched on /off.

Bye Jens

[t00fri]

Hi Chris,

I checked in the fix last night, so it will appear in 1.3.0. The only thing I didn't do was make the fix for non-vertex shader render paths. Does anyone still use the NVIDIA combiner render path without either NV or ARB vertex programs? I'm considering removing it . . .

--Chris

Sorry Chris the bug is not fixed in pre6. I'm so angry about that i've not checked this sooner. Now it's probably to late for Celestia 1.3.0. I don't know what you have modified. But i can say this about the render pathes:

1. Basic: no shadder
2. Multitexture: no shadder
3. NVidea combiners: orientation of bumbmapping is 90 degree off on equator
4. OpenGL vertex program: bump effect on the night site but with the old normal map correct orientated
5. OpenGL vertex program/NVidea combiners: work with the old normal map correct but not with a standard normal map.

I would say the problem is still the same.

I've uploaded my test normal maps with round holes again. I hope the new links work.

http://home.arcor.de/jimpage/files/stan ... al_map.jpg
http://home.arcor.de/jimpage/files/cele ... al_map.jpg

I agree with you that we don't need all the render path. I would say render path 1 and 5 are enougth. But it should be possible that the bump effect, the specular effect and the haze effect can be separate switched on /off.

Bye Jens

I have also noticed that after Chris' "fix" the nv-generated normal maps still work considerably worse than the ones I get by inverting the alpha channel in the bumpmap before converting it with nvdxt to a normal map.

I did just not have enough time yet to follow this up more closely.

But Jens, one also has to be careful with NV tools, since any JPG format involved has RGB flipped, incorrectly. This is an officially recorded bug in Doug's page somewhere.

Bye Fridger

[jim]

Hi Frigder,

I can exclude a problem with the normal map tools because I have tested 4 different progams (3 tools from Nvidea and one from ATI) and I think it's impossible that all tools have the same bug. Further I have checked the normal maps in DOOM3 and I don't think that ID use wrong normal maps. You can see that I have checked this bug very serious.

I know that the command line tools from Nvidia work best with TGA files therefore I've never used a JPG.

Another thing, I hope that a DXT1 normal map from you is soon available because I want check if it works really without artifacts on my computer. I can't beliefe that.

Bye Jens

[t00fri]

Hi Frigder,

I can exclude a problem with the normal map tools because I have tested 4 different progams (3 tools from Nvidea and one from ATI) and I think it's impossible that all tools have the same bug. Further I have checked the normal maps in DOOM3 and I don't think that ID use wrong normal maps. You can see that I have checked this bug very serious.

OK, good to know...

I know that the command line tools from Nvidia work best with TGA files therefore I've never used a JPG.

Another thing, I hope that a DXT1 normal map from you is soon available because I want check if it works really without artifacts on my computer. I can't beliefe that.

Bye Jens

I thought you are "slowly connected", how do you want to download my 44MB 8k-normalmap? You must really use the 8k map to zoom into the details to check for artifacts.

I am ready to upload my new textures to the Texture Foundry, as soon as

1) Chris comes around reactivating my Perl/Cgi code
2) This issue with the non-effective normalmap "fix" is settled. Neither I want to upload unnecessarily nor you want to download the wrong 40 MB maps;-)

Bye Fridger

[jim]

I thought you are "slowly connected", how do you want to download my 44MB 8k-normalmap? You must really use the 8k map to zoom into the details to check for artifacts.

44MB and 8k ? I thougt you use dxt1c compression or is the dimension 8kx8k? Have you tried to zip this texture? My 44MB DXD3 Mars normal map shrinks to 5MB. I have this not uploaded because the normal map bug sould be first solved. But a 4k normal map should be enought to see if your normal map works artifactless.

I am ready to upload my new textures to the Texture Foundry, as soon as...

I deny you for the possibility to build 16k textures. I can't get working the NV tools with 16K textures on my system.

Bye Jens

p.s. There is a nice download service that burn a CD with any stuff for only 3,50 EUR.

[t00fri]

I thought you are "slowly connected", how do you want to download my 44MB 8k-normalmap? You must really use the 8k map to zoom into the details to check for artifacts.

44MB and 8k ? I thougt you use dxt1c compression or is the dimension 8kx8k? Have you tried to zip this texture? My 44MB DXD3 Mars normal map shrinks to 5MB. I have this not uploaded because the normal map bug sould be first solved. But a 4k normal map should be enought to see if your normal map works artifactless.

I am ready to upload my new textures to the Texture Foundry, as soon as...

I deny you for the possibility to build 16k textures. I can't get working the NV tools with 16K textures on my system.

Bye Jens

p.s. There is a nice download service that burn a CD with any stuff for only 3,50 EUR.

The main 8k textures are 22MB, but the DXT1c 24bit normal maps are 44MB. I can't help it;-)

Here is the command I use, <file.tga> is RGBA! <====

nvdxt -file <file.tga> -alpha -n4 -scale 10.0 -dither -wrap -24 dxt1c

It may actually be a bug in the tools, that the alpha channel is still carried along as a 'dummy' channel, since the starting file was RGBA. I look into that.

All 16k conversions I usually do with my own tools, since they are much faster than the NVIDIA ones. Typically less than a minute (!) for *.tga => DXT1.

I told you also that I am using the mipmaps from the 8k SG image together with a 16k converted DXT texture that does not have mipmaps! And so on...

Did you use /exactly/ the same procedure???
Otherwise your results may be quite different.

You are welcome to compile my tools for yourself and use them. What OS do you have again?

And finally: I do not appreciate people pretending that I am not saying the truth, only because they are not able to reproduce what I am saying;-)

Bye Fridger

[jim]

All 16k conversions I usually do with my own tools, since they are much faster than the NVIDIA ones. Typically less than a minute (!) for *.tga => DXT1.

I know that the last try to compile this for windows does wrong. Maybe that it's possible to build a light version only a simple exe that convert a TGA to a DXT1c and anohter exe that convert a DDS file to a TGA.

You are welcome to compile them for yourself and use them.

I can do many things but not compile a program. I have only the free Borland C++ Compiler 5.5 and my attempts to compile something was not sucessfull. I think this is not the best tool for a C greenhorn.

What OS do you have again?

I use Win98SE.

And finally: I do not appreciate people pretending that I am not saying the truth, only because they are not able to reproduce what I am saying;-)

I have never doubt this (your Himalaya shot is proof enougth). But it's possible that Linux and Windows produce different results. I don't know if someone has tested this so far.

Bye Jens

[t00fri]

All 16k conversions I usually do with my own tools, since they are much faster than the NVIDIA ones. Typically less than a minute (!) for *.tga => DXT1.

I know that the last try to compile this for windows does wrong. Maybe that it's possible to build a light version only a simple exe that convert a TGA to a DXT1c and anohter exe that convert a DDS file to a TGA.

You are welcome to compile them for yourself and use them.

I can do many things but not compile a program. I have only the free Borland C++ Compiler 5.5 and my attempts to compile something was not sucessfull. I think this is not the best tool for a C greenhorn.

What OS do you have again?

I use Win98SE.

And finally: I do not appreciate people pretending that I am not saying the truth, only because they are not able to reproduce what I am saying;-)

I have never doubt this (your Himalaya shot is proof enougth). But it's possible that Linux and Windows produce different results. I don't know if someone has tested this so far.

Bye Jens

There is also a safe Windows method to handle 16k textures that is from my Celestia friend 'Pixel' (who was active before you came in and is now quiet, unfortunately):

He sent me a command line tool, called 'bmp2dxt' which converts arbitraily large *.bmp files to DXT1c without using RAM at all! It uses essentially virtual memory on the HD. So it takes a long time, but always works....

Also one should always do the conversion /without mipmaps/ which saves some space and subsequently 'stitch' the converted 16k dds texture together with the rest (mipmaps) from an 8k texture.

I hope Doug will show up soon again in this department and present an improved version of nvdxt for 'monster' textures. I think they are not optimized for large size stuff and also there are probably bad memory leaks!

Bye Fridger